26 Comments

  1. Ryan Duff

    So… Linode?

    Report

    • Dave Warfel

      I realize the breach happened at the “cloud provider,” but a very similar incident happened 2.5 years ago, and WP Engine was a lot more transparent with who their provider was (Linode).

      I’d love to see them be more transparent (unless, of course, it’s for legal reasons, but I have serious doubts that’s the reason).

      I’m 99% sure they still use Linode. They are listed on Linode’s website as a partner, and Lincode confirmed their recent data breach (here).

      Report

  2. Donna

    When did “managed hosting” with “best-in-class customer service” turn into not-managed-well hosting with customer service that fails not once, but twice, within a month’s time – and then it’s always someone else’s fault. (Linode, Cloud Infrastructure Provider). If this was cheap shared hosting, I’d understand, but expensive managed hosting? Ugh.

    Report

    • Amanda Rush

      Hi Donna,

      I don’t work for WPEngine, and I can’t speak specifically to your customer service experiences, but I can tell you that infrastructure, cloud or otherwise, problems are something that every single hosting provider has to contend with. I can’t think of a hosting provider, shared or otherwise, who can give you hosting for less than $50/month for a single site and do it on their own infrastructure that isn’t rented/leased from somewhere else. You, as a person, could totally go get a dedicated server from somewhere like Hurricane Electric, and expect to pay several hundred bucks a month just for that server, and be in charge of maintaining the entire stack. Linode is having troubles now, and it’s massive, but the same thing will happen to Amazon and any of the other big cloud providers. Sort of the nature of the beast.

      Report

      • Peter Knight

        To me it reinforces my belief that it’s not worth paying for ‘premium’ managed WordPress specific hosting in the expectation that it will spare me various security/infrastructure issues. Barring a few really bad hosting companies, generally you get a decent enough experience with similar exposure to risks/downtime etc.

        Report

        • Jason Cohen

          Hi Peter, it’s true that both managed and unmanaged hosting can have the same sorts of problems. The difference is what happens when those problems arise.

          Using this incident as the example, if you were in this position as your own sysadmin with your own VPS, you would need to hire and manage an external security firm (which we did, but we have an internal security team who could do that), you would need a lawyer to work with the authorities (which we did, but we have in-house council), you would need to perform diligence on an investigation and any remediation, and more things which sadly I’m not at liberty to discuss here.

          In the case of managed hosting, we did all of that on behalf of our customers. That is a substantial service.

          Of course, it’s small consolation that it’s “a service,” considering that even better would be to not have the incident at all! So it’s not like we’re patting ourselves on the back — it’s simply doing the right thing.

          But, to your point, it can (and does) happen to anyone, and only in a managed environment will the vendor take care of that for you.

          Report

        • Greg Hyatt

          Peter,

          While many will see this as a complete joke, but my primary site is still being hosted through Hostgator and other than a few snags here and there, I have not had any issues like what I have seen from companies who are providing pricey Managed WP Hosting solutions.

          Of course the other company that I feel very comfortable with is InMotion hosting. They seem to have a very knowledgeable staff when it comes to any issues with WP.

          However, most of the larger companies are owned by IEG, but as long as my sites are up and running, I think I’ll stay with the ones that most people seem to talk bad about. They haven’t done me wrong yet!

          Report

    • Amanda Rush

      As an alternative, you could also go purchase a VPS, but you’re still depending on an infrastructure provider.

      Report

    • Alex R.

      Agree. WPEngine is not cheap and we expect a better stability. This kind of hosting is mostly for companies which requires 99% uptime. I think they should take the responsibility and work harder with their infrastructure provider.

      Report

  3. Paul Davidson

    I’d be interested to know a little more about the breach was discovered and how the decision was made to involuntarily reset SFTP and admin passwords.

    Report

    • Jason Cohen

      Hi Paul, while we still can’t talk about the nature of the breach due to the on-going investigation by two US Federal authorities on their cases, I can answer the question about why we required so many password resets.

      When we first became aware of the incident, we elected to inform our customers as quickly as possible, rather than wait for a full investigation to complete (which can take weeks), during which time our customers would be blissfully unaware.

      Because we informed as quickly as possible, we couldn’t yet know exactly what data was involved, or even any details of what had or hadn’t happened. Therefore, we took the maximum precaution and elected to roll *all* passwords, even though in retrospect you could argue that not all of that activity was strictly necessary.

      While that caused work for our customers, and of course hurt us in the press, it was the right thing to do for our customers’ safety and knowledge.

      Report

      • Paul Davidson

        Thanks for the additional info. It was painful, and moreso because we didn’t know why, but I trust your judgment.

        Report

      • Rick Rottman

        Is there a gag order in place issued by a federal judge? If not, then you can talk about it, especially to the people who’s info was stolen.

        Report

        • Jason Cohen

          Hi Rick, as stated in the article above and in our own public report, we indeed have contacted and spoke on the phone with every one of our customers who were directly impacted, so you’re right about that, and we’ve already done that.

          What’s not appropriate is to reveal information that could compromise the investigation by revealing what is and isn’t known about the criminal. This is true in many types of legal investigations, not just in security matters, and we will adhere to industry and legal guidelines in this matter.

          Report

      • Marty Kassowitz

        I actually agree with your decision to reset passwords. Yes, it involved some work, but far less than it could have been if the situation had progressed. When I saw the notice, I was actually impressed. The ability to rapidly refresh security settings is actually a plus in my mind as well as an absolute necessity in today’s world.

        Report

  4. Al

    Does WP-Engine own their own datacenter, housing their own hardware and staffed by their own people? Or are they simply buying “time and space” on a service like Amazon AWS or Rackspace or RagingWire might offer?

    Report

  5. Michael

    Apparently, this attack is straining WPE resources. I have a client whose site went down with 502/504 errors on Saturday, and only was back up today at noon after rolling it back, and it’s still not 100%. They have a dedicated server and of course pay a hefty premium for their service, and I still haven’t gotten an explanantion.

    Report

  6. Chris Cree

    WP Engine had a rough month with their infrastructure provider between the data breach and the massive DDoS attack this week against Linode. Losing their entire Atlanta data center and having all our sites offline for the better part of 48 hours straight was painful, no doubt.

    Personally I’m grateful that WP Engine is on the job when things go sideways like this. Changing all the passwords across all our hosting accounts is a hassle. But having cleaned up hacked websites for clients I know full well that the hassle of sorting through mass password changes is far less than dealing with a slew of hacked sites. And that would have been a very likely scenario if they’d been more worried about appearances than the security of their customer’s accounts.

    Their willingness to own up to the problems and provide solutions moving forward builds trust and makes me more inclined to stay with them rather than moving to another hosting provider.

    Report

  7. Greg Ferro

    When doing business with WP Engine I was convinced it was a bunch of high school kids who got lucky.

    This pretty much proves my viewpoint. Thats why I quit their platform.

    Report

Comments are closed.

%d bloggers like this: