My Performance Woes

For some unknown reason, the performance of this website over the past 2-3 weeks has been terrible. While writing posts, I would routinely see a warning that said I was disconnected from the server while taking forever to reconnect. Navigating the back-end of the site was a major pain as each page seemed to take forever to load. I also saw Internal 500 Server errors as well as MySQL database timeouts. While trying to diagnose what the problem was, I used the YSlow FireFox Extension with Firebug and saw GET requests taking 3-20 seconds to load.

Long Get Requests

Fed up with the performance, I created a trouble ticket with HostGator. I’ve never had a problem with this company in the two years I’ve used them and the support has been stellar. You can read their response to me in its entirety here but the gist of their response was that certain plugins were creating such a load on the server that they were the source of my problems. They specifically mentioned these plugins:

  • All related posts plugins (WordPress Related Posts, YARPP) can cause significantly high load in most cases.
  • WPRobot3 and other auto-posters can also cause high load issues, and should be disabled if they are causing issues
  • StatPress and other wordpress statistics software should also be disabled, as these too can consume too much CPU in certain cases. Use Google Analytics instead for statistics as well as Awstats which already comes by default on your account.
  • WP Post Views is also a plugin that you’ll want to avoid as it causes significant resource usage.
  • All In One SEO Pack has also been found to cause issues in many cases and we generally recommend using Greg’s High Performance SEO Pack in its place.
  • Any other plugins that are not vital to your WordPress should be disabled.

I knew at some point, YARPP was going to give me problems because of the resources it uses to create the relationships between posts. The weird thing is that my WordPress install has performed fine for a several weeks without a problem. All of the issues occurred suddenly. I wonder if YARPP reached the point where it was hitting the performance limits. Another plugin I’ve had problems with over the years is W3 Total Cache. Caching plugins have caused me so many problems that I’ve finally decided not to use them. In my opinion, using various techniques and optimizations, this site doesn’t need to use either to be fast. I only used one or the other because of the majority of WordPress literature that says you have to.

So while I didn’t put a lot of thought and credibility into their response, I decided to go through my list of installed plugins and make the hard call on whether I needed them or not. It turns out I was able to axe about 6 plugins from my install. Those 6 did include W3 Total Cache and YARPP. I also took this opportunity to go inside of the database to drop and empty tables that were no longer necessary. While performing these tasks, I came across a few tables from plugins I had uninstalled about two years ago. This will be a subject for another post. After completing the spring cleanup, I’m happy to report that the site both on the front and back-ends is much more responsive. I’ve yet to run into a timeout or MySQL error. I can’t figure out which part of the clean-up process offered me the most benefit but I’d put money on the removal of both W3 and YARPP.

After announcing the results on Twitter, most people weren’t surprised to hear about YARPP possibly causing issues. It’s a shame because using Woopra to watch website traffic in real-time, I was able to see a bunch of people clicking on related posts through the YARPP widget. I’m now on the hunt for a service that can provide the horsepower to display related posts similar to how I use Swiftype for search. The first service I’m going to try is called nRelate. Look for a review in the near future.

Just Switch Hosts:

Why is it that at the first sign of trouble with a webhost, the first thing people tell you to do is move away to a different one? Have they not experienced what it’s like to move between webhosting companies? It’s not fun, at all. I use it as the nuclear option because the experience is so painful. This is a good time to reference an article I wrote in 2010, 14 Things To Consider When Choosing A Webhost For Your WordPress Powered Site. While there is a lot of great information in that post, the biggest take away is this:

Always type into Google “Name Of Webhosting Company Sucks” to get a grasp on the issues that have been reported with that particular webhost. When webhosting companies screw up, people are very vocal about it and will stop at nothing to complain to all who will listen. Unfortunately, the side effect to this research is that you’ll find out every webhosting company in existence, sucks.

It doesn’t matter which webhosting company I choose, someone out there will tell me how terrible they are and to switch to a different host. Talk about the ultimate feedback loop.

36

36 responses to “My Performance Woes”

  1. If you have poor database performance and many posts, YARPP will destroy your server.

    I rewrote it in-house so it never does just-in-time indexes and instead I allow it to only index when the post is first published and never again.

    Then I wrote a command line tool to slowly index the database (for imported posts) and pause based on server load so it only does a few at a time and eventually indexes the whole site over many hours transparently.

    The problem is YARPP does complex full-text queries and then a massive unindexed query that has to scan the whole table, for every post. So if you have many posts and a weak server, it will really destroy performance.

    Note that YARPP has http://wordpress.org/plugins/yarpp-experiments/ which can help pre-index your posts and you can run it at off hours (it does all those bad queries at once) but it’s also very flawed.

  2. Oh and with regard to hosting, if you cannot afford/justify dedicated (and maybe consider options like OVH if you are skilled enough to self-manage) stay the heck away from Virtuozzo (aka Parallels) VPS based hosting providers. VZ simply cannot regulate disk load properly, no matter what the host claims. Bad neighbors on the same node will completely ruin your life.

    Instead, if you must use VPS, look for Xen or KVM based hosts. Xen is much more heavily regulated and you will have a much smoother time, even with “bad neighbors”.

    If you have no other options, OVH has low-cost entry level dedicated with SSD which will allow you to get away with even the worst behaving plugins. But you absolutely must know what you are doing to use them. Strangely there isn’t much competition to their pricing yet from other hosts, but hopefully soon.

  3. It’s a bit like a last hurrah for “your time with the masses”. A quick opportunity to knock non-affiliated companies before moving into the fold. Yes, shared hosting is not that good. Oh look how amazing my new VIP hosting is. ^_^

    I have been recommending non-generic hosting to people for years, and have continued that with the rise of hosting dedicated to WordPress. This seems like an excellent opportunity for an article aimed at SOHO (http://en.wikipedia.org/wiki/Small_office/home_office) companies, extolling the benefits of WordPress specific hosting. Why it is an opportunity (like having a good lawyer) to spend money to, ultimately, save money.

    Also looking forward to your hinted-at article on plugins littering your database with unwanted tables…

  4. I’ve had great success installing cPanel on a Linode, (Xen) and with the recent increases in resources in their plans, you can buy a $20/month Linode plan with a $15 cPanel license, and get super duper performance without having to relearn everything.

    Your site loaded really slowly today. Hostgator isn’t very helpful. Actually, what you got from HostGator is the most helpful I have ever seen them, they’re usually defensive. I guarantee you will never get to the bottom of it.

    A friend who uses HostGator that just got relocated to a new server because of a major hardware upgrade on a shared plan and it made things better. If you didn’t already get upgraded, ask and see perhaps if they can relocate you ahead of time? (They got NO notice of the move, just fyi.)

  5. @Rob – That’s actually pretty expensive, you would be better off buying a VPS from a provider that buys server licenses in bulk from CPanel at a massive discount and incorporates it into their overall price. They would pay well under a dollar per year per VPS account, so, they can put more resources or management time into the VPS.

  6. @Andy Christian – Most would think that, including me. But it’s not going to happen. The VIP teams hands are full already of very expensive clients paying for their time. It doesn’t make much sense to put WPTavern in their way.

    @Eric Mesa – Considering how well this website is operating now, I’m not going to take the chance. Perhaps after a few days of speedy responsiveness, I’ll reinstall it just to see what happens.

    @Lester Chan – I have no idea why PostViews would generate so much load. All it does it move a ticker up each time the site is loaded. Maybe if a few hundred people were visiting the site it would be an issue? By the way, the WP Stats plugin they are talking about is different from the one you created. They thought I was using an analytical type of plugin when it was really your Top 10 stats plugin which probably doesn’t affect performance much at all.

    @Len – WPEngine explained on Twitter why W3 Total cache and all other caching plugins were blacklisted. It’s because they have their own caching implementation and those plugins end up interfering with their solution.

    @_ck_ – I have about 1,271 posts. Thank you for explaining why YARPP has the reputation that it does. It’s only now that I’ve had so much trouble. I know in the database, I saw a table that showed about 4,000 records for YARPP Related Cache and I wonder if between that and W3TC that the queries to update the YARPP cache was just as much as part of the problem. I guess once I hit a certain point, the effects of running YARPP become visible to the naked eye.

    As far as hosting, that will be taken care of in the near future. I don’t have to worry about hosting costs anymore. One thing I’m not ever going to get into is managing my own server or VPS.

    Off topic, how have you been CK? What have you been up to since the bbPress days? Are you still messing around with code behind the scenes? The last I read, you became so busy that you had to give up updating all of your plugins.

    @donnacha – Yes. I’ve actually thought about moving WPTavern to Drupal Gardens http://www.acquia.com/products-services/drupal-gardens I think the site would be a beautiful smelling blooming rose in their garden :)

    @Rob – Interesting. Since I performed the cleanup yesterday, I’ve noticed a substantial increase in speed and performance both on the front and back-end. When did you try accessing the site?

  7. @Rob – Look around the webhostingtalk forum, particularly the VPS offers section – I haven’t looked in a while but my expectation would be that you’ll find something around the same price but with more resources than the smallest Linode. Remember, too, that a managed service will save you time and headaches, so, figure that into your equation. My point is that $15 to CPanel is dead money, very few end-users buy those licenses direct.

  8. @donnacha – Wow, webhostingtalk. That brings back memories. That’s where I go to see how every host known to man sucks lol. It’s also where I learned how easy it was to start up a hosting company via purchasing a reseller account. Webhosting is a nightmare of a business that I never want to be in again.

  9. Hey Jeff, I have a hunch that the key to your performance woes might be the Ajax Edit Comments plugin.

    I notice that it takes a long time for the site to respond when posting, and the Edit feature and the Request Deletion feature don’t work at all, at least on my computer.

  10. @donnacha – Yep. Actually, I noticed your comment and deleted it through the AEC interface so that does work. I might have done it before you though so perhaps that’s why you saw an issue. As for editing, it’s a known issue. I created a support thread yesterday and am awaiting a response.

    http://wordpress.org/support/topic/clicking-the-edit-link-results-in-a-blank-popup?replies=1

    I’m thinking about removing all traces of AEC and starting over to see if that solves most of the problems. I really like everything AEC has to offer and I’ve yet to see any other plugin come close to its feature set. I hope Ronald finds a way to continue working on it or sells the plugin to a different company that will spend time on it.

    But to test out your theory, I’ve gone ahead and deactivated AEC for the time being. Lets see how that works.

  11. @Jeffro – Agreed, nightmarish business, especially when you depend so much on other people, but it gets a lot easier when develop a deep understanding of the underlying technologies.

    Of course, that takes years, and there are a lot of stressful situations before you get there, even some of the big name WordPress hosts have very little clue. In fact, I would go so far as to say that the entire “WordPress Specialist” niche is a scam, because they neither have deep knowledge of hosting or WordPress, they are just exercises in marketing.

    Anyone thinking about setting up as a Webhost today should learn the Hell out of Linux before they do anything else – it is way too easy to become seduced by cheap reseller account and the apparent simplicity of CPanel, you can end up in an awful mess, having to shell out hundreds of dollars to someone who has mastered the command line.

  12. @Jeffro – I really like the features of AEC, I wish every comments section allowed you to make corrective edits after posting, but I get the impression that development pretty much ended after Ronald got hired by iThemes, they certainly stopped honoring the lifetime licenses he sold.

  13. @Len – Ahh, I see now. I once tried to use Broken Link Checker on WPTavern which is also on that list and it too caused major problems lol. Never was able to see all the broken links on the site. Interesting to see Google XML sitemaps on that list. That’s another plugin I’ve used for along time but never thought it would give me problems. At least I don’t think it’s ever been a problem for me.

  14. On my other blog I use jetpack post views which get the post view data from jetpack, I have not checking that the plugin will create database table or not, but it seems to just waiting the update on jetpack and than echo the stat.

    Jane’s Related Posts is my choice to display related post, so far so good on my blog. But it only have 181 posts with around 500 views per day, don’t now if my blog getting bigger.

  15. Jeffro, I had a similar problem with Bluehost, which sits under the same umbrella as Hostgator. One of my lowest traffic sites just took ages to load, sometimes as bad as 30 seconds to load a majorly text page. This performance was bad inspite of using CloudFlare on the site.

    They gave me the usual, improve performance of WordPress and move to a premium plan. After nearly a month of frustrations, I shut my account and moved this site to my VPS.

    I can’t see how it could have been only WordPress related and not the servers. I have a feeling certain optimisations they did on their servers caused many sites to load slower (had a high wait time for each request).

    Regarding related posts, a simple pitch for my plugin Contextual Related Posts. In the latest version I added caching support that caches the output HTML directly in a meta field so there are no major FULLTEXT database look ups to find new posts after the first page load.

  16. @Jeffro – Life has been good indeed. Since our days on WLTC, I became an Investment Banker in 2009 and have been doing that as my full time job. Unfortunately, I never succeeded monetising my sites or experience to live a comfortable life as a freelancer.

    Time didn’t permit AEC and Ronald took it over completely. I think it’s pretty much out of major development and Ronald sold it off and took it back I think.

    I’ve got my own WordPress plugins the main ones being Contextual Related Posts and Top 10. And, some of which I plan on stop developing.

    I have been following WP Tavern forever, so am glad to know it’s back up and running again :)

  17. You should run static page caching. WP Super Cache, W3 Total Cache etc. are a pain sometimes, but if you switch to Varnish or a simpler caching plugin (like Batcache) then you will find things run a lot smoother. Of course, you can’t run the more advanced caching plugins without upgrading to a better host first, since most of them don’t support such features :/

    The performance advantages to running static caching a quite enormous. If you do load testing on your site (Apache Bench is useful for this), then you can see exactly how much of a difference that extra caching layer can provide. Even adding a simple object caching layer can cause massive improvements to page load times and total load capacity.

Newsletter

Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.