WordPress Async Task Library from TechCrunch Now Open Source


TechCrunch is one of the largest sites publishing with WordPress and its developers are always looking to improve performance to maximize engagement and ad revenue. This week the team open sourced its WP Async Task Library, which was created to offload time-consuming tasks into background tasks. Alex Khadiwala and Nico Vincent introduced the library at the Big Media & Enterprise Meetup in San Francisco within their “Non-Blocking WordPress” presentation.

Essentially, the purpose of the async task library is to identify anything on page load that blocks better performance and relegate it to a background task. Vincent and Khadiwala offer an example of how the library works to reduce API calls when loading CrunchBase cards:

On the TechCrunch site, loading CrunchBase cards on article pages is an example of a process that could slow down the site, since the information needed for each card becomes available via the CrunchBase API. To improve performance, we cache a copy of their data for about 12 hours. But when we initially retrieve that data or refresh it, we don’t want the API call to affect our page load time.

In both instances, we instead kick off an asynchronous call back to our site with the instruction to retrieve and cache this data to be available the next time it is needed, instead of waiting for a response from their API.

Prior to addressing TechCrunch’s performance issues, the site could take up to 17 seconds to load, as it was calling more than 100 assets. Thanks to the new task library, the situation has been dramatically improved. “Since the redesign, we’ve improved overall performance by 5 to 8 times by implementing the WP Async Task library, among other important back-end and front-end improvements,” the team reported.

WP Async Task is available as a standalone plugin or developers can bundle it into their own plugins and themes. The quick start section of the documentation shows you how to use it in your own code.

It’s always exciting when one of the larger WordPress-powered publishers make their tools available to the community, because these projects tend to be the result of many developer hours. WP Async Task has the potential to benefit many other media sites running on WordPress thanks to the TechCrunch development team. The code is now available on GitHub under the MIT license and the team welcomes your contributions.


3 responses to “WordPress Async Task Library from TechCrunch Now Open Source”

  1. It’s awesome that TechCrunch are giving back to the platform they use and open sourcing this plugin. Sites with massive traffic like TechCrunch need to solve different problems to the average website so it always great to be able to see how others tackle theses issues.

    It’s something that we try to do at our agency whenever we can, and in some cases we were able to commit code that was developed on bigger projects back into popular plugins that were used.

  2. I’ve been trying to think of use cases for this library.

    I agree with non-blocking and running heavy lifting tasks in the backend, rather than on the fly in the template etc.

    Unless the heavy lifting was directly connected to the action, ie, after save_post you need to do some stuff.. but for example of polling an API for something on the front end, this isn’t the nicest way to do it. IE save_post could be sporadic, and you’d obviously not want to do admin_init, etc.. as it’d be way too much.

    To date we just setup external crons that do heavy lifting and pre-populate transients, etc, so it’s super fast for the template to grab the data it needs.

    • Generally, I agree. Suffice it to say that, for some things, cron simply wasn’t a viable solution for us. Additionally, this allowed us to refresh ONLY the data we needed to refresh.

      With traffic like TechCrunch’s, getting a lock for the task on a front end page load is indeed a challenge. We’ve used the object cache to do it in some cases, which (if done right) only results in a couple of duplicate processes. In the end, though, 2 extra processes is much preferable to tacking on up to 30 seconds for the API request.

      Also, the majority of our uses of the library are actually on hooks like save_post. Alex and Nico just wanted to show off uses that visitors can actually see on the front end. ;)


Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

%d bloggers like this: