New Core Web Vitals Technology Report Shows Overall Pass Rate for WordPress Sites Decreases with Newer Versions

Some new data from a recent Core Web Vitals (CWV) technology report produced by the HTTP Archive shows WordPress sites running newer versions have lower CWV pass rates.

The original report was published in July by Rick Viscomi, one of the maintainers of the HTTP Archive. The site provides a permanent repository of web performance information, giving researchers a common set of data for researching and understanding trends. Contributors’ efforts are sponsored by Google, Mozilla, New Relic, Etsy, and other companies.

One of the most notable findings in Viscomi’s report showed that just 22% of WordPress-powered origins pass the Core Web Vitals “Good” threshold.

WordPress core committer Adam Silverstein wanted to dig a little deeper into this data set to see if he could extract more information about WordPress sites’ CWV performance that wasn’t represented in the initial published graphs. He proposed an analysis that would compare Core Web Vital performance across WordPress versions:

How have CWV scores changed over WordPress versions? Are there measurable improvements in the wild after recent changes like adding native image (version 5.5) and iframe (version 5.7) lazy loading and WebP image support (version 5.8)?

Silverstein worked with Viscomi to create a query that would extract performance data grouped by WordPress version. He found that core additions like native image and iframe lazy loading, and WebP image support have had no measurable improvement on CWV scores in the wild.

“Lazy loading may be too aggressive as it is applied to all images,” Silverstein said, noting that lazy loading can be detrimental if over used. This should be remedied soon. Google-sponsored WordPress core committer Felix Arntz opened a ticket to improve lazy loading, which will be included in WordPress 5.9.

“WebP adoption in WordPress has been growing since the 5.8 release, however users need to manually convert their images to WebP before uploading to take advantage of the format,” Silverstein said. “Landing WebP as the default format for sub-sized images which was started in this ticket will have a much bigger impact by automatically converting uploaded images to WebP.​​”

A few highlights of Silverstein’s observations from the analysis include:

  • 70% of origins are on the latest version of WordPress and 88% are on one of the last two versions, meaning changes we make to core reach the majority of sites relatively quickly.
  • The number of origins is quite low for older versions of WordPress, with fewer than 5k origins for most versions before 4.7
  • Overall CWV pass rates have generally decreased over WordPress versions. Although it might also be the case that “leading-edge” websites that update to the latest version are generally slower than those that linger on older versions.

Silverstein anticipates this analysis will provide the basis for tracking major improvements in the future. The Google-sponsored WordPress contributors on his team are active in certain core projects and are leading WordPress’ new performance team with the goal of improving core performance as measured by Google’s Core Web Vitals metrics.

“Basically I wanted to create a way to measure the impact of core WordPress improvements on WordPress sites (at scale),” Silverstein told the Tavern. “My team at Google is focused on helping improve the performance of the web at scale, and WordPress is a huge part of that! You may have noticed us working on features like lazy loaded images and iframes, WebP image support and now helping start the performance group. I wanted to find a way to see if our work is having a measurable impact – and not just on a vanilla WordPress site you might set up for testing, but in the wild, or real world websites that upgrade to the latest version of WordPress. That is the goal of the dashboard.”

The new dashboard, which tracks WordPress CWV performance by version, is available to the performance team for monitoring their progress with each new WordPress release. Google-sponsored contributors are using it to measure the impact of their efforts across various performance initiatives.

20 responses to “New Core Web Vitals Technology Report Shows Overall Pass Rate for WordPress Sites Decreases with Newer Versions”

  1. I think after AMP trust dilution, my take would be that CWV scores do not matter. The only thing that matters is that the websites should load fast, right ?

    • I think the issue is that if you expect great performance from any cdn out the box you will be dissapointed. WordPress has the he highest rate of use so pretty much all these points are valid but remember the web hasn’t gone through a linear change in packet size security threats or memory use, the fact there is an uptrend in the engine running 45℅ of the net has got to be good. A large amount of these sites expect out the box performance and that just doesn’t exist.

  2. According to original report 37.41% websites running on Shopify have good CWV, while WooCommerce has a paltry 11.72%. But guys at Automattic still prefer to spend their time developing Admin and Blocks…
    Partially this difference can be explained by the fact that Shopify provides decent hosting with CDN out of the box while many small WooCommerce shops are hosted on some shared hosting. But there are clearly huge problems on WordPress and WooCommerce sides.

    • CDN won’t speed up slow TTFB with WooCommerce.

      About development priorities, well, I guess you’re right.

  3. Me thinks Google keeps moving the goalposts with CWV. One day it says one thing, the next something else and across all testing platforms ( GTMetrix etc.) results vary. Think they are taking the Mick to be honest. A bit like the AMP shenanigans.

    The latest UI update of Page speed Insights even has a UX faut-pas as the results are now pushed below the fold in the browser window meaning a little bit of scrolling to view.

    To boot, with all this chopping and changing, it keeps us constantly in thrall to testing and this all adds up to burning servers unnecessarily… Have they not heard of climate change?

  4. I also didn’t find it to be informative at all. They seem to be ascribing a cause (on very dubious data) to the wrong things – they focus on lazy loading and other recent features, but the real story is the continual downwards slope across 20 versions… What’s that about? The only conclusion you could reasonably draw from this extremely limited data is that WordPress is getting inherently less performant over time. I can’t imagine that’s the case. So, there’s something else going on.

    Is there a way to measure the total “load” for these sites? Are newer sites simply using more media, plugins, features, interactivity etc…?

    What hosting are they all on? A defunct text-based blog on 3.5 will run well on even the worst shared hosting. But a “modern” site with ecommerce, social networking etc… will struggling even on many of the most popular VPS providers.

    Is there caching? CDN? etc…

    There’s surely a dozen other questions that could be asked with minimal scrutiny. If it isn’t possible to get the data to at least attempt to answer them, then perhaps publishing a “study” like this should never have happened…

    As such, its not particularly encouraging that this all seems to have evaded the people who are tasked with improving the performance of WP…

    • Why do you think that “this all seems to have evaded the people who are tasked with improving the performance of WP”?
      AFAIK Performance team never backed their decisions by these statistics.
      Also the team was created only a few weeks ago and they just started the work in the groups where there are already enough people.

      • Fair enough. I’m sure they’ve thought of a lot things with regards to the performance project. But I believe my point still stands – this “study” is so thoroughly uninformative that it begs the question why it was even published.

  5. Am I alone in my worry about “too close” a connection with Google & Google-sponsored entities (whatever “too close” means, I am not sure myself—knowing their ubiquity and influence on Web advertising, I worry, possibly excessively. [I am also an amateur.])

    • CWV are at least much better than different numeric scores. It was possible to get a perfect GTMetrix score for a very slow page, while CWV represent real user experience much better. It’s not perfect by any means but rather usable in my opinion.

      • It’s still possible for a fast site to get poor grades and vice-versa on CWV.

        • Fast site can get poor grades indeed for different reasons (usually this means that the site still can be improved) but it won’t get poor CWV.

  6. Interesting, but so many problems and limitation in a data set like this.

    The first most obvious one is that sites not getting core updates are likely simpler in design than more modern sites. That a site running TwentyTen rocks CWV has nothing to do with whether it’s running WP 3.6 or 5.8.

    This analysis really only makes sense if you control for themes and assume someone hasn’t done something stupid (like they so often do) and added 32 differently sized images to a slider.

    The other big problem, which they rightfully identified themselves, is everything is about the root/home page. No one seriously evaluates site performance based on a single page, let alone the single page most likely to be nothing like the other pages of the site.

    I get HTTP Archive doesn’t want to be a “full-blown web crawler” but even crawling the first 10 nav menu links would give WAY more meaningful results.

    The biggest thing WP needs to do is figure out how to make all the dynamic content on a page AJAX. So dynamic pages can be page cached and a single GraphQL query can pull in the dynamic bits. This is the advantage to things like JAMStack. WP wasn’t architected to be performant in dynamic page scenarios. Heck, it still doesn’t natively lazy load blog post comments!

  7. If all of this data comes from HTTP Archive sites in the wild, not in a sandbox, the differences in speed across versions could be correlation, not causation. A website running a version of WordPress that was released 11 years ago is also likely to have a very old theme with a design from a different era. It could just be that websites are loading larger media files than before. Or maybe sites that haven’t been updated for that long are smaller and simpler.

    • You could try this:

      Install various WordPress branches from 3.x to current with e.g. Twenty Fourteen theme, it existed back then and still seems to be updated today
      Import themeunittestdata.wordpress.xml from codex.wordpress.org/Theme_Unit_Test
      Run automated speed tests on all installs

  8. I think it’s extremely challenging to achieve 100/100 for dynamic site with WordPress/WooCommerce. This is because most WordPress site are built for general purpose which means one plugin are playing a lot of roles with a lot of features and functionalities. That can get WordPress site bloated with lots of unnecessary features.

    Don’t get me wrong, with WordPress it’s still possible to achieve 100/100 for both mobile and desktop. But it requires lots of customization and it needs to work with the codebase. I’ve spent years to optimize my WordPress site that have WooCommerce and LearnDash Running on it. It’s possible to achieved 100/100 with WooCommerce and LearnDash installed. But it took me tons of time to do lots of research, checking, modification and custom stuffs.

  9. Overburdened databases, cheap servers being used, etc., could all contribute to this. Not a scientific comparison unless you’re comparing apples to apples.

  10. It’s worth noting that this data is only gathered from tests on “Chrome for desktop and emulated Android (on Chrome) for mobile”. So if you care to know about actual usage across WordPress supported browsers, this dataset is very skewed. I maintain several very large traffic sites and they are about evenly split with Chrome and Safari visitors. https://httparchive.org/faq

    Also, it was stated here in other comments, but correlation is not causation.

  11. There are more tools that could give accurate reports other than CWV. I will say that word press is been greatly used by digital marketing persons where they don’t need to depend on developers, but it is as well challenging to sort out certain issues.

Newsletter

Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

%d bloggers like this: