Jetpack Re-launches Search Feature as Standalone Service

The Jetpack team announced today that its search service is now available as a separate package from the professional bundle it was previously under. The move should make it more affordable for small sites to use the service. Larger sites might have to shell out more money, depending on how much content is on the site.

“Jetpack Search had only been available as a part of the Jetpack Professional plan at $29 per month, but that pricing felt unfair: a site with one million posts costs 1,000 times as much to index as a site with one thousand posts, but both sites were paying the same amount,” wrote Greg Brown, search wrangler at Automattic, in the announcement post. “New pricing makes Jetpack Search more affordable for small sites and still costs big sites less than the competition.”

Jetpack Search now charges by the number of records:

  • Up to 100 records: $5 per month
  • Up to 1,000 records: $10 per month
  • Up to 10,000 records: $25 per month
  • Up to 100,000 records: $60 per month
  • Up to 1 million records: $200 per month
  • Over 1 million records: $200 per month per million records

“Records” are posts, pages, and custom post types. However, that may change in the future. “We decided to use the term ‘records’ because we think there are use cases where we may want to index comments as their own records sometime in the future,” said Brown. “Similarly, there are use cases for indexing authors, tags, categories, etc. as their own records.”

Jetpack Search combines comments and bbPress plugin replies, which are technically a custom post type, with their parent post. For now, these will not count against users’ record counts.

One caveat with search is that media attachments are considered individual records by default. There are good reasons for some sites to expose attachments via search results. However, this could lead to unwanted records and cost more money each month for users who have a lot of media but do not wish to count those pages. Jetpack does have a filter hook that controls which post types are cached on the WordPress.com servers, so this can at least be configured via code.

Version 8.4 of the Jetpack plugin also includes an updated search interface. When a site visitor enters a search query, a fullscreen modal appears to showcase the search results. The results are snappy because they rely on the WordPress.com API and updated technology underneath. The new interface should work well with most themes out of the box on both desktop and mobile views.

Screenshot of the new Jetpack search results.
Jetpack search results modal.

“This new version of Search is only available with the new plan,” said Brown. “The previous version of Search is still available on the Pro plan and will continue working as-is for the foreseeable future, but our focus going forward is on the new search experience.”

New Architecture Behind Jetpack Search

The new version of Jetpack Search runs on a re-architected technology layer. The old system was an Elasticsearch wrapper around the standard WordPress search feature. While it was quicker than normal WordPress search results, it was still slower than it needed to be. There was no built-in spelling correction. Searched terms were not highlighted.

The new system uses the same sync technology behind Jetpack features like Related Posts, Publicize, and Stats to cache content. It then builds a search index using the Elasticsearch engine from the cache.

The team has built an optimized API, which now supports searching as the user types and uses pageview stats as part of the algorithm in result rankings. Search input also goes directly from the browser to the API on WordPress.com, which minimizes the delay in returning results.

“All search engines work by preprocessing the data to ensure that when a user’s search query comes in the results can quickly be returned,” said Brown. “For Jetpack Search, very little of the processing time on our servers goes to processing the queries. Almost all of the server load comes from indexing the data and reindexing it as the content changes. Across our main search cluster, for instance, only 10% of the CPU time is spent processing about 850 search queries per second. Most of the time is spent on the 650 index operations per second.”

Brown explains this new architecture is why the new pricing tiers are based on the number of records cached from a site. “We preprocess the data, which expands how big it is; store it on the fastest SSDs we can buy; replicate it to multiple data centers; and then can quickly serve results as soon as a search request comes to our API.”

For readers who are interested in a more technical breakdown of the system, Brown has written a full overview titled Real-Time Elasticsearch Indexing on WordPress.com on the Data for Breakfast blog.

13 responses to “Jetpack Re-launches Search Feature as Standalone Service”

  1. I think that the search volume, which if you extrapolate out Greg’s per-second number is about 73M queries a day right now, could go up a ton. I’m curious how that number compares to other WP search solutions. Most sites underuse search because it’s slow or doesn’t have great results. This service fixes both, and I could see being a big revenue driver for eCommerce sites especially.

    • How can it be a big revenue driver for eCommerce sites especially if search results do not show images? The vast majority of e-commerce stores do not want their search results to look different than their categories and an overlay UI is in the opposite direction of that. But no images too? That’s a deal-breaker for most.

      • The vast majority of e-commerce stores do not want their search results to look different than their categories and an overlay UI is in the opposite direction of that. But no images too?

        These are both things we are aware of and have been working on. There is already an initial prototype merged, but we still have more to do before it is ready for production. One thing that may not be clear is that the overlay can also be opened by clicking on links on the site. So for e-commerce we can display a list of categories (in a sidebar or menu) and those links will then immediately open the overlay and apply the filtering. This way we make any discovery task on a site very fast which is why we think it can greatly improve revenue on eCommerce sites.

        Thanks for your patience as we continue to improve the product for different use cases.

  2. It surprizes me that this would be split it into a service since the modus operandi seems to be to just put everything and the kitchen sink in Jetpack, forcing users that want just one feature to enable the massive “plugin” (service) that is Jetpack.

    I guess this is going to be another revenue stream for Automattic – and together with the upselling within the service I think this will make Matt a nice bit of cash.

    • It doesn’t surprise me. I imagine Automattic can make more money in the long term with this tiered-pricing approach. Plus, for smaller sites, a lower monthly price is likely more enticing. $5? $10? That’s a cup of coffee or two. It’s cheaper than their Netflix bill. More people will justify that cost over the previous $29/month.

      I do wonder if larger sites will look elsewhere. Jumping to $60, $200, or more each month might put a dent in the pocketbook. At the same time, sites that large tend to have more funds on hand to cover the cost.

  3. Does anyone know how this compares to Algolia? We’ve been using Algolia’s free plan for years on a site with 500+ articles. Their WordPress plugin integration is pretty great with instant search, autocomplete, etc. It’s so good that we still use it even though it’s unsupported for about a year now (https://community.algolia.com/wordpress/).

    If anyone has experience with the two, please share your thoughts. The fact that JP is WordPress native is great, but I don’t know how it could be better than Algolia at search since that’s all they do. I guess the question comes down to: Is JP search good enough to make up for the lack of ongoing support for the Algolia plugin.

    I haven’t played with Algolia’s new PHP client yet, but maybe that’s easy enough to integrate.

    • I haven’t used the other plugin, so I can’t compare the two.

      You can check out a live demo of Jetpack Search. I’m unsure how long that demo will be up, but it may give you a better idea of how it works.

      I did want to make a small correction. Jetpack is not WordPress native. It is a third-party plugin made by Automattic. The “under the hood” part of the new search feature basically happens outside of WordPress and communicates with an API on Automattic’s servers.

    • The best place to try it right now is on CSS Tricks. Lots of content, nice integration, and when I asked Chris Coyier yesterday about how the upgrade experience went he said: “yeah there was nothing to it”.

    • I’ve used Algolia outside of WordPress and it’s worked great. One strong advantage I see with Algolia is that search is their entire business–it’s all they do, and they do it well.

    • I have been using Algolia’s free tier on my website for years. I still use their plugin (which Luke mentioned above), but that’s now discontinued and no longer officially supported. (They recommend the PHP client now, but since the plugin still works fine I see no reason to switch.) I found it pretty easy to implement my own search result and autocomplete templates, too.

Newsletter

Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

%d bloggers like this: