Elastic Hits Back at OpenSearch, Making Client Libraries Incompatible with Amazon-led Open Source Fork

After Elastic, makers of the search and analytic engine Elasticsearch, re-licensed its core product so that it was no longer open source, Amazon led a community effort to fork it. In July 2021, contributors to the project announced the first general availability (GA) release of OpenSearch 1.0, an Apache 2.0-licensed fork of Elasticsearch 7.10.2 and Kibana 7.10.2.

In what appears to be a slap back at the open source fork, Elastic has begun making its client libraries incompatible with OpenSearch. The Python client was updated to perform an API request that will verify connection to Elasticsearch and raise an error if it doesn’t receive the proper response. The PR received 40 “thumbs-down” reactions from the community and a brief round of criticism before the discussion was shut down.

“It’s disappointing to see this,” Invenio product manager Lars Holm Nielsen said. “You’re forcing us as bystanders in a battle to choose sides. We develop an Open Source product that could likely easily work with both Elasticsearch or OpenSearch and then the users can choose for themselves if they want Elasticsearch or OpenSearch.

“Now, instead, we likely have to make choice for all our users if we want OpenSearch or Elasticsearch. This and other behaviors from Elastic really does not give me any confidence in Elastic and what you might do in the future. And don’t blame it all on Amazon – you’ve already changed the server license, you didn’t have to make this move.”

Elastic Senior Engineering Manager Philip Krauss responded before turning off comments on the discussion.

“Amazon OpenSearch is a different product,” Krauss said. “And while there is some shared history, there are already many differences that cause real confusion and issues.”

https://twitter.com/amiantos/status/1423063593541541890

Elastic has also modified its .NET Connector for Elasticsearch to include “a pre-flight check on first use,” which users do not consider to be an enhancement.

Elastic Senior Engineer Steve Gordon said the change is not breaking in supported configurations and that the intent was “to make this incompatibility explicit by failing fast to avoid consumers incorrectly assuming they are running in a supported configuration which is not tested and may not function as expected.”

Last week, OpenSearch responded to Elastic’s recent changes that render many clients incompatible, by committing to create a set of new client libraries that make it easy to connect applications to any OpenSearch or Elasticsearch cluster:

Many developers who use Elasticsearch and OpenSearch in their applications also make use of the open source client libraries maintained by Elastic, which provide convenient high-level interfaces for several popular programming languages. Over the past few weeks, Elastic added new logic to several of these clients that rejects connections to OpenSearch clusters or to clusters running open source distributions of Elasticsearch 7, even those provided by Elastic themselves. While the client libraries remain open source, they now only let applications connect to Elastic’s commercial offerings.

OpenSearch published a list of a dozen clients for which contributors plan to create forks that will maintain compatibility with all Elasticsearch distributions, even those produced by Elastic.

“We do not recommend updating to the latest version of any Elastic-maintained clients, as this may cause applications to break,” OpenSearch maintainers urged users in the latest project update.

Elastic’s decision to prevent official clients from working with open source forks has further undermined any remaining goodwill the company had after re-licensing Elasticsearch.

“Looks like Elastic has sucked all the benefit they could from open source and is now spitting out the bones,” OSI Director of Standards and Policy Simon Phipps said.

10up, makers of the ElasticPress.io service, one of the most prominent Elasticsearch-powered products in the WordPress ecosystem, is still considering its next move after Elasticsearch abandoned its open source licensing. The company is not in any hurry to choose sides. Vasken Hauri, 10up’s VP of Platforms and Systems, said the dispute “isn’t something that we’re concerned about in the near term (the next 2-3 years).”

Upgrading past Elasticsearch 7.11+ would require making a choice between continuing on with Elastic’s proprietary offering or switching to the open source fork. Hauri said that the company is “barely taking advantage of most of the features Elasticsearch offers now” and projects that the current roadmap “could probably run another couple of years without any need to get new features from Elasticsearch.” For the time being, the 6,000+ users of the ElasticPress WordPress plugin and customers of the ElasticPress.io service have nothing to worry about as a result of Elastic’s renewed war with Amazon.