Interview With Stream Project Lead, Frankie Jarrett

Frankie Jarrett Featured Image

Stream 2.0 is a significant update that changes the stand alone plugin into a service. But not everyone is happy with the change. Those who use Stream in enterprise environments have voiced disappointment regarding the latest update. The following is feedback from a user on the Advanced WordPress Facebook group. “Heads up if you use Stream. The 2.0 upgrade now stores everything in the cloud instead of the local database and requires a account to use it. It’s a great plugin but this new functionality is not optional and I can no longer use it with our enterprise data.”

I reached out to Stream project lead, Frankie Jarrett, and asked why the team decided to rely on third-party services. I also inquired whether users have any options to house data on their own servers or connect it to a service of their choosing. Jarrett gives insight into the future of Stream as a Service and let’s us know if they are working on a version that is compatible with enterprise environments.

Interview With Frankie Jarrett

WordPRess Logging
photo credit: Claire L. Evanscc

Jeff – Why the decision to use an external service by default to offload Stream activity data?

Over the past 10 months we’ve learned a lot about logging events, specifically logging actions taken inside the WordPress Admin. As time went on, some of the biggest concerns we had revolved around the topics of performance and security. It became clear to us that Stream needed to be more than just a plugin to advance into a solid solution, it needed to be a service that was self-contained and lived alongside WordPress instead of trying to force it to work inside the WordPress architecture and never truly being scalable or secure.

MySQL is nice solution for storing content with simple querying, this is how WordPress uses it, but MySQL is actually bad for storing logs, especially if you want to retain them for a long time and/or run complex queries on them while also expecting those queries to be fast. Not to mention, you don’t want the performance of your website’s content to be affected at all. Since the primary purpose of the MySQL database is to store and serve up content to your website visitors, it was our view that should never be hindered by event logging.

Now that Stream is a service, we can use brand new technologies like Elasticsearch, that are better suited for (and even designed for) querying huge numbers of logs. The result is a more powerful querying performance, the possibility for users to do even more complex queries in the future (for their Reports), and have no worries about keeping logs for a very, very long time. The things we are now doing in Stream 2.0, and plan to do in the future, require the power of Elasticsearch and don’t translate into MySQL storage solutions.

In regards to security, websites and databases get hacked all the time. Unfortunately that’s just the way it is. Since all of Stream’s records had previously lived inside the website, it too was as vulnerable as the website itself. This means that any hacker that gained access could mess up a site and then cover up their tracks by simply deleting the Stream log data. This was a bad thing and meant those logs weren’t really a true security audit trail at all. Now that Stream is a service, those logs are untouchable by an intruder. Once an action is performed, it’s forever in the event history, so the site owner knows without a doubt what things have happened on their site and can go through an undo the damage.

Jeff – Why the connection between Stream and ID logins?

WordPress Stream Connection
photo credit: Manchester-Monkeycc

This was an easy decision for us, actually. Over the past few years there have been several WordPress companies that have had their sites hacked and user passwords have been compromised as a result. It’s a sad and unfortunate thing that can be avoided by simply not storing them. Our solution for this was to use SSO (Single sign on) powered by

This means Stream doesn’t have to store any login details for any customer and customers don’t have to sign up for yet another account somewhere. Furthermore, SSO supports two-factor authentication. This is a huge win for folks who are really concerned about the security of their logins, and we wanted Stream to have this capability.

The reason why we chose SSO was because of its status and reach in the WordPress community. Stream is a WordPress product and service, so it only makes sense to reach as many WordPress users as possible. When you think about all the people who use Jetpack, Gravatar, Akismet, VaultPress and Polldaddy – that’s a lot of people. Maybe not everyone, but again, we wanted to make a decisive decision not to store user login credentials at all, and that could mean some people might not be able to use it, but it’s for the good of all our users. SSO was also very easy to implement on our WordPress-powered site compared with the Facebook, Twitter or Google SSO alternatives.

Jeff – Is the connection between Stream and similar to Jetpack in that some things won’t work without the connection?

The only time Stream needs to talk to is during sign up, for login credentials. Stream doesn’t ping back to your website like Jetpack does. This means your site doesn’t have to be publicly accessible for Stream to work and can be run on a local/development environment without any problems or extra steps needed.

Jeff – Overall, what are the future plans for Stream now that it’s morphed into a service?

Future of Stream
photo credit: wwarbycc

Now that we have some scalability, performance and security milestones behind us, we are very much looking forward to making Stream even better in the coming months and years. You might have already noticed, but Stream 2.0 featured built-in integration with eight popular WordPress plugins. We intend to continue making Stream compatible out-of-the-box with tracking things that many other popular plugins do.

Another thing we plan to do is open up a REST API for people to be able to access their data and do anything they want with it. This is a very exciting prospect. Finally, we are working on ways to have a complete “mash-up” of all of your Stream data in one place. This is based on a lot of feedback we’ve been getting from folks who run not just multi-site, but multiple single-site installs for their clients and want to see everything that’s happening in one place. We think that will be another huge benefit to people and something that is only possible because Stream is now a service.

Jeff – One of the complaints I’ve seen is that Stream’s reliance on third-party services makes it incompatible with enterprise environments. What is the team doing to address this issue?

The new Stream relies on the power of Elasticsearch for performance and complex queries, but we are exploring ways for the Stream service stack to be run on-premise for Enterprise organizations who have strict internal policies that would require that. We don’t have have an ETA on when this type of solution will be ready, but we are actively pursuing it.


6 responses to “Interview With Stream Project Lead, Frankie Jarrett”

  1. Seems to me automattic is trying to control entire WordPress ecosystem slowly but surely. First there was the spech about curated list of plugins then came the buy out of WPMU. There was always Jet pack which ties everything up with WPMU plugins will surely go through this too.

    Now Stream which ties in with Its a slow but very deliberate process. Not disparaging what wordpress is or what automattic is doing. Just the direction the entire process is heading to ….

    Its becoming closer and closer to what Drupal is except there seems to be a definite control element here when you connect things to

    • You’ve obviously been fooled by an April Fools Joke post.

      Incsub, the parent company of, has not been acquired by Automattic nor will it likely ever be acquired by Automattic. It was an April Fools prank posted by it’s founder who has had a historically combative relationship with Automattic, Matt and the greater WordPress community in general.

      The decision to incorporate the single sign in functionality was entirely a decision made by the team at Stream. This has nothing to do with Automattic whatsoever. They could have just as easily went with using Facebook or Twitter’s single sign in API but chose to utilize because it’s more targeted to the market in which they cater to.

      As for JetPack, i’m not sure what that has to do with the conversation at hand other than the fact it also uses a account. But in that case i’m not sure what exactly you would be expecting considering JetPack is primarily geared towards bringing features and services to self hosted WordPress installs.

      Matt and Automattic are not trying to control the entire WordPress ecosystem. The WordPress ecosystem is far bigger than either of them.

  2. I definitely understand the issues that the Stream team is trying to tackle. We encounter them ourselves with our own product. There is functionality that we simply can’t provide the way we’d like to provide it because of the shortcomings of being a downloadable plugin with no control over the server environment in which it is executed.

    The irony here is a lot of enterprise customers want functionality and performance that won’t necessarily be possible in a standalone plugin designed for a MySQL database environment. Especially when it comes to things like analytics, logging, etc.

    I think it’s a great move for Stream and ultimately will make the product far better for all kinds of customers, but especially enterprise customers which typically have more demanding needs and more traffic. Even if some don’t quite grasp that now.


Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

%d bloggers like this: