Interview With Stream Project Lead, Frankie Jarrett

Stream 2.0 is a significant update that changes the stand alone plugin into a service. But not everyone is happy with the change. Those who use Stream in enterprise environments have voiced disappointment regarding the latest update. The following is feedback from a user on the Advanced WordPress Facebook group. “Heads up if you use Stream. The 2.0 upgrade now stores everything in the cloud instead of the local database and requires a WordPress.com account to use it. It’s a great plugin but this new functionality is not optional and I can no longer use it with our enterprise data.”

I reached out to Stream project lead, Frankie Jarrett, and asked why the team decided to rely on third-party services. I also inquired whether users have any options to house data on their own servers or connect it to a service of their choosing. Jarrett gives insight into the future of Stream as a Service and let’s us know if they are working on a version that is compatible with enterprise environments.

Interview With Frankie Jarrett

Jeff – Why the decision to use an external service by default to offload Stream activity data?

Over the past 10 months we’ve learned a lot about logging events, specifically logging actions taken inside the WordPress Admin. As time went on, some of the biggest concerns we had revolved around the topics of performance and security. It became clear to us that Stream needed to be more than just a plugin to advance into a solid solution, it needed to be a service that was self-contained and lived alongside WordPress instead of trying to force it to work inside the WordPress architecture and never truly being scalable or secure.

MySQL is nice solution for storing content with simple querying, this is how WordPress uses it, but MySQL is actually bad for storing logs, especially if you want to retain them for a long time and/or run complex queries on them while also expecting those queries to be fast. Not to mention, you don’t want the performance of your website’s content to be affected at all. Since the primary purpose of the MySQL database is to store and serve up content to your website visitors, it was our view that should never be hindered by event logging.

Now that Stream is a service, we can use brand new technologies like Elasticsearch, that are better suited for (and even designed for) querying huge numbers of logs. The result is a more powerful querying performance, the possibility for users to do even more complex queries in the future (for their Reports), and have no worries about keeping logs for a very, very long time. The things we are now doing in Stream 2.0, and plan to do in the future, require the power of Elasticsearch and don’t translate into MySQL storage solutions.

In regards to security, websites and databases get hacked all the time. Unfortunately that’s just the way it is. Since all of Stream’s records had previously lived inside the website, it too was as vulnerable as the website itself. This means that any hacker that gained access could mess up a site and then cover up their tracks by simply deleting the Stream log data. This was a bad thing and meant those logs weren’t really a true security audit trail at all. Now that Stream is a service, those logs are untouchable by an intruder. Once an action is performed, it’s forever in the event history, so the site owner knows without a doubt what things have happened on their site and can go through an undo the damage.

Jeff – Why the connection between Stream and WordPress.com ID logins?

This was an easy decision for us, actually. Over the past few years there have been several WordPress companies that have had their sites hacked and user passwords have been compromised as a result. It’s a sad and unfortunate thing that can be avoided by simply not storing them. Our solution for this was to use SSO (Single sign on) powered by WordPress.com.

This means Stream doesn’t have to store any login details for any customer and customers don’t have to sign up for yet another account somewhere. Furthermore, WP.com SSO supports two-factor authentication. This is a huge win for folks who are really concerned about the security of their logins, and we wanted Stream to have this capability.

The reason why we chose WP.com SSO was because of its status and reach in the WordPress community. Stream is a WordPress product and service, so it only makes sense to reach as many WordPress users as possible. When you think about all the people who use Jetpack, Gravatar, Akismet, VaultPress and Polldaddy – that’s a lot of people. Maybe not everyone, but again, we wanted to make a decisive decision not to store user login credentials at all, and that could mean some people might not be able to use it, but it’s for the good of all our users. WordPress.com SSO was also very easy to implement on our WordPress-powered site compared with the Facebook, Twitter or Google SSO alternatives.

Jeff – Is the connection between Stream and WordPress.com similar to Jetpack in that some things won’t work without the connection?

The only time Stream needs to talk to WordPress.com is during sign up, for login credentials. Stream doesn’t ping back to your website like Jetpack does. This means your site doesn’t have to be publicly accessible for Stream to work and can be run on a local/development environment without any problems or extra steps needed.

Jeff – Overall, what are the future plans for Stream now that it’s morphed into a service?

Now that we have some scalability, performance and security milestones behind us, we are very much looking forward to making Stream even better in the coming months and years. You might have already noticed, but Stream 2.0 featured built-in integration with eight popular WordPress plugins. We intend to continue making Stream compatible out-of-the-box with tracking things that many other popular plugins do.

Another thing we plan to do is open up a REST API for people to be able to access their data and do anything they want with it. This is a very exciting prospect. Finally, we are working on ways to have a complete “mash-up” of all of your Stream data in one place. This is based on a lot of feedback we’ve been getting from folks who run not just multi-site, but multiple single-site installs for their clients and want to see everything that’s happening in one place. We think that will be another huge benefit to people and something that is only possible because Stream is now a service.

Jeff – One of the complaints I’ve seen is that Stream’s reliance on third-party services makes it incompatible with enterprise environments. What is the team doing to address this issue?

The new Stream relies on the power of Elasticsearch for performance and complex queries, but we are exploring ways for the Stream service stack to be run on-premise for Enterprise organizations who have strict internal policies that would require that. We don’t have have an ETA on when this type of solution will be ready, but we are actively pursuing it.