Cancel

Jeff Chandler

Stream Morphs From a Plugin Into a Service

stream plugin banner

Stream 2.0 is available for download and includes a plethora of enhancements. This version features a rewrite from the ground up with a focus on scalability, security, and activity. As part of the rewrite, Stream activity data is stored in the cloud using Amazon Web Services with Elasticsearch. This is the same type of setup Jetpack uses to power its Related Posts module.

Connect To Stream Service
Request For a WordPress.com ID

The data is stored over an SSL connection making it hard to tap into your activity stream. The Stream team explains the plugin as being the black box of a WordPress site that even the NSA can’t penetrate. As part of the security enhancements, Stream uses your WordPress.com ID to authorize your account.

After connecting my WordPress.com ID to Stream, it loaded a Plans and Pricing page in place of the backend instead of just connecting my account. This is unexpected behavior and a disappointing user experience. I ended up having to load the WordPress backend in a new browser tab.

I ran into a loop where each time I logged into the backend of WordPress, I’d see the Connect to Stream notification. Each time I clicked the button, it would load the Plans and Pricing page. As it turns out, the reason for the endless loop is because I didn’t have a subscription registered with the Stream website. Once I completed the process of registering for a free account, the WordPress backend loaded the Stream records screen.

Successful Connection To Stream
Successful Connection To Stream

I recommend text be added to the top of the Plans and Pricing page. The text should explain that in order to complete the connection to Stream, a subscription plan needs to be selected. It’s not obvious and gave me the impression the plugin is broken.

Support For SMS Notifications Thanks to an Outside Source

One of the neat features in 2.0 is the ability to set up SMS notifications. For instance, every time a theme, plugin, or WordPress is updated, you can configure Stream to send you a text message.

Configuring SMS Notifications In Stream
Configuring SMS Notifications In Stream

SMS notifications ended up in 2.0 thanks to the contributing efforts of Jeff Matson. Matson is the author of the WP SMS Notifications plugin we highlighted on the Tavern back in July. Matson explains why he decided to contribute to the Stream project, “When I created WP SMS Notifications, the biggest comment I received was that I should work with Stream to add my functionality to their plugin. The team behind Stream agreed and I was given access to their Github account. Now, I can proudly say that my code is behind one of the greatest activity tracking plugins out there.” However, the only way to take advantage of SMS notifications is to use the Pro account which is available for $2 per month.

Older Version of Stream Will Remain Available For Download

Stream has undergone major changes and is now a service versus a stand alone plugin. For those who don’t want to update to the new version, the Stream Team is leaving the previous version online via Github. Versions 1.4.9 and below won’t receive any more updates outside of patching major bugs or security vulnerabilities.

Overall, a Solid Update

Stream 2.0 is a solid update. The latest edition supports activity tracking for eight of the most popular WordPress plugins out-of-the box including: Advanced Custom Fields, bbPress, BuddyPress, Easy Digital Downloads, Gravity Forms, Jetpack, WooCommerce and WordPress SEO by Yoast. SMS notification is a great enhancement and I think it’s respectable of the team to keep 1.4.9 available for those that don’t like the new direction Stream is heading in.

Are you satisfied with the latest update to Stream? Does using WordPress.com and Amazon Web Services turn you off from using it?

Category:
Tags: , , ,
38

38 responses to “Stream Morphs From a Plugin Into a Service”

  1. Thanks for the write-up about the new direction for Stream, Jeffro.

    Also I really appreciate your feedback in regards to the user experience when signing up. We will make some tweaks to that right away and hopefully nail something that is easier for users to understand what is going on.

    • Thanks. I felt like an idiot when trying to connect Stream to my WordPress.com account but I always remind myself that I can’t possibly be the only one that will go through an experience like that. I think some text or something would go a long way.

      • Just to follow up with your suggestions from today, Jeffro. Here is now how the plan page looks when you are connecting a site. http://note.io/1rEGxlX Thanks again for the great suggestion.

  2. It’s late. Another long day. Perhaps I’m beyond my best thinking moments? In that context…

    What’s so top secret (?) about high level activity meta data? I understand what off-loading to AWS does. What I’m not quiet seeing is what that buys. What’s the value add? Or is it just part of a new biz model to product revenue? Nothing wrong with making a buck. Kudos! Just askin’.

    Is there an actual security gain? For example, does the activity logging produce a signal that says “you’ve been hacked” and then that can trigger an SMS?

    Help me out. I’ve honestly never used Stream. I feels interesting. But perhaps without that context (and/or examples / benefits) in the intro I was doomed to be lost from the start? :)

  3. While the information above seems all correct, this post is still missing something. It looks like Jeff did not use the plugin before and therefore might miss some “feeling” of the drastic changes – from a user point of view.

    The plugin before 2.0 was a genius “product”, awesome helper tool! It still is, but the picture now is a bit bigger and more aspects go into it.

    As a hardcore user (of this plugin) I was testing a beta version before release via their (public) GitHub repo, so I was aware of these changes, however, total shocked about those. I wrote a post in their support forum on their product site (now no longer there, forum is removed it seems) and expressed my concerns. This was answered from their stuff, but I couldn’t manage to reply again, as with the 2.0 release their former forum was removed completely and seems no longer accessable.

    Some of my concerns are:

    1) No communication to existing users that v2.0 will almost change everything about the “handling” of the plugin (but not the core functionality itself)

    2) Pricing for Multisite is very very high! It is priced per SUB SITE of a Multisite install! So for example I maintain 4 Multisites currently, with about 10 sub sites each. That would mean about 200 US-Dollars A MONTH (not a year, per month!) for filling “some” log files…! To high for me at the moment, really! — To be fair: as an existing customer I got a bonus until the end of 2015, which is very generous, however, after 2015 I have to evaluate again! I just gave this use case example here, as I assume a lot of us freelancers are managing stuff for clients in similar way. (To be precise: those 4 Multisite are from 4 different clients and their sub sites are project-dependend similar sub sites.)

    3) It is currently not communicated for what the connection to WordPress.com really is: users are totally confused, as they think the data itself is stored on WordPress.com (which is not), but it seems so to end users. This is clear fail of the current copy etc. Also, for a lot of clients it’s not an option to have a connection to WordPress.com, for whatever reason (and there are many reasons), and even if it is for the login handling. For enterprise & business this is sensible terrain, offering no alternatives I consider a “fail”.

    4) Users have no choice: no choice of server location, which should be a bare minimum, at least in the Amazon cloud ecosystem. They have locations in Europe (Ireland) and so on. Such a choice is really important. However, it would be way better if users had a lot more choice. For example I would chose Iceland as my server location because this country is not a member of EU and not affiliated with the U.S. so it has “a bit” more security from an “NSA point of view”! Also, users have no choice for type of storage – cloud or local. Lots of users are upset about this.

    Otherwise, and I only repeat myself: the core plugin is pure genius, and the developers of it are true masters! I only criticise the business aspect of the whole thing and the handling & communication of their upgrade/ update strategy.

    If you look at the reviews they “gained” only in a few hours after the 2.0 release, you’ll get a feeling that I am not alone with my concerns and that other users express it way more “harsh”…

    Technically this plugin is sure 5-star thing, and it almost had this rating like all of its current lifetime before 2.0. This drastic change changed everything.

    I hope other developers and companies get a feeling how users see these things and act more sensible and TRY EVERYTHING to do all that together with their existing users, not against them.

    My hope really is, this will lead to a learning experience for all involved. And yes, I also develop and offer (free) plugins. So I am aware of both sides a bit.

    • To add to my previous comment: I just decided to give them a 5-star review at .org – which I gave for the plugin in itself, for the technical aspects. All this business etc. stuff should go in those reviews there, at least not as a priority. The plugin works, it does all what it says and the code is very good.

      I just wanted to give this message: it’s not all black and white. The “issues” I laid out above are still not “resolved” for me. However, I know it surely is not easy for the team/ company behind Stream at the moment.

      Sometimes it seems easy to “complain” as a user, and it is, yeah, but it’s a bigger picture in all aspects.

  4. I am confused by this upgrade. The previous plugin worked really well and did not require connection to cloud services. I’m not seeing any benefits to end-users here, just negatives.

    • I have nothing to do with Stream but I can say that there are most definitely benefits to what they have done that can directly impact the existing users in a positive way.

      The product as a standalone plugin is extremely limited by what is possible in a standalone environment utilizing MySQL in what is on average going to be a bare bones shared hosting environment. Not exactly robust.

      When dealing with a data intensive product such as Stream that is similar to work with web site analytics you are going to be handcuffed by the limitations that come with running such a data intensive plugin on a limited server environment that is completely out of your control.

      By leveraging more advanced cloud technology for the data store they can do much more with the data and do so much quicker than they ever would be able to on your average shared hosting based WordPress site.

      We run into similar issues with Gravity Forms. There are things that we would love to do that we simply can’t do right now because of the fact we have no control over the hosting or database environment in which the code is executed. It’s most definitely a limitation that can be overcome by leveraging cloud based services.

      • This is a good point, as there are indeed performance advantages in off-loading this stuff outside of WordPress. But you could gain the same advantages via self-hosted but external (from WordPress) software. In my experience, businesses don’t like to send sensitive information like this to third-parties.

        • I would have to disagree as far as your statement regarding businesses and storing sensitive information with third-party services.

          Steam is effectively a specialized analytics solution. Companies have absolutely no issue leveraging Google Analytics and the many, many hosted analytics solutions that are out there. Stream is really no different.

          The fact that Google Analytics, GoSquared, Kiss Metrics, or the countless other analytics solutions are hosted services hasn’t hurt their adoption in the least.

          In fact, most businesses would rather put their trust in a hosted service for analytics than host and manage their own analytics server and solution. There are of course always exceptions and for those exceptions there are solutions such as Piwik that you can host yourself. But you need to be prepared to to scale and more importantly know what you need to do in order to scale when it happens. That’s not something most businesses are prepared for.

          You host your web site with a web host, a third party. You’re already trusting your data with this web host. In many cases doing so simply because you read a good review online about this host and nothing more. And in this case it’s not JUST analytics data, it’s everything that encompasses your site as a whole. (I use “YOU” here generically, not specifically meaning you individually).

          Smart businesses know when to do things themselves and when to leverage 3rd parties. I wouldn’t call the vast majority of self-hosted WordPress installs as being managed by users who truly know what they are doing. Businesses that require this type of data analytics should want to leverage a service that can provide a stable, secure, scalable solution.

          Scaling is not as easy as it sounds. ESPECIALLY when it comes to data and data analytics. It’s why there is an entire industry specifically focused on data analytics, data mining, etc.

          Where the team at Stream wants to take the product, AND the functionality it’s users want to see simply isn’t the type of thing that is ideal to power using your average shared web host and MySQL.

          High traffic WordPress web sites already require optimized hosting environments, caching, etc. Which works great. Those same tactics don’t work with a data analytics application. The more traffic a WordPress site gets, the more Dashboard interaction, the more plugins, the more posts… all of this increases the volume of data related to this type of analytics.

          When you get into things like trigger and alerts based on activity like Stream does your datastore needs to be blazing fast and realtime. That means caching traditionally leveraged by WordPress site owners is thrown out the window because it’s not applicable to this type of use.

          These types of situations are precisely why solutions such as Amazon’s AWS, Microsoft’s Azure, Google’s Cloud, Keen.io, Iron.io and countless other cloud solutions have gained massive popularity.

          I haven’t even touched cloud services like SendGrid, Mailgun, Mandrill, Amazon’s SES, Postmark and many others like them that are used every single day by people to send sensitive data: email. Not only that but many web hosts already leverage these services and are already sending your email through a cloud service, even if you don’t know it’s happening.

          Just because you can host something yourself doesn’t mean you should and data analytics functionality is a great example of something you shouldn’t be hosting on a typical WordPress hosting account.

          • Carl,

            You, like Frankie at Stream, talk a lot about the technicalities. In doing so, you (and they) are largely missing the point.

            You (and they) are not hobby coders now. This is business. So where’s the due diligence?

            Ryan is right that businesses are reluctant to send personal data off-site. And with good reason.

            What precautions are in place for those with sites within the EU to prevent the illegal export of data outside the EU?

            Where’s the HIPAA compliance confirmation? Or FINRA, or …?

            You get the picture. (At least, I hope you do.) The trouble is, if any regulator or compliance auditor now decides to take a look, it’s almost certainly too late for Stream.

          • I’d bet the Steam team might make an update to let you select data center sometime down the road. As for HIPPA and FIRNA, neither of those apply here. If you’re concerned with your data, there’s nothing stopping you from using 1.4.9. It’s available on GitHub.

          • See Chris Christoff’s reply above.

            HIPPA and FINRA aren’t exactly things that most WordPress sites have any need for. There is definitely a need for HIPPA and FINRA compliant products, but those are a niche markets that call for more niche specific solutions.

            Frankly you are unlikely to find a solution that is indeed HIPPA or FINRA compliant that ISN’T a hosted SaaS solution like Stream because by their very nature they require things that a hosted plugin is not going to be able to bring to the table.

            If enough Stream users voice a concern regarding the location of their data not being within the EU then I am sure that the team at Stream would explore allowing users to select the data center location as Chris mentioned above. That’s the beauty of building products on cloud solutions like Amazon AWS and others. You can introduce the ability for users to do things like select their data center location, etc.

            You are correct. I and the team at Stream are not hobby coders. I can’t speak for the guys at Stream (although I know they originate from the X-TEAM which does enterprise work) but I come from an enterprise background. I’ve never been a WordPress hobby coder. I got involved in WordPress development after already having an extensive background building enterprise solutions.

            Don’t assume that because you have qualms about using a hosted solution that the vast majority of business also have qualms about doing so. It’s simply not true. If it were true then the SaaS industry would not be as big as it has become. Solutions like Amazon AWS, Google Cloud, etc. would not have grown like they have.

            Your screaming regulation, compliance and audits but haven’t taken a minute to actually think about what Stream does and if regulation, compliance and audits are going to be an issue. For the vast majority of businesses using WordPress this is going to be a non-issue because Stream simply doesn’t provide functionality that falls under regulation such as HIPPA, FINRA, etc.

            Those are definitely important topics. But only in the context of a SaaS solution that provides caters to users who need HIPPA and FINRA compliance. That isn’t Stream.

            Do you use Google Analytics? Or any analytics service? There really isn’t much difference than them and Stream. One is logging actions by site visitors and the other is logging actions by people logged in. The end result is still a form of site analytics.

            You need to trust in the services and solutions that you use. If you trusted the Stream team enough to run their plugin on your WordPress site then you should have enough trust in them to allow their SaaS to power Stream. Either way you have to trust the developer of the plugins you use on your site because at the end of the day their plugin code can impact your web site.

            If you don’t trust them, don’t use them. But that certainly doesn’t mean they won’t find a market with both existing users and new users.

          • So, Carl and Chris, what you are saying, essentially, is that Stream’s legal due diligence consists of this plea:

            “Yes, we probably have breached laws and regulations in the US and EU, and in many other jurisdictions besides, but we have only done so in connection with a minority of the sites whose data we store. What’s a little illegality among friends?”

            I wish them the best of luck!

    • Ryan,

      I seem to remember that you are based in the EU. Any site based there won’t legally be able to use Stream 2.0 anyway because of the data protection laws.

      • Yep, I’m based in Germany. They have very strict laws on this here, but I think other parts of Europe have very different laws, so perhaps this doesn’t apply across the whole EU? I have no idea myself (I’m based here, but am not from here).

        • Yes, I remember you’re a Kiwi. But I can assure you that those laws apply throughout the EU.

          Another thing I don’t understand about the Stream decision is that they have given themselves no way of avoiding accepting data from the EU. The point is that failing to adhere to the EU regulations makes not just the site owner liable, but also the business that is illegally storing the data.

        • There are still some differences (don’t ask, I don’t know them in detail), BUT: the EU government (commission) is about to unify all this in the future.

          So if you take our German laws (I am from here), you have good idea of where all this is heading to in the not so distant future.

          They’ve already unified some restrictions for shop owners which took effect on June 13rd this year, as of January 1st 2015 similar unifying in the area of VAT for digital product shopping within EU.

          Only a few examples.

          The whole area of (digital) privacy laws is expected to be “unifyed” as next.

          I do not welcome a lot of these laws, however, sometimes they could be an advantage sometimes for a user to really help protect user rights. Sadly, often they only increase time and cost for administration which is bad of course – and decreases power of startups and entrepeneurs for example.

          It’s a wide field. My hope is that companies like behind Stream plugin will take advantage of these realities and offer services that fit better to businesses and companies in other parts of the world.

          • David,

            Some of these laws are already unified throughout the EU. The one I am talking about forbids the export of personal data outside the EU.

            It’s been in place for quite a while now, and caused EU governments quite a few problems when they were trying to find a way to give the US government details of airplane passengers.

          • @KTS915:
            Thanks for clarifying! Absolutely, there’s a lot of stuff going on in that whole field!

            Storage of data and the precise locations becoming more and more important for business users/ clients. For quite a year now I get asked by (potential) clients about such things – it rarely got asked the years before. People are much more sensitive about their data these days. And this IS a very good thing!

            Yeah, and I still hope our government(s) would be way more sensitive with our data!

      • I forgot to mention that the folks at Interconnect IT have a similar plugin called the Auditor … https://interconnectit.com/products/the-auditor/

        I don’t have much experience with either of these plugins, but the old Stream plugin seemed very similar to the Auditor plugin during my brief experiences. So anyone who is concerned about these changes might want to check out that one.

        The Auditor is a little pricey, but in my experience the sort of sites who use this functionality usually have fairly large budgets anyway, so the price doesn’t tend to matter much anyway.

  5. I have commented before in the Tavern about how some developers seem to have no idea how to promote themselves. This, unfortunately, is a classic example. And the damage to Frankie’s reputation won’t be undone by an interview in the Tavern. (Tip to developers: you do your PR stuff before launching something new, not to defend yourself afterwards.)

    As David Decker says, the plugin before the upgrade was a work of genius, with a lot of admiring users. Would many of us have paid for that? Absolutely! Were we asked? No.

    What’s the real reason for the switch? It isn’t for enhanced security, obviously, because one centralized database is way more attractive a target to hack than any of the individual sites using it. (It could hardly be more ironic that they launch this just after the iThemes hack.)

    This change can only be to monetize the product. Which is fine. But now it’s a product many users don’t want, can’t have, or can’t afford.

    And, to cap it all, there was no explanation or information beforehand!

    This is about as crazy a way to conduct a business as you could get.

    • @KTS915 – I think saying that there is reputation damage going on is a bit extreme. I love this plugin, will I pay for it, probably not, but I can do what it does in v1 with a bit of programming or use something else. This isn’t a marketing or PR failure. It’s simply a business shift. Also, where have you seen that it is one database stored in the cloud? That hardly makes sense and would force them into similar constraints as keeping it local. Much more likely that the data is being stored in separate instances or at least multiple instances. They had already monetized the product. This is simply growing and shifting the business focus to grow more and increase longevity. While their will be growing pains, give the company and its leaders some credit. If you don’t like it, vote with your wallet (or lack there of).

  6. @mrpritchett – Have you seen the comments on wordpress.org? There clearly is reputation damage. It clearly is a PR failure.

    When you make a “business shift,” there are good ways to go about it (which minimize any criticism and promote the advantages) and bad ways (which defensively react to criticism that’s already been loudly voiced).

    Stream went the second route. As I said, that’s crazy.

  7. I agree with Ryan here and many others. Many companies, especially those who take security seriously do not like to have their own “security audit logs” hosted somewhere else, not to mention the legal implications there are in the EU.

    Apart from that, yes such plugins do take up some resources from the server because they are constantly writing to the database though again this should not be an issue. Most WordPress sites have caching hence database is not so busy, and the database is definitely not the bottleneck for WordPress. Therefore such plugins are able to run on big websites without any problem, even when storing data on the same WordPress database if written properly. The excuse of “MySQL” database was not enough in terms of resources is not so true.

    I’ve seen some implementations of our own auditing plugin on WordPress multisite which has 160 sites running and we never noticed any performance degradation. So if you develop the plugin correctly, this should not be of a big issue.

    • Caching won’t really help with MySQL usage. I think the thing which saves these type of plugins from killing the server is simply that there aren’t enough users navigating the backend to cause any major problems. If you had thousands of users constantly using the admin panel, then you would definitely want to offload that elsewhere. However, a handful of editors sporadically updating posts aren’t likely to make a big impact.

      • HI Ryan,

        Caching does help because if most of the content is cached there is not much reading going on from the DB, thus such resources can then be used from such auditing plugins. I don’t have the exact numbers right now but I’ve seen implementations with hundreds of editors working and there was no bottleneck generated by the plugin. Keep in mind that the majority of the time the editors are writing text and maybe adding some images to the posts, and such activity does not generate a lot of events in such auditing plugins.

        Having said that I am not excluding the fact that having an external source where to store the alerts is a good idea, actually I do believe in it. What I don’t like in this case is the way Stream executed the idea, which could have been much better. But then again, everyone has got his own opinion :)

          • Agreed and that was my idea :) I am not saying these plugins do not generate load, but I am saying if websites have caching enabled then there are more free resources available for these plugins, hence the chances of affecting the performance of a site is minimized.

    • Writing log files is easy. Displaying and manipulating that data in realtime when the datastore grows exponentially over time is most definitely NOT.

      You cannot compare caching and displaying WordPress content with providing more advanced analytics and data logging that makes use of realtime data manipulation, reporting, and actions/triggers.

      They simply aren’t the same thing and anyone who thinks MySQL and WordPress is the ideal platform to more complex applications involving extremely large amounts of data hasn’t tackled this type of thing before.

      The developers at Stream obviously plan on taking things in a direction that simply running it locally on a $9/month BlueHost shared hosting account is not going to be able to handle and I can tell you from experience that it doesn’t take very much to get to the point where your typical shared hosting account is going to run into major performance issues.

  8. One last thing I forgot to mention that yes, from the security point of view it does make sense to have such setup but there are many different ways how this can be tackled, rather than having all your data stored at your developer.

You may also like

Newsletter

Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

%d bloggers like this: