WP-ABTesting – Split Testing As A Service

There is a new up and coming AB Split testing service for WordPress called WP-ABTesting that aims to help you optimize websites based on data, not opinions. Anytime I’ve heard of anyone wanting an A/B testing plugin, I’ve referred them to ShrimpTest by Mitcho and Automattic however, it appears that development on that plugin has ceased as it hasn’t been updated in over 2 years. There is also a plugin mentioned by PostStat.us called AB Press Optimizer. WP-ABTesting took some time out to answer a few questions I had about their service.

WP AB Testing Logo

Tell us a bit about the WP AB Testing service.

We provide a native conversion optimization service for WordPress sites. Our aim is to increase your conversion rates (whatever your conversion goal is: more sign-ups, sales, visitors,…) in a systematic and scientific way. Opinions are great but only if they are used to start a data-driven experiment to (dis)prove them before deciding on whether to implement them or not. Nobody (not even you) is good enough to predict what will work for your site.

At the core of our service you have an A/B testing (also known as split testing) mechanism. In its simplest form, A/B testing works by predefining two or more alternatives for a given post or page and the conversion goal you want to achieve with them, and then randomly showing those alternatives to your visitors to check which one works better, i.e. which one has a higher conversion rate. For instance, you may want to create two alternative versions of your landing page, show 50% of the time each alternative and monitor which one makes more of your visitors to sign-up for your mailing list. Once the results are statistically significant (don’t be scared, we will hide all the math from you, just keep reading!) you consolidate the winning alternative and start again with a new test.

What will WP AB Testing have that may not be found in plugins or other similar services?

Most tools focus on the classic AB testing: you define the tests and the tools report on the number of conversions of each alternative. We believe this is just the tip of the iceberg.

Our service goes beyond this in two main directions:

  • Technical level. We don’t want to restrict people to test only posts / pages with a standard AB Testing algorithm. Since our service is a native WordPress solution we can offer fine-grained tests in which other WordPress components (like menus, widgets or the theme itself) can be tested as well. Then, the plug-in automatically evolves the site based on the results of those tests (obviously, only if you want the plug-in to do that for you). We’ll also complement A/B testing with other testing / optimization strategies like greedy algorithms and heat maps to offer a better picture of what’s going on in the site. Another example we have in our future roadmap is to enable the use of social metrics as conversion goals (i.e. an alternative post is better if it gets reshared more times) which we believe can be interesting for some sites.
  • Personalization level. Raw data on the percentage of improvement of an alternative may be enough for math savvy users but won’t do much for the rest. We believe in A/B testing for everybody which means that our service will help people to understand the meaning of the numbers they see to make an informed decision. And, more importantly, for those that want a more hassle-free experience, we will offer a personalized service where one of our company experts will suggest possible tests for your site (e.g. based on ours and the community experience on what has worked in other sites in the same domain) and even create and monitor the tests for you.

Why is AB Testing an important practice to conduct on websites from time to time?

First, I’d like to “challenge” the fact that AB Testing should be done “from time to time”. In our opinion, you should always be optimizing your site. Even if some optimizations may only bring a small improvement, they add up so the overall effect may well surprise you!.

A second reason, which is not usually mentioned, is that you are going to learn a lot about your site and your business when doing A/B tests. Many people regard A/B tests as mere small changes on the look and feel of your site (e.g. changing the color of a button here, increasing the size of an image there,…) but they can be much more powerful than that. I regard them as a learning tool for the business owner. Just thinking about possible tests gives you a perfect excuse to reconsider your business assumptions and offers a low-risk approach to test alternative approaches that otherwise you would have never dared to try. And if the test fails (meaning that your current version doesn’t need to be changed) you should even be happier. Failed tests are as useful as successful ones. They allow you to get rid of possible scenarios that now you know for sure wouldn’t work.

WP ABTesting Service

Why the decision to turn WP AB Testing into a dedicated service instead of a fully featured plugin?

We try to combine the best of both worlds: a WP plug-in on the “client” side that the user can install to define the tests, see the graphics with the results, consolidate the winning alternative,… all without leaving the WordPress environment (meaning there is no need to learn a new tool) plus an external back-end that takes care of recording and processing all the relevant data to evaluate the performance of the alternatives tested.

We believe this makes a lot of sense, again both at the technical and business level. On the technical side we lift all the processing load from your own server so that the tests do not slow down your site or interfere with your hosting provider limits. On the business side, because an ongoing relationship is needed to be able to advise you on your best testing strategy (by checking what has worked and what hasn’t worked so far for you), on new tests that we believe could be specially useful for you, etc.

Don’t forget that A/B Testing is only the means to an end (optimize your conversions) and not the end itself. Our mission is to make sure you bring your site to the next level, not just to provide you with a tool to do all the work on your own.

How can people be notified and participate in the beta?

Just go to our website and join our mailing list in less than 30 seconds. We’ll immediately contact you to get to know you better (what are your needs, expectations, etc) and give you more details about the beta program.

18 Comments


  1. FYI, got this when trying to access the link:

    Google Chrome has blocked access to this page on wp-abtesting.com.
    Content from demos.shapingrain.com, a known malware distributor, has been inserted into this web page. Visiting this page now is very likely to infect your computer with malware.

    Report


  2. max ab from max foundry is a good one also

    Report


  3. @chris mccoy

    This one hasn’t been updated in almost 2 years and showing compatibility only up to WP ver 3.2.1.

    This would probably put it in the ‘dead plugin’ category (i.e support has ENDED) and I would therefore not feel comfortable using it on my site.

    Any other suggestions?

    Report


  4. Ok, I too finally received the malware message in Google Chrome. That’s a shame because the AB Testing site is not bad. I’ve put the domain through a couple of scanners and they all report the site is clean.

    http://sitecheck.sucuri.net/results/wp-abtesting.com

    I suppose the domain has a bad history previous to the two months they have owned the domain. Any of you folks know how they could remove that message?

    Report


  5. Really weird (and really unfortunate that this is happening right now :-( ).

    Google webmaster tools says that the site is clean. We´ll investigate why some of you are getting the message.

    Thanks for your patience!

    Report


  6. @Jordi Cabot – Yep this sort of thing sucks but thankfully, you are in Beta :) Hope Google can quickly get this mess cleaned up.

    Report


  7. @Michael Ginsburg
    Hi Michael, really sorry about this. We have finally found out why Google flagged us as malware. Full details here: http://wp-abtesting.com/worst-timing-google-flagged-our-site-as-malware-on-our-prelaunch-announcement/
    (in case you still see the malware message, the short story is that yesterday Google blacklisted by mistake the theme shop where we bought our theme and flag as malware all sites that had a link somewhere to that web)
    So, worst timing ever but I hope you still decide to evaluate our service, you can join the beta in our site or just send us an email to info@wp-abtesting.com and we’ll set you up

    Report


  8. @Jonathan Dingman -Hi Jonathan, please see our replies above. Hopefully, we can get now a fresh start with all of you! Let us know if you´re interested in joining the beta and we´ll set you up!

    Report


  9. @Jordi Cabot explains the snafu:

    … Google flagged us as malware. Full details here: http://wp-abtesting.com/worst-timing-google-flagged-our-site-as-malware-on-our-prelaunch-announcement/
    (in case you still see the malware message, the short story is that yesterday Google blacklisted by mistake the theme shop where we bought our theme and flag as malware all sites that had a link somewhere to that web) – [emph. added]

    My condolences to Mr. Cabot and the WP-ABTesting project … and to our host Jeffro and his WPTavern website. The unintended story that this has become, does appear to be mostly/largely/probably just ‘unfortunate’ … but with some meaningful caveats.

    I read the “Full details” post on WP-ABTesting, and they do deserve credit for responding/reacting to the Google identification of a security issue. The response does fall short in 2 areas, though.

    Secondly, it implies that Google is, ‘[Sigh], you know, such a mindless clod, negligently wreaking other kids projects’. That’s a tad rich. Google run tech/security-circles around everybody here.

    Firstly, what actually is going on at/with Theme Forest, “the theme shop where we bought our theme“? And, the “Full details” post offers a link explicitly to this, quote “WordPress theme we bought“, end quote, which is in fact not a link to the theme they acquired, but simply dumps us at the Theme Forest front door …. where we see it loudly proclaimed that they sell, quote, “10,785 Site Templates and Themes from $3”, end quote.

    It is obviously of interest, which specific theme (out of ten thousand offerings) was actually involved … whether some think that was important, or not. That information would be a key part of the “Full details” … and has been made, really, ‘conspicuous by its absence’.
    =====

    Is there something icky/dubious going on at Theme Forest? This is a well-known provider of themes & etc, for the wider WordPress community. Are those folks actually a slippery or fast-and-loose outfit (or too accompanying of such), and we should be saying so? Are Google really security-lamers, or is that a red herring to deflect attention from ‘back-alley’ behavior within the WordPress commercial sector?

    I get no impression that WP-ABTesting were active ‘bad actors’ here, in terms of the reported security-issue … but they do seem to be giving this affair that ol’-time ‘Move along now – nothing to see here’, treatment.
    =====

    … (don’t be scared, we will hide all the math from you, just keep reading!) …

    Unfortunately, my site is on my localhost, and I can’t test WP-ABTesting. If I was online, though, I would at least be checking out the terms of their offer closely. (I won’t ‘sign on’, under every & any set of terms that we see offered/required, even for things that are of interest – and WP-ABTesting is of interest.)

    But as far as hiding the math (and other tech-details) … please, yes, do abuse me! ;)

    Report


  10. @Ted Clayton – I don’t see any caveats here. They wanted to promote their new service, yes, but considering being blacklisted as a marketing strategy is too much for me. I’ve also launched online beta/demo programs and the last think you want is your web site down.

    About Google, the methodology of ‘shoot first, then ask’ that is used when putting a website in a blacklist is, in my opinion, too severe. I’m sure they have the tools and technology to check false positives more accurately, specially when, as it seems in this case, the site wp-abtesting.com was safe according to Google Webmaster Tools.

    About Theme Forest, Michael Ginsburg in a previous comment also mentioned that the problem he found was with ShapingRain, which is a theme provider that sells in Theme Forest. As far as I know, they only sell the Just Landed theme for WordPress. Following the comments (and also the content) in WP-AB Testing explanatory blog post you can also see that explicitly mentioned. It wasn’t hard to find for me.

    I don’t see the point about second interpretations of what here happened. Maybe I’m too naive but, you know, I was never a follower of conspiracy theories. To me, Neil Armstrong walked on the moon, and Mr. Cabot’s web just went blacklisted by mistake in a promotion day. It seems to be solved now. End of story.

    Report


  11. Just in case people wanted to follow along, the commentary on HackerNews about Google blacklisting sites in general is pretty good https://news.ycombinator.com/item?id=6242533 I’d hate to be the victim of such an event, especially if it damaged my companies reputation and lost me a ton of money. Why Google Webmaster tools doesn’t alert website administrators of the malware warning is beyond me.

    Report


  12. @Tom Gasville

    … it seems in this case, the site wp-abtesting.com was safe …

    From what we can tell/see, the WP-ABTesting site itself was not doing whatever is was that got Google’s attention … but they were linked-in or networked with others, who were & did.

    It’s worth noting, especially for the less-technical and for those who think they need networked services to run a web-business (or to be cool), that being linked-together can make multiple web-entities functionally a single site. When a problem arise with one member of such a network, no matter how inconsequential they might seem, what’s going on with them can affect (ie, eg, ‘take down’) the whole network. Being more independent may seem old-fashioned, but it’s the stronger & smarter security-posture.

    I don’t see the point about second interpretations … It seems to be solved now. End of story.

    On the contrary, Mr. Gasville, we have a Comment section as this, precisely & expressly to invite & explore “second interpretations”.

    Theme Forest, and parent company Envato, have been the topic of controversy within the WordPress community, for some while now.

    Report


  13. @Jeffro

    I did not have real high hopes for this page, but stand pleasantly corrected. ;)

    When I looked, the top comment showing (also on Jeffro’s link), begins:

    I’ve suffered from the same malware issue earlier. Since we use ad networks for advertising it works as follows. 1 ad network has about 3000 different ads running in different locations. If any of those domains get compromised and blacklisted and Google notices that you served something from that domain – BOOM![emph. added]

    This reinforces the remark I made to Tom Gasville, that having yourself ‘all networked up’ with gawd-knows who-all & what-all, means that when any one of your network partners does something squirrelly, shady, too-rule-bendy or downright nefarious … the whole dang bunch of you are taking the fall!.

    It’s not just “ads”. It’s not just “image-linking”. ANYTHING too far outa the envelop, on any of potentially huge numbers of sites, involving linkages that may be totally unknown to many of them, can become a problem for all of them.

    This is what appears to have happened to WP-ABTesting.

    Report

Comments are closed.