Mitcho who is the author of the Yet Another Related Posts Plugin is currently working on a new project called ShrimpTest, sponsored by Automattic. ShrimpTest is a simple, A/B testing plugin that enables users to create various experiments on their site. For example, with ShrimpTest, you can create an experiment to track conversions based on the text used. In the video demo, Mitcho runs an experiment between a large version of a PayPal donation button versus a smaller one to see which provides a better conversion rate. Out of about 3,000 people, the results were almost split with a slightly higher percentage converting through the larger image. I encourage you to watch the following video to get a better idea as to what ShrimpTest is all about.
You can read all of Mitcho’s status updates regarding the plugin via the development blog. Also, if you’re interested in testing out ShrimpTest, Mitcho has recently packaged up version 0.1 that is available for download here. Please reports bugs or other issues within the ShrimpTest trac area.
This plugin is really cool and I suspect that it will be heavily used for UI testing within WordPress. I can also see marketers drooling at the mouth for something like this.
There are A/B open source projects out there why not just go with one of those instead?
Why reinvent the wheel?
Also ppl have been splittesting using for example Google Websiteoptimizer for like since it was made public. I know I have. Even written a plugin that integrates directly to the TinyMCE editor. Couldnt get creation of tests from inside of WordPress to work last time I tried ages ago. Could probably make it work now.