A/B Split Testing Reference Guide

Australian eCommerce Guide Reference Guide

On the web, A/B (or Split) Testing is the name given to the process of displaying two versions of a webpage to visitors to the page, and then analysing the results of the actions the visitors took to see which version of the page worked better. (Version A or Version B).  Which version resulted in more conversions?

Typically, you only have one single difference between the two versions so you can make sure that different conversion rates are as a result of the single difference you are testing.

Also, you will typically display Version A to 50% of  your visitors and display Version B to the other 50% of visitors. Ideally, the pages are displayed at random, and with a big enough sample of visitors you can make a decision which page is working better.

After the test, you keep the better performing page and show it to all visitors. Then you find something else to test and do your next round of Testing.

Split Testing can be extended to become what is known as Multi-Variate Testing.  With Multi-Variate Testing, you test multiple variations. Typically, you run multiple different A/B tests in the one testing operation on the one page at the same time.

There is a good explanation here on Optimizely’s website about the difference between A/B testing and Multi-Variate testing.

Testing needs to be easy and low cost to do, otherwise businesses seldom do it.  The successful eCommerce businesses do it frequently, and some are testing something ALL of the time and you may never know it.

Some  of the better eCommerce shopping systems now  come with easy capability to do this A/B split testing. If you have a choice between systems, and one system allows you to easily do split testing and the other does not, then this issue can become an important consideration.

It can be very expensive to get your web development firm to custom-build simple A/B testing functionality into your website just for you.

Alternatively you can use a third party online testing service to help manage the testing process.  You will find links to Split Testing tools here in the Australian eCommerce Guide.



Here is a handy guide to A / B Testing from Unbounce. It also includes a useful glossary and explanation of terms.  

Download the A/B Split Testing Reference Guide. (PDF)

June 27, 2014

Comments Please Note: Comments are reviewed and may be edited before publication.

Leave a Reply

5 + 4 =