A/B Testing for Ecommerce: 5 Things You Need to Know
You’ve taken the plunge and finally entered the world as an Ecommerce merchant. With your spanking brand new website, your top-of-the-line web design and your premium web copy, you think that you’re in it to win it.
But let’s take a moment to set these aside first and focus on what really matters.
Think about the current conversion rate of your Ecommerce website visitors.
How many of them are converting to customers? And how many of them remain as visitors who simply look browse through your website and then leave without purchasing anything?
Now, let’s talk about the figure that you have. Is it okay, or do you believe that it can still be improved?
If you think you have the potential to increase your conversion rates by testing your marketing strategies and promotional campaigns and finding out which techniques were effective, then you need to learn about about A/B testing.
What is A/B testing?
A/B testing is a method that measures two versions (“A” version and “B” version) of a marketing component in order to find out which version is more effective:
- in increasing sales,
- converting more customers; or,
- encouraging your website visitors to do the action that you intended them to do.
How do you implement this to your Ecommerce website?
An example may be using A/B testing to determine which “Checkout” page is more effective to reduce your cart abandonment rate.
Your current checkout page, the control, will be known as version A.
On the other hand, your revised checkout page that you want to use, the variant, will be known as version B.
You will then equally divide the amount of traffic into two equal groups and expose them to version A or version B.
After your testing period, you can quantify your results by measuring which checkout page generated more conversions to customers, and subsequently, an increase in sales.
Still with us? Here’s a simple A/B test case study that you can read about.
In understanding A/B testing, just knowing its definition is not enough.
As an Ecommerce merchant who wants to delve more into the topic, here are five basic A/B testing-related concepts you need to know, presented in a beginner-friendly format, just for you:
1. Which is Better: A/B Testing or Multivariate Testing?
Multivariate testing measures the effectivity of different elements within a single page.
In A/B testing, you are testing a different version of a web page.
But in multivariate testing, you are testing the effects of varying the elements inside one web page.
Let’s take this as an example: if you’d like to test 5 different headers, 4 different headlines, 3 different client reviews or 9 different product descriptions, then you’re using multivariate testing because you’re keeping the same main page but testing specific elements on your page.
Which one should you choose?
As a widely used testing methodology, A/B testing is generally used because of its simplicity and ability to generate fast results as it’s easier to determine the more effective version, given that there are only 2 variations to be tested.
Don’t have a lot of web traffic? Use A/B testing also.
If you want to redesign the elements of your page according to which elements will have the most significant impact, use multivariate testing.
Keep in mind that for your testing to be effective, though, you need a highly trafficked site to get meaningful results because multiple variables will be tested.
You need both high-traffic website and high-traffic individual pages for multivariate testing to work.
Further readings:
- Comparing a Multivariate Test to an A/B Test
- The Critical Difference Between A/B and Multivariate Tests
2. Hypothesis Development in A/B testing
What is it?
A hypothesis is a proposed statement based on limited evidence that can be proved or disproved through testing. Your hypothesis is used as a starting point for further experimentation.
In A/B testing, your testing hypothesis invites you to look for potential solutions. In this case, your hypothesis is the reference that you base your optimized variant on.
In developing your own hypothesis, it should contain:
- A proposed solution; and,
- The projected effects that the solution will bring about.
Here’s a sample template for your hypothesis:
“If __[Variable] , then __[Result] [Rationale].”
Your variable is your website element such as sign-up button, testimonials, or checkout page, among others.
The result is the predicted outcome of your experimentation, such as increased conversion or more clicks on your button.
The rationale is the “why?”
Your rationale explains why you think your hypothesis will be proven to be valid based on the qualitative and quantitative behavior of your traffic.
Further readings:
- State Your Hypothesis: A Scientific Approach to A/B Testing
- How to Formulate A Smart A/B Test Hypothesis (and Why They’re Crucial)
3. Statistical Significance
What you need to know
Statistical significance is finding out if the difference between two results is relevant or just a simple fluke.
You use statistical significance in order to find out if version A is actually better than version B – assuming, of course, that the sample size (traffic) is large enough.
Your statistical significance is your “confidence level” which is commonly set as 95%.
This means that given a valid testing model, there is only a 5% chance of detecting the difference in effectiveness between the two versions.
A statistical significance of 95% means that there’s only a 5% chance that your A/B test results are a fluke.
If your A/B test results are statistically significant, you need to take action and then test again. If it’s not, you also need to take action and test another hypothesis.
Further readings:
- Statistical significance & other A/B pitfalls
- Why Reaching and Protecting Statistical Significance is So Important in A/B Tests
4. A/B Testing Your Website Pages
In using A/B testing for your Ecommerce website, the choice of website page to be used in the testing process depends entirely on your end-goal.
Which pages should you use in your A/B tests?
If you want to increase your conversion rates, then your checkout page may be the priority page for your testing.
If your goal is to improve your email signups for your newsletter, then your subscription page may be your best bet.
A/B testing is not limited to website content – you can also use it to determine which email template is more effective in encouraging clicks to your Ecommerce store.
If your focus is on lead generation, then your landing page may be your subject for your A/B testing.
On the other hand, if you want to find out which design can increase the number of items purchased from your website, then it’s best to test your product page for this specific desired outcome.
Whichever page you’re going to use in your A/B tests, though, you need to keep in mind that consistency is the name of the game.
Additionally, always test both versions (version A and version B) at the same time. You do this by splitting the traffic between the two versions. Be sure to expose the same visitors to the same variations.
Further readings:
5. A/B Testing Your Marketing Elements
Which elements should you test?
If you are using A/B testing to improve your conversion rates, here are several elements that you can test and subsequently improve on:
Call-to-action buttons: Their color, content, font face, font size and location in the website can spell the difference between a sign up and an exit button.
Take it from CareLogger who increased their conversion rate by 34% just by using red instead of green!
Navigation bar/Menu: You can revise the order of which your navigation bar is presented; or you can also invest your resources in changing the content of each navigation item to determine which version receives more clicks from your visitors.
Formstack found out that renaming “Why Use Us” to “How It Works” increased page views by nearly 50%.
Images/Graphics used: What your image is, where it is placed, how many they are and what their sizes are may be significant in determining how effective your image is in increasing conversions.
For example, Highrise tested their website image and found out that Michael was more effective, based on his performance metric.
Headlines: Drafting eye-catching headlines isn’t enough – you also need to measure exactly how attention-grabbing a headline is.
Source Patrol discovered that by adding a straight-forward word, “supplement”, improved sales by 89%.
Conclusion
It’s important to consistently use A/B testing whenever you change the layout, design or content of any of your website’s pages.
A/B testing ensures that you’re constantly experimenting and improving your elements in order to find out what works best for you.
Have you already tried A/B testing for yourEcommerce website? If so, how did it go?
Leave your stories in the comments below. We love talking about improvement and experimentation!