The science to growth. How to A/B test for improved user flow and conversion.

A/B Testing Example

When it comes to digital marketing, website design or anything else digital, it’s important that you do A/B testing. A/B testing makes up a critical part of the refinement process and it can be used to continuously improve your results. Without it, you won’t be able to improve and increase your ROI.

So first, let’s talk about what A/B testing is and why it’s important.

What is A/B Testing?

The process of simultaneously running two, sometimes multiple (multivariate), versions of content or design to determine which option yields better results.

A/B Testing is important because even subtle adjustments such as a different font color or different wording can actually make a big difference in conversion rates and overall performance. Therefore, by doing A/B testing, you can confirm that the small changes you make are actually helping you to improve your marketing results.

There are three main types of A/B testing that you can apply to your website:

Element Level Testing

CTA wording variation

When you only alter one element on a page such as variations in:

  • Call to Action (CTA): differences in wording, placement, color, size
  • Content: differences in length of content, formatting, wording, type of content
  • Price: differences in the way prices of products are displayed (e.g. $10 vs. $10.00 or $10 vs. $9.99)

This level of testing is the simplest and easiest to carry out because the variations can be easily changed in your testing platform. It’s also your best bet when you’re just starting A/B testing because you’ll be able to get started quickly. The only problem is that although some people have experienced massive success in these small variations, it’s pretty rare to find a change that will make a huge impact on your conversion rates.

Page Level Testing

Page Layout Variation

Page level testing is more complex than element level testing because it involves the testing of multiple elements on a page at once. For example, you could test page designs that have differences in overall style, organization or engineered marketing additions. It is more difficult to implement than element level testing because it requires more effort, skill sets, and time to develop the variations throughout the entire page, but when it is done correctly, it is more likely to yield significant results- up to a 10-30% increase in conversion rates.

Visitor Flow Testing

Visitor Flow Testing involves changing the journey that users go through when navigating your website. For example, you could test different variations of a user’s journey through the check-out process comparing single-step checkout against multi-step checkout. This type of testing can get much more complicated, so it requires the most effort for your development team to properly execute. However, if it is executed well, it will have the highest impact on your conversion rates.

If you’re not sure where to start, your best bet is to start with the content at the top of your funnel because it is the part that will be seen by the most people and can help you to generate more leads.

Focus on testing one variable at a time. Give enough allowance to gather quality data on which variation works best before testing another variable. A common mistake is testing too many variables at one time. If you test too many variables at once, it will be near impossible to isolate which change made the biggest impact.

Once you’ve chosen which element you want to test, then you can start your experiment.

Here’s a breakdown of how to conduct a good A/B test.

1. Form hypotheses

thinking of hypothesis gif

First, you have to propose a hypothesis about what you think will happen as a result of making a particular change. For example, you could hypothesize that changing your CTA button from red to green will increase your conversion rate by 20%. Use your hypothesis to design an appropriate plan that will test the validity of your claim. You should create at least one hypothesis for each element on your page that you wish to test across an extended period of time. Then, work on evaluating your hypotheses one by one.

Keeping an open mind to design and product features is important. Commit to a Key Performance Indicator (KPI) – the goal or result of the hypothesis – and rigorously change to meet your objectives.

2. Choose a hypothesis to start

Once you’ve created a bunch of hypotheses for the many elements on your page/product, you need to choose one hypothesis to start. It’s important that you only test one at a time because otherwise you won’t know which changes are attributed to which results.

not easy gif

There are several factors that you should take into consideration when deciding your testing order, and ideally you want to start by getting some “quick wins”. You can figure out which tests are easier by looking at:

  • Speed: How long will it take to gather enough data to make a statistically significant conclusion about your hypothesis? Pages with more traffic will hit statistical significance faster, making the test easier to complete. Tests that include a more noticeable difference will also finish faster because there will be a more significant difference between the results of the two versions.
  • Impact: How much will the change actually contribute to the improvement of your conversions? The evaluation of a test’s impact depends on your goal, but typically, tests with more dramatic changes will have a greater impact on overall conversion. Another factor that will affect the test’s impact is the placement of the change in the funnel. In general, pages at the top of the funnel have more impact than those at the bottom because they are exposed to more traffic.
  • Confidence: How confident are you that your hypothesis will yield favorable results? This is hard to measure and relies on your instincts and past experiences. Make sure to record your guesses and keep track of how well you predict your hypothesis so you can improve your instincts.
  • Ease: How easy is it to make the change? This one is pretty simple. Choose the test with changes that are easier to make so that you can get started faster.

Need help with optimization? We offer complimentary consultations in our discovery process. We can help audit your website to set a baseline on starting points for split testing.

3. Design your test

Now that you know the hypothesis you want to test, you need to actually design the two different versions that you will be running simultaneously. When you are designing your variations, make sure that you make informed decisions about your changes rather than going off on a whim and attempting a change that doesn’t make sense for your website.

Ultimately, A/B testing is best used to optimize the website that you currently have to improve conversion relative to what already exists. It is not meant to be used for innovation, like when you want to completely redesign your website and try a new approach. You should definitely avoid testing multiple huge, irrational changes in your A/B testing because you won’t be able to discern which change is attributed to which result. Perhaps one change led to an increase in conversion rate while the other led to a decrease – which definitely screwed up your final results.

So when you’re designing your variations, keep in mind these general tips:

  • Use Urgency and Positive Reinforcement: If you want to increase your conversion rates, you have to push people in the right direction with a sense of urgency coupled with positive reinforcement. Of course, you don’t want to overdo it to the point where it seems fake, but you can definitely take advantage of tactics that will help people feel better about their choices. For example, you can use colors and icons that are generally associated with success to encourage them that they are making the right decision.
  • Maintain Usability: You always want to make sure that your users feel in control. Constantly test and iterate your design to find ways to make user’s journey more intuitive. You can do this by creating clear and prominent CTAs that take users forward and clearly communicating meaning.

mobile usability


  • Experiment with Copy: The wording of the content on your page can actually make a large impact on site visitors. To start, even different variations of tone and length of sentences can make a difference in the message that is conveyed to customers. You should try to keep your CTAs clear and informative and avoid using jargon and filler words.
  • Track Behavioural Analytics: You can use behavioural analytics to monitor how customers interact with your web page. With tools such as heat maps, you can see where customers tend to spend the most time looking and what parts of your site are going unnoticed. With this information in mind, you can redesign your page layout to put your CTAs in the spots where people look at more.

heat map behavioural analytics

4. Choose your testing tool

Next, you have to choose a testing tool that will direct visitors to version A and B of your test content simultaneously in a random but equal way. There are many different tools that you could use to run your test including:

  • – This tool is very easy to use! You’ll be able to get a test started in minutes.
  • VWO – Similar to Optimizely, but also includes heat maps and sessions recording included at a cheaper price. The only problem is that it’s a little buggy.
  • Unbounce – Gives you the opportunity to build and test landing pages without coding
  • Google Content Experiments – Free, but you need to know some HTML and Javascript to get it going

Choose the tool that best suits your needs.

If you like to stay away from third-party software and have developer support, there are many ways to implement scripts that split test and communicate with your current analytics stack.

5. Evaluate your results

You can evaluate your results once you have statistically significant data. What this means is that you have done enough tests to make an educated conclusion about whether your hypothesis is true or not. It is recommended that your results are at least statistically significant at the 95% confidence level. To make sure you achieve this level of certainty, you can use the Effin A/B Test Calculator to determine if your results are significant. If they are not, you need to continue to run your test for a longer period of time to increase your sample size and make a better conclusion.


Now that you know the importance of A/B testing and how to get started, what are you waiting for? If you want to improve your results, you need to always be on the lookout for different tactics you can try to get those conversion rates up. At the beginning, it will be a lot of trial and error, but don’t be discouraged! Keep at it and you’ll get better at predicting and forming good hypotheses and you’ll able to see a real impact in your conversion rates.

Most importantly, remember that the best part of A/B testing is that the results don’t lie. Menikmati Labs is a strong believer in agile, data-driven decisions and we ultimately always look at the numbers to figure out the best course of action. We encourage you to rely on data as well!

If you have any questions about A/B testing or how you can use your analytics to make more informed choices, don’t hesitate to give us a shout!

Show some love!

This is a unique website which will require a more modern browser to work!

Please upgrade today!