A/B Testing

A/B testing, also known as split testing, is a method of comparing two versions of a web page or app to see which performs better. You can test anything on a web page: the text, the layout, the colors, the images, the buttons, and even the form.

A/B Testing, or split testing, is a test where you are testing one thing against another. an “A” vs a “B”  and making decisions on a better outcome. For instance Grape Jelly vs Strawberry Jelly on your PBJ. In the PBJ testing usually, you are using your own judgment but in marketing, we like to test against a bigger audience.

TL;DR What is A/B Testing

Testing one thing against another (A vs B) in the hopes of finding a better solution.

Why A/B Testing is Important (in marketing)

TL;DR: A/B Testing will improve conversion rates of your current sales funnel leading to more sales/form fills for the same amount of effort.

You have an e-commerce store selling widgets for $20.00 apiece and are receiving 10000 visitors to your site a month.

On that website, 4% of visitors visit your e-commerce pages, and 5% of those visitors purchase.

Through split testing, you change the navigation to highlight widgets better. This takes your site from 4% to e-commerce pages to 8%. Even if you stay at a 5% conversion rate of those visitors the number of sales will be much higher through simple testing.


10000 visitors x 4% = 400 e-commerce visitors

400 e-commerce visitors x 5% = 20 sales

20 sales x $20 widgets = $400 dollars

A/B testing

Test A: (original)

5000 visitors x 4% = 200 e-commerce visitors

200 ecommerve visitors x 5% = 10 sales

10 sales x $20 widgets = $200 dollars

Test B: (updated navigation)

5000 visitors x 8% = 400 e-commerce visitors

400 e-commerce visitors x 5% = 20 sales

20 sales x $20 widgets = $400 dollars

In this made-up example, the split testing actually brought in more review at $400 vs the $200 from the original numbers. This is not always the case but since Test B proved to be more profitable the next month we would send all 10000 visitors to the new navigation (what we testing) and see a 100% increase in sales at $800 if I mathed correctly.

Of course, some split testing results show to be worse, but over time split tests prove to be a strong way to get more out of your current efforts.

When to use A/B Testing

  • Advertising
  • Email Marketing
  • Landing Pages
  • website CTAs

When to not use A/B Testing

We would recommend not using A/B testing is when you don’t have a clear objective. If you don’t know what you are trying to improve you won’t know what is performing better or not.

In Marketing, we recommend not wasting time on A/B testing until you are seeing a big enough sample size to make an informed decision.

How A/B Testing Works

The way Search Envy runs our A/B testing is we first state our SMART goal. Brainstorm ways on what changes we can make to test this idea by changing CTAs, Colors, or even images.  Testing for a number of days against a certain sample size and choosing the better outcome then repeating the process. The number of tests we run is usually limit to 1 to 2 changes and waiting to see which is the best outcome.

A/B Testing in Email Marketing

Usually done by testing a small % of your current list to see which CTA converts better, which subject line gets more opens, or which email gets more replies. After

A/B Testing in Digital Marketing

Before all the marketing efforts of most companies went online. Businesses and Agencies would also test mailers to small groups before sending them to the entire list. Back then it was much more important to get the message right before it was too late. Now with digital marketing CTAs and caught spelling errors can be changed on the fly and continuously improved upon (with split testing). This is part of how we run our website development with our clients.

A/B Testing in Advertising

A proper PPC agency will run split testing against all ads to work towards better click-throughs and ultimately better conversions.

A/B Testing Tools

Google Optimize – Free

Google Website Optimizer was a free website optimization tool that helped online marketers and webmasters increase visitor conversion rates and overall visitor satisfaction by continually testing different combinations of website content. Wikipedia

Google Analytics – Free

Google Analytics is a web analytics service offered by Google that tracks and reports website traffic, currently as a platform inside the Google Marketing Platform brand. Google launched the service in November 2005 after acquiring Urchin. Wikipedia

Leave a Reply

Your email address will not be published. Required fields are marked *

Glossary Quicklinks

Free Growth Strategy Session

Our Services

Our Industries

Table of Contents