A-B testing is a smart tactic if you know how and when to use it


This guide outlines when and how to use A-B testing to determine more about what works and resonates with your audience. When you use A/B testing — also known as split testing — you’re taking two versions and comparing them to see what performs better. You start an A-B test by initially figuring out what exactly you want to test.

For example, maybe you have two videos, and you want to determine which one is going to work best for conversions. You can show two sets of users who are assigned at random the different versions and determine which one most influenced your success metric. In this case, it would be which video led to the most conversions.

We can consider A-B testing as the most basic randomized form of a controlled experience. There are two treatments. One is the control for the other. You’re making changes to a single variable, and once you have that change, you show the two versions to audiences that are similar in size to analyze which one does better over a certain period of time.

As is true with other randomized controlled experiments, you have to estimate the sample size that you’ll need for statistical significance. Different audiences are going to behave differently from one another, which is why A/B testing is valuable. Something working for one organization may not work for another. In marketing, A/B tests are valuable because they’re low-cost and have a high reward.

How to perform A-B testing

We’ve touched on the very basic elements of performing A-B testing, but the steps to actually do a split test can include the following:

  • Pick one variable that you’re going to test. You usually need to isolate a single independent variable and measure the performance of just it because otherwise, you’re not going to know which variable led to the performance changes. Even small changes can lead to big improvements, and it’s actually often easier to measure the small changes compared to the big ones.
  • Identify your goal. You want to choose a primary metric to put your focus on before running A/B testing. You might even have a hypothesis before you set up your test. If you think about the metrics you care most about after setting up your split test, then changes that you’re using could affect user behavior, so you aren’t setting up your test in the most efficient and effective way.
  • Once you have an independent variable, a dependent variable, and the desired outcome, you can set up the unchanged version of what you’re testing as a control.
  • If you’re running a test where you have more control over the audience, such as with email marketing, you want to test at least two audiences that are equal so that your results will be conclusive.
  • Determine a sample size. You need to make sure you’re choosing a big enough sample size that you’re going to get statistical relevance, as mentioned above. If you’re testing someone without an audience that’s finite, like a page of your website, then how long you run the test affects the sample size. You need to run the test for long enough to generate adequate views. Many experts say that you should run A/B testing for at least seven days.
  • Figure out how significant you feel your results need to be to justify picking one variation over another. Most marketers are aiming for 95% results certainty, but you need to consider your particular situation because that’s not always the right answer.

When should you use A-B testing?

The following are times that you can specifically think about using an A-B test.

  • If your organization posts on social media, there are a lot of ways you can split test. One particular thing to look at is the optimal social media post length. You can experience not only the length and number of characters but also the style of the post, the use of emojis, and the tone of voice.
  • For videos that you’re using on social media, you can test text only compared to posts that have a video or image. You can also test the length of videos to see which works best.
  • A-B testing ad format is a big one on social media. You can try out different formats to see how you’re going to get the most effective. In Facebook advertising, for example, you might find that carousel ads are best for product announcements, but then if you’re launching a new store, you might want to use a button for people to get driving directions to your location.
  • Target audiences are a variation of A-B testing on social media where you’re showing the same ad to different audiences to determine which gets a better response. If you’re using Facebook ads, it might show that some audiences respond favorably to retargeting ads, and other audiences don’t like them at all and find them invasive. When you use A-B testing for targeted audiences, you can develop tailored campaigns that are very specific to each audience.
  • Split testing can help you decide on how to use the content on your website. If you’re testing blogs, you can see which length of content is favored more by your audience.
  • Test mobile calls-to-action to see which is going to convert best. You could vary whether or not someone can close out your call-to-action boxes. This would be largely classified as a user experience test.
  • Some tests can be as simple as making minimal design changes. What happens to your click-through rate if you change the color of your CTA button, as an example?
  • You can use testing when you change prices. You might want to change your price to raise revenue, but this comes with risks. If you’re weighing the pros and cons of a price change, test it first. Some customers might be more likely to convert if the price is above a certain point, and others will want the lowest prices.


These are just a tiny sample of the places you can use split testing in your business and marketing efforts, regardless of your specific goals.

About Business Woman Media

Our women don’t want to settle for anything but the best. They understand that success is a journey involving personal growth, savvy optimism and the tenacity to be the best. We believe in pragmatism, having fun, hard-work and sharing inspiration. LinkedIn

Recommended for you