What is A/B testing?

A/B testing is a method of comparing two versions of an advertisement, webpage, email, or other marketing element to determine which version performs better against a specific goal. It is also known as split testing.

In advertising, marketers typically compare Version A (the control) with Version B (the variation) to identify which generates better results, such as more clicks, leads, purchases, or a higher return on ad spend (ROAS).

How does A/B testing work?

1. Define the objective: Decide what you want to improve, such as click-through rate, conversions, or sales.

2. Create two versions: Prepare Version A and Version B, changing one key element when possible.

3. Split the audience: Show the versions to comparable audience groups under similar conditions.

4. Run the test: Collect enough data over an appropriate period.

5. Compare results: Analyze the chosen metrics and assess whether the difference is meaningful.

6. Implement the winner: Use the better-performing version and continue testing other improvements.

Example of A/B testing in advertising

A Shopify jewellery store wants to increase purchases through Instagram ads.

Version A — Product image

Headline: “Discover Your Perfect Ring”

CTR: 1.8%

Version B — Lifestyle image

Headline: “Find a Ring for Your Forever”

CTR: 2.5%

In this hypothetical example, Version B has a higher CTR. However, the store should also compare purchases, CPA, and ROAS before deciding that it is the better sales ad.

What can be tested?

• Ad images and videos

• Headlines and ad copy

• Calls to Action (CTAs)

• Audience segments

• Landing page layouts

• Offers and discounts

• Email subject lines

• Product page designs

A/B testing vs. multivariate testing

In short: A/B testing helps marketers make data-driven decisions by comparing two versions of a marketing element and identifying which performs better against a defined goal.