Split Testing – What it is & Why You Should Be Doing it

You have probably heard the term “split testing”. But what exactly does it mean?

Here’s what we are going to discuss today:

• What is split testing?
• Why should you use split testing to optimize your sales funnel?
• How to conduct a proper split test?

Want to make more money with the same amount of traffic?

Split testing can help you do that.

Continue reading…

What is Split Testing?

Split testing, also known as A/B testing, is a conversion rate optimization technique that marketers use to increase conversion rates of web pages.

Here’s how it works:

1. You create two variants of the same page – variant A and variant B. There should be only one difference between these two variants. That’s the element that you are testing
2. You split the traffic into two and send half of it to the variant A and half of it to the variant B
3. You keep the winning variant

Example: Green CTA Button vs. Red CTA Button

Performable once did an A/B test to see whether the color of the call to action (CTA) button had an effect on the conversion rate of their landing page.

Here’s how variant A and the variant B looked like:

Their hypothesis was that there would be no significant difference between the conversion rates of these two landing page variants.

“My hunch was that even if one color performed better than the other, the difference would be small. 

I could imagine that one color might be more appealing or grab the user’s attention better than another, but that the overall conversion numbers would be overwhelmed by the overall message of the page. 

I assumed that the results of this test would show what we’ve seen in testing before — that the major difference between good and poorly converting pages was the message the page was communicating,”shared Joshua Porter, then VP of Product and Community at Performable.

shared Joshua Porter, then VP of Product and Community at Performable.

So imagine how surprised the team must have been when the red button outperformed the green button by 21%. That’s a huge increase in the conversion rate!

This is a great illustration of the fact that while it’s tempting to think that you “just know” what the outcome will be, you can’t possibly know until you run the test, which is why you shouldn’t skip it no matter how confident you feel.

Why Should You Use Split Testing to Optimize Your Sales Funnel?

Want to increase your sales?

There are two ways to do that:

1. Increase the amount of traffic that you drive to your sales funnel
2. Increase the conversion rate of your sales funnel

Ideally, you should do both.

However, as you might already know from experience, traffic is expensive. You are either paying for it with your money (paid advertising) or with your time (content marketing).

That’s why it makes sense to optimize the conversion rate of your sales funnel first, then start increasing its traffic.

That way, you will maximize the ROI of your marketing efforts, instead of pouring water into a leaky bucket by driving traffic to a sales funnel that doesn’t convert well.

Here’s how Joshua Porter explained it:

“Consider this: a 21% increase in the conversion of this page is potentially a 21% increase to all downstream metrics.

So by getting 21% more people to click at the top of this process, we added 21% at the bottom as well.

This is why optimizing pages is so valuable. We did not have to increase traffic to the page to see improved results.”

Of course, not every split test will lead to such spectacular results, in fact, the vast majority of them probably won’t produce a lift in the conversion rate at all.

Meanwhile, some of them will result in seemingly insignificant increases in the conversion rate.

However, over time a 1% increase here and a 3% increase there can add up to a drastic increase in the overall conversion rate of your sales funnel and have a huge effect on your bottom line.

Finally, occasionally you will stumble upon a big win like Performable did, which will increase your sales funnel conversion rate overnight if you keep the winning variant.

What Should You Split Test? (A/B Testing Ideas)

Is there a way to increase the likelihood of you getting a big win with split testing? Yes, there is.

Not all page elements are made equal when it comes to the conversion rate of that page. Some of them are more important and some of them are less important. 

So it makes sense to start by testing the most important elements first because they are likely to have the greatest impact on the conversion rate.

Here are the key page elements that you might want to test:

#1 The Headline

The headline of the page is arguably the most important page element because it’s the first thing that the potential customer notices.

Its purpose is to grab the potential customer’s attention and make them curious enough to continue reading.

The most important copywriting principle that you need to understand if you want to write a compelling headline is the distinction between the features and the benefits of a product:

• A feature is a quality or a function of a product (e.g. “These shoes are waterproof!”)
• A benefit is the value that the potential customer will get from that product (e.g. “These shoes will keep your feet dry!”)

People buy based on benefits, then use features to justify the purchase.

Think about the #1 benefit of your offer. How will it make the potential customer’s life better? That’s what you want to emphasize.

So brainstorm various headlines and then test them against each other to see which one performs the best. 

#2 The Subheadline

The purpose of the subheadline is to explain the offer in more detail.

However, it should also be benefit-driven, not feature-driven. You should use it to reinforce the value of the offer by expanding on its #1 benefit.

Once you are happy with your headline, start testing various benefit-driven subheadlines to see which one works best.

For example:

Look at the headline and the subheadline that Brennan Dunn uses on his Double Your Freelancing online course sales page:

• The headline makes it clear what the #1 benefit of the product is. Is there a single freelancer on this planet who would say no to doubling their rate?
• The sub-headline explains the offer in more detail while reinforcing the number #1 benefit of the product. Adding a timeline of “less than 30 days” makes it especially powerful.

Note this headline and subheadline combination doesn’t even make it clear what the product is…

But you bet you are going to keep reading to find out if you are a freelancer.

#3 The Call to Action

The call to action is the part of the copy where you tell the potential customer what you want them to do next:

• Subscribe to your newsletter
• Download your lead magnet
• Buy your product

…etc.

It should be:

• Clear
• Concise
• Benefit-driven

Experiment with various calls to action to see which one resonates with your potential customers the most.

#4 The Call to Action Button Color

As we have learned from Performable’s experience, the color of the call to action button can have a huge effect on the conversion rate of a landing page.

The guiding principle when it comes to the CTA button color is this:

Pick a color that stands out in the color scheme of that page so that the CTA button would be impossible to miss.

So instead of testing a bunch of random colors, test ones that contrast with the page’s color scheme.

For example:

We use a dark background above the fold on our homepage. Note how the bright blue CTA button immediately draws your attention.

#5 The Call to Action Button Copy

Entrepreneurs often don’t give much thought to the call to action button copy but it can have a significant effect on the conversion rate of a landing page.

What’s important is to make sure that it’s benefit-driven.

For example:

Let’s say that your lead magnet is a free ebook.

Instead of using generic phrases (e.g. “Download Here”), you could make your CTA button copy more persuasive by reminding the potential customer what’s in it for them (e.g. “Get Your Free Ebook”).

Test the Most Important Page Elements First!

A/B testing can be a huge waste of time if you focus on-page elements that are unlikely to have a significant impact on the conversion rate.

Don’t go down to the rabbit hole of testing random stuff!

It’s best to go after the lowest hanging fruits first (e.g. the five-page elements that we have just discussed) since that is what is the most likely to lead to big wins.

Once you have the core elements of your page optimized, then you can get creative and start testing minor tweaks.

How to Conduct a Proper Split Test

So how do you conduct a split test?

Unless you:

1. Are a statistics aficionado
2. Have a ton of free time

… you will need to use split testing software that can do the statistical heavy lifting for you (e.g. ClickFunnels has an in-built A/B testing functionality).

Once you have the software, it’s time to run the experiment:

#1 Pick the Page Element That You Want to Test

We recommend you start with the headline because that’s what’s most likely to lead to a big win.

Alternatively, you can pick one of the other four page elements we have discussed.

Once again, test the most important page elements first.

#2 Create a Hypothesis

You should then create a hypothesis:

1. What are you testing?
2. What result do you expect?
3. Why do you expect that result?

For example:

1. You are testing a feature-driven headline against a benefit-driven headline
2. You expect that the variant with a benefit-driven headline will win
3. You expect that because pages with benefit-driven headlines typically convert better

You might wonder why do you need to go to the trouble of formulating a hypothesis when the only thing that matters is the result of the experiment.

It’s because formulating a hypothesis helps you develop a better understanding of conversion rate optimization.

Consequently, that better understanding makes you:

1. More likely to come up with tests that produce an increase in the conversion rate
2. Less likely to waste time on tests that lead nowhere

In other words:

Formulating a hypothesis before running the test makes you more effective at A/B testing (over time).

#3 Create the Variant B

We assume that you already have a page that you want to test. That current page is variant A.

Now you need to create the variant B that is identical to variant A except for the one-page element that you are testing (e.g. the headline).

Note that there must be only one difference between variant A and variant B.

Otherwise, you won’t be able to interpret the test results because you won’t be able to tell what exactly has caused the difference between the conversion rates of the two variants.

#4 Drive Traffic to the Variants A and B

Once you have set up variant A and variant B, it’s time to start driving traffic to them.

Obviously, you need to drive the same amount of traffic to each variant, otherwise, you might struggle to interpret the results.

The reason this conversion rate optimization technique is called split testing is that you split the traffic to that page into two and send half of it to variant A and half of it to the variant B.

#5 Evaluate Test Result

Your A/B testing software will tell you when you can end the experiment. Keep the winning variant.

Common Split Testing Mistakes That You Need to Avoid

Here are the three most common split testing mistakes that you should be wary of:

#1 Your Sample Size Isn’t Large Enough

Statistical significance is an important concept to understand if you want to be effective at split testing.

Basically, the larger the sample size, the more likely it is that the difference in the conversion rate between the two variants can be explained by the page element that you are testing.

For example:

Imagine that you are testing the headline of your sales page.

You then run a split test with just two unique visitors:

• One person visits variant A and doesn’t buy your product
• One person visits variant B and buys your product

But does that mean that variant B has a better headline?

It doesn’t.

When the sample size is that small, there’s no way to tell what made the person who visited variant B buy your product.

There’s an infinite number of potential explanations:

• Maybe it was indeed the headline
• Maybe they were planning to buy your product for a while and just happened to do it that day
• Maybe they were blackout drunk and went on an online shopping spree that they don’t even remember (expect a refund request)

Etc.

The point being:

You can’t possibly know what happened!

Therefore:

The test result is useless!

Now imagine that you run the same split test but you send one million unique visitors to each variant instead:

• 100,000 people buy your product on variant A (10% conversion rate)
• 200,000 people buy your product on variant B (20% conversion rate)

Is it likely that 100,000 more people just happened to buy your product on variant B for various reasons unrelated to its headline?

It’s extremely unlikely.

At that point, the experiment has reached statistical significance that is high enough to conclude that the difference in the conversion rates must have been caused by the headline.

So how do you know when the experiment has reached statistical significance that is high enough to make the result valid?

Fortunately, you don’t have to do the math yourself, you can use A/B testing software for that. It will tell you when you can stop the experiment.

But you need to understand that if you don’t have enough traffic, A/B testing might not be feasible because reaching that point will take ages.

In that case, it’s best to use other conversion rate optimization techniques instead, then reconsider A/B testing once you have enough traffic to make it feasible.

Note that running split tests that don’t reach a high enough statistical significance is a waste of time. Why?

Because the outcome of an experiment like that is bound to be nothing more than statistical noise.

In fact, relying on “results” from poorly designed experiments is worse than guessing because in the case of the latter you at least aren’t deluding yourself into thinking that your decisions are “data-driven”.

You might want to read up on A/B testing statistics to avoid mistakes like that:

“A/B Testing Statistics: An Easy-to-Understand Guide”(Conversion XL)

#2 There’s More Than One Difference Between Variant A and Variant B

We have already discussed this but it’s such a common split testing mistake that we want to reiterate:

There should be only one difference between variant A and variant B.

Otherwise, the result of the test is useless, since you have no way to know which change caused the difference between the conversion rate of variant A and variant B.

That being said, there is an advanced conversion rate optimization technique called multivariate testing.

You might want to read up on it if you want to be able to test several page elements at the same time:

“When To Do Multivariate Tests Instead of A/B/n Tests”(Conversion XL)

#3 You Aren’t Following the Best Conversion Rate Optimization Practices

Online marketers have been using A/B testing for more than two decades now. At this point, we know what works and what doesn’t.

So before you start optimizing your sales funnel with A/B testing, make sure that every page in that funnel follows the best conversion rate optimization practices.

That will prevent you from wasting time reinventing the wheel.

For example:

When people ask our co-founder Russell Brunson for feedback on their landing pages, the #1 mistake he sees is this:

Their landing pages are way too complicated!

That’s why he often advises people to delete 90% of the stuff on them.

Here’s how Russell explains it:

Now, you could A/B test each piece of clutter on your landing page… But who has the time for that??

It makes more sense to follow Russell’s advice and simplify your landing page because that is a best practice that is extremely likely to have a positive effect on its conversion rate.

We recommend you to:

1. Read up on conversion rate optimization, copywriting, and landing page design
2. Learn what the best practices are
3. Implement them in your own sales funnel

Once you have done that, then you can start A/B testing page elements.

Want Russell to Show You How to Build a Sales Funnel?

But what if you don’t have a sales funnel yet?

We understand that building one from scratch can seem like a daunting task.

That’s why we created our 5 Day Challenge where Russell walks you through it step-by-step.

You will learn how to:

• Generate unlimited leads
• Create your first lead magnet
• Build your first sales funnel
• Create a simple 6-email follow-up sequence
• And launch your funnel!

…in just five days.

So don’t hesitate.

Join our 5 Day Challenge. It’s completely free!