Welcome, or welcome back, to our Growth Hacking Academy! In this lesson we’re tackling A/B testing. Plenty of people talk about it, very few genuinely use it. They don’t realise what A/B testing brings to defining an objective, effective marketing strategy.
How often have your efforts produced results you didn’t expect and couldn’t explain? A good growth hacker has to know every secret of A/B testing.
On the contrary, A/B tests, split testing and multivariate testing should be their daily bread.
Follow along and let’s test together.
Need a list of growth hacking tools?
Need tools to grow your business? Want to use growth hacking for your clients and work out which tools are available? No problem, we’ll give you the full list. FREE!
Request the free growth hacking tool list
A/B testing: what is it?
Does this outfit suit me better, or that one? Have you ever asked a group of friends or family that? If so, you’ve unconsciously run an A/B test. The idea is simplified a little, but not far from reality.
Let’s take it step by step. An A/B test, or split-run test, is an experiment that puts two versions of a single variable (hence A, the control version, and B, the varied version) in front of a statistically meaningful audience to determine which of the two is more effective.
A/B tests: what has to be in place
Let’s drop that definition into the concrete reality of a website, or better, a single page. Picture a landing page in particular. Anything we put on the web for business reasons has one goal: to convert.
Identify the goal
Your user converts if they take the action you want them to take, the one you designed the landing page for. Actions vary with the business goal. The goal of a landing page on a site selling weight-loss products differs from that of a freelancer’s site or an advertising agency’s. What’s your goal? That’s the first answer you need clear in your mind when it comes to A/B testing.
We’re often convinced we know what we want to achieve, when in fact we don’t.
Imagine you want a better car than the one you own. What does better mean to you? Faster? More powerful? More features? Shorter? Longer? Cheaper to run? There’s no right answer. There’s your right answer.
The same goes for your landing page: define your goal clearly and numerically.
Identify the method and the audience for the test
Who do we want the A/B test to reach? Everyone who visits our site? A specific portion of them? In that case, on what criteria do we choose the portion?
Once the audience is identified, by what method do we serve the two versions? In 99% of cases version A goes to 50% of the audience and B to the other 50%. Nothing stops you choosing differently for good reasons.
Define the variable
We have our landing page. We know what action the user has to take for the result of our A/B test to count as positive. We’ve decided who to test and how. One detail is missing: what’s our variable? That is, which element separates version A from version B?
It could be the colour of the button serving as the call to action (CTA), or the heading font, or the heading text, or any other element on the page. Better that it’s one element only — we’ll come to why.
Choosing when to test and for how long
How long does an A/B test have to run to be meaningful?
This is delicate and not easy to pin down. With a large number of users, is a day enough? Two days? Do they have to be consecutive? Or can we run one test today and another in a week? Does it make sense to run the test over a weekend? Or to run it a specific number of times at the weekend and the same number of times during the rest of the week? How might public holidays such as Christmas or Easter affect the outcome?
Here too there’s no right answer. There are businesses where testing during holidays makes sense, because a differing reaction from the audience might suggest running different versions at different times of year. In other cases tests run during a holiday may be meaningless and skew the results.
In short, nobody becomes a growth hacker overnight — you get there with experience and by using your head. But we’re confident you’ll have no trouble on that front, especially if you keep following us.
Need a list of growth hacking tools?
Need tools to grow your business? Want to use growth hacking for your clients and work out which tools are available? No problem, we’ll give you the full list. FREE!
Request the free growth hacking tool list
A/B testing: a worked example
Let’s work through an example to clarify how an A/B test is run.
Imagine we handle the web presence of a spa whose franchise we want to push. Can the conversions we’re getting be improved? In our case a conversion is the user clicking an arrow-shaped button — our call to action.
This is our reference page, the one corresponding to version A of the test:
Single-variable test
The page doesn’t hold many elements, so isolating one to change isn’t hard. Let’s try moving the call to action.
We then run the A/B test for the period we’ve set, serving it to the users we judged reliable for the purpose.
Which of the two versions converted better? We’d have bet on version B, and instead A converted better? That’s the great advantage of A/B testing: objectivity. Going on personal taste, intuition or a presumed ability to read other people’s minds almost always leads to mistakes. Data: objective, real data is what a well-designed test gives back.
If the conversion result satisfies us, we stop there as far as the CTA goes. If we want to be more precise we can keep the colour that converts better and change the position for a further reading — but either way we know exactly which variable moved the result.
Multivariate test
Now imagine we’d run our first test with a version B changing more than one variable at once (position and colour of the CTA, background colour, text colour).
Whatever the result, how do we isolate the variable responsible? To know precisely, we’d have to repeat the test, breaking it into sub-tests each with a single variable.
A/B testing tools
We’re confident we’ve convinced you of the value of good, continuous A/B testing.
And how do I run the tests? How do I serve different pages to different groups of people at the same time? Wondering that? Don’t worry, we haven’t forgotten to point you at the tools for putting this into practice — although, once you’ve grasped the idea, finding tools online takes a moment.
Meanwhile it’s worth remembering that everything in digital marketing is testable: you can run A/B tests on Facebook, on posts on any social network, on a page, an email campaign, an article — anything at all.
Here are our tool recommendations.
Google Optimize
Assuming you have at least one Google account, we’d suggest starting with Google Optimize, which lets you set up an A/B testing campaign professionally and completely free.
It requires the site you’re working on to be connected to Google Analytics — but a growth hacker like you will have done that already. So on with the online A/B test!
Visual Website Optimizer
Known as VWO, one of the most widely used and most complete paid tools.
Opteo
A tool built for A/B testing ads, letting you automate tests on ad copy alongside plenty of other SEM features.
Mailchimp
Good old Mailchimp comes to the rescue for A/B tests in email marketing too.
Conclusion
In this lesson we’ve covered one of a growth hacker’s most important skills: recognising the importance of data, not least through proper A/B testing. You’ve learned that A stands for the control and B for the varied element. You’ve seen that results can surprise you, and that this is precisely why tests matter — because they’re objective. And you’ve grasped that what makes the difference isn’t so much the tool you use as the strategy you’re able to set.
Growth hacking is a question of mindset before it’s a question of technique. Which is why we’ll never suggest often enough that you read and re-read the earlier lessons and stay tuned to your favourite Growth Hacking Academy.
Want to become a growth hacker? Get in touch! We’ll find the right course for you.
Want to ask us something or just talk to us? Get in touch! That’s what we’re here for.