An introduction to A/B split testing

Recently, my Personal Trainer decided that I should do excessive amounts of weightlifting in a short period of time. Towards the end of the last set I couldn’t do it. My arms were screaming like a choir of feral cats. I stopped short of getting to the target number of lifts and dropped the dumbbells to the ground. “I hate failing.” “Perfect,” he said. “The idea is that you fail at the end. That means you have burnt up all the energy in your arms and completely pushed yourself to the limit. You have burnt up more fat and added more strength than if you hadn’t done this.” he advised me. “Great, so I’m a successful failure” I said.

It got me thinking, how much do we hold back so we don’t fail? How good could we be if we push ourselves to the point of failure, learn our limits, and then get back up again and carry on with renewed knowledge and confidence? Failure can be a better teacher than success.

I am not advocating reckless actions and taking risks for the sake of it. But what would happen if we pushed ourselves further to find out if we have a good idea or initiative?

It’s ok to fail at the gym, it’s a safe place to fail in. Not too much is at stake. But what about advertising? Advertising can be costly and failure can be expensive. Is there a safe place to be a successful failure in advertising?

A history of failure to win

Claude Hopkins was one of the successful great grandfathers of advertising. He was a big advocate of experimental advertising. He would take his best performing advertising and beat it with incremental improvements. Claude called it Scientific Advertising. It was a process of success and failure. He risked failure in small amounts to create more successful and effective advertising. He said “It is not uncommon for a change in the headline to multiply returns from five to ten times over”. Sometimes changing a few words around in your headline can have a huge impact on the number of sales and conversions. You have to test a few different headlines until you find something that converts well.

What Claude was doing over 100 years ago is now called A/B Split Testing. This approach is a bit like an ad contest. You show half of your customer audience one ad and the other half a slightly different ad. You eliminate the worst performing ad and then introduce another ad to try and beat the hero. The process goes on until your best ad is difficult to beat. You can then take your winning ad to a wider audience with a larger budget, with confidence.

Your safe place to fail

Today, digital advertising is the perfect platform for developing powerful advertising messages. It allows us to take lots of small risks without losing too much money. We can test one headline, offer or message against another. For example, when we run Facebook and Google advertising campaigns we test safe headlines against more risky ones with the aim of getting more attention and better results. We test one benefit against another, one offer against another and so on. We keep trying new (sometimes risky) headlines until we beat the existing headlines. We measure success as the number of people who take our desired actions (conversions). A desired action could be filling out a form, clicking a button, making a phone call or completing a purchase online. You have to be ready to fail to succeed with this approach, but in the long run, you win.

A/B testing can also be used on website pages to discover the most effective page to achieve your goals. You can test a multitude of factors. A good place to start is with testing the headline, page colour, main image, offer or call to action. You don’t want to test all these things at once. You choose one thing to change up and then check the results and then add another thing to change and so on.

Of course this thinking can be used in traditional advertising too. In fact, this is where the strategy was invented.

I once did a direct mail campaign for a bank targeting large commercial customers. As a first step, we were trying to test an offer to see what prize was more appealing to our audience. We sent one group the offer of the chance to win a holiday if they booked a meeting with a Bank Manager to discuss their finance. We offered the other group the chance to go in the draw to win cash. The cash prize won. We then went to a wider group with the cash prize draw offer and had a successful response.

A/B Testing can also be used when a team can’t decide on which ad should be run. There can be two or three strong ad ideas on the table and the team can’t decide which one is best. You don’t need to lose sleep over this. Put the ideas to the test on a small group and let the customers decide which is best before you roll it out to the bigger audience.

A process to follow:

  1. Define your goal. What specific result do you want to improve?
  2. Create the Challenger. Design the idea that you think will beat the current winning champion ad or web page
  3. Prioritise the variables – Step two often delivers a number of possible ideas. Narrow it down to one thing to test.
  4. Run the test
  5. Measure the results and review
  6. Start the process again

How does it work practically

If you are running the A/B test on a digital platform like Facebook or Google they make it easy to run multiple ad ideas. If you want to run the test on your website then most of the major website platforms have extra tools that allow you to do this. There is often more to setting this up than in Facebook or Google so you may need the assistance of a Web Developer to do this. If you wanting to use traditional media to run split test then you will need to use other strategies like coupon codes and offers where you have to enter a code to allow you to track and measure the results.

Free 30-minute consultation.

How branding services can boost your business

If you would like to discuss your brand with an experienced branding agency, please call to make an appointment for a free 30-minute telephone consultation. One of our experienced Account Managers will discuss your brand and will do their best to answer your questions.

Our qualified and experienced team provide quality creative branding services and management skills to deliver work at the highest standards.

A well-crafted brand, consistently applied, is an immensely powerful tool for creating exceptional customer experiences, making sales,  building your marketing, business or organisation. So contact us today to see how we can help you produce world-class branding, marketing and communication.