Why Split Testing is Pure Profit Science

The second Monday of October is Canadian Thanksgiving. Yes, we Canadians have our own Thanksgiving, six weeks earlier than America.

It’s one of my favorite times of the year. The weather is often sunny, starting off with crisp, clear mornings followed by comfortable afternoons, and finishing with beautiful early evening sunsets while we eat our big dinners.

There might be a little less football, but we don’t have to think about racing to the mall next day for Black Friday sales. It’s a relaxing holiday.

Several years ago, I spent Thanksgiving on my family’s farm. While sitting around my mother’s kitchen on Saturday afternoon, I noticed she had a fruit fly problem.

“Mom, what’s with all these bugs?” I asked. They seemed to be everywhere, and a few were even biting me on the ankles.

“I know, they are worse than ever this year,” she said.

The next morning I returned for breakfast and noticed she had set up a jar of apple cider vinegar beside her fruit bowl. There were dozens of fruit flies stuck in her trap, but there were still more swarming around the apples and bananas.

That got me thinking. Didn’t the old saying go, “You catch more flies with honey than with vinegar?”

The marketer in me recognized this was a perfect opportunity for a split test.

I threw out the old jar of vinegar and put a fresh bowl on the counter. Beside it I added a little jar of honey.

The test was on!

My mother laughed at me.

“The honey doesn’t work, I already tried it,” she said.

Undeterred, I said, “We’ll see.”

It was a classic A/B test just like we do in my businesses all the time.

• We were directing the visitors to one of two choices.
• We changed only one variable.
• And it was going to be easy to measure the responses.
• Now all we had to do was wait to get a big enough sample size.

For example, during one of recent product launches at ETR, we set up a split test on two headlines.

We sent thousands of visitors to the site through email marketing.

Half the visitors saw Headline A.

The other half saw Headline B.

Within two days we noticed that twice as many people bought when they saw Headline A, and so we ended the split test, declared a winner, stopped sending traffic to Headline B, and started looking for something else to test.

Back to the fruit fly split test on the farm…

A few hours had passed. I went off to the gym for a workout, ran a few errands, caught up with some old friends, and returned about an hour before Thanksgiving dinner was ready.

It was time to check in on the results.

Mom was right. The apple cider vinegar trap was full of fruit flies, and there wasn’t a single one in my jar of honey.

What the heck?

Did the fruit flies know about the honey trick? Were they able to Google this on their little Fruit Fly Internet?

I was shocked.

But that’s how split tests tend to go. You might expect that your over-the-top headline is going to outperform a boring headline suggested by one of your marketing team, only to find out that your idea was a dud and their suggestion was a stud.

Or maybe you think you’ll get twice as many sales with a lower price, only to find that a higher price (with everything else remaining the same) brings in just as many sales, and a better customer overall because the higher-price point leads to more upsells.

Split testing is pure profit science when done right.

Let me show you how you can be using split tests in your online (and offline) business.

It’s very easy to set up a split test.

There are all sorts of software programs out there. If you’re on a budget, you can use the free split test software from Google Analytics. If you have a few bucks to spend, you can set up Optimizely. That’s what we use at ETR.

In a real world split test, you can often track the results manually. For example, let’s say you run a radio, television, or newspaper ad. You can have the respondents call one of two different numbers, or reply by mail to one of two different post office boxes.

For example, as a kid, I noticed that print ads often gave a return address of Post Office Box 900A. I’m sure other ads gave a return address of Post Office Box 900B.

Those were ways to collect responses to two variations of the advertisement.

If you have a call center, or someone answering the phones at a dentist office, you could give two different sales scripts and see which works better for booking appointments or closing sales.

For our websites, the two most important things to test are:

1. Price
2. Headline

To be accurate, you must follow the scientific method and only test one variable at a time. You also need to let the split test run longer than most people expect. Most people stop a split test after just 50 orders, but that’s probably too soon.

One of my friends has run thousands of split tests, and here’s what he says:

“We generally look at about 400-500 sales to make a decision. If we’re doing a split test on price — which is the most important variable to test — we might run it for 700 or 800 sales. The only times we’ll end a split test early is if one of the options is winning big (by 25% or more) after 150 sales. But I never cut a test short of 400-500 sales if it’s only up 10%. I’ve seen something be up 10% and then lose when we got more data.”

In the case of my fruit fly test, I applied the “mercy rule.”

After all, when no one is taking one of the options, and everyone is taking the other option, it’s safe to say you can end the test early.

That’s what I did with the honey. In the end, I was glad to be wrong (as I have been in so many marketing split tests too), because now I could have my split test results, and eat the honey, too. That made the experiment — and Thanksgiving — all the sweeter.

Do you want to accomplish 10x more in half the time? Download the free 90-Day Blueprint my coaching clients use to achieve greater success: https://www.earlytorise.com/90-day-blueprint/