Email A/B Testing: 10 Takeaways from PostUp’s Webinar

email marketing a/b testing webinar recap

You don’t need a webinar to know that winning feels good.

email A/B testing

Whether on the ice or in the inbox, there’s nothing like being able to declare victory, take home a trophy, and maybe even do a little dance.

This week, PostUp’s Manager of Strategic Services Kelly Ford and Campaign Manager Tim White hosted “Big Wins: Tips for Efficient and Meaningful Email A/B Tests,our webinar with the Email Experience Council. Attendees learned how to implement A/B testing, analyze the data, and use it to achieve big wins in the inbox.

Did you miss the webinar (and our livetweet)? No worries, you can catch it on-demand here!

In the meantime, we’ve got a recap you can check out. Here are ten takeaways that will get you dumping Gatorade on your email program.

(Figuratively, of course. Unless you’ve got a whole lot of rice handy.)

Don’t start A/B testing without a plan!

Decide what email metrics you want to improve, and keep this in mind as you design your A/B test. Identify the areas and aspects you want to test, then develop a hypothesis.

Know your baseline.

It’s hard to draw meaningful conclusions from your A/B testing efforts if you don’t take note of your email program’s history. Look at past data and use that to form your test and interpret its results.

Don’t assume!

You might think you know what email version will prove the winner, but the results can surprise you! Don’t let preconceived notions skew your analysis. After all, you know what they say about people who assume: their email A/B testing results are meaningless.

Try testing your preheader text.

Plenty of marketers test the subject line, but they often neglect preheader text. In most email clients, it’s extra space getting your message across, which means it’s real estate that’s ripe for testing. Try tweaking your preheader to see if your ideas flip or flop.

Don’t be afraid to take risks.

While tiny tweaks like CTA color can improve conversions, they might not always move the needle. If you’re not seeing significant results, try raising the stakes by testing bigger changes.

When you go big, send small.

Taking risks can pay off in an email program, but you don’t want to alienate your entire audience. Try testing your out-of-the-box ideas or more drastic changes on smaller segments of your email list. But not too small, because you also have to…

Make sure you have a statistically significant sample size.

For results to accurately reflect your email audience, you need the 4 Ss: a Statistically Significant Sample Size. Make sure your testing audience has at least a thousand members, or else “SSSS” will just be the sound of the air coming out of your email program.

Ultimately, the size of the split depends on the test.

A 50/50 A/B test might get the job done for your test, but you might also try different splits. PostUp allows senders to use 10/10/80 splits: 10% of the list gets the A version, 10% gets the B version, and the winning version is automatically sent to the rest of the list. Your audience wins, and so do you.

When do you declare a winner? That depends too.

If you’re testing a time-sensitive email, such as a daily news digest, the winner might surface in a few hours. Other kinds of emails, such as an email in a drip series, are more of a long game. Take the kind of communication into consideration before rolling out the trophy.

Finally, keep track of your email A/B testing results!

Testing is a continuous process. Let previous results inform future tests, and you can make sure your email program is always on the way up.

Looking for more ways to win in the inbox? Check out our library of on-demand webinars, e-books, and solution guides in our resources section.

Editor, PostUp PlayBook