How To Perform A/B Testing In Email Marketing

A/B testing, also known as Split Testing, is a method of comparing two versions of an email against each other to determine which one performs better. A/B testing is an experiment where two or more variants of an email are shown to users at random, an analysis is than used to determine which variation performs better for a given goal such as opens or clicks.

A/B testing can be used to test almost any element of an email, but only perform one test at a time to get accurate results.

In the above scenario, we are testing two emails which are the same but have different call-to-actions. To see which gets the most clicks, we send 20% of the audience each email and wait 1 hour to review the results. The winning email is the one with the most clicks. Than we send the remaining 60% of the audience that email.

A/B testing allows companies to make careful changes to there emails while collecting data on the results. A/B testing can be used regularly to continually improve a given metric, such as open rate over time.

Here are some of the best practices to follow for A/B testing in email marketing:

Only test one variable at a time: This will give you the most accurate results, as if you make multiple changes it’s difficult to see what change improved the testing metric.
Split your groups equally and randomly: This will help determine the winning email as there is no bias in the testing groups.
Test early and test often for best results: By testing early and often you can continually improve the testing metric over time.
Test as large a sample as you can for more accurate results: If you use a small test group, it’s easy for the metrics to be skewed due to normal variation. Try to include as many recipients as you can in the test.
Trust the data collected, not your gut instinct: If you prefer the blue CTA but the red variation performs better, go with the red one!

A/B testing in email marketing is one of the most common testing methods in email marketing as it’s quick and easy to implement. There are many advantages of A/B testing emails including:

Better understanding of each segment: By continually testing, over time you will begin to see what type of content/format each segment prefers.
Determine the best subject line: Testing two different subject lines for your email on a subset of the audience will help determine which one to use on the larger audience for the best open rate.
Improved content: You can perform small changes for each test, and the end result will help improve the content in your email.
Increased conversion rates and more sales: A/B testing will help to improve your open and click rate. The more users that open your email means the more users that can click the CTA in your email, which gives you the best chance of getting a sale.

Now we will take a look at reporting and some of the metrics you should be paying attention to and how to calculate. After every email send you should be looking at the campaign metrics to see how it performed.

Click-through rate: The CTR is the percentage of recipients who clicked on one or more links within your email. To calculate, you take the number of clicks, divide by the number of delivered emails, and multiply by 100. You can use the total clicks to get the CTR, or use the unique clicks to get the unique CTR.
Conversion rate: The conversion rate is the percentage of recipients who clicked a link and completed the desired action, such as make a purchase. To calculate, take the number of users who completed the desired action, divide by the number of delivered emails, and multiply by 100.

Here are two more metrics you should pay attention to and calculate after each email campaign:

Bounce rate: Bounce rate is the percentage of emails sent that bounced and couldn’t be delivered. To calculate, take the number of bounced emails, divide by the number of emails sent and multiply by 100.
Share/Forward rate: Share/Forward rate is the percentage of recipients who clicked “Share this” button to post to social media. It’s calculated similar to CTR but you only use the number of clicks on the share/forward button. Take the number of clicks, divide by the number of delivered emails, and multiply by 100.

Here are two final metrics you should keep an eye on for your email marketing campaigns:

List Growth: This is the overall growth of your subscriber list. While this is a relatively simple metric, the more subscribers you have the bigger of an audience you have to market to. (As long as you keep your list healthy). To calculate, just count the number of new subscribers you get per week or month.
Overall ROI: The Return On Investment (ROI) of a campaign will show you whether the campaign was worth running. To calculate, take the additional amount of money made from a campaign, subtract the amount invested, divide that by the amount invested, and multiply by 100 to get the percentage return on investment for the campaign.

The metrics we just covered all relate directly to the email but they are not the only metrics you should be monitoring.

It is also important to monitor non-email engagements such as:

Web logins
Ad traffic
Store purchases

The recipient may not have clicked in your email, but because they read it this may have prompted them to login, make a purchase etc.

While the email campaign goals might not be any of the above, they can still contribute.

a/b testing in email marketing

As we saw in the section on campaign delivery, the last step in an email campaign is analyzing your results after you have sent your campaign and collected data. The Mail Chimp reporting feature contains campaign data on clicks, opens and total revenue from the campaign.

Recipients: the number of subscribers the campaign was sent to
Delivered: date and time the campaign completed sending from MailChimp servers
Open rate: percentage of successfully delivered campaigns that have been registered as an open
Click rate: percentage of successfully delivered campaigns registered as a click
Industry opens & clicks: the average percentage of opens or clicks campaigns in the selected industry set for your account

Now tell us your own side of this story.

We would also like to hear what you feel about the topic we discussed today. Your feedback is very important to us. Feel free to drop your comments and recommendations. If you have a contrary opinion, you can drop that too.

You can also joint our Facebook Page CRMNigeria for more updates. You can do that by clicking on the link or searching for our page on Facebook.

You can also joint our WhatsApp Group Here.

Finding the right Long-tail Keywords? Start your journey here.