How to create an A/B split test campaign

Don't know which version of MailerLite you're using? Click here to find out!

A/B testing, also known as email split testing, is sending 2 different versions of your email to a sample of your email list to see which version performs better. The version that receives the most engagement is then sent out to the rest of your list. 

There are three different types of A/B test options to choose from: 

Subject line A/B - Testing out language, tone, length, personalization, emojis and more.

Sender A/B - The sender name helps subscribers recognize your brand. Test out the sender name and email address to see which variation encourages the most opens.

Content A/B - You can test variations of all aspects of your content including headlines, images, CTAs, and more to gain valuable insights into what subscribers like and optimize click rates.

Sending time - Test different sending times among subscribers, to see which offers the most engagement.

Once you have decided on a test type, you can create the two different versions of your email using our campaign editors. MailerLite will run the test by sending version A to a small portion of your subscribers list, and version B to the other portion. The winning version that performs the best will be sent out to the remaining subscribers.

You can choose how you would like the winning campaign to be decided. The options are:

Clicks - Winner will be chosen based on which version had the higher click rate after the testing period ends.

Opens - Winner will be chosen based on which version had the higher open rate after the testing period ends

Then, you choose a time for the testing period, which is how long the system will collect stats before the winner is decided.

Note: Keep in mind that the winning campaign is decided and sent out based on the stats at the end of the testing period. However, the stats continue to be collected after this, so it is possible that some time after the testing period has ended, the stats appear different. 

A/B split testing without a winner

It’s also an option to test your campaigns on the entire selected recipient list and use the results to guide future campaigns, rather than sending the winning version mid-campaign. 

A checkbox - Run A/B test on the entire list - is available when you set up your A/B campaign.

When selected:

Versions A and B will be sent to equal splits of the entire selected list.

• The slider for the test group size is hidden, as it’s not applicable.

• No “winner” is automatically selected during the campaign.

After the campaign is sent, you’ll see both versions’ metrics in the report (opens, clicks, etc.). You can use these insights to refine future campaigns.

To create an A/B split campaign:

  1. Navigate to the Campaigns tab and click Create campaign.

  2. Under Type, select A/B split campaign.

3. Under Campaign details, enter an internal campaign name for your campaign. Choose what you would like to test in this campaign between Email Subject, From name, or Email content.

4. Once you reach the Recipients step, choose who will receive the campaign. For A/B testing, at least 4 subscribers must be included. Click Next: A/B Settings.

5. Select your Tracking options and campaign Language.

6. Click Next: Content. First, you will be prompted to create version A of your campaign, followed by version B.  

7. Decide on test group size or opt to test the entire list:

  • If the option Run A/B test on the entire list is unchecked, use the slider to determine what portion of your list receives versions A and B, and how large the remaining group is that will receive the winning version.

  • If the option Run A/B test on the entire list is checked, both versions will be sent to all recipients in two equal splits, with no mid-campaign winner determined.

Once you have determined the test group, you can decide if you would like the winner to be chosen based on the highest click rate or open rate, as well as the time period of the testing. 

Review your settings on the next step, then you are ready to schedule your campaign.

For a visual walkthrough, check out our video tutorial on how to set up an A/B split campaign. Need inspiration? Check out our blog on 7 ways to optimize your lead generation using A/B testing.

A/B testing, also known as email split testing, is sending 2 different versions of your email to a sample of your email list to see which version performs better. The version that receives the most engagement is then sent to the rest of your list.

There are three different types of A/B test options to choose from: 

Subject line A/B - Testing out language, tone, length, personalization, emojis and more.

Sender A/B - The sender name helps subscribers recognize your brand. Test out the sender name and email address to see which variation encourages the most opens.

Content A/B - You can test variations of all aspects of your content including headlines, images, CTAs, and more to gain valuable insights into what subscribers like and optimize click rates.

Once you have decided on a test type, you can create the two different versions of your email using our campaign editors. Mailerlite will run the test by sending version A to a small portion of subscribers list, and version B to the other portion. The winning version will be sent out to the remaining subscribers.

You can choose how you would like the winning campaign to be decided. The options are:

Clicks - Winner will be chosen based on which version had the higher click rate after the testing period ends.

Opens - Winner will be chosen based on which version had the higher open rate after the testing period ends

Then, you choose a time for the testing period, which is how long the system will collect stats before the winner is decided.

Note: Keep in mind that the winner is decided and sent out based on the stats seen at the end of the testing period. However, the stats continue to be collected after this, so it is possible that some time after the testing period has ended, the stats appear different

To create an A/B split campaign:

  1. Navigate to the Campaigns tab and click Create campaign.

  2. Choose the A/B split campaign tab.

  3. Choose what you would like to test between Email Subject, From name, or Email content

  1. Select your Language and Tracking options.

  2. Click Next: Content. First, you will be prompted to create version A of your campaign, followed by version B. 

Once you reach the Recipients step, choose who will receive the campaign. For A/B testing, at least 4 subscribers must be included. Click Next: A/B Settings.

Next, choose the portion of your list that will receive the fist version, the second version, and lastly the “winner” between the two.

Once you have determined the test group, you can decide if you would like the winner to be chosen based on the highest click rate or open rate, as well as the time period of the testing. 

Review your settings on the next step, then you are ready to schedule your campaign.

For a visual walkthrough, check out our video tutorial on how to set up an A/B split campaign. Need inspiration? Check out our blog on 7 ways to optimize your lead generation using A/B testing.

Last time edited: Jan 22, 2024