What Should Ecommerce Email Marketers A/B Test?

There are dozens of things you can A/B test in an ecommerce email: subject lines, CTA copy, images, layout, send time, email length and a whole lot more.

That does not mean all of them deserve the same amount of attention.

If a campaign is going to the wrong audience, promoting the wrong product, or giving away a discount the customer did not need, changing the button copy is unlikely to make much difference.

A better testing program starts with the decisions that have the biggest impact on whether someone buys, then moves into the details of the email itself.

Test which customers should receive the campaign

Most email A/B testing starts after the audience has already been chosen, while the audience itself is one of the biggest variables.

For example, you could test:

• all engaged customers versus customers with recent category interest
• all repeat buyers versus customers approaching their normal repurchase window
• a broad customer segment versus a higher-intent group
• one lifecycle segment versus another

This helps answer a more important question than whether one subject line beat another:

Did the campaign work because of the email, or because it reached customers who actually had a reason to buy?

For ecommerce brands, audience testing can often teach you more than testing two versions of the same creative against the same list.

Test which products or categories you promote

The next major decision is what you are asking the customer to buy. A campaign can underperform even when the copy and design are good if the product selection is weak.

Useful tests include:

• one hero product versus a category
• best sellers versus products based on customer interest
• new products versus familiar products
• one product versus a small collection
• different product selections for different customer groups

This matters more for brands with larger catalogs, where the same audience responds very differently depending on what is featured. Instead of asking which email version performed better, test whether a different product choice changes the result.

Test whether the customer needs an offer

Offers have a much bigger impact on revenue and margin than small creative changes.

Make sure to test:

• no discount versus discount
• free shipping versus percentage discount
• threshold offer versus sitewide discount
• bundle versus individual-product offer
• different discount depths where the economics support it

But conversion should not be the only metric.

A discounted version may generate more orders while producing worse margin. A 15% discount may lift conversion slightly, but many of those customers may have purchased at full price anyway.

Look at revenue, margin, average order value, and repeat purchase where relevant.

The test should answer whether the incentive actually improved the commercial outcome, not simply whether more people used it.

Test when the campaign should be sent

Send-time testing is usually reduced to questions like Tuesday morning versus Thursday afternoon.

For ecommerce marketing, timing can be much more useful than that.

You can test:

• day of week
• time of day
• time since the customer last purchased
• time since recent browsing activity
• time since cart activity
• timing around the customer’s normal repurchase window

A customer who usually reorders every 45 days may respond differently on day 40 than on day 20. Similarly, someone who browsed a category yesterday may be more receptive than someone who showed the same interest six weeks ago.

So the better question is not only when your list tends to open emails. It is also when a particular customer is most likely to have a reason to buy.

Test email structure and design

After the larger campaign decisions are in place, move into execution.

Useful tests include:

• one hero product versus multiple products
• short email versus longer email
• product-first versus copy-first
• image-heavy versus simpler layout
• social proof near the top versus later in the email
• one CTA versus several relevant CTAs

The test should still have a reason behind it.

If customers are not reaching the products lower in the email, shortening the layout may be worth testing. Do not change the layout simply because the email builder makes it easy to create another version. There needs to be a clear reason for this decision.

Test subject lines and preview text

Subject lines matter because they determine whether many customers see the rest of the email at all.

Test the following:

• product-led versus benefit-led
• direct versus curiosity-driven
• offer in the subject line versus offer revealed in the email
• personalized versus non-personalized
• urgency versus neutral wording when urgency is real

A campaign with a stronger open rate is not necessarily the better campaign if the extra opens do not turn into more clicks, purchases, or revenue.

Test what the CTA asks the customer to do

CTA testing is useful when the wording or destination changes what the customer understands they should do.

For example: Shop the collection versus Shop running shoes

Or: Learn more versus See how it works

Those tests can tell you whether a more specific next step helps.

Testing small cosmetic changes such as button color is much less useful unless there is a clear reason to believe visibility is the problem. The test should teach you something about the customer’s decision, not simply tell you which visual variation won.

Do not assume one winner applies to every customer

Traditional A/B testing usually ends with one question:

Which version won?

For ecommerce marketers, another question can be just as useful:

Who did each version work for?

1. A discount-led campaign may perform better overall, while the full-price version works better for loyal customers.
2. A product-led email may work better for customers who already know the category, while an educational version works better for first-time buyers.
3. One customer group may respond to a single-product email, while another prefers a broader category selection.

That means the result does not always have to become:

Version A won, so everyone gets Version A.

It should instead become:

Version A works better for this customer group, while Version B works better for another.

That is where A/B testing starts becoming useful for future personalization rather than only improving one campaign.

Pick the metric based on what you are testing

Not every test should be judged using the same metric.

If you are testing an offer, open rate tells you very little about whether the offer was commercially better.

If you are testing the audience, revenue per recipient may tell you more than total revenue because the segment sizes may be different.

The metric should match the decision you are trying to improve.

Test one meaningful hypothesis at a time

Do not change the audience, offer, subject line, design, and CTA at the same time and then try to work out why one version won.

Start with a clear hypothesis.

For example:

Customers who have recently browsed the category will generate more revenue per recipient than the broader engaged audience.

Or:

Free shipping will produce a better commercial result than 15% off for this customer group.

Then change the variable that tests that idea. Once you have a result, carry that learning into the next campaign.

Over time, the goal is not to collect a long list of winning subject lines or CTA variants. It is to understand which audiences, products, offers, timing, messages, and creative choices work under different conditions.

Ecommerce email marketers can test almost anything. The better question is what is worth testing first.

Start with the decisions that have the biggest effect on the campaign: who should receive it, what you should promote, whether an offer is needed, when the message should arrive, and how the campaign should be positioned.

Then optimize the subject line, design, CTA, and smaller execution details.

The most useful testing program does more than find a winning email. It helps you make better marketing decisions the next time around.

Nexie is a retention marketing agent for e-commerce brands. It reads your customer data across engagement, recency, category interest and lifetime value together, surfaces the audiences worth campaigning to, and builds those campaigns for your team to review and approve, inside the guardrails and marketing policies you set. See how it works →