[deliver]
Deliver article · 2026-11-17 · Charlotte Rodrigues

Klaviyo Campaign A/B Test Before BFCM: Rules and Duration

A campaign A/B test in Klaviyo is prepared like a send: audience, variations, metric, test size, duration. In our view, before BFCM, one badly filled box can be enough to produce an unusable result or a send that is hard to correct. This article covers what the help pages let you test, their rules on size, duration and reading, then offers a decision framework for the window. Black Friday falls on Friday, November 27, 2026 and Cyber Monday on Monday, November 30.

TL;DR. An email campaign can test subject lines, message content or send times, with the winning metric Open rate, Click rate or Placed order rate. A send-time test goes to 100% of the audience: no winner is sent afterwards. A result is labeled statistically significant when the win probability is above 90% and each variation has more than 50 successful deliveries. In our view, whatever should inform BFCM is tested and read before the window.

Want an outside view before you build? We offer a Q4 readiness audit: 30 minutes, free, no strings attached.

This article is part of our Klaviyo BFCM guide. For the general testing method, see our article on Klaviyo A/B testing.

What Klaviyo lets you test in an email campaign

The reference page lists three families: subject lines, message content and send times (Klaviyo, how to A/B test an email campaign). You create the test from Campaigns > Create campaign, choosing Single channel then Email, then Create A/B test above the subject line field. Klaviyo creates a second variation identical to the first.

In A/B test setup, you choose Test content for content and subject lines, or Test send times for send times. A content test changes a single element: if you edit the subject line, you change nothing else. The page recommends two variations, and the best practices page asks you not to exceed four variations of one variable per test.

An AI-generated subject line variation also exists, on free and paid accounts: under the subject line field, you choose personalization or not, short or long, question or statement, emoji or not. Variation A keeps the original subject line, variation B gets the generated one.

The choice of strategy (Winning variation or Personalized variations for each recipient) is only offered to accounts with at least 400,000 total profiles. Below that, Klaviyo applies the winning variation: a single winner goes to the rest of the recipients after the test period. The rest of this article assumes that strategy.

Choosing the winning metric

The page offers three metrics, each with a recommended use.

Metric Use recommended by the page Limit to know
Open rate Subject line, preview text, sender name and email address Counted opens include those from Apple Mail Privacy Protection (MPP)
Click rate Email content, such as button size or color No limit flagged
Placed order rate Content, for example displaying best sellers or new items Only for accounts with a Placed Order metric, unavailable with personalized variations

MPP prefetches the tracking pixel, which can inflate opens. An Open rate test counts recorded opens, MPP included: the winner may reflect automatic opens as much as human engagement. If you test regularly and more than about 45% of opens come from Apple Mail, Klaviyo suggests a custom report with an MPP property. The statistical significance page invites you to consider Click rate when MPP makes Open rate hard to read. See our article on Apple Mail Privacy Protection.

In our view, for a test meant to steer BFCM, Click rate or Placed order rate speak better than an open as soon as the test concerns something other than the subject line.

Test size and duration

For a content test, Klaviyo recommends a test size based on the audience and a duration based on the winning metric. A slider sets the size; below 100%, you can also adjust the duration. The page gives an example: with a 40% test size and a 6-hour duration, 20% of recipients get variation A, 20% get B, a winner is chosen after 6 hours and the rest of the recipients get the winning variation.

The email page provides no numeric duration to adopt. The SMS page gives one, for content tests below 100% with Placed order rate: at least 24 hours are recommended (Klaviyo, how to A/B test SMS campaigns). For email, we start, in our view, from the duration Klaviyo proposes.

The best practices page asks you to allow enough time to collect results from a larger audience, then check significance (Klaviyo, best practices for A/B testing reference).

The rules that weigh on a BFCM calendar

A send-time test has no winner to send. It goes to 100% of recipients, with the same content and subject line in both variations, each with its own date and time in the account's time zone. In our view, this is a cost to weigh on a segment you want to serve as well as possible.

The recipient's time zone requires a 100% test. For a content test to go out at each recipient's local time, the test size must be 100% and you choose Recipient's Local Time Zone when scheduling. There is then no winner sent afterwards.

The winner is chosen by hand provided recipients remain. To end a running content test, the test size must be below 100%. In the A/B test results tab, the variation's options menu offers Choose as winner: the test stops and the sending of that variation to the rest of the audience starts immediately.

The best practices page advises against editing a running test. For a campaign already sending, if the content must change, you cancel the remaining send. Cancellation does not recall emails already delivered. You then clone the canceled campaign, or create a new one, then set up a new test.

Reading the result without over-reading it

Results are in the A/B test results tab of the report (Klaviyo, how to review your A/B test results for campaigns): winning variation, status, Win probability, lift and winning metric. Lift compares the gap with the other variation's rate. The page's example: A at 5% clicks and B at 4% give a one-point gap, that is (5% minus 4%) divided by 4%, or 25% lift.

When the test group is smaller than the audience, Test period results shows the group's interactions during the test period, and Continuous results the later interactions of the same group. The leading variation can change between the two. The significance page recommends reading Test period results to judge the data collected before the winner was chosen (Klaviyo, understanding statistical significance in Klaviyo campaigns).

The four categories, for email:

  1. Statistically significant: win probability above 90% and more than 50 successful deliveries per variation. The result does not assure the same effect with another audience, another message or another send time.
  2. Promising: same minimums, win probability above 75% and at most 90%. No significance label; the page recommends repeating the test.
  3. Not statistically significant: win probability of 75% or less with more than 1,800 deliveries per variation, or win probability below 60% with less than 4 percentage points between the top two variations, provided each variation has more than 50 deliveries. The page recommends retesting two or three times, then moving on to another subject.
  4. Inconclusive: a variation at 50 deliveries or fewer, or a result that fits no other category. The page suggests widening the audience of the next tests.

In our view: a decision framework before the BFCM window

The readings that follow are Deliver's, not Klaviyo's.

The first question concerns the decision the test must inform: which offer leads the email, which subject line for the main send, which send time for the reminder. The best practices page also asks you to write a hypothesis before building the test.

Second trade-off, the window itself. The significance page advises against retesting near a holiday weekend, when your audience is much more likely to be looking for your emails. We read the Black Friday and Cyber Monday weekend as falling under that case: a result obtained those days will transpose poorly. This is a Deliver reading that the documentation does not state. It leads to launching tests before the window and sending, during the window, the variations already settled.

Third trade-off, repetition. A promising result calls for a second test, a non-significant result for two or three. This dictates the start date: the first test must go out early enough for its repetition to fit before the window. If the calendar does not allow it, one well-chosen test is better than a series read in a hurry.

Fourth trade-off, the audience. The page's thresholds (more than 50 deliveries per variation, more than 1,800 to conclude there is no difference) show that a small segment will not separate anything. On a small segment, we prefer to apply the most cautious choice rather than read an "Inconclusive" result. Our article on BFCM campaign audiences helps size those segments, and our BFCM email send calendar helps place the test sends.

Ordered checklist

These points are Deliver advice, based on the rules above.

  1. Write the decision to inform and the hypothesis, in one sentence.
  2. Pick a single variable and the test type: Test content or Test send times.
  3. Pick the winning metric, taking MPP into account for Open rate, and check that a Placed Order metric exists before choosing Placed order rate.
  4. Set the test size below 100% if a winner must be sent afterwards.
  5. Do not edit the running test: if a correction is needed, cancel the remaining send, clone, relaunch.
  6. Read Test period results, then the significance category, and decide: apply, repeat, retest or drop the subject.
  7. Find past tests in Analytics > Experiments, which gathers campaign and flow tests and shows the last 90 days by default (Klaviyo, understanding the experiments section).

FAQ

Which elements of an email campaign can you test in Klaviyo?

Subject lines, message content and send times, one element at a time.

What duration should you choose for an email campaign test?

The email help page gives no numeric duration: Klaviyo recommends a duration based on the winning metric, adjustable when the test size is below 100%. For an SMS with Placed order rate, the page recommends at least 24 hours.

Does a send-time test send a winner afterwards?

No. It goes to 100% of recipients, so nobody remains to receive a winning variation.

When is a result statistically significant?

When the win probability exceeds 90% and each variation has more than 50 successful deliveries for email.

Can you edit a running test?

The best practices page advises against it: for a campaign already sending, you cancel the remaining send then clone the campaign or create a new one.

A question about your flows? You can talk directly with the Deliver team.

Provenance and verification

Faithful English translation of the French source. Each technical claim was compared on 2026-11-14 with the raw text of the six Klaviyo pages listed in sources, opened through the help center article body, read in full and dated October 22, 2026 (update date shown by the help center): elements testable in an email campaign, AI-generated variation, test strategy and the 400,000-profile threshold, winning metrics and the note on Apple Mail Privacy Protection, test size and duration, send-time test at 100% of the audience, manual choice of the winner, results and lift calculation, significance categories and thresholds, best practices (one variable, no more than four variations, do not edit a live test), the Experiments section, SMS-specific rules. The email help page gives no numeric test duration: the article does not put one forward. The decision framework, the checklist order and the trade-offs are Deliver readings, flagged with in our view, with no figures or real account examples.

Sources checked on
Reviewed by
Claude (Claude Code session, November 14, 2026) English localization of the French source, with every vendor-documentation claim checked against the Klaviyo help pages listed in sources, reopened in raw form the same day.
AI assistance
Yes
Sources
  1. help.klaviyo.com/hc/en-us/articles/115005228148
  2. help.klaviyo.com/hc/en-us/articles/360052794352
  3. help.klaviyo.com/hc/en-us/articles/360052793012
  4. help.klaviyo.com/hc/en-us/articles/360045012632
  5. help.klaviyo.com/hc/en-us/articles/18631098703515
  6. help.klaviyo.com/hc/en-us/articles/4406796460443
Going further · Deliver sets up and runs Klaviyo for e-commerce brands: flows, campaigns and deliverability. See our Klaviyo agency page, or request a Klaviyo audit.
CR
Charlotte Rodrigues · CRM Lead at Deliver. Questions about this article? charlotte@agence-deliver.com

Want to apply this to your stack?

Spend 30 minutes with Charlotte to review your CRM setup, size the opportunity and leave with a practical action plan.

Book a 30-minute call →