A/B testing mailings

Available on Starter, Basic, Professional, and Unlimited plans

Test 2 to 4 variants of a subject line or body. FastLinkIt sends each variant to a sample slice (configurable, default 20%), waits a window you choose (1–24 hours), picks the winner by open rate, and ships the winning variant to the remaining recipients automatically.

When to use

• Subject-line tests — same body, different subjects. Cheapest test to run, biggest impact on open rates.
• Body tests — same subject, different bodies. Use to compare CTAs, layouts, length.
• Combined tests — different subject AND body. Use when you have two clearly distinct campaign concepts.

Good test design has 1,000+ recipients per variant in the sample. With sample size 20% and 5,000 total recipients, that's 1,000 per variant in a 2-variant test — comfortable.

Running a test

1. /mailing/compose → toggle A/B test on (top of the message card).
2. The card expands to show controls:
   • Sample size (default 20%) — what percentage of recipients gets the variants
   • Pick winner after — 1h / 2h / 4h / 8h / 12h / 24h
   • Variant selector — segmented button-group with Variant A and B by default
   • Add variant button — extends to up to 4 variants
3. Active variant tab — the segmented control's highlighted button. Subject + body fields edit that variant only. Switching to another variant tab loads its subject + body.
4. Subject and Body fields — show a small badge ("Variant A", "Variant B", etc.) so you always know which one you're editing.
5. Click Send.

What happens at send time

1. FastLinkIt shuffles the recipient list and partitions:
   • Sample slice (e.g. 20%) — split round-robin across variants
   • Rest of list (e.g. 80%) — held with Status=ab_pending
2. The mailing transitions to Status=ab_waiting after the sample sends.
3. AbTestWinnerService (5-minute poll) finds A/B mailings whose SentAt + AbTestWindowMinutes <= now and pick the winner:
   • Highest open rate wins
   • Tie → highest click rate
   • Tie → random
4. The winner is stamped (AbTestWinnerVariantId, AbTestWinnerPickedAt).
5. The held ab_pending recipients are promoted to MailingVariantId=winner, Status=pending and the mailing flips back to Status=queued.
6. The regular sender loop ships the rest of the list with the winning variant.

If sample is 100%, no recipients are held — the test is purely informational and the mailing transitions to sent after the window with the winner stamped.

Reading results

The mailing's stats page (/mailing/{id}/stats) shows:

• Per-variant table: label, subject (or "—" if same as parent), recipients, opens, open rate, clicks, click rate
• Green Winner badge once AbTestWinnerPickedAt is stamped
• Live counts (refreshes every 30s during the window — opens come in throughout the sample period)
• After winner pick: rest-of-list send numbers append to the winning variant's totals

Subject-only vs body-only vs both

Variants can have null Subject or null HtmlBody — both fall back to the parent Mailing's value. So:

• Subject-only test — set Variant B's subject only; leave body null. Both variants ship with the same body, different subjects.
• Body-only test — set Variant B's body only; leave subject null. Both variants ship with the same subject, different bodies.
• Both — set both fields on each variant.

The compose UI manages this for you; null fields show "(uses parent)" in stats reports.

Worked example — subject-line test

Goal: pick the better subject for a launch announcement.

1. Compose → toggle A/B test.
2. Variant A subject: "Your March update". Variant B: "5 things we shipped in March". Body identical (or leave Variant B body null).
3. Sample size: 20%. Pick winner after: 4 hours.
4. Send. 1,000 emails ship (500 per variant against 5,000 total).
5. Four hours later: B wins (34.7% opens vs A's 28.4%). The remaining 4,000 ship with subject B.
6. Total time from send to full delivery: ~4 hours plus the regular send queue.

Related

• Visual email builder — build different bodies per variant in the canvas
• Mailing templates — A/B test two saved templates by setting each variant from a template
• Email Mailing — compose page overview