AI Agents · Experiments & A/B

Test variants; keep the winner.

Try more than one version of a campaign — the subject line, the copy, the creative, the offer, the channel or the send time. The platform splits your audience, measures real member response, and names a winner only when a genuine statistical test agrees. And a small holdout proves the lift is real.

Book a Demo Campaign Intelligence →

What you can test

Test the part of the campaign that matters most.

An experiment can be as simple as two subject lines or as meaningful as two different offers. Pick what you want to learn, and let real member response answer the question.

Subject line

Try two ways of opening the conversation and see which one earns the open — without guessing which tone your members prefer.

Copy

Test a short, punchy message against a warmer, longer one, and let real response decide which voice moves people to act.

Creative

Compare hero images, layouts or a bold banner versus a clean look, and keep the design that actually gets clicks.

Offer

Weigh 20% off against a gift-with-purchase, or points versus a coupon — find the incentive members respond to, not the one you assume.

Channel

See whether email, app push or the inbox reaches a given audience best, and lean into the channel that lands.

Send timing

Test a morning send against an evening one to find the moment your members are most likely to respond.

Experiments live inside the normal build — in the campaign canvas, different branches can be delivered differently, so a test is just part of the flow, not a separate tool.

How it works

From two ideas to one proven winner.

You set up the variants; the platform handles the fair split, the honest measurement, and the call on which version won.

Define your variants

Pick what to test — subject, copy, creative, offer, channel or timing — and set up two or more versions of the campaign. It's part of the normal build, right in the canvas.

Split the audience

The platform automatically divides your audience across the variants so each version reaches a fair, comparable slice of members.

Measure real response

As sends go out, real member response is measured — opens, clicks, redemptions — counted honestly, never double-counted or inflated.

Keep the winner

A winner is named only when a genuine statistical test agrees — judged on clicks, with enough responses in every variant to trust the result. Below that bar the read stays honestly directional, so you scale what works instead of calling it early on a hunch.

The experiment

Two variants, one confident winner.

Variant A

Winner

Members responded more

Variant B

Fewer responses

The winner is chosen by a real statistical test on clicks, not a hunch

No winner is called until every variant has enough response to trust the math

Scale the version that works — the rest of your audience gets the best variant

Prove it's real

A holdout proves the win is real, not noise.

Picking a winner tells you which variant did better. A holdout tells you whether the campaign did anything at all — because some members buy anyway.

A small group, deliberately left untouched

A holdout is a slice of your audience that gets nothing — no send, no offer. If the members who received the campaign respond more than the untouched group, the difference is incremental lift: real value the campaign caused, not sales that would have happened anyway.

The same rigor behind SoClaw's win-back proof

This is exactly how our autonomous win-back agent proves its worth: it only claims the lift it can show against an untouched control. So the number you report to the business is honest — it's the value you added, measured the same trustworthy way.

See how SoClaw proves win-back lift →

For recurring campaigns

It keeps optimizing while the test runs.

One-time sends pick a winner and stop. Recurring campaigns — automations and journeys — can do more: switch on “Auto-optimize traffic over time”and, each hour, the platform shifts more of your sends toward the variant that's getting more clicks — while the test keeps running.

Opt-in, and only for recurring sends

It's a setting you turn on per experiment, for automations and journeys. One-time sends are not affected, and your configured split stays untouched as the baseline.

A comeback is never impossible

Every variant always keeps at least 10% of the traffic, so a slow starter can still prove itself. The platform spends more on what's working without ever silencing an alternative.

It waits until the comparison is fair

Traffic only starts shifting once every variant has enough deliveries to compare honestly — about 100 each. Until then, the split stays even.

Every shift is logged and explained

Each hourly reallocation is recorded in a decision log with plain-language reasoning, so you can audit exactly why the traffic looks the way it does.

Honest by design: this never declares a statistical winner — it simply spends more of your sends on what's working while the test keeps running.

Every experiment feeds the bigger picture.

Winners, response and lift roll up into honest measurement that makes your next campaign smarter. See the full measurement story on Campaign Intelligence.

Explore Campaign Intelligence →

Variant response

The platform measures real member response and keeps the winner once there's enough to be confident.

Test the version that wins — and prove the lift is real.

Book a demo

400M+

50+

12+