A/B test a flow
Test a second version of a flow against the current one, split checkouts 50/50, and keep the version that earns more.
Last updated
An A/B test runs a second version of a flow, Version B, against the current one, Version A. Each checkout that matches the flow gets one of the two, 50/50, and the test page shows which version earns more per checkout. When you end the test, the version you keep becomes the flow.
Both versions share the flow's name, time limit and conditions. What differs is under Customer journey: the products, discounts, shipping, texts and design of the post-purchase screens and of the thank-you page. On a Collection screen, Version B can use another collection or order, for example Best sellers against Collection order; see Offer products from a collection.
Create a test
Save the flow
Version B starts from the saved flow, so save your changes first. An archived flow has to be published before you can test it.
Open Advanced
In the flow's editor, scroll to Advanced at the foot of the page and click Create A/B test in the A/B test row.
Choose how to set up Version B
Set up manually, selected by default: Version B starts as a copy of Version A, and you make the changes yourself.
Describe it: describe your test in words under What should Version B change?, for example "Offer the vitamin C serum instead" or "Show the Summer collection, best sellers first". AI drafts Version B from your description in up to a minute. It can change products, discounts, shipping, texts and layout, and switch a screen to a collection, Buy again or EliteAI™. It doesn't add or remove screens.
Get ideas: AI suggests three to six tests for this flow, each marked Product, Discount, Layout or Text. It looks at the flow's past tests too, so it doesn't suggest one that already gave a clear answer. Click an idea to put it in Describe it, then change it if you like. Get other ideas asks again.
Create it
Click Create Version B. The A/B test card appears under Customer journey, and the editor switches to Version B.
When AI drafts Version B, a banner lists what it changed, and the changes are unsaved until you save them. To find a product or collection, AI searches your store and uses only what it finds there. If it can't find what you asked for, it leaves that screen as in Version A and the banner says why. A collection with no products is still used, with a warning. An EliteAI™ engine that hasn't trained yet trains when the test starts.
Use the Version A | Version B switch in the A/B test card to choose which version you're editing. While you edit Version B, the Products and Sections and style cards say Editing Version B. Save from the save bar as usual.
Start the test
Click Start test in the A/B test card, or on the test's own page. If you have unsaved changes, or the flow is unpublished, the app asks first: Save and start, Publish and start or Save, publish and start. Starting an unpublished flow's test publishes the flow.
A couple of minutes after you start, each checkout that matches the flow's conditions gets Version A or Version B, 50/50. If a buyer's version has nothing to offer them on its Offer 1, for example because every product is already in their order, they see the other version, and that checkout is left out of the comparison. While a test is open, the flow's row on Flows & modules has a badge with the test's name that opens it, and once it runs, the editor's status badge reads Published · A/B test live.
In a flow with a thank-you page offer, the thank-you page is part of the test: a checkout that saw Version A after payment also gets Version A's thank-you page. While a test is open, you can't add or remove Offer 1 or the thank-you page. If the two versions' Only show if post-purchase didn't show switches differ, Version B's card lists it under Changed from Version A:.
Read the results
Click View details in the A/B test card to open the test's page. Its title reads A/B test: and the test's name, or just the name when it already starts with "A/B test", like A/B test 1. It has:
- Test name and Note, for you only, for example what you expect to learn.
- A card per version, with Edit A and Edit B to open it in the editor. Version A's card gives an Overview: of its offers, and Version B's lists what Changed from Version A:.
- Results, per version: Shown checkouts, Not shown, Accept rate and Gross revenue per checkout, with Refresh to update them. Gross revenue includes tax and added shipping.
- Thank-you page, when the flow has one: per version, Shown, Added, Paid, Conversion (paid out of shown) and Gross revenue per shown. A thank-you page addition counts once the buyer has paid for it.
Under the results, a hint compares the versions on gross revenue per checkout that saw the post-purchase offer. It doesn't read the thank-you page's numbers:
| The hint says | When |
|---|---|
| Not enough data yet | Before 7 days and 100 shown checkouts per version |
| ... is likely better | After that, once one version leads with at least a 95% chance |
| ... is the winner | After 14 days, 300 shown checkouts and 20 accepting checkouts per version, with at least a 95% chance |
| No clear winner yet | Neither version leads with a 95% chance |
The hint is a guide, not proof. Give a test time: checking it every day and stopping at the first good-looking number makes a wrong call more likely.
End the test
On the test's page, click End test, keep A or End test, keep B, then confirm. The version you keep becomes the flow. The other is archived: buyers already viewing it can still finish, and you can still view it under Past tests. This can't be undone.
To drop a test that hasn't started, click Discard test on its page. Version B and its changes are deleted, with the test's name and note, and Version A stays as it is.
A flow has one open test at a time. Ended tests stay under Past tests on the test page, with their results.
A/B tests in Analytics
On the Analytics page, a flow with a running test has an A/B test active badge in the Detailed breakdown that opens the test. See Read your analytics.
Still stuck?
Chat with us and we'll help you get your offer live. Or email contact@elitecart.app.