AB testing

Looking for instructions on how to evaluate your AB testing? Check out our step-by-step guide here.

Exponea offers automatic A/B testing in scenarios, web layers and experiments. A customer is assigned a variant instantly when reaching an AB split node in a scenario or when matching the conditions for displaying a web layer/experiment. You can specify the distribution for the probability of occurrence for each variant.

Automatic AB testing (Optimize automatically)

Exponea allows the usage of AI to determine the best performing variants of the web layer and then preferring these variants over the less performing ones. However, even low performing variants will be still used in a small percentage of cases, so that the AI can continuously test for any changes in performance. In case of such changes, AI changes its preference as well, always choosing the best alternative. In the beginning, all alternatives have the same probability. This is based on the conversion goal that is set in the settings.

This feature will maximize the effect gained from A/B testing.

Calculating statistical significance

You can calculate the performance and statistical significance of your AB test with our AB Test Significance Calculator.

Control group in web layers

The control group by default does not show any variant. If there are any custom conditions specified in the JS code, it is important to create a custom control group that will have the same conditions despite the fact it doesn't show an actual web layer. This is important as otherwise, the compared groups would not be homogenous and the evaluation would be flawed.

Additional technical notes about AB tests

You will not be able to edit any variant that has already been visited by a customer.

  • When going through the same A/B split in scenario, a customer is assigned same variant as before.
  • In web layers, each customer is assigned the variant in event “ab test” and will have the same variant in subsequent visits.
  • It is possible for a customer to be assigned to more variants if she visited the website from different browsers or devices. In that case, the system does not know it is the same customer until she is identified (through a login for example) and hence can assign different variants on visit.
  • For correct A/B test evaluation, all groups should be identical in conditions except for the variant.
  • Web layers offer automatic A/B testing, which allows a preference of the best performing variant, based on the chosen goal.

AB test event attributes

You can read on ab test event attributes in the System events article.

AB testing


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.