A/B Test Segments

You can distribute test segments across a maximum of 12 simultaneous A/B tests, as long as the total segments for all active A/B tests doesn't exceed 12. Each A/B test can have up to five unique test segments, one of which is always the control group. Each test segment targets a portion of the total test participants. Participation in the control group results in a storefront experience that is the same as if the A/B test did not exist.

Participation in one of the other segments results in a modified storefront experience that represents one or more changes to the control group experience. These changes, only visible to a particular segment, can be from one or more of these built-in Salesforce B2C Commerce A/B testing experiences, and script customizations.

  • Promotions: what bargains give you the best response?
  • Content slots: how effective are your graphics and content?
  • Sorting rules: do search results make a difference?
Tip:

Consider testing, for example, one experience that includes a new slot content and promotion against another experience that includes another, different slot content and experience.

When learning to use B2C Commerce A/B testing, avoid using existing promotions, slot content, and sorting rules because they’re included in your controls. Instead, create an experience to test, possibly against the controls.

Note: A customer can only participate in one A/B test at a time.

Test Group Effects on the Storefront Experience

Each A/B test group can have effects that automatically change the experience for its participants:

Within the test... Effect...
Include a promotion. Makes a promotion available that might otherwise not be. When you include a promotion in a test, you can specify a rank. The rank you assign doesn't change the rules governing the exclusivity, qualifying products, or the like, of a promotion. These rules still apply, and might cause another promotion to take precedence over one included for an A/B test.

Each promotion can only be assigned to each A/B test segment once. Assigned promotions run for the length of the test.

Include a slot configuration. Makes a slot configuration available that might otherwise not be. When you include a slot configuration in a test, a rank is provided. Each slot configuration can only be assigned to each A/B test group once. Slot configurations run for the length of the test.
Override a sorting rule. Overrides the default sorting rule for a category that might otherwise have been applied.

When you assign a category sorting rule to an A/B test, it replaces what you selected on the Sorting Rule tab on the Category page. In situations where a category sorting rule is ignored, it's ignored whether the rule was selected via the Sorting Rule tab or an A/B test. For example, an explicit sorting rule is set using the ProductSearchModel API or the Search pipelet for a search refined by category. That explicit rule is used and the default rule is ignored, regardless of whether the default was set on the Category page or imposed as an effect for a test segment.