Salesforce B2C Commerce 24.7 > Merchandising Your Site > A/B Testing > A/B Testing Examples

A/B Test: Getting Complicated

A merchant wants to simultaneously test changes to the order of four recommended products and the background image for a home page slot. There is a limited time to perform the test and can't run the tests one after the other.

The merchant does the following on the Staging instance:

  1. Selects site > Merchant Tools > Online Marketing > A/B Tests, and creates an A/B test called recommended-products.
  2. Enables the test and enters an optional description that is for usage in the testing.
  3. Includes an email address to be notified when there are statistically significant results for the key metric.
  4. Selects Adds to Baskets as the key metric.
  5. Configures the test to:
    1. Expire along with sessions.
    2. Run for 1 week.
    3. Be available for the Everyone customer group.
  6. Creates four segments plus a control group and allocates 20% of the participants to each.
  7. Selects site > Merchant Tools > Online Marketing > Content Slots and opens the global content slot, home-right. They then create four slot configurations, one for each order of recommended products and background image.
    ID Content Type Content Template
    bgImage-1 Content Asset Image-1 slots/product/verticalcarousel.isml
    bgImage-2 Content Asset Image-1 slots/product/verticalcarousel.isml
    Product-1 Product product1, product2, product3, product4 slots/product/verticalcarousel.isml
    Product-2 Product product1, product2, product3, product4 slots/product/verticalcarousel.isml
  8. Returns to the test and assigns the corresponding slot configurations to the test segments. Each segment has the image applying first and the product order second, as follows:
    Segment ID Content Asset Slot Product Slot
    recommended-products-1 bgImage-1 Product-1
    recommended-products-2 bgImage-1 Product-2
    recommended-products-3 bgImage-2 Product-1
    recommended-products-4 bgImage-2 Product-2

The test is replicated to production and runs for the week. These are the results:

  • One recommended order of products leads to more Adds to Baskets than the other.
  • One background image leads to more Adds to Baskets than the other.

The combination of the better product order and background image shows a statistically significant difference in Adds to Baskets over the other combinations. Thus, the winning combination is used for an upcoming change to the homepage slot content.

Infocenter Retirement: On June 30, 2023, the Infocenter was retired, and documentation currently hosted on the Infocenter will be published to Salesforce Help, Commerce Cloud Developer Center, and Salesforce B2C Commerce Developer Documentation Resources. For more information, see the release note.