Salesforce B2C Commerce 24.7 > Merchandising Your Site > A/B Testing > A/B Testing Examples

A/B Test: Comparing Content Slots

A business user would like to refresh the content on a specific slot on the home page. There are two pieces of creative content to test.

The business user does the following:

Selects site > Merchant Tools > Online Marketing > A/B Tests, and creates a new test with ID Creative-Test-1.
Enables the test and enters an optional description that is used in the test.
Sets the metric to Checkout Rate because the site has been having issues with dropout from the checkout process. The goal is to eliminate that as a variable, although it isn’t clear what effect the creative pieces can have on conversion.
Sets participation to expire per session. (The business user decides that it's okay for the same customer to see different content in different visits. The customer is assigned to a test group again when they return.)
Configures the test to run all day the coming Monday through the end of Wednesday.
Configures the test for Registered Customers.
Creates two test groups, each with 40% allocation, leaving 20% for the control group.
Selects site > Merchant Tools > Online Marketing > Content Slots. The menu creates two slot configurations, one for each content asset representing each creative set of creative content.
Returns to the test and includes the corresponding slot configuration for each test group besides the control.

By Wednesday morning, there is already a statistically significant result, but the business user lets the test run to the end. On Thursday morning, these are the results.

One creative content shows a significantly higher Average Discount Amount With Coupon, but the other shows a significantly higher Checkout Rate, the key metric.

The conclusion is that the higher Average Discount Amount With Coupon is because the first content includes a coupon code and the other doesn't. The business user reports the findings back to the marketing department, who decide to go ahead with a site-wide creative effort based on the winning piece because of the effect on checkouts.