Historical Reports: A/B Testing

This method of A/B testing has been deprecated.

A/B testing is a method by which you can provide different option paths to random segments of your visitors so that you can observe the behavior of each segment. You can use this method to test the effectiveness of a certain business logic path before implementing it for all visitors. For example, you might want to test whether visitors are more likely to purchase a product when they are offered a 10% discount or a free bonus.

The A/B testing report in the Analytics module provides you with an overview of the A/B test results for a designated time period when using the ABCTest pipelet. (See the API documentation for details.) The information this report provides includes group segmentation and conversion rate. These A/B testing reports are run automatically via Business Manager. From a business perspective, the information provided in this report can help you compare the effectiveness of each option path and adjust your marketing plan accordingly.

In the Historical Reports module, you can filter the data for a certain time period by defining the start and end dates for the report by using the calendar box. Depending on the report, you can use the tabs to view the data for the selected period in different time increments (for example, daily, weekly, monthly, quarterly, and yearly). The report shows the total number of visits in the time period and the total number of visits that participated in the A/B test. Among the numbers in the detailed list, you can see the segmentation of the visits (in total number and percentage) as distributed across the different test paths.

For the Analytics reports version of A/B testing, each of the following statistics appear for each testing segment (A and B), as well as the total for both (visits created by robots are not included):

• Total Number of Visits (#)
• Visits that included this test (#, %)
• Visit Segmentation (#, %)
• Visits with Carts (%)
• Visits with Checkout (%)
• Visits with Orders (%)
• Cart Conversion (%)
• Checkout Conversion (%)
• Order Conversion (%)
• Total Carts (#, %)
• Total Checkouts(#, %)
• Total Orders (#, %)
• Carts to Orders (%)
• Checkouts to Orders (%)
• Average Revenue per order (currency)
• Average Revenue per Visit (currency)
• Average Orders per Cart (#)
• Average Orders per Visit (#)
• Average Items per Order (#)
• Average Registrations per Visit (#)
• Average new Customers per visit (#)
• Average Visit Duration (h:m:s)