A/B testing

After you set up your A/B test in Sitefinity CMS, you track and measure the results in the automatically generated reports in Sitefinity Insight. After you start the A/B test and once any of the page variations or the original page is visited, Insight collects the data and displays detailed reports about the current state of test results:

• Number of visitors exposed to the test and to each of the page variations
• Number of goal completions and the goal completion rate for each of the page variations
• Improvement in conversion rate of each variation
• Statistical significance
• Winner variation, if any

For details about A/B tests and how to manage them in Sitefinity CMS, see Work with A/B tests.

IMPORTANT Your connection to Sitefinity CMS must be properly configured, so that Sitefinity Insight collects data. If connection is lost, the A/B test may still be going on, but Sitefinity Insight does not track and accumulate data. In this case, you are also not able to manage the A/B test. For more information, see Troubleshoot connection issues.

To see a list of all A/B tests reports, open the A/B tests page.
Tests are ordered by the date when the reports for the respective test were first generated. That is, when any of the page variations for a specific test was visited and data accumulated.

You can choose whether to see all tests or test with one of the following statuses:

• Active
  The test is started and is accumulating data.
• Stopped
  The test is on hold and no new data is accumulated at the moment.
  Once you resume the test, Sitefinity Insight continues to collect data.
• Ended
  There is a page variation already published as default content for the page. You can still examine the data accumulated for this test up until it ended.

NOTE: You can also access an A/B test for a specific page from Sitefinity CMS.
Next to each page, which you have set up an A/B test for, click the Results link to go directly to the respective A/B report.

For each A/B test, you see the following summary:

• Whether there is winner page variation
• The date when the test started
• For how many days the test is active

  NOTE: In case you stopped the test for some time, this metric displays the sum of all days during which the test was active – before you stopped the test and after you resumed it.

• Total visits for all page variations
• Target audience
  You can see this field only if you have created the A/B test with visitor segments.

The Goal completion rate graph visualizes how goal completion rate changes over time for each page variation. Hovering over each point on the graph drills into the details of the rate for a specific date. In addition, if you added more than one goal, you can select the goal whose completion rates you want to analyze.

A/B test results give you the key metrics you need to decide which page variation to use as default. You may use the Winner variation, if there is a winner, or any other variation, based on what you aim at with your scenario.

Following is a list with all metrics for each variation and detailed description.