How to Run A/B Testing and Optimize Campaigns with a CDP

What types of elements can be tested in A/B experiments?

Using Sitefinity CMS and Sitefinity Insight, you can test various elements in A/B experiments by creating different page variations. This allows you to determine which variations perform better in terms of user engagement and conversion rates.

How does Sitefinity Insight determine statistical significance in A/B tests?

Sitefinity Insight determines statistical significance in A/B tests by analyzing the data collected during the experiment and calculating the probability that the observed differences between variations are not due to chance. The platform designates a test winner only after achieving a statistical significance of 95%.

Can we segment test participants based on predefined criteria?

Yes. When defining an A/B test in Sitefinity CMS, you can choose between running it for all users or a specific user segment. The platform allows you to isolate specific audience segments and run A/B tests targeted to these groups, enabling more precise and relevant testing.

How are test results visualized and reported?

Sitefinity Insight presents A/B test results through a detailed summary and visual analytics. Each test includes information such as the presence of a winning variation, the start date, and the total number of days the test was active (including any paused periods). It also shows total visits across all variations and, if applicable, the defined target audience.
The goal completion rate graph illustrates how each variation performs over time. Users can hover over data points to view daily completion rates and select specific goals for deeper analysis when multiple goals are set.
These tools provide a clear view of user behavior and conversion trends, enabling data-driven decisions.

Can A/B tests run concurrently without affecting each other?

Yes. A/B tests can run concurrently in Sitefinity Insight without affecting each other. The platform's architecture supports multiple experiments running simultaneously, allowing you to test different elements and strategies at the same time.

Are there guidelines for setting up effective A/B tests?

Yes. There are guidelines for setting up effective A/B tests in Sitefinity Insight. These include defining a clear hypothesis, focusing on key metrics, testing one variable at a time and gathering enough data to reach statistical significance.

Can multi-variant testing be performed within the system?

Yes. Multi-variant testing can be performed within Sitefinity Insight. This allows you to test multiple variations of different content simultaneously, providing a more comprehensive understanding of how different combinations impact user behavior and conversions.

How are underperforming test variations managed?

Underperforming test variations in Sitefinity Insight can be identified by analyzing report data to find which variations are not meeting performance goals or show low completion rates. These variations can be adjusted or discontinued based on the insights gained from the results.

What is the process for deploying a winning variation after a test?

The process for deploying a winning variation after a test in Sitefinity Insight involves reviewing test results, confirming statistical significance, and selecting the winning variation as the default version in Sitefinity CMS. This makes sure the most effective version is used to improve engagement and conversions.