Interpreting Test Results – Performance
This article is designed to ensure you understand the data found in the ‘Performance’ section of a test report:
First up you’ll notice a bar chart which gives a graphical representation of the views and conversions for each experiment. By hovering over the sections on the graph you will see the more details of the particular experiment.
You can toggle the graph on or off by clicking on the Graph icon on the right-hand side, just above the graph itself .
Scroll down to view the tabular data where the ‘meat’ of the report is located.
You’ll notice that each conversion point has its own row of results. Pay close attention to the three columns on the far right-hand side as these help you determine how the test is performing for each conversion point:
In the above example, the control of each experiment is shown at the top of each section and therefore the columns will show a dash. Within each experiment, there are three variations and in this particular report there are three experiments on ‘Globetrotters’ ‘Hover’ and ‘Title and image’
The first column “Lift Over Control” in the first experiment shows a -0.07% decrease against the control for the first variation. The next conversion point shows a -0.12% decrease for the second variation and the final conversion point shows also -0.87% decrease. Therefore you can safely say that this experiment shows that the control is always superior to any new variation.
Within the second experiment, things look a light more positive, with lifts aginst control on every variation. The column chance to beat control column also shows a high percentage and the third column solidifies the result by confirming that the results of the test were significant.
The ‘Significant?’ column enables you to see if you have reached significance for a given conversion point. Webtrends Optimize currently uses a 95% confidence rate to calculate statistical significance.