SiteSpect Reports
      Back to home
      1. Help Center
      2. Analyze
      3. SiteSpect Reports

      Performance Matrix Predictive Statistics

      Confidence Indicator (Conf)

      The Confidence Indicator is a quick way to spot statistical significance for any Variation or Variation Group. Confidence is indicated when there is at least 90% statistical significance (1 bar), 95% (2 bars) or 99% (3 bars). Green bars indicate that the Variation or Variation Group range is higher than Control. Red bars indicate that the range is lower than Control. If there are no bars, it means that the difference is significant at less than 90%.

      Confidence Plot (Plot)

      The Confidence Plot provides a visual representation of how a Variation or Variation Group (the top bar) is performing in comparison to its Control (the bottom light gray bar). A useful guideline for interpreting this measurement is that when the top bar is entirely green (better than control) or red (worse than control), there is at least a 95% chance that the observed difference is the result of a real effect as opposed to random chance.

      This measurement uses an 80% confidence interval. The short vertical line in the center of each bar represents its average. Any green color in the top bar indicates that the Variation or Variation Group is at least partially outperforming the Control; red indicates that it is under performing. Note that it is possible for the top bar to show both green and red if the variance (standard deviation) is much larger in the Variation or Variation Group as compared with the Control.

      % Significance (% Sig)

      Significance is the probability that the difference between the Variation or Variation Group and Control is the result of a real effect as opposed to random chance.

      Note: Do not use Significance as the only indicator when making decisions on the success of the Campaign. Use it to gain insight into the progress and stability of the Campaign results.

      Z-Score (Z)

      Z-Score is a quick way to judge how likely a difference is to occur by chance. This is useful in making a decision about the effectiveness of the Variation or Variation Group vs. the Control. Z scores bigger than ± 1 occur 33% of the time (there is a 33% probability that the difference is due to chance), Z scores bigger than ± 2 occur only about 5% of the time (there is a 5% probability that the difference is due to chance), and bigger than ± 3 only about 3 times in 1000 (there is a 0.1% probability that the difference is due to chance).