Monitoring Alerts

Table of Contents

• Alerts Overview
• Metric Alert Details
• Alerts Configuration

Alerts Overview

Alerts inform you about specific conditions such as a winning Campaign, a disabled Campaign, a Metric not receiving data, and a number of other conditions. When Alerts are enabled they are presented in a few places within SiteSpect:

• The SiteSpect Dashboard.
• The Campaign List page.
• The top of Campaign and Metric detail pages 
• Email messages related to Alert conditions.

The following are examples of alerts as shown on the SiteSpect Dashboard. These same icons are used on the Campaign List page.

There are five categories of alerts:

• Hurting KPI – “Hurting KPI” is denoted by a  red flame icon and indicates that the KPI of one or several Campaigns are being negatively impacted as they continue to run. To be considered “Hurting KPI”, a Campaign must reach at least 80% statistical significance for 2 consecutive days and have a minimum of 1,000 visits per Variation Group. 
• Disabled or not receiving traffic – “Disabled or Not Receiving Traffic” is denoted by a red exclamation point icon and indicates that one or several Campaigns have been placed or left in a disabled state, or are not receiving any traffic. SiteSpect alerts you to this condition so you can change the Audience, and/or discover why the Campaign is disabled and remedy that condition.
• Winning – “Winning” is denoted by a green victory cup icon and indicates that the KPI of one or several Campaigns are being positively impacted as they continue to run. To be considered “Winning”  a Campaign must reach at least 90% statistical significance for 2 consecutive days with at least 1,000 visits per Variation Group AND have a positive change when compared to the Control.
• Underperforming – “Underperforming” is denoted by a  trending-down arrow icon and indicates that one or several Campaigns have reached at least 90% significance for a specified period of time with at least 1,000 visits per Variation Group AND have a negative change when compared to the Control.
• Metrics without data – “Metrics Without Data” is denoted by a red exclamation point (the same icon as used for “Disabled or not receiving traffic” alerts) and indicates that one or several Campaigns contain Metrics that are not receiving data. SiteSpect alerts you to this condition so you can review the Metric's Triggers and the Campaigns associated with them, and make adjustments as needed.

Metric Alert Details

How does the metric alert work? 

This Alert is meant to detect when a Metric has not received data in the last hour.

Specifically, the alert engine runs every hour and triggers an alert on a Metric when the following conditions are met: 

• The Metric is Active;
• The Alert is enabled for the Metric;
• The Metric is included in at least one active and running Campaign;
• At least one Variation Group in the Campaign has visits (default of 1,000 visits per Variation Group);
• The Metric has not received any data since it was last checked;
• The Metric does not already have an active No Data alert.

What happens when a metric alert occurs?

When an alert is triggered, an email is sent to all users of a site that have system alerts enabled. This email will contain a list of all metrics that are not receiving data. Previously alerted metrics will remain in the list as long as they are not receiving data.

An alert will also appear on your Dashboard, and on Metric and Campaign detail pages. 

When SiteSpect alerts you by email about a new Metric not receiving data, it will also list all of the other Metrics that are still not receiving data. You will also see the list of these Metrics in different sections of the product including Dashboard, Campaign Definition & Reports, and Metrics.

A Metric keeps triggering the alert, what can I do?

Certain Metrics may not receive a lot of data, by definition. For example, a Metric meant to detect a checkout error, or other rare condition. You can easily turn off the Alert on the Metric detail page to avoid false positives for such Metrics.

Alerts Configuration

SiteSpect allows you to specify settings for a Hurting KPI Alert and for Metrics without data Alert. All other alerts are not configurable.

Settings for the “Hurting KPI” Alert

To specify settings for the Hurting KPI Alert, use the Metrics page.

1. Select Metrics from the left nav.
2. Locate the Metric you want to edit and select it.
3. Click On to turn on the Hurting KPI Alert. It is Off by default.
4. In the Raise Alert When area of the page, click the edit icon (the little pencil) in the upper right corner.

1. Use the visits field to determine how many visits have been exposed to the Variation Group before SiteSpect evaluates the Metric’s performance against the control.
2. In the % below the Control field, enter the percentage difference that triggers the alert.
3. In the % significance field, enter the percentage significance that you want to use for the alert.
4. Finally, enter the number of days to examine for the alert.
5. Click Apply. When all of the conditions are met, SiteSpect displays an alert.

Settings for “Metrics without data” alert

Minimum Visits Threshold

The No Data Alert will only trigger for Metrics that are included in Active Campaigns receiving live traffic with at least 1,000 new visits per Variation Group. 

If you wish to change the default minimum visits setting for this alert, you can make this request to your professional services team.

Disabling the “Metrics without data” Alert at the Metric Level

By default, newly created Metrics have the No Data Alert enabled. Certain Metrics are purposely meant to receive very little data, however, such as Metrics that are built to detect errors on your website. You have the option to turn off the No Data Alert for any Metric. To do so:

1. Select Metrics from the left nav.
2. Locate the Metric you want to edit and select it.
3. Click Off to disable the No Data Alert.

Disabling  the “Metrics without data” Alert at the Site Level

For staging or other non-production sites, it may be preferable to turn off Metric Alerts completely, for the site.

If you wish to turn off Metric Alerts for your site, please make this request to your professional services team.