SiteSpect Reports
      Back to home
      1. Knowledge Base
      2. Analyze
      3. SiteSpect Reports

      Comparing SiteSpect data to third party data

      At times the question arises that the data from a third party analytics system does not align with the user or metric data from SiteSpect

      SiteSpect uses a proxy based architecture and tends to be more accurate than tag based systems. The data from different systems can be as close as about 3% and range to 15%. There are a variety of other considerations at play that we will explore below.

      The number of users or visits don't align

      • Bot identification uses different methods. SiteSpect uses a combination of User Agent Strings and Automatic Robot Detection (ARD) to pass through those users identified as a bot so they don't end up in your Campaign data. It is also possible to utilise a further layer of bot detection provided by IAB/ABC International Bots List and you can confirm this with helpdesk@sitespect.com. SiteSpect tends to be more stringent at identifying bots. Idicaotrs that this couild be a problem can be found by reviewing rhe User Agents of your campaign data and if JS error metric hits are elivated.
      • Many third-party analytics tools will be configured with extra filters to remove traffic from sources like call centers, internal users, performance tools or similar. Additionally, some analytics tools are not integrated into every type of page, such as internal management tools. SiteSpect will see all traffic by default. Check to ensure configuration filters line up between tools.
      • Triggers are complex. Variation Triggers determine when an experience is applied and the user is counted in the Campaign. SiteSpect supports complex Triggers that can look at not only a page URL, but also page source contents, cookies, referrers, and other headers for example. Ensure that you are comparing apples to apples. For validation campaigns global HTML and defensive triggers are recommended. Ensure Find & Replace contains HTML tags as well.
      • OmniChannel, if configured, can identify users from different devices as the same user. Consider applying the OmniChannel segment in the Performance Matrix to see how it changes the User counts.
      • Audiences will target user groups and therefore reduce assigned users.
      • Percentage of traffic that is routed to SiteSpect can be configured to a value other than 100%. This is a decision that is made during initial configuration. Check with your Optimization Consultant to ensure 100% of traffic is being routed to SiteSpect.
      • Traffic Management options such as non-overlay sets to keep assignments between campaigns mutually exclusive and Campaign Frequency to control the percentage of traffic allocated to a Campaign can result in less than 100% of your visits being assigned to a given campaign. Make sure you sue an Overlay Set with the Campaign and Variation set to 100%.
      • Visit Expiration settings caused users to be expired from the Campaign.
      • Purgeing campaign data can impact traffic volumes. Variation Group that are turned [OFF] or set  to 0% while the campaign was active may also impact traffic. If you believe this to be the case you may be missing users and starting with a fresh copy of the campaign is advised.
      • Assignment Ceiling has been reached and therefore one or more Campaigns are experiencing traffic starvation. Using SiteSpects campaign priority settings can help to resolve this challenge.
      • Are you comparing your web analytics raw data vs the validation campaign or filtered to the validation campaign IDs passed by SiteSpect? If the latter, how does the raw unfiltered data compare? If better aligned, review the Global Variation for your WATTS integration with web analytics.
        • Is your Global Variation for your WATTS integration configured correctly? i.e. set to live traffic, set to show for users who are assigned or counted?
        • Is your validation campaign seeing the expected traffic or is it missing certain segments? Review things like device type, geo, payment methods.

      If Metric hits don't align

      • Metrics can be set up to use the same types of Triggers that a Variation uses.
      • Metrics will be recorded in your Campaign if the user is counted in their visit.
      • Therefore Metrics are subject to the list above that pertains to Campaigns.
      • Additionally, Campaigns have the option of either counting all metrics from the start of a visit or only counting metrics that occur after a user is counted.

      If the split looks different

      SiteSpect randomly splits users in an A/B/n test into one of the Campaign Variation Groups when assignment happens. Users are Assigned to a Campaign once they visit any page that is routed. A user will only be counted in that visit should they Trigger a Variation. As a result, users who do not get counted during their visit will not be included in the Campaign data for that visit.

      Other systems can perform the split and assignment when the user matches a certain page. This logic is different from what SiteSpect does by default and could account for a difference in the percentage split for counted users.

      SiteSpect has an Audience type called Visited URL. This Audience will delay Assignment until the user matches on a specific Page URL.

      Verification and Validation

      • WATTS (Web Analytics Tag Transformation and Segmentation) is a way to push Campaign information (most commonly the Assigned and Counted Variation Group IDs) to a third party system such as Google Analytics or Adobe. If this is not set up you might consider doing so as it provides additional visibility. In addition to passing Campaign information SiteSpect can also pass its user ID (SSID / SSGUID) which can be helpful when validating data across tools.
      • Validation Campaign. Consider running a simple single Variation Group A/ campaign that counts on some pages well-known to the team that queries that third-party system. Add metrics for those and any other pages or actions that are commonly tracked. Start with a validation campaign that counts all users before trying to validate a campaign with more restricted counting. 
      • Ancillary cookie. If your site has any sort of cookie per user, SiteSpect can pick that cookie up and include it in the raw data export for investigation purposes and cross identification. This is called the Ancillary cookie (please work with your Optimization Consultant to get this cookie set up).
      • Custom Variable Metric. A type of Metric called a Custom Variable can be created to capture any type of data in the source or in a dataLayer for example. This can be used to capture a specific site ID that can help with comparison across tools e.g. user ID, basket ID, order ID, store ID etc (please work with your Optimization Consultant or Solutions Developer to get this set up).
      • In general, running one or more Validation Campaigns with the mentioned tools and approaches should provide the necessary feedback to validate data alignment exercises.