Intro to SiteSpect
      Back to home
      1. Help Center
      2. Get Started
      3. Intro to SiteSpect

      Introduction to Testing

      We've put together a little introductory information about testing. Check out these new topics and let us know how we can help.

      What is Testing?

      Testing compares two versions of a visitor experience (for example: two versions of the same web page or app, different promotional offerings, different check-out or new account sign-up processes, etc.) and determines the best performing one.

      There are two key methodologies for testing:

      • A/B (also known as split testing)
      • MVT or Multivariate testing

      Testing allows you to:

      • Improve conversion and revenue
      • Mitigate against risk of the unknown
      • Remove guess work using data-backed results
      • Understand site visitors better
      • Resolve or avoid time consuming internal debates

      A/B Testing

      A/B tests allow you to compare a Variation with your current web page (Control) and measure which one is performing better and meeting the goals you’ve established.

      Introduction to Testing - AB Testing

      Your website traffic is randomly split equally between the Control and the Variation in real-time. Visitors behavior is measured in real time and captured for analytics.

      Typically A/B tests select one element to change within a page but could be much larger such as a complete web page redesign or full funnel journey. An A/B test can also include more than one Variation A/B/C/X.

      MVT or Multivariate Testing

      MVT testing expands on the concept of A/B testing by combining multiple A/B tests at the same time.

      Introduction to Testing - MVT or Multivariate Testing

      The result is a larger number of Variations within a test that allows you to measure how each of these performed individually as well as combined, e.g., AA, AB, BA, BB.

      Your website traffic is randomly split equally between the Variations in real-time. SiteSpect measures the visitors’ reactions to the changes. MVT test results may provide further intelligence on what made the difference, by how much, which parts work together, and which conflict.

      MVT tests can focus on a small area such as the Add to Cart button, a full web page, or even the complete visitor journey.

      Personalization or Behavioral Targeting

      Personalization or Behavioral Targeting takes A/B & MVT testing one step further by allowing you to select (target) which visitors should see which changes.

      You can target many different attributes including device type, browser version, geo location, time of day, day of week, marketing email, and so on. You can further consider specific visitor types such as New vs. Returning visitor, Premium vs. Standard, Guest vs. New vs. Existing Customer or anything available on your website/app and any combination of those elements that you’d like to explore.

      Targeting your testing using these audience criteria allows you to focus on specific groups of visitors with set needs to drive supreme engagement and higher conversion.

      Conversion Rate

      A conversion rate is a measure of successful actions taken divided by total chances for that action to be taken.

      Conversion Rate = Conversions / Visitors

      Conversion rate is a standard metric for comparing the Control and Variations since random samples across each will never have exactly the same volume.

      Conversion Difference

      Conversion rate difference is calculated as ((New Conversion Rate – Old Conversion Rate) / Old Conversion Rate).

      It is most often referred to as Uplift or Conversion Increase and signifies the improvement in performance between the Control and the Variation.

      SiteSpect 101

      How Does SiteSpect Work?

      The following illustration shows how a web page is normally delivered.

      Introduction to Testing - How Does SiteSpect Work

      The following illustration shows how a web page is delivered when you use SiteSpect.

      Introduction to Testing - How Does SiteSpect Work II

      Some SiteSpect Terminology

      The following is a short list of some of the most important terms we use in SiteSpect.

      Variation: The change you want to make (e.g., reordering the nav bar or testing a 3rd party vendor)

      Metric: The behavior you’d like to track (e.g., placing an order or interacting with a part of the site)

      Triggers: Tell SiteSpect what you want to match or change (e.g., URL, Page Source, Geolocation, etc.)

      Campaign: A combination of Variations and Metrics. Visitors are assigned to a Campaign. SiteSpect shows visitors Variations as we track their behavior real-time using Metrics.

      Audience: Visitors who are included or excluded from a Campaign. They are determined based on things like visitor behavior, device type, and location.

      Variations

      A Variation is one change you want to make. Name this to describe where and what the change is.

      A Variation Group is a group of one or more Variations that a visitor experiences. Name this to describe the visitor experience as a whole.

      Types of Variations

      • Search and Replace (inject or alter HTML, CSS, JavaScript)
      • Visual Editor (great for simple changes)
      • Redirect (server side Internal, 301, or 302)
      • Origin Experiments (signal your server to respond with a specific experience)
      • Client-Side Variations (great for script based dynamic pages such as Single Page Apps)

      Metrics

      • Trigger based: Page or JSON requests that route through SiteSpect
        • Based on Triggers such as URL and Source (see Triggers below for all the options)
      • JavaScript based: sends requests to SiteSpect for visitor interactions, such as:
        • Clicking on an element
        • Hovering over a region
        • Scrolling past a certain point
        • Opening a dialog
        • Essentially any interaction on a page

      Triggers

      Specify one or more Triggers for a Variation or Metric. Triggers support plain text or Regular Expressions for powerful pattern matching.

      • URL (Pathname): Everything after http://www.yourdomain.com. For example: /product/1234.html
      • Page Source: Anything in the response content (page source).
      • Parameters: One or more query string parameters in the URL. For example: utm_source=facebook and utm_medium=social
      • Headers: A Cookie, Referring URL, User-Agent (or pattern), Host, or Custom Header
      • Visitor Behavior/Personalization: A visitor's behavior such as triggering a Metric or custom content
      • Geo Location: Visitor location such as Country, State, City, or Postal Code
      • Page Category: Predefined Trigger for a page or group of pages (e.g., home page, search shelf, browse page, order confirmation)

      Audiences

      The visitors who are included or excluded from a Campaign. Audiences can be based on a number of different criteria:

      • Geographic location
      • Entry point URL
      • Referring URL
      • Device Type
      • Cookie values
      • IP Addresses
      • Time
      • Assignment to other Campaigns

      Campaigns

      A Campaign is a combination of Variations, Metrics, Triggers, and Audiences. Visitor are assigned to a Campaign based on the Triggers and Audience settings. SiteSpect shows them Variations as we track their behavior real-time using Metrics. A Campaign can be an A/B, A/B/C/X or a MVT.

      SiteSpect provides advanced analytics that capture Campaign results in real-time and report on all aspects of your visitors’ digital experiences. Stakeholders can monitor the effect of changes and quickly analyze and understand Campaigns and their outcomes.

      Solutions: Breadth Beyond A/B/MVT

      • Origin Experiments – Test the features, functionality, algorithms, and infrastructure that your customers may not see, but have a meaningful effect on the digital experience.Introduction to Testing - Campaigns
      • Mobile – Test, measure, and deliver the most effective content, layout, and promotional offers for each mobile device category.
      • Single Page App – Optimize the entire experience without modifying or tagging your application regardless of which SPA technology framework you are using.
      • OmniChannel – Use OmniChannel to ensure a smooth and seamless experience for all customers, regardless of channel mix.
      • Engine-Only API – Automate more of your testing practice programmatically.

      SiteSpect Support