Selecting a KPI might seem easy, but for most situations, there are many possible ways to measure success. The art comes when finding the balance between measuring the direct engagement with a change and measuring a final, end-of-funnel conversion. Metrics that immediately measure a change, like clicks to a new widget, typically need fewer visits to detect a difference, but won't necessarily affect your business goals. On the other hand, down-funnel metrics like conversion rate can sometimes be so far removed from what you're testing that you will need to run your campaign for quite a long time to detect a change.
The more you test, the more familiar you will become with your metrics and which behaviors are the most reliable to measure. But here are some general guidelines and best practices to help you find a balance when selecting a KPI for your campaign.
Measure your business goals
Select a metric that aligns with your business goals. Typically, business goals are metrics that support the objectives of the company. That might include new user registration, appointment booked, completed checkout, or subscription signup.
Metrics such as product views, blog articles read, and clicks to the navigation are helpful when understanding the user journey, but are not always correlated with overall business success. For example, someone viewing many products might be highly engaged, or they might not be finding what they need.
Consider sample size
Smaller conversion rates need more visits to reach a conclusion. If the estimated duration in your test plan is very long, consider using a KPI that is closer to the change that you're testing. For example, when testing a new product carousel on the homepage, you may want to measure add-to-cart instead of checkout conversion.
Think about offline metrics
Often, the true goal of a campaign does not occur within a web visit. This includes metrics like manually confirmed leads, completed appointments, product return rate. While it may be useful to encourage more people to submit a lead, if the quality of those leads is low, the campaign might not be considered a success overall. Or a campaign could result in no change in total products ordered but does reduce the return rate. These metrics can only be measured offline but might be the best choice for your KPI. Read more about how to measure offline metrics in your campaigns.
Guidelines specific to the Sequential Test Plan
In addition to the general guidelines around selecting a KPI, there are a few things to keep in mind when configuring your campaign to use SiteSpect's group sequential test plan.
The test planner works best when measuring unique, binomial, values, such as conversion rate, form submission, or email signup. But the planner can also support count metrics like total add-to-carts, or total articles viewed, and value metrics like order value.
Please note that calculations set as the KPI will always use visits as the denominator for the checkpoint evaluation, even if your calculation uses a custom metric for the denominator in other reports.