After you have spent so much time and energy building your product, it would be a shame if it didn’t come to fruition. Even worse, your users might become unhappy, disengaged, and stop using your service.
Engaging products don’t happen by themselves. You will need to do some work to make your users happy. Don’t worry, though. You’ll learn how to do that in this article.
We will be using the insights about your users to identify what separates the happy users from those that leave your platform. To do this, you will need to have a substantial amount of data about your users. If you don’t have enough data, you might want to try this method later.
Why use insights?
Most people in your organization probably like to see the facts before taking an action. And the same goes for your growth and product team. By using the existing information about your users, we will be able to formulate a clear strategy that is backed by the facts.
In addition to keeping your users engaged, a clear strategy will make your colleagues happy as well by getting them on the same page. If you all share the same objective, it becomes very easy to bond together and push yourselves to hit that target. Ready? Let’s do it.
Define your target
The first thing you should do is to define the goal you will be trying to achieve. This will be a unique objective that suits your business. If you’re focused on monetizing your service, this will be probably your user lifetime value. If your product relies on the active user base, you might want to focus on the user retention.
Define the success metrics
Now that you know what your goal is, you need to formulate some potential success metrics. You will be running a regression analysis against these values later. The goal is to find out what behavior separates those users that achieve your objective from those that don’t.
For example, you might be looking at the activity in the last month. If your product helps your users to increase their sales, you might be looking at how well they are performing in this aspect. You can also talk to your users directly and ask what keeps them coming back.
You should also focus on those users that don’t hit your success objective. Why do they keep leaving your platform? What actions do they take? These values will also highly depend on the type of your product. You should include as much explanatory data as possible.
Create the user cohorts
Once you have defined your metrics, you will create the user cohorts. Create a table for every metric similar to the one below.
Objective achieved: true
Objective achieved: false
Metric achieved: true
true positive
false positive
Metric achieved: false
false negative
true negative
For example, if our product was helping companies to send better email campaigns, we could be tracking their open rate. If our product relies on the subscription model, our overall objective might be to increase the user retention.
Retention after 6 months - YES
Retention after 6 months - NO
Open rate >= 5%
100
13
Open rate < 5%
4
30
Running the regression
After we created our tables with the metrics, it is time to run the regression analysis on them. The regression analysis simply states the correlation between the metric and the overall objective. At the end of this step, we should have a simple hypothesis about which metrics influence our target. If you arrive at a wide range of possibilities, you might want to alter your threshold values.
You will be calculating two values for each table - Positive Predictive Value(PPV) and Negative Predictive Value(NPV). Their values can range from 0 to 1 and you can calculate them based on the formulas below.
In our example, the open rate correlates strongly with our objective. This will not be the case with every metric. If you get values that don’t correlate strongly with your target, these metrics probably don’t influence your results and you can dismiss them.
You need to pay attention to the NPV as well. If you have a specific metric that is causing your users to miss your target, you might want to get more users to hit that metric. You need to end up with a model that contains only those metrics that significantly affect your target objective.
Ideally, you would find a metric that has high correlation for both the PPV and NPV. This will be your hypothesis. In our example, hitting the open rate of 5% almost definitely keeps the user around, while missing this target will almost surely result in a churn. We could further tweak this value to 4.5% or 5.5% until we come up with the best hypothesis.
Lastly, you need to make sure you have enough room to tweak your metric. In our example, almost 77% of the users already exhibit this behavior, so we don’t have much space to convert more users.
Why use insights?
Define your target
Define the success metrics
Create the user cohorts
Running the regression
Positive Predictive Value = true positive / (true positive + false positive)Negative Predictive Value = true negative / (true negative + false negative)Positive Predictive Value = 100 / 113 = 0.885 = 88.5%Negative Predictive Value = 30 / 34 = 0.8824 = 88.24%Verifying the hypothesis