Why players churn
We all think we know why players leave games. Or at least, every developer moderates their game on the pre-conceptions they have about churn. Maybe they limit the ads shown, or they may reflect these ideas in their on-boarding process. But, what are the realities about why players leave?
Is there only one dominant reason, or are the reasons different for players who leave at different stages of the game?
If we knew the answers to these questions, developers could identify those at risk and adjust the game for those specific players. So, what’s the best way to find out, and what type can you learn from predicting churn?
Before you start, you must have access to event level game data. We would typically track a month’s worth of new installs through their engagement with the game, so the actual data set for these players will be up to two months. There are two typical methods used, one that predicts whether the latest session played will be the last, which uses binary classification methods, and another that calculates the number of subsequent sessions a player will play after a given session, predicting the value using linear regression models.
Based on the linear regression model, which is often simpler, we aggregate the player data for each day of play, based on selected metrics. Working out which metrics to use is a combination of experience and trial and error. From our experience, we know there are several metrics which can generally perform well when it comes to identifying defined player segments, and so we’ll start by testing these on our data to see if these metrics distinguish unique player behaviors.
These are some of the typical metrics we use, plus others that may fit the specific game. The number of metrics we test, depends on the complexity of the game.
1. Activity – How much do they play?
Frequency and length of sessions are a simple detail to track for players.
2. Competency – Are they good at the game?
We often base game competency on the top score a player achieves, but this is highly dependent on the player’s progress, so it is usually normalised by the median score achieved by all players that are on the same day since install.
While score is the overall most important measure of competency, in most F2P games, non-spending players will need to employ tactics within the game to grind sufficient currency to maintain their progress. This can involve completing missions to earn grind currency, so when this is the case, we also include the total number of missions completed as a measure of competency.
3. Spending – Do they have the right ideas for doing well?
A simple measure of how much a player has spent (up to a given day) is a good guide to showing how aware players are of the items available for purchase with grind currency and how they impact gameplay.
Real money spends are not considered for churn modelling, as they don’t add much statistical significance, due to the typically low fraction of players who make in-app purchases.
4. Progression – How quickly do they progress?
Where the main aim of the game is to level-up, we assign an integer to the level of progress. Where the aims are more complex, we would assign an integer based on a combination of the factors involved.
Testing the metrics
To check the validity of our metrics, we plot them against each other to visualize their distribution. Clusters of data in these plots can indicate whether or not a metric is working in the context of it being a reasonable differentiating factor, or whether, for example, two metrics are describing the same thing, enabling us to discard one from the model.
We use a type of linear regression model that best fits the type of data and spread to determine the coefficients of each metric on each playing day to determine their contribution to that day’s churn for days numbered from zero to 5+. By applying the coefficients to the metric scores for each player on a given day’s play, the model predicts the number of days left for that individual player. While the model’s accuracy for predicting the precise day each player will churn is quite low, it is very good at identifying individual players who are most at risk of churning on any given day, and of course, the reason why. This data can be pulled through into creating a target list for these players for real-time mitigations to be A/B tested, or making more general improvements to the game.
In addition to predicting which players are likely to churn, when the coefficients for each metric on each day are compared on a table, it is easy to see how the influence a metric exerts on the probability of churn for players as they progress through the game. So, for example you may see players with low activity metrics on day zero having a high probability of churn, whereas by day 4 this metric has much less influence on the probability of churn. Progression can become more important later on in the game, so the game can be adjusted to support these. Obviously, once the game has been adjusted, each of the coefficients will need to be re-calculated, but the fundamental model will hold-up.
Simplified sample metric matrix for deltaCrunch
The above table provides a simplified example of the potential outcomes, using generic labels for the specific metrics used.
In the case of deltaCrunch, a game we created for demonstrating our platform, we can see that the level of activity is important throughout the early days, but its influence declines and actually becomes negative in days 4 and 5, telling us that we need to make the game easier at the start, but more challenging further on, as players may be leaving through boredom or lack of challenge.
Spending doesn’t seem to affect retention at the start, but from days 1 to day 3, it appears that players are being turned-off by the lack of value from grind currency expenditure, whereas the influence of this declines as the game progresses. Reinforcing this, we can see that the progression metric shows that we should slow down the gameplay for players who are progressing through the game too rapidly, as it makes a significant contribution to churn from day 4 onwards.
The specific metrics used, and supplementary data mining will provide much greater clarity and direction, helping with player targeting.
If you would like to find out more about calculating churn in you game and its implications on game design, visit our consultancy page on www.deltadna.com/consultancy, or contact us for a demonstration.