Obviously knowing what your players think of your game is important. The Google Play store and the iTunes store give app users a chance to review and rate any game they’ve played. The quality of these reviews are, to put it generously, variable. It’s often tempting to think that reviews of your game may not be representing the opinion of your players as a whole.
To put this theory to the test, we took a selection of popular games and compared their day one retention (the percentage of players who still played the game one day after install) to their average rating on the iTunes store. If the ratings are a meaningful reflection of user experience we should expect to see highly rated games having a high fraction of players returning, while poorly rated games would see more players leaving after their first experience. The graph below shows what we found.
Retention Compared to Average Rating
The pink line shows the estimated relationship between retention and average rating; we found that on average a 4-star game can expect a 4% increase in retention compared to a 3-star game.
It’s clear from the graph that this relationship is not a strong one, as some of the poorly rated games had a surprisingly high retention rate and some well-rated games had a low user retention.
Obviously, many factors affect retention and many factors affect iTunes rating. One day retention is a measure of first time user experience, while the ratings may be more representative of longer term user experience. Well made, fun games can end up losing a lot of their players with a poor first time experience, so it’s important to know how to get this right.
A note of caution for free-to-play developers: we see the opposite pattern when comparing average rating to monetization. Higher rated games have, on average, a lower percentage of players paying. The graph below shows average rating against payer fraction.
Payer Fraction Compared to Average Rating
Notice in particular the small clump of games in the bottom right of graph with an average rating of around 4.5 out of 5 but with a very low payer fraction. This suggests that in order to get good ratings you must accept a low payer percentage.
However, there are some caveats to this graph.
Firstly, the relationship between payer fraction and average rating is weak; we found on average a 1% decrease in payer fraction for each extra star.
Secondly, we found that many of the high payer fraction games with low ratings had very few players. It seems these games can only appeal to a very narrow niche of players, who maybe have higher expectations but greater loyalty, and so are willing to pay to understand the potential of the game. Relying on single payments and/or early payments is not necessarily a good strategy for making a profitable game.
Do Game Reviews Matter?
Although there may not be a strong correlation between ratings and retention, it would be foolish to presume that reviews and reviewers don’t matter. Listening carefully to feedback from your players can help you to identify how you can improve your game.
The latest release of the deltaDNA platform now includes a review dashboard with text mining and sentiment analysis of reviews from iTunes, Google Play and Steam. This gives you the ability to understand what your players love about your game and what you might want to change in your game.Sign up for a free trial of the deltaDNA platform.