In part one of this series, we looked at the techniques a game publisher or developer might deploy to transform the billions of in-game events generated each day into powerful actionable insights. In this installment, we’re going even deeper, looking at some practical examples of data mining in action and problem solving, data-scientist style.
Data mining is great fun for analysts. You throw them a problem and they come back with big smiles telling you something fascinating about the performance of the game that you are now compelled to address.
But there are certain things that will remove the smiles and create frustration for analysts and their co-workers alike, with the biggest headaches being caused by:
- Gaps in data: not being able to map clearly how players move through the game. Perhaps ‘mission end’ is collected so we know the outcome of a mission but if mission start is not we don’t know how many missions are abandoned.
- Aggregated data: The funnels will tell you where the blockers are for the playing base as a whole. But if there is no access to raw events then after that you are guessing.
- Slow query responses: Analytics is creative, and there is great value in allowing data scientists to range widely across the data – trying hunches, moving quickly through topics to find significant insights. It is vital that the analysts are able to move at the same speed as the rest of the business.
So let’s look at a typical scenario on a good day.
The analysts will first become aware of a problem through the metrics dashboard, which might show a drop in retention rates on a particular day.
Their first move will be to select a cohort of new players from before and after the drop to compare differences in game experience. By looking at levels of missions failed for both cohorts they will be able to see whether there is a general problem with difficulty balancing or whether the problem is more complex.
By using the funnels they will be able to focus the problem down to a barrier in mission 4 in the game and see that a low number of players are now making it through mission 4 compared to the previous cohort results.
So they know there is a problem with difficulty balancing, and they know where it is, but here is where it gets interesting.
In data mining, you start with a blank sheet of paper and start to ask and answer questions deeper and deeper in the data until you reach an inescapable truth. So by using advanced analytics tools you can understand the different reasons why players are leaving.
As a good analyst, you would start to explore potential reasons why players leave. First, a “signal” is explored – a metric that may uncover an interesting behavior. Then, “insight” is delivered – a behavior that results in churn. Finally, an “action” is proposed to address the issue uncovered.
Often, signals do not result in insights, but in this case we simply move on to the next idea.
The following examples show a typical workflow.
1) SIGNAL: By looking at the distribution of how long players had spent in mission 4 and at which point they decide to leave the mission, we may identify a strong peak in the number of players leaving after 10 minutes.
2) INSIGHT: Next we look at the number of mission starts versus mission fails for those players that left after 10 minutes. We notice that each player had five or more attempts with no success.
3) ACTION: A logical move would be to reduce the difficulty after four attempts for this cohort while leaving the rest of the players alone.
4) SIGNAL: Then we might look at the players that left the game before 10 minutes in mission 4. We see that they only had two attempts and two fails before leaving.
5) INSIGHT: For this group we might then look at their mission of resource when they started the mission and we can see that they are already struggling with low amounts of health. Not surprisingly, a couple of further mission fails in mission 4 takes them out of the game.
6) ACTION: So for these players, the answer is to gift them resources at the start of mission 4 to give them more ability to get a good outcome.
These are specific solutions for a specific group of players experiencing a particular difficulty. We leave the rest alone.
That is why data mining is so important. We need this level of sophistication to isolate particular behaviors and address them directly with an appropriate intervention at the right time for the right players. Otherwise, you end up improving the game play for some players whilst destroying it for others.
Moving away from a one-size-fits-all approach to game development really unlocks engagement and lifetime value. By using data mining in combination with game personalization delivers a great experience for all players.
This sponsored post originally appeared on Gamesbeat.