Throughout CoolaData’s work implementing behavioral analytics with gaming companies, we’ve seen some mistakes that repeat more often than others. This post shares some of what we’ve learned. We hope it will help you gain better, faster insights about your own games. Keep these in mind and avoid them!
Mapping Event Tracking at a Late Development Stage
Competition is tough, deadlines are short and developers need to get their game out to market fast. A common practice we encounter is designing a game’s features without thinking of event tracking, handling analytics at a late stage of development. This approach may get the game out faster, but in the long run it’s a mistake.
Here is an example: CoolaData worked with a developer who wanted to figure out why certain in-game store items were more popular than others. There was no evident reason until we noticed that the items came in two distinct color ranges. This was not a game item attribute, so it was never properly tracked. Once this attribute was added to the code and to related event tracking, the developer could filter purchases by item color and the suspected correlation was indeed validated.
Adding a new attribute post-launch was a painful process, delaying the ability to gain insights. As a mobile app, it took our developer a couple of weeks before the game’s new version got approved by appstores and updated by most active players. It would have made more sense to map this out while the in-game store was being designed. Define the business questions you want to have answered as early as possible to ensure event tracking is part of your game design!
Relying on Vanity Metrics
“We focus on tracking player retention and conversion – these are our top priorities.” Well… not quite. Industry standard metrics are popular and easy to track. They enable a comparison with competition and to some extent, even indicate the status of your business. The key problem: they focus on numbers instead of behavior.
CoolaData has been working with game developers to dive deeper than vanity metrics and expose the WHO, HOW and WHY behind games: conversion is going up – for whom? Are these your low-value or high-value players? Why is this happening?
Successful data-driven game developers use emerging techniques like path analysis to understand what really drives conversion. They use cohort analysis and segmentation to gain insights on the people behind retention numbers. Power cohorts can lead to an understanding of whether retention is driven by a specific device type, game version, acquisition source, or other factors.
Use actionable metrics! A good way to test whether a metric is actionable is to ask: “what would I do if I lost this metric?” If you cannot answer this question, then most likely your metric is not actionable.
Relying on Generic KPIs
Generic KPIs frequently emerge as vanity metrics. MAU, conversion and day-2 retention numbers are popular and serve as interesting snapshots of what is happening, but they’re highly prone to misinterpretation. A figure can be great for one game and bad for another. Aim to use game-specific metrics that are unique to your needs. True, this means more work and customization, but it will reveal WHY things are happening and will more accurately reflect your game’s potential success.
Think Before You track
We often meet analysts who are eager to track every player action, hoping that an unexpected gem will expose relevant insights. Our experience shows that this leads to drowning in data and to an overall loss of focus. Effective analytics means choosing a handful of high-priority events so you can focus on relevant metrics.
Priorities change across a game’s lifecycle. As you learn more about your players’ behavior, your business goals will change and you will find yourself asking very specific questions. For example: as your game matures, you will likely transition from optimizing game mechanics to tweaking retention campaigns. Define your business goals clearly at each stage of your lifecycle and make sure you do not focus on more than a few metrics each time.
Designing a Feature to Support a Single Metric
You have identified an engagement problem: session duration is decreasing and it’s probably because the game is too easy. You decide to tweak game difficulty in order to drive engagement. Session duration indeed goes up but revenues start to decline. What happened…?
Obviously your change successfully drove engagement, but negatively impacted some other metric. Players might have been frustrated by the increase in difficulty and stopped playing, or are not converting as well as they used to. The problem is – you don’t know because you did not prepare to measure secondary effects in advance.
We’re used to associating a feature with a KPI. We implement retention features, engagement features or monetization features, ignoring their side-effects. The effects can be positive or negative. The important thing is to prepare in advance for measuring secondary effects so that you know what is going on. Otherwise you may end up left in the dark.
To conclude, we hope these tips have been helpful and will allow you to make more effective use of your data. We’d love to hear your thoughts on this so please comment below or drop us a note!