We were recently approached by one of our existing clients about writing some design documents and doing some game balance for upcoming high-priority features in one of their games.  Being a design agency owner, I replied “Of course.  And what features are those?”  Each member of the client’s management team then began to list off different features, meanwhile arguing with the other managers on the phone about why their choices didn’t make sense.  As politely as I could, I told the customer that it would be impossible for us to work on their high priority features until we all agreed on which features were actually high priority.  

I set up an in-person meeting with the team to walk them through a set of exercises I use to help prioritize features and plan development.  The client emerged from the meeting with a solid roadmap, a repeatable process, and a strong sense of gratitude.  I will walk you through that same set of exercises and tools, and hope that you will find it valuable and useful too.

There are four key pieces to the process, and they need to be performed in a very particular order:

1. Understanding your goals
2. Generating ideas
3. Evaluating ideas
4. Prioritizing ideas

There is a right way and wrong way to do each of these pieces.  Here’s the way that has worked best for us and our customers.

1. Understanding Your Goals

Without a clear goals and a strong prioritization system in place, it’s easy to get caught up in pointless battles of opinion over what the team should be doing. This slows everything down and increases risk enormously. 

Your designers think that you should be adding new mini-games because they sound like fun.  Community thinks you should add pets because three people said so on the forum yesterday.  Your product manager thinks you should add a daily login bonus roulette because two of your biggest competitors already have one.  And your artists want to add a player avatar system because they just know they can make it look incredible.  

Who is right here and who is wrong?  This battle is typically decided for all the wrong reasons – things like who is the most persuasive, who has the most political power in the company, or who shares the most genes with the founder.  But there is a right way to settle this dispute, and is by forging a common understanding of what you are trying to achieve by these new features.  For live free-to-play games, that means a focus on Key Performance Indicators (KPIs for short).

What are KPIs, you ask?  KPIs are simple quantitative measurements of how the players in your game behave in the aggregate.  For better or worse, free-to-plays are businesses.  You’re hoping that players will like your game enough to try it out, return to it day after day, play it frequently, and pay for your various add-ons.  And this means that your game lives or dies on the strength of its KPIs.  And that means that it’s not only your product managers that need to understand what your KPIs are and why they matter, but your game designers as well.

I could (and perhaps will someday) write a book about KPIs for gaming, but for the moment I’ll just list out a few of the most important:

• D1, D7, and D30 retention
• ARPDAU
• Daily conversion and lifetime conversion
• ARPPU
• Sessions/Player/Day
• Minutes Played/Player/Day

Depending on how your game is structured, there may be other KPIs that are at least as important, but these numbers are pretty critical for just about every free-to-play game.

When thinking about how to prioritize features for your game, it’s important to know three things about each of these numbers:

• What is that number currently?
• How is it trending?
• How does it stack to to our goals (or industry averages)?

Take a look at each key number that is below the target or is trending significantly downward.  This lets you pinpoint the KPIs that you need to increase in order for your game to be successful and profitable. And if all your numbers are looking good, look at those you’ve been most consistently able to increase in the past or those that would make the most difference in your profitability and long-term health.  These are the numbers that you will target through the items in your roadmap.

Now that you have your target KPIs, it’s important to prioritize them.  Is it more important to get more players to convert to payers?  Or to get more players to stick around past D7?  The answer will vary from game to game and situation to situation, but thinking it through clearly and precisely is critical to the success of your roadmapping process and your game. 

I strongly advise building a stacked rank list of the KPIs you think it’s critical to move in the next few weeks or months.

2. Generating Ideas

There is definitely a right way (and many wrong ways) to run a brainstorm.  For more information on running a productive brainstorming session see my follow-up article – The Proper Conduct of Brainstorming.

For now, let’s just say this is the part of the process where the whole team (or whatever part of it you choose to involve in the process) gets to have some fun generating ideas.  Let your artists build ideas ideas for what would look cool.  Let your game designers think about maximizing fun.  Let your product managers think about incremental payment conversions.  Hell, go wild and let your product managers think about fun (but keep your expectations in the right place).

At this point in the process, the more ideas the merrier. The goal of the brainstorming phase (or any proper brainstorm) is to generate the widest possible variety of ideas so you can winnow them down later.  Brainstorm and keep brainstorming until your team is exhausted and ideas have stopped flowing. 

Make sure that all the ideas are written down (albeit tersely) and that all key stakeholders understand at least the basic thrust of each idea.

3. Evaluating Ideas

Now that we have some clear goals and a big list of ideas that might possibly address them, we need a way of meaningfully connecting them.  And that’s where the evaluation matrix comes into play.  The evaluation matrix allows us to get a good look at which ideas best support which goals, along with other important criteria that will help us prioritize work.

So what do we need to know to figure out whether and when an idea is worth pursuing?  There are four key criteria: Relevance, Benefit, Cost, and Risk, which we will ultimately put into a matrix.  Let’s take a quick look at each of these items.

Relevance

Relevance is a measure of how well the idea addresses our current stated goals.  Focusing on relevance first keeps us from getting random ideas on the plan because they sound cool or fun or competitive.  Since we began this exercise by focusing on the KPIs we want to move, we will use those KPIs as a measure of the idea’s relevant to our business. 

For my relevance measure, I like to look at each idea and figure which of our top priority KPIs it might reasonably be able to move.  Is this an idea intended to help people get hooked on their first day?  To keep long term players around?  To persuade more people to spend money?  To get spenders to spend more?  Just building a quick list of the KPIs targeted by this feature will really help you understand whether the idea can possibly have a meaningful impact on your business.  And you can winnow down your list of ideas by quickly removing any that aren’t likely to have an impact on your most important target KPIs.

So, after completing this exercise, your matrix might look something like this:

Benefit

Benefit is a very rough quantitative measure of how much you think the surviving ideas will impact those target KPIs – basically whether the feature will have a little impact or a lot on the KPIs you using it to move. 

Unless I have close real-world analogs with well-documented results I’m generally uncomfortable estimating an exact numerical impact that any feature will have on a specific KPI, but I’m generally comfortably sorting ideas into High, Medium, and Low Impact buckets based on the general concept and my experience launching similar features.

After performing this step on each of your ideas, your matrix might look something like this:

Cost

Cost is a measure of what it would take to actually make this feature a reality.  It’s a rough measure of how much time and effort (or non-salary cash expenditures) it would take to get the feature out the door. 

Since are still basically at the brainstorming stage of the implementation process, it doesn’t make sense to halt the process for a week so every department can go through a detailed planning process of huddling up, creating a detailed estimate of hours, and assigning work out to individuals, so instead we will once again put our ideas into High, Medium, and Low buckets.

So after estimating this for each idea, your matrix might look something like this:

Risk

There are two main risks when adding new features to a game (or changing existing ones) – first that the change may screw up some critical existing piece of the design and second that the community may react poorly to the change. 

Game designs are tightly interlocked sets of systems where changing something in one place can result in dramatic upheavals elsewhere, like the mythical butterfly wings.  Your system designers need to be able to offer a quick, intelligent take on whether the change at hand is likely to have negative repercussions elsewhere – preferably in real time.  Likewise, player communities are finicky beasts; they have very strong opinions about what works and doesn’t work in the game and they are not afraid to let you know.  There are some things that player communities generally react to poorly, like surprise price increases and other heavy-handed monetization tactics.  But there are other subtle nuances for each individual player community, and your community managers should be able to give you insight on how they are likely to react to the ideas on the table.  You may also want to shade your Risk evaluations based on your level of confidence that you can actually execute the feature at a high enough level to get the benefits you anticipate.

Once again, we will bucket the ideas into High, Medium, and Low buckets like this:

4. Prioritizing ideas

Once you’ve been gone through the exercise of building your matrix, the building of the actual prioritized list of features is pretty straightforward. 

Ideally, you’re looking for easy, low-risk, high-reward features to implement.  But even absent those, at least you can quickly figure out intelligent tradeoffs between benefit, effort, and risks, and quickly eliminate cool-sounding but irrelevant ideas.

Putting this system to work will help you create better roadmaps, focus your team more, and get significantly better yield from your development efforts.  Give it a try in your next roadmapping session; you’ll be surprised at how easy it is to implement and how well it leads your team to strong, converged agreement.