‘Roadmap’ can be a dirty word – creating impressions of a big, immovable plan that must be stuck to. And as any game developer will tell you, that kind of waterfall thinking is incompatible with making great games – fun and opportunities are discovered on the way, which will throw the whole plan into disarray. 

However some upfront thinking can still be helpful – and coming up with an outline of ‘what do I need to learn, when’ can be the prompt needed to get playtests running.  Your game’s success is measured in the experience of the player – and without prompts to regularly expose the state of your game to players, it’s impossible to tell how well development is going – until it’s too late! 

In this article, we’ll look at what to test, and when playtests (and user research studies) should be happening throughout development, to de-risk development and release the best possible version of your game.

Avoiding the “big playtest” trap

I’ve spoken to many teams suffering from disasters, caused by playtesting too late.

Teams who treat playtesting as a ‘tickbox’ exercise often decide to sync it with big milestones – running one large study at Alpha or Beta gates. This is usually due to a combination of practical reasons (an appropriate build won’t be generated until the publisher needs it), logistics (it’s hard to dedicate time to playtests amongst everything else), and an auteur mindset of ‘I am the expert and know what players need’.  

That leads to slow and unwieldy studies, with objectives ranging from “just tell us what players think”, or long lists of open questions from years of uncertain development ranging across every department – from level design, to combat balance.

Developers then run one big playtest, late in development. And find a whole bunch of problems that they can’t do anything about because they have built dependencies on them, invested too many expensive assets into, or just run out of time to deal with.

There is another way. 

Frequent, small-scope studies timed intentionally throughout development create more opportunities to address the issues coming out of playtests, more opportunities to pivot – and ultimately throwing away less work due to realising earlier when you’re on the wrong path. 

Test what is important and relevant

There’s no point learning things you can’t act on. 

There’s also no point learning things that don’t matter. So, the first step for bringing sanity to your playtesting process should always be “work out what matters, that we can act on”.

All game design decisions have assumptions about players’ behaviour. For example, when we decide “this game will be a metroidvania-farmsim mashup”, we assume that there is a market for players of that genre. When we decide “this enemy will have a flashing weak point on his chest”, we assume players will notice the weak point, and be able to hit it. 

Assumptions about player behaviour are implicit in every game design decision, and our first step should be recognising them. 

🚀 Action: Gather the leads, and lead them through a process of articulating their assumptions about player behaviour from their recent and most pressing decisions. 

When we have a list of assumptions, you can ask yourself some questions to assess which are most important to validate with players. Some questions I use to prioritise are as follows: 

🚀 Action: Rank the list of assumptions, to identify your most pressing playtest priorities. This can make a nice workshop when combined with the previous action.

Keep up with decision-making 

Once again, game development isn’t a linear process – and new decisions are made throughout development. 

To run effective playtests, prioritisation of ‘what don’t we know’ can’t be a one-time activity. It needs to match the cadence that decisions are being made. 

For some teams, that could be the sprint cycle, and it’s not impossible to plan a regular cadence of playtests every few weeks – but it’s unlikely that appropriate progress is made between each sprint to justify the time, as teams also need to react to what they learned from the last playtest.

Bi-monthly or quarterly may be more appropriate cadences to consider, but the exact answer should match ‘how often do we make important decisions about our game’.

🚀 Action: Decide an appropriate cadence for these prioritisation workshops, and schedule them in

Tighter scope makes playtesting more feasible

A natural outcome of this process is that the objectives for your study will become more focused. Rather than wanting to learn everything, each playtest will be focused on the two or three most important things for right now – and everything else can wait until the next playtest. 

This changes the methods you’ll need. With better-scoped studies, you’ll be able to more successfully match the right research method to your objective. You don’t need a thousand players to see if your tutorial works – just six people will be enough to spot any major issues around understanding. 

Here’s a little help with picking the right playtest method, so you can scope your study correctly.

This unlocks the potential for smaller, more focused playtests.

A games user research roadmap

In general, the shape of what teams need to know changes as they go through production, and the scope of decision-making changes.

Here’s a map that gives a steer to decisions typically made at each stage of production, and how that might influence your playtest objectives. 

Lots of little studies > One big test

My message, as always, is that ‘one big playtest’ is a risky approach to game development. 

Unless you’re making a direct sequel or clone, there is no reliable method to anticipate ‘what the game will be like’ without building and experimenting, and seeing how players interact with it. 

Delaying learning how players experience your game introduces unnecessary risk, and increases the chance of failure for your game (leading to some high profile melt-downs!).

By changing your mindset away from ‘playtests as a step in the process’ to ‘playtests are a tool to de-risk decisions’, you start to run more frequent, tighter-scoped studies, and unlock more reliable game production.