When should I run playtests?

Learn what to test throughout production, to de-risk development and release the best possible version of your game.

Last updated:

‘Roadmap’ can be a dirty word – creating impressions of a big, immovable plan that must be stuck to. And as any game developer will tell you, that kind of waterfall thinking is incompatible with making great games – fun and opportunities are discovered on the way, which will throw the whole plan into disarray. 

However some upfront thinking can still be helpful – and coming up with an outline of ‘what do I need to learn, when’ can be the prompt needed to get playtests running.  Your game’s success is measured in the experience of the player – and without prompts to regularly expose the state of your game to players, it’s impossible to tell how well development is going – until it’s too late! 

In this article, we’ll look at what to test, and when playtests (and user research studies) should be happening throughout development, to de-risk development and release the best possible version of your game.

Avoiding the “big playtest” trap

I’ve spoken to many teams suffering from disasters, caused by playtesting too late.

Teams who treat playtesting as a ‘tickbox’ exercise often decide to sync it with big milestones – running one large study at Alpha or Beta gates. This is usually due to a combination of practical reasons (an appropriate build won’t be generated until the publisher needs it), logistics (it’s hard to dedicate time to playtests amongst everything else), and an auteur mindset of ‘I am the expert and know what players need’.  

That leads to slow and unwieldy studies, with objectives ranging from “just tell us what players think”, or long lists of open questions from years of uncertain development ranging across every department – from level design, to combat balance.

Developers then run one big playtest, late in development. And find a whole bunch of problems that they can’t do anything about because they have built dependencies on them, invested too many expensive assets into, or just run out of time to deal with.

There is another way. 

Frequent, small-scope studies timed intentionally throughout development create more opportunities to address the issues coming out of playtests, more opportunities to pivot – and ultimately throwing away less work due to realising earlier when you’re on the wrong path. 

Test what is important and relevant

There’s no point learning things you can’t act on. 

There’s also no point learning things that don’t matter. So, the first step for bringing sanity to your playtesting process should always be “work out what matters, that we can act on”.

All game design decisions have assumptions about players’ behaviour. For example, when we decide “this game will be a metroidvania-farmsim mashup”, we assume that there is a market for players of that genre. When we decide “this enemy will have a flashing weak point on his chest”, we assume players will notice the weak point, and be able to hit it. 

Assumptions about player behaviour are implicit in every game design decision, and our first step should be recognising them. 

🚀 Action: Gather the leads, and lead them through a process of articulating their assumptions about player behaviour from their recent and most pressing decisions. 

When we have a list of assumptions, you can ask yourself some questions to assess which are most important to validate with players. Some questions I use to prioritise are as follows: 

🚀 Action: Rank the list of assumptions, to identify your most pressing playtest priorities. This can make a nice workshop when combined with the previous action.

Keep up with decision-making 

Once again, game development isn’t a linear process – and new decisions are made throughout development. 

To run effective playtests, prioritisation of ‘what don’t we know’ can’t be a one-time activity. It needs to match the cadence that decisions are being made. 

For some teams, that could be the sprint cycle, and it’s not impossible to plan a regular cadence of playtests every few weeks – but it’s unlikely that appropriate progress is made between each sprint to justify the time, as teams also need to react to what they learned from the last playtest.

Bi-monthly or quarterly may be more appropriate cadences to consider, but the exact answer should match ‘how often do we make important decisions about our game’.

🚀 Action: Decide an appropriate cadence for these prioritisation workshops, and schedule them in

Tighter scope makes playtesting more feasible

A natural outcome of this process is that the objectives for your study will become more focused. Rather than wanting to learn everything, each playtest will be focused on the two or three most important things for right now – and everything else can wait until the next playtest. 

This changes the methods you’ll need. With better-scoped studies, you’ll be able to more successfully match the right research method to your objective. You don’t need a thousand players to see if your tutorial works – just six people will be enough to spot any major issues around understanding. 

Here’s a little help with picking the right playtest method, so you can scope your study correctly.

This unlocks the potential for smaller, more focused playtests.

A games user research roadmap

In general, the shape of what teams need to know changes as they go through production, and the scope of decision-making changes.

Here’s a map that gives a steer to decisions typically made at each stage of production, and how that might influence your playtest objectives. 

Lots of little studies > One big test

My message, as always, is that ‘one big playtest’ is a risky approach to game development. 

Unless you’re making a direct sequel or clone, there is no reliable method to anticipate ‘what the game will be like’ without building and experimenting, and seeing how players interact with it. 

Delaying learning how players experience your game introduces unnecessary risk, and increases the chance of failure for your game (leading to some high profile melt-downs!).

By changing your mindset away from ‘playtests as a step in the process’ to ‘playtests are a tool to de-risk decisions’, you start to run more frequent, tighter-scoped studies, and unlock more reliable game production. 

Integrate player insight throughout development

Every month, get sent the latest article on how to plan and run efficient high quality playtests to de-risk game development. And get Steve Bromley’s free course today on how to get your first 100 playtesters for teams without much budget or time.

Plus your free early-access copy of ‘Playtest Plus’ – the essential guide to the most impactful playtests to run throughout development of your game

Free Book Free Course
Author image

Meet the author

Steve Bromley is an expert user researcher, who works with studios of all sizes to run playtests, and integrate user research into the game development process.

Learn more

Keep Exploring

Record A Mobile Playtest

A simple set-up for recording mobile games playtests

A reliable set-up for recording mobile games user research studies and playtests, that fits into every budget. 

Choose a playtest method

Choose the right playtest method

A reliable method for picking the right playtest method, and getting data you can trust.

Run a playtest on discord

Run a playtest with your Discord community

The ultimate guide to running a playtest with your Discord community

Master Games User Research

Free monthly new articles teaching playtesting & how to be a games user researcher. Join the most interesting conversations about games user research, discover job opportunities, and be introduced to new ways to think about game development.

Which best describes you?(Required)