Find usability issues in games with playtests

How to use playtests and UX research studies to find usability issues in games. Plus an exclusive playtest of Pokemon Unite to practice on.

Last updated:

Usability issues can ruin games.

When players can’t understand what they are meant to do, where they are meant to go, or what is happening, they get confused, bored and ultimately drop out. This impacts their opinion of the game, retention, and creates an experience that just isn’t fun.

User researchers can help.

One of the core skills for researchers, and one of the focus areas for the How To Be A Games User Researcher book is how to find and deal with usability issues. This helps make games that players understand and enjoy. 

Today we’re going to look at how to use playtests and UX research studies to find usability issues in games.

Find games usability issues 

Today we’re looking at usability issues. What we mean by usability is “can players do what they are expected to do”. 

That’s different to opinions about a game. “I don’t like that the weapons break in Zelda” is an opinion.  “I don’t understand why my weapons keep breaking” is a usability issue. Today we’re tackling the latter.

1. Start with understanding the design intent

Usability issues are ‘when the player can’t do what we expect them to do’. To spot that, we need to know ‘what do we expect them to do’.

This involves talking to our colleagues. Designers have an idea about how they expect the game to work, and what they think players should understand and do. You need to learn what their expectations are (the ‘design intent’).

Exactly who to speak to, and what to ask depends on what you’re testing – which is why our studies start with scoping around research objectives.  You may need to talk to combat designers, UI artists, level designers, producers, or others, But for an idea about the range of things we may need to understand look at Liz England’s ‘The Door Problem’. You need to know what the designer would say in answer to every one of those questions. 

This can be a lot of chatting.

But it will make the other steps a lot more effective! 

2. Create the right context to create realistic issues

Next comes creating the task. What are we actually going to ask the player to do. 

Some tasks can be very broad “Complete this level” or “play for thirty minutes”. Sometimes they can be very specific “Craft an upgrade to your horse armour” or “Place this object on the shelf”. Exactly what level to set your task depends on your research objectives, but my preference is to lean towards broader tasks where possible, because it allows unexpected issues to emerge.

If you are live in your playtest, you can be looser on the task. Because you are in the room, and watching them play, you can jump in with an improvised task at the right time. 

Designing tasks in UX tests can be tricky. Revealing too much information will artificially lead players to the solution. Revealing not enough information (such as missing tutorials) will create artificial situations that won’t occur in the final game. This needs a lot of thought to balance.

My preference is to be as undirected as possible, while meeting the research objectives. “Play through this bit” is fine, as long as they are exposed to the right content, any tutorials or pre-requisite content, and is easier when you are on hand to subtly guide players towards the right content.

3. Be quiet, watch and look out for deviations

Every time you intervene with players, you are introducing artificial elements to the playtest. Unless you intend to answer questions live for every player post-launch, you should resist answering questions, or revealing information in the session. 

Instead your attention should be focused on listening and watching. Observe what players are doing in the game, and when they do something unexpected, write it down. Looking back at your research objectives, and a good understanding of the design intent will help you recognise unexpected behaviour. 

Write down what you see (we’ll cover good note taking in a future issue). 

When necessary, hand out new tasks to direct players attention to the right areas. (but still be very careful that the words you use or instructions you give don’t reveal information that the player wouldn’t otherwise have)

4. Ask questions to understand what youre seeing

Observed behaviour is only half of the story. In order to understand why that behaviour occurs, we need to know what’s happening in player’s heads.

This requires asking them questions. 

As described above, there is a massive risk that your questions reveal information to the player that they wouldn’t normally have – artificially influencing your playtest. Bland, non-leading questions are required – such as “what is happening now”. We wrote more about how to ask good questions in a previous issue. 

5. Use the impact to rate severity

It’s common to spot a lot of problems, and as covered in the How To Be A Games User Researcher book, researchers go through a structured analysis process to uncover and describe them all.

In order to focus the team’s attention, we need a process for identifying which problem is most pressing. We do this by anticipating what issues will be most disruptive to players when they play.

This extract from How To Be A Games User Researcher explains one approach for doing this:

A method I like is a four-point scale for issues:

  • Critical
  • High
  • Medium
  • Low

Each issue starts as ‘Low’. Then ask three questions about the issue. Each time the answer is yes, raise the severity up a level.

These questions are:

  • Is this something that the player needs to do to progress?
  • Did the moderator need to step in to help resolve the issue?
  • Once the player had overcome the issue, did they know how to avoid it when they encountered it again?

When running your playtest, noting “what happened to the player as a result of this problem”, gives you the raw information you need to do this analysis + rating process. We will cover analysis in more depth in future issues.

A chance to practice

I’m sharing a session of a player who is playing Pokemon Unite for the first time. During their play session, they encounter some usability issues with the game.

Watch an unmoderated usability test of Pokemon Unite

As you watch, think about:

  • The design intent (what did the game team expect players to understand or do)
  • The issues the players encounter
  • What questions you’d ask, if the session was live
  • What impact the issues had on the player’s experience

(Feel free to share what you spot with me on Twitter!) 

Be ready for a career in games user research

Dont miss the next issue

Every month get an email from me with a free games user research lesson, and the latest entry-level games UX and UR jobs.

Ready to finally start your games user research career?

Every month, get sent the latest articles on how to start a career in game development, and find games user research jobs.

Plus get two free e-books of career guidance from top games companies

Free Games User Research Ebooks
Author image

Meet the author

Steve Bromley is an expert user researcher, who works with studios of all sizes to run playtests, and integrate user research into the game development process.

Learn more

Keep Exploring

Scope Studies For Impact

Scope your studies to increase research impact

How to scope games user research studies for impact – uncovering the context for decision making, and setting sensible objectives.

budget for games user research

How to budget for games user research

How much does user research cost, what to budget for user research, and how best to spend the budget you have to de-risk game development.

Master Games User Research

Free monthly new articles teaching playtesting & how to be a games user researcher. Join the most interesting conversations about games user research, discover job opportunities, and be introduced to new ways to think about game development.

Which best describes you?(Required)