Categories
Newsletter Research Skills

Find usability issues in games with playtests

How to use playtests and UX research studies to find usability issues in games. Plus an exclusive playtest of Pokemon Unite to practice on.

Usability issues can ruin games.

When players can’t understand what they are meant to do, where they are meant to go, or what is happening, they get confused, bored and ultimately drop out. This impacts their opinion of the game, retention, and creates an experience that just isn’t fun.

User researchers can help.

One of the core skills for researchers, and one of the focus areas for the How To Be A Games User Researcher book is how to find and deal with usability issues. This helps make games that players understand and enjoy. 

Today we’re going to look at how to use playtests and UX research studies to find usability issues in games.

Get future games user research lessons direct to your inbox

Find games usability issues 

Find Games Usability Issues

Today we’re looking at usability issues. What we mean by usability is “can players do what they are expected to do”. 

That’s different to opinions about a game. “I don’t like that the weapons break in Zelda” is an opinion.  “I don’t understand why my weapons keep breaking” is a usability issue. Today we’re tackling the latter.

1. Start with understanding the design intent

Usability issues are ‘when the player can’t do what we expect them to do’. To spot that, we need to know ‘what do we expect them to do’.

This involves talking to our colleagues. Designers have an idea about how they expect the game to work, and what they think players should understand and do. You need to learn what their expectations are (the ‘design intent’).

Exactly who to speak to, and what to ask depends on what you’re testing – which is why our studies start with scoping around research objectives.  You may need to talk to combat designers, UI artists, level designers, producers, or others, But for an idea about the range of things we may need to understand look at Liz England’s ‘The Door Problem’. You need to know what the designer would say in answer to every one of those questions. 

This can be a lot of chatting.

But it will make the other steps a lot more effective! 

2. Create the right context to create realistic issues

Next comes creating the task. What are we actually going to ask the player to do. 

Some tasks can be very broad “Complete this level” or “play for thirty minutes”. Sometimes they can be very specific “Craft an upgrade to your horse armour” or “Place this object on the shelf”. Exactly what level to set your task depends on your research objectives, but my preference is to lean towards broader tasks where possible, because it allows unexpected issues to emerge.

If you are live in your playtest, you can be looser on the task. Because you are in the room, and watching them play, you can jump in with an improvised task at the right time. 

Designing tasks in UX tests can be tricky. Revealing too much information will artificially lead players to the solution. Revealing not enough information (such as missing tutorials) will create artificial situations that won’t occur in the final game. This needs a lot of thought to balance.

My preference is to be as undirected as possible, while meeting the research objectives. “Play through this bit” is fine, as long as they are exposed to the right content, any tutorials or pre-requisite content, and is easier when you are on hand to subtly guide players towards the right content.

3. Be quiet, watch and look out for deviations

Every time you intervene with players, you are introducing artificial elements to the playtest. Unless you intend to answer questions live for every player post-launch, you should resist answering questions, or revealing information in the session. 

Instead your attention should be focused on listening and watching. Observe what players are doing in the game, and when they do something unexpected, write it down. Looking back at your research objectives, and a good understanding of the design intent will help you recognise unexpected behaviour. 

Write down what you see (we’ll cover good note taking in a future issue). 

When necessary, hand out new tasks to direct players attention to the right areas. (but still be very careful that the words you use or instructions you give don’t reveal information that the player wouldn’t otherwise have)

4. Ask questions to understand what you’re seeing

Observed behaviour is only half of the story. In order to understand why that behaviour occurs, we need to know what’s happening in player’s heads.

This requires asking them questions. 

As described above, there is a massive risk that your questions reveal information to the player that they wouldn’t normally have – artificially influencing your playtest. Bland, non-leading questions are required – such as “what is happening now”. We wrote more about how to ask good questions in a previous issue. 

5. Use the impact to rate severity

It’s common to spot a lot of problems, and as covered in the How To Be A Games User Researcher book, researchers go through a structured analysis process to uncover and describe them all.

In order to focus the team’s attention, we need a process for identifying which problem is most pressing. We do this by anticipating what issues will be most disruptive to players when they play.

This extract from How To Be A Games User Researcher explains one approach for doing this:

A method I like is a four-point scale for issues:

  • Critical
  • High
  • Medium
  • Low

Each issue starts as ‘Low’. Then ask three questions about the issue. Each time the answer is yes, raise the severity up a level.

These questions are:

  • Is this something that the player needs to do to progress?
  • Did the moderator need to step in to help resolve the issue?
  • Once the player had overcome the issue, did they know how to avoid it when they encountered it again?

When running your playtest, noting “what happened to the player as a result of this problem”, gives you the raw information you need to do this analysis + rating process. We will cover analysis in more depth in future issues.

A chance to practice

I’m sharing a session of a player who is playing Pokemon Unite for the first time. During their play session, they encounter some usability issues with the game.

Watch an unmoderated usability test of Pokemon Unite

As you watch, think about:

  • The design intent (what did the game team expect players to understand or do)
  • The issues the players encounter
  • What questions you’d ask, if the session was live
  • What impact the issues had on the player’s experience

(Feel free to share what you spot with me on Twitter!) 

Be ready for a career in games user research

Get my free ebook of career tips from researchers at top game studios such as EA, Blizzard, PlayStation, Ubisoft, Activision and more. Based on exclusive interviews, they reveal ten essential tips to kick-start your GUR career today.

Plus receive a monthly email with free games user research lessons and curated early-career Games UX and UR jobs.

Sign up now to get the free e-book (no spam, just a nice email from me each month!)

Ten essential tips for new and aspiring games user reseachers ebook

Don’t miss the next issue

Every month get an email from me with a free games user research lesson, and the latest entry-level games UX and UR jobs.

Leave a Reply

Your email address will not be published.