Expert playtest moderation – ask unbiased questions

Ask unbiased questions. Practice and develop your playtest moderation and run excellent qualitative research.

Last updated:

Moderating playtests is hard. The wrong question or an inappropriate tone can derail your playtest. Accidentally upsetting the player, biasing their response, or revealing inappropriate information can severely reduce the value of the data you are collecting.

But one to one interaction with a player is also a core researcher skill, and one of the most impactful methods we have for understanding player behaviour.

As a skill moderation takes practice (I remember how poorly my first moderation went 🤦 ). This month we share some guidance on how to improve your moderation.

Sign up at the end of the article to get future games user research lessons direct to your inbox.

Prepare appropriately

Excellent moderation starts with preparation. Before you step into a room with a player, you will want to be clear on what your research objectives are, and have written a discussion guide to prepare for the session.

This will ensure that your session has structure, and that you know your questions will be relevant. Even if you have to free-style questions, having a reminder of the research objectives in front of you will help remind you of the objectives we’re trying to answer.

Here is a free template for a discussion guide from my first book, which can be adapted for your playtest.

Consider the space

We want players to be at ease when they arrive for their playtest. The room is an important part of creating a friendly atmosphere where players can give feedback freely.

Think about the room you are playtesting in – we want to avoid the space biasing player’s opinions or intimidating them. Remove the many trophies that say you are an excellent game designer, and the marketing posters to create a more realistic play space.

Also think about the playtester’s experience before they reach the playtest room. How will they know what to do when they arrive at the building? Will someone be there to meet them? Will they know what to say? Spending time thinking about and designing the experience of the playtester arriving at your building will help avoid them arriving unsure or upset, and create a more positive atmosphere for feedback.

Start the session off well

The introduction helps set the tone for the rest of the session. As Meta Quest’s Rich Ridlen explains:

Click through to read more great tips from Rich

This can be rehearsed, and again preparation helps. Use the template to remind yourself what to cover at the beginning, including explaining:

  • Your relationship to the game (“I didn’t make this”)
  • This is not a test (“We’re not testing you, we are testing the game”)
  • What is expected from them (“Today we’re going to play the game, and ask you some questions about it afterwards”)

Do not help.

As a moderator, your presence is unnatural – when the game is released, you won’t be there to help. This means you have to be extremely careful not to introduce new information into the playtest that the game won’t introduce.

Don’t help players when they are stuck*. We need to learn what players would do without outside help, to recreate the authentic player experience and see whether they can overcome the issue.

*The exception to this rule is you can help if it’s something that isn’t in the game yet, but will be. As long as you don’t forget that this is untested because you helped, you can help players overcome unfinished parts of the game.

**The other exception is you can help if everyone understands and agrees it’s an issue. If one player has the problem, let it occur. If the whole team then understands and agrees that this is a problem that should be fixed, you can consider helping other players overcome it, so that you can see new issues.

Do not confirm, deny or answer questions.

Part of not helping is not letting players know if they are doing the right or wrong thing. If they ask questions about how the game is meant to work, or if they are doing it right, you shouldn’t answer.

A good phrase is What would you do if I wasn’t here. Or turning their question back on them What do you think?

Ask questions, but be careful with what you ask.

So, if we’re unnatural, why are we in the room at all? Part of our role as a moderator is to ask questions to understand why players are acting the way they are. Just watching will show us what players do, but we have to ask questions to understand why.

Questions can be dangerous, as they can introduce information the player didn’t have. Asking How did you know that was the right way to go reveals that this was the right way to go.

Because of this, my favourite questions are usually very bland, and just get players explaining what is going on currently (which you can then follow up on): What is happening currently? or How did you realise that? get players talking, without revealing any information.

Also remember to be careful when you ask questions. As Camille explains:

Read more tips from Camille

Probe deeper

We need players to feel comfortable giving feedback. This requires creating a friendly atmosphere, but also listening to the points they raise.

When players say something, or give feedback, as a moderator we need to consider “is this enough information”. A moderator should always be thinking about what would level of detail would be useful to your team. This requires asking further questions to reveal enough detail that a designer can take action.

This game is too hard isn’t enough detail for a team to action. We need to ask follow up questions to ask what happened that made them believe the game is too hard, so that the team can fix it.

As Laure says this doesn’t always have to be asking a question, leaving space can also help reveal more information:

Read more tips from Laure

Focus on moderation

Moderation is a skill and takes attention. As a moderator we have to carefully watch players to make sure we’re asking timely relevant questions. As Francesca explains, it can’t be multitasked.

https://twitter.com/core2design/status/1501306315867623430

Note-taking is an equally valuable skill, which takes preparation to do correctly. As Donat describes:

https://twitter.com/Donatbihr/status/1501158204486606849

Read more tips from Donat

Moderation is a skill that takes practice. We’ll cover more tips in a later issue.

Support How To Be A Games User Researcher

Thanks for reading. If you like this and want to support, consider:

  • Sharing the newsletter with a friend or colleague
  • Leaving an Amazon review for the How To Be A Games User Researcher book
  • Sending me some feedback about the newsletter!

Have a great month.

Steve

Dont miss the next issue

Every month, get an email with a free games UX research lesson, and the latest entry-level games UX and UR jobs from me.

Start your games user research Career

Every month, get sent the latest articles on how to start a career in games user research, and hand-picked job opportunities.

Plus get two free e-books of career guidance from top games companies

Free Games User Research Ebooks
Author image

Meet the author

Steve Bromley is an expert user researcher, who works with studios of all sizes to run playtests, and integrate user research into the game development process.

Learn more

Keep Exploring

Game Dev for user researchers

Game development for new games user researchers

Understand game development, and how user researchers interact with their industry peers, and be more likely to pass the interview. 

How to run unbiased player interviews

How to run unbiased player interviews

Every stage of preparing, running and analysing your playtest interviews. Plus tips on how to reveal player’s honest thoughts, and apply that to game design decisions.

Master Games User Research

Free monthly new articles teaching playtesting & how to be a games user researcher. Join the most interesting conversations about games user research, discover job opportunities, and be introduced to new ways to think about game development.