Categories
Newsletter Research Skills

Expert playtest moderation – ask unbiased questions

Better moderation | Activision’s Hannah Murphy | Entry Level Jobs | Games UX Scholarship, Resources and the #gamesUXchallenge

Moderating playtests is hard. The wrong question, or an inappropriate tone can derail your playtest. Accidentally upsetting the player, biasing their response, or revealing inappropriate information can severely reduce the value of the data you are collecting.

But one to one interaction with a player is also a core researcher skill, and one of the most impactful methods we have for understanding player behaviour.

As a skill moderation takes practice (I remember how poorly my first moderation went 🤦 ). This month we share some guidance on how to improve your moderation. We also have an exclusive interview with Activision’s Hannah Murphy, a free scholarship to learn Games UX, and the next #gamesUXchallenge

Get future games user research lessons direct to your inbox

Learn to moderate playtests

Prepare appropriately

Excellent moderation starts with preparation. Before you step into a room with a player, you will want to be clear on what your research objectives are, and have written a discussion guide to prepare for the session.

This will ensure that your session has structure, and that you know your questions will be relevant. Even if you have to free-style questions, having a reminder of the research objectives in front of you will help remind you of the objectives we’re trying to answer.

Here is a free template for a discussion guide from my first book, which can be adapted for your playtest.

Consider the space

We want players to be at ease when they arrive for their playtest. The room is an important part of creating a friendly atmosphere where players can give feedback freely.

Think about the room you are playtesting in – we want to avoid the space biasing player’s opinions or intimidating them. Remove the many trophies that say you are an excellent game designer, and the marketing posters to create a more realistic play space.

Also think about the playtester’s experience before they reach the playtest room. How will they know what to do when they arrive at the building? Will someone be there to meet them? Will they know what to say? Spending time thinking about and designing the experience of the playtester arriving at your building will help avoid them arriving unsure or upset, and create a more positive atmosphere for feedback.

Start the session off well

The introduction helps set the tone for the rest of the session. As Meta Quest’s Rich Ridlen explains:

Click through to read more great tips from Rich

This can be rehearsed, and again preparation helps. Use the template to remind yourself what to cover at the beginning, including explaining:

  • Your relationship to the game (“I didn’t make this”)
  • This is not a test (“We’re not testing you, we are testing the game”)
  • What is expected from them (“Today we’re going to play the game, and ask you some questions about it afterwards”)

Don’t help.

As a moderator, your presence is unnatural – when the game is released, you won’t be there to help. This means you have to be extremely careful not to introduce new information into the playtest that the game won’t introduce.

Don’t help players when they are stuck*. We need to learn what players would do without outside help, to recreate the authentic player experience and see whether they can overcome the issue.

*The exception to this rule is you can help if it’s something that isn’t in the game yet, but will be. As long as you don’t forget that this is untested because you helped, you can help players overcome unfinished parts of the game.

**The other exception is you can help if everyone understands and agrees it’s an issue. If one player has the problem, let it occur. If the whole team then understands and agrees that this is a problem that should be fixed, you can consider helping other players overcome it, so that you can see new issues.

Don’t confirm, deny or answer questions.

Part of not helping is not letting players know if they are doing the right or wrong thing. If they ask questions about how the game is meant to work, or if they are doing it right, you shouldn’t answer.

A good phrase is What would you do if I wasn’t here. Or turning their question back on them What do you think?

Ask questions, but be careful with what you ask.

So, if we’re unnatural, why are we in the room at all? Part of our role as a moderator is to ask questions to understand why players are acting the way they are. Just watching will show us what players do, but we have to ask questions to understand why.

Questions can be dangerous, as they can introduce information the player didn’t have. Asking How did you know that was the right way to go reveals that this was the right way to go.

Because of this, my favourite questions are usually very bland, and just get players explaining what is going on currently (which you can then follow up on): What is happening currently? or How did you realise that? get players talking, without revealing any information.

Also remember to be careful when you ask questions. As Camille explains:

https://twitter.com/rehe_m/status/1500896495637250052

Read more tips from Camille

Probe deeper

We need players to feel comfortable giving feedback. This requires creating a friendly atmosphere, but also listening to the points they raise.

When players say something, or give feedback, as a moderator we need to consider “is this enough information”. A moderator should always be thinking about what would level of detail would be useful to your team. This requires asking further questions to reveal enough detail that a designer can take action.

This game is too hard isn’t enough detail for a team to action. We need to ask follow up questions to ask what happened that made them believe the game is too hard, so that the team can fix it.

As Laure says this doesn’t always have to be asking a question, leaving space can also help reveal more information:

Read more tips from Laure

Focus on moderation

Moderation is a skill and takes attention. As a moderator we have to carefully watch players to make sure we’re asking timely relevant questions. As Francesca explains, it can’t be multitasked.

https://twitter.com/core2design/status/1501306315867623430

Note-taking is an equally valuable skill, which takes preparation to do correctly. As Donat describes:

Read more tips from Donat

Moderation is a skill that takes practice. We’ll cover more tips in a later issue.

Other games user research news

Activision’s Hannah Murphy’s Career Journey and Tips

This month I’ve been talking to Hannah Murphy, User Researcher at Activision about her experience working in games, and top tips for people looking to join the industry.

She shared her own journey into user research, her experience running the Games User Research mentoring scheme, and how to develop your own games user research skills.

Read the full interview with Hannah Murphy

A big list of Games UX resources

Nida Ahmad (who very kindly reviewed a preview version of the How To Be A Games User Researcher book) has put together an excellent list of Games UX resources for people new to the industry. See the full list here.

Entry level games user research jobs

Blizzard are looking for an associate research scientist, suitable for someone leaving university with some quantitative research experience.

Futuregames have a scholarship for their Games UX course – see this great pitch from Joel Fällbom 

Hello people considering studying in UX for games but feel that the cost of education is stopping you. Our schools (Futuregames) in Sweden have a merit scholarship program. The fee for the 2 year program would be 0 EUR which is a whole lot cheaper than the regular cost of 18000 Eur for students outside of Scandinavia.

Last year we had a girl from Bolivia joining us as well as Fatima which ended up with the ABK Women in Games Scholarship from Activision Blizzard King. No matter who you are, you have a good chance, since the amount of applicants are few and we will do our best to help you along the way. 

You would have to be able to cover the cost of living (Depending on the location, where the UX program is, the student apartments are about 230 eur/month (2 summer months are free) as well as the cost of food.

Perks of working in Sweden you may ask? 5 Week paid vacation minimum, a industry that is getting more mature with less and less crunch, many with contracted no crunch policies. Very talent starved industry so if you are good it is usually no problem getting work. Many AAA, AA, and indie companies hiring, we need a couple of thousand people to fill the need of professionals here. And most important, a chance to live a complete normal middle to upper middle class life with your own home, paid maternity leave for both parents, etc etc. It is just a pretty nifty place to be no matter if you study or stick around!

Learn more on the FutureGames website.

#gamesUXchallenge

We have a new games UX challenge. This is a stand-alone challenge, so you don’t need to have taken part in the previous one.

This month, learn and practice how to pick research methods + write tasks.

Learn more about the gamesUXchallenge on research methods.

Support How To Be A Games User Researcher

Thanks for reading. If you like this and want to support, consider:

Have a great month.

Steve

Don’t miss the next issue

Leave a Reply

Your email address will not be published.