How To Run A Games User Research Playtest

I’ve written almost one hundred articles covering every aspect of playtesting and games user research. And I recognise – that can be overwhelming, and hard to navigate.

If you’re planning and running a playtest or user research study, this article should be your guide – helping you through each part of the process. I’ll update it over time as new articles get added, so start here!

In this article we’ll cover the end to end playtesting process, with spin-off guides for each step:

Deciding when and what to test
Finding participants for research
Picking the right method and designing your study
Collecting unbiased data
Analysing and reporting playtest data

Following these steps will lead to pragmatic, reliable mixed method studies that inspire your whole development team throughout development.

Deciding when and what to test

Running successful playtests is a team effort, and needs everyone to be aligned behind ‘what we want to learn’ and ‘when is best to run it’. In this first collection of articles, we explore when to start (and stop) playtesting, how to decide what to learn from your playtests, and tips for bringing the whole team on the journey.

Before leaving this stage of the playtesting process, you should have a list of research objectives, linked to your development priorities, and a wider team who are excited about the upcoming playtest!

When should we playtest?

Teams often leave playtests too late, leading to low impact studies. One reason for this can be not recognising what they should be playtesting, and when – leaving the team to run big unwieldy studies that learn too many things the team can’t act on. We want to avoid that!

In these guides, I cover what playtests to run in each stage of development, in order to provide relevant and actionable results.

Read on to learn when to playtest:

How to define research objectives

Research objectives are the agreed list of ‘what do we want to learn from our playtest’, and should reflect the current priorities of your whole development team. In these guides I cover how to work with your team to come up with a definitive list of research objectives.

Read on to learn about defining research objectives:

Bring your team along on the journey

Research is a team sport. Making sure that your teammates understand and engage with research and playtests is essential in order for your studies to have impact. In these guides, I cover tips on how to make your team care about research, and how to prioritise it in a busy development schedule.

Read on to learn how to convince your team to playtest:

Finding participants for playtests

One of the most common challenges that teams have is finding reliable playtesters – it can be an expensive and time consuming process (it’s also the first part of the process I outsource when I do have budget).

It’s also one of the most important steps – effort put into finding and screening your playtest participants will ensure you generate reliable and representative results, and avoid one of the major sources of bias that can creep into your playtests.

Before leaving this stage of the playtest process, you will want to have a good understanding of who your players are, and a plan for how you will

Deciding who your players are

To recruit playtesters, you need to have a realistic understanding of who will be playing your game at launch.

In this guide we look at what criteria are worth considering when defining your players, and how to focus on what really matters when finding and defining who should take part in your playtests.

Read on to learn who your players are:

How to define your players

Build your own playtest panel

If you don’t have the budget for external playtest recruitment, it’s sensible to create your own panel of playtesters who represent your real players, that you can call upon for your research studies. This guide to building your own playtest panel will help you recruit the right participants for your playtest.

Read on to learn how to make your playtest panel:

A simple process for making a playtest panel

Making the most of less-ideal participants

Sometimes finding the ‘perfect’ participants is not feasible, and you’ll have to make do with other groups. In these guides we cover how to test with your existing community, and with internal staff members, and some steps you can take to minimise bias with each group.

Read on to learn about playtesting with specialist audiences:

Picking the right method and designing your study

Having found your participants, you have to make smart decisions about how to design your study.

Templates for all of these studies, and more are available in the 5-star rated Playtest Kit.

You’re ready to move on from designing your study once you are confident what you’ll be doing in your time with players, and that you’ll be gathering appropriate data that will answer your research objectives.

Choose the right method for your study

There are a lot of methods you can use the collect data from a games user research study – including observation, interviews, surveys and analytics. These guides help you understand the range of methods, and guide you through the process of selecting the right one for your playtest.

Read on to learn how to pick the right playtest method:

Decide your number of players

This guide will allow you to work out how many players are enough, for your qualitative or quantitative playtests.

Read on to learn how many players you need for a playtest:

How Many Players For A Playtest

Make and analyse a playtest survey

Surveys are a popular way to gather quantitative opinion data, supporting your qualitative observation and interviews. These guides cover how to design and analyse playtest surveys.

Read on to learn how to design a reliable playtest survey:

Research Method Guides

Games user researchers have a range of methods they should be ready to deploy, including surveys, usability tests, multi-seat playtests, interviews and more.

Read on for some specialist method guides:

How To Run A Multi-Seat Playtest

Getting the right technical setup for your playtest

Playtests fail when builds break, and the technical setup for a playtest can let you down (Don’t forget to hit record!).

These guides cover the setup and operations side of playtesting, and will help you overcome some of the technical challenges in organising your playtest. After following them, you should be confident you’ll minimise the risk of technical failures disrupting your study.

What to think about when preparing your study

An overview of what research operations things to organise for your playtest, from encouraging players to turn up, to running your pilot test.

Read on to learn how to prepare your playtest study:

Technical tools for playtests

Recording a playtest can be low tech (OBS is a good starting point). In these guides I cover some tools for in-person and remote playtests, and how to run playtests remotely.

Read on to learn how to set up your playtest tech.

Collecting unbiased data

By far, observing players, and asking probing questions to assess their current understanding and opinions is the most powerful tool available for playtesting, and can provide a huge amount of insights at a reasonably small cost.

It takes bravery to speak to players one-to-one, and experience to avoid accidentally leading players or introducing bias through your question choice or phrasing. Mastering these skills is essential to being a successful games user researcher.

Moderate Playtests & Interviews

These guides cover tips for the time you spend with players – how to ask unbiased questions that elicit genuinely useful information from players, while keeping an appropriate emphasis on observed behaviour.

Read on to learn how to get the most from your players during a playtest:

Taking fast and efficient notes

Mindmaps can drastically cut down the time it takes to capture and analyse playtest data. In this video, I cover an approach for speeding up your notetaking and analysis, including a live demo of the approach.

Read on to learn faster methods of note-taking and analysis:

Mindmaps – Get In Front Of Game Design Decisions (YouTube video)

Analysing and reporting playtest data

Playtests generate a lot of data, and working through the raw observations, interviews and analytic behaviour to find ‘the meaning’ is a slow and deliberate process. However crafting a clear and compelling narrative that conveys the most important point is essential for impact – long, dull or vague reports get ignored.

These guides cover how to analyse and share playtest data, and craft a compelling narrative that makes teams care about your results. When you can make a compelling playtest report that leads to action, you’re ready to move on!

Analysing playtest data

Taking an deliberate process through your playtest data will ensure that you get authentic, unbiased results that you can be confident representing to the whole game development team (no hallucinations here!).

Read on to learn how to work through your playtest data to find the real meaning within it.

Sharing games user research findings

Your relationship with the development team will influence how best to share your research findings. Finding a compelling, but lightweight approach is essential in a busy game development process

Read on to learn how best to share your playtest results with others:

Develop your playtest process further

Playtesting is an iterative process – and your process will improve over time. If your playtesting needs a kick-start, get the Playtest Kit, used by hundreds of developers to run reliable and robust playtest everytime.

Integrate player insight throughout development

Every month, get sent the latest article on how to plan and run efficient high quality playtests to de-risk game development. And get Steve Bromley’s free course today on how to get your first 100 playtesters for teams without much budget or time.

Plus your free early-access copy of ‘Playtest Plus’ – the essential guide to the most impactful playtests to run throughout development of your game

Meet the author

Steve Bromley is an expert user researcher, who works with studios of all sizes to run playtests, and integrate user research into the game development process.

Learn more