Sharing research findings from playtests and games UX tests is essential to making sure your studies have an impact, and the findings are not ignored.

Read on to learn about debriefing a study and making a research report.

Debriefing a study

In the previous section, we covered how to identify and describe issues from a user research study. These then need to be communicated with the game team accurately so that the right action is taken to fix them. Doing this well is essential for research studies to have an impact – teams won’t be able to act on findings they don’t understand. 

A debrief can have two stages, and it’s sensible to separate them. First, they should communicate what was learned in the research study. Then, because you have gathered a lot of experts in the same room, it is also an opportunity to start to decide what to do to address the issues discovered.

Sharing what was learned

Following analysis, we should have some fully explained findings written in accessible language. Assuming the study design was appropriate for the research objectives, the findings will be relevant and address the team’s current priorities. 

Creating a research report

These findings can then be communicated in a report, for example in a presentation or a word document. Presenting the findings live, rather than just sending a report, will create a better understanding of the issues for the audience. This leads me towards recommending capturing findings in a presentation, such as PowerPoint or Google Docs, supported with videos and screenshots where required.

A format for this report can be:

• Remind the team of the research objectives
• Briefly cover the method & the audience for the study
• Then present each issue in turn, in order of priority

This can then transition into a structured activity on deciding what to do about the issues covered, as we’ll discuss next.

The two audiences of a research report

When creating a presentation or document, to share with the team, keep in mind that the report will have two audiences. One is the immediate room being presented to. The second is people who the report gets forwarded too, or who come back to it in six months, when the researcher isn’t around to present it. This means that the report needs enough context about what was tested, which build it was and what the tasks were that the report will make sense to an audience who wasn’t there for the original debrief.

Although presenting the report is best because it gives the researcher the opportunity to take questions and explain any misunderstandings, this isn’t always possible and sometimes the report is sent ahead without being presented. Because of this, it’s important that the report covers all of the context for the issues within it, and explains why the issues occurred in enough detail that the development team can confidently make the right decisions to address the issues. 

Deciding what to do

The point of games user research is to make games better, so teams need to make changes based on what was learned in the study. Although deciding and prioritising tasks will largely be the responsibility of a producer, a researcher can help start this process by starting the discussion and acting as a facilitator to help teams decide what to do.

The ideal format for this is one that encourages divergent thinking – coming up with a variety of ideas – then convergent thinking in order to assess which solutions are best. 

The risks of including design recommendations

Some researchers decide to include recommendations for actions in their report. This can be helpful when direct contact with the game team isn’t possible, but it does have some risks.

Although researchers have a lot of understanding about player behaviour, this is only one aspect of the ‘best’ solution, and they lack a lot of other domain expertise. For example producers will understand what’s feasible in the time available, artists will have better domain expertise about what is aesthetically appropriate and programmers will be able to advise on what’s technically feasible. A researcher’s recommendation won’t understand these elements to the same depth that other disciplines do, and may come across as naive.

When a researcher gives solutions, it can also influence the ideas that follow it, reducing the breadth or creativity of ideas explored. Setting challenges instead of solutions can help prevent this. A challenge can encourage creativity, such as ‘how might we ensure players know this puzzle is essential for progress?’, instead of a closed recommendation ‘An NPC should tell the player this is the way to continue’.

When possible, the ideal solution will take the insight from all of the disciplines and combine them. This isn’t always possible, as scheduling all of the relevant leads can be challenging – one of the reasons the calendar invite for the debrief is sent out during the preparation for the test. If relevant colleagues are available, one way of generating recommendations is with a workshop, which can be run back to back with the debrief. 

Workshopping solutions

A potential structure for this workshop could be:

• Reviewing the highest priority issue
• Spending five minutes individually thinking about potential ways of addressing the top priority issue
• Sharing the ideas with the group and having a short discussion to evaluate each
• Voting on which should be explored further, incorporating elements from other ideas discussed as necessary.
• Assigning it to a specific person to explore further, either the producer or a lead from a relevant discipline
• Then repeating this cycle with the next highest priority issue, until the group has run out of time, issues, or enthusiasm

As this workshop runs, the researcher can facilitate it and note the ideas discussed and the favoured options. These can be shared when emailing the research report. A researcher can also help steer the group to avoid the solutions which seem easy but aren’t compatible with what we know about player behaviour. A very common example of this is trying to fix every issue with another tutorial, which players rarely read, without appropriate thought given to the context in which the tutorial will appear.

This collaborative method of deciding solutions allows all of the disciplines to input on the potential solution, will increase everyone’s engagement with the findings from the study, and should lead to better quality design decisions.

Regardless of the final design decisions made, it’s likely that further studies will be required to check that they do fix the issue. Design is an iterative process. As this study ends, it’s sensible to start again with identifying what the most important research objectives are, and planning the next study.