This month, we explore what goes into writing a games user research report, and how to communicate research results effectively.
When a tree falls in the woods, does it make a sound? I’m unsure. But I’ve seen plenty of research studies make no sound because of how they were communicated.
The hard work that goes into planning and running the study is undermined by poor communication. Excellent reporting requires a balance of building trust, excellent storytelling and clear communication – today we’ll look at some tips on how to do that (and look at a real report!).
Write a games user research report
What is a user research report
A games user research report summarises the findings from a playtest or research study – they are the conclusion of all of the work defining objectives, writing tasks, moderating sessions and analysing data.
Writing a formal report is only one way of sharing research findings with teams. Its strength is that it is a stand-alone document. A report should hopefully make sense to someone who didn’t see the research study, or to yourself in six months time when you’ve forgotten what you learned in that test.
The importance of storytelling
A dull report that just lists findings will be easy to forget and will have low impact. With time researchers learn the value of sharing fewer findings, but making the ones they share more impactful.
Having a narrative thread through the report is one way of achieving that. Here’s some tips from EA’s Jess Tompkins.
A report is not the only way to share findings. Depending on the relationship with the team, an email or conversation may be more appropriate.
When deciding how to share findings, remember a report is also not the point of research. We’re running studies to improve games. At all times think about ‘what is the clearest way to communicate my findings with my team’, and use that to inform how you present your research findings – whether you are making a report or not.
What is in the report
VidyaResearcher covers the benefits of keeping the structure for your report consistent:
Making a template makes it easier to consistently structure your report. Here’s what’s typically included within the report:
A one-page summary
A report will have multiple audiences. Some people will need all the detail explaining exactly what was discovered and what caused the issues. This information is helpful for the people who have to design and implement fixes. This can include designers, artists and programmers working on the features described.
Other team members don’t need to know the specifics of each problem, and just need an overview of the scale of the issues and how much attention they will need – most commonly important execs, or producers who need to manage the schedule.
Many games user research reports start with a high level summary of the test, including the most significant findings – what the most important issues were, what areas of the game need the most attention, and the overall state of the game. A good model for thinking about the exec summary is “if someone asked me in a lift how that test went, what would I say”.
Context
To increase the impact of the work, it’s helpful to explain why the study is relevant and important to what team’s need to know today. Here’s a tip from Tom Lorusso:
Again, effective storytelling is important to allow your findings to have the impact they deserve.
Grouping issues
Putting similar issues together makes it easier for teams to follow your findings. It allows teams to section the report, and share relevant sections with the right people to fix them. All of this makes communication of the points clearer, and increases the potential your report will be impactful. Your research objectives might inspire suitable ways to group your findings – whether that is by game mechanic (such as ‘issues with crafting’), by ‘type’ of issue (e.g. ‘issues with the UI)
Think about the ordering of the groups, and put them into a logical order. For linear games, that might be chronological – the order the player encounters them. Or you may consider ordering them by severity – the mechanics with the biggest issues first.
Within each section, again consider how you are ordering the findings. Most severe first? Or chronological? Think about the experience of the reader, and pick an order that makes sense when read.
Each issue explained fully
Proper data collection and analysis should allow you to give appropriate detail to your findings. Describing ‘what is the issue’ isn’t enough by itself. We also need to give enough detail about the causes of the issue so that teams understand where their attention needs to be to fix the issue. Describing the impact on the player will also help teams prioritise, by anticipating the impact of fixing the issue on players.
All of this adds up to a cause/impact structure being common when reporting usability issues – where the intended experience is contrasted to what players actually experienced, such as on this slide:
This structure works for usability issues – but again isn’t the only way of reporting findings. With richer findings (such as describing truths about players), you should consider other ways of describing what you learned so that the key points are understood by the reader.
Some teams also share recommendations or prompts to inspire solutions on their slides also. We’ll cover more about this in future lessons.
Across the whole report, clarity in writing is important. Some tips for writing clearly:
- Have only one idea or finding per slide.
- Use uncomplicated words that people use in normal everyday speech. Avoid jargon.
- Use short sentences.
- If you have to use a font-size smaller than… 16? you probably have too many words on your slide. If you’re not presenting findings, but trying to make a report that tells a story, aim for nothing under a font size of 30.
- Pictures and diagrams can make your points clearer – but only if they are understandable. Overwhelming diagrams will quickly lose the audience.
Next steps
We want our research to lead to action – and a next steps section is one way to encourage that. If you intend to run a workshop to help teams apply your findings, you can suggest that as the next step. Or if teams are going to be addressing the issues themselves, you could suggest further research objectives to tackle, or remind teams to re-test to check the issues have been fixed successfully.
See a real research report
In my talk last year on ‘how to get games user research experience without having a job’, I partnered with Cat Floor Studio who kindly allowed me to share the report from our review together.
It doesn’t incorporate all of the best practices discussed above (where is the storytelling!) and more work could be done on making the most important points obvious. However to see an example of what a report might look like, take a look at this usability review of The GoD Unit.
Other reporting tips
Laure Duchamp shared her tips on LinkedIn
- Think about your audiences, all of them. If your communication speaks to a variety of stakeholders, chances are it will have more reach and will be used for various purposes.
- I usually divide it into parts and at the end of each part sum up key insights and recommendations. That way you have more rhythm, and people don’t have to wait until the end to get your point.
- Make it more alive with images, gifs and videos clips! Varying inputs helps keep attention
- Add all your « further data » (any complex graph or raw data) at the end, in a part you might not use when presenting. Having extra slides can be good for picky stakeholders who like to dig into the evidence, or just to add more context to your insights.
- Your report is a material that has to stand and live on its own. It has to be reusable and shareable without you being there to talk it through.
We’ll explore other ways in which research findings can be shared in future issues, so stay tuned!
Ready to finally start your games user research career?
Every month, get sent the latest articles on how to start a career in game development, and find games user research jobs.
Plus get two free e-books of career guidance from top games companies
