Categories
Newsletter

Sharing Games User Research Findings

User researchers find problems with games. But it’s usually someone else who has to fix them.

This means communication is one of the most important skills for a researcher to have – and sharing games user research results is critical to having an impact on the quality of games. 

In this issue, we’ll look at some approaches for sharing research findings, and share some tips from the pros! 

Get future games user research lessons direct to your inbox

Debriefing games user research results

Sharing games user research results accurately creates an impact

From our playtests, we learn how does the game currently differ from the design intent. Much more crucially, we also understand why it differs – what caused players to not understand, or to do the wrong thing. 

Understanding the why is essential to making the right decisions about how to fix the problems. Usually, it is someone else who actually makes the final decision on what to do to fix it – a game designer, art director, level designer, etc…

Full and accurate communication of the issues, their causes, and the impacts mean that our colleagues make the right choices. 

Making a traditional research report 

The most common way of sharing research findings is to write a report. I have an old example of one on my blog, which – although I would do some things differently today, is still a good idea of what a research report might look like. I’m also sharing a new, real, research report as part of an upcoming GRUX Online talk – look for that in a future issue. 

As you will see, the report runs through each of the issues in turn. They are grouped by topic. Matching your groups to ‘what discipline would be working on fixing this issue’ can make distributing the findings to the right person easier. 

For each issue it explains what the issue was, why it occurred, and what impact it had on the player – this information will help teams make the right fix and prioritise the issue appropriately. 

When writing reports, it’s important to write concisely, and use plain language – we’re trying to make sure we’re understood. Complicated words don’t impress anyone, and just confuse your readers! Here’s some tips on how to write better. 

I would always recommend presenting the report to the team live. People are much more likely to pay attention, will have the opportunity to ask questions if they don’t understand, and it just gets a whole lot more engagement. Although it can be tempting to send a report and say “let me know if there are other questions”, I suspect that this leads to reports being ignored, and reduces the chance the team will react to your findings. Ignored findings make running user research pointless, so let’s try and avoid that when we can.

A report isn’t right for everyone

It’s really easy to get into the trap of thinking ‘a user researchers job is to make research reports’. Our job is to make the game better, and that a research report might not be the best way of doing that. 

For some teams, a report isn’t appropriate and would be ignored – instead consider alternative methods of sharing findings:

  • An interactive workshop
  • An email covering the top 5 issues
  • A message on slack
  • A research insight database that they can interrogate
  • Access to the raw data 
  • A conversation 

When starting with a team, think about how much time they have, how they communicate currently, and use that to decide what is the right way to share the results so that they don’t get ignored.

Research has multiple audiences

As well as the team who asked for the study, there are some secondary audiences who will also be interested in aspects of your study. They might need a different method for the findings to be shared with them.

Some of those audiences can include:

  • Executives who don’t need to know the detail, but do need to know if there is a big problem
  • Other teams working on different projects which some of the findings might be relevant too
  • Yourself in 12 months time, trying to remember if this study is a topic you’ve done before or not. 

Many of these won’t have the time to read through all of the detail of your report – consider some of the other methods that might be more appropriate for sharing research findings with them.  

Some prompts to think about:

In interviews, you are likely to be asked about your experience sharing research findings. Some aspects to think about in your answers include:

  • How did you decide what was the right method of sharing the findings?
  • Did you understand the needs of the audience and use that to inform your decisions?
  • How did you evaluate if you had been understood correctly?

Some research results tips from the pros

I asked the community if they had any tips for a successful debrief. Here’s what they said. 

I like giving my results a hierarchicy that relates to the research problem. That way we circle them back to the research questions and it helps staying in scope. I also use a usability severity rating attached to insights so that it can have an impact on priorities in the debrief with designers :)

Laure described how prioritisation is important for making sure the results are relevant. A common mistake I see (and do myself) is trying to fit in all of the findings, and make them all prominent, when all the team really need is ‘what is the most important thing we need to do next’ – which Laure’s method addresses.

For a usability rating scale, I personally like using this severity scale described by userfocus – but other’s exist too! 

You should definitely make sure to know for whom you are writing your report. The vocabulary you use might be different based on who reads it. Also, the level of details might vary based on how much time readers are willing to dedicate to your report. It's also a good idea to have quick takeaways at the very beginning for people who don't have time or to pick up interest of others.

As Raphael says, understanding your audience is incredibly important. At the start of a project, make sure you know who will be interested in the results, so that you can tailor your delivery to them.

Games Research and UX Design Mentoring

The fantastic team from the IGDA-GRUX SIG have opened up their free mentoring scheme for another batch of applicants. 

The mentoring programme has previously partnered over one hundred students with more than fifty industry professionals from top companies such as Sony, EA, Valve, Ubisoft, and Microsoft, and helped many people get their first job in games. It was one of the initiatives I started when on the steering council many years ago, and so I’m really glad it’s had a new boost of life with a new team! 

If you’re interested in applying, see the mentoring website here (I think the application deadline is the end of September)

User Research and UX Practice Exercises 

Callum Deery is an indie game designer working with Alphabet Soup games, who recognised that it’s hard to get in-development builds that can be used to practise research and UX skills at different development stages. He’s working on an RPG and created snapshots, and associated activities, to help people practice. 

One: A competitor analysis and usability review of some early menu prototypes

Two: Evaluating two potential features and deciding which works better.

Three: Helping a team evaluate their demo 

Callum would like to thank Kanesha Patterson and Patrick Tan for their feedback so far and would love to hear your experience with the practice exercises, so do start a conversation with him on twitter about how you get on!

An exclusive interview with Jimmy Zhou from Riot Games

Jimmy Zhou is a user researcher, working on League of Legends at Riot Games. In this exclusive interview, he covers how an internship helped him get into the industry, how to expand your methodology experience, and the importance of starting conversations in the industry.

Read the full interview with Jimmy here.

A quantitative research internship with Epic Games

Ben Taels, UX Research Director at Epic shared an fantastic internship they have available on their team.

Ben Taels - Tweet looking for UX Data Research Intern

The internship looks to require combining survey responses, analytics and other data about users to influence design decisions. Learn more and apply for the job on Epic’s site.

Epic are also advertising an entry-level playtest coordinator role, setting up internal and external playtests for Fortnite.

Playtesting at Develop Conference

A busy month ahead for me. I’ll be talking at Develop Conference next month sharing some lessons on running affordable playtesting for teams who can’t afford full-time research support. 

Steve Bromley Develop - Better Playtesting for Indie Developers

Do come and say hi if you’re attending! (and if it’s a topic you’re interested in, do sign up for updates on when I publish more resources to help indie teams)

Have a great month and good luck on your games user research journey!

Steve (follow me on Twitter for more games user research news throughout the month!)

Don’t miss future issues

Thanks for reading this issue. If you’re reading online, and want to get future issues directly to your inbox – sign up here.

Categories
Newsletter

Remote games user research

For the first five years of my career, most of the research studies I ran were in person. It was much easier to bring people to us, rather than transport our technically complex setup to them.

This had started to change for me prior to the pandemic, and then exploded last year when we could no longer safely bring participants to us. Now remote games user research is an essential part of our toolbox. In this issue we explore how to do it, and some of the things to look out for.

Get future games user research lessons direct to your inbox

How to run games research remotely

Running remote games user research studies

Remote games user research is when participants can take part at home, rather than coming to us. That’s always been the case for large-scale beta tests, which use analytics and surveys to gather feedback. It has traditionally been challenging for earlier stage studies, like a typical usability test. 

There are some advantages from running usability studies remotely. Previously our studies were limited to ‘people who could travel to us in a reasonable timeframe’. Running these remotely allows us to overcome our geographical bias, and recruit people from around the world to take part. Remote research makes the logistics of accessibility testing much easier too – it allows us to find more people with access needs who can take part in studies, and doesn’t put the burden on them to travel to us.

However there have always been some difficulties with running these studies remotely. Many studios are very secretive about their games, and the risk of leaks is increased when we can’t keep an eye on everything that our participants are doing. There are also technical and logistical obstacles that are more difficult remotely. It’s much easier for me to install a prototype of a mobile phone game on my phone and give the phone to a participant, than to get them to install the prototype successfully on their own phone. 

Many of the challenges have got easier as specialist tools have developed – we’ll look at some today.

How to do remote games user research

Live moderated research

Moderated research – where you are able to speak to the participant live and ask questions, is often the best way of gathering the most data possible from your playtest. 

Many teams have been exploring using Parsec to allow remote players to stream access to builds on your own device, similar to how Google Stadia works. This solves some of the technical issues (because the game is running on your own device it doesn’t need to be set up on their computer). It also reduces some opportunities for leaking, so can be preferable to installing the game on the playtester’s computer.  There is a video of how Parsec works on their website, as well as a case study of Ubisoft using it to distribute builds for a marketing event.

If your team are comfortable with participants installing the game on their own device, conference tools like Google Meet allow you to view the player’s screen, and talk to them live. It also has the built-in ability to record the session. Google Meet requires very little technical knowledge from the player, so is simple to set up – and it’s free! 

Unmoderated research

Another alternative to consider is unmoderated research – asking players to play in their own time, and receiving a video recording. This has some downsides – it’s not possible to ask questions in reaction to the behaviour you are seeing in the session. However, it can often be logistically easier to organise.

Some teams use automated services like PlaytestCloud to do this. This allows teams to specify a ‘type’ of player, upload a build, and receive videos of people playing it. This eliminates many of the logistical challenges of distributing games and finding players and makes running unmoderated user research studies easier. 

However not every team has the budget to outsource this (or are not working on a mobile game), and will have to recreate this setup manually. This can be done using tools like Obs or DScout (recommended by @eddiepearson) to capture videos from players, and combine that with a free survey created on Google Forms.

This kind of unmoderated research can allow us to see many more players than we could observe linearly. It can be very powerful for combining qualitative feedback (‘observing what they do’) with quantitative feedback from surveys. 

Non Disclosure Agreements

The risk of leaks is difficult to avoid entirely. Non-disclosure agreements can warn participants and reduce the likelihood of them sharing confidential information. This template from the research agency ping-pong will allow you to create your own NDA, and includes a plan to get it signed remotely. 

Some remote research tips

I asked others on LinkedIn for their tips about remote research. 

Christian Ress from PlaytestCloud gave this great advice:

Some great tips for the study design of remote studies (and good advice to run a pilot. Pilots are always an essential part of my in-person research plan, but easy to forget when working remotely!).

For a different take to addressing the issues from the pandemic, Player Research spoke last year about how they had adapted their lab to be safe for in-person studies, and I know this is something other research teams are looking into. 

Career Interview with Joe Florey – Senior User Researcher at PlayStation

Each issue, we speak to a new or experienced user researcher about how they started their career in games. Joe Florey is an experienced user researcher, working in PlayStation’s European user research team. In this exclusive interview, he explained how mentoring helped him join the industry, and his experience converting a psychology PhD into a games user research role.

#ActivisionBlizzard

It’d be impossible to talk about the games industry this month, without mentioning the allegations of sexual harassment at Activision Blizzard

In the ‘how to be a games user researcher’ book, I covered some of the downsides of working in the games industry. In particular, we focused on the risk of toxic fans, and misogynistic abuse from members of the public. Something I entirely failed to cover was that this also occurs from colleagues within the industry, not just from fans. Although the lawsuit is against a single company, I believe this happens throughout the industry.

Other people are much better placed to comment than I am, but I wanted to share some twitter threads I’ve found helpful over the last month to consider my own actions, and how to create a more inclusive work culture.

Frankie Ward covers how to recognise warning signs in the workplace.

Julie Buchanan shares small-scale interactions that create a unwelcome atmosphere 

Javiera Cordero covers some things those with privilege can do to help.

GUR Café

I’ve been listening to a great new games user research podcast. GUR Café is a podcast with Lanie Dixon, Olivier De Maeyer, & Sébastien Lourties of Ubisoft Montreal. In their second episode, they cover how important communication skills are for user researchers, and some tips on how to work with new teams.

Find all of the episodes of GUR Cafe here.

Entry level games user research jobs

Adams Greenwood-Ericksen shared these exciting roles on the GRUX discord, working on games such as Call of Duty, Skylanders, World of Warcraft, Overwatch, Diablo, Candy Crush, and Bubble Witch.

Hey folks! Activision User Research is hiring for 3 (!) user research moderators to work on-site in our Woodland Hills CA lab. Since these are entry level roles, we’re looking for folks with interest, education, and/or skills in game user research, but we don’t expect any significant prior experience in the field. Come learn the trade and grow with us!

Learn more and apply for the user research moderator roles.

Improving playtesting

People who follow me on twitter have heard about this already, but I’m excited to share a bit about a current project I’m working on for release next year.

Both while working on the book, and in conversations with developers since, I’ve become increasingly aware of an unmet need for many games studios. Our deep dives on a career in games user research are helpful to teams who can give time and attention to user research. But many teams don’t have the time or budget to dedicate a full-time role to understanding players and testing games.

I see a need for converting the best practice from games user research. Making it achievable for teams with no time or money, making their current playtesting practise more efficient and more useful. I think this will be a ‘toolkit’ for game teams to apply to improve their playtests.

My plan is to develop this toolkit in the open. I want to open it up to critique throughout development and improve the quality of the final thing. As a first step I’ve been interviewing game developers to understand how they playtest currently, and what is difficult. Here’s a short thread covering some of what I’ve learned. This has inspired some ideas about what to do to help!

I believe the audience for that project (‘game developers who want to playtest better’) is different to this newsletter (‘people who want to be games user researchers’). So I don’t intend to regularly cover updates for that project here. If you’d like to hear when it’s ready, stick your email here. (no spam, just a nice email from me sometime next year).

Don’t miss future issues

Thanks for reading this issue. If you’re reading online, and want to get future issues directly to your inbox – sign up here.

Have a great month! 

Categories
Newsletter

Quantitative Research for new user researchers

Let’s start with my confession. I have been a user researcher working with games for over ten years. I have run hundreds of studies, and overseen thousands of hours of playtests (over 25,000 player hours at last count!). And yet, I know very little about stats.

There are two quantitative research things I know how to do. Today, I will explain both.

Get future issues direct to your inbox

Quantitative Research for new Games User Researchers

Comparing two sets of numbers

The first thing is how to compare two sets of numbers. I was taught this by Cyril and Mirweis at PlayStation, and I am grateful to them both for teaching me the only stats I know – how to compare two sets of numbers. This is useful when comparing things such as ‘how many times did the player fail’ or ‘how long did it take people to complete this level’

This method is appropriate for when the data is numerical rather than categorical (or ordinal). Here’s a short explanation of what that means. 

When you have some numerical data, it’s quite common to want to compare it. This allows you to learn “is there a difference between these two things”, and then inspire conversations such as “do we want players to fail more times on this level than on the next one?”.

To do this, you want to find the average, and then work out some confidence intervals to anticipate whether the difference between them is real or whether it was potentially caused by not measuring enough people. 

So, after counting how many times people died on level 3, you can take an average – which looks like this. 

We can see that on average, players died around 2.5 times on level 3. 
We can then do the same thing for the next level.

(This is probably a good moment to mention there is a template that does the maths for you later in this post…)

Looking at the average for Level 4 shows us that people died on average more often on Level 4 than they did on Level 3. 


But we don’t know if this is because Level 4 causes more deaths, or just random chance that it occurred in this study.

To identify that bit, we calculate confidence intervals. Which looks like this…

And we can see that the confidence intervals (the uppey-downy bits) overlap. The top of Level 3 overlaps with the bottom of Level 4.

Level 5’s confidence intervals do not overlap with any of the other levels. If the confidence intervals don’t overlap, there is a real difference between them. It’s true that more people died, and will die, on level 5 than level 4.

This hopefully means that Level 5 harder – although you should watch people play to understand actually why the difference in deaths occurred.

If the confidence intervals don’t overlap, we can’t tell if there is a difference. This is the case for Level 3 + 4. This either means that the number of times people die are the same, or that we haven’t seen enough players to draw an appropriate conclusion.

(There are probably errors in the terminology above, but as I said, I know little about stats – I just know how to compare two sets of numbers).

I use this all of the time – to count and compare deaths, completion time, etc. I made a template that you can duplicate to see the formulas required, and to have a go at doing it yourself.

Go deeper on quantitative research

Beyond this one technique, I’ve found two other tools very helpful.

Adjusted wald calculators like this allow you to state your completion rate (e.g. 3 out of 10 people encountered this issue), and from that anticipate how many people in the real world would encounter the same issue (between 10% and 60% apparently).

And the book ‘Quantifying the User Experience’ which has lots of nice decision maps like these, which tells me what tools I should (and shouldn’t) be using … and includes a crash course in stats to explain how to do them! 

Picture from the book ‘Quantifying the user experience’
(Picture from the book ‘Quantifying the user experience’).

Avoid common quantitative research errors

The second thing I’ve learned is a collection of things not to do. By recognising some stats errors, it helps me know when I should seek out someone better with stats than me to help out. 

Avoiding common errors include:

  • Don’t do the kind of maths I described above on ordinal data (such as likert scales). People often do, and get away with it, but it’s somewhat inaccurate as you’re treating categories like they are numbers.
  • Think about the sampling bias you have created in your study, and don’t over-emphasise how representative your conclusions are
  • Don’t assume that because you are measuring what players say they think or do, you are actually measuring what they think or do.
  • Recognise that when you are limiting the options you allow people to select from, you are limiting the range of results you will get back, potentially distorting the truth.
  • Avoid dogmatic rules about sample sizes. There’s lots of rules out there that have become dogmatic (‘quant studies need 30 users’, ‘qual studies need 5 users’), and many people repeat them without understanding the reason behind them. Understand why those guidelines exist, think about what you are trying to learn, and make conscious decisions rather than following ‘rules’.

The job is not just ‘qualitative research’

I sometimes encounter the idea that user researchers are synonymous with qualitative research. I don’t think that is appropriate or correct. Even if you are more comfortable with qualitative research, you shouldn’t allow your skillset to determine the method you apply for answering research questions. 

Instead always lead with ‘what does the team want to know’, and then ‘what is the most appropriate way of discovering that’. If that method isn’t one you are comfortable with, use it as an opportunity to learn how to do a new thing, ask for help from the community, or bring in some help from someone who is comfortable with it. Our job is to “help the team make evidence-based decisions”, regardless of the methods we are most comfortable with.

What quantitative research skills should I be ready for in the job interview?

If you can answer the following questions, I would say you would be a stand-out candidate…

  • What is p-value?
  • How would you compare the difficulty between two levels? What would you measure, and how should that be interpreted?
  • How would you measure if players are enjoying a game?
  • How would you handle being asked ‘I think this study should have a larger sample size’?

You will notice that these questions are often not about ‘how do I do the stats’, but much more interested in ‘when is quantitative research appropriate, how should it be applied, how should I explain things to my colleagues, and what are the caveats for this kind of work’. Which I think is where the real challenge lies! 

Interview with Adam Lobel – User Experience Researcher at Blizzard Entertainment

On the subject of quantitative data, I was lucky to speak to Adam Lobel recently about his journey into games user research. He has been working as a user experience researcher at Blizzard for two years, and we talked about meeting new people in the workplace, turning a PhD into a games job and spotting opportunities in the data that others miss.

Read the full interview with Adam Lobel here.

Games user research jobs

Sprung Studios are looking for a Junior User Researcher to work with their clients which include EA, Activision, LucasArts, Rare, Ubisoft, Microsoft Studios, ArenaNet, and many others. Find more details and apply on the GUR Jobs board here.

Epic Games are hiring a contract user research analyst to work with their fantastic team on titles such as Fortnite. Apply on Epic’s website here.

EA are looking for an Associate UX Researcher to work on FIFA. Apply on EA’s website here.

More games user research reading

Last month we featured the first of Alistair Greo’s articles about Getting Started in Games User Research. Part two is out now, and answers questions such as ‘what research methods should I know’, ‘how to get and demonstrate experience’, and being prepared for interviews. Read Alistair’s article on Getting Started in Games User Research.

If you are writing a cover letter this month, read this twitter thread on how to write a good cover letter.

Games cover letter advice

Also, check out this article from Splash Damage’s Jason Tzaidas on tips for a successful games industry interview.

Thanks, and hello…

Thanks for making it to the end, and particularly thank you to all of the new subscribers – we’ve had over 500 people sign up to this newsletter over the last few months. Many of the new readers joined this month, so welcome to all the new people starting their games user research careers.

I’ve written a book about how to be a games user researcher, do take a look if you haven’t already. If you had, I’d really appreciate a review on Amazon – it does have a huge impact on the book, and I value it very much.

As always, do email me or tweet me with feedback, questions, etc and I’ll see everyone next month! 

Don’t miss the next issue

Categories
Newsletter

Networking in games user research

I once read that 70% of all jobs are not published on jobs sites. And 80% of jobs are filled through personal or professional connections. Although those stats are not specific to games, it’s likely that if you are only looking on job sites, you are missing out on the vast majority of games user research roles.

Even when jobs do make it to job sites, having a personal connection to the hiring manager must make a difference. There are usually a huge number of applications for every games job. This means that hiring managers have to find ways of identifying good candidates, and eliminating unsuitable people at pace. If the hiring manager recognises your name, it’s more likely that you will  make it through that first initial pass, and progress through to the interview and task stages where you can demonstrate your excellence. 

So, how do we make sure that the hiring manager recognises your name? This is our focus for today’s issue on networking.

Networking doesn’t have to be corny handshakes, business cards and scary approaches across busy conference floors. This is helpful, because I’m no good at any of those. Instead, I wanted to share some of my own tips of what I have seen work well, and ask others in the industry their networking advice, to help you start a career in games user research. 

Get future issues direct to your inbox

Networking Tips for new games user researchers

Be useful

That first approach to talk to a stranger can be very difficult. If you are doing something useful, or involved in an activity together that approach becomes much easier. It gives you a reason to be talking and something to talk about.

People new to the Games Research and UX community assume you have to be on the steering committee to start a new initiative or to help out. This is not the case – the conferences need volunteers, and the community welcomes new initiatives being led by anyone, regardless of experience. I started the mentoring scheme and jobs board, and both of these can always do with more support. As can the other initiatives that people have started, such as the excellent reading club.

Join the discord, see what is going on, and then volunteer to help out. It will give you something to talk about. As an added benefit people will start to come to you and open conversations, avoiding the need for intimidating approaches! 

Ask genuine questions

I have found the games research community to be very friendly, and willing to give up their time to people interested in understanding the discipline better. This has been evident through events like the office-hours initiative in the GRUX community. If you are not already part of the mentoring scheme, sign up for the opportunity to meet someone in the industry and learn about it.

Meet existing researchers, ask them about their role, how they got into the industry and show a genuine interest. It’s often very obvious if you’re just interested in a job, rather than treating every conversation as a learning opportunity (particularly if you keep scanning the room in a conversation at a conference rather than giving them your full attention). Leave everyone you speak to with a positive impression about you and your enthusiasm for games user research – It’s a very small industry, and so you will bump into the same people again and again.

This will start to make connections with people outside of the job application process, and can inspire some great ideas about what you can do to get into the industry. We have an interview with Tyler Sterle coming up later in the newsletter where I did exactly that, and he gave some great ideas about his own journey.

Some of our experienced researchers had more advice about talking to people – more follows later in this newsletter! 

You don’t have to travel

The last year has highlighted how events can work remotely. The games user research community have done a great job of combining their conferences with socialising opportunities that work remotely. 

I have found this a great asset, and have appreciated being able to take part in conferences and events that would otherwise take too much time or money to attend, and join the social activities after.

I hope that events like the GRUX Online conference continue this tradition as the world re-opens. Keeping an eye on the GRUX discord is a good way to become aware of remote social opportunities.

Join the conversation

As well as the GRUX discord, games user research conversations are happening everywhere – the GamesUR twitter account does a great job at collating some of them. Find people talking about games user research by searching on the hashtags #gamesUR or #GRUX, and join in the conversations. Reply to other people’s threads and ask questions. Eventually you will start to build up a network of people in the industry, and start to be recognised as someone serious about games user research.

I believe that questions reflect well on the person asking them (it’s better to ask what might be a silly question than to remain ignorant). Here’s me on twitter, if you want a friendly face to talk too.

Networking advice from the experts

I also asked the industry for advice about networking. Here’s some of the excellent points they raised.

//1

Donat's Twitter Header

Most of the people I know will kindly answer to someone who gets in touch in private messages on social media, or by email. Be polite, introduce yourself and briefly explain why you’re sending a message. No need to try to impress, you’re learning, it’s cool.

Be patient, maybe the person won’t have time now, but will answer later. Do not insist, look for someone else. And don’t be angry if the person doesn’t answer.

Finally, people are not your direct contact with HR. Do not ask for a job or training position, at least not right now.

(Donat’s tweet)

//2

Alistair's Twitter Header

Try and find a buddy who can offer support and encouragement. It can be someone else trying to engage also, or someone already in the community (who can hopefully do a few introductions for you).

Also, remember they’re just people 🙂

(Alistair’s Tweet)

//3

John's Twitter Header

Most people are in the games industry because they love games, and most people who love games love talking about games. That’s maybe half the barrier to networking gone – you’re joining a community who share the same passion and interests as you do! You don’t get that everywhere.

(John’s tweet)

An exclusive interview with Tyler Sterle

I first heard of Tyler when he joined the games research mentoring scheme from his very impressed mentor. There’s a lot of inspiration from Tyler’s journey for others interested in joining the industry. 

I asked Tyler about his journey to become a games user researcher. He had a lot to share including essential games user research resources, networking and making your own luck.

Read the full interview with Tyler on the website.

(Image credit – Alistair Greo)

Essential reading for new games user researchers

This month Alistair Greo (who shared some networking advice above) wrote an article about getting started in games user research, with links to some great resources (and the book!), and the GRUX discord.

Read Alistair’s article ‘Getting Started in Games User Research’ here.

Work with Blizzard

Blizzard are currently looking for an Assistant User Researcher. Although this is only a temporary role, it would offer fantastic real life experience and would make getting future roles in the games industry much easier.

Read the full details of the Assistant User Researcher role at Blizzard.

What’s coming up

Thanks to everyone who has sent in questions or inspiration for future issues. Some of the things we have coming up soon include:

  • Interviews with more games user research experts about their journey
  • Getting real experience before joining the industry
  • How does UX and User Research relate?

I also have started talking about my next project, which aims to help game developers run playtests. I will share more on this over the next six months. But don’t fear – I have no intention to stop supporting How to be a games user researcher for a long time yet. We have some great bonus content being prepared, so do stick around!

As always, if you’ve found this useful – the most helpful thing you can do for me is to leave a review of my book on Amazon. The second most helpful thing you can do is tell someone else about the newsletter, and encourage them to sign up

Thanks friends, and have a great month!  

Steve

Get future issues direct to your inbox

Categories
Newsletter

Moving from academia into the games industry

Welcome to issue four of our newsletter about entering the games industry as a user researcher. In this issue:

Congratulations also to Vanissa who won the GUR poster and stickers from the competition last month! 

Get future issues direct to your inbox

Games user research and academia

Moving into the games industry from academia

It’s common that user researchers have academic experience. In this years games user research salary survey, 70% of people who took part had a postgraduate degree. 

Planning reliable playtest studies requires the application of scientific methods. A masters degree or PhD is one way of demonstrating that you are qualified to apply research methods. It is not the only way – plenty of great researchers don’t have academic experience. However postgraduate experience is an indication that you know how to pick an appropriate method, reduce bias and draw reliable conclusions.

I wanted to explore how to transition from academia to industry in this issue. My own experience was very fortunate. My postgraduate course gave me the opportunity to work on projects with real game studios which, supplemented by personal projects, helped me get my first job in games. 

From my own personal experience, using academic study to get some experience in running in-person research is one of the biggest potential benefits of postgraduate study. We covered this in more depth in the HTBAGUR book. Jessica Tompkins, PhD and UX researcher at EA, help on this chapter was invaluable.

I also asked this question on twitter and LinkedIn, and Jessica gave some additional great points on LinkedIn:

Jess's comment about games user research. Linked above

Irena Pavlovic also raised the importance of pragmatism when performing research in industry.

Irena's comment about games user research. Linked above

On twitter, Natali, working as a user researcher at Ubisoft, raised the difference in pace between academia and industry, and that one of the challenges is adapting research methods, analysis and debriefing so that it finds answers when it’s still relevant. 

Natali's tweet about games user research - linked below

Link to Natali’s original tweet

Blake is a UX Researcher at XBox. He also highlighted how the change of environment impacts the goal of quantitative studies, and what we would consider an appropriate conclusion.

Blakes Tweet about Games User Research - linked below

Link to Blake’s original tweet  

Lots of great tips there from the community, and the need for speed comes up multiple times. In industry a study that provides findings too late is pointless, as the decision will have been made. The development team have already moved on to the next topic.

One way of speeding up analysis is through preparing templates for note-taking and analysis, and using mindmaps as a way to speed up analysis, as I covered in the Games User Research Summer Camp event last year.  We also explore this, and other aspects of moving from academia to industry in the book – but it’s a popular topic, so I know we’ll cover different aspects of the transition in more depth in the future.

Charles Somerville Bethesda

How I got started in games user research

Charles Somerville is a user researcher at Bethesda. I caught up with him to ask about his journey into games. He talked about some great topics including applying academic experience into the games industry.

What is it you currently do at Bethesda?

I’m a user researcher, and my job is to conduct and facilitate user research at the publisher level for the many talented dev studios in the ZeniMax Media family. This means that while the actual science of designing and executing user research is the core of my job, it is equally (if not more) important that I also help maintain positive relationships with the many stakeholder groups Bethesda’s UR team interacts with.

What was your journey into games user research?

The abridged version of this story always starts in the same place at GDC 2018 in San Francisco. While attending graduate school in southern California, I had scraped up enough money to attend my first GDC, and by that time I was still unaware that games user research was a field. When exploring the various talks being given, I discovered the roundtable discussion on user research in games chaired by Ben Taels, who would end up being my boss a year later.

Fast forward to 2019 and many, many rejected applications later, I landed a contract role at Epic Games as a UX Lab Analyst, helping to moderate Epic’s lab and support the talented UX team there.

In the middle of 2020, right after the pandemic began, I knew my contract was expiring soon and began exploring full-time opportunities. Fortuitously, a user research role opened at Bethesda, and I applied with all due haste. Many interviews and several months later, I began my first full-time role as a mid-level user researcher.

What did you find the most challenging step of getting a role working in games? How did you overcome that?

The first step is always the hardest, and that is absolutely true for landing a job in the gaming industry. I had several advantages when I started applying specifically to GUR roles – my location in California meant that I was close to many studios, and my recently acquired graduate degree in social psychology was a good way to prove my research acumen.

However, the biggest gap in my history was, funnily enough, displaying a passion for games and doing so in a way that coincided with some kind of work history or academic project. I did what I could to remedy this by working with my academic advisor on incorporating games into my doctoral studies while also doing independent research on the side.

Did your experience working in academia help with getting a job in the games industry? How?

Yes and no. Academia taught me the fundamentals of research and the importance of the scientific method, especially when dealing with something so subjective as games. However, academia never adequately prepared me for the sheer speed at which GUR operates. Turnaround time has varied at both the companies I’ve worked for and across different projects, but I think anyone else in the field will agree that there is an expectation that results are due sooner rather than later. Academia just really doesn’t prepare a person for that – or at least my experience didn’t.

What would your top tip be for someone looking to become a games user researcher?

It’s okay to fail. This field is highly competitive at the junior level. As I said earlier on, the entirety of my 2018 and part of 2019 was spent applying and getting rejected to a lengthy list of roles (recorded in a spreadsheet I keep to this day as a reminder and confidence boost). Every rejection is an opportunity to reach out, be graceful and gracious, and learn why you didn’t make the cut. They are chances to patch the holes in the metaphorical armor of your resume and try again. Be persistent and willing to improve, and you will get there!

As a bonus tip, perhaps even more important than accepting failure, learn how to network. Attend conferences as you’re able. Go to talks. Join the GRUX SIG Discord. Our community is still relatively small, which means that good impressions are critical. Conduct yourself in a way so that your name has a positive association, and your chances will greatly improve.

Thanks Charles for the great introduction to his journey, and top tips! To hear more from Charles, do follow him on twitter. This is a popular topic, so we have more interviews from experienced games user researchers about the transition from academia to industry over the next few issues, including Joe Florey from PlayStation and Adam Lobel from Blizzard. 

Games User Research Community

The #gamesUR summit

This month was also the annual games user research conference organised by the Games Research and UX team.

There were some great live events around mentoring and getting your first role in the industry, hopefully you were able to attend.

There were also some great pre-recorded sessions on topics such as teaching players without tutorials, the accessibility of The Last Of Us, academic and industry partnerships, and why ‘the boring stuff’ is also your job as a user researcher. All of the videos are online now, and are obviously excellent:

Find them on the GRUX youtube channel.

Games User Research Jobs

Games User Research Jobs

A games research apprenticeship: 

This UK based opportunity is quite rare – an apprenticeship in a games research related role. The criteria for eligibility is quite niche, but if you are between 16 to 24-year-olds, not in education and are currently claiming Universal Credit this is a fantastic opportunity.

User Researcher at Xbox:

The fantastic Tom Lorusso posted about this junior-appropriate role working with his team at Xbox – take a look! 

Get 1:1 career support

I’ve enjoyed all of the office hour chats I’ve had last month and it’s been great meeting everyone. If you are after free 1:1 advice on how to start your career in games user research (or how to playtest your game!), then do book some time for a chat.

The types of things we can discuss on a call include: 

  • Answering questions you have about games user research
  • Giving feedback on something you’ve done
  • Talking about how to tackle a specific research challenge

I’m looking forward to meeting more newsletter subscribers this month!  

How To Be A Games User Researcher

The Book

The paperback version of the book was unavailable for a few weeks last month, thanks for everyone who got in touch about it.

How To Be A Games User Researcher is back in stock, and available now

As always, if you’re feeling kind, please do leave a review on Amazon – it makes a huge difference!

Help this newsletter

I’m putting together a page for the website that introduces this newsletter. I want to make it easier for people to discover it.

I would really appreciate it if you would write a few sentences about the newsletter, and how (/if) it has been helpful so far. I hope to use these to help other people discover it, and decide whether the newsletter is right for them.

Send me an email (ux@stevebromley.com) with some kind words – I would appreciate it very much! 

Thank you! 

Thanks for reading another issue of the How To Be A Games User Researcher newsletter. Personally, I have a busy month ahead – some exciting game projects to look at! 

I’ve also started thinking about my next community project for the games industry. Follow me on twitter to be the first to hear about what I’m up too (…although it’s still a year away!).

As always get in touch with questions, feedback, etc – and have a great month!

Steve (@steve_bromley)

Don’t miss the next issue

Categories
Newsletter

How to convert user research experience into a games user research job

Welcome to issue three of the How To Be A Games User Researcher

This month:

…and more! 

Every month we’ll be tackling topics important to people looking to develop their career in games, and run better playtests.

Get future issues direct into your mailbox:

Read on to learn how to become a games user researcher.

Transitioning into games from other user research sectors

This month, I wanted to go a bit deeper into how to take user research experience gained in other industries, and translate it into a games user research job. 

This is a reasonably common route that people use to join the industry, and one of the pieces of advice I give to people who are having difficulty joining the industry at a junior level is to look for a broader user research role. This helps develop very relevant experience you can bring into a games job application. 

This month’s interview is with Surabhi Mathur, a user researcher at EA who made the jump into games last year, who has some great advice based on her own experience. Before that though, let’s consider some of the differences between games user research and other sectors.

Expect friction

In most other types of software, ‘difficulty’ is to be avoided. No-one wants it to be hard to order a book from an online shop. However for games, a game without difficulty isn’t (usually) fun. Work like Raph Koster’s A Theory Of Fun For Game Design and Mihaly Csikszentmihalyi’s Flow both put ‘overcoming challenges that were previously unachievable’ as a key part of creating compelling experiences. Games need friction.

Games user researchers need to be nuanced and put in more upfront work to understand the thing being tested. They need to understand the game well enough to know what difficulty the designer intended. This usually involves a lot of time working with designers to identify what the difficulty is meant to be and what players are meant to understand or be able to do before they should be allowed to progress. They need to moderate the session and capture notes in a way that recognises ‘intended’ challenge (that has been designed) and separates this from ‘unintended’ challenge (that is accidental). 

This also means that some of the typical things a user research team working outside of games (like ‘time to complete’), are not appropriate for games. These measures are not able to separate intended and unintended challenges.

Methodological constraints

Many games have big marketing budgets, and excited fans who care deeply about the games being made. This puts them at high risk of leaks, and the consequences of leaks can be very expensive for the studio. So, secrecy is extremely important.

This includes asking participants to sign non-disclosure agreements and finding ways of actively preventing recording, such as removing their phones.

This need for secrecy has a big impact on the methods available to us. For extremely secretive games, all research has to be lab-based. This is particularly difficult for quant research.  Other types of software can be distributed to users to use at their home, with unmoderated tools, surveys and analytics to capture their behaviour. With games, the huge risk of leaks means we can’t allow players to participate at home, and we need to bring them into a research lab. This leads to the need to build labs which can hold 10-40 players playing simultaneously

For a researcher coming from other industries, this requires them to adapt their study designs so that it works as a mass playtest and adapt their moderation so that they can control a room of many players simultaneously.

Research Objectives

Games are entertainment. Often your game team want to know some variation of ‘is this fun?’.

For clear usability issues, this is fine – we can frame usability problems as ‘barriers to fun’. But it also means working out a reliable method of measuring enjoyment or fun. One approach is using a combination of quant measures, benchmarking and qualitative data to explain any quant scores. Think about the challenges of measuring fun, and how to explain the limitations of measuring fun to a game team who will want to equate it with a review score. This will be excellent preparation for a researcher joining the games industry.

Converting your experience

If you are applying from another user research role, your experience of planning and running studies is extremely valuable. Strongly emphasise this in your application. Supplementing your experience with an understanding of how games are different to other software design projects, will make you a very strong candidate. 

Tom Lorusso is a Research Manager at Xbox, and also had some advice:

(Tom’s original tweet)

I think Tom’s point, about the implications of games being art, is interesting. There is not only the impact on ‘how much people care’ about what they are making. It also affects how user researchers describe their work. 

In other fields, it’s common to talk about how user research uncovers and fulfills the needs of it’s users. In games, we often describe that our role is to ensure the game design vision is being experienced correctly. We frame this around the vision of the designer, not the needs of the user. (At least, that’s what one of the posters says!).

I personally believe that this framing will change over time, as the maturity of research in the game design process grows. That’s a subject for another time. 

As Tom says, it’s definitely a high passion environment, which creates different dynamics to working in many corporate environments. 

How I got started in Games User Research

Surabhi Mathur is an Associate UX Researcher with EA Sports, working on FIFA. In this exclusive interview, I asked about her experience moving from user research at Uber, and the challenges of transitioning into the games industry

What was your journey into games user research?

As a UXR I look for opportunities that will help me grow, expand my skills and pose exciting challenges. A job posting by EA was one such opportunity that I took up. 

When I switched from tech to games user research, I didn’t expect GUR to be so different and distinct. Tech UXR has a big, cohesive community which allows a general understanding of how UXR functions in different businesses. Since GUR is largely absent from mainstream UXR conversations, I was oblivious to its uniqueness. 

Even though my journey began subconsciously, I quickly realised that GUR has its own tightnit network of highly qualified and passionate individuals. As soon as I joined( and even before), I started leveraging the information shared by this community and it helped rapidly ramp up. 

What did you find the most challenging step of getting a role working in games? 

I faced two key challenges when trying to enter games industry:

  1. Since games/GUR is unique in the way it functions, the industry prefers people with prior experience in the field. Experience in playing/interacting with games is the next best thing one can bank upon. Since I had neither, I started interacting with different games to get familiar with game design. I also read as much literature as possible on GUR, Game Design etc which helped me understand how I could use my previous experience to draw parallels with the GUR process. 
  2. GUR is more inclined towards quantitative research than tech UXR (subjective experience) and basic quant skills are tested during the interview process. To prepare for this, refreshed all my quant skills before the interview and took a few classes to learn basic statistical languages (R, Python).

Did your experience working in UX elsewhere help with getting a job in the games industry? 

Definitely! Working in Tech made me a versatile UXR with the ability to conduct research in any given circumstances, methods, timelines, stakeholders etc. These skills helped me during my interviews because I was able to think on my feet and be creative with my approach.

What would your top tip be for someone looking to become a games user researcher?

  1. Get in touch with the GUR community and leverage its resources to get started.
  2. Play the games you want to work on or of similar genre: you don’t have to play the whole thing but do experience it’s gameplay. 
  3. Focus on your quant skills. 

Fantastic advice from Surabhi. I personally found the book ‘Quantifying the User Experience’ a really helpful introduction into learning enough stats to get by in most user research roles, if this is a topic you don’t feel comfortable with.

Last month Crystal Dynamics released their 2018 book Women in Gaming: 100 Professionals of Play as a free download to celebrate Women’s History month.

I was particularly impressed to see how heavily represented games user research was in the book. The book includes a day in the life feature from Elizabeth Zelle (who featured in issue 2 of this newsletter), a history of Carol Kantor’s work establishing Games User Research as part of game development, and profiles from other games user researchers. Read more about the book, and download it here.

Win a games user research poster and set of stickers

This month we have some of the games user research posters to give away exclusively to newsletter subscribers.

Winners get a poster of their choice (signed if you live in the UK!), and a pack of all three stickers. Designed by Chloe True, the posters and stickers are a fun way of sparking off games user research conversations – and look great in an office or home environment! 

To enter, fill out the attached form – I’ll be in touch with the winners by the end of May.

A junior/mid-level user research role

CCP are looking for a Player Researcher to join their team at Eve Online. Although the role requires a postgraduate degree, the listing on the GRUX site describes this is a junior and mid-level role. Take a look and pack your bags for Reykjavík.

Player Researcher role at CCP

Speak to experienced games user researchers

The GRUX Community

John Hopson has once again arranged a set of Office Hours for the Games Research and UX community. From his original post on the Discord:

 This is intended as a way for anyone in the community to have a chance to sit down and get advice. Feel free to offer yourself as a mentor even if you're fairly new to the profession, your experiences getting hired are probably more valuable to someone looking for their first job than mine are. Also, feel free to sign up as a mentee even if you're fairly senior yourself. Everyone needs to talk through career stuff with someone sometimes.
 Here's a sign up sheet where potential mentors can post their availability and mentees can sign up for a slot: GRUX Office Hours
 How it works:
 1) This is an unofficial experiment in trying to make basic mentoring more easily available with less obligation and commitment on the part of either the mentor or mentee.
 2) People who want to volunteer as one-off mentors should post some times they're available.
 3) Folks who are interested in having one-off conversations with a mentor should claim a time by entering their name next to it.
 4) Mentor is responsible for finding the mentee on the GRUX Discord and arranging a call at the scheduled time.
 5) The mentee is responsible for coming to the session with a set of questions and topics.
 6) Mentor is strictly a volunteer and can cancel or quit at any time if the session goes off the rails.
 7) John has no authority to do this or control over this. If you think there's a better way, make it happen and I'll sign up there too. 

This is a great opportunity to speak to some top industry professionals. Do take a look! 

Time with Steve

I’m loving the chats I’m having with people looking for advice, feedback or asking questions about the industry. However to help manage my calendar, I’m going to explore using calendly to schedule these automatically.

I’m available to talk about anything games user research related – such as answering questions about user research, reviewing some work you’ve done or discussing how to get relevant experience. Book some time for a chat here

Help GRUX Online conference

Emma Varjo (interviewed in issue one) has started the process of organising another Games Research and UX online conference for late 2021. 

The team is currently looking for volunteers to help:

Could you volunteer to make this event even bigger and better? As a volunteer you get to help shape the event: we can make it even more awesome this time around. Check the form for a list of roles and their approximate responsibilities. We have some ideas of what we need to get the conference up and running, but if you don’t see your superpower listed, we can adjust the roles or do bigger and better things! Please send us your ideas!
https://forms.gle/UbeWz4jyz8HzzF2E7

As we’ve discussed before, volunteering is a great way to start to get introduced into the community – and a way of networking without appearing sleazy. I highly recommend getting involved and becoming an active member of the community.

You made it!

That’s it. Thanks for reading. I have a few more podcasts scheduled as part of the book tour, and some games projects I’m excited about. I’m also trying to get the paperback version of How To Be A Games User Researcher back in stock on Amazon. It should be up again in a few days.

I’d love to chat about games user research, and what would be helpful for you in this newsletter. Do email me by replying to this newsletter if you have any questions, or drop me a line on twitter!

If you’ve enjoyed this newsletter, there are two very helpful things you could do for me:

  1. Share it with people you think would find it helpful
  2. Leave a review for the book How To Be A Games User Researcher on Amazon.

I’d be extremely grateful if you decide to do either of these, as both are very helpful!

Have a great May! 

Steve (@steve_bromley)

Get the latest information on how to become a games user researcher direct to your inbox

Categories
Newsletter

Why do games user researchers start by defining research objectives?

Welcome to the second issue of the How To Be A Games User Researcher – and thanks for everyone’s kind words about issue one

  • Why is defining research objectives so important?
  • Represent games user research on Zoom
  • Elizabeth Zelle’s journey into games user research 
  • What interview questions should you be prepared for?
  • An entry level role with Warner Brothers Seattle

…and more! 

Get future issues direct into your mailbox

Every month we’ll be tackling topics important to people looking to develop their career in games, and run better playtests

Read on to learn how to become a games user researcher.

Games User Research Tips

Why do we start by defining objectives?

User research studies, or playtests, can give reliable answers to many useful game development decisions. They are strongest with questions around behaviour or understanding such as “what do we need to tell players”, “will players understand what they are meant to go” or “can players complete this challenge”. The right playtest can also answer harder questions, like “Do players like my game”, but this requires more skill to reliably answer.

The simplest playtests involve watching people play, and asking them questions. This reveals issues where they are not experiencing the game as intended, and help us identify what needs to be changed to improve their experience. 

However in order to design appropriate tasks, or pay attention to the right parts, we need to know what we want to learn from the study. Researchers call these the ‘objectives’ of the study.

Coming up with objectives is best done regularly throughout development by understanding what the team have been working on recently, and working with them to describe how they think players should experience it – what do we want players to understand, think and do. You can then use this information to decide what you need to measure, to check if they are experiencing it as intended.

I wrote more about how to define research objectives in the book, and as a free extract on the website. Read more here to learn how to come up with research objectives for your study.

Represent - Fight For Better Games

Educate people about games user research

Last month, I shared the posters and stickers created for the book by Chloe True which promote some key games user research principles.

Jess Tompkins (who also helped with the book!), suggested making them into Zoom backgrounds. So we did! 

Jess Tompkin's Tweet - Research Makes Games Better

Link to Jess’s tweet

Hannah Murphy's Tweet- Game Design Is Revealed By User Research

Link to Hannah’s original tweet

The zoom backgrounds are free to download from the website, alongside free printable posters to advocate for user research.

Get the free zoom backgrounds here

Do share your pictures of the backgrounds if you pick them up! 

Competition - Win Free Games

Congratulations to last month’s winners.

Last month we gave away two copies of ‘The GoD Unit’. Congratulations to Alex and Jel who each won a copy! 

It’s too late to enter, but if you’re feeling helpful I’d still love to hear about you on the survey link, so I can continue to tailor the newsletter contents

We’ll have a new competition soon! 

Career - Get Your First Job In Games User Research

How I got started in Games User Research

In the GRUX Discord, their regular QOTW feature asked last year how people got started in games user research. There were many great answers, including from Elizabeth Zelle, User Research Manager at Amazon Games. I caught up with Elizabeth – here’s her journey and a top tip for other people following in her footsteps! 

Elizabeth Zelle – User Research Manager at Amazon Games

Follow Elizabeth on Twitter

What was your journey into Games User Research?

In 2008 I applied to work in the QA department of Volition. At the time I had a number of friends from high school already working in QA there, and they encouraged me to apply. It was the middle of the recession and jobs in marine biology (what my actual degree was in) were few and far between, so there was nothing to lose. I got hired fairly easily, but I will admit that networking and name dropping my mom (who had worked at Volition years before) totally played a role there.

I was laid off a year later, but was in one of the first rounds of rehiring in 2010 when they started staffing up again. I was so high up the rehire list because I’d proven my skills, both in doing QA and in being a good team member, and people in the department specifically wanted me back, which felt good. 

I transitioned to User Research a little while later when I discovered it was a great fit for my skillsets and matched well with my interests. The only problem was that we didn’t have a researcher opening. So I approached the solo researcher at the company and told him that it looked like he had enough work that he needed an assistant. Luckily for me, he agreed!

My takeaway: Getting that first job was easy but also a lot of luck; getting rehired after the layoff was based on skill. Being detail-oriented, having strong note-taking skills, and being highly observant all were traits that helped me excel in QA and then transition over to UR (not surprising, since those are all skills that research work in college had hammered into me). 

My advice: Self-assess and identify which skills are your strengths, and then own them. Seek out the opportunities that you want and shoot your shot; don’t count yourself out before trying. Make friends; being someone that people value and want on their team will always benefit you.

What interview questions should you be prepared for?

Members of the GUR community discussed questions that could come up in games user research interviews. 

Callum Deery (@CallumDeery2 on twitter) shared some practice questions he’d gathered for UX applications.

Seb Long (@seb_long on twitter)  spoke to hiring managers about their interview questions for Games User Research jobs back in 2015.

Juney Dijkstra (find Juney on LinkedIn) shared some links she’d collated including:

Although not games specific, I’ve previously found this medium article helpful to think about the range of technical and situational questions a user researcher may be asked.

All of these links were originally shared on the #job_hunting_chat on the GRUX-DIG discord. Do join the community and continue the conversation! 

An entry level user research role

Warner Brothers are looking for two user research assistants for their Seattle team to help conduct research studies, including moderating studies and observing player behaviour. The role includes training but is ideal for people who have had previous experience running research studies with people, potentially gathered through a university degree.

Find more details, and apply here.

Help the Games User Research conference

The GUR conference is looking for volunteers for their 2021 summit. Find more details and sign up here.

Helping at the conferences is a great first step for getting involved in the community, so I’d recommend taking a look!

You made it!

British Pub - You Are Not Your Players

That’s it. Thanks for reading. For me personally, I’m continuing on the book tour (I’m popping up on many podcasts), and have some exciting game projects coming up in April so I know it’s going to be a good one! 

I’d love to chat about games user research, and what would be helpful for you in this newsletter – do email me by replying to this newsletter if you have any questions, or drop me a line on twitter!

If you’ve enjoyed this newsletter, there are two very helpful things you could do for me:

  1. Share it with people you think would find it helpful (there’s a share button just below)
  2. Leave a review for the book How To Be A Games User Researcher on Amazon.

I’d be extremely grateful if you decide to do either of these, as both are very helpful!

Have a great April! 

Steve (@steve_bromley)

Found this issue helpful? Sign up to receive the next issue here:

Categories
Newsletter

Test early and often

How to run better usability reviews, educate people about research, a top puzzle game to give away & Emma Varjo’s journey to becoming UX lead at Frozenbyte

Welcome to issue one of the How To Be A Games User Researcher newsletter – we’ve got a jam-packed issue this month including topics such as: 

  • How to run better usability reviews
  • Educating people about research
  • A top puzzle game to give away
  • Emma Varjo’s journey to becoming UX lead at Frozenbyte 

…and more! 

Get future issues direct into your mailbox

Every month we’ll be tackling topics important to people looking to develop their career in games, and run better playtests

I will start with a thank you. The response to the launch of the book has been overwhelming. I’ve loved seeing everyone’s photos shared on Twitter and LinkedIn – and am really happy everyone’s finding it useful. There’s plenty more to come, so stay tuned. 

Let’s get stuck into the first issue! 


Why do games user researchers always talk about ‘testing early and often’?

“It’s not yet ready to test” is a phrase I hear frequently from game teams. It is also a phrase I shouldn’t be hearing from teams who understand the value of playtesting. 

It’s very common to wait to playtest. Teams assume it’s only useful near the end of development. Sometimes developers know that bugs exist, which they believe will stop players from progressing. Other times developers hold back because the art isn’t finished yet, and they are worried people will think the game is ugly.

Although the game isn’t polished, testable bits of the core experience exist very early in development. And this creates a risk – the longer these go untested, the more ‘baked in’ they become, and the harder it becomes to change them. When problems are only discovered late in development, it becomes very expensive to change them. A lot of work needs to be thrown away and redone.

The safest approach is to expose the core mechanics to players early, from the first prototype. Watch people play, and ask questions to check if players are understanding the mechanics in the way you hoped. Check that they are able to use them as expected. You might even see interesting behaviour that can inspire new gameplay ideas.

I loved this Twitter thread from Katie Chironis, giving a concrete example of how this could work for Dark Souls, and how it’s possible to create and evaluate the core experience very early on.  

Moderated usability testing makes this easy

By sitting with players as they play, or streaming their session live, you can help them work around bugs, and ask them questions about the things you do care about such as their understanding of what’s happening – avoiding irrelevant comments about the unfinished assets. Let’s make sure our colleagues understand they can come to us from the very start of development – more on how to do that coming soon! 

How to run better usability reviews

Last year Seb Long, of Player Research fame, ran an exercise where he reviewed a whole bunch of usability ‘expert’ reviews from people at the start of their career and noticed some patterns in the errors. This featured at the games user research summit in 2019.

There is a huge wealth of valuable information here about the mistakes people make when running usability reviews, packed into 60 minutes. Do take a look! 


Educate people about games user research

In the book, we talked about how it’s not enough just to run good user research studies. I hear from many frustrated researchers that they are upset teams don’t come to them earlier in development, or don’t react to their findings.

Communication is just as important as running studies for user researchers.

To help spark conversations about research, I asked designer Chloe True to create some posters which advocate and educate colleagues about games user research principles.

These are available free to download from the gamesuserresearch.com website. I’ve also partnered with Printful to offer professionally printed stickers and posters, which (I believe) they deliver worldwide. I’ve framed my own copies at home, and they look excellent! 

Get the posters and stickers

Take a look at the website to find download links and a link to order the posters and stickers. They’d look great in your office or game studio! 


Win a copy of The GoD Unit

I’m really happy to have had the opportunity to work with the team behind ‘The GoD Unit’. They’ve allowed us to use in-development builds for some of the updates coming soon to follow the book, which I’m extremely grateful for.

The GoD Unit is a first person puzzle game, suitable for fans of The Talos Principle, or Portal. It’s one of my favourite genres, and The GoD Unit definitely had me stumped at points – nice and difficult! 

They’ve kindly given us two copies to give away on Steam to newsletter readers. 

To enter, fill out this short form. I’d also love to hear a bit about you (although the questions are not mandatory) so I can make the contents of these newsletters more relevant to you! 

Win a copy of The GoD Unit

I’ll draw two winners in the middle of March, and contact the winners before the next newsletter. Good luck!


How I got started in Games UX

In the GRUX Discord, the regular QOTW feature asked how people got started in games user research. There were many great answers, including from Emma Varjo, UX Lead at FrozenByte games. I caught up with Emma – here’s her journey and a top tip for other people following in her footsteps! 

Emma Varjo – UX Lead at FrozenByte

Follow Emma on Twitter and LinkedIn

What was your journey to games user research and UX?

I didn’t know what I wanted to do after high school. I figured I like math, languages, and videogames, so I should study programming. I’d either end up working on games (dream) or making a good living (also acceptable), as long as it didn’t turn out I hated it.

I nearly quit during my first semester, but once I got it, I felt I could do anything the program threw at me. I did struggle with all new programs we needed to learn in order to do the work and felt it was dumb it was so hard to just get stuff done. Others told me it’s normal that you just have to learn the tools. I felt so happy when I discovered HCI and the people agreed with me! 

I tried to get internships in games studios, but thanks to a general economic downturn practically all studios in my city shut down and I could not afford to move cities for the summer and pay rents for two apartments so I could do an unpaid internship in games. Besides I could get paid nicely interning in other tech companies. 

Years later I was working in tech as a programmer and looked at games jobs again as there were more of them in the capital. Most were unpaid internships unless you had games experience. One AAA studio looked for a GUR person with “5+ years of experience in UR and at least one shipped game.” I did not apply, but in hindsight I really should’ve. 

Frozenbyte had all applicants do a task they’d have to submit with the application. For game design it was an expert evaluation and a user test of a demo of their game. Right up my alley! They replied months later saying it was one of the best applications they ever got, but couldn’t hire anyone at the time, but I should try again in the future. I took another job and forgot about it. A year later I was burning out in my new job and FB emailed me and asked if I was still interested. Heck yeah I was and here I am!

What’s your top tip for people looking to get started in games user research?

Empathy and communication skills are paramount. 

I was remembered a year after the initial conversation not only because I did a good evaluation and user test on their demo, but because I empathized with the designers reading my application and wrote my report on a way that it was easy for them to find where the issues were, how severe the issues were, as well as some ways to possibly fix them. (I was applying for a UXD position, so it was important to have – for our GUR that can be dangerous.) 

When you’re breaking in, and ever after too, it’s important to use your skills not only to do great research but also affect change based on it. 

Apply to work at Nintendo

You don’t hear much from the user research team at Nintendo (I’d love to see them pop up at a Games User Research conference). But here’s a rare user research internship with their American team.

Find more information about the Nintendo internship here.

It’s not explicit whether this will be working on games, or their digital store/website – but some interesting challenges regardless!


*Whew*. That was a long first issue. I’d love to hear what you think! 

Please do fill out the survey (in the competition), as it’ll help these newsletters become more relevant to you in the future. You can do it without leaving your email address, if you don’t mind not winning the game! 

Also I’d love to chat about games user research, and what would be helpful for you in this newsletter – do email me by replying to this newsletter if you have any questions, or drop me a line on twitter!

Oh, one last thing – don’t forget to get your ticket for the #gamesUR summit in May – rescheduled from last year. It’s a great line-up (and some amazing value swag ☕📓), so do pick up a ticket! 

Have a great March! 

Steve (@steve_bromley)

Don’t miss an issue

Every month we’ll be tackling topics important to people looking to develop their career in games, and run better playtests