Test early and often

I will start with a thank you. The response to the launch of the book has been overwhelming. I’ve loved seeing everyone’s photos shared on Twitter and LinkedIn – and am really happy everyone’s finding it useful. There’s plenty more to come, so stay tuned. 

Let’s get stuck in! 

Get future issues direct into your mailbox

Every month we’ll be tackling topics important to people looking to develop their career in games, and run better playtests

Why do games user researchers always talk about ‘testing early and often’?

“It’s not yet ready to test” is a phrase I hear frequently from game teams. It is also a phrase I shouldn’t be hearing from teams who understand the value of playtesting. 

It’s very common to wait to playtest. Teams assume it’s only useful near the end of development. Sometimes developers know that bugs exist, which they believe will stop players from progressing. Other times developers hold back because the art isn’t finished yet, and they are worried people will think the game is ugly.

Although the game isn’t polished, testable bits of the core experience exist very early in development. And this creates a risk – the longer these go untested, the more ‘baked in’ they become, and the harder it becomes to change them. When problems are only discovered late in development, it becomes very expensive to change them. A lot of work needs to be thrown away and redone.

The safest approach is to expose the core mechanics to players early, from the first prototype. Watch people play, and ask questions to check if players are understanding the mechanics in the way you hoped. Check that they are able to use them as expected. You might even see interesting behaviour that can inspire new gameplay ideas.

I loved this Twitter thread from Katie Chironis, giving a concrete example of how this could work for Dark Souls, and how it’s possible to create and evaluate the core experience very early on.  

Moderated usability testing makes this easy

By sitting with players as they play, or streaming their session live, you can help them work around bugs, and ask them questions about the things you do care about such as their understanding of what’s happening – avoiding irrelevant comments about the unfinished assets. Let’s make sure our colleagues understand they can come to us from the very start of development – more on how to do that coming soon! 

How to run better usability reviews

Last year Seb Long, of Player Research fame, ran an exercise where he reviewed a whole bunch of usability ‘expert’ reviews from people at the start of their career and noticed some patterns in the errors. This featured at the games user research summit in 2019.

There is a huge wealth of valuable information here about the mistakes people make when running usability reviews, packed into 60 minutes. Do take a look! 

How I got started in Games UX

In the GRUX Discord, the regular QOTW feature asked how people got started in games user research. There were many great answers, including from Emma Varjo, UX Lead at FrozenByte games. I caught up with Emma – here’s her journey and a top tip for other people following in her footsteps! 

Emma Varjo – UX Lead at FrozenByte

Follow Emma on Twitter and LinkedIn

What was your journey to games user research and UX?

I didn’t know what I wanted to do after high school. I figured I like math, languages, and videogames, so I should study programming. I’d either end up working on games (dream) or making a good living (also acceptable), as long as it didn’t turn out I hated it.

I nearly quit during my first semester, but once I got it, I felt I could do anything the program threw at me. I did struggle with all new programs we needed to learn in order to do the work and felt it was dumb it was so hard to just get stuff done. Others told me it’s normal that you just have to learn the tools. I felt so happy when I discovered HCI and the people agreed with me! 

I tried to get internships in games studios, but thanks to a general economic downturn practically all studios in my city shut down and I could not afford to move cities for the summer and pay rents for two apartments so I could do an unpaid internship in games. Besides I could get paid nicely interning in other tech companies. 

Years later I was working in tech as a programmer and looked at games jobs again as there were more of them in the capital. Most were unpaid internships unless you had games experience. One AAA studio looked for a GUR person with “5+ years of experience in UR and at least one shipped game.” I did not apply, but in hindsight I really should’ve. 

Frozenbyte had all applicants do a task they’d have to submit with the application. For game design it was an expert evaluation and a user test of a demo of their game. Right up my alley! They replied months later saying it was one of the best applications they ever got, but couldn’t hire anyone at the time, but I should try again in the future. I took another job and forgot about it. A year later I was burning out in my new job and FB emailed me and asked if I was still interested. Heck yeah I was and here I am!

What’s your top tip for people looking to get started in games user research?

Empathy and communication skills are paramount. 

I was remembered a year after the initial conversation not only because I did a good evaluation and user test on their demo, but because I empathized with the designers reading my application and wrote my report on a way that it was easy for them to find where the issues were, how severe the issues were, as well as some ways to possibly fix them. (I was applying for a UXD position, so it was important to have – for our GUR that can be dangerous.) 

When you’re breaking in, and ever after too, it’s important to use your skills not only to do great research but also affect change based on it. 

Don’t miss an issue

Every month we’ll be tackling topics important to people looking to develop their career in games, and run better playtests

2 replies on “Test early and often”