It’s often hard for production to prioritise user research & playtesting. It requires an upfront investment, often of tens of thousands of pounds, to generate savings later.
The financial (and time) cost is immediately felt, especially when teams need to stop to polish up a build ready for a test. But the benefits are abstract and in the future – and easy to leave as a problem for ‘future you’.
We know user research and playtesting will make production easier later on. Eventually the problems will be surfaced, and many of them will create additional unplanned work. There are many stories from the industry of late minute changes creating crunch, throwing away work already created, or expensive last minute changes. I wanted to explore ways to make that trade-off visible.
To help with that, I’ve put together a small tool that will allow you to demonstrate how much money is saved when running a test and discovering issues earlier. This builds on the idea from the much-cited IBM paper that “fixing a defect post launch costs 100 times more than preventing it during the design phase” – but allows us to adapt it to our context of game development.
Games are complex, and research return on investment cannot be proven with a single calculation. I’m hoping this tool can help surface the potential cost of re-work, and lead to better conversations about the cost of playtesting.
A tool to estimate THE SAVINGS FROM playtesting
After running a test, entering the number and severity of usability issues discovered will give a potential cost save, based on how far through development you discovered them (compared to resolving them at launch)
The ‘true’ cost save is very dependent on your context (and ultimately impossible to know) – factors such as the genre, the team’s experience, and the amount of technical debt that has built up. Every assumption the tool makes is editable, so can be adjusted to your team’s situation – for example the day rate of a team member, or updating the estimates of how many issues get fixed when discovered. I’ve started with some defaults tuned for a ‘generic’ AA or AAA team, but you’ll want to make changes where you know the data – especially for smaller teams.
The tool stores no data (and doesn’t ask you for any contact details), for safety.
It’s a beta – starting to play with this idea, so I’d love to hear your feedback/experience with it. Get in touch if you find value in it, or with other feedback.
Loading calculator…
The big caveat
Ultimately showing the value of ‘a path we didn’t take’ is impossible – we can’t genuinely know what would have happened if the team hadn’t discovered the issue until later. But these figures, taken as an estimate, might be useful internally for discussing the value of playtesting and visualise the cost of deferring playtests until later.
I’d love to hear your experience with this internally – do let me know how you end up using it.
