Getting playtesters together is hard work. It gets harder if a game is long or if you’re past testing with friends and are looking for more input from strangers. Playtesting is the most critical part of making your game any good, it’s not somewhere that you can cut corners. Playtests are a valuable opportunity so it’s important not to waste any of them.
This is distinct from wasting playtesters themselves: A process that results from putting incomplete games in front of people, taking negative feedback with poor grace and failing to provide snacks. (I’ve written about it before but the 341 strong archive has become unmanageable, I must put the time into indexing it properly soon so I can find the link at moments like this.) This post assumes that you’re treating your testers well and that they’re happy to participate – but focuses on ensuring that you get the most out of your time together.
The first steps occur before the test starts, you need to decide on exactly what it is that you’re trying to test. It’s perfectly possible early in the process that this might be as broad as “The game” but as testing goes on you’re more likely to be considering whether particular tweaks are having their intended effects. You should also decide on how much you’re willing to change as a result of the tests. Imagine various outcomes, are you willing to throw away the design? Scrap vast swathes of the design? Change a few mechanics? Tweak individual cards and powers? It’s worth knowing what you’re willing to change before you get feedback.
Having defined the scope you should select testers based on it. If you want to find out if the tweaks have made the game more approachable for new players it’s a waste of time to reuse testers from previous rounds. Conversely if you want to deeply understand the impact of a change it can be helpful to get feedback from people who played before and after the change (though ideally not anyone who was strongly in favour of or against it).
The testers then need to know the scope! A person can only pay attention to so much at once and testers will do a lot of self-censorship – it’s good to be able to say “This is very early, I might lose whole chunks of this, let me know about anything that’s not working.” or “This is nearly done, I’m looking for suggestions on which card abilities are working best but I’m not entertaining changing the core mechanics around how you get and use those cards in the first place”.
A focused test will provide you with higher quality information on the focus of your test. Be careful not to confuse focus with desire though – “I want to know whether the game is ready” is different to “I want testers to tell me that the game is ready”. Pressuring testers to give positive feedback (or even being neutral with people who have already invested time and energy into this game) will just lead to you missing out on opportunities to improve the game.
When sitting down to test it’s important to remember that the goal of the exercise is not to enjoy your game – it’s to improve the game to the point that lots of people will enjoy it. This can mean taking shortcuts in order to test the parts of the game that most need it.
Suppose you have a battle game in which the player who gets the last turn always wins – because they can attack with no regard to keeping their defences intact and that’s enough of an advantage to swing a win from most positions. If you make a few tweaks to fix that problem – say making the exact timing of the game end unpredictable – you don’t really need to play the game from start to end to test if that’s working.
It’d be better to play the last 20 minutes of the game six times than a full two hour game once. There’s no reason you can’t do that – come up with six “80% complete” positions that you reckon are plausible (from your previous testing) and just play the last 20% of the game until you’ve got the information that you need about the part that you’re trying to tune.
I’ve not been involved in the development of a legacy game (Though have some fantastic ideas if I ever had the backing to do one) but even with longer games like Wizard’s Academy found it helpful to section the game out and test in chunks. In part that sectioning is what made it so easy to move to “expert scenarios” that start the game in medias res once it became apparent there was a divide between players who enjoyed the slow build up and ones that wanted to jump straight in at the action.
It may be a helpful exercise to get into the habit of mentally dividing your game up into phases – even if nothing formal changes play between them. Having the lexicon to realist that (for instance) the game is slow between “The phase where there are still empty spaces not yet claimed” and “The phase just after someone has hit a top tier tech” makes it easier to recognise the problem. Having done so it’s easier to implement a fix and test that fix by rapidly iterating through the surrounding phases.
Another way to get more out of each playtest is to develop a willingness to change things on the fly. It’s good to spend a lot of time thinking about your games and to fully understand the ramifications of any change that you make – but trying a different idea should be seen as less of “making a change” and more as “getting more data to inform the change that you’ll eventually make”.
“That’s an interesting idea, is everyone up for playing a couple of turns using that rule?” is a great phrase to be able to say. Often it’s not a big time commitment and it lets you immediately see how a sort of change might play out – even if the idea turns out to be terrible you learn something by the experiment that can be applied in other ways. It’s also got the fringe benefit of letting your testers know that you’re listening to them and interested in what they have to say.
But by far and away the absolutely best way to avoid wasting a playtest session is to make sure that you record everything! It’s super easy to have a wonderful session in which you make loads of precise observations and the testers are filled with brilliant suggestions and you’re absolutely sure that you couldn’t possibly forget something this brilliant – only to find that by tomorrow it’s gone.
Write everything down.
Video it if you can.
Once the testing is done, there’s still one more thing you can do to help avoiding wasting time with your next test – that’s to maximise what you’re going to get out of the next test. It’s good to be sure of changes that you make and not to weight any test too heavily – a game might never reach a stable state if you totally rebuild it following each and every test.
However sometimes one is enough. You can prove to yourself mathematically that yes, the thing that one tester said is true and will be a dominant strategy in every game after a player has spotted it. If you’re at a point that you’ve got something that you’re sure you need to address – don’t wait! Get some sort of response down before the next test, so that the test is giving you some feedback on your tweak rather than just confirming what you already know: That you need to make one.
Those are my top tips for avoiding wasting playtests – what’ve I missed? How can I squeak even more progress out of each and every one?