This past month has been characterised by lots and lots of playtesting. Testing in my house, testing at friends, testing in the office, testing at gaming events, testing on tabletop simulator, testing by lending out prototypes, testing by making print and play files available. I found that I’ve started writing things at my notes along the lines of “X is a problem because we’re playtesting here” and thinking more about how the venue for a playtest is affecting the outcome of that test and how to interpret the data for developing a game that’ll predominantly be played in different environments.

It’s well known that the relationship of the testers to the designer and the presence or absence of the designer during the test has a significant impact on how testing goes. There’s not a “right” way to do things, but it’s important to recognise how presence impacts a test in both positive and negative ways, particularly when trying to get an idea for how finished something is.

If a game is going well and the players are really getting into it they’re likely to “forget” that the designer is present (or that they’re being video’d – there’s a marked difference between recording people playing a game and recording people who’re concious that they’re performing for a camera while playing a game). The ease with which this happens is affected noticeably by the venue, if someone’s in your house they’re not very likely to forget that you’re there. At a gaming event it’s easier to become unnoticeable. In digital environments it’s trivial.

When testing in an environment that makes your presence more obvious it’s important to adjust the valence of all feedback down by an order of magnitude. “It’s great” becomes “It’s good” and “It’s okay” means something is wrong. Most people don’t like to make other people feel bad at will at least soften their criticism in the direct presence of the creator of the thing being criticised. A few people pride themselves on not doing so, but they have a minority impact.

Focus also has an important impact on how a game plays. If a group is all concentrating primarily on the game then it’ll play very differently to if they’re interested in other things. Games can have various levels of resilience to this sort of distraction – in some games if you stop to talk to someone for five minutes then the game takes five minutes longer. In others the distraction makes you lose track of things and need to do more mental work to get back to where you were and the delay is multiplied.

A game that’s “about the right length” in most settings can become “far too long” in a social setting. Particularly in games clubs where the regular attendees know each other and drop in and out throughout the day needing to say their ‘hellos’ and ‘goodbyes’ as this happens. Obviously it is better for a game to have perfect resiliency than not, but the ability to maintain pace despite interruptions is less important than a host of other issues – it’s important not to let a playtesting environment dictate the relative importance of factors in your game. If the game will only ever be played at conventions and gaming clubs then there’s a way to optimise a design for that environment, but if you’re planning on a general release that’s primarily played between friends then optimising for that will hurt other aspects of your game that matter more to your target audience.

Playing in more public spaces also gives a better idea of how a game is perceived by a casual onlooker and how it’ll first appear to a lot of people. Running a playtest at someone’s house will let you see their reaction to your prototype once – and only their reaction to it as a static thing in a box rather than a dynamic thing being played. Running a public space will show you how people see it a dozen times.

Your environment can also have a strong impact on the motivations of playtesters for taking part. In most gamer centric environments I’ve found that the main motivations people have shown have been a curiosity about what the game is, a genuine desire to help something be better and seeking the chance to make a mark on the world in some small way. While these motivations play out slightly differently I’ve found moving to more exotic environments produces testers with different motivations which in turn generates different sorts of feedback.

In my work with The People’s Orchestra I’m fortunate enough to have playtesters with me in the office – a rare luxury for a designer. Generally these are volunteers who are present due to some goal that’s got nothing to do with games, often looking for routes into paid work, but they’re not so much motivated to play games as they’re motivated to do what our mutual boss tells them to.

These tests are slightly unusual in that people who aren’t really in to games don’t conceptualise things in the same way as games fans. It’s much more necessary to look for times that they’re unhappy with a thing and probe into why that is, suggesting reasons and seeing what sticks and what doesn’t. Normally I wouldn’t feel the need to ask if something’s a downtime issue because games fans, even if they lack that language, will express something like “I was bored between turns” but total non-gamers will sometimes express something like “I didn’t understand the game” which will trace back to “I didn’t want to understand it” which will trace back to “I was bored” which will trace back to “I understood the first couple of turns but got fed up of paying attention to stuff that happened when it wasn’t my turn.”

The digital environment produces a different sort of response. There are a lot of people on TTS flitting from room to room looking for things that entertain them. Some are looking for things to fill a slot before a big game they’re looking forward to starts, or are in a different timezone and are online for a few moments before bed or similar. Attention is easily gained, but just as easily lost.

In this environment – and to a lesser extent at conventions – it’s vitally important to be playing the game within a couple of minutes of someone expressing an interest. Most gamers are gaming nights (as opposed to conventions) like to know all of the rules before they start playing, so that they can make informed decisions and start to strategise. I’d got used to explaining games in this context and took that approach on TTS which lead to what seemed like a very high refusal rate in which players listened to thirty seconds of explanation and when that didn’t encompass the whole ruleset went “Nah, I’m out” and logged off.

I switched to a strategy of saying “This game is about gods creating a world, you pick four of those piles to represent your new world, the title on the bottom left says what it’s about. Which four do you want?” and people’d immediately jump in saying “I’ll take catastrophe” or “Give me love”. In meatspace players generally seem to want to know what the rest of the numbers and images on the card means, but in a digital environment it seems far more popular to just jump in.

People also like it when a test is fair, generally in the real world I set up groups that don’t include me so a game is entirely populated by newbies. In a digital environment it seems that players are willing to wait about three seconds for someone else to join and will look for a different game if it’s going to take longer than that. To counteract this I’ve taken to telling people in advance “Don’t worry about all of the rules, I’ll introduce each one as it comes up, but to keep things fair I’ll make my play decisions based on a dice rolls until you know all of the rules”. That seems to keep things level, since in most games it’s eminently possible to crush someone who doesn’t know what they’re doing if you chose to – but in this environment the preference seems very much for starting playing without knowing what you’re doing.

The physicality of the environment also makes a degree of difference. This isn’t tremendously noticeable in physical environments. Public gaming is more prone to physical disruption than private gaming so some venues will notice a slight difference in feedback based on whether knocking a few pieces around is something that’s quickly corrected or irrevocably ruins the game – but the difference is really very slight. Digital playtesting on the other hand involves people interacting with components in a radically different way, which can significantly alter feedback.

For instance there is a card in Genesis that has received no complaints in the real world, but is decried every time it comes up in TTS. It allows the player to search their deck for any card and add it to their hand. A player’s deck cannot possibly be more than nine cards and very often will be smaller so as a real operation this is rapid. However as a computerised operation you can’t hold a deck and flick through the cards in the same way, so it takes longer and is more inconvenient – to the point that some players just choose not to use the ability. As I’m developing primarily to make a real game I’m almost certainly not going to remove the card due to this drawback, which I suppose comes back to the fundamental point of this article: Which is to recognise when a playtest environment is different from the environment in which your game will ultimately be played and to reduce the importance of feedback that’s tied to those differences.

The point could also be reversed towards designing for a specific environment. It’d be interesting to see what could be done designing a game specifically for TTS. As some operations are more difficult, some are much easier. There’s no limitation on components due to manufacturing costs. It’s trivial to count the cards in a deck. You could shuffle a sixty card deck three times a turn without noticeably slowing down play.

Similarly convention focused games exist, that do things with the nature of drop in and drop out play or having an audience or simply that get something out of there being a convention atmosphere surrounding them.

The take home point is that environments change games and that when you’re designing a game you need to be aware of the features that you’re designing for. Take advantage of the specifics of that environment, the more unusual it is the more it’ll impact on your design and let you do exciting, creative things that are beyond the scope of most games. Conversely if you’re designing for a “typical” environment (and don’t get lazy thinking about what that is – even if it’s being played in the home what that means varies depending on the characteristics of your target audience) then be aware of the characteristics of the places that you playtest and be sure to take it into account when interpreting the feedback that you get.

You want to be able face your testers and truthfully say “I see where you’re coming from…”