This morning I came across this post from Kevin G Nunn, I think via a twitter notification (It turns out that I’m really bad at twitter, I haven’t posted anything at all this year and can’t remember my login details.) In it he describes a designer using the line “Give me your problems, not your solutions.” and the arguments in favour of getting playtesters to report feedback in this manner.

I found this interesting because it’s a view that I subscribe too, I’m far more interested in the problems my playtesters encounter than the solutions that they generate to these problems. I found myself agreeing with some of the reasons that the other designer had identified to Kevin. To quote his article:

“First, he expressed that the suggestions playtesters give are usually bad.  Second, that while the playtester is relating the suggestion, he was generally running other–presumably better–variants in his head.  Third, that playtesters tended to get attached to their suggestion and resentful if that suggestion isn’t used in the final design.  Fourth, he stated that the playtesters could not see the inner workings of the design and were therefore ill-equipped to make informed suggestions.  Fifth, that many players don’t want to raise a problem without having a solution to offer as well so asking players to only express issues freed them to raise issues to which they see no solution.”

The rest of the article started on the point by point takedown of these issues and promised to conclude it over the week. I won’t repeat his article here, you can go and read it straight from the source. I agree with his objections to the first two points and can see how one could very reasonably disagree with the remaining points. This teaches us two things: Firstly I am very suggestible and despite being treated as a reliable source will apparently always believe whatever has been said to me most recently.

Secondly, I need to examine why I like “Give me problems, not solutions” and reconcile that with the view that a lot of the justifications for it are nonsense (Especially that “the designer could be thinking of other things” one, nevermind efficiency, it’s just plain rude to want people opposite you to stop speaking because what you’re thinking is more valuable than them.)

Ultimately it comes down to what the outcome for the game is, depending on whether the playtester reports the problem or the solution. So if a playtester finds a problem and I could generate a good solution but they also generate a solution and it’s not very good then we…let’s draw a table.

It’s pretty even whether producing a report of a problem or a report of a potential solution leads to a good outcome. The deciding factor is whether the playtester or designer is likely to have a better solution, I’d argue that the designer has a much better chance of generating a good solution, since they’ve seen many more playtests, understand the game much more deeply and generally have more information to work with (and more capacity to test their solutions until they get it right!)

So case closed! Well, not quite, there are two more things to account for. The first is whether the designer is capable of deriving the problem from the proposed solution, if you can see what problem the playtester is trying to solve then you get the best of both worlds, you can use all solutions and pick whichever one is best. That comes down to the perceptiveness of the designer (which I once argued was the most important quality for a designer in an interview) so depending on the designer a solution may be preferable.

The second minor problem is that the premise of the entire thing is based on an informal logical fallacy and a large portion of the above represents a serious thinking error. Specifically false dichotomy.

Why can’t a playtester report both the problem that they experienced and their proposed solution? I think that this is the crux of the matter. I think that it’s valid to be more interested in problems than solutions so “Give me your problem not your solution” appeals to me, but I acknowledge that it would be best to have both so arguments against that absolute also ring true.

It’s not always possible to get everything that every playtester has to offer. In group conversations if someone dominates the conversation they tend to shut out other opinions and you can lose access to them. A solution often takes a lot of floor time to express compared to a problem. I tend to prefer to split up groups of testers to have one to one conversations, but again time becomes an issue. If you spend a lot of time hashing out a proposed solution then people tend to skip over other things they wanted to say.

There are also times that playtesters aren’t aware of the problem, I’ve had testers suggest changes to the game and when I’ve asked “Why do you want to make that change?” they don’t know. In these cases you need to listen to the solution to have a chance of grasping what problem they’re trying to solve.

So I guess it comes down to this: A problem report is more helpful than a solution report, get both if you can, but gear your questions and approach towards the former. You might lose something important by saying “Give me your problems, not your solutions.” but it’s probably worth saying “Give me your problems.”

(Unless their problem is an angry housecat.)