Game Playtesting: the Heart of Game Design

Unlike most other works of individual art, games are altered by the user/consumer. Hence, the designer loses a lot of control over the user's experience. The best way to regain some control, to at least eliminate undesirable results, is to test the game with lots of players from the target market, and modify it to remove that which is undesirable. That's why we playtest.

This is a summary I created for aspiring game designers in general. I've added it here, though we'll go into more detail about most of these points as we go through this course.

We're interested in testing to make sure that the way a game works, is interesting and enjoyable ("fun testing"). This contrasts with more traditional (in video games) bug testing, which is to make sure the software works the way it was designed to work.

From my "Learning Game Design" course.

Yes, every prototype (and many published games) is broken in some way. But do you fix enough of your brand-new prototype that when other people play it, they can more or less enjoy it? This requires solo testing and fixing, no way around it.

I decided it was worthwhile to make a video just about how much solo playtesting you might expect to do, before asking anyone else to play your game.

Categories of playtesters and how they differ, and where to find playtesters.

From my "Learning Game Design" course.

Metrics is the technique, sometimes becoming a near-fetish, of using a large array of computer-monitored measurements to modify a video game that’s either in “early access”/released, or in a widespread beta. It amounts to a form of A/B testing common on Websites

To me, it easily becomes a form of “throw things against the wall and see what sticks”, of trial and error (guess and check), rather than of informed DESIGN.

I discuss that technique, and the differences in playtesting between tabletop and video games.

–"Many of the things you can count, don't count. Many of the things you can't count, really count." - Albert Einstein

Near-worship of statistics is a road to Loserville in playtesting.

Playtesting is not focus groups, or bug testing, or validation testing. It's testing how the game plays to make sure it's entertaining (or otherwise effective, if it's a "serious" game).

From my "Learning Game Design" course.

Perfect?

You’ve heard the phrase “practice makes perfect”. Evidently it’s a really old phrase:

“Practice is everything. This is often misquoted as Practice makes perfect.” Periander (an ancient Greek from before the Golden Age).

But “everything” is not really true either, you have to practice the right way or you waste much of your time, and that’s the topic for today/

•To get the best from playtesting, you need “objective” opinions

–That is, ones with no emotional connection to you or to the development of the game

•Those who are connected with a game tend to lose their objectivity, just as in anything else

•Those who are emotionally connected with you similarly lose their objectivity

Some novice designers hope that there's a formula, an easy solution, to any problem, sort of like the pill (or the cassette) in The Matrix. No such luck! Game design doesn't work that way.

Contemporary gamers tend to be finicky about playtesting games that look (and probably are) unfinished. Here are some ways to make playtesting more attractive, but there's no substitute for a regular group who are familiar with you and your games.

This is largely about tabletop playtesting, which presents additional problems to video game testing. It's about where people congregate to play games, and how you can locate them.

What are alpha, beta, and gamma (post-release) playtesting?

From my "Learning Game Design" course. Additions are in the next (written) "lecture".

I always think of stages of playtesting as relating to the state of the game, but some people relate stages to the nature of the playtesters. Discussed here.

Generally, you playtest to fix what’s broken, and to make sure the target market enjoys the game.
But there can be more specific goals, and that’s the topic here.I imagine that in some circumstances there are goals other than the ones I list here,these are the most common ones, I think.

More Specific Goals:

• Test to see “if there’s something in it”
• Testing with new people to get their reaction – broadening the testing pool – and to see how they see things differently
• Testing to see how people change their play with experience
• Test with those outside your target market
• Test one change

There are three major reasons why it can be hard to find people to playtest your prototype.

Blind testing - having someone play your game without your intervention of even presence, so they must read the rules to learn the game - is ideal, but it's hard to find reliable tabletop blind testers. REALLY HARD.

On the other hand, video games blind testing is the norm, because video games hardly ever come with rulebooks or other reading materials.

Playtesting is the heart of game design, and there are a lot of things to look for during a playtesting session (or from reports from blind testers). Which results in a two-part discussion. From my "Learning Game Design" course. Part 1

Playtesting is the heard of game design, and there are a lot of things to look for during a playtesting session (or from reports from blind testers). Which results in a two-part discussion. From my "Learning Game Design" course. Part 2

•At some point your game prototype will be fairly stable

•The rules are set, but you’re looking for improvement (as usual)

•You’re trying little things to see which of 2 or 3 choices works best

•This is what I call the tweaking stage of playtesting

Short answer, most of the time, no. It skews the results. But if you design games primarily so that you can play them (I do not), you might ignore my advice.

The rules are the most important component of a published game. You need to test the rules just like you test everything else, and that means writing the rules fairly early in the process, revising and retesting them as you go along.

From my "How to Write Clear Rules" course.

I understand that some designers often stop playtests. I hardly ever do. I explain why.

I don't use questionnaires. Why Not?

First, it is widely known that if you ask people for opinions that involve their own actions and preferences, they’ll often tell you WRONG. Not intentionally, they just often get it wrong.
It’s much more reliable to watch what people actually do, how they actually react.
Second, unbiased questionnaires that actually find out what you need are VERY HARD to create
Which is likely one reason why I don’t try to.

I never ask people if they'd buy a game (prototype). First, because the answer wouldn't be accurate. Second, what they've just seen/played doesn't look cool (usually), where the published game can. Third, because people are often polite and might say when they think you want to hear. Fourth, because amongst young people (college-age), a few buy lots of games and the rest buy none. The answer would depend largely on whether I had some of the former.

If someone volunteers that they'd buy it, fine.

And more reasons, as discussed.

I like playtesting to be as much like normal game testing as possible. But when I want to "go into more detail," I don't use a questionnaire, I use the "Six Hats" method for evaluating ideas, modified slightly for evaluating the game just tested.

Depends on what “bad” means.
Bad in the sense of “lacking in information?”:
Every playtest offers something, but some times less than other times.

Bad in the sense of “not fun?”:
Yes, that can happen, just as you can play any game and it isn’t fun for some reason(s)

Bad in the sense of “indicating your game is weak?”:
Yes, but if it IS weak, you want to find that out ASAP so you can fix it! That’s why we playtest

Bad in the sense of “someone broke the game?”:
But we need to know that NOW rather than later!

Some "designers" think that ideas can be ignored if they don't originate with the designer: the "not invented here" syndrome. This is an example of the ad hominem logical fallacy.

Don't be foolish. Evaluate an idea on its *own merits*, not because of where it came from. You are not god, nor are you infallible.

This is from one of my most-tested designs, but also one that has required major changes along the line. These are notes from a recent test, with more significant changes. Sometimes that's how it works out. Sigh.

I probably typed these notes as I played, on a portable computer. More recently, I dictate to a smartphone in Evernote, then sync the text result to my desktop.

This particular game has not been played in years, these notes from 2009. It works, but I'm not keen on it at all.

Notes from Info Select (my free-text database) about the first (solo, of course) play of a space wargame with both strategic and tactical elements.

I generally make a copy of the original notes for archival/historical purposes, then make notations/modifications to the notes in preparation for the second solo play.

During an interview I was asked this question. Good question: it's important to make sure your game isn't broken before it's released.

Someone on Quora asked "how do you find the fun" in a game? Sorry, you're supposed to design the fun in from the start! If you need to find it . . .

This is a different look at stages of testing, from the point of view of what the designer is thinking, more than who the testers are.

This is one of the more important screencasts I've made. I try to teach people a method for designing games, one that relies on thinking and problem-solving. Some people appear to use a much free-er form, I daresay sloppy, method, which amounts in large part to trial-and-error (guess and check) and "throwing things against the wall to see what sticks." Bad Idea.

I used to teach Project Management (graduate level), among other things, and game design has many elements of project management to it. Even though projects can change drastically, those who spend all their time "fighting fires" do a poor job. Those who plan and focus on the desired results are more successful. You need to do the same thing in game design, I think, to have the best chance to succeed.

This is Part 1.

This is part 2.

This is one of the more important screencasts I've made. I try to teach people a method for designing games, one that relies on thinking and problem-solving. Some people appear to use a much free-er form, I daresay sloppy, method, which amounts in large part to trial-and-error (guess and check) and "throwing things against the wall to see what sticks." Bad Idea.

I used to teach Project Management (graduate level), among other things, and game design has many elements of project management to it. Even though projects can change drastically, those who spend all their time "fighting fires" do a poor job. Those who plan and focus on the desired results are more successful. You need to do the same thing in game design, I think, to have the best chance to succeed.

This is a summary of thoughts about what makes a game "good". There is no single way, as tastes vary so much in "games" (many of which are much more puzzle than game). So in the end, after describing many alternatives, I inevitably settle on my point of view. Part 1

This is a summary of thoughts about what makes a game "good". There is no single way, as tastes vary so much in "games" (many of which are much more puzzle than game). So in the end, after describing many alternatives, I inevitably settle on my point of view. Part 2

Some "designers" think that ideas can be ignored if they don't originate with the designer: the "Not Invented Here" syndrome. This is an example of the ad hominem logical fallacy.

Don't be foolish. Evaluate an idea on its *own merits*, not because of where it came from. You are not god, nor are you infallible. And many heads are better than one.

How do you deal with (or cope with, since it can be critical) feedback from playtesting?

From my "Learning Game Design" course.

I was once asked whether to ask for/accept suggested solutions from PTers. As Neil Gaiman said (paraphrasing), if a PTer tells you what doesn't work for them, they're probably right, if they tell you how to fix it, they're probably wrong.

Like most game design questions, the answer is clearly "it depends". If you're playtesting with a lot of game designers, you'd be nuts not to solicit and listen to their suggested solutions. If you're playtesting with people who are just playing what may turn out to be fun games, even though they're prototypes, people who don't really think of themselves as playtesters, then you're unlikely to solicit much comment, and often you won't get many comments. The PTers are helping me identify problems, whether explicitly, or by my observations of their behavior. Though I cannot imagine not listening to a suggested change, the time it takes is minimal and "two heads are better than one", though I also agree that the problems are mine to solve.

In other words, it depends on the mindset of the testers, and on how much they think like designers. There is no single answer.

Testing and modifying a game design should be a continuous process of improvement. 

Designers have to ask themselves, when a playtest results in an "outlier", whether it's in the game, or a result of the people playing the game (and the location).

Even when your game works well, you should be looking for ways to improve it. Small changes can make small improvements. But they can also make big differences, and we'll hope you test those fairly early in the testing process.

This turned out to be quite a bit longer than I expected.

Symmetry is no guarantee of balance. A turn-based symmetric game can be quite unbalanced if the first mover (or last) gains a significant advantage. An asymmetric game can be balanced through careful playtesting. Part 1 of 2.

Symmetry is no guarantee of balance. A turn-based symmetric game can be quite unbalanced if the first mover (or last) gains a significant advantage. An asymmetric game can be balanced through careful playtesting. Part 2 of 2.

You don't write the rules just once, or twice. You write them, rewrite them, revise them, rewrite them, revise them, and on and on.

What are your standards? Are you trying to make a game that will be OK for one to three plays (like most published boardgames and video games these days), or do you want one that can be played 25 or a hundred times or more by those who like it? This will have a big impact on your testing.

In order to help the most inexperienced among us, I've described the stages or junctures at which a game design might be abandoned (for months, years, or permanently). There are lots of possibilities, and lots of possible reasons.

If you're NOT willing to abandon a design, you're likely to turn out a lot of weak if not awful games.

These extensive, play-by-play, notes were written to post in an online forum for development of Britannia Third Edition. This is much more than I would normally do for a playtest.

If you're fortunate enough to have a following (for you, or your game) you may be able to set up a forum (such as on Yahoo Groups) for discussion of playtest results with testers. Britannia has been around for nearly 30 years in various editions, and I can get help from fans of the game when testing the third editions.

•I try not to say, when one of my games is playtested, that I consider it “done” •Because I KNOW how often the “done” game gets changed, if only in small ways –Keep in mind, some “improvements”, aren’t

Ideally, a game would work equally well with:
Novices: beginning gamers (especially, beginners at this game)
Mediocre: people who generally aren’t good game players (which is the majority), however many times they’ve played a game
Experts: both as game players, and in this particular game

But it almost never happens, except in really simple games that even children can play well:
In other words, where expertise makes no difference.

The more gameplay depth to a game, the bigger difference there can be between experts, novices, and people who aren’t good game players

If you've designed for experts, then you want to have some expert players, especially some who have playtested the game a lot.

If you've designed for mediocre players, especially those who expect a game to be transparent, and who only expect to play a few times, then you don't need to worry about expertise. Or about how people play when they've played many times.

Part 1

Ideally, a game would work equally well with:
Novices: beginning gamers (especially, beginners at this game)
Mediocre: people who generally aren’t good game players (which is the majority), however many times they’ve played a game
Experts: both as game players, and in this particular game

But it almost never happens, except in really simple games that even children can play well:
In other words, where expertise makes no difference.

The more gameplay depth to a game, the bigger difference there can be between experts, novices, and people who aren’t good game players

If you've designed for experts, then you want to have some expert players, especially some who have playtested the game a lot.

If you've designed for mediocre players, especially those who expect a game to be transparent, and who only expect to play a few times, then you don't need to worry about expertise. Or about how people play when they've played many times.

Part 2

Playtesting does more than identify problems in the game and test suitability for the target market.

You have to be willing to change your game again and again, and if necessary, to kill it. If it's "my baby", how are you going to do that? It's a game, folks.

I'm sorry the sound is too low. My new (at the time) computer wasn't doing well.

Often during playtesting, you change a rule in order to modify the game in a desirable way. Of course, you watch to see whether it works as you intended. But you also need to look for unintended consequences, as games are logical structures, much like computer programs in their reaction to change. (Programmers amongst us know very well how fixing one bug can introduce others unintentionally.)

The old 80/20 rule actually has a name, and applies to many things in life. Including game design, especially testing.

In addition to what you'd normally track when playtesting a two- or one- player game, there are some additional things to watch for in multi-sided games.

Here is a discussion, and some examples from my games, of what paper prototypes might look like. Not real pretty, nor hand-made.

From my course "The Joys of Game Design".

While I haven't emphasized it in the screencast, when you design a game you're practically choosing a play style that will be most efficient, depending on how fluid the game is. I discuss what I mean by "fluidity" and the three playstyles that result (one being kind of "in between the two extremes"). Part 1.

I regard this unnamed "block" space wargame for two (or four, playing partners) as one of my better games. But it's only a year and a half old.(I refer to it as "4X".)

I tried developing a four-sided version; while it worked strategically, I don't believe you can play a block game for four because it's too easy to see the hidden side of some opposing blocks, especially in a game without strict "battle lines" (blocks scattered all over the board). I've included notes from a solo 4 player test.

This is an example of the brief notes I keep of each game playtest. (Sometimes I write much longer notes, that I keep separate, as in another example in this course from "Age of Expansion".) Since I made this file in 2008, I've discovered that the leader at midpoint nearly always won the game, so I've had to modify it and test further. Right now it's "lying fallow," not played in some years.

Why write a book in the 21st century, an era when people rarely read non-fiction books? And why a game design book? Here's why…

Describes my other classes and my games and other game-related activities.