The Business: Making A Not-Crap RPG: Part Two: Parsing Playtests

MMORPGs and Tabletop RPGs have some things in common. One of them is that playtesting is often a frustrating experience for the developers.

Gamers are very good at breaking things. They will find out how to exploit gaps in your procedures and mechanics that you did not intend for their benefit. You need this information for the revision process.

That is what your playtesting needs to focus upon. How to do this?

You tell the playtesters what you expect out of the gameplay test. You tell the playtesters that they are to see if what you designed--your procedures and mechanics--actually result in the fulfillment of those expectations. You insist that they document what they did and how they did it. Then you wait for their reports, provided by way of the channels you established for this purpose and in the format that you specified.

Your job now is to parse those reports.

You need to go over the written reports, compare those to the supplied documentation--these days, that should include A/V logs and chat logs; Unlisted YouTube uploads will do for the former--and figure out exactly what went wrong and how.

That's not glamorous or exciting, but you want to make a RPG that isn't crap so that is what it takes. This is why development time of any RPG worth a damn is measured in years, not months or less, if it is not a clone of an existing game.

You need to sit down and go over the play reports in exhaustive detail, much like someone trying to figure out why a machine malfunctioned by retracing the events that led up to it, to see how each step went down and thereby detect the specific point where a misfunction of your design occured and what caused it.

Once you know what and how that malfunction occured, then you can figure out what to do about it. At the very least, in the case of user error, you need to review your technical manual documentation (i.e. your rulebook) to ensure that you convey the correct meaning to the user. If your playtesters encounter an unforseen effect or synergy, then you need to revise your design to eliminate that unwanted outcome. Whatever it is, this meticulous process will reveal what you need to remedy and point you to a means to do so.

"And then I'm ready to publish?"

No. Now you're ready for the next round of testing.

You will repeat this playtesting and revision process until your game design actually functions as intended from start to finish. When not a single element fails to produce the desired result, and your playtesters have no problem comprehending what your game is meant to do or how it goes about doing it, then you're done and no sooner.

(Yes, this is a large reason for why so many games now are variations of existing ones; cuts down R&D time by a lot.)

This is why you should not be in a great hurry to bring your game to market once that first draft is done. You need testers to break it. You need to review the documenation of those breakages. You need to revise your design to account for those revelations, test it again, and repeat this process until your design does not break under its intended usage.

Then, in your final draft, you need to spell out with all the subtlty of a brick to the face, exactly what the parameters are--what your game does--to properly set user expectations. You do not publish a RPG about exploring strange new worlds across far across the seas and neglect to include rules for ships, shipping, piracy/privateering, and so on. (Yes, this sort of fuckup has happened.)

Now you're ready to publish.

But you're not ready to sell. Time to talk about something a lot of us find distasteful, tomorrow.