Playtesting

From QBWiki
Jump to navigation Jump to search

Playtesting is a step in set production in which questions are read to a core of playtesters for the purposes of giving feedback. Unlike discussion, playtesting is almost always done before the first mirror, though in recent years the initial online open mirror of a set has doubled as a playtest mirror.

There has been recent discussion of playing playtesters being compensated monetarily for their work, at least on major sets like nationals.[1] This would be a large departure from existing norms - as of 2025 playtesting is a role that is almost unpaid and sometimes even requires players to pay.

How to playtest

The first goal of playtesting is to get datapoints on the difficulty, ordering, and quality of questions. Commonly the focus is on portions that have less data, like first lines and bonus hard parts.

The second goal is to serve as an expert proofreader who can go beyond typos and grammar to call out oversights like ambiguous clues, possible hoses, missing prompt instructions, and incomplete answerlines. A similar function is served by NAQT's second eyes editors, though those editors will sometimes also do internal playtesting.

For the writers

Playtesting affords a unique opportunity for an editor to directly interrogate players about the construction of a question - as such the goal should be to squeeze juice as much as possible. As an extreme example: 2025 ACF Nationals head editor John Lawrence had every editor and playtester look at the drafts of every question using a Discord bot written by Jordan Brownstein, followed by a second round with additional playtesters.[2] As a rule of thumb it is good to have every major draft of a question go through at least one round of playtesting.

The two essential halves of receiving playtesting feedback are "hearing a valid criticism and fixing it" and "hearing an invalid criticism and not fixing it" - being able to distinguish these cases is an important skill. The value of minimizing "invalid criticism" can be so great as to outweigh the benefits of a large playtesting core - in practice there are many scenarios where a single experienced playtester may be more valuable than a cohort of inept ones.

For the players

Being a good playtester means thinking about the question and trying to help the editor rather than being selfish and focusing on personal gratification. In most cases playtesting is an opportunity to hear novel questions for free so it's important to pay back into the process. One way to straddle the line is to "stress test" by making early buzzes that apply to only some of the clues to see how well the tossup is pinned and how robust the answerline is.

Some useful tips:

  • Trust your personal expertise. Playtesting is not the time to be humble. If you're a specialist and have studied a subject extensively but cannot convert a question, it deserves mention.
  • Don't muddy the waters. The converse of the previous point. There's no need to say you don't know something on every question, nor do you need to give justifications for how you know something unless it is actually relevant.
  • Give replacements. If you think a clue is non-unique, don't just point it out: sketch out an alternative. If you think a clue is too easy, try to suggest another one.
  • Propose alternate answers. If you learned a different name for an answer, make sure it's included.

References

  1. Jump up Re: 2025 PACE NSC Thanks and Discussion by DavidB256 » Thu Jun 12, 2025 10:16 am
  2. Jump up ACF Nationals 2025 Thanks and General Discussion by ThisIsMyUsername » Mon Apr 21, 2025 11:24 am