On measuring difficulty [I of II]

• From: stipenn
• Date: Mon, 16 Apr 2001 11:03:44 -0700
• Message-ID: <000TDMD00.001FBN6P1.12170551-quizbowl_at_...>

One thing I did for Penn Bowl 10---to see whether
it was of any use for future events---was to try and
come up with a gauge of the difficulty of each packet
as it was submitted. The basic idea was this--for
each question, a point value was assigned: full value
for a question that I could correctly answer *as
written*, half value for a question that I could not
correctly answer as written, but had an answer I've heard
of _before I began editing for the tournament_, and
no points for an unanswerable question whose answer
I *hadn't* heard of. [Repeats, therefore, always
receive the same value: if a Q on "Trouton's rule" got 0
points the first time around, it'd get 0 the second time
too.]

I used these values to get a difficulty rating for
the tossups and boni in each packet, ranging from 1.0
[I could answer every TU as written], down to 0
[never heard of *any* of the answers]. I then used the
following formula, where T is the tossup difficulty, and B
the bonus difficulty, to estimate the number of
points scored per 20 TUs:

Est. score = 200T(1 +
3B) [= 20 * (10T + 30BT)]

It should be fairly
obvious from this that tossup difficulty plays a larger
role than bonus difficulty: if you can't answer the
tossups, you won't hear the boni.

I calculated this
number for almost every packet received (it took about
10-20 minutes per packet). I will present the numerical
data later.

In the second half of this, I will
just note the most important results of this
'experiment.'

--STI