> once again, paul, you are conflating "more speed-based/less knowledge- > based" with "inferior." it's not. you (and i) may like it less, but > that doesn't make it a worse product. > > you raise a good point, though, and one which i'm sort of curious > about: does NAQT itself (or its head honchos, if there is no > unanimous voice) feel that the ICT questions adequately fulfilled > NAQT's goals for the set? my own impression was that they were a very > good packet set based on the parameters of what i had come to expect > from NAQT. that's one of the many reasons i enjoyed the ICT so much. Joon -- Hey, it's not often that *we* get asked for feedback on the set. :-) In Division I, statistically, the set essentially fulfilled our goals. Bonus conversion and power conversion were within one percentage point of our ideals. Tossup conversion was a bit low (78% versus 85%) which is something to work on for next year. Of course, the end goal is not the numbers themselves, but a set of questions that is playable and results in reasonable scores (and thus reasonable differentiation) at all levels. We think the set did that. The numbers are simply an objective way to measure it. At the same time, questions that are too easy result in upsets and randomness for the good teams; the upset rate does not indicate (so far as we can tell), that this was a problem. Questions that are too hard result in upsets and randomness for the bottom teams; there were relatively more upsets among teams 17-32 than 1-16, so the questions did a worse (but not a terrible) job of differentiating them. We've spent a lot of time in the past few days looking at the statistics to find some support for the principal complaint that a huge, fairness- or fun-disrupting number of the tossups were poorly structured and led to unearned powers or improper game results, but we just don't see the numbers to back that up. Maybe more digging will reveal it. I've gone and looked at all of the questions that people have mentioned by name as problematic (both on the Yahoo! board and in private e-mail). For some of them (Miro, Mandelbrot, Foucault's Pendulum, etc.) I'll admit that they were not ICT-caliber (possibly not by long shot) and should not have been used without additional editing. In a majority of cases, though, questions that players claimed had a giveaway in the first line were not, in fact, powerered any more often than the average question. In some cases clues that players remember as being "in the first line" are actually in the second or third line (very few tossups were more than four lines long). All in all, I would say that NAQT considered this to have been a relatively successful set, at least up until last Friday. We are in the process of re-evaluating that judgement. -- R. Robert Hentzel President and Chief Technical Officer, National Academic Quiz Tournaments, LLC
This archive was generated by hypermail 2.4.0: Sat 12 Feb 2022 12:30:47 AM EST EST