Misuse of statistics

In Quiz Bowl, misuse of statistics can arise by using more and more elaborate formulae, computer programs, and statistical analysis tools in a futile attempt to make up for the fact that the premises or underlying data being fed into the model are incorrect or do not exist. Proponents can both be blinded by the elegance of their algorithms and be more able to discuss their details than less mathematically-trained dissenters, giving them an easy way to bulldoze simple but correct objections to the process. It can be a sign of bad quizbowl.

Examples include:

• Tournament formats which rank teams based on factors other than win-loss through complicated formulae and end up producing self-evidently absurd results (2003 ICT)
• Power-matching systems which make teams play widely different strengths-of-schedule based on their performance in each game, but then take teams to the playoffs based on the incommensurable win-loss records produced
• "Correlation tests" which attempt to mathematically "prove" which tiebreakers are more fair by starting with the unproven, unaccepted assumption that the point of a tiebreaker is to "predict who will win the next game between the two teams"
• PATH, which is laden with the assumption that all players on a team are perfect generalists and are being denied a chance to answer any tossup their teammates answer
• Every PPG-replacement or other player- or team-ranking formula ever produced, which seeks to sidestep the fact that the data collected about quizbowl games are insufficient by manipulating that data harder
• The formula for determining the JV champion at NAC, which involves taking arbitrary values related to teams at each site such as "the number of teams in the tournament times 10" and "average PPG of playoff opponents times 2" and adding them together, in order to compare an 8-0 team who played in DC to an 8-0 team who played in Chicago without actually arranging any way for them to play each other. In 2014, the Chicago champion defeated the DC champion 3748 to 3255 using this system of made-up points. The DC team is perhaps the only team to ever travel to a national tournament, win all of their games, and not win the title.
• Attempts to "prove cheating" that reduce to proving the tautology that exceptional performances are outliers, then deciding which outliers constitute cheating based on the same subjective assessment of the situation that could have been made without any statistical analysis. E.g., the assertion that "anyone who powers 3 tossups in every game of a tournament must be cheating, except if I decide that this could have plausibly happened without cheating based on my own evaluation of the player's skill and the tournament's difficulty."