One of the big things that differentiates circuit quiz bowl from television is an emphasis on statistical analysis. There's a lot of importance applied to statistics, so much so that a lot of people take only one thing away from a tournament, their personal stat line.
I want you to be able to put this into perspective, when stats become important to your players. Because when stats become important to your players, it’s almost always at a time when individual stats lie, and tell them something that isn’t quite true about them. For your first year as a coach, don't ignore your teams' response to stats, but remind them that stats aren't predictive, a player’s stats line lags what they actually know.
A lot of stat lines at the start of a player's career are more noise than trend. When you have a little knowledge of a number of categories, you can have a match where everything lines up, and a player goes on a tear because they just happened to have just the right knowledge. And then the next game, nothing lines up with their knowledge, and they score nothing. So you’re going to see lots of variance until you get enough repetitions in.
Another problem you will face with following stats is that the most radical changes in performance comes early in the player’s career, but not until they have actually played a few matches and practices. In that time, they take in the rules and begin playing according to the rules, and begin experiencing repeat questions. You learn by seeing the same clues and answers come up again and again. A player first has to see the repeat to recognize the importance of things repeating, and then they have to start picking up the value of new facts that they will repeat in future. That development time, when they’re encountering repeats in practice and in competition, is both when they are most likely to first explore stats, and the time when they will have the least relevance to them.
Televised quiz bowl has the problem that you can't really use statistics from game results. It's both that the sample space of your team’s performances is too small, but even that where the there is repeatable data, there's too many confounding factors which thwart results for a school's record. Since most opponents only play one match, you can’t really sample the field’s average strength. If there are specific theme rounds, they will have an unknown effect on whatever team encounters it. There’s also the problem that the formats of televised quiz bowl doesn’t match the format in most locations, so playing more events creates two incompatible and irreconcilable sets of statistics,
The problem this poses for us is that to actually figure out who on your team knows what, and how a team should be constructed, we need to use some sort of statistics, or we’re forced into blind guessing. We don’t want our players to blind guess, so that’s a bad example to use. Since we don’t have a sufficient sample space of questions from the show, we need a different sample space, and the only one available to us are the results of our own practice.
The good policy would be to get lots of data by keeping stats during practice, and you can do this, but it doesn't have the focus you intend.
The stats which would be really useful to us:
Performance of each player broken down by category.
When players have wrong answers, what kind of wrong answer is it?
Who on your team knew the answer at the same time as the buzz?
The problem with these pieces of data are that they:
- demand category breakdowns, and demand you figure out the categories.
- demand analysis in the moment, you need to pause, examine the wrong answer and the right one in the context of the clues given, and deduce the relationship and where it went wrong. This takes time and effort, and it doesn't form a pattern until you've spent weeks of practice cataloging wrong answers.
- demand a large sample space
We can sort of cheat this, I mentioned using the J!Archive before as a way to get short questions on a category quickly. Taking the results of practicing on those gives you some data. But by and large we can’t create this data fast enough in a week for it to be useful.
But even if we had that data available, we’d need to massage it. Here’s the data you can get easily out of questions, with a formatted scoresheet.
The stats you get:
PPG (points per game)
Tossup/Interrupt ratio
Bonus conversion (if you have bonus questions)
Total team score
When in the question they buzzed (in some advanced applications and in Buzzword through the celerity score.)
These are the neat categories, they have won out because they have emotional feelings attached to them: a strong PPG makes a player feel better, superior to their opponents (and unfortunately, sometimes feel that way over their teammates) A high celerity score or a quick buzz on the first clue can lift a player’s spirits or put fear in their opponent’s hearts.
These stats also lie a lot to new players: what your teammates know that they get to before you affects your PPG, and for new players who experienced the same curriculum, there’s a lot of knowledge in common (separate from Common Knowledge). Bonus conversion can be dumb luck of the draw, if you don’t get a lot of bonus questions. And where you buzz can be a factor of a player’s confidence as much as their knowledge.
More than that, all stats they are looking at past results. And if during the game when not answering, a player is listening, really listening, they’re not the same player those stats say they are. They’re a little better, a little more experienced, and they’ll use that knowledge gained on some future question.
There's really only two stat lines I put much faith in from my career. The first was a tournament where in addition to points per game, they recorded each player's range of scoring. In that, I ended up with a range of 100-50. I put value in that 50 as evidence that no matter what the writers put before me during that tournament, I still was able to hit five questions every round (this was before powers were invented), which means even when things went against me in subjects, I was able to luck into enough knowledge, or rather, I had enough knowledge in enough categories that I couldn't help but run into something I knew.
The second stat line came from a Trashmasters, and it was 72/1 over fourteen rounds. I spent most of that tournament without a neg and still leading the tournament in scoring.
Tossup/Interrupt ratio does have some value to tell you whether you are too risk averse or attempting to guess to often, but stat has high degree of variance due to difficulty, and the behavior of teammates, and the optimal value is highly dependent on the average opponent’s average score, and the particular category of any particular question. It gives you a general feeling if you should be risking answers you’re not certain about more or less. How a player uses that gut value, like how they use a mental filter, are ways to tune the knowledge accumulated and use it more effectively. As good as I was running during that Trashmasters, I could have taken more chances and still collected more bonus opportunities, and at the time, I kind of knew it.
Two observations I forgot to make in last week's tirade on filtering:
First, COVID has busted my chronological filter for very recent times. Like lots of people I'm suffering through the "was that last year or last month?" feeling that all things have taken on since March 2020. I imagine that will become less of a problem as we proceed, as most things of note from this time will be tied to an era, and attempts to quiz people on when in the pandemic will be properly seen as pedantic. But it is a similar problem to what I had when I was sick. I have mental markers down for the two months before hospitalization (“I heard about that when I was sick”) and for the five months when I was rehabilitating, but anything finer detail within that is lost for me.
Second, a filter is much less taxing on the brain than actual answer seeking. If it's simply a snap decision, no this doesn't fit, you don't use as much mental energy as actually thinking through an answer, which can save the stress that actually thinking through every question can do to you over the course of a tournament. A filter like that also is more useful than the filter of tuning out the question when you think that you don't know anything in that category.