Week 111: Tying it back to the first
Generating high frequency repeats
Short one this week. Between getting the post-tournament press releases out for SSNCT and beginning with the pre-MSNCT press releases, and getting a t-shirt design for HSNCT through review, I've been busy. I've really only had time to ponder the following idea for the book, and how it ties in with The 99 Critical Shots of Quiz Bowl.
Let's start with the two word takeaway lesson of that book: Everything repeats. Every subject and theme of quiz bowl questions is going to be repeated by some writer in future. There will occasionally be new answers added, or new clues added, but those new concepts will either be repeated, consciously or unconsciously, in questions by later writers, or forgotten and inadvertently rediscovered at some point down the line. Some of these concepts, most of them will be repeated at a much higher frequency, such that they will appear, written by many people over the course of a year of competitions. And while one person usually refrains from writing the same answer in consecutive contributions, or separates those questions over multiple years, writers in total, will produce the same themes as quickly as in tournaments separated by a week. The 99 Critical Shots were some of those themes which appeared frequently enough, and reliably enough, that I felt that 1/3 of them would show up in a typical high school or college tournament. Thus far, when I've sampled a set of questions written by anyone, and they're not trying to write for higher difficulty, my prediction has more or less held, in the worst case I've seen 26 questions matched themes of the 99, and in the best case 40 questions did. If you carefully choose a set of examples which you predict will appear in a set based on how often they repeat, and what categories they fill, you can probably do the same.
The lesson still applies to televised quiz bowl, but a lot of the pieces of the method of prediction have to change. One of the things I'm asking of a new coach is to do exactly what I did in the first book: Given a set of data, predict the contents of a set of questions, and train the team to expect some subset of the questions. I'm asking them to collect their own data (all those instructions of charting previous rounds), I'm giving them some training material based on my observations, and I'm giving them how to train for those pieces of data, but I'm also asking them to make a prediction of which of my suggestions to not worry about.
But the material in this case can't be the Critical Shots. Their coverage is too small, they're too unlikely to match up with the product of a single match. We can't even really widen the concept out to categories, as the circuit defines them. They're too broad to cover in a week, and we can't really even estimate a packet distribution for a single show. Between themed rounds, questions that could fit multiple categories based on a limited set of clues, and the possibility that they don't even have an idea of a distribution, we can't import that circuit idea.
I had tried to introduce this without using circuit ideas by couching this in terms of high and low frequency repeats. A high frequency repeat being something that appears in almost every circuit set, and a low frequency being something you see maybe once a year. But this is also an idea that doesn't translate to our problem. Someone with a year's experience on the circuit could tell the difference between these two concepts. But a new coach who has only seen maybe two or three episodes of the TV show? No, they don't have the sample space to make this happen. If they see a pattern, it's undoubtedly a high frequency pattern, but by circuit standards, they're not going to see repeats of answers or themes, unless those themes are very, very broad. There’s not a prediction you can make from such a small sample. And it was only at this point where I started to make some headway on the idea.
There are some broad categories of answers, where the answers are grouped together. In a much earlier one of these newsletters, I had made the example of questions about planets appearing in consecutive rounds when I watched some episodes of the same show in sequence. That was the level of granularity necessary to get patterns out of the data, that would produce high frequency repeats. What this book needs to do is give the coach some set of groups of answers which broadly appear, at high enough frequencies that they can be seen with just a few episodes viewed. With these, we can give them a starting point for training their team, and an idea of how the writers set up their questions. Once we show the coach that, they can know what to expect, and begin imposing some sort of order and predictiblity on some fraction of the questions their team will face. That, combined with what their students already know might be enough for them to win.
So what sort of groups of answers should we be looking for? It should be orderly information, so we can relay the information to students quickly, or be able to give them a concise study guide. It ideally should be something they're already familiar with, so we don't have to teach an entirely new concept but just review, things were taught at lower grades would help. They should have readymade last clues, so that we can expect them to end with a reliable ending. They shouldn't be very large sets of answers, or possessed of a very long tail of answers which will never appear, or have major gradations of importance within them which a coach would have to teach alongside the list. And they should fit inside the constraints of the show itself.
The last paragraph I had already applied when I started the book, and come up with sets of answers. But prior to last week, I hadn't anchored that part of the process I was asking the coach to go through to the rest of the reasoning above. I think this piece now fits, and I can move everything forward.