Questions Are King

Bad news: you have to take a final in three days. If you took the test today, you’d probably get a B-, but you feel like you could get an A if you studied. Unfortunately, you will only have access to exactly one of three study materials:

1. The class textbook. Everything on the exam is directly related to information in this textbook.

2. Every test the class took that semester. The teacher gave tests every three weeks.

3. The notes from the smartest kid in the class. This person takes great notes.

You’re free to use the internet, but you can’t talk to other students. Which of the three would you choose?

For me, it’s not close: I want the tests. After all the concepts and variations and explanations and discussions and analogies and experiments and exercises, there will be a test. That test will be composed entirely of questions. And your answers to those questions will determine your score. Questions are king: master the questions, and you master the test.

The textbook might have all of the questions, but there are too many of them. You could easily waste time on something that has no chance of appearing. The notes might have great explanations and examples of difficult questions, but there won’t be enough questions to practice with. You might easily miss a difficult variation that is likely to appear. 

How do you find the best questions? By looking at old tests. Testmakers have tendencies. Think of it from the teacher’s point of view. She’s written all of these tests with questions that target the most important skills from every unit. It wouldn’t be fair to target new skills at the end of the year. And it’s very difficult to write good questions¹. An excellent teacher might develop novel ways to target those same skills; a mediocre one might simply reuse questions or change the numbers. Either way, that collection of old tests is pure gold: their questions reveal which skills are most important and provide a way for you to practice them. When you’re studying for a test, no resource is more valuable than a good question set, and old tests form the best possible set of questions.

It may seem a bit brutal to reduce the beauty and creativity of your favorite subject to a set of questions. But here is a surprising and wonderful truth about a well-made test: in the process of striving to beat the test, you acquire the skills it is testing. It’s as though you were trying to cross a river and found that you could jump from one stone to the next, and then, looking behind you, noticed that the stones had somehow risen out of the water and combined to form a stable bridge.  

The SAT Math section is a good example. If a student starts with a 470 on SAT Math and earns a 620, it is certain that they got much better at math. No calculator tricks or test strategies can possibly raise scores that much. The student was focused on beating the test, but they could only do that by getting better at linear equations or quadratics or proportions (or, most likely, all of them). Scores improved because skills improved. And these skills always transfer to the classroom. Every time a student makes an improvement of this size, a parent tells me that the student is doing much better in school.

Questions are therefore the key component of a powerful learning process: Testmakers use questions to create fair and informative tests. Students target these questions in pursuit of higher scores. But as students chase these higher scores, they build skills.

1. More on this in a future post.