How good are our educational theories?

I know comparison is the thief of joy, but...

Can you guess who said this, and approximately when?

“...there is an awful lot of studying of the methods of education going on, particularly of the teaching of arithmetic—but if you try to find out what is really known about what is the better way to teach arithmetic than some other way, you will discover that there is an enormous number of studies and a great deal of statistics, but they are all disconnected from one another and they are mixtures of anecdotes, uncontrolled experiments, and very poorly controlled experiments, so that there is very little information as a result.”

Nobel Prize-winning physicist Richard Feynman said that over 60 years ago, shortly after he was asked to help California with its math curriculum. And I think his main point still resonates: We don’t understand math education like we understand physics…or medicine, or even sports education. Have we made any progress?

We do have useful theories, many of which have been bolstered considerably over the last 6 decades through random controlled trials and longitudinal studies. I wrote about 10 of them recently – things like spaced repetition and retrieval. We also have general approaches that seem to work quite well (like tutoring!). And, of course, individual educators know very well what tends to work with certain types of students. But our theories are still fairly abstract and difficult to apply to individual students.

It might be easier to explain what I mean by using examples from other fields.

If I drop a glass on the concrete, it will shatter. A scientist could explain what force drew the glass to the ground and why it shattered on impact. She’d have a story, and that story would be so accurate that anyone who understood the story could enact it in the real world.

Suppose my daughter has strep throat. The doctor will have a story about what happens to the body when it encounters that bacteria, and how it will respond if it receives a certain antibiotic. It may not work 100% of the time, like my glass meeting concrete. Maybe the diagnosis is wrong, or maybe the patient won’t respond to treatment for some reason. But that story is good enough to work in the real world maybe 95% of the time.

Now let’s move closer to the realm of education. Suppose I’d like to throw a baseball faster. A good coach might look at how I’m storing and unleashing energy in my body. He thinks of a pitch as one giant whip, from my coiled legs and torso to my rotating shoulder and elbow to my cocked forearm and wrist, which snaps forward as my fingers release the ball. Or that’s how it should work, anyway. Maybe I’m not using my legs or twisting my torso. Maybe I’m not putting it all together in some way. But this good coach will have a story (the “kinetic chain”), a mental model of how a good pitch is thrown. He’ll be able to describe my current pitching motion in relation to an ideal one. Through lessons and exercises, he can help me achieve that ideal, or at least get closer to it. What percentage of the ideal will I achieve? What percentage of his students will achieve the ideal? I don’t know. But at least he has a compelling story based on biomechanics. If I’m able to understand and enact that story, I’ll achieve success.

Math education is on shakier ground. My three examples above had three features in common:

• A clear definition of the subject’s starting point (an intact glass, a patient with specific symptoms, a pitching motion with specific features).

• A clear definition of the goal (a broken glass, a symptom-free patient, a faster pitch).

• A comprehensive and useful theory of the domain (the story) that allows us to identify states and create successful plans for moving from one state to another.

We can’t precisely define a student’s starting point. Is it merely a configuration of neurons? Even if that’s true, we can’t create model brains, let alone simulate specific brain states.

We can attempt to represent a student’s learning state in terms of questions they can and can’t answer, but our students each answer unique assortments of questions in mostly uncontrolled environments, and only a tiny fraction of these answers are recorded.

That is one reason why good tutors ask lots of questions, require students to show their work, and try not to give away too many answers. Yes, students learn more when they do the work, but these methods also give us valuable inklings about what students might know, and what they might be able to do.

These inklings, in combination with experience and theory, often yield good plans, which in turn produce good results. But I don’t have diagnoses as definitive as strep throat, nor are my prescriptions as effective as antibiotics.

I’ve picked a nice example for myself in strep throat here. Certainly there are many other conditions that can’t be reliably diagnosed or treated (Lyme Disease? Long COVID?). And my pitching comparison is even less favorable – I’m sure many pitching students are incorrectly diagnosed, or fail to improve despite reasonable instruction.

But I struggle to come up with any diagnoses and prescriptions in the math education world that are as widely applicable and effective as the “kinetic chain” theory baseball coaches use to instruct pitchers. The pitching motion is complex. Each pitcher has a different body and different athletic experiences. And yet the theory can be applied to everyone: an expert coach can diagnose deviations from the ideal quickly and suggest exercises that will help the student achieve it.

In my world, a student may arrive with a 580 SAT score, but it’s going to take me a long time to figure out why he’s getting that score – which problem types he’s missing, which test strategies he’s not applying. Once I have enough context, I can suggest exercises that are very likely to work. If asked, I can point to various pedagogical principles that support those exercises:

• Why it’s good to answer lots of questions, preferably without multiple choice (retrieval)

• Why it’s crucial to focus on questions at your level (chunking, cognitive load, Zone of Proximal Development)

• Why practicing many topics at once is better than practicing one at a time (blocked practice is bad; spaced repetition and interleaving are good)

I can even track down the scientific studies that prove those principles are correct. And I can also share my own results, along with probabilities and anecdotes. We are no longer in Feynman’s position of having “very little information” about the “better way” to teach math.

But it feels so much more complicated and bespoke than the three examples I provided earlier. If my SAT student needs help with linear equations, I can’t say, “This is the ideal path students follow when learning how to solve linear equations. I can see you are at this point of the path, or deviating from the path in the following way, and therefore I will use this proven method to help you reach your destination.”

---

After a tutoring session, I was talking to the student’s father, and I happened to notice a book on his shelf by William Osler. My Dad had frequently spoken to me about Osler, a Canadian physician who was one of the “Big Four” founders of the medical school at Johns Hopkins, and I mentioned that it seemed like education was in a similar spot today – some things work (like washing your hands back then), a lot of things don’t (like prescribing cocaine and ‘laudanum’). He laughed and said that he envied me. I was pretty surprised. He was a very successful and highly respected doctor – I believe he actually wrote some of the questions for one of the medical board exams. Why would he envy me?

“We have so many answers now,” he said. “There’s no room for experimentation or finding your own solutions.” True, you could be a researcher, but that would probably mean focusing on one question for a very long time. A clinical physician, seeing one patient after the next, rarely has the freedom to adopt his own course of treatment, let alone discover new diseases or invent novel cures.

On the other hand, we tutors get to create our own materials, our own methods, our own explanations, even our own adaptive math practice tools! It’s frustrating that education isn’t a science, but it’s also an opportunity for us. Maybe education is a little behind other fields, but we are the ones who get the chance to push it forward.