Hansonian Education? 7/9
why don't education interventions matter more?
Recent study, Pelham et al: The Effect Of Stimulant Medication On The Learning Of Academic Curricula In Children With ADHD.
It’s gotten popular buzz as “scientists have found medication has no detectable impact on how much children with ADHD learn in the classroom” and “Medication alone has no impact on learning”. This probably comes as a surprise if you’ve ever worked with stimulants, ADHD patients, or classrooms, so let’s take a look.
In the short run, students do better, but in the long run the difference fades out. That happens with many educational interventions. I have often stated what I call The Null Hypothesis, which is that educational interventions do not make a significant, long-term difference. Careful studies typically find little or no effect, even in the short run. But even if a study shows an effect in the short run, it often fades out a few years later.
I assume that a big cause of fade-out is that people’s brains revert to their long-term capacity. Helping someone do better on a test is temporary. But that does not improve their long-term ability to perform tasks.
Robin Hanson found similarly disquieting results for health care spending. He summaries many studies in which two identical populations are compared, with one population receiving “better” medical care (more spending on procedures) and the other receiving less: the average outcomes do not end up different. Since some medical procedures obviously help, he hypothesized that in the aggregate these must be canceled out by procedures that cause unintended harm. I call the harmful procedures “Hansonian medicine.”
Similarly, there may be educational interventions that help some students but harm others. Maybe medications that are prescribed for ADHD help some students but harm other students somehow. So in the aggregate, the results are minimal, especially in the long run.
Probably every form of education you can name might make a slight positive difference for some students and a slight negative difference for others.
Distinguish two potential outputs of education:
1. Education might impart technical skills and knowledge of facts and mechanisms.
2. Education might inculcate norms and ideology (in a word, civics).
Consider four combinatorial possibilities, re: the impact of an intervention:
a. An intervention durably changes skills/knowledge and civics.
b. An intervention durably changes skills/knowledge, but not civics.
c. An intervention durably changes civics, but not skills/knowledge.
d. An intervention changes neither civics nor skills/knowledge.
If an intervention has a durable effect, then what is the sign of the effect? (Think of Sonny Corleone's quip to his brother Michael: "Whatcha go to college? To get stupid?") Now we have seven combinatorial possibilities.
As Arnold Kling suggests, usually an intervention's effect on *skills/knowledge* is weak, its sign varies among individuals, and the result is noise or a null effect.
By contrast, discourse around an intervention's effects on *civics* typically assumes:
i. The effect on civics is strong.
ii. The sign of the effect is unambiguous.
However, there is polarization about whether the sign fo the effect is positive or negative. Think discourse around CRT in schools, the 1619 project, or sex education.
It would be interesting to poll the minority of social scientists, who believe that interventions in education have null effects on skills/knowledge, to ask them whether they believe also that interventions in education that target civics have null effects. My hunch is that most of them would say that civics interventions have durable effects -- and do more harm than good.
My interpretation of the study is that you have to care about retaining whatever the material is you’re learning. The medicine clearly seemed to help them to focus better, but most 7-12 year old boys (unsurprisingly) don’t care very much about knowing vocabulary.
Regardless, helping children attain short-term success may still be worthwhile. Staying out of trouble and turning things in on time is a big part of life, and insofar as adhd medicine availability has a positive “disparate impact” on boys I think that could be a worthwhile intervention.