Freddi DeBoer has a piece called Education Doesn't Work 2.0 (https://freddiedeboer.substack.com/p/education-doesnt-work-20) which is the definitive takedown of "education interventions". It's the lazy person's summary of what he wrote in his book "The Cult of Smart". Highly recommended.
Freddie says: "But there’s no question that in study after study, for decade after decade, researchers have found that interventions presumed to have causal influence on educational outcomes simply don’t. Almost everything is clustered right around zero. Most things we try in education simply don’t do anything. This is reality. Bet on the null."
But does the Null Hypothesis itself survive scrutiny for selection bias?
For example, one wonders if the most researchers (and people who decide what gets funded, published, reported, etc.) have been artificially constraining the space of "things we want to achieve and study" to precisely those limited areas that are politically-desirable objectives in conflict with ideologically-inconvenient truths about human reality.
That is, if you keep trying to do the impossible, and you keep failing, then it's not that "everything fails," it's that "choosing impossible is nuts," and "trying over and over and hoping for different results is insane."
With regards to things like crime, health, and education, doesn't it seem like the vast bulk of these studies all start out by heading in the wrong direction? That doesn't prove that there are no interventions which can work in those fields. That proves that if researchers are constantly trying to find a land-based bike path from Boston to Bermuda, that the problem is not with the whole enterprise of bike-path-discovering research, but with the dumb ideas in the brains of the researchers.
Exactly. Researchers are far more likely to study the impact of fad ideas like DEI training or Head Start than, say, maintaining discipline in the classroom.
Yes! This. An *obvious* change would be to lower, rather than raise as has been more commonly done, the age at which you may opt out of school.
Thus, rather than wasting so much energy on the worst-behaved students, which demoralizes teachers and makes a mockery of the classroom, of school itself - let them go.
Not to alternate school. Not home for 3 days and then you're back at it. Not - return but with an aide we have to pay to babysit you all day.
"Clearly you are not school material. You are dismissed. You needn't come back. If you want to come back, it will have to be very different. It's a matter of total indifference to us."
ETA: I'd suggest the school year you turn 13, could be your last.
Both K-12 and higher education are reliable bastions of Democratic Party electoral support. Spending more money there may not improve educational outcomes but it helps keep the Blues in power.
Education does not improve maximum gene/culture IQ of each person, tho bad edu might reduce it. Bad school might have a 100 IQ 16 y.o. person score only 95, or even only 90. But no school nor coaching gets him up to 105, much less college level 110.
What can happen is to have more training in making and doing things that low IQ folk can do, like assembly work. Or McDonald’s, tho they seem to far prefer to hire young who so often leave. K-12 schools should be far more concerned about the success of their low IQ students.
Here is an experiment worth trying as a pilot program:
Allow teens to replace high school with vouchers for internships at firms/orgs, apprenticeship programs, training programs, and the like.
Would entrepreneurs and/or established orgs develop new and better ways to prepare youths for the labor market? (Supply-side response)
The current one-size-fits-all approach systematically delays adulthood and alienates a large fraction of boys. Pace the null hypothesis, there should be radical experiments in new paths to enable teens to find their way, spend much more time with adults in the workplace (rather than most of their time with peers in a total institution whose main function is daycare), bring something to market, and stand on their own two feet.
My own expected value of any UBI program is negative, because it incentivizes bad / lazy behavior. Social interventions fail because of the unwillingness of social planners to say behavior X is worse than behavior Y, so we need to reward Y behavior more. Instead, because the worse behavior X results in bad economic outcomes, the social program tries to equalize the outcome, thus subsidizing the bad behavior. Trying to get quite different results without a quite different behavior culture will naturally fail.
The unwillingness to do slut shaming, nor even agree on a negative phrase for uncaring, promiscuous men, is an example of this. For children, and society, promiscuous, slutty behavior leads to bad outcomes.
If I thought the problem of US Blacks was, actually, “systemic racism”, I would certainly be on board with shaming the racists. But the evidence according to behavior is that the Black poor are that way because of slut-jerk men who become the absent fathers of kids with mothers who choose to be sluts. Which increases other bad behavior: not studying/ learning in school, crime, drugs, bad work habits & jobs, and of course more promiscuity. Shameful behavior, done by Whites and Blacks, and deserving of cultural dis-incentive, shame.
Capitalism is pretty strictly a carrot only set of incentives—get the good results by good work, or not have a boss willing to peacefully pay you. Or else be poor, and shamed, because of your own failure.
But, yes, there are many cases, maybe 10% (tho I see no estimates nor measurement attempts in America) when it’s far more bad luck and others’ problems which create a terrible situation. Big temporary help for those folk does help, because their lifestyle choices are helpful.
This is the fundamental problem in dealing truly complex multi-variable problems where in a N dimensional space you try and describe the problem with N-X dimensions. It can't be done. If you think about conic sections from your math classes a long time ago you get an infinite number of 2-D objects from a point to an ellipse from sectioning a 3-D cone. Any relevant variable not included can sink the analysis.
This is how humanity from around the world took the night sky patterns (random dots in 2-D from a position on a rotating earth moving around the sun) and created Astrology models connecting that patterns to human events. Creating widely "believed" nonsense.
For example, minor family cultural values not included can have major effects. Visiting a Japanese friends house where the dinner conversation was about "which major university will you attend" instead of my fast car culture (LA in the 50's) shifted me from being an excellent mechanic like my advisers at school were recommending into a Ph.D. level scientist creating significant values for humanity as a whole.
In todays world of using check-box categories like race, ethnic, etc. which contain full overlapping distributions of other parameters of individuals from IQ to foot size any model based upon the check boxes only will be Astrology relative to individuals. With the modern non-STEM belief that you can exclude unmeasured variables, like the influence of a grandmother on a kids development, will be wrong.
The same problem exists with most biology and ecology where a microscopic parasite can control the distribution of cattle and people in Africa. What a bacterial virus (phage) does in your gut and impacts on your brain gets to a complexity level hard to imagine but still real to you as an individual. However, in science we do know that we can't exclude "inconvenient" variables like culture in our analysis.
Microcredit lending targeted at women was a big thing when I was involved in development assistance decades ago. I always thought it was nonsense, combining as it does what Stevenson calls 'the engineering view' with an earlier version of DEI. The thing to remember, however, is that development professionals earned hefty salaries implementing this and other types of interventions, as well as enjoying perks like travel (albeit often to places with less than ideal conditions) and attending conferences, as Stevenson mentions. This reminds me of an anecdote from a former colleague, about an economist who was a consultant on a different type of policy intervention in a developed country. Other economists were razzing this economist about whether the policy intervention provided the benefits it was intended to produce, and the economist responded to the effect that 'I don't know whether the policy is beneficial in general, but it has been greatly beneficial to me personally.'
Re: the null hypothesis in education interventions
Arnold and commenters, from time to time, have addressed a conundrum:
(a) Education interventions don't reliably and durably improve target outcomes.
(b) Education interventions seem to have a cumulative impact on normative beliefs of youths. This outcome may or may not be the explicit goal of the interventions, depending on the specific intervention. Sometimes it's brainwashing, sometimes it's an ideological side-effect.
Or maybe the real cause of ideological change among youths isn't schooling, but some other social factors; for example, prosperity or the massive growth of the entertainment industry?
Before arguing too much about any particular policy, there should be agreement about the goals. This goal, making people better, doesn’t include much testing of reading ability, none. That makes it a more universal goal, tho not as specific as I want for me my kids. But being better is quite a good goal.
If you reject race/IQ realism, you are naturally going to reject the null hypothesis. So your left trying this or that intervention to fix what “must” be a problem given the outcomes.
One has to ask "how bright do you have to be to read at grade level?" If 90% of IQ 100 kids can read at grade level with reasonably low levels of training and 60% of IQ 90 kids can do the same, that suggests one thing. If it is 100% of IQ 100 and 80% of IQ 90 that gives you another suggestion. I don't know what the numbers look like, what average IQ leads to 20% reading at grade level, but reading at grade level has never struck me as something terribly difficult.
So race/IQ might explain a lot of the gaps in achievement, but you aren't going to figure it out by looking at extremely high level statistics. You will need to look at school/class level statistics, because all of the relevant variables are at the individual and class level. What are these kid's IQs, how do they respond to these teachers over time, etc. I have seen some absolutely abysmal teachers and absolutely abysmal teaching practice, some that seem worse than doing nothing. I suspect that those on the lower ends of the IQ scale are more harmed by bad teaching than those on the upper end, not in just a linear fashion.
Actually, it’s largely because of the way in which reading is / has been “taught” for so many decades that kids’ reading ability has fallen off a cliff - ie the ideologically pushed “whole language” approach, which rejects any phonic element in the process. This can work for a minority who learn to decode/read words more or less intuitively. But for most kids it reduces the act of reading to a fruitless ‘look/say’ routine of guesswork,; whereas most kids need Explicit Instruction in order to sequentially develop basic skills in reading and mathematics - see Sweller et al re working memory. Take a look at the “reading wars” controversy, and at the OECD’s international PISA results if you want to see the relative decline in reading capacity across Anglo-Saxon countries, where the Left has throttled education for the last 50 years.
Agreed, I taught my girls to read using phonics to good effect. I think there is more to it though, and the question “why did teachers stop teaching phonics?” really leads to a lot of more general insights into why kids are doing so much worse across the board.
I taught in a middling school district in Massachusetts and knew plenty of kids who could not read at "grade level". This was even though we had reading specialists to give special instruction and Massachusetts always comes out on top or near the top in the National Assessment of Educational Progress. I would be willing to bet substantial amounts of money that a large proportion of young people never read at grade level.
The education business is full of standards and "expectations" that lots of students never meet.
Yea, but that's just it: is it because they have never had good teachers, or because the kids are all dumb, or because the bar is really high? Or is it a hellacious mix of all three? My sense, particularly based on my experience teaching college, my mom teaching elementary reading for ever, and my dad working in youth detention centers (and my teaching there a very little bit) is that there is no real measurement of teaching skill at those low levels, and very little grasp of what works and what doesn't. New teaching techniques are driven almost entirely by fad or politics and not at all about what works to help kids read or get them interested in reading. By the time the kids are in middle or high school there is a terrible compounding effect where they didn't learn well at lower levels, then they didn't catch up the next and just keep falling behind. I have also seen remarkably clever delinquents who can't read well at intake but improve drastically over six months or so.
So at a high level, some good experienced teachers are really good at teaching kids to read (and have an interest in doing so, which is necessary I think) and very little understanding of how to teach other teachers to do that, not much interest in identifying it, and very little interest in getting rid of bad teachers and getting mediocre teachers to improve, or ability to tell the difference. There just isn't much interest in finding out what actually works, because it would involve doing a lot of things people don't like and there is a lot of money to be made with new initiatives to teach new and exciting ways, and if those don't work there is more money to be made selling a new one next year.
It is sad to compare what a new drug has to go through to get approved and what a new educational procedure does. For the drug, it's a number of stages (one of the first being that it doesn't make things worse, that it's "safe) of randomized controlled trials. For things in education, it's maybe a few underpowered studies and a happy feeling. Some times, changes even make things worse. A tragedy was the educational establishment deciding that phonics was right-wing and pushing it out in favor of "memorize the look of the word". That seems to be changing.
There are some teachers who can reach otherwise unreachable students but, as you say, they are rare and generally can't pass on their skills. Most teachers are in the broad middle, not bad, especially if the kids are middling bright and with middling motivation. Most people who read and comment in places like this have no idea how "not bright" many people are. Because they have no contact with them outside of structured situations like retail or food service. The bar is too high because of this and because people in the business think it is mean and undemocratic and possibly racist to think a lot of kids can't perform up to a high level. If I was a poet, I might call it toxic optimism.
Right, grade level reading is not that hard, and even if the whole school had a special education diagnosis IQ alone would probably not be the explanation.
IQ alone is never a complete explanation. But it is much better than "almost all the young people here are bright enough to meet the standards if we only just do it right". So since they're not, there must be lots of things we could be doing better. But every time we think we've found something, it doesn't work at scale.
Average IQ for blacks is 85. In depressed urban areas like say Baltimore it gets down to 75. 83 is the cutoff where are army won’t even accept you let enlistment because your too dumb to be useful.
It would not surprise me if a ghetto school lacked the mental capacity to have a majority of its kids read at “grade level”.
Does the null hypothesis depend on the intervention occurring in a bureaucratic environment captured by interests whose primary goals are different from the goals of the intervention?
What if we could somehow reduce bureaucratic inertia and re-align the interests of education leaders (or criminal justice)? If the department of education were destroyed and the new, more agile interests controlling education were primarily concerned with student outcomes and didn’t care how much they needed to blow up the current system - then could we temporarily put aside the null hypothesis until the new leaders themselves were captured by other interest groups? (I’m thinking Mancur Olson “The Rise and Fall of Nations.”)
I'm curious what are the educational or criminal justice policies that have actually made a big positive difference? Mandatory high school education? The GI Bill?
That maze analogy is excellent, and I am going to steal it the next time I'm at a 12 step meeting and someone else there steals the "fall down a hole" story from the West Wing.
On the other hand we know that different countries have different results. They do something so in a different way and get different outcomes. I know it is very complex, but there must be a way to experiment with that, maybe hard to do but theoretically you could move people around and see if that changes their outcomes.
It is also true that many (most) RCTs are under-powered, sometimes due to financial constraints, sometimes due to over-optimistic effect size priors encourage by program advocates. And, somehow, many individuals (indeed, nearly all) eventually desist from criminal behavior, and the schools do actually manage to educate most students to some level of proficiency, and some students to high levels of proficiency.
No doubt many interventions don't work and some even make things worse but none of them have any benefit? Really?
- so if a health insurance lottery didn't help the winners, compared to what? Someone on Medicare or charity care? Is there a control group who has to pay for care?
- did social security reduce poverty and improve the well-being of elderly?
- did/does City college of New York provide a way out of poverty to many of the poor and immigrants? What about heavily subsidized community colleges? Isn't public K-12 an intervention?
If we can find 50 examples that don't work does that mean there aren't 10 examples that do? Color me skeptical of claims none work.
The problem with trying to do RCTs on interventions on criminals is that you will have a lot of definitional problems because of the many differences between states in criminal law. These differences are often quite arbitrary and significant. I think to do a better set of RCTs you should restrict it to programs of various kinds targeting federal offenders only. While you would still have the same problem across different circuits, in different prisons, and at different times, it would not hit you as badly as trying to compare things in State X vs State Y. There are similar issues that make comparing legal statistics between foreign countries a waste of time.
A lot of new District Attorney Researchers in many major Blue cities have had no trouble running good trials and getting solid, statistically significant, and replicated results about which interventions really work to lower crime.
That's because, when they just recently all implemented the total opposite of those interventions, crime shot way up, almost immediately. It's a good thing they've done this important research for us, because otherwise, who could have guessed?
The fact the crime rose so much and so quickly after application of the interventions allowed those District Attorney Researchers performing difference-in-difference analysis to assume that other local factors didn't change much and so could be ignored as causal factors when the same interventions cause the same kinds of crimes to shoot up similarly in a variety of different local scenes with difference local and state laws and other local factor. I mean, not varying in terms of which party is locally in charge, that's the same for all of them.
Seriously folks, come on. What works to reduce crime - and what will cause it to explode if you stop doing it - has been common sense for millennia.
As I mentioned previously, the whole problem with trying to infer a "null hypothesis for crime" from all those failed studies is because the vast majority of them were selected to try "progressivist opposite of traditional common sense" approaches. This proves something about the relative merits of progressivism and traditional common sense when it comes to crime, but not much about whether nothing at all can do anything about crime.
Even Stevenson didn't discount the classical "tough on crime" approaches as ineffective - since they were in general backed up by the research she cited - though she was careful to make those facts seem weaker than they were in order to fit in with the crowd of clearly-useless soft-on-crime interventions by complaining about lack of replication (which is not quite fair when it's much harder to do that kind of research if local politicians won't cooperate and grant awarders fear it might come up with the wrong answer) or that actions like "hot-spot policing" intended to have only short-term impact - get this - don't also have long-term impact.
Um, duh. This is like complaining that weeding your organic garden, while it's proven to work keeping weeds out of garden for a while, is not very good at keeping weeds out of your garden, "in some larger sense", because you have to keep doing it or else the weeds will come back. If you encounter someone who points this out like it's some kind of smart and important insight, are you impressed by them, or, in the alternative, do you pity them?
These kinds of studies only persuade people in government-sociology-etc. world.
If this were a set of private corporations, they would have blamed some people when something stopped working and fired some people or tried something else rather than to try to imagine that the same technique, program, or set of slogans (as it seems like that is what a lot of the "interventions" were in the article). They would not assume that the techniques that work in Pennsylvania will work in New Mexico. And in fact, in business, no one assumes that a technique that works in one year will work a year afterwards because the market changes.
Freddi DeBoer has a piece called Education Doesn't Work 2.0 (https://freddiedeboer.substack.com/p/education-doesnt-work-20) which is the definitive takedown of "education interventions". It's the lazy person's summary of what he wrote in his book "The Cult of Smart". Highly recommended.
Freddie says: "But there’s no question that in study after study, for decade after decade, researchers have found that interventions presumed to have causal influence on educational outcomes simply don’t. Almost everything is clustered right around zero. Most things we try in education simply don’t do anything. This is reality. Bet on the null."
But does the Null Hypothesis itself survive scrutiny for selection bias?
For example, one wonders if the most researchers (and people who decide what gets funded, published, reported, etc.) have been artificially constraining the space of "things we want to achieve and study" to precisely those limited areas that are politically-desirable objectives in conflict with ideologically-inconvenient truths about human reality.
That is, if you keep trying to do the impossible, and you keep failing, then it's not that "everything fails," it's that "choosing impossible is nuts," and "trying over and over and hoping for different results is insane."
With regards to things like crime, health, and education, doesn't it seem like the vast bulk of these studies all start out by heading in the wrong direction? That doesn't prove that there are no interventions which can work in those fields. That proves that if researchers are constantly trying to find a land-based bike path from Boston to Bermuda, that the problem is not with the whole enterprise of bike-path-discovering research, but with the dumb ideas in the brains of the researchers.
Exactly. Researchers are far more likely to study the impact of fad ideas like DEI training or Head Start than, say, maintaining discipline in the classroom.
Yes! This. An *obvious* change would be to lower, rather than raise as has been more commonly done, the age at which you may opt out of school.
Thus, rather than wasting so much energy on the worst-behaved students, which demoralizes teachers and makes a mockery of the classroom, of school itself - let them go.
Not to alternate school. Not home for 3 days and then you're back at it. Not - return but with an aide we have to pay to babysit you all day.
"Clearly you are not school material. You are dismissed. You needn't come back. If you want to come back, it will have to be very different. It's a matter of total indifference to us."
ETA: I'd suggest the school year you turn 13, could be your last.
Shout it from the rooftops.
Both K-12 and higher education are reliable bastions of Democratic Party electoral support. Spending more money there may not improve educational outcomes but it helps keep the Blues in power.
Education does not improve maximum gene/culture IQ of each person, tho bad edu might reduce it. Bad school might have a 100 IQ 16 y.o. person score only 95, or even only 90. But no school nor coaching gets him up to 105, much less college level 110.
What can happen is to have more training in making and doing things that low IQ folk can do, like assembly work. Or McDonald’s, tho they seem to far prefer to hire young who so often leave. K-12 schools should be far more concerned about the success of their low IQ students.
College level IQ 110? A sad comment on the dumbing down of higher education.
The piece reminds me of Jeffrey Friedman’s book _Power Without Knowledge_.
Here is an experiment worth trying as a pilot program:
Allow teens to replace high school with vouchers for internships at firms/orgs, apprenticeship programs, training programs, and the like.
Would entrepreneurs and/or established orgs develop new and better ways to prepare youths for the labor market? (Supply-side response)
The current one-size-fits-all approach systematically delays adulthood and alienates a large fraction of boys. Pace the null hypothesis, there should be radical experiments in new paths to enable teens to find their way, spend much more time with adults in the workplace (rather than most of their time with peers in a total institution whose main function is daycare), bring something to market, and stand on their own two feet.
Interesting ideas.
My own expected value of any UBI program is negative, because it incentivizes bad / lazy behavior. Social interventions fail because of the unwillingness of social planners to say behavior X is worse than behavior Y, so we need to reward Y behavior more. Instead, because the worse behavior X results in bad economic outcomes, the social program tries to equalize the outcome, thus subsidizing the bad behavior. Trying to get quite different results without a quite different behavior culture will naturally fail.
The unwillingness to do slut shaming, nor even agree on a negative phrase for uncaring, promiscuous men, is an example of this. For children, and society, promiscuous, slutty behavior leads to bad outcomes.
If I thought the problem of US Blacks was, actually, “systemic racism”, I would certainly be on board with shaming the racists. But the evidence according to behavior is that the Black poor are that way because of slut-jerk men who become the absent fathers of kids with mothers who choose to be sluts. Which increases other bad behavior: not studying/ learning in school, crime, drugs, bad work habits & jobs, and of course more promiscuity. Shameful behavior, done by Whites and Blacks, and deserving of cultural dis-incentive, shame.
Capitalism is pretty strictly a carrot only set of incentives—get the good results by good work, or not have a boss willing to peacefully pay you. Or else be poor, and shamed, because of your own failure.
But, yes, there are many cases, maybe 10% (tho I see no estimates nor measurement attempts in America) when it’s far more bad luck and others’ problems which create a terrible situation. Big temporary help for those folk does help, because their lifestyle choices are helpful.
This is the fundamental problem in dealing truly complex multi-variable problems where in a N dimensional space you try and describe the problem with N-X dimensions. It can't be done. If you think about conic sections from your math classes a long time ago you get an infinite number of 2-D objects from a point to an ellipse from sectioning a 3-D cone. Any relevant variable not included can sink the analysis.
This is how humanity from around the world took the night sky patterns (random dots in 2-D from a position on a rotating earth moving around the sun) and created Astrology models connecting that patterns to human events. Creating widely "believed" nonsense.
For example, minor family cultural values not included can have major effects. Visiting a Japanese friends house where the dinner conversation was about "which major university will you attend" instead of my fast car culture (LA in the 50's) shifted me from being an excellent mechanic like my advisers at school were recommending into a Ph.D. level scientist creating significant values for humanity as a whole.
In todays world of using check-box categories like race, ethnic, etc. which contain full overlapping distributions of other parameters of individuals from IQ to foot size any model based upon the check boxes only will be Astrology relative to individuals. With the modern non-STEM belief that you can exclude unmeasured variables, like the influence of a grandmother on a kids development, will be wrong.
The same problem exists with most biology and ecology where a microscopic parasite can control the distribution of cattle and people in Africa. What a bacterial virus (phage) does in your gut and impacts on your brain gets to a complexity level hard to imagine but still real to you as an individual. However, in science we do know that we can't exclude "inconvenient" variables like culture in our analysis.
Microcredit lending targeted at women was a big thing when I was involved in development assistance decades ago. I always thought it was nonsense, combining as it does what Stevenson calls 'the engineering view' with an earlier version of DEI. The thing to remember, however, is that development professionals earned hefty salaries implementing this and other types of interventions, as well as enjoying perks like travel (albeit often to places with less than ideal conditions) and attending conferences, as Stevenson mentions. This reminds me of an anecdote from a former colleague, about an economist who was a consultant on a different type of policy intervention in a developed country. Other economists were razzing this economist about whether the policy intervention provided the benefits it was intended to produce, and the economist responded to the effect that 'I don't know whether the policy is beneficial in general, but it has been greatly beneficial to me personally.'
Re: the null hypothesis in education interventions
Arnold and commenters, from time to time, have addressed a conundrum:
(a) Education interventions don't reliably and durably improve target outcomes.
(b) Education interventions seem to have a cumulative impact on normative beliefs of youths. This outcome may or may not be the explicit goal of the interventions, depending on the specific intervention. Sometimes it's brainwashing, sometimes it's an ideological side-effect.
Or maybe the real cause of ideological change among youths isn't schooling, but some other social factors; for example, prosperity or the massive growth of the entertainment industry?
Just read this on the goal education: to make people better.
https://modernwomaninrecovery.substack.com/p/what-is-the-ultimate-goal-of-education
Before arguing too much about any particular policy, there should be agreement about the goals. This goal, making people better, doesn’t include much testing of reading ability, none. That makes it a more universal goal, tho not as specific as I want for me my kids. But being better is quite a good goal.
If you reject race/IQ realism, you are naturally going to reject the null hypothesis. So your left trying this or that intervention to fix what “must” be a problem given the outcomes.
Race/IQ realism does not explain large gulfs in attainment. It doesn't explain 20% of a school reading at grade level.
Why not? To be blunt, why isn't "most of them aren't very bright" a good answer to "why do only 20% of them read at grade level?"
One has to ask "how bright do you have to be to read at grade level?" If 90% of IQ 100 kids can read at grade level with reasonably low levels of training and 60% of IQ 90 kids can do the same, that suggests one thing. If it is 100% of IQ 100 and 80% of IQ 90 that gives you another suggestion. I don't know what the numbers look like, what average IQ leads to 20% reading at grade level, but reading at grade level has never struck me as something terribly difficult.
So race/IQ might explain a lot of the gaps in achievement, but you aren't going to figure it out by looking at extremely high level statistics. You will need to look at school/class level statistics, because all of the relevant variables are at the individual and class level. What are these kid's IQs, how do they respond to these teachers over time, etc. I have seen some absolutely abysmal teachers and absolutely abysmal teaching practice, some that seem worse than doing nothing. I suspect that those on the lower ends of the IQ scale are more harmed by bad teaching than those on the upper end, not in just a linear fashion.
Actually, it’s largely because of the way in which reading is / has been “taught” for so many decades that kids’ reading ability has fallen off a cliff - ie the ideologically pushed “whole language” approach, which rejects any phonic element in the process. This can work for a minority who learn to decode/read words more or less intuitively. But for most kids it reduces the act of reading to a fruitless ‘look/say’ routine of guesswork,; whereas most kids need Explicit Instruction in order to sequentially develop basic skills in reading and mathematics - see Sweller et al re working memory. Take a look at the “reading wars” controversy, and at the OECD’s international PISA results if you want to see the relative decline in reading capacity across Anglo-Saxon countries, where the Left has throttled education for the last 50 years.
I have to admit I haven't thought of that.
Agreed, I taught my girls to read using phonics to good effect. I think there is more to it though, and the question “why did teachers stop teaching phonics?” really leads to a lot of more general insights into why kids are doing so much worse across the board.
I taught in a middling school district in Massachusetts and knew plenty of kids who could not read at "grade level". This was even though we had reading specialists to give special instruction and Massachusetts always comes out on top or near the top in the National Assessment of Educational Progress. I would be willing to bet substantial amounts of money that a large proportion of young people never read at grade level.
The education business is full of standards and "expectations" that lots of students never meet.
Yea, but that's just it: is it because they have never had good teachers, or because the kids are all dumb, or because the bar is really high? Or is it a hellacious mix of all three? My sense, particularly based on my experience teaching college, my mom teaching elementary reading for ever, and my dad working in youth detention centers (and my teaching there a very little bit) is that there is no real measurement of teaching skill at those low levels, and very little grasp of what works and what doesn't. New teaching techniques are driven almost entirely by fad or politics and not at all about what works to help kids read or get them interested in reading. By the time the kids are in middle or high school there is a terrible compounding effect where they didn't learn well at lower levels, then they didn't catch up the next and just keep falling behind. I have also seen remarkably clever delinquents who can't read well at intake but improve drastically over six months or so.
So at a high level, some good experienced teachers are really good at teaching kids to read (and have an interest in doing so, which is necessary I think) and very little understanding of how to teach other teachers to do that, not much interest in identifying it, and very little interest in getting rid of bad teachers and getting mediocre teachers to improve, or ability to tell the difference. There just isn't much interest in finding out what actually works, because it would involve doing a lot of things people don't like and there is a lot of money to be made with new initiatives to teach new and exciting ways, and if those don't work there is more money to be made selling a new one next year.
It is sad to compare what a new drug has to go through to get approved and what a new educational procedure does. For the drug, it's a number of stages (one of the first being that it doesn't make things worse, that it's "safe) of randomized controlled trials. For things in education, it's maybe a few underpowered studies and a happy feeling. Some times, changes even make things worse. A tragedy was the educational establishment deciding that phonics was right-wing and pushing it out in favor of "memorize the look of the word". That seems to be changing.
There are some teachers who can reach otherwise unreachable students but, as you say, they are rare and generally can't pass on their skills. Most teachers are in the broad middle, not bad, especially if the kids are middling bright and with middling motivation. Most people who read and comment in places like this have no idea how "not bright" many people are. Because they have no contact with them outside of structured situations like retail or food service. The bar is too high because of this and because people in the business think it is mean and undemocratic and possibly racist to think a lot of kids can't perform up to a high level. If I was a poet, I might call it toxic optimism.
Right, grade level reading is not that hard, and even if the whole school had a special education diagnosis IQ alone would probably not be the explanation.
IQ alone is never a complete explanation. But it is much better than "almost all the young people here are bright enough to meet the standards if we only just do it right". So since they're not, there must be lots of things we could be doing better. But every time we think we've found something, it doesn't work at scale.
Average IQ for blacks is 85. In depressed urban areas like say Baltimore it gets down to 75. 83 is the cutoff where are army won’t even accept you let enlistment because your too dumb to be useful.
It would not surprise me if a ghetto school lacked the mental capacity to have a majority of its kids read at “grade level”.
People just don’t understand how dumb people are.
Does the null hypothesis depend on the intervention occurring in a bureaucratic environment captured by interests whose primary goals are different from the goals of the intervention?
What if we could somehow reduce bureaucratic inertia and re-align the interests of education leaders (or criminal justice)? If the department of education were destroyed and the new, more agile interests controlling education were primarily concerned with student outcomes and didn’t care how much they needed to blow up the current system - then could we temporarily put aside the null hypothesis until the new leaders themselves were captured by other interest groups? (I’m thinking Mancur Olson “The Rise and Fall of Nations.”)
I'm curious what are the educational or criminal justice policies that have actually made a big positive difference? Mandatory high school education? The GI Bill?
That maze analogy is excellent, and I am going to steal it the next time I'm at a 12 step meeting and someone else there steals the "fall down a hole" story from the West Wing.
On the other hand we know that different countries have different results. They do something so in a different way and get different outcomes. I know it is very complex, but there must be a way to experiment with that, maybe hard to do but theoretically you could move people around and see if that changes their outcomes.
It is also true that many (most) RCTs are under-powered, sometimes due to financial constraints, sometimes due to over-optimistic effect size priors encourage by program advocates. And, somehow, many individuals (indeed, nearly all) eventually desist from criminal behavior, and the schools do actually manage to educate most students to some level of proficiency, and some students to high levels of proficiency.
No doubt many interventions don't work and some even make things worse but none of them have any benefit? Really?
- so if a health insurance lottery didn't help the winners, compared to what? Someone on Medicare or charity care? Is there a control group who has to pay for care?
- did social security reduce poverty and improve the well-being of elderly?
- did/does City college of New York provide a way out of poverty to many of the poor and immigrants? What about heavily subsidized community colleges? Isn't public K-12 an intervention?
If we can find 50 examples that don't work does that mean there aren't 10 examples that do? Color me skeptical of claims none work.
The problem with trying to do RCTs on interventions on criminals is that you will have a lot of definitional problems because of the many differences between states in criminal law. These differences are often quite arbitrary and significant. I think to do a better set of RCTs you should restrict it to programs of various kinds targeting federal offenders only. While you would still have the same problem across different circuits, in different prisons, and at different times, it would not hit you as badly as trying to compare things in State X vs State Y. There are similar issues that make comparing legal statistics between foreign countries a waste of time.
A lot of new District Attorney Researchers in many major Blue cities have had no trouble running good trials and getting solid, statistically significant, and replicated results about which interventions really work to lower crime.
That's because, when they just recently all implemented the total opposite of those interventions, crime shot way up, almost immediately. It's a good thing they've done this important research for us, because otherwise, who could have guessed?
The fact the crime rose so much and so quickly after application of the interventions allowed those District Attorney Researchers performing difference-in-difference analysis to assume that other local factors didn't change much and so could be ignored as causal factors when the same interventions cause the same kinds of crimes to shoot up similarly in a variety of different local scenes with difference local and state laws and other local factor. I mean, not varying in terms of which party is locally in charge, that's the same for all of them.
Seriously folks, come on. What works to reduce crime - and what will cause it to explode if you stop doing it - has been common sense for millennia.
As I mentioned previously, the whole problem with trying to infer a "null hypothesis for crime" from all those failed studies is because the vast majority of them were selected to try "progressivist opposite of traditional common sense" approaches. This proves something about the relative merits of progressivism and traditional common sense when it comes to crime, but not much about whether nothing at all can do anything about crime.
Even Stevenson didn't discount the classical "tough on crime" approaches as ineffective - since they were in general backed up by the research she cited - though she was careful to make those facts seem weaker than they were in order to fit in with the crowd of clearly-useless soft-on-crime interventions by complaining about lack of replication (which is not quite fair when it's much harder to do that kind of research if local politicians won't cooperate and grant awarders fear it might come up with the wrong answer) or that actions like "hot-spot policing" intended to have only short-term impact - get this - don't also have long-term impact.
Um, duh. This is like complaining that weeding your organic garden, while it's proven to work keeping weeds out of garden for a while, is not very good at keeping weeds out of your garden, "in some larger sense", because you have to keep doing it or else the weeds will come back. If you encounter someone who points this out like it's some kind of smart and important insight, are you impressed by them, or, in the alternative, do you pity them?
These kinds of studies only persuade people in government-sociology-etc. world.
If this were a set of private corporations, they would have blamed some people when something stopped working and fired some people or tried something else rather than to try to imagine that the same technique, program, or set of slogans (as it seems like that is what a lot of the "interventions" were in the article). They would not assume that the techniques that work in Pennsylvania will work in New Mexico. And in fact, in business, no one assumes that a technique that works in one year will work a year afterwards because the market changes.