Having Rich Friends, 8/27

Questioning the causal inference

Re-reading Rob Henderson’s essay made me want to think more about the Raj Chetty studies.

He is the lead author on two recent studies on how youthful friendships affect one’s life prospects. To do this, Chetty and his colleagues analyzed the social networks of more than 70 million Facebook users. They concluded a key to upward mobility for poor kids is interacting, especially at school, with well-to-do ones.

… Chetty and his colleagues also describe “friending bias.” That is, when low-income and high-income kids go to the same school, they tend not to become friends. (In other words, the economists discovered class-based cliques.) The studies conclude that this bias is so powerful that even if every school in America was perfectly balanced by student income level, half of the social disconnect between the rich and the poor would remain. 

The interpretation of these studies is causal. That is, having rich friends causes you to be more likely to be rich.

In an interview with Russ Roberts, Chetty says,

If we look at kids who move at younger ages to such neighborhoods, they have better outcomes in the long run than kids who move to those same neighborhoods at later ages. And, under a set of identification assumptions--a set of assumptions that we think make our estimates of these neighborhood effects as good as causal--the way you can interpret this is: If I were to take a given child and put that child at a young age in a connected community, I would see better outcomes for that given child than if I were to put that child in a different community.

But causal inference with non-experimental data is always risky. There could be hidden variables that are driving the results.

In this case, suppose that there is a psychological determinant of success that the researchers do not observe. As a possible example, suppose that it is impulse control, and it is highly correlated with income. (I don’t mean perfectly correlated, just highly correlated.) Also suppose that it is highly correlated between parents and children.

Next, suppose that people with high impulse control and people with low impulse control do not enjoy being around one another. So people with high impulse control cluster together. When Chetty does his study it will look as though the children grew up rich because they lived among rich children. But in fact, impulse control is the causal factor.

Next, suppose that in a neighborhood with some rich children who mostly have high impulse control, there are poor children with different levels of impulse control. The poor children with high impulse control tend to befriend other children with high impulse control, some of whom happen to be rich. So it looks like having a “friend bias” toward rich kids helps you escape poverty, but in fact it is your impulse control that helps you escape poverty.

Just as a reminder, I am not saying that impulse control is the hidden factor here. There could be many psychological traits that matter. Also, note that it is possible for causality to go in the direction from the friends you choose to your psychological traits. Maybe if we ran an experiment in which some poor kids were exposed to rich kids as friends and others were not, we would find that impulse control (or other psychological traits) were affected.

Some of Chetty’s analysis may be robust to this sort of confounding effect that I think could raise doubts about causality. But I remain highly skeptical. Of all of the anti-poverty interventions that have ever been tried, only cash transfers such as the EITC seem to have an effect. The “base rate” would say that an intervention based on Chetty’s studies also would fail. Moreover, there are other results that go against the theory that having rich friends makes a difference. For example, the Dale and Krueger study showing that being admitted to a highly selective college but going elsewhere is as good for future prospects as going to the highly selective college suggests to me that meeting the rich kids at the highly selective college is not a life-changer.

Suppose that policy makers try an intervention that the Chetty study would predict an effect E on reducing poverty. I bet that in practice the intervention would have an effect less than .2E.