Economists may think we have the biggest shovels, but does that mean we can build the best castles?
And here is where my disillusionment with Freakonomics, and Freakonomics-style academic publications, began. You see, Freakonomics is, on balance, more about statement b than statement a. It is more about applying the tools of economics to counter-intuitive settings than using the theory of economics to explain real-world phenomena. For two books that take the opposite approach, see Tim Hartford's Undercover Economist and David Friedman's Hidden Order. To be sure, Freakonomics was meant to be a book of popular interest, not an economics tome. And the book deserves credit for bringing to a wide audience some of the most interesting research going on in academia, and showing its practical usefulness. Where I become concerned is in the concurrent rise of wacky econometrics articles in the mainstream academic press that value novelty over rigor: The Freakonomizing of economics itself.
The rise of Randomized Controlled Trials in development economics provides a useful example. RCTs offer an incredible opportunity for us to understand, once and for all, whether X causes Y in a given setting. Applying this to development allows economists to answer questions we could only speculate about previously: Do micronutrients improve health? Does class size matter? These questions are incredibly important in economic development policy. Yet, as the use of RCTs became more mainstream, they came under criticism for amounting to little more than program evaluations. "We shall experiment, but how shall we learn," worried development economist Dani Rodrik. His problem wasn't really that randomized experiments weren't teaching us anything--by all accounts they were. Rather, his worry was that these experiments weren't advancing economics. We might know that a certain program to do AB and C in African country J resulted in favorable outputs XY and Z, but what did that teach us about how people made decisions? What kind of incentives they responded best to? What policies were best for economic development as a whole? The reason this is worrying is that if these studies weren't designed to answer such questions, they weren't really economics studies at all. And if they weren't, why were economists doing them? Economists certainly hadn't invented RCTs, and we'd been using them for a lot less time than the medical profession. So what was the advantage of an economist running a randomized experiment, if it wasn't informed by economic theory? Using RCTs or regression analysis to try to explain everything turns economists into statisticians. Yes, we receive a lot of training on identifying significant causal effects in data, but I'm not convinced we have a competitive advantage over other disciplines in this pursuit.
Another example comes from the misadventures of a respected young economist in the field of demography. Emily Oster is by all accounts a brilliant economist, and has been championed by Freakonomics authors Levitt and Dubner. For her dissertation, Oster decided to try to use econometrics to investigate the relationship between sex ratios (male to female birthrates) and hepatitis B infections. She had seen some research suggesting women tended to have sons more frequently than daughters when infected with hepatitis B, and decided this might be responsible for the high male-female sex ratios in places like China, rather than the more conventional explanation that people were actively influencing the gender of their children (through selective abortion). Using regression analysis, she found that hepatitis B could be responsible for as much as 75% of China's missing female babies. Unfortunately, she neglected the fact that in China, the sex ratio is highly dependent on birth order--male babies are increasingly more likely the higher the number of previous girls, something that hepatitis B cannot explain, but voluntary sex selection can. Her findings were quickly debunked by a number of other economists, and she soon recanted. Note that her study employed econometrics, but not economics. Economics would have led her to look at the incentives in a society for having either a girl or a boy and investigate the relative costs of voluntary sex selection versus the perceived benefits to understand the plausibility of the accepted explanation before claiming the superiority of an alternate one.
Elsewhere, Oster has tried her hand at epidemiology, postulating that untreated STD infections play a large role in HIV transmission rates in Africa, and that previous HIV infection estimates were biased. Tim Hartford offered a critique of her work in a similar vein to this, worrying that the problem wasn't economists venturing into other fields, it was us presuming we knew those fields better than the people who in fact studied them. We think our superior toolkit gives us license to explore outside our sandbox. But, to belabor the metaphor, having a better set of shovels does not necessarily make one an expert on sandcastles.
I agree with you about this. One thing I would add though, is that I'm not sure there is that much of a split between a) and b). In particular, economic theorizing (thinking about incentives, decisions, tradeoffs) can also sometimes go wrong when applied in a setting about which an economist might not know as much about. For example, a theory where people maximize their utility by moving to the communities that best match their preferences (accounting for the costs of relocation, etc.) might not take into account the historical context of issues like racial segregation. And while the economic theory and its implications could be quite sound, without a real deep knowledge of the context, it could be taken to justify policies or opinions that might not be socially (or morally) optimal.
ReplyDeleteI've thought about this a lot as well, and I am no fan of Levitt's (kind of the opposite, actually.) However, I don't think it's all bad for economists to venture into other fields. The problem with Levitt is that he has gone into so many other fields, failing to gain expertise in any (the study of crime is perhaps the exception). I think there is a lot of value in being cross-disciplinary, ie an economist who also focuses on epidemics, or education, or crime. I think developing different specialties produces good analysis, but being a complete generalist does not.
ReplyDeleteI definitely agree! Check out the Tim Hartford piece on Oster--I think it makes sense to bring economic techniques and methodology to other fields, but the key thing is to be *interdisciplinary*, not to say "My discipline is better than yours." If Oster had done her study by collaborating with demographers who had worked in this area before, some of the issues might have naturally been brought out. I was just trying to argue that having the tools doesn't necessarily mean you should be the one doing the building--you need expertise, either your own (through lots of work in an area) or someone else's (though collaboration). But a lot of economists have taken a really snotty view in their papers, saying "this other field got this wrong." I think that does both fields a disservice!
ReplyDelete