Economists may think we have the biggest shovels, but does that mean we can build the best castles?
When I first read Freakonomics, I was captivated. I was an undergraduate economics major who already delighted in describing things to my family in terms of "opportunity cost," "marginal utility," and "widgets." I was enthralled with the idea of applying economic concepts and techniques to puzzles that had nothing to do with firms, finance, or fiscal policy. This, I realized, was what I wanted to do with my life: a) apply the study of incentives and game theory from economics to households, businesses, and individuals in the real world; b) use econometric techniques to understand questions in development and policy. It was only later, after furthering my study of economics, and reading and critiquing countless articles doing similar things, that I realized the difference between statements a and b. Economics is fundamentally about understanding scarcity, incentives, decisions, and tradeoffs. Econometrics has become a useful technique for studying how these forces have played out in the real world. Econometrics is a tool. Economics is a science. Yes, the tools of econometrics can be applied in many different settings. The question is, do economists have a competitive advantage in doing so?
And here is where my disillusionment with Freakonomics, and Freakonomics-style academic publications, began. You see, Freakonomics is, on balance, more about statement b than statement a. It is more about applying the tools of economics to counter-intuitive settings than using the theory of economics to explain real-world phenomena. For two books that take the opposite approach, see Tim Hartford's Undercover Economist and David Friedman's Hidden Order. To be sure, Freakonomics was meant to be a book of popular interest, not an economics tome. And the book deserves credit for bringing to a wide audience some of the most interesting research going on in academia, and showing its practical usefulness. Where I become concerned is in the concurrent rise of wacky econometrics articles in the mainstream academic press that value novelty over rigor: The Freakonomizing of economics itself.
The rise of Randomized Controlled Trials in development economics provides a useful example. RCTs offer an incredible opportunity for us to understand, once and for all, whether X causes Y in a given setting. Applying this to development allows economists to answer questions we could only speculate about previously: Do micronutrients improve health? Does class size matter? These questions are incredibly important in economic development policy. Yet, as the use of RCTs became more mainstream, they came under criticism for amounting to little more than program evaluations. "We shall experiment, but how shall we learn," worried development economist Dani Rodrik. His problem wasn't really that randomized experiments weren't teaching us anything--by all accounts they were. Rather, his worry was that these experiments weren't advancing economics. We might know that a certain program to do AB and C in African country J resulted in favorable outputs XY and Z, but what did that teach us about how people made decisions? What kind of incentives they responded best to? What policies were best for economic development as a whole? The reason this is worrying is that if these studies weren't designed to answer such questions, they weren't really economics studies at all. And if they weren't, why were economists doing them? Economists certainly hadn't invented RCTs, and we'd been using them for a lot less time than the medical profession. So what was the advantage of an economist running a randomized experiment, if it wasn't informed by economic theory? Using RCTs or regression analysis to try to explain everything turns economists into statisticians. Yes, we receive a lot of training on identifying significant causal effects in data, but I'm not convinced we have a competitive advantage over other disciplines in this pursuit.
Another example comes from the misadventures of a respected young economist in the field of demography. Emily Oster is by all accounts a brilliant economist, and has been championed by Freakonomics authors Levitt and Dubner. For her dissertation, Oster decided to try to use econometrics to investigate the relationship between sex ratios (male to female birthrates) and hepatitis B infections. She had seen some research suggesting women tended to have sons more frequently than daughters when infected with hepatitis B, and decided this might be responsible for the high male-female sex ratios in places like China, rather than the more conventional explanation that people were actively influencing the gender of their children (through selective abortion). Using regression analysis, she found that hepatitis B could be responsible for as much as 75% of China's missing female babies. Unfortunately, she neglected the fact that in China, the sex ratio is highly dependent on birth order--male babies are increasingly more likely the higher the number of previous girls, something that hepatitis B cannot explain, but voluntary sex selection can. Her findings were quickly debunked by a number of other economists, and she soon recanted. Note that her study employed econometrics, but not economics. Economics would have led her to look at the incentives in a society for having either a girl or a boy and investigate the relative costs of voluntary sex selection versus the perceived benefits to understand the plausibility of the accepted explanation before claiming the superiority of an alternate one.
Elsewhere, Oster has tried her hand at epidemiology, postulating that untreated STD infections play a large role in HIV transmission rates in Africa, and that previous HIV infection estimates were biased. Tim Hartford offered a critique of her work in a similar vein to this, worrying that the problem wasn't economists venturing into other fields, it was us presuming we knew those fields better than the people who in fact studied them. We think our superior toolkit gives us license to explore outside our sandbox. But, to belabor the metaphor, having a better set of shovels does not necessarily make one an expert on sandcastles.