Jumping to Conclusions: Disease Burden and "Intelligence"

Last week, The Economist publicized some research by Christopher Eppig, a graduate student at the University of New Mexico. Eppig's research posits that the relationship between national disease burden and national IQ scores is causal, and that this decreased IQ level is thus a symptom, and not a cause, of underdevelopment in countries that lack health and hygiene infrastructures. The Economist summarizes how
Eppig and company reached their conclusion of causation (emphasis mine):

The correlation is about 67%, and the chance that it might have come about at random is less than one in 10,000. But correlation is not causation, so Mr Eppig and his colleagues tried to eliminate other possible explanations. Previous work has offered income, education, low levels of agricultural labour (which is replaced by more mentally stimulating jobs), climate (the challenge of surviving cold weather might provoke the evolution of intelligence) and even distance from humanity’s African homeland (novel environments could encourage greater intelligence) as explanations for national differences in IQ. However, all of these, except perhaps the last, are also likely to be linked to disease and, by careful statistical analysis, Mr Eppig and his colleagues show that all of them either disappear or are reduced to a small effect when the consequences of disease are taken into account.

Wow! They accounted for a lot of different stuff there! Do you think they accounted for every possible confounding factor? (Not possible!) It's also certain that the previous work they rely on included major ambiguities and assumptions in itself. And did you notice that the authors included distance from Africa as a cause for increased intelligence? It's like magic! Or really crappy analysis. I expected them to throw in craniometry as a potential confounding factor for good measure.

Now, a longstanding complaint of mine is that The Economist doesn't rigorously source articles, making it difficult for readers to go back to the studies and decide for themselves. Sometimes (oftentimes?) journalists don't understand a correct / responsible way to interpret data. So here's the full text of Eppig's article as published in Proceedings of the Royal Society.

First of all, I take issue with the scientific measurement of intelligence on such a broad, multi-cultural scale. Like love, joy, or pain, intelligence is something that is really difficult to define. And like, IQ tests, how bullshitty are those? Perhaps it is true that intelligence would vary on a national level - but I suspect that you would find even greater variance in definitions of intelligence across cultures. Even within our own culture, how would you describe a star mathematician, brilliant actor, and shrewd politician in a way that classified them all as equally intelligent.

Secondly, to develop these conclusions, Eppig combined the results of several different studies. This presents huge methodological issues which I don't even fully understand, but think of it like this: You have one set of data showing the price of apples in grocery stores in 30 of the 50 US states. Another set of data shows the average amount families spend on groceries in 40 of the states, with some degree of overlap in coverage. You combine the two data sets to show that apple prices are driving more than half of the variation in family grocery budgets! In my firm, we have to combine tons of data from various sources of differing levels of credibility and detail. Any of the analysts will tell you, that by the time you get to a final model, it's a pretty tenuous representation of reality. When assumptions build on assumptions to reach tentative conclusions, there's not much there there.

Thirdly, and not to be too much of a nitpicker, countries don't catch infectious diseases, people do (so use protection kids!) Wouldn't it be more meaningful to look at the effect of disease on individual's psychological abilities? Although it would still be difficult to ensure that those people who have diseases are comparable in all other ways to those who don't, this would get us closer to a reasonable conclusion.

This graphic from NewScientist perfectly encapsulates the study: "People born in a country with a high rate of infectious disease are likely to have a lower IQ." Even if they move? What about people who never get sick? Does treatment make a difference? Does the disease in question matter? (This graphic mentions tetanus and malaria - I was under the impression that these diseases had different effects on the human body, but maybe I'm crazy.) What about sub-geographies of low disease burden countries where there is a high rate of infectious disease? Would the results hold on a county level in the US? Like, seriously, WTF!

It was in looking for Eppig's current university post that I discovered he was a grad student currently managed by Dr Randy Thornhill, a zoologist with a research focus in evolutionary psychology (a field that itself gets pretty hocus-pocus, but let's not get too off-topic). Eppig's research interests are described as "sex steroids and dominance in men, concealment of estrus in women, estrus detection by men." As the great Dave Barry would say, I am not making this up. Boy would I like to hear what this guy has to say on gender interactions. I would bet the blog that it is complete bullshit. Does it count as "estrus detection" if my brother sees a tampon wrapper in the bathroom wastebasket? Why exactly is the nature of this researcher's fascination with menstruation, and why does it revolve around concealment and detection? It's like James Bond meets Are you there God? It's me Margaret. Also, if this is what Eppig studies, isn't he pretty far afield in publishing on infectious diseases and IQ distribution?

In conclusion, be a critical reader! And Economist, please, check your sources.