The Politics of the Null Hypothesis

Stephanie Zvan in Scientific American:

In late April, Dr. Angela Lee Duckworth and her team published a study demonstrating that some of the variability in IQ test results–and in the life outcomes known to be correlated with IQ scores–varied significantly and substantially as a function of how motivated the test subject was. As the author herself points out in the paper, this is a fairly humdrum result. Those who developed IQ testing predicted that this would happen:

Despite efforts to “encourage in order that every one may do his best” on intelligence tests (ref. 41, p. 122), pioneers in intelligence testing took seriously the possibility that test takers might not, in fact, exert maximal effort. Thorndike, for instance, pointed out that although “all our measurements assume that the individual in question tries as hard as he can to make as high a score as possible . . . we rarely know the relation of any person’s effort to his maximum possible effort” (ref. 42, p. 228). Likewise, Wechsler recognized that intelligence is not all that intelligence tests test: “from 30% to 50% of the total factorial variance [in intelligence test scores remains] unaccounted for . . .this residual variance is largely contributed by such factors as drive, energy, impulsiveness, etc. . . .” (ref. 9, p. 444).

Yet this study that should be eliciting simple head nods was published in PNAS and is generating a fair amount of buzz. Ed Yong covers it nicely, emphasizing both underlying ability and motivation as factors in test results and educational and employment outcomes. ScienceNOW reports the findings, and Maria Konnikova of Artful Choice notes that motivation is a factor over which society has a certain amount of control.

The study is also receiving less positive notices. Steve Sailer at VDARE says the study tell us nothing new because IQ tests are still predictive, despite the researchers' determination that a model that includes motivation predicts life outcomes better than one that doesn't. StatSquatch runs a separate analysis taking out some of the data, but declines to submit the analysis as a peer-reviewed comment on the paper. And at EconLog, Bryan Caplan also visits the motivation factor:

For example, instead of saying, “IQ tests show that people are poor because they're less intelligent – and intelligence is hard to durably raise” we should say, “IQ tests show that people are poor because they're less intelligent and less motivated – and intelligence and motivation are hard to durable raise.” If, like me, you already believed in the Conscientiousness-poverty connection, that's no surprise.

The interesting thing about the disparity in views on this “non-controversial” study is how the views are divided. The straightforward reporting comes from science sites. The criticisms and assertions that the results are meaningless come from a linked group of political blogs. VDARE is an anti-immigration site; EconLog is a an economics blog. StatSquatch is perhaps most easily defined by the rate at which those on the blogroll perspire over “political correctness.”