Scientific Literacy and a Storm in a Tea Party Cup

Tea Party conservatives were beside themselves when they discovered that a Yale political scientist named Daniel Kahan had seemingly admitted that Tea Party members were more scientifically literate than the general population. Kahan went on to say:

I’ve got to confess, though, I found this result surprising. As I pushed the button to run the analysis on my computer, I fully expected I’d be shown a modest negative correlation between identifying with the Tea Party and science comprehension.

But then again, I don’t know a single person who identifies with the Tea Party. All my impressions come from watching cable tv — & I don’t watch Fox News very often — and reading the “paper” (New York Times daily, plus a variety of politics-focused internet sites like Huffington Post & Politico).

I’m a little embarrassed, but mainly I’m just glad that I no longer hold this particular mistaken view.

Of course, I still subscribe to my various political and moral assessments–all very negative– of what I understand the “Tea Party movement” to stand for. I just no longer assume that the people who happen to hold those values are less likely than people who share my political outlooks to have acquired the sorts of knowledge and dispositions that a decent science comprehension scale measures.

Tea Party favorites like Glenn Beck were ecstatic:

Glenn Beck and his radio co-hosts Pat Gray and Stu Burguiere discussed the poll on Thursday…

All three agreed that it was a “positive” development to find someone “honest enough to go through and run the data.”

“I hope you have tenure, Dan, because Yale’s not going to keep you around, dude,” Beck said.

This led to some furious backtracking and accusations of anti-science from the usual suspects. Chris Mooney, author of The Republican War on Science, for instance, had this to say:

Beck and his co-hosts were citing the research of Yale psychology professor Dan Kahan, who had been analyzing some data and found a slight positive relationship between scores on a science “comprehension” scale and identifying with the tea party. How slight? For data nerds, the correlation was .05. Correlations range between 0 and 1, where 1 is a perfect positive correlation, and from 0 to -1, where -1 is a perfect negative correlation. In social science (or any science) a correlation of .05 would be considered “weak” or perhaps even “trivial,” because it is pretty close to zero. (For a very approachable explanation of correlations, see here.)

That’s true as far as it goes. The question begged by Mooney, however, is correlation to what. For the answer to that we have to turn as always to consultant statistician William M. Briggs, who does not mince words:

This isn’t Kahan’s first foray into “science comprehension.” He and several others were responsible for media-noted “The polarizing impact of science literacy and numeracy on perceived climate change risks.” (See notes below.)

Much the same thing: small trivia quiz matched with iffy political affiliation questions. Then an enormous and creaky statistical apparatus erected on top of these thin reeds with the purpose of teasing out infinitesimal differences. My God! Did they go on and on and on and then on some more? Did they ever. All completely worthless.

These kinds of “studies” come out with depressing regularity. By “these kinds” I mean those concocted by professors who have found ways to escape teaching by engaging in “research.” They gather in the nearest Starbucks (faculty lounges are passé), think of questions which they convince each other are profound and which support a fashionable theory, call the battery of questions an “instrument”, give it to a few students, then analyze, analyze, analyze.

Seriously, now. Just how far can you go with a question like “Lasers work by focusing sound waves [true/false]“? Could you tell how the country should best be run by gleaning insights from a survey which showed that only 69% of respondents knew the right answer? Could you tie the way a person answered that to some obscure psychological, jargon-laden theory? How plausible is it that most people’s politics and religious beliefs would cause people to answer correctly or incorrectly this particular question?

Why not require people to know what happens when a teaspoon of Seaborgium is mixed with a pinch of Livermorium as a prerequisite for marriage? How about insisting people know which has more mass, a neutrino or top quark, before allowing them to purchase soda pops larger than sixteen ounces?

In other words, Kahan’s “instrument” did not really measure scientific comprehension or its relevance to public policy. So when Mooney suggests that “Tea partiers are a teeny bit better at science comprehension than non-tea partiers—even as liberal Democrats are a teeny bit better at science comprehension than conservative Republicans,” and accuses Beck of missing the point (which he did), he himself misses the point. The naturalistic fallacy – “is implies ought” – is one thing. This fallacy – that knowing what one set of people know about one set of questions about science can inform our knowledge of what should be done about public policy – is one that GE Moore would have laughed at.

And that should hold true whether you’re a Tea Partier or a progressive.