Recent Studies Suggest Too Much

Wall Street Journal science columnist Robert Lee Hotz has a fascinating profile of epidemiologist John Ioannidis and his view that because of unacknowledged biases and sloppy study design, “most published research findings are wrong.” Dr. Ioannidis, by the way, believes this is true not just in his area of speciality, but of peer-reviewed scientific studies across the board:

These flawed findings, for the most part, stem not from fraud or formal misconduct, but from more mundane misbehavior: miscalculation, poor study design or self-serving data analysis. “There is an increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims,” Dr. Ioannidis said. “A new claim about a research finding is more likely to be false than true.”

The hotter the field of research the more likely its published findings should be viewed skeptically, he determined.

[…]

Every new fact discovered through experiment represents a foothold in the unknown. In a wilderness of knowledge, it can be difficult to distinguish error from fraud, sloppiness from deception, eagerness from greed or, increasingly, scientific conviction from partisan passion. As scientific findings become fodder for political policy wars over matters from stem-cell research to global warming, even trivial errors and corrections can have larger consequences.

The column also goes on to make the point which is often made about inaccuracies in traditional newspapers like the Journal – that even when a retraction or correction is issued, it generally receives much less attention than the original inaccurate story or study.