Questioning the Authority of Scientific Journals
A Tufts University School of Medicine reporter has realized that a pretty large amount of scientific findings are, well, wrong. This work follows on from a recent publication of his that found that a third of medical research articles published in major scientific journals and then cited over a thousand times in the literature are later contradicted or have major questions raised over them. The reasons are many: <?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />
One of these factors is that many research studies are small. "The smaller the studies conducted in a scientific field, the less likely the research findings are to be true," says Ioannidis. Another problem is that in many scientific fields, the "effect sizes" (a measure of how much a risk factor such as smoking increases a person's risk of disease, or how much a treatment is likely to improve a disease) are small... Financial and other interests and prejudices can also lead to untrue results. And "the hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true," which may explain why we sometimes see "major excitement followed rapidly by severe disappointments in fields that draw wide attention."
I'm glad he mentions "other interests and prejudices." The reaction of some on the left to the news that the recent fetal pain study was conducted by abortion providers is especially interesting. Public choice theory suggests that there are a whole range of influences on researchers rather than just financial, yet current disclosure requirements focus exclusively on financial interests. I should not need to add that arguing that the results must be wrong because of conflicts of interest, whether financial or ideological, is a bad example of the ad hominem fallacy, but if we're going to consider the possible effects of conflicts of interest the we need to consider the whole range (I argue in my recent science paper that conflicts of interest are actually useful to the development of science). One other point; this is a useful conclusion:
We acknowledge that most studies published should be viewed as hypothesis-generating, rather than conclusive.
A shame that so many in the environmental movement choose to read scientific studies that cannot even be proved by experimental evidence as conclusive. A tragedy that so many in the scientific establishment agree with them.