One of the things I remember most clearly about my freshman year in high school was that my biology teacher, Mr. Richardson, warned us about the dangers of popular science. Not Popular Science the magazine, but popular science, as in scientific findings that are reported to news organizations for dissemination to the general public. He told us that the more popular the topic, the more likely it was that the researchers would be in more of a hurry to get the results out, and therefore more likely to turn up some specious results. Well, almost 20 years later, an article in the Wall Street Journal (jacked from Evangelical Outpost) is backing him up.
Here’s the money quote:
These flawed findings, for the most part, stem not from fraud or formal misconduct, but from more mundane misbehavior: miscalculation, poor study design or self-serving data analysis. “There is an increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims,” Dr. Ioannidis said. “A new claim about a research finding is more likely to be false than true.”The hotter the field of research the more likely its published findings should be viewed skeptically, he determined.
Take the discovery that the risk of disease may vary between men and women, depending on their genes. Studies have prominently reported such sex differences for hypertension, schizophrenia and multiple sclerosis, as well as lung cancer and heart attacks. In research published last month in the Journal of the American Medical Association, Dr. Ioannidis and his colleagues analyzed 432 published research claims concerning gender and genes.
Upon closer scrutiny, almost none of them held up. Only one was replicated.
The article is careful to point out that this trend is not necessarily indicative of any malevolence; sometimes it’s just messing with the variables to make the work say something significant or different. Even when the error on the scientist’s part is not necessarily malicious, this jack rabbit reporting gets into the hands of people who will use it with a definite agenda, despite the fact that the data doesn’t really what they say it does.
Think about this next time somebody comes at you about anthropomorphic global warming with religious fervency.
Entries (RSS)