If there are any doctors who are congratulating themselves for basing
their treatment decisions on rigorous clinical evidence--not the
questionable studies on things like homeopathy that my colleague Jerry
Adler wrote about in his health column this week--they can stop.
Physicians and even biomedical researchers may say they rely solely on
the gold standard of randomized clinical trials, but they’re as likely
to believe much weaker studies as is the most desperate cancer patient
combing the Web for laetrile.
So
finds an alarming study published this afternoon in the Journal of the
American Medical Association. Scientists led by John Ioannidis of the
University of Ioannina School of Medicine in Greece find that well-known
claims based on poor studies are repeated over and over again in
medical journals even when better studies contradict them. The earlier,
refuted conclusions continue to pop up in medical journals like satanic
whack-a-moles.
It’s important to understand why some ways of
doing a study are better than others. Less-good are observational
studies. In these, you basically watch and measure a group of people who
are doing something related to their health--women taking estrogen for
menopause, say--and compare them to a similar group who are not doing
that thing. In this example, observational studies concluded that
estrogen reduces women’s risk of heart disease, Alzheimer’s and other
bad things. But then a rigorous study looked at the same question (the
effect of estrogen), by randomly assigning some women volunteers to take
estrogen and others to take a dummy pill. Lo and behold, estrogen
suddenly didn’t look like such a good idea: it raised the risk of heart
disease, stroke and breast cancer, the Women’s Health Initiative reported in a blockbuster announcement in 2002.
Why
the different results in the two kinds of studies? One reason is that
in observational studies, women who chose to take estrogen are
inherently different from women who did not. They were, de facto,
seeing a doctor (for prescriptions if nothing else), and were likely
wealthier and better educated. Those three factors alone, not the
estrogen, might have made them healthier than non-estrogen women,
distorting the results.
The well-publicized negative verdict on
estrogen makes it hard for biomedical researchers to slip a sentence
into their papers off-handedly saying estrogen protects against heart
disease. Not so with other studies. To wit:
- Two
observational studies in 1993 suggested that vitamin E could decrease
the risk of developing cardiovascular disease by some 40 percent. But
randomized trials in 1996 and 2000 showed no benefit or even increased
harm. Yet papers in medical journals continued to cite the dubious
findings. In 2005, 50 percent of articles mentioning vitamin E and
cardiovascular disease cited the favorable study, leaving readers with
the erroneous impression that vitamin E helps prevent heart attacks and
stroke.
- A 1981 observational study claimed that beta-carotene
can prevent cancer. Randomized control studies in the mid-1990s refuted
that, finding that taking beta-carotene actually increases a person’s
risk of developing cancer. Yet 10 articles that mention the positive
1981 study do so favorably, while three are equivocal and only three are
(correctly) unfavorable.
- A 1996 observational study claimed
that estrogen can reduce older women’s risk of developing Alzheimer’s. A
randomized control study in 2004 refuted that. Yet 29 articles that
cited the 1996 study mentioned it favorably, while 14 were equivocal and
only four were unfavorable.
It’s difficult to fathom
why scientists can’t tell a rigorous study from a dubious one and keep
citing the latter. It’s fine to start with observational studies,
especially to generate hypotheses. But once a randomized control study
shows that the compound or other treatment originally deemed beneficial
is not, surely it’s time to stop saying otherwise. That’s especially
true when studies are available to anyone with an Internet connection to
PubMed.
By continuing to make health claims for vitamin E and heart health,
beta carotene and cancer, estrogen and Alzheimer’s, and who knows what
else that has been refuted, scientists mislead the public and promote
dangerous treatments. |