Research shows some single studies don't stand up under scrutiny

When it comes to original research studies, you shouldn't believe everything you read -- or at least you should wait until follow-up studies are done.

That was the essence of a review of major studies published in three influential medical journals between 1990 and 2003, including 45 highly publicized studies that initially claimed a drug or other treatment was effective.

Subsequent research contradicted results of seven studies -- 16 percent -- and reported weaker results for seven others, an additional 16 percent. In other words, nearly one-third of the original results did not hold up, according to a report of the study in a recent issue of the Journal of the American Medical Association.

Study author John Ioannidis, M.D., a researcher at the University of Ioannina in Greece, says contradictions and potentially exaggerated findings are not uncommon in the most visible and most influential original clinical research. Dr. Ioannidis' study looked at research published in the New England Journal of Medicine, JAMA and Lancet.

The refuted studies dealt with a wide range of drugs and treatments. For example, hormone pills, once thought to protect menopausal women from heart disease, were found in a later study to do the opposite, and vitamin E pills have not been shown to prevent heart attacks, contrary to initial results. Contradictions also included a study claiming that nitric oxide does not improve survival in patients with respiratory failure, despite earlier claims that it does, and one suggesting that an antibody treatment did not improve survival in certain sepsis patients -- a smaller, previously published study found the opposite.

According to some, the report is a reminder to doctors and patients that they should not put too much stock in a single study and that they should understand that treatments often become obsolete with medical advances. Dr. Ioannidis noted that in all 14 cases in which results were contradicted or softened, the subsequent studies were either larger or better designed. Also, none of the contradicted treatments is currently recommended by medical guidelines.

"A single study is not the final word, and that is an important message," says a statement issued by editors at the New England Journal of Medicine.