Why there are so many contradicted or exaggerated findings in highly-cited clinical research?

Mengyi Lu, Suyu Liu, Ying Yuan

Research output: Contribution to journalArticlepeer-review

Abstract

It is not uncommon that clinical studies of the same intervention contradicted with each other, e.g., one study produced positive results, while the other produced negative results. Ioanndis (2005a) found that among 49 highly-cited original clinical research studies, published in New England Journal of Medicine, Journal of the American Medical Association, Lancet or in a high-impact medical specialty journal, 32% of them were contradicted in subsequent large-scale studies, or were shown to have potentially overestimated the efficacy of the experimental intervention. This finding is disturbing and of serious concern given the widespread impact of these highly-cited studies and the rigorous standards used to design and conduct the studies. We perform Bayesian analysis of these highly-cited clinical studies based on Bayesian factor. We identified one cause of the issue: p values strongly overstated the experimental evidence. For the highly-cited studies, when the p value was 0.05, there was a 74.4% percentage chance that the null hypothesis was true. The use of a p value of 0.05 as the criterion for significance caused many researchers to mistakenly draw conclusions of positive findings, which were then contradicted by subsequent large-scale studies.

Original languageEnglish (US)
Article number106782
JournalContemporary Clinical Trials
Volume118
DOIs
StatePublished - Jul 2022

Keywords

  • Contradicted
  • Highly-cited
  • p values
  • Test statistics

ASJC Scopus subject areas

  • Pharmacology (medical)

MD Anderson CCSG core facilities

  • Biostatistics Resource Group

Fingerprint

Dive into the research topics of 'Why there are so many contradicted or exaggerated findings in highly-cited clinical research?'. Together they form a unique fingerprint.

Cite this