The Big Data Paradox in Clinical Practice

Research output: Contribution to journalArticlepeer-review

19 Scopus citations

Abstract

The big data paradox is a real-world phenomenon whereby as the number of patients enrolled in a study increases, the probability that the confidence intervals from that study will include the truth decreases. This occurs in both observational and experimental studies, including randomized clinical trials, and should always be considered when clinicians are interpreting research data. Furthermore, as data quantity continues to increase in today’s era of big data, the paradox is becoming more pernicious. Herein, I consider three mechanisms that underlie this paradox, as well as three potential strategies to mitigate it: (1) improving data quality; (2) anticipating and modeling patient heterogeneity; (3) including the systematic error, not just the variance, in the estimation of error intervals.

Original languageEnglish (US)
Pages (from-to)567-576
Number of pages10
JournalCancer Investigation
Volume40
Issue number7
DOIs
StatePublished - 2022

Keywords

  • Bias-variance trade-off
  • big data
  • patient relevance
  • relevance-robustness trade-off
  • robustness

ASJC Scopus subject areas

  • Oncology
  • Cancer Research

Fingerprint

Dive into the research topics of 'The Big Data Paradox in Clinical Practice'. Together they form a unique fingerprint.

Cite this