Confidence Calibration: An Introduction With Application to Quality Improvement

Behrang Amini, Roland L. Bassett, Tamara Miner Haygood, Kevin W. McEnery, Michael L. Richardson

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

A probabilistic forecast is one that assigns a probability (or likelihood) to the occurrence of an event. Radiologists commonly make probabilistic judgments in their reports, even if these predictions are not explicitly expressed as numbers. There are calls for radiologists to commit to their probabilistic predictions in a standardized fashion; however, without a mechanism for feedback, there is no opportunity for improvement. Analysis techniques familiar to radiologists (eg, calculation of sensitivity and specificity and construction of receiver operating characteristics curves) have a blind spot with regard to calibration of these probabilities to reality and are the main obstacle to improvement along this axis. We review statistical and graphical methods for calibration analysis in wider use outside the medical literature and present a framework for implementation of these techniques for quality improvement and radiologist self-assessment.

Original languageEnglish (US)
Pages (from-to)620-628
Number of pages9
JournalJournal of the American College of Radiology
Volume17
Issue number5
DOIs
StatePublished - May 2020
Externally publishedYes

Keywords

  • Brier score
  • confidence calibration
  • physician judgment

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

MD Anderson CCSG core facilities

  • Biostatistics Resource Group

Fingerprint

Dive into the research topics of 'Confidence Calibration: An Introduction With Application to Quality Improvement'. Together they form a unique fingerprint.

Cite this