Overconfident Physician Opinion on the Effectiveness of BRCA1 Risk Reduction Measures

Introduction: We explored whether clinicians are overconfident in their judgments about the effectiveness of risk reduction measures in women with mutations in the BRCA1 gene. In this context, "overconfidence" is defined as the expression of too much certainty in subjective estimates, regardless of whether estimates are large or small. Methods: We asked physicians to estimate the percent decrease in the lifetime probability of breast and ovarian cancer in carriers who received various prophylactic interventions. Respondents were also asked to indicate their 90% plausibility interval. Subjects were breast cancer clinicians and principal investigators on NCI-sponsored Specialized Programs in Oncology Research and Education (SPOREs) in breast cancer at six US cancer centers. Results: Clinicians varied widely in their estimates of effectiveness. Many had plausibility intervals that did not include the best estimate offered by other clinicians. It was not uncommon to find two clinicians with plausibility intervals that did not overlap. In addition, many clinicians expressed 90%-plausibility intervals that were so narrow that they did not capture findings from large robust studies of the effectiveness of prophylaxis. While, by definition, 10% of clinicians should have been surprised to learn that a scientific finding was outside their 90% plausibility interval, we found that 34-67% would have been surprised. This is because their plausibility intervals were too narrow. Conclusion: We found that clinicians are overconfident in their estimates of the effectiveness of BRCA1 risk-reduction measures.