TY - JOUR
T1 - Prevalence and implications of significance testing for baseline covariate imbalance in randomised cancer clinical trials
T2 - The Table 1 Fallacy
AU - Sherry, Alexander D.
AU - Msaouel, Pavlos
AU - McCaw, Zachary R.
AU - Abi Jaoude, Joseph
AU - Hsu, Eric J.
AU - Kouzy, Ramez
AU - Patel, Roshal
AU - Yang, Yumeng
AU - Lin, Timothy A.
AU - Taniguchi, Cullen M.
AU - Rödel, Claus
AU - Fokas, Emmanouil
AU - Tang, Chad
AU - Fuller, Clifton David
AU - Minsky, Bruce
AU - Meirson, Tomer
AU - Sun, Ryan
AU - Ludmir, Ethan B.
N1 - Publisher Copyright:
© 2023 The Authors
PY - 2023/11
Y1 - 2023/11
N2 - Background: The ‘Table 1 Fallacy’ refers to the unsound use of significance testing for comparing the distributions of baseline variables between randomised groups to draw erroneous conclusions about balance or imbalance. We performed a cross-sectional study of the Table 1 Fallacy in phase III oncology trials. Methods: From ClinicalTrials.gov, 1877 randomised trials were screened. Multivariable logistic regressions evaluated predictors of the Table 1 Fallacy. Results: A total of 765 randomised controlled trials involving 553,405 patients were analysed. The Table 1 Fallacy was observed in 25% of trials (188 of 765), with 3% of comparisons deemed significant (59 of 2353), approximating the typical 5% type I error assertion probability. Application of trial-level multiplicity corrections reduced the rate of significant findings to 0.3% (six of 2345 tests). Factors associated with lower odds of the Table 1 Fallacy included industry sponsorship (adjusted odds ratio [aOR] 0.29, 95% confidence interval [CI] 0.18–0.47; multiplicity-corrected P < 0.0001), larger trial size (≥795 versus <280 patients; aOR 0.32, 95% CI 0.19–0.53; multiplicity-corrected P = 0.0008), and publication in a European versus American journal (aOR 0.06, 95% CI 0.03–0.13; multiplicity-corrected P < 0.0001). Conclusions: This study highlights the persistence of the Table 1 Fallacy in contemporary oncology randomised controlled trials, with one of every four trials testing for baseline differences after randomisation. Significance testing is a suboptimal method for identifying unsound randomisation procedures and may encourage misleading inferences. Journal-level enforcement is a possible strategy to help mitigate this fallacy.
AB - Background: The ‘Table 1 Fallacy’ refers to the unsound use of significance testing for comparing the distributions of baseline variables between randomised groups to draw erroneous conclusions about balance or imbalance. We performed a cross-sectional study of the Table 1 Fallacy in phase III oncology trials. Methods: From ClinicalTrials.gov, 1877 randomised trials were screened. Multivariable logistic regressions evaluated predictors of the Table 1 Fallacy. Results: A total of 765 randomised controlled trials involving 553,405 patients were analysed. The Table 1 Fallacy was observed in 25% of trials (188 of 765), with 3% of comparisons deemed significant (59 of 2353), approximating the typical 5% type I error assertion probability. Application of trial-level multiplicity corrections reduced the rate of significant findings to 0.3% (six of 2345 tests). Factors associated with lower odds of the Table 1 Fallacy included industry sponsorship (adjusted odds ratio [aOR] 0.29, 95% confidence interval [CI] 0.18–0.47; multiplicity-corrected P < 0.0001), larger trial size (≥795 versus <280 patients; aOR 0.32, 95% CI 0.19–0.53; multiplicity-corrected P = 0.0008), and publication in a European versus American journal (aOR 0.06, 95% CI 0.03–0.13; multiplicity-corrected P < 0.0001). Conclusions: This study highlights the persistence of the Table 1 Fallacy in contemporary oncology randomised controlled trials, with one of every four trials testing for baseline differences after randomisation. Significance testing is a suboptimal method for identifying unsound randomisation procedures and may encourage misleading inferences. Journal-level enforcement is a possible strategy to help mitigate this fallacy.
KW - Covariate imbalance
KW - Oncology
KW - Phase III
KW - Randomised controlled trials
KW - Significance testing for baseline characteristics
KW - Table 1 Fallacy
KW - Testing for baseline differences
UR - http://www.scopus.com/inward/record.url?scp=85173272900&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85173272900&partnerID=8YFLogxK
U2 - 10.1016/j.ejca.2023.113357
DO - 10.1016/j.ejca.2023.113357
M3 - Article
C2 - 37827064
AN - SCOPUS:85173272900
SN - 0959-8049
VL - 194
JO - European Journal of Cancer
JF - European Journal of Cancer
M1 - 113357
ER -