A Novel Dual Layer Cascade Reliability Framework for an Informed and Intuitive Clinician-AI Interaction In Diagnosis of Colorectal Cancer Polyps

We present a novel Cascade Reliability Framework (CRF) that integrates two independent cascade layers of reliability (i.e., variational temperature scaling and conformal prediction) with a pre-trained Machine Learning (ML) model in order to provide clinicians with a more reliable and tunable tool for early-stage diagnosis of Colorectal Cancer (CRC) polyps. The conformal prediction layer generates predictive sets that are guaranteed to contain the true polyp type with an adjustable error rate tuned by clinicians, while the confidence calibration generates meaningful confidence estimates for each predicted label. These two layers provide additional information and an error-tuning-ability for clinicians to assist them in making informed and intuitive decisions considering the outputs of the pre-trained ML model. Utilizing a novel vision-based tactile sensor and unique 3D-printed CRC polyp phantoms, we evaluated the trustworthiness of the proposed architecture and particularly dual outputs of four different types of CRF models, integrated with two different pre-trained ML models (i.e., ResNet18 and Dilated Residual Network) to highlight the model-agnostic feature of the architecture. To thoroughly assess the performance of the proposed approach, we used reliability diagrams and metrics such as accuracy, coverage, and average set size, while also addressing inter-class performance. Results demonstrate that the calibrated CRF models are well capable of handling non-ideal inputs with noise and blur. Moreover, using the conformal prediction with a user-defined error rate and various experiments, we show how clinicians can intuitively interact with a pre-trained ML model to make informed decisions and minimize the risk of CRC polyps misdiagnoses.