Optimising the computerised adaptive test to reliably reduce the burden of administering the CLEFT-Q: A Monte Carlo simulation study

Background: Computerised adaptive testing (CAT) has the potential to transform plastic surgery outcome measurement by making patient-reported outcome measures (PROMs) shorter, individualised and more accurate than pen-and-paper questionnaires. Objectives: This paper reports the results of two optimisation studies for the CLEFT-Q CAT, a CAT intended for use in the field of cleft lip and/or palate. Specifically, we aimed to identify the optimal score estimation and item selection methods for using this CAT in clinical practice. These represent two major components of any CAT algorithm. Method: Monte Carlo simulations were performed using simulated data in the R statistical computing environment and incorporated a range of score estimation and item selection techniques. The performance and accuracy of the CAT was assessed by mean items administered, correlation between CAT scores and paired linear assessment scores, and the root mean squared deviation (RMSD) of these score pairs. Results: The accuracy of the CLEFT-Q CAT was not significantly affected by the choice of score estimation or item selection method. Sub-scales which originally contain more items were amenable to greater item reduction with CAT. Conclusion: This study shows that score estimation and item selection methods that need minimal processing power can be used in the CLEFT-Q CAT without compromising accuracy. This means that the CLEFT-Q CAT could be administered quickly and efficiently with basic hardware demands. We recommend the use of less computationally intensive techniques in future CLEFT-Q CAT studies.