TY - JOUR
T1 - A risk assessment of automated treatment planning and recommendations for clinical deployment
AU - Kisling, Kelly
AU - Johnson, Jennifer L.
AU - Simonds, Hannah
AU - Zhang, Lifei
AU - Jhingran, Anuja
AU - Beadle, Beth M.
AU - Burger, Hester
AU - du Toit, Monique
AU - Joubert, Nanette
AU - Makufa, Remigio
AU - Shaw, William
AU - Trauernicht, Christoph
AU - Balter, Peter
AU - Howell, Rebecca M.
AU - Schmeler, Kathleen
AU - Court, Laurence
N1 - Publisher Copyright:
© 2019 The Authors. Medical Physics published by Wiley Periodicals, Inc. on behalf of American Association of Physicists in Medicine.
PY - 2019/6
Y1 - 2019/6
N2 - Purpose: To assess the risk of failure of a recently developed automated treatment planning tool, the radiation planning assistant (RPA), and to determine the reduction in these risks with implementation of a quality assurance (QA) program specifically designed for the RPA. Methods: We used failure mode and effects analysis (FMEA) to assess the risk of the RPA. The steps involved in the workflow of planning a four-field box treatment of cervical cancer with the RPA were identified. Then, the potential failure modes at each step and their causes were identified and scored according to their likelihood of occurrence, severity, and likelihood of going undetected. Additionally, the impact of the components of the QA program on the detectability of the failure modes was assessed. The QA program was designed to supplement a clinic's standard QA processes and consisted of three components: (a) automatic, independent verification of the results of automated planning; (b) automatic comparison of treatment parameters to expected values; and (c) guided manual checks of the treatment plan. A risk priority number (RPN) was calculated for each potential failure mode with and without use of the QA program. Results: In the RPA automated treatment planning workflow, we identified 68 potential failure modes with 113 causes. The average RPN was 91 without the QA program and 68 with the QA program (maximum RPNs were 504 and 315, respectively). The reduction in RPN was due to an improvement in the likelihood of detecting failures, resulting in lower detectability scores. The top-ranked failure modes included incorrect identification of the marked isocenter, inappropriate beam aperture definition, incorrect entry of the prescription into the RPA plan directive, and lack of a comprehensive plan review by the physician. Conclusions: Using FMEA, we assessed the risks in the clinical deployment of an automated treatment planning workflow and showed that a specialized QA program for the RPA, which included automatic QA techniques, improved the detectability of failures, reducing this risk. However, some residual risks persisted, which were similar to those found in manual treatment planning, and human error remained a major cause of potential failures. Through the risk analysis process, we identified three key aspects of safe deployment of automated planning: (a) user training on potential failure modes; (b) comprehensive manual plan review by physicians and physicists; and (c) automated QA of the treatment plan.
AB - Purpose: To assess the risk of failure of a recently developed automated treatment planning tool, the radiation planning assistant (RPA), and to determine the reduction in these risks with implementation of a quality assurance (QA) program specifically designed for the RPA. Methods: We used failure mode and effects analysis (FMEA) to assess the risk of the RPA. The steps involved in the workflow of planning a four-field box treatment of cervical cancer with the RPA were identified. Then, the potential failure modes at each step and their causes were identified and scored according to their likelihood of occurrence, severity, and likelihood of going undetected. Additionally, the impact of the components of the QA program on the detectability of the failure modes was assessed. The QA program was designed to supplement a clinic's standard QA processes and consisted of three components: (a) automatic, independent verification of the results of automated planning; (b) automatic comparison of treatment parameters to expected values; and (c) guided manual checks of the treatment plan. A risk priority number (RPN) was calculated for each potential failure mode with and without use of the QA program. Results: In the RPA automated treatment planning workflow, we identified 68 potential failure modes with 113 causes. The average RPN was 91 without the QA program and 68 with the QA program (maximum RPNs were 504 and 315, respectively). The reduction in RPN was due to an improvement in the likelihood of detecting failures, resulting in lower detectability scores. The top-ranked failure modes included incorrect identification of the marked isocenter, inappropriate beam aperture definition, incorrect entry of the prescription into the RPA plan directive, and lack of a comprehensive plan review by the physician. Conclusions: Using FMEA, we assessed the risks in the clinical deployment of an automated treatment planning workflow and showed that a specialized QA program for the RPA, which included automatic QA techniques, improved the detectability of failures, reducing this risk. However, some residual risks persisted, which were similar to those found in manual treatment planning, and human error remained a major cause of potential failures. Through the risk analysis process, we identified three key aspects of safe deployment of automated planning: (a) user training on potential failure modes; (b) comprehensive manual plan review by physicians and physicists; and (c) automated QA of the treatment plan.
KW - FMEA
KW - automated treatment planning
KW - external beam radiation therapy
KW - quality assurance
KW - risk analysis
UR - http://www.scopus.com/inward/record.url?scp=85065302381&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85065302381&partnerID=8YFLogxK
U2 - 10.1002/mp.13552
DO - 10.1002/mp.13552
M3 - Article
C2 - 31002389
AN - SCOPUS:85065302381
SN - 0094-2405
VL - 46
SP - 2567
EP - 2574
JO - Medical physics
JF - Medical physics
IS - 6
ER -