Developing an Automatic Treatment Record Review System for Quality Assurance of Patient Treatment Delivery in Radiation Therapy

Developing an Automatic Treatment Record Review System for Quality Assurance of Patient Treatment Delivery in Radiation Therapy | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Developing an Automatic Treatment Record Review System for Quality Assurance of Patient Treatment Delivery in Radiation Therapy Peng Huang, Yingjie Xu, Fukui Huan, Yanxin Zhang, Min Ma, Kuo Men, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4432121/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 15 Jan, 2025 Read the published version in Radiation Oncology → Version 1 posted 9 You are reading this latest preprint version Abstract Background and Purpose Treatment record contains most of information related to treatment plan delivery in radiation therapy. Reviewing treatment record is an important quality assurance (QA) task for safety and quality of patient treatments. This task is usually performed by senior medical physicists. However, it is time-consuming, tedious, and error-prone. To assist this process, a treatment record review system (TRRS) is developed to automatically review items related to treatment delivery record. Methods The treatment record is firstly extracted from oncology information system(OIS). Based on the daily patient treatment information, the original plan from the treatment planning system is identified. Then the original plan and the delivered plan are correlated. Eight review categories (parameter consistency, treatment completeness, treatment progression, image guidance, override, treatment couch, documentation, and treatment mode) are defined. Tailored rules are designed for various review items to automate the review process. As a result, for each treatment record on a daily basis, a review flag (pass, failure, warning, and N/A) is determined by TRRS. Finally, this system is evaluated using six months patient treatment records collected in our institute and compared to the manual process on the same database. Results TRRS automatically reviewed a total of 76651 treatment fractions from 4230 patients with an average of 574 treatments per day. The average abnormality rate was 0.76%. The average processing time per treatment record was 3.9 seconds and 282 seconds for the automatic and manual processes, respectively. Comparing with the manual process, the time efficiency of TRRS is improved by a factor of 72. The average numbers of anomalies detected by the automatic and manual processes are 21 and 13 per day, respectively. TRRS detects 61.5% more anomalies than those of the manual process. Conclusion TRRS is not only efficient in processing a large amount of treatment records on a daily basis but also effective in finding more anomalies than those of physics weekly check. The application of the automatic review system could significantly reduce the work of review physicists and let them focus on more important works related to patient safety. treatment record plan delivery radiation therapy review anomaly Figures Figure 1 Figure 2 Figure 3 Figure 4 1. Introduction Radiotherapy became more complex and powerful in dealing with various clinical requirements. The modern radiotherapy techniques such as intensity-modulated radiation therapy (IMRT), volumetric modulated arc therapy (VMAT), and stereotactic body radiation therapy (SBRT) are capable of delivering high-precision dose to tumors while safeguarding the surrounding health tissues. On the other hand, a small mistake happened in one step of the whole process could result in serious problem at the end of treatment.[ 1 – 4 ] Therefore, a careful inspection of daily treatment accuracy of the radiotherapy plans is necessary. Shafiq et al. presented a survey on international radiotherapy incidents. They found 19% of 3,125 incidents happened in the treatment stage.[ 5 ] Ezzell et al. analyzed 173 problematic events and found 43% events happened in the treatment stage.[ 6 ] Treatment record review is a comprehensive inspection of various data associated with a patient's treatment, including the plan, delivery, patient setup and monitoring phases.[ 7 ] Eric et al. showed that weekly review of treatment record by a physicist could effectively reduce the occurrence of radiotherapy accidents. It was one of the most effective measures to ensure the quality control of patient treatment, with an effectiveness rate of more than 40%.[ 8 ] American Association of Physicists in Medicine (AAPM) Task Group (TG) 275 report and Medical Physics Practice Guideline (MPPG) 11.a further emphasized the importance of treatment record review in radiation therapy.[ 9 , 10 ] Both reports recommended that a Qualified Medical Physicist (QMP) should perform treatment record review at least weekly and document it. In brief, treatment record review plays a crucial role in ensuring the accuracy, quality, and safety of radiation therapy treatments. Manual review of treatment records is a time-consuming process, especially when dealing with a large amount of complex treatment plans.[ 9 ] It requires significant human resources, including the time and expertise of qualified personnel such as medical physicists. In hospitals where staffing was limited, allocating many physicists for treatment record reviews is difficult. Given the complexity of treatment plans, only relying on manual method may increase the risk of missing critical details. Physicists conducting manual reviews may also apply individual criteria. It could lead to inconsistencies between reviewers, affecting the reliability and uniformity of the quality assurance program. In addition, Manual review processes is mentally demanding. Repetitive work would lead to fatigue, which may affect the attention to detail and thoroughness of the review, potentially increasing the risk of oversights.[ 11 , 12 ] Several researchers developed methods to complement manual review process with computer-aided solutions.[ 13 – 18 ] Holdsworth et al. developed an in-house software called Verifier, which was designed to improve the efficacy and efficiency of radiation therapy treatment planning and quality control review.[ 19 ] Yang et al. introduced the development and implementation of a framework to automate the quality control (QC) step in radiotherapy treatment plan verification.[ 20 ] Currently, studies on automatic treatment record review are rare. Xia et al. developed an automated system called CATERS (Computer Aided Treatment Event Recognition System) to analyze electronic treatment records and detect treatment events in radiation therapy. The system improved the efficiency of treatment monitoring by automating the search for deviations from the physician's intention.[ 21 ] The physics group in our institute developed a treatment plan review system (TPRS), also called Automatic review (AutoReview), which improved the efficiency of review by nearly 60 times and increased the anomaly detection rate by 19.2% .[ 22 ] Based on the TPRS and the recommendation of AAPM TG275 report, we further developed an automatic treatment record review system (TRRS). TRRS was built upon the foundation of TPRS and integrates with the MOSAIQ Version 2.80 (Elekta Medical Systems, USA). It is expected that this automatic system could greatly improve the reliability and efficiency of treatment record review, and help physicians, physicists and therapists quickly and accurately detect errors and potential risks that may occur during the treatment process. 2. Methods and Materials 2.1 System architecture The system architecture of TRRS follows the B/S (Browser/Server) model, utilizing Java and HTML languages for programming. The primary program server operates on the Windows 2016 platform, with MySQL serving as the database management system. This architecture enables review physicists to access TRRS from any workstation within the hospital LAN via a standard web browser, facilitating the viewing and analysis of review results. It consists of five main components, data extraction, data processing, the core automated review program, parameter configuration, and review report generation, as shown in Fig. 1 . 2.2 Data acquisition The data type of TRRS was mainly structured data and from two sources as shown in Fig. 1 . The original plan was generated in TPS and transferred to MOSIAQ for machine delivery. After delivery, the treatment record was generated. The structured delivered plan data was extracted from the database of MOSAIQ and sent to TRRS for further analysis. The original plan was also transferred to TPRS for physics review. After review, the structured reviewed plan data was extracted from the database of TPRS and sent to TRRS for further analysis. Once both delivered and reviewed plan data were available, a one-to-one correspondence between them was established by matching key field. Specifically, the Prescription Unique Identifier (SIT_ID) within the MOSAIQ database was used for this correlation. 2.3 Review Items The review items are classified in eight categories: parameter consistency, treatment completeness, treatment progression, image guidance, override, treatment couch, documentation, and treatment mode. The descriptions of these items are listed in Table 1 . The review result of each treatment record is presented as one of four flags: Pass: the value is normal for this item. Failure: the value is incorrect for this item. Warning: the value is questionable and further manual review is needed. N/A: the value is no applicable for this item. Table 1 The descriptions of the review items Categories Details Parameter consistency • Consistency between treatment delivery parameters and treatment plan parameters, which included scrutiny of machine specifications, modality, energy, beam type, source-to-surface distance (SSD), segment count, monitor units (MU), gantry angle, collimator settings, couch angle, jaw configuration, and multi-leaf collimator (MLC) positions, etc. Treatment completeness • Delivery of all treatment fields in the plan • Delivered MU did not exceed planned MU Treatment progression • Cumulative dose and remaining dose • Accuracy of the remaining session • Consistency between the daily treatment dose and the prescribed dose • Treatment calendar that has been postponed for an extended period or discontinued altogether • Dose verification prior to stereotactic treatment Image guidance • Image approval in accordance with departmental policies • Applied shifts • Isocenter on the CBCT(cone-beam computed tomography) matched plan • Selection of IGRT (image-guided radiation therapy) scan template and parameters meet clinical requirements • IGRT frequency adherence to medical directives • IGRT registration deviation did not exceed predefined threshold Overrides • Override records, including instances such as couch position exceeding tolerance, inconsistent field parameters, abnormal dose tracking, treatment fractionation mode not consistent with the prescription, and any other deviations Treatment couch • Discrepancies between the treatment couch position (vertical, lateral, longitudinal, and rotational) and the reference couch position Documentation • Completeness of treatment-related documentation • Approval of documents by both the planning physicist and review physicist Treatment mode • Treatments completed out of clinical mode • The individual performing the QA model treatment 2.4 System design The core of the automatic review system resides on the server, where it systematically retrieves treatment records of all patients from the MOSAIQ system on a daily basis. These records contain data such as prescriptions, isocenters, treatment fields, positioning fields, treatment couch, and images. With patient prescription information, TRRS seamlessly matches and retrieves corresponding original plan data from TPRS database, establishing an association stored within the local database of TRRS. Tailored rules are designed for various review items to automate the review process. Upon completion, TRRS generates a detailed report for each treatment fractional review. Subsequently, the review physicist focused primarily on TPRS review results, manually scrutinizing and addressing any flagged abnormal items highlighted with prominent reminders in TRRS. Figure 2 illustrated the workflow of TRRS. 2.5 System Evaluation The treatment records over a period of six months from August 2023 to January 2024 were collected in our institute. Two senior physicists manually reviewed these treatment records meanwhile these data were also processed by TRRS. A comparative analysis was conducted between the manual review results and those processed by TRRS. The statistical analysis was performed using SPSS 21.0 software and χ2 test was employed to evaluate the consistency between the TRRS and manual review results. A significance level of P < 0.05 was established, with any disparities considered statistically significant. 3. Results TRRS automatically reviewed a total of 76,651 treatments from 4,230 patients with an average of 574 treatment fractions per day. The average abnormality rate was 0.76%. The result of daily treatment records processed by TRRS is shown in Fig. 3 . The list on the left is the summary of review results arranged by date. The statistics include the total number of patients reviewed, the count of patients passing the review, the number of cases marked as N/A, the total instances of warnings, the overall count of failures, the failure rate, and the breakdown of failures across eight distinct review categories. The list on the right shows the overall review results of the selected day. The information include medical record number, patient name, physician, treatment room, technique, treatment time, and the review outcomes for the eight review categories. Note that the technique area highlighted with green color indicates the patient is undergoing stereotactic therapy. The symbols are used to represent different review results. A "red cross" represents a failure. For instance, if the total monitor units (MU) of a patient's treatment fail to reach the planned value due to machine failure, the "completeness" item will display a red cross. An "yellow exclamation mark" represents a warning. For example, if a patient's treatment couch position deviation exceeds the specified lower limit but not the upper limit, the "couch position" item will display an orange exclamation mark. A "blue N/A" indicates that the review item is not applicable. For instance, in the case of electron beam therapy, IGRT checking is unnecessary, so the result will display blue N/A. Lastly, a "Green check" represents a pass. If there are no abnormal values detected, the review item will display a green check. The specific partial review results of a patient treatment records are shown in Fig. 4 . The "parameter consistency" item checks whether the detailed parameters of the treatment fields consistent with the original plan. The "Treatment completeness" item checks whether all treatment fields have been conducted as planned without any unexpected interruptions. Additionally, it checks if the total MU administered during the treatment session consistent with the planned value. The average processing time per treatment record is 3.9 seconds and 282 seconds for TRRS and manual process, respectively. Comparing with manual process, the time efficiency of TRRS is improved by a factor of 72. The average numbers of anomalies detected by the automatic and manual processes are 21 and 13 per day, respectively. TRRS detected 61.5% more anomalies than those of the manual process. Note that the anomalies detected by human operator are also included in the anomalies detected by TRRS. The statistical analysis indicates that no significant difference between the result of manual and automatic processes (P = 0.164). 4. Discussion The goal of treatment delivery is to execute the radiotherapy plan while patient is on the treatment couch, thereby the planning dose distribution in patient body can be achieved. The implementation of treatment delivery consists of two steps: (1) Positioning patient using body fixation and immobilization devices, (2) Operating treatment machine to execute treatment plan downloaded from treatment planning workstation. Both tasks highly rely on the proficiency and responsibility of radiation therapists. Under proper quality assurance of treatment machine and plan by engineers and physicists, the delivery accuracy of treatment plan can be guaranteed by well-trained therapists. However, the daily repetitive and high-demanding clinical work is susceptible to human errors. Despite advancements in treatment control system and record & verify (R&V) system aimed at reducing the likelihood of treatment errors, complete elimination these errors is impossible. Huang et al. conducted an analysis for a period of five years and found 555 errors among 28136 patient treatments (average 1.97 error per 100 patients).[ 2 ] Bissonnette et al. analyzed 1063 incident reports from 2001 to 2007, revealing an average incident rate of 1.7 per 100 radiotherapy courses.[ 3 ] The AAPM TG 275 provides a comprehensive list of inspection items relevant to the review of treatment records. For patients undergoing treatment, a minimum weekly review of treatment records is recommended. For patients undergoing Stereotactic Radiosurgery (SRS) or Stereotactic Body Radiotherapy (SBRT), a more rigorous review frequency of treatment records is advised to enhance quality assurance measures. In clinical practice, conducting a thorough review of treatment records often is less than that of treatment plans. The manual review of treatment records imposes a considerable work on clinical physicists.[ 13 – 18 ] Alternatively, many institutions opt to simplify review items or extend review intervals to reduce the workload. The introduction of computer-aided systems relieve physicists from this repetitive work and let them focus on more valuable tasks such as checking those anomalous treatment records detected by TRRS. Both TPRS and TRRS are QA procedures which are implemented in our department for clinical use. TRRS is an extension of TPRS. Upon completion of a radiotherapy plan, TRRS automatically checks relevant parameters to ensure plan delivery accuracy.[ 22 ] Previously, checking the consistency between the treatment plan and delivery is a challenging task. The implementation of TRRS perfectly solves this issue. During the treatment plan review process, TPRS extracts original radiotherapy plan files from the Treatment Planning System (TPS), converting them into structured data. It is then linked to the "site" table in the R&V system, allowing TRRS to accurately match the current treatment fraction to its corresponding original plan in TPRS using the "primary key" in the "site" table. Note that the current commercial R&V systems or the Treatment Management Systems (TMS) performs thorough consistency check between plan parameters during delivery and those store in the R&V system, which make it unnecessary to double checked by TRRS. The review items in TRRS are formulated with reference to the checklist recommended by the AAPM TG275 and MPPG11.a. In addition, they also based on many years’ experience on daily treatment record review in our department. The system includes most of potential anomalous events happened in radiotherapy plan delivery. The review rules are carefully designed to address problems in various clinical scenarios. With the rigorous tests by review physicists, the system is expected to minimize the false negative rate to zero at our best efforts. While comparing with the checklists provided by TG275 and MPPG11.a, the majority of the recommended review items were implemented in our TRRS except few of them which are no applicable in our institute, such as special instructions and in vivo dosimetry. It is cautious to devise overly strict tolerances or criteria. Stricter rules may result in high false positive rate, leading to unnecessary errors or warning message, and misleading the review physicist's attention. For instance, after completing positioning verification for the first treatment fraction using CBCT, TMS performs a 6D correction of the treatment couch based on the registration results on Edge (Varian Medical System). Consequently, there is a substantial deviation between the couch position recorded by the R&V system and the preset position before treatment. This kind of deviations can be judged as normal or anomalous events according to the different clinical protocols. Therefore, review rules should be carefully devised to avoid high false positive results. In conclusion, TRRS significantly improved the efficiency and effectiveness of reviewing process for daily patient treatment records of radiotherapy plans. The system not only extends the scope and frequency of review process but also promotes the detection rate of anomalies comparing to those of manual process. The implementation of TRRS can significantly relieve the workload of review physicists and enable them to focus on more important tasks related to the safety of patient treatment. Declarations Ethics approval and consent to participate The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The institutional Ethics Committee of the Cancer Hospital, Chinese Academy of Medical Sciences, and Peking Union Medical College approved this study. Informed consent was waived in this retrospective study by the institutional Ethics Committee of the Cancer Hospital, Chinese Academy of Medical Sciences, and Peking Union Medical College. The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Consent to publication Not applicable. Availability of data and materials The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. Competing interests The authors declare that they have no competing interests. Funding This work is supported by CAMS Innovation Fund for Medical Sciences(CIFMS), 2023-I2M-C&T-B-076. Authors’ contributions PH wrote the main manuscript text. PH, YX, FK, YZ and KM performed the experiments and have made substantial contributions to the conception. YZ and MM collected the data. PH and YX analyzed and interpreted the data. JD supervised the whole study. All authors wrote and have approved the manuscript. Acknowledgments Not applicable. References Ford EC, Evans SB. Incident learning in radiation oncology: A review. Med Phys. 2018;45:e100–19. Huang G, Medlam G, Lee J, Billingsley S, Bissonnette J-P, Ringash J, et al. Error in the delivery of radiation therapy: Results of a quality assurance review. Int J Radiat Oncol Biol Phys. 2005;61:1590–5. Bissonnette J-P, Medlam G. Trend analysis of radiation therapy incidents over seven years. Radiother Oncol. 2010;96:139–44. Greenwalt JC, Mittauer K, Liu C, Deraniyagala RL, Yeung AR. Reducing Errors in Radiation Treatment Through the Implementation of Electronic Safety Checklists. Int J Radiat Oncol Biol Phys. 2014;90:S128–9. Shafiq J, Barton M, Noble D, Lemer C, Donaldson LJ. An international review of patient safety measures in radiotherapy practice. Radiother Oncol. 2009;92:15–21. Ezzell G, Chera B, Dicker A, Ford E, Potters L, Santanam L, Weintraub S. Common error pathways seen in the RO-ILS data that demonstrate opportunities for improving treatment safety. Practical Radiation Oncol. 2018;8:123–32. Gopan O, Zeng J, Novak A, Nyflot M, Ford E. The effectiveness of pretreatment physics plan review for detecting errors in radiation therapy. Med Phys. 2016;43:5181–7. Ford EC, Terezakis S, Souranis A, Harris K, Gay H, Mutic S. Quality Control Quantification (QCQ): A Tool to Measure the Value of Quality Control Checks in Radiation Oncology. Int J Radiation Oncology*Biology*Physics. 2012;84:e263–9. Ford E, Conroy L, Dong L, Los Santos LF, Greener A, Gwe-Ya Kim G et al. Strategies for effective physics plan and chart review in radiation therapy: Report of AAPM Task Group 275. Med Phys. 2020;47. Xia P, Sintay BJ, Colussi VC, Chuang C, Lo YC, Schofield D, et al. Medical Physics Practice Guideline (MPPG) 11.a: Plan and chart review in external beam radiotherapy and brachytherapy. J Appl Clin Med Phys. 2021;22:4–19. Kalet AM, Luk SMH, Phillips MH. Radiation Therapy Quality Assurance Tasks and Tools: The Many Roles of Machine Learning. Med Phys. 2020;47:e168–77. Xu H, Zhang B, Guerrero M, Lee S-W, Lamichhane N, Chen S, Yi B. Toward automation of initial chart check for photon/electron EBRT: the clinical implementation of new AAPM task group reports and automation techniques. J Appl Clin Med Phys. 2021;22:234–45. Furhang EE, Dolan J, Sillanpaa JK, Harrison LB. Automating the initial physics chart-checking process. J Appl Clin Med Phys. 2009;10:129–35. Siochi RA, Pennington EC, Waldron TJ, Bayouth JE. Radiation therapy plan checks in a paperless clinic. J Appl Clin Med Phys. 2009;10:43–62. Sun B, Rangaraj D, Palaniswaamy G, Yaddanapudi S, Wooten O, Yang D, et al. Initial experience with TrueBeam trajectory log files for radiation therapy delivery verification. Practical Radiation Oncol. 2013;3:e199–208. Moore KL, Kagadis GC, McNutt TR, Moiseenko V, Mutic S. Vision 20/20: Automation and advanced computing in clinical radiation oncology. Med Phys. 2014;41:010901. Dewhurst JM, Lowe M, Hardy MJ, Boylan CJ, Whitehurst P, Rowbottom CG. AutoLock: a semiautomated system for radiotherapy treatment plan quality control. J Appl Clin Med Phys. 2015;16:339–50. Hadley SW, Kessler ML, Litzenberg DW, Lee C, Irrer J, Chen X, et al. SafetyNet: streamlining and automating QA in radiotherapy. J Appl Clin Med Phys. 2016;17:387–95. Holdsworth C, Kukluk J, Molodowitch C, Czerminska M, Hancox C, Cormack RA, et al. Computerized System for Safety Verification of External Beam Radiation Therapy Planning. Int J Radiation Oncology*Biology*Physics. 2017;98:691–8. Yang D, Moore KL. Automated radiotherapy treatment plan integrity verification. Med Phys. 2012;39:1542–51. Xia J, Mart C, Bayouth J. A computer aided treatment event recognition system in radiation therapy. Med Phys. 2014;41:011713. Huang P, Xu Y, Tian Y, Ma P, Dai J. Realization and application of automatic independent check software for radiotherapy treatment plans. Chin J Radiation Oncol. 2019;28:909–13. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 15 Jan, 2025 Read the published version in Radiation Oncology → Version 1 posted Editorial decision: Revision requested 15 Aug, 2024 Reviews received at journal 15 Aug, 2024 Reviewers agreed at journal 08 Aug, 2024 Reviews received at journal 09 Jun, 2024 Reviewers agreed at journal 20 May, 2024 Reviewers invited by journal 19 May, 2024 Editor assigned by journal 17 May, 2024 Submission checks completed at journal 16 May, 2024 First submitted to journal 16 May, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4432121","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":306817544,"identity":"240362e4-13cd-493e-83ab-1d9991ded867","order_by":0,"name":"Peng Huang","email":"","orcid":"","institution":"National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","correspondingAuthor":false,"prefix":"","firstName":"Peng","middleName":"","lastName":"Huang","suffix":""},{"id":306817545,"identity":"123fc917-5125-42f6-96e5-09bbbb9dc670","order_by":1,"name":"Yingjie Xu","email":"","orcid":"","institution":"National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","correspondingAuthor":false,"prefix":"","firstName":"Yingjie","middleName":"","lastName":"Xu","suffix":""},{"id":306817546,"identity":"39edf6a5-395c-42c3-a01e-d918172140b6","order_by":2,"name":"Fukui Huan","email":"","orcid":"","institution":"National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","correspondingAuthor":false,"prefix":"","firstName":"Fukui","middleName":"","lastName":"Huan","suffix":""},{"id":306817547,"identity":"a2f5c26b-d78a-48fe-82bb-fc4d48732605","order_by":3,"name":"Yanxin Zhang","email":"","orcid":"","institution":"National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","correspondingAuthor":false,"prefix":"","firstName":"Yanxin","middleName":"","lastName":"Zhang","suffix":""},{"id":306817548,"identity":"1819ad23-d2ea-4150-82d1-bbfbe4270016","order_by":4,"name":"Min Ma","email":"","orcid":"","institution":"National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","correspondingAuthor":false,"prefix":"","firstName":"Min","middleName":"","lastName":"Ma","suffix":""},{"id":306817549,"identity":"fb9fa56c-dd60-42bf-9026-0782da6cba24","order_by":5,"name":"Kuo Men","email":"","orcid":"","institution":"National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","correspondingAuthor":false,"prefix":"","firstName":"Kuo","middleName":"","lastName":"Men","suffix":""},{"id":306817550,"identity":"a1af4a3c-033a-4b19-be01-153adb323897","order_by":6,"name":"Jianrong Dai","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/0lEQVRIiWNgGAWjYDCCA4wNDBIGUM4HBgkStTDOIE4LEpuZhxh38R0/3MBgUWCXZ97ee/i1TY2FPP+04w8YftQwyJvj0CJ5JhHksORimTPn0qxzjkkYzridY8DYc4zBcGcDdi0GB8BamBNnSOSYGec2SCQw3M5hYOBtYEgwOIBDy/mHIC31EC2WQC3yt9MfMP7Fp+UG2JbDIC3GjxmBWgxuJxgw47NF8gbYluOJM3jOmAG9IGG4EeiXwzJAxgYcWvjOpz9glvhTnTiDvcf4w4+aOnm52+kPH76psZHHZQsQsP+Gxh8bPB6BivHHKeMHCM38Aa+yUTAKRsEoGLEAABYNWUMfq5iTAAAAAElFTkSuQmCC","orcid":"","institution":"National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","correspondingAuthor":true,"prefix":"","firstName":"Jianrong","middleName":"","lastName":"Dai","suffix":""}],"badges":[],"createdAt":"2024-05-16 15:34:53","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4432121/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4432121/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s13014-024-02582-8","type":"published","date":"2025-01-15T15:57:48+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":57519184,"identity":"9e3fed6b-d83e-407f-babc-a774ac4b633c","added_by":"auto","created_at":"2024-05-31 20:39:09","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":354392,"visible":true,"origin":"","legend":"\u003cp\u003eThe architecture of TRRS\u003c/p\u003e","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4432121/v1/aab7ef46e69c52572dfb561b.jpeg"},{"id":57519182,"identity":"9ed08060-9a06-44a2-82f6-39c70ad2c2ed","added_by":"auto","created_at":"2024-05-31 20:39:09","extension":"jpeg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":319187,"visible":true,"origin":"","legend":"\u003cp\u003eWorkflow of TRRS.\u003c/p\u003e","description":"","filename":"floatimage2.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4432121/v1/31aaf38b1bce530e973af6d1.jpeg"},{"id":57519181,"identity":"bbcd9db9-6b26-4b13-a2b2-72dae029b3c9","added_by":"auto","created_at":"2024-05-31 20:39:08","extension":"jpeg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":2267858,"visible":true,"origin":"","legend":"\u003cp\u003eThe review results processed by TRRS, where NOR: Number of reviews, P: Pass, N/A: Not available, W: Warning, F: Failure, FR: Failure rate, PC: Parameter consistency, TCmp: Treatment completeness, TP: Treatment progression, IG: Image guidance, OVR: Override, TC: Treatment couch, DOC: Documentation, TM: Treatment mode, MRN: Medical record number, TR: Treatment room, TECH: Technique.\u003c/p\u003e","description":"","filename":"floatimage3.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4432121/v1/e63a975daea4489c93e62284.jpeg"},{"id":57519183,"identity":"b225d88a-9568-422f-a400-17e5f5d87c7c","added_by":"auto","created_at":"2024-05-31 20:39:09","extension":"jpeg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":1918187,"visible":true,"origin":"","legend":"\u003cp\u003eThe partial review results of a patient treatment records at a selected day.\u003c/p\u003e","description":"","filename":"floatimage4.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4432121/v1/1903c78ed581920dd0e7fec8.jpeg"},{"id":74284658,"identity":"8bd5f9e3-cbb5-44b5-afdd-c8905f897d79","added_by":"auto","created_at":"2025-01-20 16:10:26","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":5357519,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4432121/v1/dcebc9d1-2eb7-42db-92a9-0c4aec5ed017.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Developing an Automatic Treatment Record Review System for Quality Assurance of Patient Treatment Delivery in Radiation Therapy","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eRadiotherapy became more complex and powerful in dealing with various clinical requirements. The modern radiotherapy techniques such as intensity-modulated radiation therapy (IMRT), volumetric modulated arc therapy (VMAT), and stereotactic body radiation therapy (SBRT) are capable of delivering high-precision dose to tumors while safeguarding the surrounding health tissues. On the other hand, a small mistake happened in one step of the whole process could result in serious problem at the end of treatment.[\u003cspan additionalcitationids=\"CR2 CR3\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e] Therefore, a careful inspection of daily treatment accuracy of the radiotherapy plans is necessary. Shafiq et al. presented a survey on international radiotherapy incidents. They found 19% of 3,125 incidents happened in the treatment stage.[\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e] Ezzell et al. analyzed 173 problematic events and found 43% events happened in the treatment stage.[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]\u003c/p\u003e \u003cp\u003eTreatment record review is a comprehensive inspection of various data associated with a patient's treatment, including the plan, delivery, patient setup and monitoring phases.[\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e] Eric et al. showed that weekly review of treatment record by a physicist could effectively reduce the occurrence of radiotherapy accidents. It was one of the most effective measures to ensure the quality control of patient treatment, with an effectiveness rate of more than 40%.[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e] American Association of Physicists in Medicine (AAPM) Task Group (TG) 275 report and Medical Physics Practice Guideline (MPPG) 11.a further emphasized the importance of treatment record review in radiation therapy.[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] Both reports recommended that a Qualified Medical Physicist (QMP) should perform treatment record review at least weekly and document it. In brief, treatment record review plays a crucial role in ensuring the accuracy, quality, and safety of radiation therapy treatments.\u003c/p\u003e \u003cp\u003eManual review of treatment records is a time-consuming process, especially when dealing with a large amount of complex treatment plans.[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e] It requires significant human resources, including the time and expertise of qualified personnel such as medical physicists. In hospitals where staffing was limited, allocating many physicists for treatment record reviews is difficult. Given the complexity of treatment plans, only relying on manual method may increase the risk of missing critical details. Physicists conducting manual reviews may also apply individual criteria. It could lead to inconsistencies between reviewers, affecting the reliability and uniformity of the quality assurance program. In addition, Manual review processes is mentally demanding. Repetitive work would lead to fatigue, which may affect the attention to detail and thoroughness of the review, potentially increasing the risk of oversights.[\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]\u003c/p\u003e \u003cp\u003eSeveral researchers developed methods to complement manual review process with computer-aided solutions.[\u003cspan additionalcitationids=\"CR14 CR15 CR16 CR17\" citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] Holdsworth et al. developed an in-house software called Verifier, which was designed to improve the efficacy and efficiency of radiation therapy treatment planning and quality control review.[\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] Yang et al. introduced the development and implementation of a framework to automate the quality control (QC) step in radiotherapy treatment plan verification.[\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e] Currently, studies on automatic treatment record review are rare. Xia et al. developed an automated system called CATERS (Computer Aided Treatment Event Recognition System) to analyze electronic treatment records and detect treatment events in radiation therapy. The system improved the efficiency of treatment monitoring by automating the search for deviations from the physician's intention.[\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]\u003c/p\u003e \u003cp\u003eThe physics group in our institute developed a treatment plan review system (TPRS), also called Automatic review (AutoReview), which improved the efficiency of review by nearly 60 times and increased the anomaly detection rate by 19.2% .[\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e] Based on the TPRS and the recommendation of AAPM TG275 report, we further developed an automatic treatment record review system (TRRS). TRRS was built upon the foundation of TPRS and integrates with the MOSAIQ Version 2.80 (Elekta Medical Systems, USA). It is expected that this automatic system could greatly improve the reliability and efficiency of treatment record review, and help physicians, physicists and therapists quickly and accurately detect errors and potential risks that may occur during the treatment process.\u003c/p\u003e"},{"header":"2. Methods and Materials","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1 System architecture\u003c/h2\u003e \u003cp\u003eThe system architecture of TRRS follows the B/S (Browser/Server) model, utilizing Java and HTML languages for programming. The primary program server operates on the Windows 2016 platform, with MySQL serving as the database management system. This architecture enables review physicists to access TRRS from any workstation within the hospital LAN via a standard web browser, facilitating the viewing and analysis of review results. It consists of five main components, data extraction, data processing, the core automated review program, parameter configuration, and review report generation, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2 Data acquisition\u003c/h2\u003e \u003cp\u003eThe data type of TRRS was mainly structured data and from two sources as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. The original plan was generated in TPS and transferred to MOSIAQ for machine delivery. After delivery, the treatment record was generated. The structured delivered plan data was extracted from the database of MOSAIQ and sent to TRRS for further analysis. The original plan was also transferred to TPRS for physics review. After review, the structured reviewed plan data was extracted from the database of TPRS and sent to TRRS for further analysis. Once both delivered and reviewed plan data were available, a one-to-one correspondence between them was established by matching key field. Specifically, the Prescription Unique Identifier (SIT_ID) within the MOSAIQ database was used for this correlation.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e2.3 Review Items\u003c/h2\u003e \u003cp\u003eThe review items are classified in eight categories: parameter consistency, treatment completeness, treatment progression, image guidance, override, treatment couch, documentation, and treatment mode. The descriptions of these items are listed in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. The review result of each treatment record is presented as one of four flags:\u003c/p\u003e \u003c/div\u003e\n\u003col\u003e\n \u003cli\u003ePass: the value is normal for this item.\u003c/li\u003e\n \u003cli\u003eFailure: the value is incorrect for this item.\u003c/li\u003e\n \u003cli\u003eWarning: the value is questionable and further manual review is needed.\u003c/li\u003e\n \u003cli\u003eN/A: the value is no applicable for this item.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eThe descriptions of the review items\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCategories\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDetails\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eParameter consistency\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026bull; Consistency between treatment delivery parameters and treatment plan parameters, which included scrutiny of machine specifications, modality, energy, beam type, source-to-surface distance (SSD), segment count, monitor units (MU), gantry angle, collimator settings, couch angle, jaw configuration, and multi-leaf collimator (MLC) positions, etc.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTreatment completeness\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026bull; Delivery of all treatment fields in the plan\u003c/p\u003e \u003cp\u003e\u0026bull; Delivered MU did not exceed planned MU\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTreatment progression\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026bull; Cumulative dose and remaining dose\u003c/p\u003e \u003cp\u003e\u0026bull; Accuracy of the remaining session\u003c/p\u003e \u003cp\u003e\u0026bull; Consistency between the daily treatment dose and the prescribed dose\u003c/p\u003e \u003cp\u003e\u0026bull; Treatment calendar that has been postponed for an extended period or discontinued altogether\u003c/p\u003e \u003cp\u003e\u0026bull; Dose verification prior to stereotactic treatment\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eImage guidance\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026bull; Image approval in accordance with departmental policies\u003c/p\u003e \u003cp\u003e\u0026bull; Applied shifts\u003c/p\u003e \u003cp\u003e\u0026bull; Isocenter on the CBCT(cone-beam computed tomography) matched plan\u003c/p\u003e \u003cp\u003e\u0026bull; Selection of IGRT (image-guided radiation therapy) scan template and parameters meet clinical requirements\u003c/p\u003e \u003cp\u003e\u0026bull; IGRT frequency adherence to medical directives\u003c/p\u003e \u003cp\u003e\u0026bull; IGRT registration deviation did not exceed predefined threshold\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOverrides\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026bull; Override records, including instances such as couch position exceeding tolerance, inconsistent field parameters, abnormal dose tracking, treatment fractionation mode not consistent with the prescription, and any other deviations\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTreatment couch\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026bull; Discrepancies between the treatment couch position (vertical, lateral, longitudinal, and rotational) and the reference couch position\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDocumentation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026bull; Completeness of treatment-related documentation\u003c/p\u003e \u003cp\u003e\u0026bull; Approval of documents by both the planning physicist and review physicist\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTreatment mode\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026bull; Treatments completed out of clinical mode\u003c/p\u003e \u003cp\u003e\u0026bull; The individual performing the QA model treatment\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e2.4 System design\u003c/h2\u003e \u003cp\u003eThe core of the automatic review system resides on the server, where it systematically retrieves treatment records of all patients from the MOSAIQ system on a daily basis. These records contain data such as prescriptions, isocenters, treatment fields, positioning fields, treatment couch, and images. With patient prescription information, TRRS seamlessly matches and retrieves corresponding original plan data from TPRS database, establishing an association stored within the local database of TRRS. Tailored rules are designed for various review items to automate the review process. Upon completion, TRRS generates a detailed report for each treatment fractional review. Subsequently, the review physicist focused primarily on TPRS review results, manually scrutinizing and addressing any flagged abnormal items highlighted with prominent reminders in TRRS. Figure\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e illustrated the workflow of TRRS.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e2.5 System Evaluation\u003c/h2\u003e \u003cp\u003eThe treatment records over a period of six months from August 2023 to January 2024 were collected in our institute. Two senior physicists manually reviewed these treatment records meanwhile these data were also processed by TRRS. A comparative analysis was conducted between the manual review results and those processed by TRRS. The statistical analysis was performed using SPSS 21.0 software and χ2 test was employed to evaluate the consistency between the TRRS and manual review results. A significance level of P\u0026thinsp;\u0026lt;\u0026thinsp;0.05 was established, with any disparities considered statistically significant.\u003c/p\u003e \u003c/div\u003e"},{"header":"3. Results","content":"\u003cp\u003eTRRS automatically reviewed a total of 76,651 treatments from 4,230 patients with an average of 574 treatment fractions per day. The average abnormality rate was 0.76%. The result of daily treatment records processed by TRRS is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. The list on the left is the summary of review results arranged by date. The statistics include the total number of patients reviewed, the count of patients passing the review, the number of cases marked as N/A, the total instances of warnings, the overall count of failures, the failure rate, and the breakdown of failures across eight distinct review categories. The list on the right shows the overall review results of the selected day. The information include medical record number, patient name, physician, treatment room, technique, treatment time, and the review outcomes for the eight review categories. Note that the technique area highlighted with green color indicates the patient is undergoing stereotactic therapy.\u003c/p\u003e \u003cp\u003eThe symbols are used to represent different review results. A \"red cross\" represents a failure. For instance, if the total monitor units (MU) of a patient's treatment fail to reach the planned value due to machine failure, the \"completeness\" item will display a red cross. An \"yellow exclamation mark\" represents a warning. For example, if a patient's treatment couch position deviation exceeds the specified lower limit but not the upper limit, the \"couch position\" item will display an orange exclamation mark. A \"blue N/A\" indicates that the review item is not applicable. For instance, in the case of electron beam therapy, IGRT checking is unnecessary, so the result will display blue N/A. Lastly, a \"Green check\" represents a pass. If there are no abnormal values detected, the review item will display a green check.\u003c/p\u003e \u003cp\u003eThe specific partial review results of a patient treatment records are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e. The \"parameter consistency\" item checks whether the detailed parameters of the treatment fields consistent with the original plan. The \"Treatment completeness\" item checks whether all treatment fields have been conducted as planned without any unexpected interruptions. Additionally, it checks if the total MU administered during the treatment session consistent with the planned value.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe average processing time per treatment record is 3.9 seconds and 282 seconds for TRRS and manual process, respectively. Comparing with manual process, the time efficiency of TRRS is improved by a factor of 72. The average numbers of anomalies detected by the automatic and manual processes are 21 and 13 per day, respectively. TRRS detected 61.5% more anomalies than those of the manual process. Note that the anomalies detected by human operator are also included in the anomalies detected by TRRS. The statistical analysis indicates that no significant difference between the result of manual and automatic processes (P\u0026thinsp;=\u0026thinsp;0.164).\u003c/p\u003e"},{"header":"4. Discussion","content":"\u003cp\u003eThe goal of treatment delivery is to execute the radiotherapy plan while patient is on the treatment couch, thereby the planning dose distribution in patient body can be achieved. The implementation of treatment delivery consists of two steps: (1) Positioning patient using body fixation and immobilization devices, (2) Operating treatment machine to execute treatment plan downloaded from treatment planning workstation. Both tasks highly rely on the proficiency and responsibility of radiation therapists. Under proper quality assurance of treatment machine and plan by engineers and physicists, the delivery accuracy of treatment plan can be guaranteed by well-trained therapists. However, the daily repetitive and high-demanding clinical work is susceptible to human errors. Despite advancements in treatment control system and record \u0026amp; verify (R\u0026amp;V) system aimed at reducing the likelihood of treatment errors, complete elimination these errors is impossible. Huang et al. conducted an analysis for a period of five years and found 555 errors among 28136 patient treatments (average 1.97 error per 100 patients).[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e] Bissonnette et al. analyzed 1063 incident reports from 2001 to 2007, revealing an average incident rate of 1.7 per 100 radiotherapy courses.[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]\u003c/p\u003e \u003cp\u003eThe AAPM TG 275 provides a comprehensive list of inspection items relevant to the review of treatment records. For patients undergoing treatment, a minimum weekly review of treatment records is recommended. For patients undergoing Stereotactic Radiosurgery (SRS) or Stereotactic Body Radiotherapy (SBRT), a more rigorous review frequency of treatment records is advised to enhance quality assurance measures. In clinical practice, conducting a thorough review of treatment records often is less than that of treatment plans. The manual review of treatment records imposes a considerable work on clinical physicists.[\u003cspan additionalcitationids=\"CR14 CR15 CR16 CR17\" citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] Alternatively, many institutions opt to simplify review items or extend review intervals to reduce the workload. The introduction of computer-aided systems relieve physicists from this repetitive work and let them focus on more valuable tasks such as checking those anomalous treatment records detected by TRRS.\u003c/p\u003e \u003cp\u003eBoth TPRS and TRRS are QA procedures which are implemented in our department for clinical use. TRRS is an extension of TPRS. Upon completion of a radiotherapy plan, TRRS automatically checks relevant parameters to ensure plan delivery accuracy.[\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e] Previously, checking the consistency between the treatment plan and delivery is a challenging task. The implementation of TRRS perfectly solves this issue. During the treatment plan review process, TPRS extracts original radiotherapy plan files from the Treatment Planning System (TPS), converting them into structured data. It is then linked to the \"site\" table in the R\u0026amp;V system, allowing TRRS to accurately match the current treatment fraction to its corresponding original plan in TPRS using the \"primary key\" in the \"site\" table. Note that the current commercial R\u0026amp;V systems or the Treatment Management Systems (TMS) performs thorough consistency check between plan parameters during delivery and those store in the R\u0026amp;V system, which make it unnecessary to double checked by TRRS.\u003c/p\u003e \u003cp\u003eThe review items in TRRS are formulated with reference to the checklist recommended by the AAPM TG275 and MPPG11.a. In addition, they also based on many years\u0026rsquo; experience on daily treatment record review in our department. The system includes most of potential anomalous events happened in radiotherapy plan delivery. The review rules are carefully designed to address problems in various clinical scenarios. With the rigorous tests by review physicists, the system is expected to minimize the false negative rate to zero at our best efforts. While comparing with the checklists provided by TG275 and MPPG11.a, the majority of the recommended review items were implemented in our TRRS except few of them which are no applicable in our institute, such as special instructions and in vivo dosimetry.\u003c/p\u003e \u003cp\u003eIt is cautious to devise overly strict tolerances or criteria. Stricter rules may result in high false positive rate, leading to unnecessary errors or warning message, and misleading the review physicist's attention. For instance, after completing positioning verification for the first treatment fraction using CBCT, TMS performs a 6D correction of the treatment couch based on the registration results on Edge (Varian Medical System). Consequently, there is a substantial deviation between the couch position recorded by the R\u0026amp;V system and the preset position before treatment. This kind of deviations can be judged as normal or anomalous events according to the different clinical protocols. Therefore, review rules should be carefully devised to avoid high false positive results.\u003c/p\u003e \u003cp\u003eIn conclusion, TRRS significantly improved the efficiency and effectiveness of reviewing process for daily patient treatment records of radiotherapy plans. The system not only extends the scope and frequency of review process but also promotes the detection rate of anomalies comparing to those of manual process. The implementation of TRRS can significantly relieve the workload of review physicists and enable them to focus on more important tasks related to the safety of patient treatment.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The institutional Ethics Committee of the Cancer Hospital, Chinese Academy of Medical Sciences, and Peking Union Medical College approved this study. Informed consent was waived in this retrospective study by the institutional Ethics Committee of the Cancer Hospital, Chinese Academy of Medical Sciences, and Peking Union Medical College. The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent to publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis work is supported by CAMS Innovation Fund for Medical Sciences(CIFMS), 2023-I2M-C\u0026amp;T-B-076.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026rsquo; contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003ePH wrote the main manuscript text. PH, YX, FK, YZ and KM performed the experiments and have made substantial contributions to the conception. YZ and MM collected the data. PH and YX analyzed and interpreted the data. JD supervised the whole study. All authors wrote and have approved the manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgments\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eFord EC, Evans SB. Incident learning in radiation oncology: A review. Med Phys. 2018;45:e100\u0026ndash;19.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuang G, Medlam G, Lee J, Billingsley S, Bissonnette J-P, Ringash J, et al. Error in the delivery of radiation therapy: Results of a quality assurance review. Int J Radiat Oncol Biol Phys. 2005;61:1590\u0026ndash;5.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBissonnette J-P, Medlam G. Trend analysis of radiation therapy incidents over seven years. Radiother Oncol. 2010;96:139\u0026ndash;44.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGreenwalt JC, Mittauer K, Liu C, Deraniyagala RL, Yeung AR. Reducing Errors in Radiation Treatment Through the Implementation of Electronic Safety Checklists. Int J Radiat Oncol Biol Phys. 2014;90:S128\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShafiq J, Barton M, Noble D, Lemer C, Donaldson LJ. An international review of patient safety measures in radiotherapy practice. Radiother Oncol. 2009;92:15\u0026ndash;21.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEzzell G, Chera B, Dicker A, Ford E, Potters L, Santanam L, Weintraub S. Common error pathways seen in the RO-ILS data that demonstrate opportunities for improving treatment safety. Practical Radiation Oncol. 2018;8:123\u0026ndash;32.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGopan O, Zeng J, Novak A, Nyflot M, Ford E. The effectiveness of pretreatment physics plan review for detecting errors in radiation therapy. Med Phys. 2016;43:5181\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFord EC, Terezakis S, Souranis A, Harris K, Gay H, Mutic S. Quality Control Quantification (QCQ): A Tool to Measure the Value of Quality Control Checks in Radiation Oncology. Int J Radiation Oncology*Biology*Physics. 2012;84:e263\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFord E, Conroy L, Dong L, Los Santos LF, Greener A, Gwe-Ya Kim G et al. Strategies for effective physics plan and chart review in radiation therapy: Report of AAPM Task Group 275. Med Phys. 2020;47.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXia P, Sintay BJ, Colussi VC, Chuang C, Lo YC, Schofield D, et al. Medical Physics Practice Guideline (MPPG) 11.a: Plan and chart review in external beam radiotherapy and brachytherapy. J Appl Clin Med Phys. 2021;22:4\u0026ndash;19.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKalet AM, Luk SMH, Phillips MH. Radiation Therapy Quality Assurance Tasks and Tools: The Many Roles of Machine Learning. Med Phys. 2020;47:e168\u0026ndash;77.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXu H, Zhang B, Guerrero M, Lee S-W, Lamichhane N, Chen S, Yi B. Toward automation of initial chart check for photon/electron EBRT: the clinical implementation of new AAPM task group reports and automation techniques. J Appl Clin Med Phys. 2021;22:234\u0026ndash;45.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFurhang EE, Dolan J, Sillanpaa JK, Harrison LB. Automating the initial physics chart-checking process. J Appl Clin Med Phys. 2009;10:129\u0026ndash;35.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSiochi RA, Pennington EC, Waldron TJ, Bayouth JE. Radiation therapy plan checks in a paperless clinic. J Appl Clin Med Phys. 2009;10:43\u0026ndash;62.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSun B, Rangaraj D, Palaniswaamy G, Yaddanapudi S, Wooten O, Yang D, et al. Initial experience with TrueBeam trajectory log files for radiation therapy delivery verification. Practical Radiation Oncol. 2013;3:e199\u0026ndash;208.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMoore KL, Kagadis GC, McNutt TR, Moiseenko V, Mutic S. Vision 20/20: Automation and advanced computing in clinical radiation oncology. Med Phys. 2014;41:010901.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDewhurst JM, Lowe M, Hardy MJ, Boylan CJ, Whitehurst P, Rowbottom CG. AutoLock: a semiautomated system for radiotherapy treatment plan quality control. J Appl Clin Med Phys. 2015;16:339\u0026ndash;50.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHadley SW, Kessler ML, Litzenberg DW, Lee C, Irrer J, Chen X, et al. SafetyNet: streamlining and automating QA in radiotherapy. J Appl Clin Med Phys. 2016;17:387\u0026ndash;95.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHoldsworth C, Kukluk J, Molodowitch C, Czerminska M, Hancox C, Cormack RA, et al. Computerized System for Safety Verification of External Beam Radiation Therapy Planning. Int J Radiation Oncology*Biology*Physics. 2017;98:691\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYang D, Moore KL. Automated radiotherapy treatment plan integrity verification. Med Phys. 2012;39:1542\u0026ndash;51.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXia J, Mart C, Bayouth J. A computer aided treatment event recognition system in radiation therapy. Med Phys. 2014;41:011713.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuang P, Xu Y, Tian Y, Ma P, Dai J. Realization and application of automatic independent check software for radiotherapy treatment plans. Chin J Radiation Oncol. 2019;28:909\u0026ndash;13.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"radiation-oncology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"raon","sideBox":"Learn more about [Radiation Oncology](http://ro-journal.biomedcentral.com/)","snPcode":"13014","submissionUrl":"https://submission.nature.com/new-submission/13014/3","title":"Radiation Oncology","twitterHandle":"@OncoBioMed","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"treatment record, plan delivery, radiation therapy, review, anomaly","lastPublishedDoi":"10.21203/rs.3.rs-4432121/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4432121/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground and Purpose\u003c/h2\u003e \u003cp\u003eTreatment record contains most of information related to treatment plan delivery in radiation therapy. Reviewing treatment record is an important quality assurance (QA) task for safety and quality of patient treatments. This task is usually performed by senior medical physicists. However, it is time-consuming, tedious, and error-prone. To assist this process, a treatment record review system (TRRS) is developed to automatically review items related to treatment delivery record.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eThe treatment record is firstly extracted from oncology information system(OIS). Based on the daily patient treatment information, the original plan from the treatment planning system is identified. Then the original plan and the delivered plan are correlated. Eight review categories (parameter consistency, treatment completeness, treatment progression, image guidance, override, treatment couch, documentation, and treatment mode) are defined. Tailored rules are designed for various review items to automate the review process. As a result, for each treatment record on a daily basis, a review flag (pass, failure, warning, and N/A) is determined by TRRS. Finally, this system is evaluated using six months patient treatment records collected in our institute and compared to the manual process on the same database.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eTRRS automatically reviewed a total of 76651 treatment fractions from 4230 patients with an average of 574 treatments per day. The average abnormality rate was 0.76%. The average processing time per treatment record was 3.9 seconds and 282 seconds for the automatic and manual processes, respectively. Comparing with the manual process, the time efficiency of TRRS is improved by a factor of 72. The average numbers of anomalies detected by the automatic and manual processes are 21 and 13 per day, respectively. TRRS detects 61.5% more anomalies than those of the manual process.\u003c/p\u003e\u003ch2\u003eConclusion\u003c/h2\u003e \u003cp\u003eTRRS is not only efficient in processing a large amount of treatment records on a daily basis but also effective in finding more anomalies than those of physics weekly check. The application of the automatic review system could significantly reduce the work of review physicists and let them focus on more important works related to patient safety.\u003c/p\u003e","manuscriptTitle":"Developing an Automatic Treatment Record Review System for Quality Assurance of Patient Treatment Delivery in Radiation Therapy","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-31 20:39:02","doi":"10.21203/rs.3.rs-4432121/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-08-15T10:23:32+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-08-15T09:13:33+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"318147930937586026917702892314235128879","date":"2024-08-08T08:40:03+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-06-09T12:02:03+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"339374076175635131386749540146671051560","date":"2024-05-20T19:38:58+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-05-19T09:18:23+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-05-17T18:36:26+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-05-17T02:35:34+00:00","index":"","fulltext":""},{"type":"submitted","content":"Radiation Oncology","date":"2024-05-16T15:33:42+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"radiation-oncology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"raon","sideBox":"Learn more about [Radiation Oncology](http://ro-journal.biomedcentral.com/)","snPcode":"13014","submissionUrl":"https://submission.nature.com/new-submission/13014/3","title":"Radiation Oncology","twitterHandle":"@OncoBioMed","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"748f785a-1255-458c-bbda-3e6a5c3efdb5","owner":[],"postedDate":"May 31st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-01-20T16:03:33+00:00","versionOfRecord":{"articleIdentity":"rs-4432121","link":"https://doi.org/10.1186/s13014-024-02582-8","journal":{"identity":"radiation-oncology","isVorOnly":false,"title":"Radiation Oncology"},"publishedOn":"2025-01-15 15:57:48","publishedOnDateReadable":"January 15th, 2025"},"versionCreatedAt":"2024-05-31 20:39:02","video":"","vorDoi":"10.1186/s13014-024-02582-8","vorDoiUrl":"https://doi.org/10.1186/s13014-024-02582-8","workflowStages":[]},"version":"v1","identity":"rs-4432121","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4432121","identity":"rs-4432121","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}