Comparison of Large Language Model and Manual Review for Clinical Data Curation in Breast Cancer

Comparison of Large Language Model and Manual Review for Clinical Data Curation in Breast Cancer | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Comparison of Large Language Model and Manual Review for Clinical Data Curation in Breast Cancer Young-Joon Kang, Hocheol Lee, Jae Pak Yi, Hyobin Kim, Chang Ik Yoon, and 7 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7089616/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Purpose : Large language models (LLMs) offer the potential to automate the extraction of clinical data; however, few studies have been published on their feasibility in the field of oncology. To evaluate the feasibility and accuracy of LLM-based processing compared with manual physician review for the extraction of clinical data from breast cancer records. Results: The groups yielded comparable results for most clinical parameters. The LLM group yielded better documentation of lymph node assessment (91.2% vs. 78.5%) but had a larger proportion of missing data for cancer staging (12.2% vs. 3.1%). Breast-conserving surgery rates were similar (63.5% vs. 63.9%). The LLM achieved 90.8% accuracy in validation analysis while requiring significantly less processing time (12 days vs. 7 months) and fewer physicians (two vs. five). The LLM group’s stage distribution aligned better with the national registry data than the manual-review group(Cramér’sV = 0.03 vs. 0.07), and it captured more survival events (41 vs. 11; P = 0.002). Discussion : LLM-based processing demonstrated comparable effectiveness to manual review by physicians, while significantly reducing processing time and resource utilization. Despite limitations in integrated assessments, this approach shows potential for efficient clinical data curation in oncology research. Methods : This retrospective study analyzed breast cancer records from five academic hospitals (2019). Two independent cohorts were compared: manual physician review (n=1,366) and LLM-based processing using Claude 3.5 Sonnet (n=1,734) groups. Primary outcomes included missing value rates, accuracy of data extraction, and inter-cohort concordance. Secondary outcomes included comparison with national registry data, processing time, and resource utilization. Biological sciences/Cancer Health sciences/Health care Health sciences/Medical research Health sciences/Oncology Natural Language Processing Breast Neoplasms Data Mining Clinical Oncology Figures Figure 1 Figure 2 Figure 3 Introduction Recent advances in artificial intelligence (AI), particularly in large language models (LLMs), have demonstrated remarkable capabilities of automated data extraction and organization from complex clinical documents [1–3]. These AI-driven approaches can be used to process large volumes of clinical data with a consistent methodology, potentially reducing human bias and improving the collection efficiency of research data. Although LLMs show promise in healthcare applications, few studies on their practical efficacy compared with that of traditional, manual processing by physicians have been published, particularly in complex areas such as the extraction of surgical oncology data [4]. In the field of breast cancer surgery, retrospective data analysis presents unique challenges owing to the complexity of unstructured clinical data. Electronic health records (EHRs) contain diverse information across clinical charts, operation records, and pathology reports, often in free-text format. The complexity is compounded by breast cancer-specific characteristics, including bilateral nature of the organs, concurrent malignant and benign lesions, and multiple radiological features. This complicates automated data curation and has traditionally necessitated manual review by physicians for accurate data interpretation and collection. However, the manual approach has important limitations. As the volume of clinical data increases, consistency in physician reviews becomes increasingly difficult to maintain, potentially leading to discrepancies in data interpretation [5, 6]. The time-intensive nature of manual review and the risk of errors in the processing of large volumes of clinical data present considerable challenges in retrospective research [7–10]. In addition, direct EHR access for manual data extraction raises privacy concerns when sensitive patient information is handled [11–13]. Although the LLM-based automation of information extraction from anonymized EHR data may address these challenges, its effectiveness compared with that of traditional physician review remains to be evaluated. Although LLMs have been validated for the extraction of specific medical data, their potential for the curation of comprehensive data of patients with cancer remains largely unexplored [14]. In this study, we compared traditional physician review with LLM-based processing of anonymized clinical data in the field of breast cancer, focusing on the development of a practical approach for surgical oncologists. We hypothesized that LLM-based analysis would yield comparable results to manual review in the handling of large volumes of clinical data while reducing the processing time and resource utilization. Results Outcomes The manual review (group 1) and LLM processing (group 2) groups comprised 1,366 and 1,734 cases, respectively. Although both groups completely captured age data, they exhibited different patterns of missing data for other parameters. Group 2 had higher missing rates in terms of cancer stage (12.2% vs. 3.1%) and HER2 status (15.1% vs. 11.0%), whereas group 1 had more missing data for lesion size (20.5% vs. 5.9%) and lymph node assessment (21.5% vs. 8.8%). Both groups maintained high documentation rates (>90%) for hormone receptor status (Fig. 2). The validation analysis encompassed 1,800 data points (900 per group) across clinical factors. Group 1 demonstrated perfect accuracy with no discrepancies. Group 2 exhibited 83 discordant factors out of 900 data points, achieving an accuracy rate of 90.8% (817/900), which exceeds the predetermined threshold of 90% for acceptable performance. Demographics and Clinical Characteristics The baseline characteristics of both groups are summarized in Table 1. The mean age differed slightly between groups 1 and 2 (55.0 vs. 53.5 years, P < 0.001, Cohen’s d = 0.13). For breast surgery, total mastectomy was performed in 19.2% of cases in group 1 and 26.5% in group 2, while nipple (skin)-sparing mastectomy rates were 15.7% and 9.6%, respectively (χ 2 = 164.3, degrees of freedom [df] = 3, P < 0.001, Cramér’s V = 0.29). When these procedures were combined, groups 1 and 2 exhibited similar proportions of breast-conserving surgery (63.5% vs. 63.9%) and mastectomy (34.8% vs. 36.0%), with a small effect size (Cramér’s V = 0.10). In terms of axillary surgery, group 1 had a higher rate of sentinel lymph node biopsy (SLNB) than group 2 (68.4% vs. 59.7%), whereas the rates of axillary lymph node dissection (ALND) were similar (χ 2 = 47.2, df = 2, P < 0.001, Cramér’s V = 0.13). The mean ± standard deviation number of harvested lymph nodes was similar between the groups (7.79 ± 7.03 vs. 7.11 ± 7.20, P = 0.016). Stage distribution differed between groups (χ 2 = 68.9, df = 8, P < 0.001), but only slightly (Cramér’s V = 0.16), with group 1 identifying more cancers as advanced. The hormone receptor status was similar between the groups (estrogen receptor: 78.3% vs. 76.2%, P = 0.172; progesterone receptor: 68.7% vs. 67.5%, P = 0.525). The HER2 status differed negligibly and Ki67 expression was similar between the groups (HER2: P=0.003, Cramér’s V = 0.003; Ki67: P = 0.391, Cramér’s V ≈ 0.00). Histological grade distributions were similar between the groups ( P = 0.764). Interrater Agreement Analysis ICC analysis of continuous variables demonstrated a consistently low agreement: age (ICC = 0.013, 95% confidence interval [CI]: -0.035-0.060), tumor size (ICC = 0.029, 95% CI: -0.021-0.078), number of metastatic lymph nodes (ICC = 0.031, 95% CI: -0.019-0.081), number of harvested lymph nodes (ICC = 0.025, 95% CI: -0.025-0.075), and Ki67 expression (ICC = 0.027, 95% CI: -0.023-0.077). All ICC values were negligible and all CIs included zero. Survival Outcomes Survival analysis revealed significant differences between the groups (hazard ratio [HR] = 2.917, 95% CI: 1.496-5.688, P = 0.002). Group 2 captured more events (11 vs. 41). The proportional hazards assumption was met (χ 2 = 2.37, P =0.12), and the log-rank test confirmed a difference in survival distributions (χ 2 = 10.9, P = 0.001). Comparison with National Registry Data Comparison with the KBCS 2019 registry data (n = 9,447) revealed small differences in breast surgery patterns for both groups (Cramér’s V = 0.03-0.04, P < 0.001; Fig. 3A). In axillary surgery, both groups had lower SLNB rates (group 1: 68.3% and group 2: 59.7% vs. KBCS: 73.2%) and similar ALND rates (20.5% vs. 20.9% vs. 18.6%; Fig. 3B). Group 2 had a higher rate of no axillary surgery than group 1 (19.4% vs. 11.2%). Stage distribution analysis revealed significant but small differences from the national data (group 1: Cramér’s V = 0.07, P < 0.001; Group 2: Cramér’s V = 0.03, P = 0.003; Fig. 3C). Regarding biomarker subtypes, both groups had slightly higher proportions of HR-positive/HER2-negative (group 1: 67.0% and group 2: 67.5% vs. KBCS: 63.1%) and triple-negative cases (12.7% and 12.5% vs. 12.0%) with minimal effect sizes (Cramér’s V = 0.03-0.04; Fig. 3D). Discussion In this study, we compared traditional manual reviews by physicians with LLM processing of anonymized data for breast cancer clinical research. Although LLMs have yielded favorable results in the extraction of data from radiology and pathology reports, surgical oncology presents unique challenges owing to its requirement for the complex integration of multiple clinical parameters and temporal relationships [15-18]. Our study addresses this gap as the first comprehensive evaluation of the utility of LLMs in surgical oncology data curation. Whereas statistical differences were observed between the groups, most were clinically insignificant, suggesting that LLM processing may be a viable alternative to manual review despite the challenges. Digital extraction exhibited advantages in case identification and collection of survival data, achieving an accuracy > 90%. LLM processing performed well with continuous variables such as tumor size and lymph node assessment but had limitations in integrated analysis such as that of surgery methods and staging. This suggests that, although LLM-based processing effectively captures most clinical categories, it needs to be refined in terms of complex clinical information that required integrated analysis. The patterns of missing data differed between the groups: missing data upon LLM processing primarily stemmed from the initial CDW extraction, whereas manual review was vulnerable to individual variations in data collection, as evidenced by clustering of missing values in the documentation of nodal assessment (21.5% missing). Such reviewer-dependent variations highlight a fundamental limitation of manual review, as maintaining consistency across multiple reviewers can be challenging despite using standardized forms. Although clinical specialists are ideally suited for disease-specific data curation, technical barriers and the need for continuous collaboration with programming experts often limit their direct involvement in large-scale data analysis. In this context, LLM processing offers an accessible solution for clinicians, demonstrating vast efficiency gains; LLM processing over 12 days by two physicians yielded comparable accuracy to manual review over 7 months by five physicians for the same study period. The higher number of cases in the LLM group reflects the advantages of automated extraction rather than specific LLM capabilities. The axillary surgery data demonstrated the strengths and limitations of LLM-based processing in integrated analysis. The LLM group indicated a close alignment between the no axillary surgery rate (19.4%) and the in situ cancer rate (20.3%), whereas manual review suggested a difference (11.2% vs. 18.2%). The difference revealed by manual review reflects real-world clinical practice, in which SLNB is often performed in patients with DCIS, who are at a high risk of invasion [19]. The discrepancy between the data extraction methods highlights a key limitation of LLM processing: whereas manual reviewers can identify and integrate multiple surgical steps (such as lymph node assessment after initial diagnostic excision), LLM typically captures data from the record of a single, representative operation. This explains the higher rates both of no axillary surgery and of missing breast surgery data in the LLM group, suggesting the need for refinement of the ability or prompting of LLMs to integrate temporal surgical information. This study had several limitations. First, the LLM-based approach exhibited limitations in the integrated analysis of multiple clinical events. Although the model performed well in the extraction of explicit data points, it struggled to synthesize information across multiple clinical events, particularly in cases requiring the interpretation of sequential surgical procedures. This was evidenced by a higher rate of missing surgical data in patients who underwent multiple operations. Second, we used stages directly from pathological reports rather than from component factors, resulting in higher missing rates in the LLM group despite its better documentation of individual staging factors such as tumor size and nodal status. The LLM approach also had technical limitations in this study. The model’s performance was dependent on the quality and standardization of the input data. Additionally, although it extracted explicit clinical data points with high accuracy, it exhibited limitations in making implicit clinical judgments that experienced physicians routinely make during manual review. The generalizability of our results to other LLM models needs to be evaluated, as model performance may vary among different updates and versions. The accuracy of the manual review process might have been limited by the scope of this study, as it was not conducted for specific research purposes. This was evidenced by reviewer-dependent variations in data collection. Furthermore, differences in data extraction methods between groups may affect direct comparability. This study demonstrated that the LLM-based processing of anonymized clinical data is a viable alternative to traditional manual review by physician for surgical oncology research. The automated approach yielded superior efficiency in terms of processing time and resource utilization while it maintained accuracy in key clinical parameters. Despite its limitations with regard to integrated clinical assessments, LLM-based processing offers improved efficiency and scalability for large oncology datasets while enhancing patient privacy. Future research is needed on the following aspects: (1) development of improved prompting to handle complex clinical scenarios requiring integrated assessment, (2) comparison of LLM processing with manual review of identical raw data sources, (3) feasibility studies of integrated data curation across multiple clinical events, and (4) examination of the performance characteristics of different LLM models. Methods Study Design and Data Collection This study included the data of patients with breast cancer who underwent surgery at five academic hospitals from January 1, 2019, to December 31, 2019. We compared two data extraction methods: manual physician review (group 1) and LLM-based processing (group 2). In group 1, one dedicated breast-surgical oncologist from each hospital reviewed data spanning 2 years (2019-2020) over 7 months (May- November 2021) by using a standardized data collection form (Supplementary Fig. 1).The data encompassed 89 clinical variables across three domains: patient demographics (basic information, medical history, and family history), treatment information (surgical details, neo/adjuvant therapy, complications, and follow-up treatment), and pathological information (tumor characteristics, tumor stage, biomarker status, margin status). Follow-up observations regarding recurrence and mortality were updated through January 2024. Patients for group 2 were initially identified using the clinical data warehouse (CDW) of Catholic Medical Center (CMC), an integrated data platform of eight affiliated academic hospitals in Korea [20, 21]. The CDW supports research by providing anonymous clinical data to investigators following institutional review board approval [20]. The LLM structured 31 clinical factors from the raw data, including patient demographics (basic information, survival data, and diagnostic data), treatment information (surgery types and neo/adjuvant therapy), pathological information (tumor characteristics, tumor stage, biomarker status, and nodal status), and imaging features. Data extraction and curation were performed from October 20, 2024, to November 1, 2024. For the comparative analysis, 18 key clinical factors were selected from both groups (Fig. 1). The CDW query identified 17,317 patients diagnosed with invasive breast cancer or ductal carcinoma in situ (DCIS) from July 2018 to July 2021. From this cohort, we selected patients diagnosed during the study period (January-December 2019) who underwent breast cancer surgery. The CDW extraction included unstructured EHR reports containing clinical information, operation records, and pathology reports (Supplementary Fig. 2-4). Follow-up data through October 31, 2023, were used. This study was approved by the Institutional Review Board of CMC (approval number: OC24WIDI0138). Due to the retrospective nature of the study, the Institutional Review Board of CMC waived the requirement for obtaining informed consent. All methods were performed in accordance with the relevant guidelines and regulations, and the study was conducted in accordance with the Declaration of Helsinki. Data Curation in LLM-Processing Group Unstructured data extracted from the CDW were processed using Claude 3.5 Sonnet (Anthropic, San Francisco, CA, USA), an LLM, to extract and structure the required factors into predefined categories (Supplementary prompt). Prompts were developed through an iterative process of testing with raw sample data (October 20-21, 2024), focusing on the accurate identification and extraction of predetermined clinical factors while maintaining consistency across different documentation styles. The LLM prompt was developed through a three-phase iterative process: (1) initial prompt development using a test set of 10 diverse cases; (2) refinement through error analysis of 20 additional cases; and (3) validation using a separate set of 30 cases before full implementation. Each iteration focused on improving the accuracy of clinical factors such as surgical procedures and biomarkers (especially the interpretation of receptor tyrosine-protein kinase erbB-2 [HER2]status). The original prompt used for data extraction and analysis is available upon reasonable request from the corresponding author. Data curation was performed using a standard hospital workstation with typical computing resources (Intel Core i5, 32GB RAM) from October 20 to November 1, 2024. Objectives and Statistical Analysis The objective of this study was to assess the feasibility of replacing manual physician reviews with LLM-based processing of breast cancer-related clinical data. We compared demographic characteristics, clinical parameters, treatment patterns, disease characteristics, and survival outcomes between the two groups. Categorical variables were compared using chi-square or Fisher's exact tests, with agreement assessed using Cohen’s kappa coefficient (κ 0.80: very good). Continuous variables were analyzed using Student’s t-test and intraclass correlation coefficient (ICC). Effect sizes were calculated using Cohen’s d (continuous) and Cramér’s V (categorical). Overall survival was analyzed using the Kaplan–Meier method and compared with the log-rank test. Both approaches were validated using the Korean Breast Cancer Society (KBCS) 2019 national registry data by comparing age, tumor stage, surgical procedures, molecular subtypes, and survival trends [22]. Data Quality Assessment and Validation For validation, 50 cases from each group were selected using proportionate stratified random sampling. Stratification was based on the cancer stage (0-IV) and type of surgical intervention (breast-conserving surgery vs. mastectomy) to ensure representative sampling across key clinical categories. The random selection was performed using Python (version 3.8) with the NumPy (v.1.21.0) and pandas (v.1.3.0) libraries and a fixed random seed of 20241201 for reproducibility. Four breast-surgical oncologists (S.J. Oh, J.P. Yi, H. Kim, and S. Lim) independently evaluated 18 predefined clinical factors in each case (900 data points per group). Accuracy rates were calculated as the percentage of correctly extracted factors relative to the total number of factors. A dual-reference validation approach was implemented: group 1 was validated against the EHR, whereas group 2 was compared to the CDW raw data. The evaluation included both present and missing values, and the accuracy threshold was set to 90% [23]. Declarations Acknowledgments Not applicable Author’s contributions Conceived and designed the analysis: Y-J Kang, H Lee; Collected the data: Y-J Kang, CI Yoon, JM Baek, Y-S Kim, YW Jeon, J Rhu; Contributed data or analysis tools: Y-J Kang, CI Yoon, JM Baek, Y-S Kim, YW Jeon, J Rhu, JP Yi, H Kim, SH Lim, SJ Oh; Performed the analysis: Y-J Kang, H Lee; Wrote the paper: Y-J Kang, All authors reviewed and edited the manuscript, and have read and approved the final version. Data availability statement The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author upon reasonable request. Prompt availability The prompts used in this study are available in Supplementary Material. The detailed original prompts can be obtained from the corresponding author upon request. Competing interests The authors declare that there is no conflict of interest regarding the publication of this paper. Ethics approval and consent to participate This study was approved by the Institutional Review Board of Catholic Medical Center (approval number: OC24WIDI0138). Funding No funding. References Baclic O, Tunis M, Young K, Doan C, Swerdfeger H, Schonfeld J: Challenges and opportunities for public health made possible by advances in natural language processing . Can Commun Dis Rep 2020, 46 (6):161-168. Minaee S, Mikolov T, Nikzad N, Chenaghlu M, Socher R, Amatriain X, Gao J: Large language models: A survey . arXiv preprint arXiv:240206196 2024. Bedi S, Liu Y, Orr-Ewing L, Dash D, Koyejo S, Callahan A, Fries JA, Wornow M, Swaminathan A, Lehmann LS et al : Testing and Evaluation of Health Care Applications of Large Language Models: A Systematic Review . JAMA 2024. Stroganov O, Schedlbauer A, Lorenzen E, Kadhim A, Lobanova A, Lewis DA, Glausier JR: Unpacking unstructured data: A pilot study on extracting insights from neuropathological reports of Parkinson's Disease patients using large language models . Biol Methods Protoc 2024, 9 (1):bpae072. Guo C, Chen J: Big Data Analytics in Healthcare . In: Knowledge Technology and Systems: Toward Establishing Knowledge Systems Science. edn. Edited by Nakamori Y. Singapore: Springer Nature Singapore; 2023: 27-70. Wani MA, Jabin S: Big Data: Issues, Challenges, and Techniques in Business Intelligence . In : 2018; Singapore : Springer Singapore; 2018: 613-628. Garza MY, Williams T, Ounpraseuth S, Hu Z, Lee J, Snowden J, Walden AC, Simon AE, Devlin LA, Young LW et al : Error rates of data processing methods in clinical research: A systematic review and meta-analysis of manuscripts identified through PubMed . Int J Med Inform 2024, 195 :105749. Wang W, Krishnan E: Big data and clinicians: a review on the state of the science . JMIR Med Inform 2014, 2 (1):e1. Hoffman S: Big data analytics: what can go wrong . Ind Health L Rev 2018, 15 :227. Fan J, Han F, Liu H: Challenges of big data analysis . National science review 2014, 1 (2):293-314. Kim M, Kim Y, Kang HJ, Seo H, Choi H, Han J, Kee G, Park S, Ko S, Jung H et al : Fine-Tuning LLMs with Medical Data: Can Safety Be Ensured? NEJM AI 2025, 2 (1):AIcs2400390. Rengers TA, Thiels CA, Salehinejad H: Academic Surgery in the Era of Large Language Models: A Review . JAMA Surg 2024, 159 (4):445-450. Jeyaraman M, Balaji S, Jeyaraman N, Yadav S: Unraveling the Ethical Enigma: Artificial Intelligence in Healthcare . Cureus 2023, 15 (8):e43262. Wulcan JM, Jacques KL, Lee MA, Kovacs SL, Dausend N, Prince LE, Wulcan J, Marsilio S, Keller SM: Classification performance and reproducibility of GPT-4 omni for information extraction from veterinary electronic health records . arXiv preprint arXiv:240913727 2024. Lopez-Ubeda P, Martin-Noguerol T, Juluru K, Luna A: Natural Language Processing in Radiology: Update on Clinical Applications . J Am Coll Radiol 2022, 19 (11):1271-1285. Park HJ, Huh JY, Chae G, Choi MG: Extraction of clinical data on major pulmonary diseases from unstructured radiologic reports using a large language model . PLoS One 2024, 19 (11):e0314136. Soni N, Ora M, Agarwal A, Yang T, Bathla G: A Review of The Opportunities and Challenges with Large Language Models in Radiology: The Road Ahead . AJNR Am J Neuroradiol 2024. Cheng J: Applications of Large Language Models in Pathology . Bioengineering (Basel) 2024, 11 (4). Davey MG, O'Flaherty C, Cleere EF, Nohilly A, Phelan J, Ronane E, Lowery AJ, Kerin MJ: Sentinel lymph node biopsy in patients with ductal carcinoma in situ: systematic review and meta-analysis . BJS Open 2022, 6 (2). Choi IY, Park S, Park B, Chung BH, Kim CS, Lee HM, Byun SS, Lee JY: Development of prostate cancer research database with the clinical data warehouse technology for direct linkage with electronic medical record system . Prostate Int 2013, 1 (2):59-64. Park SJ, Lee SJ, Kim H, Kim JK, Chun JW, Lee SJ, Lee HK, Kim DJ, Choi IY: Machine learning prediction of dropping out of outpatients with alcohol use disorders . PLoS One 2021, 16 (8):e0255626. Choi JE, Kim Z, Park CS, Park EH, Lee SB, Lee SK, Choi YJ, Han J, Jung KW, Kim HJ et al : Breast Cancer Statistics in Korea, 2019 . J Breast Cancer 2023, 26 (3):207-220. Woodfield R, Grant I, Group UKBSO, Follow-Up UKB, Outcomes Working G, Sudlow CL: Accuracy of Electronic Health Record Data for Identifying Stroke Cases in Large-Scale Epidemiological Studies: A Systematic Review from the UK Biobank Stroke Outcomes Group . PLoS One 2015, 10 (10):e0140533. Table 1 Table 1. Baseline Characteristics of Study Groups Characteristics Manual Review LLM Processing P -value n=1,366 n=1,734 Demographics Age, mean (SD), y 55.0 (11.5) 53.5 (11.4) <0.001 Surgical Procedures, No. (%) Breast Operation Breast-conserving surgery 868 (63.5) 949 (63.9) <0.001 Total mastectomy 262 (19.2) 393 (26.5) N(S)SM 214 (15.7) 142 (9.6) Other procedures 22 (1.6) 1 (0.1) Combined mastectomy a 476 (34.8) 535 (36.0) Axillary Surgery No surgery 153 (11.2) 321 (19.4) <0.001 SLNB 934 (68.3) 990 (59.7) ALND 280 (20.5) 346 (20.9) Pathological Results Tumor Size, mean (SD), mm 20.5 (16.4) 21.5 (16.9) 0.156 Lymph Node Status Harvested nodes, mean (SD) 7.79 (7.0) 7.11 (7.2) 0.016 Metastatic nodes, mean (SD) 0.95 (3.0) 0.98 (3.2) 0.802 Stage Distribution, No. (%) 0 237 (17.9) 237 (15.6) <0.001 IA 472 (35.7) 659 (43.3) IB 8 (0.6) 20 (1.3) IIA 294 (22.2) 316 (20.7) IIB 147 (11.1) 165 (10.8) IIIA 99 (7.5) 86 (5.6) IIIB 11 (0.8) 1 (0.1) IIIC 49 (3.7) 39 (2.6) IV 6 (0.5) 0 (0) Biomarker Status, No. (%) ER positive 1012 (78.3) 1198 (76.2) 0.172 PR positive 886 (68.7) 1059 (67.5) 0.525 HER2 positive 249 (20.5) 298 (20.2) 0.003 Ki-67, mean (SD) 25.4 (23.0) 26.6 (22.6) 0.204 Histologic Grade, No. (%) Grade 1 169 (22.8) 283 (21.5) 0.764 Grade 2 331 (44.7) 610 (46.4) Grade 3 241 (32.5) 423 (32.1) Nuclear Grade, No. (%) Grade 1 137 (16.7) 199 (12.9) <0.001 Grade 2 348 (42.5) 795 (51.5) Grade 3 334 (40.8) 549 (35.6) Survival Outcomes, No. (%) Death 11 (0.8) 42 (2.4) 0.001 Abbreviations: ALND, axillary lymph node dissection; ER, estrogen receptor; HER2, human epidermal growth factor receptor 2; N(S)SM, nipple or skin-sparing mastectomy; PR, progesterone receptor; SD, standard deviation; SLNB, sentinel lymph node biopsy. Statistical significance set at P < 0.05. Analyses were performed using chi-square test for categorical variables and t test for continuous variables. a Combined mastectomy includes total mastectomy and N(S)SM. Additional Declarations No competing interests reported. Supplementary Files SupplementaryFigure1.tif SupplementaryFigure2.tif SupplementaryFigure3.tif SupplementaryFigure4.tif SupplementPrompt.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7089616","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":505560170,"identity":"971b8a47-fb7a-499e-910a-e59d61921a3d","order_by":0,"name":"Young-Joon Kang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA30lEQVRIiWNgGAWjYFCCY1D6eAOQMLAgRcuZAyAtEsRoYYPSNxJAJBFaDA4eS/xc8Ishmu/m86sbfhRIMPC3dyfg13Lg2GHpmX0MuTNv55Td7AE6TOLM2Q0EtBxvkObtYcjdcDsn7QYPUIuBRC5BLc2/wVpunkm7+Yc4LceOSfP8AGq5wX7sNlG2SB44lmbN2yCRO/NMDtttGQMJHoJ+4btxzPg2zx+b3L7jx5/dfPPHRo6/vRe/FoUbBxgYGNtA0cFjABLgwascBOT7G4DkHxCT/QFB1aNgFIyCUTAyAQARf1Kl6x9o1AAAAABJRU5ErkJggg==","orcid":"","institution":"The Catholic University of Korea","correspondingAuthor":true,"prefix":"","firstName":"Young-Joon","middleName":"","lastName":"Kang","suffix":""},{"id":505560171,"identity":"6c6d8a2f-7b7e-4b22-812a-1254234f6be9","order_by":1,"name":"Hocheol Lee","email":"","orcid":"","institution":"Yonsei University (Mirae)","correspondingAuthor":false,"prefix":"","firstName":"Hocheol","middleName":"","lastName":"Lee","suffix":""},{"id":505560172,"identity":"f0909e35-c4f7-4b6e-8078-83a177c92148","order_by":2,"name":"Jae Pak Yi","email":"","orcid":"","institution":"The Catholic University of Korea","correspondingAuthor":false,"prefix":"","firstName":"Jae","middleName":"Pak","lastName":"Yi","suffix":""},{"id":505560173,"identity":"a1a23b66-dd64-4a2d-bd54-ce46de7b2994","order_by":3,"name":"Hyobin Kim","email":"","orcid":"","institution":"The Catholic University of Korea","correspondingAuthor":false,"prefix":"","firstName":"Hyobin","middleName":"","lastName":"Kim","suffix":""},{"id":505560174,"identity":"da1da9a6-43bd-4f88-ab70-cf8aa450ee64","order_by":4,"name":"Chang Ik Yoon","email":"","orcid":"","institution":"The Catholic University of Korea","correspondingAuthor":false,"prefix":"","firstName":"Chang","middleName":"Ik","lastName":"Yoon","suffix":""},{"id":505560175,"identity":"ae13326d-5c57-49f3-9dd6-2c1faefde840","order_by":5,"name":"Jong Min Baek","email":"","orcid":"","institution":"The Catholic University of Korea","correspondingAuthor":false,"prefix":"","firstName":"Jong","middleName":"Min","lastName":"Baek","suffix":""},{"id":505560178,"identity":"b4923d8c-5544-4d67-a6f9-75837e3b94d1","order_by":6,"name":"Yong-seok Kim","email":"","orcid":"","institution":"Uijeongbu St. Mary’s Hospital, The Catholic University of Korea","correspondingAuthor":false,"prefix":"","firstName":"Yong-seok","middleName":"","lastName":"Kim","suffix":""},{"id":505560179,"identity":"cc81886c-608f-4dbb-a291-7d6b6fcbecb2","order_by":7,"name":"Ye Won Jeon","email":"","orcid":"","institution":"St. Vincent's Hospital, The Catholic University of Korea","correspondingAuthor":false,"prefix":"","firstName":"Ye","middleName":"Won","lastName":"Jeon","suffix":""},{"id":505560181,"identity":"4d47efdd-7241-4dec-abef-ef5a13d207e4","order_by":8,"name":"Jiyoung Rhu","email":"","orcid":"","institution":"The Catholic University of Korea","correspondingAuthor":false,"prefix":"","firstName":"Jiyoung","middleName":"","lastName":"Rhu","suffix":""},{"id":505560182,"identity":"b9d8572b-f848-4bdb-a098-4358c374a5b9","order_by":9,"name":"Su Hyun Lim","email":"","orcid":"","institution":"The Catholic University of Korea","correspondingAuthor":false,"prefix":"","firstName":"Su","middleName":"Hyun","lastName":"Lim","suffix":""},{"id":505560183,"identity":"4202f695-0d27-448d-b0b4-0a3ad18aab46","order_by":10,"name":"Hoon Choi","email":"","orcid":"","institution":"The Catholic University of Korea","correspondingAuthor":false,"prefix":"","firstName":"Hoon","middleName":"","lastName":"Choi","suffix":""},{"id":505560186,"identity":"4ca4057c-c901-4318-ae89-7e6c201b8193","order_by":11,"name":"Se Jeong Oh","email":"","orcid":"","institution":"The Catholic University of Korea","correspondingAuthor":false,"prefix":"","firstName":"Se","middleName":"Jeong","lastName":"Oh","suffix":""}],"badges":[],"createdAt":"2025-07-10 06:38:12","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7089616/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7089616/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":90310999,"identity":"8a48db16-457e-434e-8f55-78fce634fdb1","added_by":"auto","created_at":"2025-09-01 09:47:47","extension":"tif","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":298296,"visible":true,"origin":"","legend":"\u003cp\u003eStudy design chart showing the comparison between manual review group and LLM-processing group\u003c/p\u003e","description":"","filename":"Figure1.tif","url":"https://assets-eu.researchsquare.com/files/rs-7089616/v1/5c3a0e3570c852be3468f510.tif"},{"id":90312379,"identity":"22496a8b-4be0-4c04-92b2-340b5831273b","added_by":"auto","created_at":"2025-09-01 09:55:47","extension":"tif","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":184095,"visible":true,"origin":"","legend":"\u003cp\u003eMissing data rates (%) comparison between manual review (group 1) and LLM-processing (group 2) groups\u003c/p\u003e","description":"","filename":"Figure2.tif","url":"https://assets-eu.researchsquare.com/files/rs-7089616/v1/d9fc43138c78bbbafd00a916.tif"},{"id":90311002,"identity":"dd017e6e-4879-4dd7-a055-474e2a4f451b","added_by":"auto","created_at":"2025-09-01 09:47:47","extension":"tif","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":290988,"visible":true,"origin":"","legend":"\u003cp\u003eComparison of clinical characteristics between manual review (group 1), LLM-processing (group 2), and Korea Breast Cancer Registry 2019 national registry data\u003c/p\u003e\n\u003cp\u003eA) Distribution of breast surgery types\u003c/p\u003e\n\u003cp\u003eB) Distribution of axillary surgery types\u003c/p\u003e\n\u003cp\u003eC) Distribution of cancer stages\u003c/p\u003e\n\u003cp\u003eD) Distribution of biomarker status (ER, PR, HER2)\u003c/p\u003e","description":"","filename":"Figure3.tif","url":"https://assets-eu.researchsquare.com/files/rs-7089616/v1/8f3e8f5da251c06fbb96d1a4.tif"},{"id":91620914,"identity":"5ac032da-85f4-48dd-8c37-a4c97e400aac","added_by":"auto","created_at":"2025-09-18 11:24:45","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2310039,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7089616/v1/5f8cdae5-d9a3-4f02-9f1f-fb1e5b7e88e1.pdf"},{"id":90311024,"identity":"2eab1432-7dfe-4c95-83ae-bcb857975ef1","added_by":"auto","created_at":"2025-09-01 09:47:48","extension":"tif","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":32818138,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryFigure1.tif","url":"https://assets-eu.researchsquare.com/files/rs-7089616/v1/45de7537b614b8e9abc1a503.tif"},{"id":90311023,"identity":"f0b4fac1-af6f-4cb3-9073-a01295d718a6","added_by":"auto","created_at":"2025-09-01 09:47:48","extension":"tif","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":20485586,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryFigure2.tif","url":"https://assets-eu.researchsquare.com/files/rs-7089616/v1/33e4e8b000de6e30c119b445.tif"},{"id":90311025,"identity":"29c1755d-c25b-4ba6-9347-0ef60e542bac","added_by":"auto","created_at":"2025-09-01 09:47:48","extension":"tif","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":31621492,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryFigure3.tif","url":"https://assets-eu.researchsquare.com/files/rs-7089616/v1/90a91642a9296669efa81834.tif"},{"id":90311026,"identity":"d4e0ccd7-bedd-4b73-b009-ff0b84984b21","added_by":"auto","created_at":"2025-09-01 09:47:48","extension":"tif","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":28722028,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryFigure4.tif","url":"https://assets-eu.researchsquare.com/files/rs-7089616/v1/84361599740acd997fdea040.tif"},{"id":90311006,"identity":"f5de7770-22b4-493a-9f16-4318bbf30168","added_by":"auto","created_at":"2025-09-01 09:47:47","extension":"docx","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":21306,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementPrompt.docx","url":"https://assets-eu.researchsquare.com/files/rs-7089616/v1/25ba76824f95fa8830e2a992.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Comparison of Large Language Model and Manual Review for Clinical Data Curation in Breast Cancer","fulltext":[{"header":"Introduction","content":"\u003cp\u003eRecent advances in artificial intelligence (AI), particularly in large language models (LLMs), have demonstrated remarkable capabilities of automated data extraction and organization from complex clinical documents [1\u0026ndash;3]. These AI-driven approaches can be used to process large volumes of clinical data with a consistent methodology, potentially reducing human bias and improving the collection efficiency of research data. Although LLMs show promise in healthcare applications, few studies on their practical efficacy compared with that of traditional, manual processing by physicians have been published, particularly in complex areas such as the extraction of surgical oncology data [4].\u003c/p\u003e\u003cp\u003eIn the field of breast cancer surgery, retrospective data analysis presents unique challenges owing to the complexity of unstructured clinical data. Electronic health records (EHRs) contain diverse information across clinical charts, operation records, and pathology reports, often in free-text format. The complexity is compounded by breast cancer-specific characteristics, including bilateral nature of the organs, concurrent malignant and benign lesions, and multiple radiological features. This complicates automated data curation and has traditionally necessitated manual review by physicians for accurate data interpretation and collection.\u003c/p\u003e\u003cp\u003eHowever, the manual approach has important limitations. As the volume of clinical data increases, consistency in physician reviews becomes increasingly difficult to maintain, potentially leading to discrepancies in data interpretation [5, 6]. The time-intensive nature of manual review and the risk of errors in the processing of large volumes of clinical data present considerable challenges in retrospective research [7\u0026ndash;10]. In addition, direct EHR access for manual data extraction raises privacy concerns when sensitive patient information is handled [11\u0026ndash;13]. Although the LLM-based automation of information extraction from anonymized EHR data may address these challenges, its effectiveness compared with that of traditional physician review remains to be evaluated.\u003c/p\u003e\u003cp\u003eAlthough LLMs have been validated for the extraction of specific medical data, their potential for the curation of comprehensive data of patients with cancer remains largely unexplored [14]. In this study, we compared traditional physician review with LLM-based processing of anonymized clinical data in the field of breast cancer, focusing on the development of a practical approach for surgical oncologists. We hypothesized that LLM-based analysis would yield comparable results to manual review in the handling of large volumes of clinical data while reducing the processing time and resource utilization.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cstrong\u003eOutcomes\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe manual review (group 1) and LLM processing (group 2) groups comprised 1,366 and 1,734 cases, respectively. Although both groups completely captured age data, they exhibited different patterns of missing data for other parameters. Group 2 had higher missing rates in terms of cancer stage (12.2% vs. 3.1%) and HER2 status (15.1% vs. 11.0%), whereas group 1 had more missing data for lesion size (20.5% vs. 5.9%) and lymph node assessment (21.5% vs. 8.8%). Both groups maintained high documentation rates (\u0026gt;90%) for hormone receptor status (Fig. 2).\u003c/p\u003e\n\u003cp\u003eThe validation analysis encompassed 1,800 data points (900 per group) across clinical factors. Group 1 demonstrated perfect accuracy with no discrepancies. Group 2 exhibited 83 discordant factors out of 900 data points, achieving an accuracy rate of 90.8% (817/900), which exceeds the predetermined threshold of 90% for acceptable performance.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDemographics and Clinical Characteristics\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe baseline characteristics of both groups are summarized in Table 1. The mean age differed slightly between groups 1 and 2 (55.0 vs. 53.5 years, \u003cem\u003eP\u003c/em\u003e \u0026lt; 0.001, Cohen\u0026rsquo;s d = 0.13). For breast surgery, total mastectomy was performed in 19.2% of cases in group 1 and 26.5% in group 2, while nipple (skin)-sparing mastectomy rates were 15.7% and 9.6%, respectively (\u0026chi;\u003csup\u003e2\u003c/sup\u003e = 164.3, degrees of freedom [df] = 3, \u003cem\u003eP\u003c/em\u003e \u0026lt; 0.001, Cram\u0026eacute;r\u0026rsquo;s V = 0.29). When these procedures were combined, groups 1 and 2 exhibited similar proportions of breast-conserving surgery (63.5% vs. 63.9%) and mastectomy (34.8% vs. 36.0%), with a small effect size (Cram\u0026eacute;r\u0026rsquo;s V = 0.10).\u003c/p\u003e\n\u003cp\u003eIn terms of axillary surgery, group 1 had a higher rate of sentinel lymph node biopsy (SLNB) than group 2 (68.4% vs. 59.7%), whereas the rates of axillary lymph node dissection (ALND) were similar (\u0026chi;\u003csup\u003e2\u003c/sup\u003e = 47.2, df = 2, \u003cem\u003eP\u003c/em\u003e \u0026lt; 0.001, Cram\u0026eacute;r\u0026rsquo;s V = 0.13). The\u0026nbsp;mean \u0026plusmn; standard deviation \u0026nbsp;number of harvested lymph nodes was similar between the groups (7.79 \u0026plusmn; 7.03 vs. 7.11 \u0026plusmn; 7.20, \u003cem\u003eP\u003c/em\u003e = 0.016).\u003c/p\u003e\n\u003cp\u003eStage distribution differed between groups (\u0026chi;\u003csup\u003e2\u003c/sup\u003e = 68.9, df = 8, \u003cem\u003eP\u003c/em\u003e \u0026lt; 0.001), but only slightly (Cram\u0026eacute;r\u0026rsquo;s V = 0.16), with group 1 identifying more cancers as advanced. The hormone receptor status was similar between the groups (estrogen receptor: 78.3% vs. 76.2%, \u003cem\u003eP\u003c/em\u003e = 0.172; progesterone receptor: 68.7% vs. 67.5%, \u003cem\u003eP\u003c/em\u003e = 0.525). The HER2 status differed negligibly and Ki67 expression was similar between the groups (HER2: P=0.003, Cram\u0026eacute;r\u0026rsquo;s V = 0.003; Ki67: \u003cem\u003eP\u003c/em\u003e = 0.391, Cram\u0026eacute;r\u0026rsquo;s V \u0026asymp; 0.00). Histological grade distributions were similar between the groups (\u003cem\u003eP\u003c/em\u003e = 0.764).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eInterrater Agreement Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eICC analysis of continuous variables demonstrated a consistently low agreement: age (ICC = 0.013, 95% confidence interval [CI]: -0.035-0.060), tumor size (ICC = 0.029, 95% CI: -0.021-0.078), number of metastatic lymph nodes (ICC = 0.031, 95% CI: -0.019-0.081), number of harvested lymph nodes (ICC = 0.025, 95% CI: -0.025-0.075), and Ki67 expression (ICC = 0.027, 95% CI: -0.023-0.077). All ICC values were negligible and all CIs included zero.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSurvival Outcomes\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eSurvival analysis revealed significant differences between the groups (hazard ratio [HR] = 2.917, 95% CI: 1.496-5.688, \u003cem\u003eP\u003c/em\u003e = 0.002). Group 2 captured more events (11 vs. 41). The proportional hazards assumption was met (\u0026chi;\u003csup\u003e2\u003c/sup\u003e = 2.37, \u003cem\u003eP\u003c/em\u003e =0.12), and the log-rank test confirmed a difference in survival distributions (\u0026chi;\u003csup\u003e2\u003c/sup\u003e = 10.9, \u003cem\u003eP\u003c/em\u003e = 0.001).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eComparison with National Registry Data\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eComparison with the KBCS 2019 registry data (n = 9,447) revealed small differences in breast surgery patterns for both groups (Cram\u0026eacute;r\u0026rsquo;s V = 0.03-0.04, \u003cem\u003eP\u003c/em\u003e \u0026lt; 0.001; Fig. 3A). In axillary surgery, both groups had lower SLNB rates (group 1: 68.3% and group 2: 59.7% vs. KBCS: 73.2%) and similar ALND rates (20.5% vs. 20.9% vs. 18.6%; Fig. 3B). Group 2 had a higher rate of no axillary surgery than group 1 (19.4% vs. 11.2%).\u003c/p\u003e\n\u003cp\u003eStage distribution analysis revealed significant but small differences from the national data (group 1: Cram\u0026eacute;r\u0026rsquo;s V = 0.07, \u003cem\u003eP\u003c/em\u003e \u0026lt; 0.001; Group 2: Cram\u0026eacute;r\u0026rsquo;s V = 0.03, \u003cem\u003eP\u003c/em\u003e = 0.003; Fig. 3C). Regarding biomarker subtypes, both groups had slightly higher proportions of HR-positive/HER2-negative (group 1: 67.0% and group 2: 67.5% vs. KBCS: 63.1%) and triple-negative cases (12.7% and 12.5% vs. 12.0%) with minimal effect sizes (Cram\u0026eacute;r\u0026rsquo;s V = 0.03-0.04; Fig. 3D).\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn this study, we compared traditional manual reviews by physicians with LLM processing of anonymized data for breast cancer clinical research. Although LLMs have yielded favorable results in the extraction of data from radiology and pathology reports, surgical oncology presents unique challenges owing to its requirement for the complex integration of multiple clinical parameters and temporal relationships [15-18]. Our study addresses this gap as the first comprehensive evaluation of the utility of LLMs in surgical oncology data curation. Whereas statistical differences were observed between the groups, most were clinically insignificant, suggesting that LLM processing may be a viable alternative to manual review despite the challenges.\u003c/p\u003e\n\u003cp\u003eDigital extraction exhibited advantages in case identification and collection of survival data, achieving an accuracy \u0026gt; 90%. LLM processing performed well with continuous variables such as tumor size and lymph node assessment but had limitations in integrated analysis such as that of surgery methods and staging. This suggests that, although LLM-based processing effectively captures most clinical categories, it needs to be refined in terms of complex clinical information that required integrated analysis.\u003c/p\u003e\n\u003cp\u003eThe patterns of missing data differed between the groups: missing data upon LLM processing primarily stemmed from the initial CDW extraction, whereas manual review was vulnerable to individual variations in data collection, as evidenced by clustering of missing values in the documentation of nodal assessment (21.5% missing). Such reviewer-dependent variations highlight a fundamental limitation of manual review, as maintaining consistency across multiple reviewers can be challenging despite using standardized forms. Although clinical specialists are ideally suited for disease-specific data curation, technical barriers and the need for continuous collaboration with programming experts often limit their direct involvement in large-scale data analysis. In this context, LLM processing offers an accessible solution for clinicians, demonstrating vast efficiency gains; LLM processing over 12 days by two physicians yielded comparable accuracy to manual review over 7 months by five physicians for the same study period. The higher number of cases in the LLM group reflects the advantages of automated extraction rather than specific LLM capabilities.\u003c/p\u003e\n\u003cp\u003eThe axillary surgery data demonstrated the strengths and limitations of LLM-based processing in integrated analysis. The LLM group indicated a close alignment between the no axillary surgery rate (19.4%) and the in situ cancer rate (20.3%), whereas manual review suggested a difference (11.2% vs. 18.2%). The difference revealed by manual review reflects real-world clinical practice, in which SLNB is often performed in patients with DCIS, who are at a high risk of invasion [19]. The discrepancy between the data extraction methods highlights a key limitation of LLM processing: whereas manual reviewers can identify and integrate multiple surgical steps (such as lymph node assessment after initial diagnostic excision), LLM typically captures data from the record of a single, representative operation. This explains the higher rates both of no axillary surgery and of missing breast surgery data in the LLM group, suggesting the need for refinement of the ability or prompting of LLMs to integrate temporal surgical information.\u003c/p\u003e\n\u003cp\u003eThis study had several limitations. First, the LLM-based approach exhibited limitations in the integrated analysis of multiple clinical events. Although the model performed well in the extraction of explicit data points, it struggled to synthesize information across multiple clinical events, particularly in cases requiring the interpretation of sequential surgical procedures. This was evidenced by a higher rate of missing surgical data in patients who underwent multiple operations. Second, we used stages directly from pathological reports rather than from component factors, resulting in higher missing rates in the LLM group despite its better documentation of individual staging factors such as tumor size and nodal status.\u003c/p\u003e\n\u003cp\u003eThe LLM approach also had technical limitations in this study. The model\u0026rsquo;s performance was dependent on the quality and standardization of the input data. Additionally, although it extracted explicit clinical data points with high accuracy, it exhibited limitations in making implicit clinical judgments that experienced physicians routinely make during manual review. The generalizability of our results to other LLM models needs to be evaluated, as model performance may vary among different updates and versions.\u003c/p\u003e\n\u003cp\u003eThe accuracy of the manual review process might have been limited by the scope of this study, as it was not conducted for specific research purposes. This was evidenced by reviewer-dependent variations in data collection. Furthermore, differences in data extraction methods between groups may affect direct comparability.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThis study demonstrated that the LLM-based processing of anonymized clinical data is a viable alternative to traditional manual review by physician for surgical oncology research. The automated approach yielded superior efficiency in terms of processing time and resource utilization while it maintained accuracy in key clinical parameters. Despite its limitations with regard to integrated clinical assessments, LLM-based processing offers improved efficiency and scalability for large oncology datasets while enhancing patient privacy. Future research is needed on the following aspects: (1) development of improved prompting to handle complex clinical scenarios requiring integrated assessment, (2) comparison of LLM processing with manual review of identical raw data sources, (3) feasibility studies of integrated data curation across multiple clinical events, and (4) examination of the performance characteristics of different LLM models.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003e\u003cstrong\u003eStudy Design and Data Collection\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study included the data of patients with breast cancer who underwent surgery at five academic hospitals from January 1, 2019, to December 31, 2019. We compared two data extraction methods: manual physician review (group 1) and LLM-based processing (group 2).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn group 1, one dedicated breast-surgical oncologist from each hospital reviewed data spanning 2 years (2019-2020) over 7 months (May- November 2021) by using a standardized data collection form (Supplementary Fig. 1).The data encompassed 89 clinical variables across three domains: patient demographics (basic information, medical history, and family history), treatment information (surgical details, neo/adjuvant therapy, complications, and follow-up treatment), and pathological information (tumor characteristics, tumor stage, biomarker status, margin status). Follow-up observations regarding recurrence and mortality were updated through January 2024.\u003c/p\u003e\n\u003cp\u003ePatients for group 2 were initially identified using the clinical data warehouse (CDW) of Catholic Medical Center (CMC), an integrated data platform of eight affiliated academic hospitals in Korea [20, 21]. The CDW supports research by providing anonymous clinical data to investigators following institutional review board approval [20]. The LLM structured 31 clinical factors from the raw data, including patient demographics (basic information, survival data, and diagnostic data), treatment information (surgery types and neo/adjuvant therapy), pathological information (tumor characteristics, tumor stage, biomarker status, and nodal status), and imaging features. Data extraction and curation were performed from October 20, 2024, to November 1, 2024. For the comparative analysis, 18 key clinical factors were selected from both groups (Fig. 1).\u003c/p\u003e\n\u003cp\u003eThe CDW query identified 17,317 patients diagnosed with invasive breast cancer or ductal carcinoma in situ (DCIS) from July 2018 to July 2021. From this cohort, we selected patients diagnosed during the study period (January-December 2019) who underwent breast cancer surgery. The CDW extraction included unstructured EHR reports containing clinical information, operation records, and pathology reports (Supplementary Fig. 2-4). Follow-up data through October 31, 2023, were used. This study was approved by the Institutional Review Board of CMC (approval number: OC24WIDI0138). Due to the retrospective nature of the study, the Institutional Review Board of CMC waived the requirement for obtaining informed consent. All methods were performed in accordance with the relevant guidelines and regulations, and the study was conducted in accordance with the Declaration of Helsinki.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Curation in LLM-Processing Group\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eUnstructured data extracted from the CDW were processed using Claude 3.5 Sonnet (Anthropic, San Francisco, CA, USA), an LLM, to extract and structure the required factors into predefined categories (Supplementary prompt). Prompts were developed through an iterative process of testing with raw sample data (October 20-21, 2024), focusing on the accurate identification and extraction of predetermined clinical factors while maintaining consistency across different documentation styles. The LLM prompt was developed through a three-phase iterative process: (1) initial prompt development using a test set of 10 diverse cases; (2) refinement through error analysis of 20 additional cases; and (3) validation using a separate set of 30 cases before full implementation. Each iteration focused on improving the accuracy of clinical factors such as surgical procedures and biomarkers (especially the interpretation of receptor tyrosine-protein kinase erbB-2 [HER2]status). The original prompt used for data extraction and analysis is available upon reasonable request from the corresponding author. Data curation was performed using a standard hospital workstation with typical computing resources (Intel Core i5, 32GB RAM) from October 20 to November 1, 2024.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eObjectives and Statistical Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe objective of this study was to assess the feasibility of replacing manual physician reviews with LLM-based processing of breast cancer-related clinical data. We compared demographic characteristics, clinical parameters, treatment patterns, disease characteristics, and survival outcomes between the two groups.\u003c/p\u003e\n\u003cp\u003eCategorical variables were compared using chi-square or Fisher\u0026apos;s exact tests, with agreement assessed using Cohen\u0026rsquo;s kappa coefficient (\u0026kappa; \u0026lt; 0.20: poor, 0.21-0.40: fair, 0.41-0.60: moderate, 0.61-0.80: good, \u0026gt; 0.80: very good). Continuous variables were analyzed using Student\u0026rsquo;s t-test and intraclass correlation coefficient (ICC). Effect sizes were calculated using Cohen\u0026rsquo;s d (continuous) and Cram\u0026eacute;r\u0026rsquo;s V (categorical).\u003c/p\u003e\n\u003cp\u003eOverall survival was analyzed using the Kaplan\u0026ndash;Meier method and compared with the log-rank test. Both approaches were validated using the Korean Breast Cancer Society (KBCS) 2019 national registry data by comparing age, tumor stage, surgical procedures, molecular subtypes, and survival trends [22].\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Quality Assessment and Validation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFor validation, 50 cases from each group were selected using proportionate stratified random sampling. Stratification was based on the cancer stage (0-IV) and type of surgical intervention (breast-conserving surgery vs. mastectomy) to ensure representative sampling across key clinical categories. The random selection was performed using Python (version 3.8) with the NumPy (v.1.21.0) and pandas (v.1.3.0) libraries and a fixed random seed of 20241201 for reproducibility. Four breast-surgical oncologists (S.J. Oh, J.P. Yi, H. Kim, and S. Lim) independently evaluated 18 predefined clinical factors in each case (900 data points per group). Accuracy rates were calculated as the percentage of correctly extracted factors relative to the total number of factors. A dual-reference validation approach was implemented: group 1 was validated against the EHR, whereas group 2 was compared to the CDW raw data. The evaluation included both present and missing values, and the accuracy threshold was set to 90% [23].\u0026nbsp;\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor’s contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eConceived and designed the analysis: Y-J Kang, H Lee; Collected the data: Y-J Kang, CI Yoon, JM Baek, Y-S Kim, YW Jeon, J Rhu; Contributed data or analysis tools: Y-J Kang, CI Yoon, JM Baek, Y-S Kim, YW Jeon, J Rhu, JP Yi, H Kim, SH Lim, SJ Oh; Performed the analysis: Y-J Kang, H Lee; Wrote the paper: Y-J Kang, All authors reviewed and edited the manuscript, and have read and approved the final version.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData availability statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author upon reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt availability\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe prompts used in this study are available in Supplementary Material. The detailed original prompts can be obtained from the corresponding author upon request.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that there is no conflict of interest regarding the publication of this paper.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study was approved by the Institutional Review Board of Catholic Medical Center (approval number: OC24WIDI0138).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003cbr\u003e\u003c/strong\u003eNo funding.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eBaclic O, Tunis M, Young K, Doan C, Swerdfeger H, Schonfeld J: \u003cstrong\u003eChallenges and opportunities for public health made possible by advances in natural language processing\u003c/strong\u003e. \u003cem\u003eCan Commun Dis Rep \u003c/em\u003e2020, \u003cstrong\u003e46\u003c/strong\u003e(6):161-168.\u003c/li\u003e\n\u003cli\u003eMinaee S, Mikolov T, Nikzad N, Chenaghlu M, Socher R, Amatriain X, Gao J: \u003cstrong\u003eLarge language models: A survey\u003c/strong\u003e. \u003cem\u003earXiv preprint arXiv:240206196 \u003c/em\u003e2024.\u003c/li\u003e\n\u003cli\u003eBedi S, Liu Y, Orr-Ewing L, Dash D, Koyejo S, Callahan A, Fries JA, Wornow M, Swaminathan A, Lehmann LS\u003cem\u003e et al\u003c/em\u003e: \u003cstrong\u003eTesting and Evaluation of Health Care Applications of Large Language Models: A Systematic Review\u003c/strong\u003e. \u003cem\u003eJAMA \u003c/em\u003e2024.\u003c/li\u003e\n\u003cli\u003eStroganov O, Schedlbauer A, Lorenzen E, Kadhim A, Lobanova A, Lewis DA, Glausier JR: \u003cstrong\u003eUnpacking unstructured data: A pilot study on extracting insights from neuropathological reports of Parkinson\u0026apos;s Disease patients using large language models\u003c/strong\u003e. \u003cem\u003eBiol Methods Protoc \u003c/em\u003e2024, \u003cstrong\u003e9\u003c/strong\u003e(1):bpae072.\u003c/li\u003e\n\u003cli\u003eGuo C, Chen J: \u003cstrong\u003eBig Data Analytics in Healthcare\u003c/strong\u003e. In: \u003cem\u003eKnowledge Technology and Systems: Toward Establishing Knowledge Systems Science.\u003c/em\u003e edn. Edited by Nakamori Y. Singapore: Springer Nature Singapore; 2023: 27-70.\u003c/li\u003e\n\u003cli\u003eWani MA, Jabin S: \u003cstrong\u003eBig Data: Issues, Challenges, and Techniques in Business Intelligence\u003c/strong\u003e. In\u003cem\u003e: 2018; Singapore\u003c/em\u003e: Springer Singapore; 2018: 613-628.\u003c/li\u003e\n\u003cli\u003eGarza MY, Williams T, Ounpraseuth S, Hu Z, Lee J, Snowden J, Walden AC, Simon AE, Devlin LA, Young LW\u003cem\u003e et al\u003c/em\u003e: \u003cstrong\u003eError rates of data processing methods in clinical research: A systematic review and meta-analysis of manuscripts identified through PubMed\u003c/strong\u003e. \u003cem\u003eInt J Med Inform \u003c/em\u003e2024, \u003cstrong\u003e195\u003c/strong\u003e:105749.\u003c/li\u003e\n\u003cli\u003eWang W, Krishnan E: \u003cstrong\u003eBig data and clinicians: a review on the state of the science\u003c/strong\u003e. \u003cem\u003eJMIR Med Inform \u003c/em\u003e2014, \u003cstrong\u003e2\u003c/strong\u003e(1):e1.\u003c/li\u003e\n\u003cli\u003eHoffman S: \u003cstrong\u003eBig data analytics: what can go wrong\u003c/strong\u003e. \u003cem\u003eInd Health L Rev \u003c/em\u003e2018, \u003cstrong\u003e15\u003c/strong\u003e:227.\u003c/li\u003e\n\u003cli\u003eFan J, Han F, Liu H: \u003cstrong\u003eChallenges of big data analysis\u003c/strong\u003e. \u003cem\u003eNational science review \u003c/em\u003e2014, \u003cstrong\u003e1\u003c/strong\u003e(2):293-314.\u003c/li\u003e\n\u003cli\u003eKim M, Kim Y, Kang HJ, Seo H, Choi H, Han J, Kee G, Park S, Ko S, Jung H\u003cem\u003e et al\u003c/em\u003e: \u003cstrong\u003eFine-Tuning LLMs with Medical Data: Can Safety Be Ensured?\u003c/strong\u003e \u003cem\u003eNEJM AI \u003c/em\u003e2025, \u003cstrong\u003e2\u003c/strong\u003e(1):AIcs2400390.\u003c/li\u003e\n\u003cli\u003eRengers TA, Thiels CA, Salehinejad H: \u003cstrong\u003eAcademic Surgery in the Era of Large Language Models: A Review\u003c/strong\u003e. \u003cem\u003eJAMA Surg \u003c/em\u003e2024, \u003cstrong\u003e159\u003c/strong\u003e(4):445-450.\u003c/li\u003e\n\u003cli\u003eJeyaraman M, Balaji S, Jeyaraman N, Yadav S: \u003cstrong\u003eUnraveling the Ethical Enigma: Artificial Intelligence in Healthcare\u003c/strong\u003e. \u003cem\u003eCureus \u003c/em\u003e2023, \u003cstrong\u003e15\u003c/strong\u003e(8):e43262.\u003c/li\u003e\n\u003cli\u003eWulcan JM, Jacques KL, Lee MA, Kovacs SL, Dausend N, Prince LE, Wulcan J, Marsilio S, Keller SM: \u003cstrong\u003eClassification performance and reproducibility of GPT-4 omni for information extraction from veterinary electronic health records\u003c/strong\u003e. \u003cem\u003earXiv preprint arXiv:240913727 \u003c/em\u003e2024.\u003c/li\u003e\n\u003cli\u003eLopez-Ubeda P, Martin-Noguerol T, Juluru K, Luna A: \u003cstrong\u003eNatural Language Processing in Radiology: Update on Clinical Applications\u003c/strong\u003e. \u003cem\u003eJ Am Coll Radiol \u003c/em\u003e2022, \u003cstrong\u003e19\u003c/strong\u003e(11):1271-1285.\u003c/li\u003e\n\u003cli\u003ePark HJ, Huh JY, Chae G, Choi MG: \u003cstrong\u003eExtraction of clinical data on major pulmonary diseases from unstructured radiologic reports using a large language model\u003c/strong\u003e. \u003cem\u003ePLoS One \u003c/em\u003e2024, \u003cstrong\u003e19\u003c/strong\u003e(11):e0314136.\u003c/li\u003e\n\u003cli\u003eSoni N, Ora M, Agarwal A, Yang T, Bathla G: \u003cstrong\u003eA Review of The Opportunities and Challenges with Large Language Models in Radiology: The Road Ahead\u003c/strong\u003e. \u003cem\u003eAJNR Am J Neuroradiol \u003c/em\u003e2024.\u003c/li\u003e\n\u003cli\u003eCheng J: \u003cstrong\u003eApplications of Large Language Models in Pathology\u003c/strong\u003e. \u003cem\u003eBioengineering (Basel) \u003c/em\u003e2024, \u003cstrong\u003e11\u003c/strong\u003e(4).\u003c/li\u003e\n\u003cli\u003eDavey MG, O\u0026apos;Flaherty C, Cleere EF, Nohilly A, Phelan J, Ronane E, Lowery AJ, Kerin MJ: \u003cstrong\u003eSentinel lymph node biopsy in patients with ductal carcinoma in situ: systematic review and meta-analysis\u003c/strong\u003e. \u003cem\u003eBJS Open \u003c/em\u003e2022, \u003cstrong\u003e6\u003c/strong\u003e(2).\u003c/li\u003e\n\u003cli\u003eChoi IY, Park S, Park B, Chung BH, Kim CS, Lee HM, Byun SS, Lee JY: \u003cstrong\u003eDevelopment of prostate cancer research database with the clinical data warehouse technology for direct linkage with electronic medical record system\u003c/strong\u003e. \u003cem\u003eProstate Int \u003c/em\u003e2013, \u003cstrong\u003e1\u003c/strong\u003e(2):59-64.\u003c/li\u003e\n\u003cli\u003ePark SJ, Lee SJ, Kim H, Kim JK, Chun JW, Lee SJ, Lee HK, Kim DJ, Choi IY: \u003cstrong\u003eMachine learning prediction of dropping out of outpatients with alcohol use disorders\u003c/strong\u003e. \u003cem\u003ePLoS One \u003c/em\u003e2021, \u003cstrong\u003e16\u003c/strong\u003e(8):e0255626.\u003c/li\u003e\n\u003cli\u003eChoi JE, Kim Z, Park CS, Park EH, Lee SB, Lee SK, Choi YJ, Han J, Jung KW, Kim HJ\u003cem\u003e et al\u003c/em\u003e: \u003cstrong\u003eBreast Cancer Statistics in Korea, 2019\u003c/strong\u003e. \u003cem\u003eJ Breast Cancer \u003c/em\u003e2023, \u003cstrong\u003e26\u003c/strong\u003e(3):207-220.\u003c/li\u003e\n\u003cli\u003eWoodfield R, Grant I, Group UKBSO, Follow-Up UKB, Outcomes Working G, Sudlow CL: \u003cstrong\u003eAccuracy of Electronic Health Record Data for Identifying Stroke Cases in Large-Scale Epidemiological Studies: A Systematic Review from the UK Biobank Stroke Outcomes Group\u003c/strong\u003e. \u003cem\u003ePLoS One \u003c/em\u003e2015, \u003cstrong\u003e10\u003c/strong\u003e(10):e0140533.\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Table 1","content":"\u003cp\u003e\u003cstrong\u003eTable 1. Baseline Characteristics of Study Groups\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"624\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"3\" style=\"width: 247px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eCharacteristics\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eManual Review\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eLLM Processing\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eP\u003c/em\u003e\u003c/strong\u003e\u003cstrong\u003e-value\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003en=1,366\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003en=1,734\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"3\" style=\"width: 247px;\"\u003e\n \u003cp\u003eDemographics\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e \u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e \u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\n \u003cp\u003e \u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 233px;\"\u003e\n \u003cp\u003eAge, mean (SD), y\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e55.0 (11.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e53.5 (11.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\n \u003cp\u003e\u0026lt;0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"3\" style=\"width: 247px;\"\u003e\n \u003cp\u003eSurgical Procedures, No. (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 233px;\"\u003e\n \u003cp\u003eBreast Operation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eBreast-conserving surgery\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e868 (63.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e949 (63.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"5\" style=\"width: 93px;\"\u003e\n \u003cp\u003e\u0026lt;0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eTotal mastectomy\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e262 (19.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e393 (26.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eN(S)SM\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e214 (15.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e142 (9.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eOther procedures\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e22 (1.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e1 (0.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eCombined mastectomy\u003csup\u003ea\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e476 (34.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e535 (36.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 233px;\"\u003e\n \u003cp\u003eAxillary Surgery\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eNo surgery\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e153 (11.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e321 (19.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"3\" style=\"width: 93px;\"\u003e\n \u003cp\u003e\u0026lt;0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eSLNB\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e934 (68.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e990 (59.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eALND\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e280 (20.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e346 (20.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"3\" style=\"width: 247px;\"\u003e\n \u003cp\u003ePathological Results\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 233px;\"\u003e\n \u003cp\u003eTumor Size, mean (SD), mm\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e20.5 (16.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e21.5 (16.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\n \u003cp\u003e0.156\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 233px;\"\u003e\n \u003cp\u003eLymph Node Status\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eHarvested nodes, mean (SD)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e7.79 (7.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e7.11 (7.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\n \u003cp\u003e0.016\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eMetastatic nodes, mean (SD)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e0.95 (3.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e0.98 (3.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\n \u003cp\u003e0.802\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 233px;\"\u003e\n \u003cp\u003eStage Distribution, No. (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e237 (17.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e237 (15.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"9\" style=\"width: 93px;\"\u003e\n \u003cp\u003e\u0026lt;0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eIA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e472 (35.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e659 (43.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eIB\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e8 (0.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e20 (1.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eIIA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e294 (22.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e316 (20.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eIIB\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e147 (11.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e165 (10.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eIIIA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e99 (7.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e86 (5.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eIIIB\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e11 (0.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e1 (0.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eIIIC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e49 (3.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e39 (2.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eIV\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e6 (0.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e0 (0)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 233px;\"\u003e\n \u003cp\u003eBiomarker Status, No. (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eER positive\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e1012 (78.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e1198 (76.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\n \u003cp\u003e0.172\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003ePR positive\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e886 (68.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e1059 (67.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\n \u003cp\u003e0.525\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eHER2 positive\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e249 (20.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e298 (20.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\n \u003cp\u003e0.003\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eKi-67, mean (SD)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e25.4 (23.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e26.6 (22.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\n \u003cp\u003e0.204\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 233px;\"\u003e\n \u003cp\u003eHistologic Grade, No. (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eGrade 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e169 (22.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e283 (21.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"3\" style=\"width: 93px;\"\u003e\n \u003cp\u003e0.764\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eGrade 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e331 (44.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e610 (46.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eGrade 3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e241 (32.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e423 (32.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 233px;\"\u003e\n \u003cp\u003eNuclear Grade, No. (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eGrade 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e137 (16.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e199 (12.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"3\" style=\"width: 93px;\"\u003e\n \u003cp\u003e\u0026lt;0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eGrade 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e348 (42.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e795 (51.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 218px;\"\u003e\n \u003cp\u003eGrade 3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e334 (40.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e549 (35.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"3\" style=\"width: 247px;\"\u003e\n \u003cp\u003eSurvival Outcomes, No. (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 15px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 233px;\"\u003e\n \u003cp\u003eDeath\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e11 (0.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e42 (2.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93px;\"\u003e\n \u003cp\u003e0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"6\" style=\"width: 624px;\"\u003e\n \u003cp\u003eAbbreviations: ALND, axillary lymph node dissection; ER, estrogen receptor; HER2, human epidermal growth factor receptor 2; N(S)SM, nipple or skin-sparing mastectomy; PR, progesterone receptor; SD, standard deviation; SLNB, sentinel lymph node biopsy.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"6\" style=\"width: 624px;\"\u003e\n \u003cp\u003eStatistical significance set at \u003cem\u003eP\u003c/em\u003e \u0026lt; 0.05. Analyses were performed using chi-square test for categorical variables and t test for continuous variables.\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u003csup\u003ea\u003c/sup\u003e Combined mastectomy includes total mastectomy and N(S)SM.\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Natural Language Processing, Breast Neoplasms, Data Mining, Clinical Oncology","lastPublishedDoi":"10.21203/rs.3.rs-7089616/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7089616/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003ePurpose\u003c/strong\u003e: Large language models (LLMs) offer the potential to automate the extraction of clinical data; however, few studies have been published on their feasibility in the field of oncology. To evaluate the feasibility and accuracy of LLM-based processing compared with manual physician review for the extraction of clinical data from breast cancer records.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003eThe groups yielded comparable results for most clinical parameters. The LLM group yielded better documentation of lymph node assessment (91.2% vs. 78.5%) but had a larger proportion of missing data for cancer staging (12.2% vs. 3.1%). Breast-conserving surgery rates were similar (63.5% vs. 63.9%). The LLM achieved 90.8% accuracy in validation analysis while requiring significantly less processing time (12 days vs. 7 months) and fewer physicians (two vs. five). The LLM group’s stage distribution aligned better with the national registry data than the manual-review group(Cramér’sV = 0.03 vs. 0.07), and it captured more survival events (41 vs. 11; \u003cem\u003eP \u003c/em\u003e= 0.002).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDiscussion\u003c/strong\u003e: LLM-based processing demonstrated comparable effectiveness to manual review by physicians, while significantly reducing processing time and resource utilization. Despite limitations in integrated assessments, this approach shows potential for efficient clinical data curation in oncology research.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e: This retrospective study analyzed breast cancer records from five academic hospitals (2019). Two independent cohorts were compared: manual physician review (n=1,366) and LLM-based processing using Claude 3.5 Sonnet (n=1,734) groups. Primary outcomes included missing value rates, accuracy of data extraction, and inter-cohort concordance. Secondary outcomes included comparison with national registry data, processing time, and resource utilization.\u003c/p\u003e","manuscriptTitle":"Comparison of Large Language Model and Manual Review for Clinical Data Curation in Breast Cancer","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-01 09:47:42","doi":"10.21203/rs.3.rs-7089616/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"4d6b05c3-8918-4d8b-9c71-d70ceda32f6d","owner":[],"postedDate":"September 1st, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":53694614,"name":"Biological sciences/Cancer"},{"id":53694615,"name":"Health sciences/Health care"},{"id":53694616,"name":"Health sciences/Medical research"},{"id":53694617,"name":"Health sciences/Oncology"}],"tags":[],"updatedAt":"2025-09-18T11:24:26+00:00","versionOfRecord":[],"versionCreatedAt":"2025-09-01 09:47:42","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7089616","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7089616","identity":"rs-7089616","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}