Artificial Intelligence–Assisted Ultrasound Interpretation Enhances Diagnostic Performance Among Trainees in Thyroid Imaging

Artificial Intelligence–Assisted Ultrasound Interpretation Enhances Diagnostic Performance Among Trainees in Thyroid Imaging | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Artificial Intelligence–Assisted Ultrasound Interpretation Enhances Diagnostic Performance Among Trainees in Thyroid Imaging Fumihiko Furuya, Keiichi Nakano, Yoshiko Matsumoto, Koki Shio, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9329400/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background: Accurate interpretation of thyroid ultrasound requires substantial experience, and diagnostic performance among trainees remains variable. Artificial intelligence (AI) has shown promise in medical image analysis; however, its role as an educational support tool for trainees has not been fully established. This study aimed to evaluate whether an AI-assisted system could improve trainee performance in thyroid ultrasound interpretation. Methods: We developed an AI-based diagnostic support system using a Mask R-CNN framework trained on pathology-confirmed thyroid ultrasound images. The model performed segmentation and classification of five lesion categories. A reader study was conducted involving five expert physicians and five trainees. Trainees evaluated ultrasound images under non-assisted and AI-assisted conditions. Diagnostic performance was assessed using sensitivity, specificity, precision, and F1 score. Results: The AI model demonstrated high segmentation accuracy for several structures, including vessels and thyroid parenchyma. However, its ability to distinguish benign from malignant tumors remained limited. Despite this, AI assistance improved trainee performance. Median precision increased from 0.53 to 0.63, and F1 score improved from 0.53 to 0.61. Specificity increased from 0.22 to 0.50, while sensitivity remained comparable. These findings indicate that AI support primarily reduced false-positive interpretations and improved diagnostic consistency among trainees. Conclusions: AI-assisted ultrasound interpretation improved the diagnostic performance of trainees, particularly by enhancing precision and reducing false-positive findings. Although the AI model alone was insufficient for independent clinical use, it demonstrated potential as an educational support tool. These findings suggest that AI may contribute to more consistent and effective training in thyroid ultrasound interpretation. Figures Figure 1 Figure 2 Introduction Thyroid ultrasound is widely used for the evaluation of thyroid morphology and the characterization of nodular lesions. Accurate interpretation of ultrasound findings is essential for appropriate clinical decision-making, including the indication for fine-needle aspiration cytology. However, ultrasound interpretation is highly operator-dependent and requires substantial training and experience, leading to variability in diagnostic performance among clinicians. In large-scale examination settings, such as those implemented following the Chernobyl nuclear disaster, there is an increased demand for rapid and consistent image interpretation [ 1 , 2 ]. In such situations, the limited availability of experienced specialists poses a significant challenge, and the need for effective training strategies for novice clinicians becomes increasingly important. Artificial intelligence (AI), particularly convolutional neural networks (CNNs), has shown promising performance in medical image analysis, including classification, object detection, and segmentation tasks. In thyroid ultrasound, AI has been primarily investigated as a tool for automated diagnosis, focusing on distinguishing between benign and malignant lesions [ 3 , 4 ]. However, despite these advances, the role of AI as an educational support tool for trainees has not been fully established. Given that clinical decision-making still requires human judgment, it is important to explore how AI can be integrated into training to enhance diagnostic performance and consistency among less experienced clinicians. In this study, we developed an AI-based diagnostic support system using a Mask R-CNN framework trained on pathology-confirmed thyroid ultrasound images. We aimed to evaluate not only its diagnostic performance but also its impact on trainee interpretation of thyroid ultrasound images. Specifically, we investigated whether AI assistance could improve diagnostic performance and reduce variability among novice trainees. Materials and Methods Development of the AI Diagnostic Support Tool Preoperative ultrasound images from patients with surgically treated and histopathologically confirmed thyroid lesions were retrospectively analyzed. Ground truth annotations were created for five classes: thyroid parenchyma, lymph node, vessel, benign tumor (including follicular adenoma, adenomatous nodule, and cyst), and malignant tumor. All cases in the malignant tumor category were pathologically diagnosed as papillary thyroid carcinoma. Each ultrasound image could include multiple classes within the same frame. The dataset consisted of 1,206 images divided into training (n = 700), validation (n = 254), and test (n = 252) sets, derived from 340 patients. To prevent data leakage, all images from a single patient were assigned exclusively to one subset. To mitigate class imbalance, data augmentation was applied during training, including horizontal flipping. These procedures approximately doubled the number of minority-class samples. Additionally, five-fold cross-validation was performed using the same hyperparameter settings to verify model robustness. Cross-validation results demonstrated consistent segmentation performance across folds (mean mAP = 0.78 ± 0.03; mean IoU = 0.81 ± 0.02), confirming that augmentation and balanced sampling effectively reduced bias toward majority classes. Ethical Approval This study was approved by the Ethics Committees of Fukushima Medical University (approval number: 2022192) and University of Yamanashi (approval number: 2740). Written informed consent was obtained from all participants. The ultrasound image data were accessed for research purposes from 01/05/2024 to 10/10/2025. All procedures performed in this study were in accordance with the ethical standards of the institutional research committees and with the Declaration of Helsinki. Participants and Diagnostic Accuracy Evaluation Five board-certified thyroid imaging expert doctors with over 10 years of experience and five medical trainees from Fukushima Medical University who had passed the Japanese national Objective Structured Clinical Examination and Computer-Based Testing were recruited. They were presented with 35 ultrasound images from 9 cases with confirmed pathology (including 29 malignant and 18 benign lesions). The AI’s predictions were matched with ground truth labels based on maximum Intersection over Union (IoU). The prediction was counted as correct if the class label and region matched. To evaluate the effect of AI support on diagnostic performance, all participants reviewed the same set of thyroid ultrasound images under two conditions. In the first round (“non-assisted” session), trainees interpreted the original ultrasound images without any AI-generated information and annotated lesions based solely on their own judgment. In the second round (“AI-assisted” session), the same trainees evaluated identical images with Mask R-CNN–generated segmentation overlays, class labels (benign or malignant), and confidence scores displayed on the screen. The AI output highlighted predicted lesion areas and showed the model’s confidence for each classification. Participants could view these AI-generated suggestions but were not allowed to modify them. All ultrasound images were obtained from patients who subsequently underwent thyroid surgery, and final pathological diagnoses were confirmed postoperatively. Statistical Analysis Group comparisons were performed using the Mann–Whitney U test. Statistical analyses were performed using STATA software (version 14.2; Stata Corp., College Station, TX, USA). Statistical significance was defined as p < 0.05. Results Performance of the AI Tool The AI achieved promising segmentation accuracy on the test dataset. Table 1 summarizes the distribution of ultrasound images used for the development and evaluation of the AI diagnostic model. The dataset was categorized into five anatomical/pathological classes: parenchyma, lymph node, vessel, benign tumor (including follicular adenoma, adenomatous nodule, and cyst), and malignant tumor. Images were randomly allocated into training, validation, and test sets. In the test dataset, the model detected 210 thyroid parenchyma regions (92.3% detection rate), 2 lymph node regions (20.0%), 130 vessel regions (82.8%), 101 benign tumor regions (82.8%), and 65 malignant tumor regions (69.1%). The Intersection over Union (IoU) values for the detected regions were 0.80 (95% CI: 0.76–0.82) for thyroid parenchyma, 0.89 (95% CI: 0.88–0.90) for lymph nodes, 0.89 (95% CI: 0.88–0.90) for vessels, 0.80 (95% CI: 0.79–0.84) for benign tumors, and 0.81 (95% CI: 0.76–0.82) for malignant tumors, indicating high localization accuracy in all classes. Due to the considerable imbalance in lesion type distribution, a confusion matrix was constructed to comprehensively evaluate the classification performance of the model across all lesion categories. As shown in Figure 1, the model demonstrated high sensitivity for vessel lesions (95.1%) and parenchymal structures (66.2%), but frequently misclassified benign and malignant tumors. Notably, 47.0% of pathologically confirmed malignant tumors were correctly identified as malignant tumors, while an equal proportion (47.0%) were misclassified as benign. Comparison with Experts and Trainees To evaluate the diagnostic utility of the AI model developed in this study, we assessed its ability to differentiate between benign and malignant thyroid tumors. The model’s diagnostic performance was compared with that of expert doctors and trainees. Furthermore, we investigated the AI's effectiveness as a diagnostic support tool for trainees. For this analysis, an additional set of 47 ultrasound images was used, consisting of 18 benign tumors and 29 malignant tumors (all histologically confirmed). Detailed IoU values for each class are shown in Table 2. The AI demonstrated the highest IoU for vessel regions (0.90, 95% CI: 0.80–0.91), followed by malignant tumors (0.83, 95% CI: 0.67–0.93), benign tumors (0.81, 95% CI: 0.75–0.86), and thyroid parenchyma (0.62, 95% CI: 0.48–0.67). These results indicate particularly accurate segmentation of vascular and tumor structures, which are clinically relevant for diagnosis. The diagnostic performance of trainees was compared between the non-assisted and AI-assisted sessions and further contrasted with that of expert physicians. The performance metrics are summarized in Table 3 and illustrated in Figure 2. Without AI assistance, trainees demonstrated relatively low diagnostic accuracy, with a median precision of 0.53 and F1 score of 0.53. When supported by AI-generated segmentation overlays, class predictions, and confidence scores, their performance modestly improved. Median precision increased from 0.53 to 0.63, and the F1 score from 0.53 to 0.61. Specificity also improved from 0.22 to 0.50, while sensitivity remained comparable (0.55 vs. 0.52). These results indicate that AI assistance mainly enhanced precision and specificity, reducing false-positive interpretations by trainees. However, statistically significant differences (p < 0.05) in sensitivity and F1 score remained between AI-assisted trainees and experts, suggesting that the AI tool only partially closed the performance gap. Discussion In this study, we developed an AI-assisted diagnostic support system for thyroid ultrasound interpretation and evaluated its impact on trainee performance. The results demonstrated that AI assistance modestly improved diagnostic performance among trainees, particularly in terms of precision and specificity. These improvements were primarily driven by a reduction in false-positive interpretations, suggesting that AI support may enhance the consistency of diagnostic decision-making among less experienced clinicians. Although the AI model achieved high segmentation performance for several anatomical structures, including vessels and thyroid parenchyma, its classification performance for distinguishing benign from malignant tumors remained limited. As shown in the confusion matrix, nearly half of malignant tumors were misclassified as benign, indicating that the standalone diagnostic performance of the model is insufficient for independent clinical use. However, despite these limitations, the AI system still contributed to improved trainee performance, highlighting its potential value as a supportive educational tool rather than a replacement for expert judgment. These findings are consistent with the concept that AI can augment, rather than replace, human expertise. Clinical interpretation of ultrasound images requires integration of multiple sources of information, including imaging features, clinical context, and experience-based judgment. In this setting, AI-generated segmentation overlays and probability-based outputs may serve as visual and cognitive aids, helping trainees to better recognize lesion characteristics and avoid overinterpretation. Previous studies have primarily focused on improving diagnostic accuracy using AI-based classification systems, such as AI-integrated Thyroid Imaging Reporting and Data Systems (AI-TIRADS), which demonstrated improved diagnostic accuracy and interobserver agreement [5]. In contrast, our approach emphasizes pixel-level segmentation combined with classification, providing intuitive visual feedback that may be particularly beneficial in educational settings. Similar findings have been reported in breast ultrasound studies, where AI assistance improved diagnostic performance, especially among less experienced clinicians. Together, these studies support the role of AI as a complementary tool that enhances, but does not replace, clinical expertise. From an educational perspective, our results suggest that AI-assisted systems may help standardize image interpretation and reduce variability among trainees. In environments where access to expert supervision is limited, such tools could support more efficient learning and improve diagnostic consistency. Importantly, rather than eliminating the need for training, the introduction of AI may redefine the learning process by requiring clinicians to develop skills in interpreting and appropriately utilizing AI-generated outputs. Several limitations should be acknowledged. First, the dataset size was relatively small and derived from a single institution, which may limit the generalizability of the findings. Second, class imbalance, particularly for lymph nodes, affected model performance despite the use of data augmentation and cross-validation. Third, although AI assistance improved trainee performance, it did not achieve expert-level accuracy, indicating that further refinement of the model and its integration into clinical workflows is necessary. In addition, future studies should evaluate the impact of AI-assisted training in larger and more diverse cohorts, as well as explore user experience and learning outcomes over time. Improvements in model interpretability, confidence calibration, and user interface design may further enhance the effectiveness of AI as an educational support tool. In conclusion, our AI-assisted diagnostic system improved the diagnostic performance of trainees in thyroid ultrasound interpretation, particularly by enhancing precision and reducing false-positive findings. While the model is not sufficient as a standalone diagnostic tool, it shows promise as an educational aid that may support more consistent and efficient training in thyroid ultrasound interpretation. Declarations Acknowledgments We sincerely thank the trainees and expert physicians who participated in this study for their valuable contributions. Ethics approval and consent to participate: This study was approved by the Ethics Committees of Fukushima Medical University (approval number: 2022192) and University of Yamanashi (approval number: 2740). All procedures were performed in accordance with the Declaration of Helsinki. Consent to participate: Written informed consent was obtained from all participants. Consent for publication: Not applicable. Competing interests: The authors declare that they have no competing interests. Data Availability Statement The datasets used or analyzed during the current study are available from the corresponding author on reasonable request. Author contributions FF conceived and designed the study. FF and KN performed data analysis. KS, YM, and SS (Satoshi) acquired the ultrasound images. SS (Satoru), HS, HA contributed to study supervision and interpretation of the data. All authors contributed to manuscript drafting and revision and approved the final version of the manuscript. Funding This study was supported by commissioned research funding from FREI and the Environmental Health Research Program on Radiation Effects of the Ministry of the Environment, Japan. References Kazakov VS, Demidchik EP, Astakhova LN. Thyroid cancer after Chernobyl. Nature. 1992;359(6390):21–21. Zablotska LB, Ron E, Rozhko AV, Hatch M, Polyanskaya ON, Brenner AV, Lubin J, Romanov GN, McConnell RJ, O'Kane P, et al. Thyroid cancer risk in Belarus among children and adolescents exposed to radioiodine after the Chornobyl accident. Br J Cancer. 2011;104(1):181–7. He K, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. IEEE Trans Pattern Anal Mach Intell. 2020;42(2):386–97. Cai Z, Vasconcelos N. Cascade R-CNN: High Quality Object Detection and Instance Segmentation. IEEE Trans Pattern Anal Mach Intell. 2021;43(5):1483–98. Li X, Zhang S, Zhang Q, Wei X, Pan Y, Zhao J, Xin X, Qin C, Wang X, Li J, et al. Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images: a retrospective, multicohort, diagnostic study. Lancet Oncol. 2019;20(2):193–201. Tables Table1. Dataset composition used for AI model development and evaluation. IoU: Intersection over Union. mAP: mean Average Precision. IoU values represent the median for each class; values in parentheses indicate the 95% confidence intervals (CI). mAP was calculated using a threshold of IoU ≥ 0.5. parenchyma (n=1,077) lymph node (n=110) vessel (n=790) benign tumor (n=1,036) malignant tumor (n=549) training 646 71 474 731 287 validation 205 29 159 131 168 test 226 10 157 174 94 true positive 210 2 130 101 65 IoU 0.80 (0.76-0.82) 0.89 (0.88-0.90) 0.89 (0.88-0.90) 0.80 (0.79-0.84) 0.81 (0.76-0.82) mAP 0.72 0.08 0.83 0.48 0.70 Table 2. IoU values for each lesion class. IoU values represent the median for each class; values in parentheses indicate the 95% confidence intervals. parenchyma vessel benign tumor malignant tumor IoU 0.62 (0.48-0.67) 0.9 (0.80-0.91) 0.81 (0.75-0.86) 0.83 (0.67-0.93) Table 3. Diagnostic performance of experts, trainees, and AI-assisted trainees. Sensitivity, Specificity, Precision, and F1 Score represent the median values for each group; values in parentheses indicate the 95% confidence intervals. Sensitivity Specificity Precision F1 Score experts 0.72 (0.56-0.79) 0.78 (0.57-0.92) 0.80 (0.75-0.94) 0.77 (0.66-0.80) trainees 0.55 (0.25-0.68) 0.22 (0.12-0.79) 0.53 (0.42-0.75) 0.53 (0.32-0.63) trainees+AI 0.52 (0.52-0.59) 0.50 (0.45-0.93) 0.63 (0.63-0.93) 0.61 (0.57-0.71) Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9329400","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":622783112,"identity":"71cb2efc-51e3-402e-98a7-a6934a0e0c18","order_by":0,"name":"Fumihiko Furuya","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAwklEQVRIiWNgGAWjYBAC9uYDYFqOvQEiwExQC8+xBDBtzHOAVC2JPQeIdRgPG4/ZZ942u/QesTMGDD9qGNjNidBiPJu3LTm3RzrHgLHnGAOzZQMBLfbyPcbMvG3MufuBWhh4GxiYDQi5EGQLUEt9Og/Ilr8kaDmcANLCTKQtbMWMc84dN+yRTis4LHNMgrBfeNiYNzO8KauW55FO3vjwTY1NMsEQAwNGNggNdJJEsgFRWhj+IJh2RGoZBaNgFIyCEQQAH88xvk8+3ugAAAAASUVORK5CYII=","orcid":"","institution":"Fukushima Medical University","correspondingAuthor":true,"prefix":"","firstName":"Fumihiko","middleName":"","lastName":"Furuya","suffix":""},{"id":622783113,"identity":"553e1785-4345-43d6-ada5-5a7ad6359e14","order_by":1,"name":"Keiichi Nakano","email":"","orcid":"","institution":"Fukushima Medical University","correspondingAuthor":false,"prefix":"","firstName":"Keiichi","middleName":"","lastName":"Nakano","suffix":""},{"id":622783114,"identity":"022bfb5a-4aa4-48c2-9680-45a97c2258ea","order_by":2,"name":"Yoshiko Matsumoto","email":"","orcid":"","institution":"Fukushima Medical University","correspondingAuthor":false,"prefix":"","firstName":"Yoshiko","middleName":"","lastName":"Matsumoto","suffix":""},{"id":622783115,"identity":"f848b631-7b96-420b-9aeb-52d2c6065f61","order_by":3,"name":"Koki Shio","email":"","orcid":"","institution":"Fukushima Medical University","correspondingAuthor":false,"prefix":"","firstName":"Koki","middleName":"","lastName":"Shio","suffix":""},{"id":622783116,"identity":"f3240ce5-598d-47d9-ae38-db44bf818a8e","order_by":4,"name":"Satoshi Suzuki","email":"","orcid":"","institution":"Fukushima Medical University","correspondingAuthor":false,"prefix":"","firstName":"Satoshi","middleName":"","lastName":"Suzuki","suffix":""},{"id":622783117,"identity":"60eea34b-4bb0-4602-81b5-0a026d8bb221","order_by":5,"name":"Satoru Suzuki","email":"","orcid":"","institution":"Fukushima Medical University","correspondingAuthor":false,"prefix":"","firstName":"Satoru","middleName":"","lastName":"Suzuki","suffix":""},{"id":622783118,"identity":"d399ed6d-b184-4fb6-9fbc-b614db3a02a6","order_by":6,"name":"Hiroki Shimura","email":"","orcid":"","institution":"Fukushima Medical University","correspondingAuthor":false,"prefix":"","firstName":"Hiroki","middleName":"","lastName":"Shimura","suffix":""},{"id":622783119,"identity":"6690ead2-1cb1-4d12-9059-33dcba25bff3","order_by":7,"name":"Hidetoshi Ando","email":"","orcid":"","institution":"Graduate School of University of Yamanashi","correspondingAuthor":false,"prefix":"","firstName":"Hidetoshi","middleName":"","lastName":"Ando","suffix":""}],"badges":[],"createdAt":"2026-04-06 03:24:01","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9329400/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9329400/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":107450476,"identity":"68f04a89-52a2-4c69-ba64-804462653117","added_by":"auto","created_at":"2026-04-21 15:12:47","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":1241887,"visible":true,"origin":"","legend":"\u003cp\u003eRow-normalized confusion matrix showing the percentage of predictions for each true class label. Each cell indicates the proportion (%) of samples with a given true label (row) that were predicted as the corresponding class (column). Numbers in parentheses represent the actual counts.\u003c/p\u003e","description":"","filename":"Figure1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9329400/v1/dab7f45146ea644fc04b6d66.jpg"},{"id":107488401,"identity":"ea0c3be3-598e-47c9-b5ba-5fb23adfb24f","added_by":"auto","created_at":"2026-04-22 02:44:34","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":856576,"visible":true,"origin":"","legend":"\u003cp\u003eBoxplots with overlaid individual data points comparing diagnostic performance across three groups: expert doctors (EXP), trainees using AI assistance (TRN+AI), and students without AI assistance (TRN). Each panel shows one of the four performance metrics: A:Sensitivity, B:Specificity, C:Precision, and D:F1 Score. Boxes represent the interquartile range, with the median indicated by the central line. Whiskers extend to the range within 1.5 × IQR. Individual dots represent observed values per evaluator. * indicate statistically significant differences at p \u0026lt; 0.05\u003c/p\u003e","description":"","filename":"Figure2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9329400/v1/8eb5e9546edf758449499e86.jpg"},{"id":108494183,"identity":"2c2e2b67-57ee-4b78-ab78-58be98ef6ea9","added_by":"auto","created_at":"2026-05-05 10:02:50","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":653370,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9329400/v1/75a8edbb-ea42-4476-98f6-4cbecbd69285.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Artificial Intelligence–Assisted Ultrasound Interpretation Enhances Diagnostic Performance Among Trainees in Thyroid Imaging","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThyroid ultrasound is widely used for the evaluation of thyroid morphology and the characterization of nodular lesions. Accurate interpretation of ultrasound findings is essential for appropriate clinical decision-making, including the indication for fine-needle aspiration cytology. However, ultrasound interpretation is highly operator-dependent and requires substantial training and experience, leading to variability in diagnostic performance among clinicians.\u003c/p\u003e \u003cp\u003eIn large-scale examination settings, such as those implemented following the Chernobyl nuclear disaster, there is an increased demand for rapid and consistent image interpretation [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. In such situations, the limited availability of experienced specialists poses a significant challenge, and the need for effective training strategies for novice clinicians becomes increasingly important.\u003c/p\u003e \u003cp\u003eArtificial intelligence (AI), particularly convolutional neural networks (CNNs), has shown promising performance in medical image analysis, including classification, object detection, and segmentation tasks. In thyroid ultrasound, AI has been primarily investigated as a tool for automated diagnosis, focusing on distinguishing between benign and malignant lesions [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eHowever, despite these advances, the role of AI as an educational support tool for trainees has not been fully established. Given that clinical decision-making still requires human judgment, it is important to explore how AI can be integrated into training to enhance diagnostic performance and consistency among less experienced clinicians.\u003c/p\u003e \u003cp\u003eIn this study, we developed an AI-based diagnostic support system using a Mask R-CNN framework trained on pathology-confirmed thyroid ultrasound images. We aimed to evaluate not only its diagnostic performance but also its impact on trainee interpretation of thyroid ultrasound images. Specifically, we investigated whether AI assistance could improve diagnostic performance and reduce variability among novice trainees.\u003c/p\u003e"},{"header":"Materials and Methods","content":"\u003cp\u003e\u003cem\u003eDevelopment of the AI Diagnostic Support Tool\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003ePreoperative ultrasound images from patients with surgically treated and histopathologically confirmed thyroid lesions were retrospectively analyzed. Ground truth annotations were created for five classes: thyroid parenchyma, lymph node, vessel, benign tumor (including follicular adenoma, adenomatous nodule, and cyst), and malignant tumor. All cases in the malignant tumor category were pathologically diagnosed as papillary thyroid carcinoma. Each ultrasound image could include multiple classes within the same frame. The dataset consisted of 1,206 images divided into training (n = 700), validation (n = 254), and test (n = 252) sets, derived from 340 patients. To prevent data leakage, all images from a single patient were assigned exclusively to one subset.\u003c/p\u003e\n\u003cp\u003eTo mitigate class imbalance, data augmentation was applied during training, including horizontal flipping. These procedures approximately doubled the number of minority-class samples. Additionally, five-fold cross-validation was performed using the same hyperparameter settings to verify model robustness. Cross-validation results demonstrated consistent segmentation performance across folds (mean mAP = 0.78 ± 0.03; mean IoU = 0.81 ± 0.02), confirming that augmentation and balanced sampling effectively reduced bias toward majority classes.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eEthical Approval\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThis study was approved by the Ethics Committees of Fukushima Medical University (approval number: 2022192) and University of Yamanashi (approval number: 2740). Written informed consent was obtained from all participants. The ultrasound image data were accessed for research purposes from 01/05/2024 to 10/10/2025. All procedures performed in this study were in accordance with the ethical standards of the institutional research committees and with the Declaration of Helsinki.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eParticipants and Diagnostic Accuracy Evaluation\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eFive board-certified thyroid imaging expert doctors with over 10 years of experience and five medical trainees from Fukushima Medical University who had passed the Japanese national Objective Structured Clinical Examination and Computer-Based Testing were recruited. They were presented with 35 ultrasound images from 9 cases with confirmed pathology (including 29 malignant and 18 benign lesions). The AI’s predictions were matched with ground truth labels based on maximum Intersection over Union (IoU). The prediction was counted as correct if the class label and region matched.\u003c/p\u003e\n\u003cp\u003eTo evaluate the effect of AI support on diagnostic performance, all participants reviewed the same set of thyroid ultrasound images under two conditions. In the first round (“non-assisted” session), trainees interpreted the original ultrasound images without any AI-generated information and annotated lesions based solely on their own judgment. In the second round (“AI-assisted” session), the same trainees evaluated identical images with Mask R-CNN–generated segmentation overlays, class labels (benign or malignant), and confidence scores displayed on the screen. The AI output highlighted predicted lesion areas and showed the model’s confidence for each classification. Participants could view these AI-generated suggestions but were not allowed to modify them. All ultrasound images were obtained from patients who subsequently underwent thyroid surgery, and final pathological diagnoses were confirmed postoperatively.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eStatistical Analysis\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eGroup comparisons were performed using the Mann–Whitney U test. Statistical analyses were performed using STATA software (version 14.2; Stata Corp., College Station, TX, USA). Statistical significance was defined as p \u0026lt; 0.05.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cem\u003ePerformance of the AI Tool\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe AI achieved promising segmentation accuracy on the test dataset. Table 1 summarizes the distribution of ultrasound images used for the development and evaluation of the AI diagnostic model. The dataset was categorized into five anatomical/pathological classes: parenchyma, lymph node, vessel, benign tumor (including follicular adenoma, adenomatous nodule, and cyst), and malignant tumor.\u003c/p\u003e\n\u003cp\u003eImages were randomly allocated into training, validation, and test sets. In the test dataset, the model detected 210 thyroid parenchyma regions (92.3% detection rate), 2 lymph node regions (20.0%), 130 vessel regions (82.8%), 101 benign tumor regions (82.8%), and 65 malignant tumor regions (69.1%). The Intersection over Union (IoU) values for the detected regions were 0.80 (95% CI: 0.76\u0026ndash;0.82) for thyroid parenchyma, 0.89 (95% CI: 0.88\u0026ndash;0.90) for lymph nodes, 0.89 (95% CI: 0.88\u0026ndash;0.90) for vessels, 0.80 (95% CI: 0.79\u0026ndash;0.84) for benign tumors, and 0.81 (95% CI: 0.76\u0026ndash;0.82) for malignant tumors, indicating high localization accuracy in all classes.\u003c/p\u003e\n\u003cp\u003eDue to the considerable imbalance in lesion type distribution, a confusion matrix was constructed to comprehensively evaluate the classification performance of the model across all lesion categories. As shown in Figure 1, the model demonstrated high sensitivity for vessel lesions (95.1%) and parenchymal structures (66.2%), but frequently misclassified benign and malignant tumors. Notably, 47.0% of pathologically confirmed malignant tumors were correctly identified as malignant tumors, while an equal proportion (47.0%) were misclassified as benign.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eComparison with Experts and Trainees\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eTo evaluate the diagnostic utility of the AI model developed in this study, we assessed its ability to differentiate between benign and malignant thyroid tumors. The model\u0026rsquo;s diagnostic performance was compared with that of expert doctors and trainees. Furthermore, we investigated the AI\u0026apos;s effectiveness as a diagnostic support tool for trainees. For this analysis, an additional set of 47 ultrasound images was used, consisting of 18 benign tumors and 29 malignant tumors (all histologically confirmed). Detailed IoU values for each class are shown in Table 2. The AI demonstrated the highest IoU for vessel regions (0.90, 95% CI: 0.80\u0026ndash;0.91), followed by malignant tumors (0.83, 95% CI: 0.67\u0026ndash;0.93), benign tumors (0.81, 95% CI: 0.75\u0026ndash;0.86), and thyroid parenchyma (0.62, 95% CI: 0.48\u0026ndash;0.67). These results indicate particularly accurate segmentation of vascular and tumor structures, which are clinically relevant for diagnosis.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe diagnostic performance of trainees was compared between the non-assisted and AI-assisted sessions and further contrasted with that of expert physicians. The performance metrics are summarized in Table 3 and illustrated in Figure 2. Without AI assistance, trainees demonstrated relatively low diagnostic accuracy, with a median precision of 0.53 and F1 score of 0.53. When supported by AI-generated segmentation overlays, class predictions, and confidence scores, their performance modestly improved. Median precision increased from 0.53 to 0.63, and the F1 score from 0.53 to 0.61. Specificity also improved from 0.22 to 0.50, while sensitivity remained comparable (0.55 vs. 0.52). These results indicate that AI assistance mainly enhanced precision and specificity, reducing false-positive interpretations by trainees. However, statistically significant differences (p \u0026lt; 0.05) in sensitivity and F1 score remained between AI-assisted trainees and experts, suggesting that the AI tool only partially closed the performance gap.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn this study, we developed an AI-assisted diagnostic support system for thyroid ultrasound interpretation and evaluated its impact on trainee performance. The results demonstrated that AI assistance modestly improved diagnostic performance among trainees, particularly in terms of precision and specificity. These improvements were primarily driven by a reduction in false-positive interpretations, suggesting that AI support may enhance the consistency of diagnostic decision-making among less experienced clinicians.\u003c/p\u003e\n\u003cp\u003eAlthough the AI model achieved high segmentation performance for several anatomical structures, including vessels and thyroid parenchyma, its classification performance for distinguishing benign from malignant tumors remained limited. As shown in the confusion matrix, nearly half of malignant tumors were misclassified as benign, indicating that the standalone diagnostic performance of the model is insufficient for independent clinical use. However, despite these limitations, the AI system still contributed to improved trainee performance, highlighting its potential value as a supportive educational tool rather than a replacement for expert judgment.\u003c/p\u003e\n\u003cp\u003eThese findings are consistent with the concept that AI can augment, rather than replace, human expertise. Clinical interpretation of ultrasound images requires integration of multiple sources of information, including imaging features, clinical context, and experience-based judgment. In this setting, AI-generated segmentation overlays and probability-based outputs may serve as visual and cognitive aids, helping trainees to better recognize lesion characteristics and avoid overinterpretation.\u003c/p\u003e\n\u003cp\u003ePrevious studies have primarily focused on improving diagnostic accuracy using AI-based classification systems, such as AI-integrated Thyroid Imaging Reporting and Data Systems (AI-TIRADS), which demonstrated improved diagnostic accuracy and interobserver agreement [5]. In contrast, our approach emphasizes pixel-level segmentation combined with classification, providing intuitive visual feedback that may be particularly beneficial in educational settings. Similar findings have been reported in breast ultrasound studies, where AI assistance improved diagnostic performance, especially among less experienced clinicians. Together, these studies support the role of AI as a complementary tool that enhances, but does not replace, clinical expertise.\u003c/p\u003e\n\u003cp\u003eFrom an educational perspective, our results suggest that AI-assisted systems may help standardize image interpretation and reduce variability among trainees. In environments where access to expert supervision is limited, such tools could support more efficient learning and improve diagnostic consistency. Importantly, rather than eliminating the need for training, the introduction of AI may redefine the learning process by requiring clinicians to develop skills in interpreting and appropriately utilizing AI-generated outputs.\u003c/p\u003e\n\u003cp\u003eSeveral limitations should be acknowledged. First, the dataset size was relatively small and derived from a single institution, which may limit the generalizability of the findings. Second, class imbalance, particularly for lymph nodes, affected model performance despite the use of data augmentation and cross-validation. Third, although AI assistance improved trainee performance, it did not achieve expert-level accuracy, indicating that further refinement of the model and its integration into clinical workflows is necessary.\u003c/p\u003e\n\u003cp\u003eIn addition, future studies should evaluate the impact of AI-assisted training in larger and more diverse cohorts, as well as explore user experience and learning outcomes over time. Improvements in model interpretability, confidence calibration, and user interface design may further enhance the effectiveness of AI as an educational support tool.\u003c/p\u003e\n\u003cp\u003eIn conclusion, our AI-assisted diagnostic system improved the diagnostic performance of trainees in thyroid ultrasound interpretation, particularly by enhancing precision and reducing false-positive findings. While the model is not sufficient as a standalone diagnostic tool, it shows promise as an educational aid that may support more consistent and efficient training in thyroid ultrasound interpretation.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe sincerely thank the trainees and expert physicians who participated in this study for their valuable contributions.\u003c/p\u003e\n\u003cp\u003eEthics approval and consent to participate:\u003c/p\u003e\n\u003cp\u003eThis study was approved by the Ethics Committees of Fukushima Medical University (approval number: 2022192) and University of Yamanashi (approval number: 2740). All procedures were performed in accordance with the Declaration of Helsinki.\u003c/p\u003e\n\u003cp\u003eConsent to participate:\u003c/p\u003e\n\u003cp\u003eWritten informed consent was obtained from all participants.\u003c/p\u003e\n\u003cp\u003eConsent for publication:\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003eCompeting interests:\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Availability Statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe datasets used or analyzed during the current study are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFF conceived and designed the study. FF and KN performed data analysis. KS, YM, and SS (Satoshi) acquired the ultrasound images. SS (Satoru), HS, HA contributed to study supervision and interpretation of the data. All authors contributed to manuscript drafting and revision and approved the final version of the manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study was supported by commissioned research funding from FREI and the Environmental Health Research Program on Radiation Effects of the Ministry of the Environment, Japan.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eKazakov VS, Demidchik EP, Astakhova LN. Thyroid cancer after Chernobyl. Nature. 1992;359(6390):21\u0026ndash;21.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZablotska LB, Ron E, Rozhko AV, Hatch M, Polyanskaya ON, Brenner AV, Lubin J, Romanov GN, McConnell RJ, O'Kane P, et al. Thyroid cancer risk in Belarus among children and adolescents exposed to radioiodine after the Chornobyl accident. Br J Cancer. 2011;104(1):181\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe K, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. IEEE Trans Pattern Anal Mach Intell. 2020;42(2):386\u0026ndash;97.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCai Z, Vasconcelos N. Cascade R-CNN: High Quality Object Detection and Instance Segmentation. IEEE Trans Pattern Anal Mach Intell. 2021;43(5):1483\u0026ndash;98.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi X, Zhang S, Zhang Q, Wei X, Pan Y, Zhao J, Xin X, Qin C, Wang X, Li J, et al. Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images: a retrospective, multicohort, diagnostic study. Lancet Oncol. 2019;20(2):193\u0026ndash;201.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"},{"header":"Tables","content":"\u003cp\u003eTable1. Dataset composition used for AI model development and evaluation.\u003c/p\u003e\n\u003cp\u003eIoU: Intersection over Union. mAP: mean Average Precision. IoU values represent the median for each class; values in parentheses indicate the 95% confidence intervals (CI). mAP was calculated using a threshold of IoU \u0026ge; 0.5.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"761\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003eparenchyma\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n=1,077)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003elymph node\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n=110)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003evessel\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n=790)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003ebenign tumor\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n=1,036)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003emalignant tumor\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n=549)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003etraining\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e646\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e71\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e474\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e731\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e287\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003evalidation\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e205\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e29\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e159\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e131\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e168\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003etest\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e226\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e157\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e174\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e94\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003etrue positive\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e210\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e130\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e101\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e65\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003eIoU\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.80 (0.76-0.82)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.89 (0.88-0.90)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.89 (0.88-0.90)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.80 (0.79-0.84)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.81 (0.76-0.82)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003emAP\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.72\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.08\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.83\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.48\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.70\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eTable 2. IoU values for each lesion class.\u003c/p\u003e\n\u003cp\u003eIoU values represent the median for each class; values in parentheses indicate the 95% confidence intervals.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"630\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd nowrap=\"\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cstrong\u003eparenchyma\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cstrong\u003evessel\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cstrong\u003ebenign tumor\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cstrong\u003emalignant tumor\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003eIoU\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.62\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.48-0.67)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.80-0.91)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.81\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.75-0.86)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.83\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.67-0.93)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eTable 3. Diagnostic performance of experts, trainees, and AI-assisted trainees.\u003c/p\u003e\n\u003cp\u003eSensitivity, Specificity, Precision, and F1 Score represent the median values for each group; values in parentheses indicate the 95% confidence intervals.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"653\" class=\"fr-table-selection-hover\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd nowrap=\"\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cstrong\u003eSensitivity\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cstrong\u003eSpecificity\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cstrong\u003ePrecision\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cstrong\u003eF1 Score\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003eexperts\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.72\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.56-0.79)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.78\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.57-0.92)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.80\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.75-0.94)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.77\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.66-0.80)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003etrainees\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.55\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.25-0.68)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.22\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.12-0.79)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.53\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.42-0.75)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.53\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.32-0.63)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e\u003cstrong\u003etrainees+AI\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.52\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.52-0.59)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.50\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.45-0.93)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.63\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.63-0.93)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e0.61\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd nowrap=\"\"\u003e\n \u003cp\u003e(0.57-0.71)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-9329400/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9329400/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground:\u003c/h2\u003e \u003cp\u003eAccurate interpretation of thyroid ultrasound requires substantial experience, and diagnostic performance among trainees remains variable. Artificial intelligence (AI) has shown promise in medical image analysis; however, its role as an educational support tool for trainees has not been fully established. This study aimed to evaluate whether an AI-assisted system could improve trainee performance in thyroid ultrasound interpretation.\u003c/p\u003e\u003ch2\u003eMethods:\u003c/h2\u003e \u003cp\u003eWe developed an AI-based diagnostic support system using a Mask R-CNN framework trained on pathology-confirmed thyroid ultrasound images. The model performed segmentation and classification of five lesion categories. A reader study was conducted involving five expert physicians and five trainees. Trainees evaluated ultrasound images under non-assisted and AI-assisted conditions. Diagnostic performance was assessed using sensitivity, specificity, precision, and F1 score.\u003c/p\u003e\u003ch2\u003eResults:\u003c/h2\u003e \u003cp\u003eThe AI model demonstrated high segmentation accuracy for several structures, including vessels and thyroid parenchyma. However, its ability to distinguish benign from malignant tumors remained limited. Despite this, AI assistance improved trainee performance. Median precision increased from 0.53 to 0.63, and F1 score improved from 0.53 to 0.61. Specificity increased from 0.22 to 0.50, while sensitivity remained comparable. These findings indicate that AI support primarily reduced false-positive interpretations and improved diagnostic consistency among trainees.\u003c/p\u003e\u003ch2\u003eConclusions:\u003c/h2\u003e \u003cp\u003eAI-assisted ultrasound interpretation improved the diagnostic performance of trainees, particularly by enhancing precision and reducing false-positive findings. Although the AI model alone was insufficient for independent clinical use, it demonstrated potential as an educational support tool. These findings suggest that AI may contribute to more consistent and effective training in thyroid ultrasound interpretation.\u003c/p\u003e","manuscriptTitle":"Artificial Intelligence–Assisted Ultrasound Interpretation Enhances Diagnostic Performance Among Trainees in Thyroid Imaging","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-21 15:12:22","doi":"10.21203/rs.3.rs-9329400/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"3d09c747-1715-4d16-8c35-f2c259d6d07f","owner":[],"postedDate":"April 21st, 2026","published":true,"recentEditorialEvents":[{"type":"decision","content":"Rejected","date":"2026-05-05T04:48:15+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-02T18:19:43+00:00","index":46,"fulltext":""},{"type":"reviewerAgreed","content":"149365404752997072619306056147561705096","date":"2026-05-02T18:09:22+00:00","index":45,"fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-05-05T04:54:49+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-21 15:12:22","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9329400","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9329400","identity":"rs-9329400","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}