Application and Evaluation of Vision-LSTM Modeling in Diagnostic Ultrasound Imaging of TI-RADS category 4b thyroid nodules

preprint OA: closed
Full text JSON View at publisher

Abstract

Abstract Background:Accurate differentiation of TI-RADS category 4b thyroid nodules remains a critical clinical challenge, with 20-50% malignancy risk necessitating invasive biopsies. Existing diagnostic methods heavily rely on physician expertise, leading to inconsistencies in resource-limited settings.To develop and validate a Vision-LSTM model for ultrasound-based diagnosis of TI-RADS 4b nodules, integrating spatiotemporal feature analysis to mimic clinicians’ dynamic decision-making. Methods:Retrospective analysis of 401 pathologically confirmed TI-RADS 4b nodules (188 malignant, 213 benign) was performed. The Vision-LSTM model, combining LSTM layers for temporal dynamics and convolutional networks for spatial features, was trained on 7:3 split data. Performance was compared against junior (AUC=0.624) and senior physicians (AUC=0.787) using ROC analysis, Delong test, and precision-recall metrics. Results:The Vision-LSTM model significantly outperformed the junior practitioners and slightly outperformed the senior practitioners in terms of diagnostic accuracy and AUC values.The AI model was able to consistently identify complex features in ultrasound images and output consistent and accurate diagnostic results, demonstrating a high degree of accuracy and reliability. Conclusion:This study demonstrates that the Vision-LSTM model significantly improves diagnostic consistency for TI-RADS 4b nodules, offering a clinically deployable tool to reduce healthcare disparities. Future work will focus on multi-center validation and real-time integration with ultrasound systems.
Full text 74,493 characters · extracted from preprint-html · click to expand
Application and Evaluation of Vision-LSTM Modeling in Diagnostic Ultrasound Imaging of TI-RADS category 4b thyroid nodules | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Application and Evaluation of Vision-LSTM Modeling in Diagnostic Ultrasound Imaging of TI-RADS category 4b thyroid nodules Xinru Zhang, Yang Li, Meng Sun, Wei Nie, Zhe Ma This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6287509/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 13 You are reading this latest preprint version Abstract Background :Accurate differentiation of TI-RADS category 4b thyroid nodules remains a critical clinical challenge, with 20-50% malignancy risk necessitating invasive biopsies. Existing diagnostic methods heavily rely on physician expertise, leading to inconsistencies in resource-limited settings.To develop and validate a Vision-LSTM model for ultrasound-based diagnosis of TI-RADS 4b nodules, integrating spatiotemporal feature analysis to mimic clinicians’ dynamic decision-making. Methods :Retrospective analysis of 401 pathologically confirmed TI-RADS 4b nodules (188 malignant, 213 benign) was performed. The Vision-LSTM model, combining LSTM layers for temporal dynamics and convolutional networks for spatial features, was trained on 7:3 split data. Performance was compared against junior (AUC=0.624) and senior physicians (AUC=0.787) using ROC analysis, Delong test, and precision-recall metrics. Results :The Vision-LSTM model significantly outperformed the junior practitioners and slightly outperformed the senior practitioners in terms of diagnostic accuracy and AUC values.The AI model was able to consistently identify complex features in ultrasound images and output consistent and accurate diagnostic results, demonstrating a high degree of accuracy and reliability. Conclusion :This study demonstrates that the Vision-LSTM model significantly improves diagnostic consistency for TI-RADS 4b nodules, offering a clinically deployable tool to reduce healthcare disparities. Future work will focus on multi-center validation and real-time integration with ultrasound systems. TI-RADS category 4b thyroid nodules Vision-LSTM model diagnostic accuracy artificial intelligence Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Background The incidence of thyroid nodules is increasing every year, and with the continuous development of ultrasound technology, more and more thyroid nodules are being detected at an early stage. Although the majority of thyroid nodules are benign, some, especially category 4b nodules in the TI-RADS classification system, have a high malignant potential, making their clinical management particularly important. TI-RADS (Thyroid Imaging Reporting and Data System) [1] is a classification system for assessing the risk of malignancy of thyroid nodules by ultrasound imaging. TI-RADS category 4b nodules usually show some significant malignant features such as irregular borders, inhomogeneous internal echoes and microcalcifications. The risk of malignancy for these nodules is generally considered to be in the range of 20-50% [2] , and in clinical workup, TI-RADS category 4b nodules are usually recommended to undergo further fine needle aspiration biopsy (FNA) to determine the nature of the nodule [3] , yet up to 40% of these biopsies yield benign results. This diagnostic uncertainty not only burdens healthcare systems but also increases patient anxiety and procedural risks. In recent years, the application of artificial intelligence (AI), especially deep learning technology, has made breakthroughs in the field of medical imaging. AI models are able to automatically analyse ultrasound images, accurately extract nodule features and effectively classify them [4,5,6] . How to improve the diagnostic accuracy of TI-RADS class 4b nodules in clinical practice has become a critical issue to be addressed. The aim of this study is to explore the potential of AI techniques based on the Vision-LSTM model [7] in the determination of benign and malignant TI-RADS category 4b thyroid nodules. Recent advancements in AI, particularly deep learning models like Vision-LSTM, have shown remarkable potential in medical image analysis. Unlike static CNN-based approaches, the Vision-LSTM model mimics clinicians’ real-time assessment by analyzing temporal changes in ultrasound sequences (e.g., nodule mobility during swallowing), thereby capturing subtle malignant features often overlooked in single-frame analysis.This capability is particularly advantageous in analyzing ultrasound images, where subtle temporal changes in nodule characteristics can provide critical diagnostic information. With the help of the AI model, we were able to significantly improve the diagnostic sensitivity and accuracy of TI-RADS category 4b nodules in mass screening, reduce unnecessary invasive procedures and optimise the clinical decision-making process. Ultimately, we expect to improve the overall efficiency of thyroid nodule management, reduce the risk of misdiagnosis and underdiagnosis, and provide patients with more scientific and personalised treatment plans through the precise support of AI technology, thus promoting the development of early screening and precision medicine for thyroid diseases. Methods 1. Data collection In this study, imaging data of 401 TI-RADS category 4b thyroid nodules in 401 patients who attended the First Affiliated Hospital of Shandong First Medical University from January 2022 to December 2024 were collected. Ultrasound images were acquired from three ultrasound systems (GE Logiq E9, Philips EPIQ 7, and Siemens Acuson S2000) to ensure device heterogeneity. All nodules were diagnosed as TI-RADS category 4b on the basis of preoperative ultrasound images by two ultrasonographers with more than 5 years of experience in thyroid ultrasound diagnosis, and all nodules were pathologically diagnosed by FNA or surgery to ensure the accuracy of the data. The study was approved by the hospital ethics committee and the requirement of informed consent was waived. 2. Grouping and processing of images A total of 401 ultrasound images were included in this study, including 213 cases in the benign group and 188 cases in the malignant group. All images were randomly divided into training and validation groups according to 7:3. The ultrasound images extracted from the database in JPG format were cropped, while the TI-RADS class 4b thyroid nodules in the images were labeled using labellmg, and the classification labels were set up, with a label of 1 for malignant pathological results and a label of 0 for benign pathological results. 3. Selection, construction and validation of the artificial intelligence model In this study, we adopted Vision LSTM (Long Short-Term Memory Network combined with visual features) as the infrastructure of the artificial intelligence model, aiming to analyze the benignness and malignancy of thyroid nodules by automatically extracting spatio-temporal features in images. The Vision-LSTM model comprises a ResNet-50 backbone for spatial feature extraction, followed by two bidirectional LSTM layers (hidden units=256) to analyze temporal dependencies across 10-frame sequences. Model training utilized the Adam optimizer (learning rate=1e-4, weight decay=1e-5) with early stopping based on validation loss. Unlike traditional CNN models, Vision LSTM not only learns spatial features in images, but also improves the recognition of nodule change trends by introducing LSTM layers to capture temporal information in image sequences [8] . In this study, a large number of TI-RADS class 4b nodule images were used for training, and the cross-entropy loss function was chosen to measure the difference between the predicted values and the actual labels. To address the issue of data imbalance, we employed several data augmentation techniques, including rotation, affine transformation, and center cropping. These techniques not only increased the diversity of the training dataset but also enhanced the model's ability to generalize to unseen data, thereby improving its robustness and classification accuracy. 4. Image feature extraction and analysis In the pre-processing stage of image data, we first normalized and denoised the ultrasound images to ensure the quality and consistency of the input data. Subsequently, combined with the powerful spatio-temporal feature extraction capability of the Vision-LSTM model, key information such as the morphology of the nodule, echogenic features, and calcification status were automatically extracted. By learning these features at the image level, Vision-LSTM is able to capture the microstructure of the nodule and its important indicators more effectively. Based on these extracted features, the AI model is able to generate predictions about the benign and malignant nature of the nodule. Even though we are dealing with static images, Vision-LSTM, with its in-depth temporal feature modeling capability, is still able to effectively mine the complex information in the image, thus improving the classification accuracy and enhancing the reliability and accuracy of the model in nodule determination. 5. Performance evaluation In order to evaluate the performance of the Vision-LSTM model, we compared it with junior doctors and senior doctors in terms of TI-RADS Class 4b thyroid nodule identification accuracy. A total of 100 rounds of training were conducted in this study, and loss curves were used to evaluate the performance of the model during training, and the Vision-LSTM model was further evaluated by precision-recall curves (PR curves). We also demonstrated the accuracy of the model through the confusion matrix and also calculated the area under the curve (AUC) and other related metrics of the model. Results 1. Comparison of basic information A total of 401 patients involving 401 TI-RADS category 4b thyroid nodules were included in this study.The basic clinical and ultrasonographic data of the TI-RADS category 4b thyroid nodules are detailed in Table 1. Statistical analyses of the basic clinical and ultrasonographic data showed statistically significant differences in age, maximum diameter of the thyroid nodules, and sex of the patients in the study ( P < 0.05). The specific data are shown in Table 2. 2. Judgment of TI-RADS category 4b thyroid nodules by junior and senior physicians In this study, we compared the performance of junior and senior physicians in judging TI-RADS category 4b thyroid nodules. Junior doctors are usually those who have less clinical experience and are new to the diagnosis of thyroid disease (less than 5 years of experience). Senior doctors, on the other hand, usually have extensive clinical experience and have practiced for many years (>15 years), and are able to take multiple factors into account when recognizing complex lesions and interpreting images, demonstrating high judgmental accuracy and clinical acumen. The results of the analysis showed that the AUC of junior doctors was 0.624 (95% CI: 0.569-0.679) in diagnosing 401 TI-RADS category 4b thyroid nodules, indicating that their classification ability was weak, while the AUC of senior doctors was 0.787 (95% CI: 0.740-0.833), which was significantly higher than that of junior doctors, indicating that senior doctors had a higher accuracy in diagnosing this type of nodule diagnosis with higher accuracy and differentiation ability. The ROC curves for both are shown in Fig. 1. 3. Vision-LSTM model judgment of TI-RADS class 4b thyroid nodules (1) In this study, we chose Vision-LSTM model and divided the TI-RADS class 4b thyroid nodule data into training and validation groups in the ratio of 7:3 for model training and evaluation. (2) We evaluated the performance of the model during the training process by means of a loss function curve . The loss curve shows the trend of the value of the loss function with the number of training iterations (or training rounds) during the training process of the model. By observing the curve, we can visualize the training progress of the model, the learning situation, and determine whether there is a risk of overfitting or underfitting. During the model training process, we use the cross-entropy loss function to continuously adjust the parameters, aiming to reduce the prediction error. If both the training loss and validation loss show a decreasing trend, it indicates that the model is learning the data effectively and shows good generalization ability on both the training and validation sets. The loss curves are shown in Fig. 2. (3) PR Curve is an important tool for evaluating the performance of a classification model. It shows the relationship between Precision and Recall at different decision thresholds.The closer the PR curve is to the upper right corner, the better the performance of the model. The upper right corner represents high Precision and high Recall, which means that the model maintains a high level of accuracy in recognizing the positive class, but also effectively recognizes most of the positive class samples. In this study, the model has a PR of 0.97 on the training set and 0.85 on the validation set, indicating that the model performs better on the training set, while its performance on the validation set slightly decreases but still maintains a good performance. The PR curves are shown in Fig. 3 (4) Confusion Matrix is an important tool for evaluating the performance of a classification model, which demonstrates the model's performance in a classification task by presenting the model's predictions in a table form, showing the comparison between the model's predicted results and the true labels. Especially in binary classification tasks, the Confusion Matrix can visually reveal the type of classification errors of the model as well as the accuracy of the classification. In this study, the Vision-LSTM model has an accuracy of 89.3% in the training group and 89.4% in the validation group during the training of TI-RADS class 4b thyroid nodules. The detailed results of the confusion matrix are shown in Fig. 4. (5) The ROC curve is an intuitive and effective tool for evaluating binary classification models, which can show the relationship between the True Positive Rate and the False Positive Rate of the model under different thresholds.The AUC value can quantify the classification ability of the model, and the larger the AUC value, the better the performance of the model. The larger the AUC value, the better the performance of the model. In this study, the ROC curve results are shown in Fig.5. The Vision-LSTM model has an AUC value of 0.97 in the training group and 0.88 in the validation group. The Vision-LSTM model's high diagnostic accuracy and AUC value suggest its potential to serve as a valuable decision-support tool in clinical practice, particularly in resource-limited settings where experienced radiologists are scarce. By providing consistent and accurate preliminary diagnoses, the model can significantly reduce the diagnostic burden on junior physicians and improve overall diagnostic efficiency. Discussion This study reveals the basic clinical and imaging features of TI-RADS category 4b thyroid nodules and finds that factors such as the maximum diameter of the nodule, age, and gender have a significant impact in the diagnosis. The significance of these variables suggests that, although imaging features are key determinants of the risk of nodule malignancy, physicians should take into account factors such as the patient's age, gender, and the growth rate of the nodule in the clinical evaluation. Further, with the gradual deepening of big data analysis and multidimensional diagnostic information, future diagnostic models should not only rely on traditional imaging data, but should also integrate patients' genomic and molecular biology information to achieve more personalized and precise diagnosis. The difference in performance between junior and senior physicians in this study reflects the subjective and experience-dependent nature of medical image interpretation. The lower AUC of the junior physicians indicated a lack of sensitivity to the imaging features and nuances of the nodules. In contrast, the high AUC values of senior physicians indicate that they are able to capture complex information in imaging more accurately through their extensive clinical experience and comprehensive judgment. This result further validates the importance of physician experience on diagnostic outcomes and raises a profound medical education question: how to improve junior physicians' ability to interpret imaging, especially for early nodules and difficult cases, through more efficient and systematic training. This phenomenon also points to the problem of uneven medical resources: in many resource-poor areas, the lack of senior physicians may lead to insufficient accuracy in nodule diagnosis. Therefore, how to make up for the lack of imaging judgment of junior physicians and improve the efficiency of medical resources through intelligent assistive systems has become a major challenge in the current development of medicine. The introduction of artificial intelligence, especially deep learning technology, has revolutionized medical image analysis [ 9 ] .The performance of Vision-LSTM model in this study highlights the great potential of AI technology in image classification. Its high-precision classification ability not only demonstrates the advantages of AI in processing complex medical images, but also reflects the self-optimization and learning ability of deep learning models in large-scale data training. Compared with the diagnostic methods of traditional doctors, the Vision-LSTM model demonstrates a high degree of consistency and stability, especially in several indicators such as AUC value, surpassing both junior and senior physicians, showing that it can still maintain a high degree of accuracy in diverse samples and complex data. This advantage is not only reflected in the improvement of diagnostic efficiency, but also significantly reduces the workload of physicians, especially in high-load, high-stress clinical environments. However, it is worth noting that the success of AI models does not mean replacing the role of doctors, but rather as an auxiliary tool is particularly prominent.AI can quickly provide preliminary screening results, but still need to rely on the professional judgment and clinical experience of doctors to complete the final diagnostic decision. Therefore, deep collaboration between AI and clinicians will become the norm rather than a competitive relationship in future medical practice. Although the Vision-LSTM model has demonstrated excellent performance in research, its promotion in practical clinical applications still faces multiple challenges. First, the large-scale high-quality dataset required for model training is a bottleneck that cannot be ignored. Patient populations in different regions and hospitals have different clinical characteristics and imaging performances, which makes the performance of AI models in terms of generalizability and generalization ability may vary. In order to ensure the wide application of the model, it is necessary to establish a standardized and diversified dataset that covers a wider range of patient groups. Second, the “black box” problem of AI remains a major obstacle in the application of the technology [ 10 , 11 ] . Although deep learning models such as Vision-LSTM can provide numerically efficient classification results, their internal decision-making process is not transparent and lacks sufficient interpretability. This makes it potentially difficult for clinicians to understand the judgmental logic of the models in practical applications, affecting physicians' trust and acceptance of the model's predicted results. Therefore, how to improve the interpretability of AI models and enable them to be more synergistic with physicians' clinical thinking is a key direction for future research. In addition, the ethical issues of AI models cannot be ignored. How to ensure the protection of patient privacy, how to avoid bias in models and how to ensure the dominance of human medical ethics in the decision-making process are all issues that need to be discussed in depth. The application of AI cannot be separated from the framework of morality and law, and only under the premise of safeguarding patients' rights and interests can we truly promote the sustainable development of AI in the medical field. Conclusion Overall, this study demonstrates the great potential of AI in the diagnosis of thyroid nodules, especially in terms of improving diagnostic accuracy, reducing the burden on physicians and improving medical efficiency, AI technology undoubtedly brings new hope to the medical community. However, the successful application of AI does not only depend on the progress of the technology itself, but also requires the reform and improvement of the medical system. Only by establishing standardized datasets, strengthening the synergy between AI and doctors, improving the interpretability of models, and resolving ethical issues can AI play its greatest role in actual clinics. Future research should focus on validating the Vision-LSTM model in multi-center studies to assess its generalizability across diverse patient populations and imaging equipment. Additionally, efforts should be made to integrate the model into clinical workflows, ensuring its seamless adoption by healthcare providers and its ability to enhance diagnostic accuracy in real-world settings. Declarations Ethics approval and consent to participate: The study was approved by the Clinical Ethics Committee of the First Affiliated Hospital of Shandong First Medical University, and it was confirmed that all experiments were conducted in accordance with the relevant designated guidelines and regulations. The retrospective study was approved by the Ethics Committee and Institutional Review under the approval number [YXLL-KY-2023 (133))], exempted from the requirement of informed consent, and registered on ClinicalTrials.gov (NCT06258044). Our research adheres to the Declaration of Helsinki. Consent for publication: The study's research data and information were obtained with the consent of the participants. Availability of data and materials: The datasets generated and analyzed in this study are not available to the public due to the privacy of the patients' cases, but are available from the corresponding authors upon request. Competing interests: The authors declare that they have no competing interests. Funding :We thank the First Affiliated Hospital of Shandong First Medical University and the Department of Ultrasound Diagnosis and Treatment for their help. Authors' contributions: We sincerely thank the following people for their contributions to this study: Xinru Zhang : Data analysis and writing the article; Meng Sun : Data organization; Wei Nie : data collection; Yang Li : Model selection and training; Zhe Ma: Software instruction and review.All authors read and approved the final manuscript. Acknowledgements :Not applicable. References Malhi, Harshawn S, and Edward G Grant. “Ultrasound of Thyroid Nodules and the Thyroid Imaging Reporting and Data System.” Neuroimaging clinics of North America vol. 31,3 (2021): 285-300. Chen, Zhiguang et al. “Diagnostic performance of simplified TI-RADS for malignant thyroid nodules: comparison with 2017 ACR-TI-RADS and 2020 C-TI-RADS.” Cancer imaging : the official publication of the International Cancer Imaging Society vol. 22,1 41. 17 Aug. 2022. Poller, David N et al. “Thyroid FNA terminology: The case for a single unified international system for thyroid FNA reporting.” Cytopathology : official journal of the British Society for Clinical Cytology vol. 32,6 (2021): 714-717. Chen, Chen et al. “Deep learning to assist composition classification and thyroid solid nodule diagnosis: a multicenter diagnostic study.” European radiology vol. 34,4 (2024): 2323-2333. Tessler, Franklin N, and Johnson Thomas. “Artificial Intelligence for Evaluation of Thyroid Nodules: A Primer.” Thyroid : official journal of the American Thyroid Association vol. 33,2 (2023): 150-158. Peng, Yun et al. “The Application of Artificial Intelligence in Thyroid Nodules: A Systematic Review Based on Bibliometric Analysis.” Endocrine, metabolic & immune disorders drug targets vol. 24,11 (2024): 1280-1290. Alkin B, Beck M, Pöppel K, et al. Vision-LSTM: xLSTM as Generic Vision Backbone[J]. arxiv preprint arxiv:2406.04303, 2024. Donahue J, Anne Hendricks L, Guadarrama S, et al. Long-term recurrent convolutional networks for visual recognition and description[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 2625-2634. Chen, Xuxin et al. “Recent advances and clinical applications of deep learning in medical image analysis.” Medical image analysis vol. 79 (2022): 102444. Belle, Vaishak, and Ioannis Papantonis. “Principles and Practice of Explainable Machine Learning.” Frontiers in big data vol. 4 688969. 1 Jul. 2021, Borisov, Vadim et al. “Deep Neural Networks and Tabular Data: A Survey.” IEEE transactions on neural networks and learning systems vol. 35,6 (2024): 7499-7519. Tables Tab.1 Basic clinical and ultrasound information on TI-RADS category 4b thyroid nodules total cases (n=401) malignant nodule (n=188) benign nodule (n=213) gender male 197 (49.1%) 82 (43.6%) 115 (54.0%) female 204 (50.9%) 106 (56.4%) 98 (46.0%) average age (years) 50 (43, 58) 54 (43, 62) 48 (43, 56) maximum diameter (mm) 9.0 (6.5, 12.9) 7.9 (6.3, 12.1) 10.0 (7.0, 13.5) junior Doctor's Result 119 (63.3%) 131 (61.5%) senior Doctor's Result 151 (80.3%) 164 (77.0%) Tab.2 Statistical analysis of basic clinical and ultrasound data in TI-RADS category 4b thyroid nodules projects total cases (n=401) malignant nodule (n=188) benign nodule (n=213) Wilcoxon's multi-sample rank-sum test H P average age (years) 54(43, 62) 48 (43, 56) 11.18 0.001 maximum diameter (mm) 7.9(6.3, 12.1) 10.0 (7.0, 13.5) 9.18 0.002 gender male 82 (43.6%) 115 (54.0%) 4.29 0.038 female 106 (56.4%) 98 (46.0%) Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 01 Sep, 2025 Reviews received at journal 08 Jun, 2025 Reviews received at journal 14 May, 2025 Reviewers agreed at journal 11 May, 2025 Reviewers agreed at journal 09 May, 2025 Reviews received at journal 06 May, 2025 Reviewers agreed at journal 01 May, 2025 Reviewers agreed at journal 01 May, 2025 Reviewers invited by journal 30 Apr, 2025 Editor assigned by journal 25 Apr, 2025 Editor invited by journal 01 Apr, 2025 Submission checks completed at journal 31 Mar, 2025 First submitted to journal 31 Mar, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6287509","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":451714200,"identity":"025a0da8-c63b-4bce-856e-d416fa458240","order_by":0,"name":"Xinru Zhang","email":"","orcid":"","institution":"The First Affiliated Hospital of Shandong First Medical University \u0026 Shandong Provincial Qianfoshan Hospital","correspondingAuthor":false,"prefix":"","firstName":"Xinru","middleName":"","lastName":"Zhang","suffix":""},{"id":451714201,"identity":"b1566d89-3328-4ad4-a9ae-c1a50ddfa1e9","order_by":1,"name":"Yang Li","email":"","orcid":"","institution":"The First Affiliated Hospital of Shandong First Medical University \u0026 Shandong Provincial Qianfoshan Hospital","correspondingAuthor":false,"prefix":"","firstName":"Yang","middleName":"","lastName":"Li","suffix":""},{"id":451714202,"identity":"a884be9d-d9ef-4fbc-81cf-de81f5aa7431","order_by":2,"name":"Meng Sun","email":"","orcid":"","institution":"The First Affiliated Hospital of Shandong First Medical University \u0026 Shandong Provincial Qianfoshan Hospital","correspondingAuthor":false,"prefix":"","firstName":"Meng","middleName":"","lastName":"Sun","suffix":""},{"id":451714203,"identity":"98cd026c-002c-4a20-bf10-16d08f76bbe6","order_by":3,"name":"Wei Nie","email":"","orcid":"","institution":"The First Affiliated Hospital of Shandong First Medical University \u0026 Shandong Provincial Qianfoshan Hospital","correspondingAuthor":false,"prefix":"","firstName":"Wei","middleName":"","lastName":"Nie","suffix":""},{"id":451714204,"identity":"4da694ec-b128-4804-a3e2-64137e968bfc","order_by":4,"name":"Zhe Ma","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAv0lEQVRIiWNgGAWjYHACNgaGHxL1/MzMhx8Qr4WxxyJBsp0tzYAEW9gqEgzO8yhIEKVet/34swcfeCTyjA/zMBgw1NhEE9RidiYh3XCGhUSx2WHeAw8YjqXlNhDUciDhmDQPjwTjtsN8CQaMDYeJ0HL+YZv0HzYJxs3NPAYSxGm5kcwmzcAmkbiBmXgtz9gke3skjCUOAwM5gSi/nE9/JvHjR50cf//hww8+1NgQ1oIKEkhTPgpGwSgYBaMAFwAANyU8pDQRA/sAAAAASUVORK5CYII=","orcid":"","institution":"The First Affiliated Hospital of Shandong First Medical University \u0026 Shandong Provincial Qianfoshan Hospital","correspondingAuthor":true,"prefix":"","firstName":"Zhe","middleName":"","lastName":"Ma","suffix":""}],"badges":[],"createdAt":"2025-03-23 09:38:14","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6287509/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6287509/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":82273407,"identity":"8e9f0a73-d66e-45f0-bd1a-4f699ac9b7a9","added_by":"auto","created_at":"2025-05-08 14:28:50","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":107986,"visible":true,"origin":"","legend":"\u003cp\u003eROC curves for TI-RADS category 4b thyroid nodules by junior and senior physicians\u003c/p\u003e","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-6287509/v1/77c7228d6c80cab407ac6610.jpeg"},{"id":82273405,"identity":"523d881c-25e8-48a3-a979-35fb675fab9d","added_by":"auto","created_at":"2025-05-08 14:28:50","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":44516,"visible":true,"origin":"","legend":"\u003cp\u003eLoss curves for Vision-LSTM models\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-6287509/v1/bdb8e2618365b2ce87709235.png"},{"id":82273411,"identity":"d9d82ccd-418a-455f-a634-c0b89c328d15","added_by":"auto","created_at":"2025-05-08 14:28:50","extension":"jpeg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":95543,"visible":true,"origin":"","legend":"\u003cp\u003ePR curves for Vision-LSTM models\u003c/p\u003e","description":"","filename":"floatimage3.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-6287509/v1/2f758d6fd4f70e193f8294c2.jpeg"},{"id":82273409,"identity":"2cc12846-337e-4f81-b0b2-83a29a5ed9e1","added_by":"auto","created_at":"2025-05-08 14:28:50","extension":"jpeg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":76049,"visible":true,"origin":"","legend":"\u003cp\u003eConfusion Matrix for Vision-LSTM Models\u003c/p\u003e","description":"","filename":"floatimage4.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-6287509/v1/4162c0436792c47d627c3fac.jpeg"},{"id":82275879,"identity":"cf8250be-0f72-4532-95a8-e682b548d543","added_by":"auto","created_at":"2025-05-08 14:44:50","extension":"jpeg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":173404,"visible":true,"origin":"","legend":"\u003cp\u003eROC curves for Vision-LSTM models\u003c/p\u003e","description":"","filename":"floatimage5.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-6287509/v1/1ba363dbd3cd29887c9e7bd5.jpeg"},{"id":82275880,"identity":"b9bf13d0-0269-4470-ade5-3eaace1b701b","added_by":"auto","created_at":"2025-05-08 14:44:55","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1061614,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6287509/v1/48db0da5-70ee-4096-99b3-49e8c230b353.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Application and Evaluation of Vision-LSTM Modeling in Diagnostic Ultrasound Imaging of TI-RADS category 4b thyroid nodules","fulltext":[{"header":"Background","content":"\u003cp\u003eThe incidence of thyroid nodules is increasing every year, and with the continuous development of ultrasound technology, more and more thyroid nodules are being detected at an early stage. Although the majority of thyroid nodules are benign, some, especially category 4b nodules in the TI-RADS classification system, have a high malignant potential, making their clinical management particularly important.\u003c/p\u003e\n\u003cp\u003eTI-RADS (Thyroid Imaging Reporting and Data System)\u003csup\u003e\u0026nbsp;[1]\u0026nbsp;\u003c/sup\u003eis a classification system for assessing the risk of malignancy of thyroid nodules by ultrasound imaging. TI-RADS category 4b nodules usually show some significant malignant features such as irregular borders, inhomogeneous internal echoes and microcalcifications. The risk of malignancy for these nodules is generally considered to be in the range of 20-50% \u003csup\u003e[2]\u003c/sup\u003e, and in clinical workup, TI-RADS category 4b nodules are usually recommended to undergo further fine needle aspiration biopsy (FNA) to determine the nature of the nodule \u003csup\u003e[3]\u003c/sup\u003e, yet up to 40% of these biopsies yield benign results. This diagnostic uncertainty not only burdens healthcare systems but also increases patient anxiety and procedural risks.\u003c/p\u003e\n\u003cp\u003eIn recent years, the application of artificial intelligence (AI), especially deep learning technology, has made breakthroughs in the field of medical imaging. AI models are able to automatically analyse ultrasound images, accurately extract nodule features and effectively classify them\u003csup\u003e\u0026nbsp;[4,5,6]\u003c/sup\u003e. How to improve the diagnostic accuracy of TI-RADS class 4b nodules in clinical practice has become a critical issue to be addressed. The aim of this study is to explore the potential of AI techniques based on the Vision-LSTM model\u003csup\u003e\u0026nbsp;[7]\u0026nbsp;\u003c/sup\u003ein the determination of benign and malignant TI-RADS category 4b thyroid nodules. Recent advancements in AI, particularly deep learning models like Vision-LSTM, have shown remarkable potential in medical image analysis. Unlike static CNN-based approaches, the Vision-LSTM model mimics clinicians\u0026rsquo; real-time assessment by analyzing temporal changes in ultrasound sequences (e.g., nodule mobility during swallowing), thereby capturing subtle malignant features often overlooked in single-frame analysis.This capability is particularly advantageous in analyzing ultrasound images, where subtle temporal changes in nodule characteristics can provide critical diagnostic information. With the help of the AI model, we were able to significantly improve the diagnostic sensitivity and accuracy of TI-RADS category 4b nodules in mass screening, reduce unnecessary invasive procedures and optimise the clinical decision-making process. Ultimately, we expect to improve the overall efficiency of thyroid nodule management, reduce the risk of misdiagnosis and underdiagnosis, and provide patients with more scientific and personalised treatment plans through the precise support of AI technology, thus promoting the development of early screening and precision medicine for thyroid diseases.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003e\u003cstrong\u003e1. Data collection\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn this study, imaging data of 401 TI-RADS category 4b thyroid nodules in 401 patients who attended the First Affiliated Hospital of Shandong First Medical University from January 2022 to December 2024 were collected. Ultrasound images were acquired from three ultrasound systems (GE Logiq E9, Philips EPIQ 7, and Siemens Acuson S2000) to ensure device heterogeneity. All nodules were diagnosed as TI-RADS category 4b on the basis of preoperative ultrasound images by two ultrasonographers with more than 5 years of experience in thyroid ultrasound diagnosis, and all nodules were pathologically diagnosed by FNA or surgery to ensure the accuracy of the data. The study was approved by the hospital ethics committee and the requirement of informed consent was waived.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2. Grouping and processing of images\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA total of 401 ultrasound images were included in this study, including 213 cases in the benign group and 188 cases in the malignant group. All images were randomly divided into training and validation groups according to 7:3. The ultrasound images extracted from the database in JPG format were cropped, while the TI-RADS class 4b thyroid nodules in the images were labeled using labellmg, and the classification labels were set up, with a label of 1 for malignant pathological results and a label of 0 for benign pathological results.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3. Selection, construction and validation of the artificial intelligence model\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn this study, we adopted Vision LSTM (Long Short-Term Memory Network combined with visual features) as the infrastructure of the artificial intelligence model, aiming to analyze the benignness and malignancy of thyroid nodules by automatically extracting spatio-temporal features in images. The Vision-LSTM model comprises a ResNet-50 backbone for spatial feature extraction, followed by two bidirectional LSTM layers (hidden units=256) to analyze temporal dependencies across 10-frame sequences. Model training utilized the Adam optimizer (learning rate=1e-4, weight decay=1e-5) with early stopping based on validation loss. Unlike traditional CNN models, Vision LSTM not only learns spatial features in images, but also improves the recognition of nodule change trends by introducing LSTM layers to capture temporal information in image sequences \u003csup\u003e[8]\u003c/sup\u003e. In this study, a large number of TI-RADS class 4b nodule images were used for training, and the cross-entropy loss function was chosen to measure the difference between the predicted values and the actual labels. To address the issue of data imbalance, we employed several data augmentation techniques, including rotation, affine transformation, and center cropping. These techniques not only increased the diversity of the training dataset but also enhanced the model\u0026apos;s ability to generalize to unseen data, thereby improving its robustness and classification accuracy.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e4. Image feature extraction and analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn the pre-processing stage of image data, we first normalized and denoised the ultrasound images to ensure the quality and consistency of the input data. Subsequently, combined with the powerful spatio-temporal feature extraction capability of the Vision-LSTM model, key information such as the morphology of the nodule, echogenic features, and calcification status were automatically extracted. By learning these features at the image level, Vision-LSTM is able to capture the microstructure of the nodule and its important indicators more effectively. Based on these extracted features, the AI model is able to generate predictions about the benign and malignant nature of the nodule. Even though we are dealing with static images, Vision-LSTM, with its in-depth temporal feature modeling capability, is still able to effectively mine the complex information in the image, thus improving the classification accuracy and enhancing the reliability and accuracy of the model in nodule determination.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e5. Performance evaluation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn order to evaluate the performance of the Vision-LSTM model, we compared it with junior doctors and senior doctors in terms of TI-RADS Class 4b thyroid nodule identification accuracy. A total of 100 rounds of training were conducted in this study, and loss curves were used to evaluate the performance of the model during training, and the Vision-LSTM model was further evaluated by precision-recall curves (PR curves). We also demonstrated the accuracy of the model through the confusion matrix and also calculated the area under the curve (AUC) and other related metrics of the model.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cstrong\u003e1. Comparison of basic information\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA total of 401 patients involving 401 TI-RADS category 4b thyroid nodules were included in this study.The basic clinical and ultrasonographic data of the TI-RADS category 4b thyroid nodules are detailed in Table 1. Statistical analyses of the basic clinical and ultrasonographic data showed statistically significant differences in age, maximum diameter of the thyroid nodules, and sex of the patients in the study (\u003cem\u003eP\u003c/em\u003e \u0026lt; 0.05). The specific data are shown in Table 2.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2. Judgment of TI-RADS category 4b thyroid nodules by junior and senior physicians\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn this study, we compared the performance of junior and senior physicians in judging TI-RADS category 4b thyroid nodules. Junior doctors are usually those who have less clinical experience and are new to the diagnosis of thyroid disease (less than 5 years of experience). Senior doctors, on the other hand, usually have extensive clinical experience and have practiced for many years (\u0026gt;15 years), and are able to take multiple factors into account when recognizing complex lesions and interpreting images, demonstrating high judgmental accuracy and clinical acumen. The results of the analysis showed that the AUC of junior doctors was 0.624 (95% CI: 0.569-0.679) in diagnosing 401 TI-RADS category 4b thyroid nodules, indicating that their classification ability was weak, while the AUC of senior doctors was 0.787 (95% CI: 0.740-0.833), which was significantly higher than that of junior doctors, indicating that senior doctors had a higher accuracy in diagnosing this type of nodule diagnosis with higher accuracy and differentiation ability. The ROC curves for both are shown in Fig. 1.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3. Vision-LSTM model judgment of TI-RADS class 4b thyroid nodules\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e(1) In this study, we chose Vision-LSTM model and divided the TI-RADS class 4b thyroid nodule data into training and validation groups in the ratio of 7:3 for model training and evaluation.\u003c/p\u003e\n\u003cp\u003e(2) We evaluated the performance of the model during the training process by means of a loss function curve . The loss curve shows the trend of the value of the loss function with the number of training iterations (or training rounds) during the training process of the model. By observing the curve, we can visualize the training progress of the model, the learning situation, and determine whether there is a risk of overfitting or underfitting. During the model training process, we use the cross-entropy loss function to continuously adjust the parameters, aiming to reduce the prediction error. If both the training loss and validation loss show a decreasing trend, it indicates that the model is learning the data effectively and shows good generalization ability on both the training and validation sets. The loss curves are shown in Fig. 2.\u003c/p\u003e\n\u003cp\u003e(3) PR Curve is an important tool for evaluating the performance of a classification model. It shows the relationship between Precision and Recall at different decision thresholds.The closer the PR curve is to the upper right corner, the better the performance of the model. The upper right corner represents high Precision and high Recall, which means that the model maintains a high level of accuracy in recognizing the positive class, but also effectively recognizes most of the positive class samples. In this study, the model has a PR of 0.97 on the training set and 0.85 on the validation set, indicating that the model performs better on the training set, while its performance on the validation set slightly decreases but still maintains a good performance. The PR curves are shown in Fig. 3\u003c/p\u003e\n\u003cp\u003e(4) Confusion Matrix is an important tool for evaluating the performance of a classification model, which demonstrates the model\u0026apos;s performance in a classification task by presenting the model\u0026apos;s predictions in a table form, showing the comparison between the model\u0026apos;s predicted results and the true labels. Especially in binary classification tasks, the Confusion Matrix can visually reveal the type of classification errors of the model as well as the accuracy of the classification. In this study, the Vision-LSTM model has an accuracy of 89.3% in the training group and 89.4% in the validation group during the training of TI-RADS class 4b thyroid nodules. The detailed results of the confusion matrix are shown in Fig. 4.\u003c/p\u003e\n\u003cp\u003e(5) The ROC curve is an intuitive and effective tool for evaluating binary classification models, which can show the relationship between the True Positive Rate and the False Positive Rate of the model under different thresholds.The AUC value can quantify the classification ability of the model, and the larger the AUC value, the better the performance of the model. The larger the AUC value, the better the performance of the model. In this study, the ROC curve results are shown in Fig.5. The Vision-LSTM model has an AUC value of 0.97 in the training group and 0.88 in the validation group. The Vision-LSTM model\u0026apos;s high diagnostic accuracy and AUC value suggest its potential to serve as a valuable decision-support tool in clinical practice, particularly in resource-limited settings where experienced radiologists are scarce. By providing consistent and accurate preliminary diagnoses, the model can significantly reduce the diagnostic burden on junior physicians and improve overall diagnostic efficiency.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis study reveals the basic clinical and imaging features of TI-RADS category 4b thyroid nodules and finds that factors such as the maximum diameter of the nodule, age, and gender have a significant impact in the diagnosis. The significance of these variables suggests that, although imaging features are key determinants of the risk of nodule malignancy, physicians should take into account factors such as the patient's age, gender, and the growth rate of the nodule in the clinical evaluation. Further, with the gradual deepening of big data analysis and multidimensional diagnostic information, future diagnostic models should not only rely on traditional imaging data, but should also integrate patients' genomic and molecular biology information to achieve more personalized and precise diagnosis.\u003c/p\u003e \u003cp\u003eThe difference in performance between junior and senior physicians in this study reflects the subjective and experience-dependent nature of medical image interpretation. The lower AUC of the junior physicians indicated a lack of sensitivity to the imaging features and nuances of the nodules. In contrast, the high AUC values of senior physicians indicate that they are able to capture complex information in imaging more accurately through their extensive clinical experience and comprehensive judgment. This result further validates the importance of physician experience on diagnostic outcomes and raises a profound medical education question: how to improve junior physicians' ability to interpret imaging, especially for early nodules and difficult cases, through more efficient and systematic training. This phenomenon also points to the problem of uneven medical resources: in many resource-poor areas, the lack of senior physicians may lead to insufficient accuracy in nodule diagnosis. Therefore, how to make up for the lack of imaging judgment of junior physicians and improve the efficiency of medical resources through intelligent assistive systems has become a major challenge in the current development of medicine.\u003c/p\u003e \u003cp\u003eThe introduction of artificial intelligence, especially deep learning technology, has revolutionized medical image analysis \u003csup\u003e[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]\u003c/sup\u003e.The performance of Vision-LSTM model in this study highlights the great potential of AI technology in image classification. Its high-precision classification ability not only demonstrates the advantages of AI in processing complex medical images, but also reflects the self-optimization and learning ability of deep learning models in large-scale data training. Compared with the diagnostic methods of traditional doctors, the Vision-LSTM model demonstrates a high degree of consistency and stability, especially in several indicators such as AUC value, surpassing both junior and senior physicians, showing that it can still maintain a high degree of accuracy in diverse samples and complex data. This advantage is not only reflected in the improvement of diagnostic efficiency, but also significantly reduces the workload of physicians, especially in high-load, high-stress clinical environments. However, it is worth noting that the success of AI models does not mean replacing the role of doctors, but rather as an auxiliary tool is particularly prominent.AI can quickly provide preliminary screening results, but still need to rely on the professional judgment and clinical experience of doctors to complete the final diagnostic decision. Therefore, deep collaboration between AI and clinicians will become the norm rather than a competitive relationship in future medical practice.\u003c/p\u003e \u003cp\u003eAlthough the Vision-LSTM model has demonstrated excellent performance in research, its promotion in practical clinical applications still faces multiple challenges. First, the large-scale high-quality dataset required for model training is a bottleneck that cannot be ignored. Patient populations in different regions and hospitals have different clinical characteristics and imaging performances, which makes the performance of AI models in terms of generalizability and generalization ability may vary. In order to ensure the wide application of the model, it is necessary to establish a standardized and diversified dataset that covers a wider range of patient groups. Second, the \u0026ldquo;black box\u0026rdquo; problem of AI remains a major obstacle in the application of the technology \u003csup\u003e[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]\u003c/sup\u003e. Although deep learning models such as Vision-LSTM can provide numerically efficient classification results, their internal decision-making process is not transparent and lacks sufficient interpretability. This makes it potentially difficult for clinicians to understand the judgmental logic of the models in practical applications, affecting physicians' trust and acceptance of the model's predicted results. Therefore, how to improve the interpretability of AI models and enable them to be more synergistic with physicians' clinical thinking is a key direction for future research. In addition, the ethical issues of AI models cannot be ignored. How to ensure the protection of patient privacy, how to avoid bias in models and how to ensure the dominance of human medical ethics in the decision-making process are all issues that need to be discussed in depth. The application of AI cannot be separated from the framework of morality and law, and only under the premise of safeguarding patients' rights and interests can we truly promote the sustainable development of AI in the medical field.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eOverall, this study demonstrates the great potential of AI in the diagnosis of thyroid nodules, especially in terms of improving diagnostic accuracy, reducing the burden on physicians and improving medical efficiency, AI technology undoubtedly brings new hope to the medical community. However, the successful application of AI does not only depend on the progress of the technology itself, but also requires the reform and improvement of the medical system. Only by establishing standardized datasets, strengthening the synergy between AI and doctors, improving the interpretability of models, and resolving ethical issues can AI play its greatest role in actual clinics. Future research should focus on validating the Vision-LSTM model in multi-center studies to assess its generalizability across diverse patient populations and imaging equipment. Additionally, efforts should be made to integrate the model into clinical workflows, ensuring its seamless adoption by healthcare providers and its ability to enhance diagnostic accuracy in real-world settings.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate:\u003c/strong\u003eThe study was approved by the Clinical Ethics Committee of the First Affiliated Hospital of Shandong First Medical University, and it was confirmed that all experiments were conducted in accordance with the relevant designated guidelines and regulations. The retrospective study was approved by the Ethics Committee and Institutional Review under the approval number [YXLL-KY-2023 (133))], exempted from the requirement of informed consent, and registered on ClinicalTrials.gov (NCT06258044). Our research adheres to the Declaration of Helsinki.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication:\u003c/strong\u003eThe study\u0026apos;s research data and information were obtained with the consent of the participants.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials:\u003c/strong\u003eThe datasets generated and analyzed in this study are not available to the public due to the privacy of the patients\u0026apos; cases, but are available from the corresponding authors upon request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests:\u003c/strong\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e:We thank the First Affiliated Hospital of Shandong First Medical University and the Department of Ultrasound Diagnosis and Treatment for their help.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos; contributions:\u003c/strong\u003eWe sincerely thank the following people for their contributions to this study: Xinru Zhang : Data analysis and writing the article; Meng Sun : Data organization; Wei Nie : data collection; Yang Li : Model selection and training; Zhe Ma: Software instruction and review.All authors read and approved the final manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e:Not applicable.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eMalhi, Harshawn S, and Edward G Grant. \u0026ldquo;Ultrasound of Thyroid Nodules and the Thyroid Imaging Reporting and Data System.\u0026rdquo; Neuroimaging clinics of North America vol. 31,3 (2021): 285-300.\u003c/li\u003e\n \u003cli\u003eChen, Zhiguang et al. \u0026ldquo;Diagnostic performance of simplified TI-RADS for malignant thyroid nodules: comparison with 2017 ACR-TI-RADS and 2020 C-TI-RADS.\u0026rdquo; Cancer imaging : the official publication of the International Cancer Imaging Society vol. 22,1 41. 17 Aug. 2022.\u003c/li\u003e\n \u003cli\u003ePoller, David N et al. \u0026ldquo;Thyroid FNA terminology: The case for a single unified international system for thyroid FNA reporting.\u0026rdquo; Cytopathology : official journal of the British Society for Clinical Cytology vol. 32,6 (2021): 714-717.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eChen, Chen et al. \u0026ldquo;Deep learning to assist composition classification and thyroid solid nodule diagnosis: a multicenter diagnostic study.\u0026rdquo; European radiology vol. 34,4 (2024): 2323-2333.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eTessler, Franklin N, and Johnson Thomas. \u0026ldquo;Artificial Intelligence for Evaluation of Thyroid Nodules: A Primer.\u0026rdquo; Thyroid : official journal of the American Thyroid Association vol. 33,2 (2023): 150-158.\u003c/li\u003e\n \u003cli\u003ePeng, Yun et al. \u0026ldquo;The Application of Artificial Intelligence in Thyroid Nodules: A Systematic Review Based on Bibliometric Analysis.\u0026rdquo; Endocrine, metabolic \u0026amp; immune disorders drug targets vol. 24,11 (2024): 1280-1290.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eAlkin B, Beck M, P\u0026ouml;ppel K, et al. Vision-LSTM: xLSTM as Generic Vision Backbone[J]. arxiv preprint arxiv:2406.04303, 2024.\u003c/li\u003e\n \u003cli\u003eDonahue J, Anne Hendricks L, Guadarrama S, et al. Long-term recurrent convolutional networks for visual recognition and description[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 2625-2634.\u003c/li\u003e\n \u003cli\u003eChen, Xuxin et al. \u0026ldquo;Recent advances and clinical applications of deep learning in medical image analysis.\u0026rdquo; Medical image analysis vol. 79 (2022): 102444.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eBelle, Vaishak, and Ioannis Papantonis. \u0026ldquo;Principles and Practice of Explainable Machine Learning.\u0026rdquo; Frontiers in big data vol. 4 688969. 1 Jul. 2021,\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eBorisov, Vadim et al. \u0026ldquo;Deep Neural Networks and Tabular Data: A Survey.\u0026rdquo; IEEE transactions on neural networks and learning systems vol. 35,6 (2024): 7499-7519.\u0026nbsp;\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e"},{"header":"Tables","content":"\u003cp\u003eTab.1 Basic clinical and ultrasound information on TI-RADS category 4b thyroid nodules\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" align=\"\" width=\"573\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003etotal cases\u003c/p\u003e\n \u003cp\u003e(n=401)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003emalignant nodule\u003c/p\u003e\n \u003cp\u003e(n=188)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003ebenign nodule\u003c/p\u003e\n \u003cp\u003e(n=213)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003egender\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"3\" style=\"width: 430px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003emale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e197 (49.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e82 (43.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e115 (54.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003efemale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e204 (50.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e106 (56.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e98 (46.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003eaverage age\u003c/p\u003e\n \u003cp\u003e(years)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e50 (43, 58)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e54 (43, 62)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e48 (43, 56)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003emaximum diameter\u003c/p\u003e\n \u003cp\u003e(mm)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e9.0 (6.5, 12.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e7.9 (6.3, 12.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e10.0 (7.0, 13.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003ejunior Doctor\u0026apos;s Result\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e119 (63.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e131 (61.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003esenior Doctor\u0026apos;s Result\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e151 (80.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 143px;\"\u003e\n \u003cp\u003e164 (77.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Tab.2 Statistical analysis of basic clinical and ultrasound data in TI-RADS category 4b thyroid nodules\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"581\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\" style=\"width: 99px;\"\u003e\n \u003cp\u003eprojects\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"4\" style=\"width: 483px;\"\u003e\n \u003cp\u003etotal cases\u003c/p\u003e\n \u003cp\u003e(n=401)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"2\" style=\"width: 157px;\"\u003e\n \u003cp\u003emalignant nodule\u003c/p\u003e\n \u003cp\u003e(n=188)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"2\" style=\"width: 155px;\"\u003e\n \u003cp\u003ebenign nodule\u003c/p\u003e\n \u003cp\u003e(n=213)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 171px;\"\u003e\n \u003cp\u003eWilcoxon\u0026apos;s multi-sample rank-sum test\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 67px;\"\u003e\n \u003cp\u003e\u003cem\u003eH\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 104px;\"\u003e\n \u003cp\u003e\u003cem\u003eP\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 99px;\"\u003e\n \u003cp\u003eaverage age\u003c/p\u003e\n \u003cp\u003e(years)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 157px;\"\u003e\n \u003cp\u003e54(43, 62)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 155px;\"\u003e\n \u003cp\u003e48 (43, 56)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 67px;\"\u003e\n \u003cp\u003e11.18\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 104px;\"\u003e\n \u003cp\u003e0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 99px;\"\u003e\n \u003cp\u003emaximum diameter\u003c/p\u003e\n \u003cp\u003e(mm)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 157px;\"\u003e\n \u003cp\u003e7.9(6.3, 12.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 155px;\"\u003e\n \u003cp\u003e10.0 (7.0, 13.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 67px;\"\u003e\n \u003cp\u003e9.18\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 104px;\"\u003e\n \u003cp\u003e0.002\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"5\" style=\"width: 581px;\"\u003e\n \u003cp\u003egender\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 99px;\"\u003e\n \u003cp\u003emale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 157px;\"\u003e\n \u003cp\u003e82 (43.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 155px;\"\u003e\n \u003cp\u003e115 (54.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"2\" style=\"width: 67px;\"\u003e\n \u003cp\u003e4.29\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"2\" style=\"width: 104px;\"\u003e\n \u003cp\u003e0.038\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 99px;\"\u003e\n \u003cp\u003efemale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 157px;\"\u003e\n \u003cp\u003e106 (56.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 155px;\"\u003e\n \u003cp\u003e98 (46.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"TI-RADS category 4b thyroid nodules, Vision-LSTM model, diagnostic accuracy, artificial intelligence","lastPublishedDoi":"10.21203/rs.3.rs-6287509/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6287509/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground\u003c/strong\u003e:Accurate differentiation of TI-RADS category 4b thyroid nodules remains a critical clinical challenge, with 20-50% malignancy risk necessitating invasive biopsies. Existing diagnostic methods heavily rely on physician expertise, leading to inconsistencies in resource-limited settings.To develop and validate a Vision-LSTM model for ultrasound-based diagnosis of TI-RADS 4b nodules, integrating spatiotemporal feature analysis to mimic clinicians’ dynamic decision-making.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e:Retrospective analysis of 401 pathologically confirmed TI-RADS 4b nodules (188 malignant, 213 benign) was performed. The Vision-LSTM model, combining LSTM layers for temporal dynamics and convolutional networks for spatial features, was trained on 7:3 split data. Performance was compared against junior (AUC=0.624) and senior physicians (AUC=0.787) using ROC analysis, Delong test, and precision-recall metrics.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e:The Vision-LSTM model significantly outperformed the junior practitioners and slightly outperformed the senior practitioners in terms of diagnostic accuracy and AUC values.The AI model was able to consistently identify complex features in ultrasound images and output consistent and accurate diagnostic results, demonstrating a high degree of accuracy and reliability.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusion\u003c/strong\u003e:This study demonstrates that the Vision-LSTM model significantly improves diagnostic consistency for TI-RADS 4b nodules, offering a clinically deployable tool to reduce healthcare disparities. Future work will focus on multi-center validation and real-time integration with ultrasound systems.\u003c/p\u003e","manuscriptTitle":"Application and Evaluation of Vision-LSTM Modeling in Diagnostic Ultrasound Imaging of TI-RADS category 4b thyroid nodules","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-05-08 14:28:45","doi":"10.21203/rs.3.rs-6287509/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-09-01T15:17:40+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-06-08T16:03:56+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-05-14T04:44:24+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"246809914589896758288319800500941150060","date":"2025-05-11T14:56:40+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"118141364099904689796083681826835162721","date":"2025-05-09T15:13:31+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-05-06T18:40:41+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"120404124463081768260700223499091494793","date":"2025-05-01T20:13:42+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"55940902235364567898834457040495518297","date":"2025-05-01T10:13:31+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-04-30T13:38:01+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-04-25T09:01:29+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-04-01T09:46:43+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-04-01T03:20:07+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Informatics and Decision Making","date":"2025-04-01T03:19:02+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"efbc25c0-774a-4683-9391-d8753172d2d2","owner":[],"postedDate":"May 8th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-04-30T14:09:02+00:00","versionOfRecord":[],"versionCreatedAt":"2025-05-08 14:28:45","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6287509","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6287509","identity":"rs-6287509","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}

Text is read by the "Ask this paper" AI Q&A widget below. Extraction quality varies by source — PMC NXML preserves structure cleanly, OA-HTML may include some navigation residue, and OA-PDF can have broken hyphenation. The publisher copy (via DOI) is the canonical version.

My notes (saved in your browser only)

Ask this paper AI returns verbatim quotes from the full text · source: preprint-html

Answers must be backed by verbatim quotes from this paper's full text. Hallucinated quotes are dropped automatically; if no verbatim passage answers the question, we say so. How this works

Citation neighborhood (no data yet)

We don't have any in-corpus citations linked to this paper yet. This is a recent paper (2025) — citers typically take a year or two to land, and the OpenAlex reference graph may still be filling in.

Source provenance

europepmc
last seen: 2026-05-20T01:45:00.602351+00:00