Development and validation of a multimodal interpretable machine learning model with SHAP for malignancy risk prediction in Bethesda III thyroid nodules: a dual-center retrospective cohort study

Development and validation of a multimodal interpretable machine learning model with SHAP for malignancy risk prediction in Bethesda III thyroid nodules: a dual-center retrospective cohort study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Development and validation of a multimodal interpretable machine learning model with SHAP for malignancy risk prediction in Bethesda III thyroid nodules: a dual-center retrospective cohort study Xinyu Gao, Kaiyi Yang, Xuan Chen, Huiting Chen, Mei Song, Jiawei Zhuo, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9258955/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract Background To develop a multimodal interpretable machine learning model integrating clinical, ultrasonographic, and BRAF V600E mutation data for malignancy risk prediction in Bethesda III thyroid nodules, explore the impact of noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP) reclassification on model robustness, and establish a risk stratification strategy for BRAF V600E -negative subgroups. Methods This dual-center retrospective cohort study enrolled 490 patients with Bethesda III nodules. Features were selected through univariate analysis, collinearity diagnosis, LASSO regression, and multivariable logistic regression. Six predictive algorithms (LR, RF, SVM, ExtraTrees, MLP, and XGBoost) and four hierarchical models (clinical, ultrasound, clinical-ultrasound fusion, and clinical-ultrasound-BRAF V600E combined) were constructed. Model performance was evaluated using the AUC, calibration curves, decision curve analysis, and SHapley Additive exPlanations (SHAP) interpretability analysis. Sensitivity analysis was conducted following NIFTP exclusion, and a multi-threshold risk stratification strategy was developed for BRAF V600E -negative subgroups. Results Thyroglobulin, BRAF V600E mutation, taller-than-wide shape, calcification, and CDFI were identified as independent risk factors. The multimodal fusion model achieved AUC of 0.959 (95%CI:0.939–0.977), 0.895 (95%CI: 0.831–0.950), and 0.886 (95%CI:0.804–0.947) in the training, internal validation, and external validation sets, respectively. The XGBoost model demonstrated optimal generalization performance in the external validation set (AUC = 0.890, 95%CI: 0.798–0.954). SHAP analysis confirmed BRAF V600E mutation as the predominant contributor. Following NIFTP exclusion, core predictors remained stable with robust overall performance, and XGBoost model maintained optimal performance (AUC = 0.884). For the BRAF V600E -negative subgroup (n = 281), four-tier stratification using thresholds of 0.75 demonstrated graded malignancy rates, with the low-risk tier achieving a negative predictive value of 92.4% (95% CI:0.867–0.961) and the high-risk tier achieving a positive predictive value of 100% (95%CI: 0.692–1.000). Conclusions The multimodal interpretable machine learning model significantly enhanced malignancy risk prediction accuracy for Bethesda III nodules. The XGBoost model combines superior cross-center generalization performance with interpretability. The multi-threshold risk stratification strategy for BRAF V600E -negative subgroups provides a quantifiable, actionable pathway for clinical decision-making. Thyroid nodule Multimodal fusion BRAFV600E Bethesda Category III Noninvasive follicular thyroid neoplasm with papillary-like nuclear features Machine learning Full Text Additional Declarations No competing interests reported. Supplementary Files file.docx Additional Files Cite Share Download PDF Status: Under Review Version 1 posted Reviewers invited by journal 30 Apr, 2026 Editor invited by journal 09 Apr, 2026 Editor assigned by journal 04 Apr, 2026 Submission checks completed at journal 04 Apr, 2026 First submitted to journal 29 Mar, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9258955","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":636261121,"identity":"0419e694-3a5d-41e9-91b8-35d1d131dd50","order_by":0,"name":"Xinyu Gao","email":"","orcid":"","institution":"The First Affiliated Hospital of Fujian Medical University","correspondingAuthor":false,"prefix":"","firstName":"Xinyu","middleName":"","lastName":"Gao","suffix":""},{"id":636261122,"identity":"45cf9fc6-d979-469b-a450-bebbe5c46ed5","order_by":1,"name":"Kaiyi Yang","email":"","orcid":"","institution":"The First Affiliated Hospital of Fujian Medical University","correspondingAuthor":false,"prefix":"","firstName":"Kaiyi","middleName":"","lastName":"Yang","suffix":""},{"id":636261123,"identity":"5781458d-68c6-4388-92f9-3d46d3338090","order_by":2,"name":"Xuan Chen","email":"","orcid":"","institution":"The First Affiliated Hospital of Fujian Medical University","correspondingAuthor":false,"prefix":"","firstName":"Xuan","middleName":"","lastName":"Chen","suffix":""},{"id":636261125,"identity":"51bc3468-368e-446f-822c-f414a5dec764","order_by":3,"name":"Huiting Chen","email":"","orcid":"","institution":"The First Affiliated Hospital of Fujian Medical University","correspondingAuthor":false,"prefix":"","firstName":"Huiting","middleName":"","lastName":"Chen","suffix":""},{"id":636261128,"identity":"4ef6c0ee-3aa5-4957-8787-3c1891e87abf","order_by":4,"name":"Mei Song","email":"","orcid":"","institution":"The First Affiliated Hospital of Fujian Medical University","correspondingAuthor":false,"prefix":"","firstName":"Mei","middleName":"","lastName":"Song","suffix":""},{"id":636261129,"identity":"cc9e1fbe-00e1-4c45-93dd-1dbecd9ad94a","order_by":5,"name":"Jiawei Zhuo","email":"","orcid":"","institution":"The First Affiliated Hospital of Fujian Medical University","correspondingAuthor":false,"prefix":"","firstName":"Jiawei","middleName":"","lastName":"Zhuo","suffix":""},{"id":636261134,"identity":"57f3aed6-a319-4839-9a97-98e3d65ca54a","order_by":6,"name":"Jinshu Zeng","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABEElEQVRIie2RsUoDQRCGd1m4avWwGxFz+gYDC2eKgzyIzRzCVifkAVIELK5RbH2Mg0DAThiITdD2Cou1EYsUF2wsAmbPVu5MKbhfsdX/sTP/CBEI/FUAhUARrx1hNoiVYrejIgy6sTWHZWRxp598Kj1wDefVsz6BvmRS3vD7cPwyOgOyglDlM9ben2TnXYq8frJDwLf8/o4WfpfIzHnvwYmFvZx2KAqKFAGZsM5LJNTHc94nlFPuVKJk9a2MsL6IgBDk7Er7t0fRoI3ziqxq2yp4WqlfFNBF6kv2RS1Xyg9GBtiXTD27JOXSfMDGD/ZYyNfPzdcgvmV2zSTrVNp1jn6egbrjLWrd9AcCgUDgv7MFWWlbFA7xWGEAAAAASUVORK5CYII=","orcid":"","institution":"The First Affiliated Hospital of Fujian Medical University","correspondingAuthor":true,"prefix":"","firstName":"Jinshu","middleName":"","lastName":"Zeng","suffix":""},{"id":636261135,"identity":"77d47aa0-0de4-4289-85ac-db8d2629a7f2","order_by":7,"name":"Zibin Gao","email":"","orcid":"","institution":"The Second Affiliated Hospital of Fujian University of Traditional Chinese Medicine","correspondingAuthor":false,"prefix":"","firstName":"Zibin","middleName":"","lastName":"Gao","suffix":""}],"badges":[],"createdAt":"2026-03-29 13:08:19","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9258955/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9258955/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":108976882,"identity":"622fa07e-e080-4d52-84a8-de2c1b3b5cc2","added_by":"auto","created_at":"2026-05-11 11:29:21","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":952811,"visible":true,"origin":"","legend":"","description":"","filename":"DevelopmentandvalidationofamultimodalinterpretablemachinelearningmodelwithSHAPformalignancyriskpredictioninBethesdaIIIthyroidnodulesadualcenterretrospectivecohortstudy.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9258955/v1_covered_02d8ce7d-cd49-4ff1-b330-bfb2110d1db5.pdf"},{"id":108833826,"identity":"cad505ef-8f9c-44aa-bdb5-ea3e6a643a2d","added_by":"auto","created_at":"2026-05-08 21:20:01","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":8333429,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eAdditional Files\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"file.docx","url":"https://assets-eu.researchsquare.com/files/rs-9258955/v1/edd9ddea489f8cb772f3c96d.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Development and validation of a multimodal interpretable machine learning model with SHAP for malignancy risk prediction in Bethesda III thyroid nodules: a dual-center retrospective cohort study","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Thyroid nodule, Multimodal fusion, BRAFV600E, Bethesda Category III, Noninvasive follicular thyroid neoplasm with papillary-like nuclear features, Machine learning","lastPublishedDoi":"10.21203/rs.3.rs-9258955/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9258955/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eTo develop a multimodal interpretable machine learning model integrating clinical, ultrasonographic, and BRAF\u003csup\u003eV600E\u003c/sup\u003e mutation data for malignancy risk prediction in Bethesda III thyroid nodules, explore the impact of noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP) reclassification on model robustness, and establish a risk stratification strategy for BRAF\u003csup\u003eV600E\u003c/sup\u003e-negative subgroups.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eThis dual-center retrospective cohort study enrolled 490 patients with Bethesda III nodules. Features were selected through univariate analysis, collinearity diagnosis, LASSO regression, and multivariable logistic regression. Six predictive algorithms (LR, RF, SVM, ExtraTrees, MLP, and XGBoost) and four hierarchical models (clinical, ultrasound, clinical-ultrasound fusion, and clinical-ultrasound-BRAF\u003csup\u003eV600E\u003c/sup\u003e combined) were constructed. Model performance was evaluated using the AUC, calibration curves, decision curve analysis, and SHapley Additive exPlanations (SHAP) interpretability analysis. Sensitivity analysis was conducted following NIFTP exclusion, and a multi-threshold risk stratification strategy was developed for BRAF\u003csup\u003eV600E\u003c/sup\u003e-negative subgroups.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eThyroglobulin, BRAF\u003csup\u003eV600E\u003c/sup\u003e mutation, taller-than-wide shape, calcification, and CDFI were identified as independent risk factors. The multimodal fusion model achieved AUC of 0.959 (95%CI:0.939\u0026ndash;0.977), 0.895 (95%CI: 0.831\u0026ndash;0.950), and 0.886 (95%CI:0.804\u0026ndash;0.947) in the training, internal validation, and external validation sets, respectively. The XGBoost model demonstrated optimal generalization performance in the external validation set (AUC\u0026thinsp;=\u0026thinsp;0.890, 95%CI: 0.798\u0026ndash;0.954). SHAP analysis confirmed BRAF\u003csup\u003eV600E\u003c/sup\u003e mutation as the predominant contributor. Following NIFTP exclusion, core predictors remained stable with robust overall performance, and XGBoost model maintained optimal performance (AUC\u0026thinsp;=\u0026thinsp;0.884). For the BRAF\u003csup\u003eV600E\u003c/sup\u003e-negative subgroup (n\u0026thinsp;=\u0026thinsp;281), four-tier stratification using thresholds of \u0026lt;\u0026thinsp;0.15, 0.15\u0026ndash;0.45, 0.45\u0026ndash;0.75, and \u0026gt;\u0026thinsp;0.75 demonstrated graded malignancy rates, with the low-risk tier achieving a negative predictive value of 92.4% (95% CI:0.867\u0026ndash;0.961) and the high-risk tier achieving a positive predictive value of 100% (95%CI: 0.692\u0026ndash;1.000).\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eThe multimodal interpretable machine learning model significantly enhanced malignancy risk prediction accuracy for Bethesda III nodules. The XGBoost model combines superior cross-center generalization performance with interpretability. The multi-threshold risk stratification strategy for BRAF\u003csup\u003eV600E\u003c/sup\u003e-negative subgroups provides a quantifiable, actionable pathway for clinical decision-making.\u003c/p\u003e","manuscriptTitle":"Development and validation of a multimodal interpretable machine learning model with SHAP for malignancy risk prediction in Bethesda III thyroid nodules: a dual-center retrospective cohort study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-05-08 21:19:57","doi":"10.21203/rs.3.rs-9258955/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewersInvited","content":"","date":"2026-04-30T21:19:33+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2026-04-09T16:35:36+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-04-04T11:18:31+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-04-04T11:18:21+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Informatics and Decision Making","date":"2026-03-29T12:58:19+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"52921207-a33c-48bf-b642-d47eff0f1cae","owner":[],"postedDate":"May 8th, 2026","published":true,"recentEditorialEvents":[{"type":"reviewersInvited","content":"30","date":"2026-04-30T21:19:33+00:00","index":"","fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-05-08T21:19:57+00:00","versionOfRecord":[],"versionCreatedAt":"2026-05-08 21:19:57","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9258955","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9258955","identity":"rs-9258955","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}