Mitigating Data Leakage and Class Imbalance in Explainable AI for Stroke Prediction

Mitigating Data Leakage and Class Imbalance in Explainable AI for Stroke Prediction | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Mitigating Data Leakage and Class Imbalance in Explainable AI for Stroke Prediction Feng Wen, Hong Deng, WenXue Li, Fanyu Du, Yang Fan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6876045/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 7 You are reading this latest preprint version Abstract Background: Stroke prediction using machine learning (ML) is highly sensitive to data preprocessing strategies, with significant implications for data leakage and model interpretability. Methods: This study systematically investigates the effects of three key prepro-cessing components—missing value imputation, class imbalance correction, and clinically guided feature binning—on the performance and explainability of six machine learning (ML) models: logistic regression (LR), support vector machine (SVM), decision tree (DT), random forest (RF), CatBoost, and XGBoost, using a public stroke dataset. Results: Our findings demonstrate that applying SMOTE before data splitting introduces significant data leakage, artificially inflating AUC values up to 0.99—a misleading representation. Mitigating this leakage by restricting SMOTE to the training set caused a marked drop in performance: CatBoost’s recall declined from 0.96 to 0.08, and XGBoost’s AUC decreased from 0.99 to 0.84. Similarly, imputing missing values before splitting led to inflated metrics, albeit to a lesser extent. In contrast, class-weight adjustment, a leakage-free strategy, consistently achieved robust and balanced results (AUC up to 0.86). While clinically guided feature binning improved interpretability with minimal performance trade-off, SHAP analysis confirmed that improper preprocessing distorted feature importance rankings, reducing the clinical plausibility and trustworthiness of model interpretations. Conclusion: These findings underscore that rigorous, leakage-free preprocess-ing is essential for developing reliable, interpretable, and clinically meaningful 1 stroke prediction models. This study offers methodologically grounded guidance for constructing trustworthy AI systems in healthcare. Stroke prediction Data leakage Class imbalance SMOTE Class-weight adjustment Explainable AI SHAP analysis Clinical feature binning Machine learning Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviews received at journal 30 Nov, 2025 Reviewers agreed at journal 30 Nov, 2025 Reviewers invited by journal 27 Nov, 2025 Editor invited by journal 08 Oct, 2025 Editor assigned by journal 14 Jun, 2025 Submission checks completed at journal 14 Jun, 2025 First submitted to journal 11 Jun, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6876045","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":552835698,"identity":"6bfe6c2b-5fa7-4580-bf88-3edc645bcca3","order_by":0,"name":"Feng Wen","email":"","orcid":"","institution":"North Sichuan Medical College","correspondingAuthor":false,"prefix":"","firstName":"Feng","middleName":"","lastName":"Wen","suffix":""},{"id":552835699,"identity":"17b726d7-57ed-467d-85bf-6bfe4a42f856","order_by":1,"name":"Hong Deng","email":"","orcid":"","institution":"North Sichuan Medical College","correspondingAuthor":false,"prefix":"","firstName":"Hong","middleName":"","lastName":"Deng","suffix":""},{"id":552835700,"identity":"dfd31438-99b0-44ef-8676-259438a352fb","order_by":2,"name":"WenXue Li","email":"","orcid":"","institution":"North Sichuan Medical College","correspondingAuthor":false,"prefix":"","firstName":"WenXue","middleName":"","lastName":"Li","suffix":""},{"id":552835701,"identity":"4c167992-6651-4082-830b-e4cd2733a910","order_by":3,"name":"Fanyu Du","email":"","orcid":"","institution":"North Sichuan Medical College","correspondingAuthor":false,"prefix":"","firstName":"Fanyu","middleName":"","lastName":"Du","suffix":""},{"id":552835702,"identity":"8c8351fc-5c11-4b8a-9627-eb4bf3946531","order_by":4,"name":"Yang Fan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAyklEQVRIie3PMQqDMBTG8Ugg6fCoHZV6iJRCcRA8SJdMmdKpawZLVw/QHsMbBIS4eICIoxdwdOhQoVMn41Zo/vP78fgQ8vl+sLy5N9M4KQhp4Wpaw+NnaZK41K7ESrYHgjNmuaMIHoIzAAKsGyqLVHZeJBgGzaM0gbgX1xQZcSmWCKGCazZ/2fbyFAVFvUwAyUPBCQbUtY4k2sgj0jPZWXAkDIwIbqWBuJy3cJctjN4Nfk0qD2ld2VFly+SLR3zN+YesFT6fz/cfvQFWLT4Fhh+RTAAAAABJRU5ErkJggg==","orcid":"","institution":"North Sichuan Medical College","correspondingAuthor":true,"prefix":"","firstName":"Yang","middleName":"","lastName":"Fan","suffix":""}],"badges":[],"createdAt":"2025-06-12 03:23:21","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6876045/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6876045/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":97369706,"identity":"89044b2b-1b28-437e-adc8-65edbca1b282","added_by":"auto","created_at":"2025-12-03 16:25:36","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":6841019,"visible":true,"origin":"","legend":"","description":"","filename":"SpringerNatureLaTeXTemplate4.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6876045/v1_covered_b53c1c3c-673c-49ab-bb9b-b6998b8484ae.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Mitigating Data Leakage and Class Imbalance in Explainable AI for Stroke Prediction","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Stroke prediction, Data leakage,Class imbalance,SMOTE,Class-weight adjustment,Explainable AI, SHAP analysis, Clinical feature binning, Machine learning","lastPublishedDoi":"10.21203/rs.3.rs-6876045/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6876045/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eBackground: Stroke prediction using machine learning (ML) is highly sensitive to data preprocessing strategies, with significant implications for data leakage and model interpretability.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eMethods: This study systematically investigates the effects of three key prepro-cessing components—missing value imputation, class imbalance correction, and clinically guided feature binning—on the performance and explainability of six machine learning (ML) models: logistic regression (LR), support vector machine (SVM), decision tree (DT), random forest (RF), CatBoost, and XGBoost, using a public stroke dataset.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eResults: Our findings demonstrate that applying SMOTE before data splitting introduces significant data leakage, artificially inflating AUC values up to 0.99—a misleading representation. Mitigating this leakage by restricting SMOTE to the training set caused a marked drop in performance: CatBoost’s recall declined from 0.96 to 0.08, and XGBoost’s AUC decreased from 0.99 to 0.84. Similarly, imputing missing values before splitting led to inflated metrics, albeit to a lesser extent. In contrast, class-weight adjustment, a leakage-free strategy, consistently achieved robust and balanced results (AUC up to 0.86). While clinically guided feature binning improved interpretability with minimal performance trade-off, SHAP analysis confirmed that improper preprocessing distorted feature importance rankings, reducing the clinical plausibility and trustworthiness of model interpretations.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eConclusion: These findings underscore that rigorous, leakage-free preprocess-ing is essential for developing reliable, interpretable, and clinically meaningful 1 stroke prediction models. This study offers methodologically grounded guidance for constructing trustworthy AI systems in healthcare.\u003c/p\u003e","manuscriptTitle":"Mitigating Data Leakage and Class Imbalance in Explainable AI for Stroke Prediction","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-12-03 09:19:06","doi":"10.21203/rs.3.rs-6876045/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2025-11-30T06:34:50+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"150756367104384753308491895037900915952","date":"2025-11-30T05:33:59+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-11-27T15:16:07+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-10-08T12:10:06+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-06-14T10:54:42+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-06-14T10:52:59+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Informatics and Decision Making","date":"2025-06-12T03:17:10+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"a6059edf-89a4-4585-b212-ada5bc2226f4","owner":[],"postedDate":"December 3rd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-12-03T09:19:06+00:00","versionOfRecord":[],"versionCreatedAt":"2025-12-03 09:19:06","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6876045","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6876045","identity":"rs-6876045","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}