A Study on Explainable Artificial Intelligence(XAI) in Malware Detection for Proactive Cyber Threat Hunting

A Study on Explainable Artificial Intelligence(XAI) in Malware Detection for Proactive Cyber Threat Hunting | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Study on Explainable Artificial Intelligence(XAI) in Malware Detection for Proactive Cyber Threat Hunting Pankaj Gajakosh S., Rama Abirami K., Nagendra Kumar Y. J. This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8386211/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract In recent years, effective malware detection requires Machine Learning models that are both accurate and interpretable. While high-performing models often suffer from poor explainability, this study addresses this gap by integrating advanced Explainable Artificial Intelligence (XAI) frameworks with well-established ML algorithms to create transparent, trustworthy detection systems. We evaluate Decision Tree, Random Forest, XGBoost, Naïve Bayes, and Kernel SVM using SHAP (SHapley Additive exPlanations) for feature importance assessment. Models with limited interpretability (Kernel SVM, Decision Tree) are excluded, while remaining models undergo ELI5 permutation importance validation to confirm feature rankings and decision logic. XGBoost emerges as optimal due to its superior accuracy, superior handling of complex non-linear relationships, and stable, reproducible explanations. We apply advanced XAI techniques—Partial Dependence Plots (PDP), Individual Conditional Expectation (ICE) plots, and 2D Accumulated Local Effects (2D-ALE)—to reveal global and local trends, feature interactions, and non-linear patterns within the model's most influential features. This analysis demonstrates that tree-based ensemble models, when paired with rigorous XAI techniques, yield transparent and operationally trustworthy tools for cybersecurity applications. The result is a methodology that bridges high predictive performance with actionable intelligence, enabling security practitioners to validate and deploy ML-based malware classifiers with confidence. Explainable Artificial Intelligence SHAP Partial Dependence Plots Accumulated Local Effects Malware Detection Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8386211","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":562692540,"identity":"52183676-8ebb-4da7-b4f2-77a56deff76f","order_by":0,"name":"Pankaj Gajakosh S.","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA5klEQVRIiWNgGAWjYNCCAwzM8uwNQIaBBQlaDHsOgLRIEK+FgeFGAohFhBZ5/7MHPxecsWNnnPn86oYfBRIM/O3dCXi1GN7IS5aecSOZmV06p+xmD9BhEmfObsCvZQaPgTTPB2Zmxtk5aTd4gFoMJHIJaOk/Y/yb50M9M8PNM2k3/xCjRZ4hx0ya58ZhZoYb7MduE2WLgURemjXPmePAQM5huy1jIMFD0C/y/WcP3+Y5Vp0sz3782c03f2zk+Nt7CdhygAdMJzMw8BiAGDx4lYNtaYCosWNgYH9AUPUoGAWjYBSMTAAAZtpHOOl9JHAAAAAASUVORK5CYII=","orcid":"","institution":"Curtin University, Malaysia","correspondingAuthor":true,"prefix":"","firstName":"Pankaj","middleName":"Gajakosh","lastName":"S.","suffix":""},{"id":562692546,"identity":"3dacea1d-af49-422d-a326-d34674b363c4","order_by":1,"name":"Rama Abirami K.","email":"","orcid":"","institution":"Curtin University, Malaysia","correspondingAuthor":false,"prefix":"","firstName":"Rama","middleName":"Abirami","lastName":"K.","suffix":""},{"id":562692547,"identity":"1f0b4484-7b17-45f3-9832-0f6234383d63","order_by":2,"name":"Nagendra Kumar Y. J.","email":"","orcid":"","institution":"Gokaraju Rangraju Institute of Engineering and Technology","correspondingAuthor":false,"prefix":"","firstName":"Nagendra","middleName":"Kumar Y.","lastName":"J.","suffix":""}],"badges":[],"createdAt":"2025-12-17 13:23:42","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8386211/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8386211/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":98855656,"identity":"624fce73-db91-4366-88fe-ea95f102a1f6","added_by":"auto","created_at":"2025-12-23 07:41:37","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1154287,"visible":true,"origin":"","legend":"","description":"","filename":"AnonymisedActualResearchPaper.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8386211/v1/d20d5721c3ff94ba56e1594d.pdf"},{"id":98855655,"identity":"84ca5ecf-40ea-4087-b4ad-105b8069c61f","added_by":"auto","created_at":"2025-12-23 07:41:37","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":6412,"visible":true,"origin":"","legend":"","description":"","filename":"9d8bab6428e44b00bdd984816bd5f30a.json","url":"https://assets-eu.researchsquare.com/files/rs-8386211/v1/22811ce4d15e2adf5f5ad1fc.json"},{"id":99794295,"identity":"36339816-4b17-4762-9da0-b2159f20fd89","added_by":"auto","created_at":"2026-01-08 13:34:31","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1022724,"visible":true,"origin":"","legend":"","description":"","filename":"AnonymisedActualResearchPaper.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8386211/v1_covered_ded7f365-9673-4f5a-aaa9-4dabd7158a59.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A Study on Explainable Artificial Intelligence(XAI) in Malware Detection for Proactive Cyber Threat Hunting","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Explainable Artificial Intelligence, SHAP, Partial Dependence Plots, Accumulated Local Effects, Malware Detection","lastPublishedDoi":"10.21203/rs.3.rs-8386211/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8386211/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"In recent years, effective malware detection requires Machine Learning models that are both accurate and interpretable. While high-performing models often suffer from poor explainability, this study addresses this gap by integrating advanced Explainable Artificial Intelligence (XAI) frameworks with well-established ML algorithms to create transparent, trustworthy detection systems. We evaluate Decision Tree, Random Forest, XGBoost, Naïve Bayes, and Kernel SVM using SHAP (SHapley Additive exPlanations) for feature importance assessment. Models with limited interpretability (Kernel SVM, Decision Tree) are excluded, while remaining models undergo ELI5 permutation importance validation to confirm feature rankings and decision logic. XGBoost emerges as optimal due to its superior accuracy, superior handling of complex non-linear relationships, and stable, reproducible explanations. We apply advanced XAI techniques—Partial Dependence Plots (PDP), Individual Conditional Expectation (ICE) plots, and 2D Accumulated Local Effects (2D-ALE)—to reveal global and local trends, feature interactions, and non-linear patterns within the model's most influential features. This analysis demonstrates that tree-based ensemble models, when paired with rigorous XAI techniques, yield transparent and operationally trustworthy tools for cybersecurity applications. The result is a methodology that bridges high predictive performance with actionable intelligence, enabling security practitioners to validate and deploy ML-based malware classifiers with confidence.","manuscriptTitle":"A Study on Explainable Artificial Intelligence(XAI) in Malware Detection for Proactive Cyber Threat Hunting","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-12-23 07:41:32","doi":"10.21203/rs.3.rs-8386211/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"805381f0-827e-4d23-b79e-6617f668b3c3","owner":[],"postedDate":"December 23rd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-01-06T16:09:38+00:00","versionOfRecord":[],"versionCreatedAt":"2025-12-23 07:41:32","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8386211","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8386211","identity":"rs-8386211","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}