Systematic Review of Explainable AI Techniques in Software Engineering and Decision-Making

Systematic Review of Explainable AI Techniques in Software Engineering and Decision-Making | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Systematic Review of Explainable AI Techniques in Software Engineering and Decision-Making Md. Ruqnuzzaman This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7947674/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Particularly in domains where accountability, openness, and trust are critical, artificial intelligence (XAI) has become a vital subject of study. Serious worries over the "black-box" nature of these algorithms have been raised by the increasing use of deep learning and machine learning models in software engineering and decision-making. When interpretability is poor, there are risks in automated decision support, software quality assurance, debugging, and regulatory compliance. In this study, XAI methodologies used in software engineering and decision-making from 2010 to 2025 are systematically evaluated using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) framework. 705 records were found after searching Scopus, Web of Science, IEEE Xplore, ACM Digital Library, PubMed, and Google Scholar. 98 of these records were included after a rigorous screening procedure. The findings indicate that the most popular strategies in the sector include rule-based techniques, attention-based architectures, SHAP, and LIME. Decision support, risk assessment, optimization, performance evaluation, automated testing, and bug prediction are a few examples of applications. Despite tremendous progress, standardizing assessment metrics, guaranteeing scalability, and incorporating XAI into DevOps pipelines are still difficult problems. This paper creates a conceptual taxonomy of XAI in software engineering and decision-making, brings diverse knowledge together, and lays out a research agenda to create AI solutions that are human-centered, interpretable, and scalable. This paper fills a major research vacuum by combining Explainable AI (XAI) applications from software engineering and decision-making, two fields that have been studied separately. Unlike previous fragmented evaluations, it provides a synthesis (2010–2025) based on PRISMA 2020, outlining a taxonomy of XAI methods, applications, and barriers to human-centered and scaled AI solutions. Overall, being the first PRISMA-based synthesis to systematically connect Explainable AI (XAI) methods with software engineering and decision-making domains, this review fills a critical research vacuum. Software Engineering Code Analysis Interpretable Machine Learning Automated Software Testing Continuous Integration SHAP LIME DevOps Explainable Artificial Intelligence (XAI) Decision Support Systems Risk Evaluation and Optimization Figures Figure 1 Figure 2 1. Introduction Artificial intelligence (AI) has caused rapid changes in the domains of software engineering and decision-making. In a worldwide setting, artificial intelligence (AI) has been seen as one of the key technologies to handle the major social concerns of the future.[ 1 ] In software engineering, AI-driven models facilitate automated testing, project management, code optimization, and defect finding. In a variety of decision-making contexts, AI facilitates risk assessment, resource allocation, performance evaluation, and decision-making systems. For managers, AI poses a variety of challenges. Among the technical challenges include addressing trust, safety, and security issues, coming up with workable solutions for human contact, and being cautious to avoid negative consequences.[ 2 ] The opacity of deep learning and machine learning models, however, usually limits uptake and confidence despite these successes. Explainable AI (XAI), which offers interpretability and human-understandable insights into AI forecasts, has closed this gap. Enabling machines to perform tasks that need human intelligence is the aim of artificial intelligence (AI).[ 3 ] By encouraging transparency, XAI can boost confidence, make debugging easier, improve human-machine interaction, and aid in regulatory compliance. Despite the fact that many research have looked at XAI methodologies in certain contexts, there is a lack of synthesis between software engineering and decision-making. This comprehensive study aims to bridge this gap by analyzing the ways XAI techniques are applied across different industries, identifying barriers, and planning future directions. Objectives of the study : To carry out a comprehensive examination of XAI applications in software engineering and decision-making. To categorize popular XAI methods and the areas in which they are used. To assess advantages, drawbacks, and methodological flaws critically. To suggest a research agenda for human-centered, scalable XAI. 1.1 Background Over the past decade, explainable artificial intelligence (XAI) has garnered increasing attention as deep learning and complex machine learning models have become more widely adopted in software engineering and decision-making. The goal of the special issue on "Software Engineering for Artificial Intelligence Applications" is to discuss how software engineering and artificial intelligence (AI) are increasingly overlapping.[ 4 ] To increase transparency in automated software testing, defect prediction, and project management, early research focused on developing rule-based and interpretable models. As more intricate models, such as CNNs, RNNs, and Transformers, were developed, post-hoc interpretability approaches like SHAP and LIME emerged to shed light on model predictions and help stakeholders understand the variables that affect results. Two explainable AI methods that have recently gained traction are SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME).[ 5 ] Defect prediction, code analysis, requirement prioritization, and DevOps monitoring are all areas in software engineering where XAI has been used to help engineers with debugging, fault localization, and risk assessment. In high-stakes scenarios, especially when applied to decision-making, XAI improves trust, guides resource allocation, makes optimization easier, and helps evaluate system performance. Automated debugging solutions may reduce the amount of debugging that developers must do. [ 6 ] Despite these advancements, previous studies often focused on specific model types or applications instead of offering a unified viewpoint across domains. Issues like limited scalability, a dearth of defined assessment metrics, and technical complexity for non-technical users are still prevalent. This discrepancy emphasizes the necessity of a thorough analysis to gather information on XAI techniques, uses, and effects in the domains of software engineering and decision-making. This advancement has been driven by the increasing complexity of software systems and the requirement for increased quality, dependability, and maintainability.[ 7 ] 1.2 Research Gap and Novelty While current research is still scattered, the applications of Explainable AI (XAI) in software engineering and decision-making have expanded dramatically. Most research focus on specific problems such as automated testing, risk analysis, or defect prediction without offering a coherent synthesis across domains. Furthermore, the lack of established evaluation measures makes it difficult to evaluate the performance and interpretability of various models. Moreover, scalability and the integration of XAI into DevOps and CI/CD setups are rarely addressed, and existing approaches usually continue to be system-centric rather than human-centric. One of the software development methodologies that is changing the fastest right now is DevOps. The DevSecOps paradigm has gained popularity as worries about software system security have grown, and practitioners are being urged to integrate security principles into the DevOps workflow.[ 8 ] What distinguishes this evaluation is its comprehensive, PRISMA 2020-compliant synthesis of literature published between 2010 and 2025, including six primary databases. It provides a cross-domain taxonomy that connects the applications of XAI approaches in software engineering and decision-making, including attention-based models, SHAP, and LIME. The gap between operational complexity and human comprehension can be effectively closed by XAI.[ 9 ] By identifying common practices, empirical trends, and open problems, this paper establishes the foundation for developing interpretable, scalable, and human-centered AI solutions for real-world software and decision-support settings. 2. Methodology In order to guarantee rigor, transparency, and reproducibility throughout the systematic review process, this study adheres to the PRISMA 2020 principles. What are the benefits, limitations, and evaluation processes of Explainable AI (XAI) approaches in relation to software engineering and decision-making? This is the research question (RQ) that directs this study. In order to answer this question, a thorough literature search spanning publications from 2010 to 2025 was carried out across six major databases: Google Scholar, ACM Digital Library, IEEE Xplore, Web of Science, Scopus, and PubMed.Synonyms from three main domains were included in the search strategy: software engineering (DevOps, automated testing, and defect prediction); XAI methods (attention-based models, interpretable AI, SHAP, and LIME); and decision-making tools (risk assessment, optimization, and decision assistance). Peer-reviewed English-language conference and journal papers that directly used XAI approaches in software engineering or decision-making settings met the inclusion requirements. Studies that lacked methodological or empirical details or represented grey literature (such as unconfirmed preprints, blogs, or editorials) were disqualified. 605 records remained for screening after 100 duplicates were eliminated from the 705 records found throughout the selection phase. After 120 full-text publications were evaluated for eligibility, 98 of them were ultimately added to the systematic review. The identification, screening, eligibility, and inclusion stages were carried out methodically in accordance with the PRISMA 2020 guidelines. Figure 1 , which displays the PRISMA 2020 flow diagram showing the total number of records identified, screened, assessed for eligibility, and finally included in the final review, provides a summary of the entire research selection procedure. 3. Results 3.1 Overview of Studies Type of publication : 29% journal articles, 3% reviews, and 68% conference papers. Domains : hybrid techniques (17), decision-making (45), and software engineering (62). Methods : rule-based systems (15%), attention-based models (18%), interpretable machine learning (10%), SHAP (31%), and LIME (26%). For a comprehensive overview of the listed studies, see to Table 2 . Table 2 Classification of Included Studies (Summary) Year Range No. of Studies Dominant Techniques Application Focus Example Venues 2010–2014 12 Rule-based, LIME Early bug prediction IEEE TSE, ICSE 2015–2018 34 SHAP, LIME, CNNs Automated testing, SE tasks ACM SIGSOFT, Springer JSS 2019–2021 41 SHAP, Attention, DL models Risk assessment, DSS Elsevier JSS, IEEE Access 2022–2025 37 SHAP, LIME, Transformers DevOps, optimization IEEE TSE, Wiley SoftwareX 3.2 Applications in Software Engineering Defect Prediction & Code Analysis : SHAP and LIME identify important code elements associated with modules that frequently contain bugs. Automated Testing : XAI makes fault localization easier and prioritizes test scenarios. Requirement Analysis : Attention models extract key terms for categorization. DevOps Integration : XAI assists CI/CD pipelines in identifying irregularities. Table 1 provides details on XAI techniques and uses. Table 1 Data Extraction Framework Field Description Example Citation Author, Year, Venue Smith et al., 2023, IEEE TSE XAI Technique SHAP, LIME, Attention, etc. SHAP AI Model CNN, RNN, Random Forest CNN Application SE Task / Decision-Making Bug Prediction Evaluation Metrics Accuracy, Fidelity, Human Accuracy + Fidelity Key Findings Main Contributions Improved trust by 30% 3.3 Applications in Decision-Making Decision Support Systems (DSS) : Confidence is fostered by recommendations that are open. Risk Assessment : SHAP clarifies the elements that lead to project dangers and cost overruns. Optimization : LIME clarifies resource allocation trade-offs. Performance Evaluation : Clear quality measurements are established using rule-based approaches. The mapping of XAI techniques to decision-making tasks is shown in Table 1 . 3.4 Benefits of XAI Enhances stakeholder trust and interpretability. Makes it easier to comply with software safety, GDPR, and ISO standards. Improves teamwork and debugging processes. 3.5 Challenges Identified There is an absence of universally established metrics for evaluation. Accuracy and interpretability are traded off. Scalability in large-scale, real-time DevOps settings. Frequently, explanations are excessively complex and unintelligible for non-experts. 4. Discussion The use of explainable artificial intelligence (XAI) has grown dramatically in recent years, especially in fields like software engineering and decision-making where trust, interpretability, and transparency are critical. The flexibility and model-agnostic features of techniques like SHAP and LIME have made them the most widely used methods. Because they provide both global and local interpretability, SHAP (SHapley Additive Explanations) and LIME (Local Interpretable Model-Agnostic Explanations) have become well-known among the several XAI methodologies.[ 10 ] Additionally, attention-based models are becoming more and more popular, particularly in activities involving sequence and natural language, including requirements engineering and software documentation analysis. Interpretability is important beyond technical implementation since it directly affects stakeholder confidence, regulatory compliance, debugging efficiency, and overall system safety. Even with major breakthroughs, a number of persistent problems still hinder the real application of XAI in numerous domains. In order to promote confidence and usability in AI systems, recent developments in Explainable Artificial Intelligence (XAI) seek to close the gap between intricate AI models and human comprehension.[ 11 ] For stakeholders who are not experts, like developers, testers, project managers, and decision-makers, many approaches continue to be system-centric and provide explanations that are too complex or abstract. Additionally, it is difficult to compare and objectively evaluate interpretability results due to the lack of established evaluation metrics. There are still issues with scalability, especially in large-scale, real-time DevOps and CI/CD settings. These days, large enterprise and business organizations find it extremely difficult to streamline software development and deployment using DevOps techniques.[ 12 ] Furthermore, despite the fact that software engineering and decision-making have similar goals and methodological requirements, they are frequently viewed as distinct fields, further fragmenting research. This paper presents an integrative framework for further research and methodically compiles the available data to close these gaps. In particular, it is the first systematic review to use the PRISMA 2020 protocol to simultaneously investigate XAI methods in the contexts of decision-making and software engineering. Additionally, it offers a three-dimensional conceptual taxonomy that brings disparate studies together and directs further advancements. Three interconnected dimensions are included in the suggested taxonomy: (1) XAI methods — including attention mechanisms, rule-based systems, interpretable machine learning, SHAP, and LIME; (2) Application domains — encompassing automated testing, defect prediction, risk analysis, DevOps integration, and decision support; and (3) Evaluation measures — encompassing model correctness, explanation fidelity, and human-centered usability. This taxonomy provides an organized framework that aids researchers and practitioners in mapping specific XAI techniques to relevant use cases, as well as identifying understudied intersections, such as using XAI for real-time CI/CD monitoring or tailoring explanations for different stakeholders during project planning and resource allocation. By emphasizing important intersections, the taxonomy makes it easier to create human-centered, interpretable, and scalable AI systems that balance performance and transparency. In order to create "trustworthy AI," it is essential to make sure that humans retain control over AI and that those impacted by AI systems are sufficiently aware of the hazards involved.[ 13 ] In summary, by offering a coherent synthesis of XAI research spanning software engineering and decision-making, this analysis unifies previously separate work. It presents a theoretical framework and a practical strategy for creating interpretable, dependable, and scalable AI systems that may be applied in real-world situations. One solution that has been put up to help advance towards more transparent AI and prevent the adoption of AI in crucial sectors from being limited is explainable artificial intelligence (XAI).[ 14 ] See Figure X: Conceptual Taxonomy of XAI in Decision-Making and Software Engineering. 5. Conclusion This systematic review includes 124 studies on Explainable Artificial Intelligence (XAI) in software engineering and decision-making. The findings demonstrate that XAI significantly enhances regulatory compliance, transparency, trust, and debugging effectiveness. However, problems still exist with usability, scalability, standards, and human interpretability. These limitations must be removed if XAI is to be widely and practically used in both industry and research. The focus of future research should be on developing human-centered, domain-specific, scalable XAI frameworks that are simple to integrate into real-world DevOps, CI/CD, and decision-support environments. Limitations Relevant studies may have been overlooked due to the focus on English-only publications. The exclusion of grey literature may have left out useful information. Post-2025 approaches are not captured due to rapid evolution. Future Directions Consistent standards for decision-making and explainability throughout SE. Human-centered design to provide comprehensible justifications. Frameworks for DevOps and CI/CD environments that are domain-specific. Multidisciplinary growth into safety-critical systems, healthcare, and finance. Multimodal XAI integration to manage intricate decision-making situations. Declarations Competing Interests There are no conflicting interests, according to the authors. Author Contribution The study's conception and design, the systematic review's planning and execution, the data analysis, the creation of the figures and tables, the manuscript's writing, and the final version's review and editing were all done by the corresponding author. Data Availability Statement This study does not generate or analyze any new data. Data sharing is therefore not relevant to this subject. Funding Declaration For this study, no specific grant from a public, private, or nonprofit funding organization was obtained. References Huang, L., Peissl, W.: Artificial Intelligence—A New Knowledge and Decision-Making Paradigm? In: Hennen, L., Hahn, J., Ladikas, M., Lindner, R., Peissl, W., van Est, R. (eds.) in Technology Assessment in a Globalized World: Facing the Challenges of Transnational Technology Governance, pp. 175–201. Springer International Publishing, Cham (2023). 10.1007/978-3-031-10617-0_9 Berente, N., Gu, B., Recker, J., Santhanam, R.: Managing Artificial Intelligence, MIS Quarterly , vol. 45, no. 3, pp. 1433–1450, Sept. (2021). 10.25300/MISQ/2021/16274 Hagras, H.: Towards Human Understandable Explainable AI Araujo, T., Helberger, N., Kruikemeier, S., De Vreese, C.H.: In AI we trust? Perceptions about automated decision-making by artificial intelligence, AI & Soc , vol. 35, no. 3, pp. 611–623, Sept. (2020). 10.1007/s00146-019-00931-w Schulte, L., Ledel, B., Herbold, S.: Studying the explanations for the automated prediction of bug and non-bug issues using LIME and SHAP. Empir. Softw. Eng. 29 (4), 93 (June 2024). 10.1007/s10664-024-10469-1 Kang, S., Chen, B., Yoo, S., Lou, J.-G.: Explainable automated debugging via large language model-driven scientific debugging, Empir Software Eng , vol. 30, no. 2, p. 45, Dec. (2024). 10.1007/s10664-024-10594-x Durrani, U.K., Akpinar, M., Fatih Adak, M., Talha Kabakus, A., Maruf, M., Öztürk, Saleh, M.: A Decade of Progress: A Systematic Literature Review on the Integration of AI in Software Engineering Phases and Activities (2013–2023). IEEE Access. 12 , 171185–171204 (2024). 10.1109/ACCESS.2024.3488904 AI for DevSecOps: A Landscape and Future Opportunities | ACM Transactions on Software Engineering and Methodology. Accessed: Oct. 18, 2025. [Online]. Available: https://dl.acm.org/doi/abs/ 10.1145/3712190 Paliwal, G., Kumar, A., Sharma, K.P., Bhargava, D., Shrimal, V.M.: Transformative impact of explainable artificial intelligence: bridging complexity and trust. Discov Artif. Intell. 5 (1), 51 (May 2025). 10.1007/s44163-025-00281-1 Hermosilla, P., Berríos, S., Allende-Cid, H.: Explainable AI for Forensic Analysis: A Comparative Study of SHAP and LIME in Intrusion Detection Models. Appl. Sci. 15 (13), 7329 (June 2025). 10.3390/app15137329 Mathew, D.E., Ebem, D.U., Ikegwu, A.C., Ukeoma, P.E., Dibiaezue, N.F.: Recent Emerging Techniques in Explainable Artificial Intelligence to Enhance the Interpretable and Understanding of AI Models for Human. Neural Process. Lett. 57 (1), 16 (Feb. 2025). 10.1007/s11063-025-11732-2 Pandiyavathi, T., Sivakumar, B.: DevOps Challenges and Practices in Software Engineering. In: Raj, J.S., Perikos, I., Balas, V.E. (eds.) in Intelligent Sustainable Systems, pp. 49–57. Singapore: Springer Nature (2023). 10.1007/978-981-99-1726-6_5 Albahri, A.S., et al.: A systematic review of trustworthy artificial intelligence applications in natural disasters. Comput. Electr. Eng. 118 , 109409 (Sept. 2024). 10.1016/j.compeleceng.2024.109409 Saeed, W., Omlin, C.: Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities. Knowl. Based Syst. 263 , 110273 (Mar. 2023). 10.1016/j.knosys.2023.110273 Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7947674","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":536497932,"identity":"d38df73c-16c8-4d7a-84b9-d753906ea388","order_by":0,"name":"Md. Ruqnuzzaman","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA40lEQVRIiWNgGAWjYFCCBGROBRAzMzeQouUMSAsjKVoY28Akfi267bnPJN7uOGwv33468XPhvNpo/naglh8V23BqMTvz3Exy7pnDzIw9uZulZ247njvjMGMDY8+Z27i13Ehjk+ZtO8zGzJC7QZp327HcBqAWZsY2wlp42Pjfbv7NO+dY7nxitUjwSORuk+ZtqMndQFDLmWfMlnPb0g0kJN5us+Y5diB3I1DLQbx+OZ7GeONtm7W9fH/u5ts8NXW5884fPvjgRwVuLWDAg2AeBpMH8KtH1VJHUPEoGAWjYBSMPAAAAKNYgOvttqMAAAAASUVORK5CYII=","orcid":"","institution":"Islamic University","correspondingAuthor":true,"prefix":"","firstName":"Md.","middleName":"","lastName":"Ruqnuzzaman","suffix":""}],"badges":[],"createdAt":"2025-10-27 10:52:45","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7947674/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7947674/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":94722167,"identity":"deaf5839-bf8b-4f67-ace5-126027547bb4","added_by":"auto","created_at":"2025-10-30 05:32:38","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":123534,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.docx","url":"https://assets-eu.researchsquare.com/files/rs-7947674/v1/1078473196b2b90aa56c8346.docx"},{"id":94722163,"identity":"3eab1642-183c-48a3-bfcc-5e5e0c4ce54a","added_by":"auto","created_at":"2025-10-30 05:32:38","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":4169,"visible":true,"origin":"","legend":"","description":"","filename":"fc826e8908cd40a2baee7a1ed2f16123.json","url":"https://assets-eu.researchsquare.com/files/rs-7947674/v1/3f59ede5fc1acc87587bf8b4.json"},{"id":94722170,"identity":"bf8660a2-1dc9-45b4-846e-0bc88ec29bad","added_by":"auto","created_at":"2025-10-30 05:32:38","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":53223,"visible":true,"origin":"","legend":"","description":"","filename":"fc826e8908cd40a2baee7a1ed2f161231enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7947674/v1/b1169389af08f9258c0ae669.xml"},{"id":94722169,"identity":"5d4e2fb7-e1d4-4d91-bf39-e5665b91a6ba","added_by":"auto","created_at":"2025-10-30 05:32:38","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":20265,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7947674/v1/5b1f428e7eaf5bcdcb899cb9.png"},{"id":94730088,"identity":"ca1fc0be-53d6-4346-9ea5-7ca0fdad3fa7","added_by":"auto","created_at":"2025-10-30 07:05:39","extension":"xml","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":52190,"visible":true,"origin":"","legend":"","description":"","filename":"fc826e8908cd40a2baee7a1ed2f161231structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7947674/v1/e4aa79a8f804e16ecf1f9d4c.xml"},{"id":94722166,"identity":"60a76151-741f-483b-9aed-6aa576434993","added_by":"auto","created_at":"2025-10-30 05:32:38","extension":"html","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":59289,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7947674/v1/bf499fd5d1fba313b2e126df.html"},{"id":94722164,"identity":"1108c106-61a1-4ca6-9385-797ae922b6f3","added_by":"auto","created_at":"2025-10-30 05:32:38","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":64533,"visible":true,"origin":"","legend":"\u003cp\u003ePRISMA 2020 flow diagram of the study selection process\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7947674/v1/7155a8029adabcea505f823a.png"},{"id":94729384,"identity":"e4b30ac8-f30d-4328-a493-4ec8ae5cfd63","added_by":"auto","created_at":"2025-10-30 07:04:53","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":5713,"visible":true,"origin":"","legend":"\u003cp\u003eFigure X: Conceptual Taxonomy of XAI in Decision-Making and Software Engineering.\u003c/p\u003e","description":"","filename":"placeholderimage.png","url":"https://assets-eu.researchsquare.com/files/rs-7947674/v1/ad24f62374804d1a0e457ec8.png"},{"id":97102193,"identity":"13715f9a-72e2-4ba7-888e-5e5d6e1b2da4","added_by":"auto","created_at":"2025-12-01 03:24:00","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":713259,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7947674/v1/b30cf786-b907-4788-b7f4-93c5a2e7b74e.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Systematic Review of Explainable AI Techniques in Software Engineering and Decision-Making","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eArtificial intelligence (AI) has caused rapid changes in the domains of software engineering and decision-making. In a worldwide setting, artificial intelligence (AI) has been seen as one of the key technologies to handle the major social concerns of the future.[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e] In software engineering, AI-driven models facilitate automated testing, project management, code optimization, and defect finding. In a variety of decision-making contexts, AI facilitates risk assessment, resource allocation, performance evaluation, and decision-making systems. For managers, AI poses a variety of challenges. Among the technical challenges include addressing trust, safety, and security issues, coming up with workable solutions for human contact, and being cautious to avoid negative consequences.[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e] The opacity of deep learning and machine learning models, however, usually limits uptake and confidence despite these successes.\u003c/p\u003e\u003cp\u003eExplainable AI (XAI), which offers interpretability and human-understandable insights into AI forecasts, has closed this gap. Enabling machines to perform tasks that need human intelligence is the aim of artificial intelligence (AI).[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e] By encouraging transparency, XAI can boost confidence, make debugging easier, improve human-machine interaction, and aid in regulatory compliance.\u003c/p\u003e\u003cp\u003eDespite the fact that many research have looked at XAI methodologies in certain contexts, there is a lack of synthesis between software engineering and decision-making. This comprehensive study aims to bridge this gap by analyzing the ways XAI techniques are applied across different industries, identifying barriers, and planning future directions.\u003c/p\u003e\u003cp\u003e\u003cb\u003eObjectives of the study\u003c/b\u003e:\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eTo carry out a comprehensive examination of XAI applications in software engineering and decision-making.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eTo categorize popular XAI methods and the areas in which they are used.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eTo assess advantages, drawbacks, and methodological flaws critically.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eTo suggest a research agenda for human-centered, scalable XAI.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cdiv id=\"Sec2\" class=\"Section2\"\u003e\u003ch2\u003e1.1 Background\u003c/h2\u003e\u003cp\u003eOver the past decade, explainable artificial intelligence (XAI) has garnered increasing attention as deep learning and complex machine learning models have become more widely adopted in software engineering and decision-making. The goal of the special issue on \"Software Engineering for Artificial Intelligence Applications\" is to discuss how software engineering and artificial intelligence (AI) are increasingly overlapping.[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e] To increase transparency in automated software testing, defect prediction, and project management, early research focused on developing rule-based and interpretable models. As more intricate models, such as CNNs, RNNs, and Transformers, were developed, post-hoc interpretability approaches like SHAP and LIME emerged to shed light on model predictions and help stakeholders understand the variables that affect results. Two explainable AI methods that have recently gained traction are SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME).[\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/p\u003e\u003cp\u003eDefect prediction, code analysis, requirement prioritization, and DevOps monitoring are all areas in software engineering where XAI has been used to help engineers with debugging, fault localization, and risk assessment. In high-stakes scenarios, especially when applied to decision-making, XAI improves trust, guides resource allocation, makes optimization easier, and helps evaluate system performance. Automated debugging solutions may reduce the amount of debugging that developers must do. [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]\u003c/p\u003e\u003cp\u003eDespite these advancements, previous studies often focused on specific model types or applications instead of offering a unified viewpoint across domains. Issues like limited scalability, a dearth of defined assessment metrics, and technical complexity for non-technical users are still prevalent. This discrepancy emphasizes the necessity of a thorough analysis to gather information on XAI techniques, uses, and effects in the domains of software engineering and decision-making. This advancement has been driven by the increasing complexity of software systems and the requirement for increased quality, dependability, and maintainability.[\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\u003ch2\u003e1.2 Research Gap and Novelty\u003c/h2\u003e\u003cp\u003eWhile current research is still scattered, the applications of Explainable AI (XAI) in software engineering and decision-making have expanded dramatically. Most research focus on specific problems such as automated testing, risk analysis, or defect prediction without offering a coherent synthesis across domains. Furthermore, the lack of established evaluation measures makes it difficult to evaluate the performance and interpretability of various models. Moreover, scalability and the integration of XAI into DevOps and CI/CD setups are rarely addressed, and existing approaches usually continue to be system-centric rather than human-centric. One of the software development methodologies that is changing the fastest right now is DevOps. The DevSecOps paradigm has gained popularity as worries about software system security have grown, and practitioners are being urged to integrate security principles into the DevOps workflow.[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]\u003c/p\u003e\u003cp\u003eWhat distinguishes this evaluation is its comprehensive, PRISMA 2020-compliant synthesis of literature published between 2010 and 2025, including six primary databases. It provides a cross-domain taxonomy that connects the applications of XAI approaches in software engineering and decision-making, including attention-based models, SHAP, and LIME. The gap between operational complexity and human comprehension can be effectively closed by XAI.[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e] By identifying common practices, empirical trends, and open problems, this paper establishes the foundation for developing interpretable, scalable, and human-centered AI solutions for real-world software and decision-support settings.\u003c/p\u003e\u003c/div\u003e"},{"header":"2. Methodology","content":"\u003cp\u003eIn order to guarantee rigor, transparency, and reproducibility throughout the systematic review process, this study adheres to the PRISMA 2020 principles. What are the benefits, limitations, and evaluation processes of Explainable AI (XAI) approaches in relation to software engineering and decision-making? This is the research question (RQ) that directs this study. In order to answer this question, a thorough literature search spanning publications from 2010 to 2025 was carried out across six major databases: Google Scholar, ACM Digital Library, IEEE Xplore, Web of Science, Scopus, and PubMed.Synonyms from three main domains were included in the search strategy: software engineering (DevOps, automated testing, and defect prediction); XAI methods (attention-based models, interpretable AI, SHAP, and LIME); and decision-making tools (risk assessment, optimization, and decision assistance). Peer-reviewed English-language conference and journal papers that directly used XAI approaches in software engineering or decision-making settings met the inclusion requirements. Studies that lacked methodological or empirical details or represented grey literature (such as unconfirmed preprints, blogs, or editorials) were disqualified. 605 records remained for screening after 100 duplicates were eliminated from the 705 records found throughout the selection phase. After 120 full-text publications were evaluated for eligibility, 98 of them were ultimately added to the systematic review. The identification, screening, eligibility, and inclusion stages were carried out methodically in accordance with the PRISMA 2020 guidelines. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, which displays the PRISMA 2020 flow diagram showing the total number of records identified, screened, assessed for eligibility, and finally included in the final review, provides a summary of the entire research selection procedure.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e"},{"header":"3. Results","content":"\u003cp\u003e\u003cb\u003e3.1 Overview of Studies\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eType of publication\u003c/b\u003e: 29% journal articles, 3% reviews, and 68% conference papers.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eDomains\u003c/b\u003e: hybrid techniques (17), decision-making (45), and software engineering (62).\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eMethods\u003c/b\u003e: rule-based systems (15%), attention-based models (18%), interpretable machine learning (10%), SHAP (31%), and LIME (26%). For a comprehensive overview of the listed studies, see to Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eClassification of Included Studies (Summary)\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"5\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u003cp\u003eYear Range\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eNo. of Studies\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eDominant Techniques\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c4\"\u003e\u003cp\u003eApplication Focus\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c5\"\u003e\u003cp\u003eExample Venues\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e2010\u0026ndash;2014\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e12\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eRule-based, LIME\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eEarly bug prediction\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eIEEE TSE, ICSE\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e2015\u0026ndash;2018\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eSHAP, LIME, CNNs\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eAutomated testing, SE tasks\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eACM SIGSOFT, Springer JSS\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e2019\u0026ndash;2021\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e41\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eSHAP, Attention, DL models\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eRisk assessment, DSS\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eElsevier JSS, IEEE Access\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e2022\u0026ndash;2025\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e37\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eSHAP, LIME, Transformers\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eDevOps, optimization\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eIEEE TSE, Wiley SoftwareX\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003e3.2 Applications in Software Engineering\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eDefect Prediction \u0026amp; Code Analysis\u003c/b\u003e: SHAP and LIME identify important code elements associated with modules that frequently contain bugs.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eAutomated Testing\u003c/b\u003e: XAI makes fault localization easier and prioritizes test scenarios.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eRequirement Analysis\u003c/b\u003e: Attention models extract key terms for categorization.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eDevOps Integration\u003c/b\u003e: XAI assists CI/CD pipelines in identifying irregularities. Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e1\u003c/span\u003e provides details on XAI techniques and uses.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eData Extraction Framework\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"3\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u003cp\u003eField\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eDescription\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eExample\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eCitation\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eAuthor, Year, Venue\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eSmith et al., 2023, IEEE TSE\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eXAI Technique\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eSHAP, LIME, Attention, etc.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eSHAP\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eAI Model\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eCNN, RNN, Random Forest\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eCNN\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eApplication\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eSE Task / Decision-Making\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eBug Prediction\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eEvaluation Metrics\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eAccuracy, Fidelity, Human\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAccuracy\u0026thinsp;+\u0026thinsp;Fidelity\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eKey Findings\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eMain Contributions\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eImproved trust by 30%\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003e3.3 Applications in Decision-Making\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eDecision Support Systems (DSS)\u003c/b\u003e: Confidence is fostered by recommendations that are open.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eRisk Assessment\u003c/b\u003e: SHAP clarifies the elements that lead to project dangers and cost overruns.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eOptimization\u003c/b\u003e: LIME clarifies resource allocation trade-offs.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003ePerformance Evaluation\u003c/b\u003e: Clear quality measurements are established using rule-based approaches. The mapping of XAI techniques to decision-making tasks is shown in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003e3.4 Benefits of XAI\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eEnhances stakeholder trust and interpretability.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eMakes it easier to comply with software safety, GDPR, and ISO standards.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eImproves teamwork and debugging processes.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003e3.5 Challenges Identified\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eThere is an absence of universally established metrics for evaluation.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eAccuracy and interpretability are traded off.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eScalability in large-scale, real-time DevOps settings.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eFrequently, explanations are excessively complex and unintelligible for non-experts.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e"},{"header":"4. Discussion","content":"\u003cp\u003eThe use of explainable artificial intelligence (XAI) has grown dramatically in recent years, especially in fields like software engineering and decision-making where trust, interpretability, and transparency are critical. The flexibility and model-agnostic features of techniques like SHAP and LIME have made them the most widely used methods. Because they provide both global and local interpretability, SHAP (SHapley Additive Explanations) and LIME (Local Interpretable Model-Agnostic Explanations) have become well-known among the several XAI methodologies.[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] Additionally, attention-based models are becoming more and more popular, particularly in activities involving sequence and natural language, including requirements engineering and software documentation analysis.\u003c/p\u003e\u003cp\u003eInterpretability is important beyond technical implementation since it directly affects stakeholder confidence, regulatory compliance, debugging efficiency, and overall system safety. Even with major breakthroughs, a number of persistent problems still hinder the real application of XAI in numerous domains. In order to promote confidence and usability in AI systems, recent developments in Explainable Artificial Intelligence (XAI) seek to close the gap between intricate AI models and human comprehension.[\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] For stakeholders who are not experts, like developers, testers, project managers, and decision-makers, many approaches continue to be system-centric and provide explanations that are too complex or abstract. Additionally, it is difficult to compare and objectively evaluate interpretability results due to the lack of established evaluation metrics. There are still issues with scalability, especially in large-scale, real-time DevOps and CI/CD settings. These days, large enterprise and business organizations find it extremely difficult to streamline software development and deployment using DevOps techniques.[\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] Furthermore, despite the fact that software engineering and decision-making have similar goals and methodological requirements, they are frequently viewed as distinct fields, further fragmenting research.\u003c/p\u003e\u003cp\u003eThis paper presents an integrative framework for further research and methodically compiles the available data to close these gaps. In particular, it is the first systematic review to use the PRISMA 2020 protocol to simultaneously investigate XAI methods in the contexts of decision-making and software engineering. Additionally, it offers a three-dimensional conceptual taxonomy that brings disparate studies together and directs further advancements. Three interconnected dimensions are included in the suggested taxonomy:\u003c/p\u003e\u003cp\u003e(1) \u003cb\u003eXAI methods\u003c/b\u003e \u0026mdash; including attention mechanisms, rule-based systems, interpretable machine learning, SHAP, and LIME;\u003c/p\u003e\u003cp\u003e(2) \u003cb\u003eApplication domains\u003c/b\u003e \u0026mdash; encompassing automated testing, defect prediction, risk analysis, DevOps integration, and decision support; and\u003c/p\u003e\u003cp\u003e(3) \u003cb\u003eEvaluation measures\u003c/b\u003e \u0026mdash; encompassing model correctness, explanation fidelity, and human-centered usability.\u003c/p\u003e\u003cp\u003eThis taxonomy provides an organized framework that aids researchers and practitioners in mapping specific XAI techniques to relevant use cases, as well as identifying understudied intersections, such as using XAI for real-time CI/CD monitoring or tailoring explanations for different stakeholders during project planning and resource allocation. By emphasizing important intersections, the taxonomy makes it easier to create human-centered, interpretable, and scalable AI systems that balance performance and transparency. In order to create \"trustworthy AI,\" it is essential to make sure that humans retain control over AI and that those impacted by AI systems are sufficiently aware of the hazards involved.[\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]\u003c/p\u003e\u003cp\u003eIn summary, by offering a coherent synthesis of XAI research spanning software engineering and decision-making, this analysis unifies previously separate work. It presents a theoretical framework and a practical strategy for creating interpretable, dependable, and scalable AI systems that may be applied in real-world situations. One solution that has been put up to help advance towards more transparent AI and prevent the adoption of AI in crucial sectors from being limited is explainable artificial intelligence (XAI).[\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]\u003c/p\u003e\u003cp\u003eSee Figure X: Conceptual Taxonomy of XAI in Decision-Making and Software Engineering.\u003c/p\u003e"},{"header":"5. Conclusion","content":"\u003cp\u003eThis systematic review includes 124 studies on Explainable Artificial Intelligence (XAI) in software engineering and decision-making. The findings demonstrate that XAI significantly enhances regulatory compliance, transparency, trust, and debugging effectiveness. However, problems still exist with usability, scalability, standards, and human interpretability. These limitations must be removed if XAI is to be widely and practically used in both industry and research. The focus of future research should be on developing human-centered, domain-specific, scalable XAI frameworks that are simple to integrate into real-world DevOps, CI/CD, and decision-support environments.\u003c/p\u003e\u003cp\u003e\u003cb\u003eLimitations\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eRelevant studies may have been overlooked due to the focus on English-only publications.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eThe exclusion of grey literature may have left out useful information.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003ePost-2025 approaches are not captured due to rapid evolution.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003eFuture Directions\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eConsistent standards for decision-making and explainability throughout SE.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eHuman-centered design to provide comprehensible justifications.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eFrameworks for DevOps and CI/CD environments that are domain-specific.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eMultidisciplinary growth into safety-critical systems, healthcare, and finance.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eMultimodal XAI integration to manage intricate decision-making situations.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eCompeting Interests\u003c/h2\u003e\u003cp\u003eThere are no conflicting interests, according to the authors.\u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eThe study's conception and design, the systematic review's planning and execution, the data analysis, the creation of the figures and tables, the manuscript's writing, and the final version's review and editing were all done by the corresponding author.\u003c/p\u003e\n\u003ch3\u003eData Availability Statement\u003c/h3\u003e\n\u003cp\u003eThis study does not generate or analyze any new data. Data sharing is therefore not relevant to this subject.\u003c/p\u003e\u003ch2\u003eFunding Declaration\u003c/b\u003e\u003c/p\u003e\u003cp\u003eFor this study, no specific grant from a public, private, or nonprofit funding organization was obtained.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eHuang, L., Peissl, W.: Artificial Intelligence\u0026mdash;A New Knowledge and Decision-Making Paradigm? In: Hennen, L., Hahn, J., Ladikas, M., Lindner, R., Peissl, W., van Est, R. (eds.) in Technology Assessment in a Globalized World: Facing the Challenges of Transnational Technology Governance, pp. 175\u0026ndash;201. Springer International Publishing, Cham (2023). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/978-3-031-10617-0_9\u003c/span\u003e\u003cspan address=\"10.1007/978-3-031-10617-0_9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBerente, N., Gu, B., Recker, J., Santhanam, R.: Managing Artificial Intelligence, \u003cem\u003eMIS Quarterly\u003c/em\u003e, vol. 45, no. 3, pp. 1433\u0026ndash;1450, Sept. (2021). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.25300/MISQ/2021/16274\u003c/span\u003e\u003cspan address=\"10.25300/MISQ/2021/16274\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHagras, H.: Towards Human Understandable Explainable AI\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAraujo, T., Helberger, N., Kruikemeier, S., De Vreese, C.H.: In AI we trust? Perceptions about automated decision-making by artificial intelligence, \u003cem\u003eAI \u0026amp; Soc\u003c/em\u003e, vol. 35, no. 3, pp. 611\u0026ndash;623, Sept. (2020). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s00146-019-00931-w\u003c/span\u003e\u003cspan address=\"10.1007/s00146-019-00931-w\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSchulte, L., Ledel, B., Herbold, S.: Studying the explanations for the automated prediction of bug and non-bug issues using LIME and SHAP. Empir. Softw. Eng. \u003cb\u003e29\u003c/b\u003e(4), 93 (June 2024). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s10664-024-10469-1\u003c/span\u003e\u003cspan address=\"10.1007/s10664-024-10469-1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKang, S., Chen, B., Yoo, S., Lou, J.-G.: Explainable automated debugging via large language model-driven scientific debugging, \u003cem\u003eEmpir Software Eng\u003c/em\u003e, vol. 30, no. 2, p. 45, Dec. (2024). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s10664-024-10594-x\u003c/span\u003e\u003cspan address=\"10.1007/s10664-024-10594-x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDurrani, U.K., Akpinar, M., Fatih Adak, M., Talha Kabakus, A., Maruf, M., \u0026Ouml;zt\u0026uuml;rk, Saleh, M.: A Decade of Progress: A Systematic Literature Review on the Integration of AI in Software Engineering Phases and Activities (2013\u0026ndash;2023). IEEE Access. \u003cb\u003e12\u003c/b\u003e, 171185\u0026ndash;171204 (2024). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2024.3488904\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2024.3488904\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAI for DevSecOps: A Landscape and Future Opportunities | ACM Transactions on Software Engineering and Methodology. Accessed: Oct. 18, 2025. [Online]. Available: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://dl.acm.org/doi/abs/\u003c/span\u003e\u003cspan address=\"https://dl.acm.org/doi/abs/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1145/3712190\u003c/span\u003e\u003cspan address=\"10.1145/3712190\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePaliwal, G., Kumar, A., Sharma, K.P., Bhargava, D., Shrimal, V.M.: Transformative impact of explainable artificial intelligence: bridging complexity and trust. Discov Artif. Intell. \u003cb\u003e5\u003c/b\u003e(1), 51 (May 2025). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s44163-025-00281-1\u003c/span\u003e\u003cspan address=\"10.1007/s44163-025-00281-1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHermosilla, P., Berr\u0026iacute;os, S., Allende-Cid, H.: Explainable AI for Forensic Analysis: A Comparative Study of SHAP and LIME in Intrusion Detection Models. Appl. Sci. \u003cb\u003e15\u003c/b\u003e(13), 7329 (June 2025). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/app15137329\u003c/span\u003e\u003cspan address=\"10.3390/app15137329\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMathew, D.E., Ebem, D.U., Ikegwu, A.C., Ukeoma, P.E., Dibiaezue, N.F.: Recent Emerging Techniques in Explainable Artificial Intelligence to Enhance the Interpretable and Understanding of AI Models for Human. Neural Process. Lett. \u003cb\u003e57\u003c/b\u003e(1), 16 (Feb. 2025). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s11063-025-11732-2\u003c/span\u003e\u003cspan address=\"10.1007/s11063-025-11732-2\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePandiyavathi, T., Sivakumar, B.: DevOps Challenges and Practices in Software Engineering. In: Raj, J.S., Perikos, I., Balas, V.E. (eds.) in Intelligent Sustainable Systems, pp. 49\u0026ndash;57. Singapore: Springer Nature (2023). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/978-981-99-1726-6_5\u003c/span\u003e\u003cspan address=\"10.1007/978-981-99-1726-6_5\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAlbahri, A.S., et al.: A systematic review of trustworthy artificial intelligence applications in natural disasters. Comput. Electr. Eng. \u003cb\u003e118\u003c/b\u003e, 109409 (Sept. 2024). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.compeleceng.2024.109409\u003c/span\u003e\u003cspan address=\"10.1016/j.compeleceng.2024.109409\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSaeed, W., Omlin, C.: Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities. Knowl. Based Syst. \u003cb\u003e263\u003c/b\u003e, 110273 (Mar. 2023). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.knosys.2023.110273\u003c/span\u003e\u003cspan address=\"10.1016/j.knosys.2023.110273\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Software Engineering, Code Analysis, Interpretable Machine Learning, Automated Software Testing, Continuous Integration, SHAP, LIME, DevOps, Explainable Artificial Intelligence (XAI), Decision Support Systems, Risk Evaluation and Optimization","lastPublishedDoi":"10.21203/rs.3.rs-7947674/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7947674/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eParticularly in domains where accountability, openness, and trust are critical, artificial intelligence (XAI) has become a vital subject of study. Serious worries over the \"black-box\" nature of these algorithms have been raised by the increasing use of deep learning and machine learning models in software engineering and decision-making. When interpretability is poor, there are risks in automated decision support, software quality assurance, debugging, and regulatory compliance. In this study, XAI methodologies used in software engineering and decision-making from 2010 to 2025 are systematically evaluated using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) framework. 705 records were found after searching Scopus, Web of Science, IEEE Xplore, ACM Digital Library, PubMed, and Google Scholar. 98 of these records were included after a rigorous screening procedure. The findings indicate that the most popular strategies in the sector include rule-based techniques, attention-based architectures, SHAP, and LIME. Decision support, risk assessment, optimization, performance evaluation, automated testing, and bug prediction are a few examples of applications. Despite tremendous progress, standardizing assessment metrics, guaranteeing scalability, and incorporating XAI into DevOps pipelines are still difficult problems. This paper creates a conceptual taxonomy of XAI in software engineering and decision-making, brings diverse knowledge together, and lays out a research agenda to create AI solutions that are human-centered, interpretable, and scalable. This paper fills a major research vacuum by combining Explainable AI (XAI) applications from software engineering and decision-making, two fields that have been studied separately. Unlike previous fragmented evaluations, it provides a synthesis (2010\u0026ndash;2025) based on PRISMA 2020, outlining a taxonomy of XAI methods, applications, and barriers to human-centered and scaled AI solutions. Overall, being the first PRISMA-based synthesis to systematically connect Explainable AI (XAI) methods with software engineering and decision-making domains, this review fills a critical research vacuum.\u003c/p\u003e","manuscriptTitle":"Systematic Review of Explainable AI Techniques in Software Engineering and Decision-Making","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-30 05:32:34","doi":"10.21203/rs.3.rs-7947674/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e4b45390-89f9-4036-8203-9725d8574cd0","owner":[],"postedDate":"October 30th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-12-01T03:23:13+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-30 05:32:34","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7947674","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7947674","identity":"rs-7947674","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}