Accountable Deployment of Agentic AI Demands Layered, System-Level Interpretability

Accountable Deployment of Agentic AI Demands Layered, System-Level Interpretability | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Accountable Deployment of Agentic AI Demands Layered, System-Level Interpretability Judy Zhu*, Dhari Gandhi*, Ahmad Rezaie Mianroodi, Dhanesh Ramachandram, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8802251/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Agentic AI systems behave through trajectories: they plan, invoke tools, update memory, and coordinate over multiple steps. However, interpretability remains largely model-centric, focused on explaining single predictions rather than tracing long-horizon behavior and responsibility across interacting components. As a result, critical failures, such as tool misuse, coordination breakdowns, or goal drift, often evade existing audits until harm occurs. We argue that interpretability for agentic systems must become system-centric, addressing trajectories, responsibility assignment, and lifecycle dynamics rather than internal model mechanisms alone. We advance three claims: interpretability must (1) co-evolve with agentic capabilities, (2) address distinct layers of opacity with tailored methods, and (3) integrate across the deployment lifecycle. To operationalize this position, we introduce ATLIS (Agentic Trajectory and Layered Interpretability Stack) , a framework integrating five interpretability layers across a five-stage deployment lifecycle. ATLIS enables lightweight continuous monitoring with risk-aware escalation to deeper system-level analysis when incidents are detected. ATLIS provides a blueprint for closing the growing gap between agentic capabilities and the interpretability infrastructure needed to govern them. Artificial Intelligence and Machine Learning Machine Learning Agentic AI Agentic Systems Interpretability Explainable AI Transparent AI AI Safety Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8802251","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":586689647,"identity":"12e07745-337d-4b00-83cd-27d55c57773a","order_by":0,"name":"Judy Zhu*","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA0klEQVRIiWNgGAWjYFCCAwwSDBX/5cDMB8RrOcNsDGYmEGuPBGMbc2IDiEWUFvnGM4a3edvY0ueHHX4ItMVOTreBgBbGhjPG1jzneHI33k4zAGpJNjY7QEALM8MZM2meMoncjbMTQFoOJG4jpIUNrIXNIN1wdvoH4rTwgLW0JSTIS+cQaYsEw7FiyzlnDhhukM4pOJBgQIRf5Gcc3njjTcUBefnZ6Zs/fKiwkyOohUHiAAMTD5A2AKs0IKQcBPgbGBh/gKxrIEb1KBgFo2AUjEgAAMKnRsAC054rAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0009-0007-2490-4476","institution":"Vector Institute for Artificial Intelligence","correspondingAuthor":true,"prefix":"","firstName":"Judy","middleName":"","lastName":"Zhu*","suffix":""},{"id":586689648,"identity":"ec78d868-a325-4a2d-bb99-a2b1d96f63f0","order_by":1,"name":"Dhari Gandhi*","email":"","orcid":"","institution":"Vector Institute for Artificial Intelligence","correspondingAuthor":false,"prefix":"","firstName":"Dhari","middleName":"","lastName":"Gandhi*","suffix":""},{"id":586689649,"identity":"7e58f634-7998-439a-b68c-f13c2b4b37e6","order_by":2,"name":"Ahmad Rezaie Mianroodi","email":"","orcid":"","institution":"Dalhousie University","correspondingAuthor":false,"prefix":"","firstName":"Ahmad","middleName":"Rezaie","lastName":"Mianroodi","suffix":""},{"id":586689650,"identity":"a04e210b-eb8d-4f35-8089-cc8e4626f48f","order_by":3,"name":"Dhanesh Ramachandram","email":"","orcid":"","institution":"Vector Institute for Artificial Intelligence","correspondingAuthor":false,"prefix":"","firstName":"Dhanesh","middleName":"","lastName":"Ramachandram","suffix":""},{"id":586689651,"identity":"be4f6404-8fa2-4d56-92f5-526dfd6e7d57","order_by":4,"name":"Sedef Akinli Kocak","email":"","orcid":"","institution":"Vector Institute for Artificial Intelligence","correspondingAuthor":false,"prefix":"","firstName":"Sedef","middleName":"Akinli","lastName":"Kocak","suffix":""},{"id":586689652,"identity":"64beb965-320b-4d59-8c93-ce3cdfacb4e7","order_by":5,"name":"Shaina Raza","email":"","orcid":"","institution":"Vector Institute for Artificial Intelligence","correspondingAuthor":false,"prefix":"","firstName":"Shaina","middleName":"","lastName":"Raza","suffix":""}],"badges":[],"createdAt":"2026-02-06 03:40:47","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8802251/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8802251/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":102172486,"identity":"9f0d00cc-038c-400f-aa03-41e21acbcd79","added_by":"auto","created_at":"2026-02-09 05:00:19","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":535278,"visible":true,"origin":"","legend":"","description":"","filename":"AccountableDeploymentofAgenticAIDemandsLayeredSystemLevelInterpretability.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8802251/v1_covered_87eb76ae-15e8-4da4-a1ca-6bc2f4511be8.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eAccountable Deployment of Agentic AI Demands Layered, System-Level Interpretability\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Vector Institute","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Machine Learning, Agentic AI, Agentic Systems, Interpretability, Explainable AI, Transparent AI, AI Safety","lastPublishedDoi":"10.21203/rs.3.rs-8802251/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8802251/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eAgentic AI systems behave through trajectories: they plan, invoke tools, update memory, and coordinate over multiple steps. However, interpretability remains largely model-centric, focused on explaining single predictions rather than tracing long-horizon behavior and responsibility across interacting components. As a result, critical failures, such as tool misuse, coordination breakdowns, or goal drift, often evade existing audits until harm occurs. \u003cstrong\u003eWe argue that interpretability for agentic systems must become system-centric, addressing trajectories, responsibility assignment, and lifecycle dynamics rather than internal model mechanisms alone. \u0026nbsp;\u003c/strong\u003eWe advance three claims: interpretability must (1) co-evolve with agentic capabilities, (2) address distinct layers of opacity with tailored methods, and (3) integrate across the deployment lifecycle. To operationalize this position, we introduce \u003cstrong\u003eATLIS (Agentic Trajectory and Layered Interpretability Stack)\u003c/strong\u003e, a framework integrating five interpretability layers across a five-stage deployment lifecycle. ATLIS enables lightweight continuous monitoring with risk-aware escalation to deeper system-level analysis when incidents are detected. ATLIS provides a blueprint for closing the growing gap between agentic capabilities and the interpretability infrastructure needed to govern them.\u003c/p\u003e","manuscriptTitle":"Accountable Deployment of Agentic AI Demands Layered, System-Level Interpretability","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-09 05:00:08","doi":"10.21203/rs.3.rs-8802251/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"68de1da8-dad1-4678-8329-9f1a4b038f6d","owner":[],"postedDate":"February 9th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":62420668,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2026-02-09T05:00:08+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-09 05:00:08","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8802251","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8802251","identity":"rs-8802251","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}