The Informational Cost of Agency: A Bounded Measurement Layer for Deployed Reinforcement Learning

The Informational Cost of Agency: A Bounded Measurement Layer for Deployed Reinforcement Learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article The Informational Cost of Agency: A Bounded Measurement Layer for Deployed Reinforcement Learning Wael Hafez, Cameron Reid, Amir Nazeri This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9683103/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Deployed reinforcement learning systems lack a principled runtime reliability theory. Reward-based monitoring reacts only after degradation accumulates, and model-internal diagnostics are often unavailable. Bipredictability (𝑃), the fraction of the observation–action–outcome uncertainty budget converted into shared predictability across the interaction loop, provides a structural alternative. 𝑃 admits a provable classical upper bound 𝑃≤0.5, derived directly from Shannon entropy subadditivity. Responsive agency suppresses 𝑃 strictly below this ceiling — the informational cost of agency — which we observe empirically at 𝑃=0.33±0.02 across 21 trained continuous-control agents. This signature is not an artifact of a particular discretization or environment; the same ratio and suppression regime have been independently confirmed in classical mechanical systems, convolutional vision networks, and multi-turn language models, and the framework extends theoretically to quantum-entangled systems where the bound is relaxed to 𝑃 ≤ 1, indicating that 𝑃 captures a substrate-independent structural property. The Information Digital Twin (IDT), a model-agnostic sidecar that computes 𝑃 from the external interaction stream, detected 89.3% of coupling degradations versus 44.0% for reward-based monitoring across 168 perturbation trials in high-dimensional continuous control, with 4.4× lower median latency. Bipredictability thus establishes the prerequisite measurement signal for closed-loop self-regulation in deployed agents — a necessary foundation for runtime reliability engineering in autonomous systems. Artificial Intelligence and Machine Learning Reinforcement Learning Information Theory Bipredictability AI Drift Detection Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9683103","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":638489500,"identity":"44030bcd-611f-47f7-bbd4-667027f3cd8c","order_by":0,"name":"Wael Hafez","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAwklEQVRIiWNgGAWjYFACHgaGCgYGORK1nGFgMCZdS2ID0RoMDvAe/HCgoi59fnt34ufKNgZ5frEDhLTwJUscOMOWu+HM2c2SZ84wGM6cnYBfi9kBHgPpj208uRskcjdINlQwJBjcJqzF+MfBfxLp8vPfbv7ZYECcFjOJgw0GCQw3eLcRZ4v9YR4ziwPHEgw3nMndZtlwRoKwXyTbe4xvHKipk5dvP7v5ZmObjTy/NAEtDMyoXAkCykfBKBgFo2AUEAUAep5DoeZOQPYAAAAASUVORK5CYII=","orcid":"","institution":"Semarx Research LLC","correspondingAuthor":true,"prefix":"","firstName":"Wael","middleName":"","lastName":"Hafez","suffix":""},{"id":638491749,"identity":"1133281d-d8f8-4ebb-9b55-1faf7de67374","order_by":1,"name":"Cameron Reid","email":"","orcid":"","institution":"Semarx Research LLC","correspondingAuthor":false,"prefix":"","firstName":"Cameron","middleName":"","lastName":"Reid","suffix":""},{"id":638491750,"identity":"0b598f44-d259-4555-ba04-fa641c94c371","order_by":2,"name":"Amir Nazeri","email":"","orcid":"","institution":"Semarx Research LLc","correspondingAuthor":false,"prefix":"","firstName":"Amir","middleName":"","lastName":"Nazeri","suffix":""}],"badges":[],"createdAt":"2026-05-11 18:02:53","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-9683103/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9683103/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":109142463,"identity":"4efb8122-8f4b-447f-88bd-f9a50ea68bd2","added_by":"auto","created_at":"2026-05-13 02:56:45","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":384746,"visible":true,"origin":"","legend":"","description":"","filename":"SMXRL.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9683103/v1_covered_67dcd9f7-12ab-4cbf-8dd5-a4d05db6abd2.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eThe Informational Cost of Agency: A Bounded Measurement Layer for Deployed Reinforcement Learning\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Semarx Research LLC","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Reinforcement Learning, Information Theory, Bipredictability, AI Drift Detection","lastPublishedDoi":"10.21203/rs.3.rs-9683103/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9683103/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eDeployed reinforcement learning systems lack a principled runtime reliability theory. Reward-based monitoring reacts only after degradation accumulates, and model-internal diagnostics are often unavailable. Bipredictability (𝑃), the fraction of the observation–action–outcome uncertainty budget converted into shared predictability across the interaction loop, provides a structural alternative. 𝑃 admits a provable classical upper bound 𝑃≤0.5, derived directly from Shannon entropy subadditivity. Responsive agency suppresses 𝑃 strictly below this ceiling — the informational cost of agency — which we observe empirically at 𝑃=0.33±0.02 across 21 trained continuous-control agents. This signature is not an artifact of a particular discretization or environment; the same ratio and suppression regime have been independently confirmed in classical mechanical systems, convolutional vision networks, and multi-turn language models, and the framework extends theoretically to quantum-entangled systems where the bound is relaxed to 𝑃 ≤ 1, indicating that 𝑃 captures a substrate-independent structural property. The Information Digital Twin (IDT), a model-agnostic sidecar that computes 𝑃 from the external interaction stream, detected 89.3% of coupling degradations versus 44.0% for reward-based monitoring across 168 perturbation trials in high-dimensional continuous control, with 4.4× lower median latency. Bipredictability thus establishes the prerequisite measurement signal for closed-loop self-regulation in deployed agents — a necessary foundation for runtime reliability engineering in autonomous systems.\u003c/p\u003e","manuscriptTitle":"The Informational Cost of Agency: A Bounded Measurement Layer for Deployed Reinforcement Learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-05-13 02:55:49","doi":"10.21203/rs.3.rs-9683103/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"4125f34d-cd79-4118-9ce0-343e6924e357","owner":[],"postedDate":"May 13th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":67954896,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2026-05-13T02:55:49+00:00","versionOfRecord":[],"versionCreatedAt":"2026-05-13 02:55:49","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9683103","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9683103","identity":"rs-9683103","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}