Functional Trust Regions (FTR): A Lagrangian Framework for Stability-Constrained Continual Learning

Functional Trust Regions (FTR): A Lagrangian Framework for Stability-Constrained Continual Learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Functional Trust Regions (FTR): A Lagrangian Framework for Stability-Constrained Continual Learning Kavya Bhand, Aadi Joshi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9205833/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The stability-plasticity tradeoff in continual learning is widely assumed to be architecture-dependent: models with higher loss-landscape curvature or larger parameter counts should require stronger regularization. We empirically challenge this assumption through extensive experiments on Functional Trust Regions (FTR), a method that enforces explicit KL-divergence constraints on functional drift during sequential task learning. Conducting 1,200 experiments across eight architectures spanning a twenty-four-fold parameter range and forty-eight-fold variation in Hessian trace, we identify a stability crossover at ε ∗ = 7.15 ± 0.35 (coefficient of variation: 4.96 percent) that is architecture-independent to measurement precision. A formal F-test for constancy yields p = 0.786, indicating that between-architecture variance is statistically indistinguishable from measurement noise. Crucially, all ten tested curvature-based normalizations including Hessian trace, Fisher trace, spectral norm, and effective dimensionality increase cross-architecture dispersion rather than reduce it. No curvature metric achieves statistically significant correlation with ε ∗ (all p > 0.06). Cross-method analysis reveals that Learning without Forgetting (LwF) exhibits moderately architecturedependent transitions (coefficient of variation approximately 14 percent), while Elastic Weight Consolidation (EWC) shows no phase transition across four orders of magnitude of regularization strength. These results indicate that stability crossovers in distillation-based constrained learning arise from task structure rather than model geometry, and that widely accepted curvature-based intuitions fail to predict the critical stability budget. Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9205833","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":611040647,"identity":"7f20300a-893f-4b35-9179-c01c63a262a2","order_by":0,"name":"Kavya Bhand","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABFElEQVRIiWNgGAWjYBACCQh1IAFIGBz4YGAjB+Y+IFKL4cMZFWnGcC4xWoyNec4cSmwA8fFpkWw/Yybxg+FOHv/s5m2SM9sOpM8PO/wQaIKdnG4Ddi3SPDlmkj0Mz4ol7hwrk/jYdid34+00A6CWZGOzA9i1yDGkpUnwMBxObLgB1Duz7VnuxtkJIC0HErfh0sL/LE3yD1DLfKAWad62w+mGs9M/4NUiLZF8TBpky4YbOSDvH06Ql87Bb4vkjMeHrWUMniVuvJFWCApkww3SOQUHEgxw+0XifGLjzTcVdxLn3UjeAIpKefnZ6Zs/fKiwk8OlBQhYJBgMkLgGYJUG2NVCAfMHFK58A17Vo2AUjIJRMAIBAL0Mbc/ubFBfAAAAAElFTkSuQmCC","orcid":"","institution":"Vishwakarma Institute of Technology","correspondingAuthor":true,"prefix":"","firstName":"Kavya","middleName":"","lastName":"Bhand","suffix":""},{"id":611040648,"identity":"2a165573-6da8-40ae-a95f-82f253aa938b","order_by":1,"name":"Aadi Joshi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA5ElEQVRIiWNgGAWjYLACHgM2BgYJxsYHIDYfQdVIWpoNQGw24rSACAkGNgkQTVCLPXvzww9vCvjk5Gc3t1V+zbGTYWNgfvjoBj5beI4ZS84xYDM2uHOw7bbstmSgw9iMjXPwaZFIMJAG+iVxg0Ri223JbcxALTxs0vi1pH/+DdIyf0ZiW7HktnpitOSYgW1puJHYxvhx22EitJw5U2YJ9suNxGZpxm3HediYCfiFvb198403f47Jyc9If/jx57Zqe35gGD7GpwUKjoFJZnAEMRNWDgI1YJLxB3GqR8EoGAWjYIQBAMMiQS59F638AAAAAElFTkSuQmCC","orcid":"","institution":"Vishwakarma Institute of Technology","correspondingAuthor":true,"prefix":"","firstName":"Aadi","middleName":"","lastName":"Joshi","suffix":""}],"badges":[],"createdAt":"2026-03-24 02:40:37","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-9205833/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9205833/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":105565722,"identity":"951f658c-c977-4384-a340-f39d87914c44","added_by":"auto","created_at":"2026-03-27 12:54:10","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":247252,"visible":true,"origin":"","legend":"","description":"","filename":"SSMLIS.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9205833/v1_covered_73d2a4b8-e639-4cc8-91bc-751957c1c80c.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eFunctional Trust Regions (FTR): A Lagrangian Framework for Stability-Constrained Continual Learning\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-9205833/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9205833/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThe stability-plasticity tradeoff in continual learning is widely assumed to be architecture-dependent: models with higher loss-landscape curvature or larger parameter counts should require stronger regularization. We empirically challenge this assumption through extensive experiments on Functional Trust Regions (FTR), a method that enforces explicit KL-divergence constraints on functional drift during sequential task learning. Conducting 1,200 experiments across eight architectures spanning a twenty-four-fold parameter range and forty-eight-fold variation in Hessian trace, we identify a stability crossover at ε ∗ = 7.15 ± 0.35 (coefficient of variation: 4.96 percent) that is architecture-independent to measurement precision. A formal F-test for constancy yields p = 0.786, indicating that between-architecture variance is statistically indistinguishable from measurement noise. Crucially, all ten tested curvature-based normalizations including Hessian trace, Fisher trace, spectral norm, and effective dimensionality increase cross-architecture dispersion rather than reduce it. No curvature metric achieves statistically significant correlation with ε ∗ (all p \u0026gt; 0.06). Cross-method analysis reveals that Learning without Forgetting (LwF) exhibits moderately architecturedependent transitions (coefficient of variation approximately 14 percent), while Elastic Weight Consolidation (EWC) shows no phase transition across four orders of magnitude of regularization strength. These results indicate that stability crossovers in distillation-based constrained learning arise from task structure rather than model geometry, and that widely accepted curvature-based intuitions fail to predict the critical stability budget.\u003c/p\u003e","manuscriptTitle":"Functional Trust Regions (FTR): A Lagrangian Framework for Stability-Constrained Continual Learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-25 08:58:04","doi":"10.21203/rs.3.rs-9205833/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e64e3b17-eb4d-44e1-87ec-7200a8b59591","owner":[],"postedDate":"March 25th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-03-25T08:58:04+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-25 08:58:04","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9205833","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9205833","identity":"rs-9205833","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}