Stability in Continual Learning as a Global Geometric Phenomenon: A Representation Manifold Perspective

Stability in Continual Learning as a Global Geometric Phenomenon: A Representation Manifold Perspective | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Stability in Continual Learning as a Global Geometric Phenomenon: A Representation Manifold Perspective Yukun Feng This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9316295/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Catastrophic forgetting in continual learning is commonly attributed toparameter drift or task interference, and most existing methods attempt to address this problem through local stabilization mechanisms such as regularization, replay, or parameter isolation. In this paper, we propose a different viewpoint: forgetting is fundamentally a geometric instability of the representation manifold. From this perspective, stable continual learning requires not only controlling local parameter updates, but maintaining the global geometric consistency of the representation space during sequential learning. To formalize this idea, we introduce a global geometric constraint,termed \emph{Global Angular--Momentum Neutrality (GAN)},which acts on the overall curvature structure of the representation manifold. We construct a geometric loss consisting of a task term,a local curvature term, and a global constraint term, and investigate its effect on representation drift and for getting through systematic parameter-scan experiments. The experiments reveal a clear geometric phase structure:when the global constraint is too weak, the representation manifold under goeslarge-scale reconstruction; when the constraint lies within a finite interval,the system enters a geometrically stable regime characterized by low curvature, low representation drift, and low forgetting;when the constraint becomes too strong, the system enters an over-constrained regime, and stability degrades again. This trend persists across different structural channels, indicating that the stabilization mechanism is structural rather than model-specific. These results suggest a geometric stability principle for continual learning: existing methods mainly provide local stabilization, while global geometric constraints regulate curvature balance at the manifold level, thereby stabilizing the representation structure and reducing forgetting. Continual learning stability should therefore be understood as a\emph{global geometric phenomenon} governed by the large-scale structure of the representation manifold. Physical sciences/Mathematics and computing Physical sciences/Physics Continual Learning Catastrophic Forgetting Representation Manifold Learning Dynamics Geometric Stability Global Regularization Information Geometry Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9316295","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":628126296,"identity":"710e7af1-086b-43b0-9738-008761004e56","order_by":0,"name":"Yukun Feng","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAtElEQVRIiWNgGAWjYDCCA4wPgKQNhMNDnBZmAyCZRrqWwyRo4bt9mPFxwa/zefzTDh9geFNBhBbJc8nMxjP7bhdL3E5LYJxzhggtBmf4j0nz9txObLidY8DM20aUFmb237w95xLng7X8I04LGzPPjwOJG8BaGojQInmGmVmatyE5cSPQLwfnHCNCC98ZZsbPPH/sEufdTj744E0NEVrAgBHq6wPEagCCPySoHQWjYBSMgpEHAEvIOXZSuM8oAAAAAElFTkSuQmCC","orcid":"","institution":"Independent Researcher","correspondingAuthor":true,"prefix":"","firstName":"Yukun","middleName":"","lastName":"Feng","suffix":""}],"badges":[],"createdAt":"2026-04-03 21:53:15","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9316295/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9316295/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":108492496,"identity":"fb07cbf7-f4bf-4cf1-a688-00802ea6b56f","added_by":"auto","created_at":"2026-05-05 09:57:55","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":699061,"visible":true,"origin":"","legend":"","description":"","filename":"GeometricStabilityinContinualLearningv7.1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9316295/v1_covered_b9bc823a-c307-41ff-a2d6-e80c3cb22251.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"\u003cp\u003eStability in Continual Learning as a Global Geometric Phenomenon: A Representation Manifold Perspective\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Continual Learning, Catastrophic Forgetting, Representation Manifold, Learning Dynamics, Geometric Stability, Global Regularization, Information Geometry","lastPublishedDoi":"10.21203/rs.3.rs-9316295/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9316295/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eCatastrophic forgetting in continual learning is commonly attributed toparameter drift or task interference, and most existing methods attempt to address this problem through local stabilization mechanisms such as regularization, replay, or parameter isolation. In this paper, we propose a different viewpoint: forgetting is fundamentally a geometric instability of the representation manifold. From this perspective, stable continual learning requires not only controlling local parameter updates, but maintaining the global geometric consistency of the representation space during sequential learning.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTo formalize this idea, we introduce a global geometric constraint,termed \\emph{Global Angular--Momentum Neutrality (GAN)},which acts on the overall curvature structure of the representation manifold. We construct a geometric loss consisting of a task term,a local curvature term, and a global constraint term, and investigate its effect on representation drift and for getting through systematic parameter-scan experiments.\u003c/p\u003e\n\u003cp\u003eThe experiments reveal a clear geometric phase structure:when the global constraint is too weak, the representation manifold under goeslarge-scale reconstruction; when the constraint lies within a finite interval,the system enters a geometrically stable regime characterized by low curvature, low representation drift, and low forgetting;when the constraint becomes too strong, the system enters an over-constrained regime, and stability degrades again. This trend persists across different structural channels, indicating that the stabilization mechanism is structural rather than model-specific.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThese results suggest a geometric stability principle for continual learning: existing methods mainly provide local stabilization, while global geometric constraints regulate curvature balance at the manifold level, thereby stabilizing the representation structure and reducing forgetting. Continual learning stability should therefore be understood as a\\emph{global geometric phenomenon} governed by the large-scale structure of the representation manifold.\u003c/p\u003e","manuscriptTitle":"Stability in Continual Learning as a Global Geometric Phenomenon: A Representation Manifold Perspective","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-05-04 02:52:57","doi":"10.21203/rs.3.rs-9316295/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"d51d1390-9286-451e-889d-f6562d0d29c8","owner":[],"postedDate":"May 4th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":67450390,"name":"Physical sciences/Mathematics and computing"},{"id":67450391,"name":"Physical sciences/Physics"}],"tags":[],"updatedAt":"2026-05-04T02:52:57+00:00","versionOfRecord":[],"versionCreatedAt":"2026-05-04 02:52:57","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9316295","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9316295","identity":"rs-9316295","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}