K–R Excitation–Regulation Learning: A Stability-Driven Framework for Robust and Generalizable Vision Transformers

K–R Excitation–Regulation Learning: A Stability-Driven Framework for Robust and Generalizable Vision Transformers | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article K–R Excitation–Regulation Learning: A Stability-Driven Framework for Robust and Generalizable Vision Transformers RamaKrishna Pasupuleti This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8918824/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Vision Transformers (ViTs) have achieved strong performance in visual recognition tasks, yet they often exhibit unstable representation dynamics, sensitivity to perturbations, and limited generalization under distribution shifts. These limitations arise from optimization processes that prioritize predictive accuracy without explicitly controlling feature evolution and stability. To address this gap, we propose a stability-driven learning framework termed K–R Excitation–Regulation Learning , which introduces nonlinear excitation and regulation mechanisms to guide representation dynamics toward equilibrium. The proposed framework models feature evolution as a dynamical process in which excitation enhances nonlinear feature interactions while regulation constrains representation drift, enabling stable embedding formation. A stability-constrained objective inspired by equilibrium principles is integrated into standard training, promoting balanced excitation–regulation behavior during learning. Unlike conventional architecture modifications, the K–R formulation directly governs representation dynamics, improving learning consistency and robustness. Extensive experiments demonstrate that the proposed method improves representation stability and generalization across multiple evaluation conditions. Specifically, K–R reduces feature drift, exhibits more controlled responses under noise perturbations, improves performance under distribution shifts and few-shot learning settings, and shows superior scaling behavior as training progresses. Notably, these gains are achieved while maintaining competitive predictive accuracy with standard ViT baselines. These findings suggest that stability-driven learning offers a principled alternative to purely optimization-based training, enabling more robust and generalizable representation learning. The K–R framework provides a new perspective on integrating dynamical systems principles into deep learning, highlighting the importance of controlled feature evolution for reliable visual recognition. K–R Excitation–Regulation learning stability-driven learning Vision Transformers representation stability nonlinear representation learning feature drift analysis dynamical systems in deep learning stability-constrained optimization domain generalization noise-robust learning few-shot learning and scaling behavior in deep learning Full Text Additional Declarations No competing interests reported. Supplementary Files Suplementary2.zip Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8918824","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":593999512,"identity":"a700ed14-7ca4-42ca-b2ca-b073515d9041","order_by":0,"name":"RamaKrishna Pasupuleti","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA9ElEQVRIiWNgGAWjYHACZoYEBjkgfbD9wwcgxcZOnBZjBgbGw22MM0BamInRwgDSwny8jZkHxscHDI73PjZ48McgcTvbwbbHNr+2yfMxMzB++JiDR8uZ48YJiW0GiTt7DrYb5/bdNmxjZmCWnLkNtxazG2nMBxIb/iRuuHGwQTq35zYjUAsbMy8hLQlAh224/7BB2rLntj1RWhIS2IBaDhxsk2b4cTuRoBb7M8eYDYB+MQZqaTbsbbid3MbM2IzXL5LtbcySP/4YyG44cPzhgx9/btvOb28++OEjHi2ogLENTDYQqx4E/pCieBSMglEwCkYKAAAdgVk9CekE7wAAAABJRU5ErkJggg==","orcid":"","institution":"Kakatiya University","correspondingAuthor":true,"prefix":"","firstName":"RamaKrishna","middleName":"","lastName":"Pasupuleti","suffix":""}],"badges":[],"createdAt":"2026-02-19 15:42:22","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8918824/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8918824/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":109280517,"identity":"12de37d5-317b-4268-8261-9b8b69cccfd7","added_by":"auto","created_at":"2026-05-14 17:13:38","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":354120,"visible":true,"origin":"","legend":"","description":"","filename":"Manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8918824/v1_covered_e5df09e1-0b3b-4225-83d5-e83fee65d159.pdf"},{"id":103063059,"identity":"043442c6-6fe1-4394-85bd-08e0e7d3e502","added_by":"auto","created_at":"2026-02-20 10:39:26","extension":"zip","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":114450,"visible":true,"origin":"","legend":"","description":"","filename":"Suplementary2.zip","url":"https://assets-eu.researchsquare.com/files/rs-8918824/v1/f9d7ceccf5e7924127882a67.zip"}],"financialInterests":"No competing interests reported.","formattedTitle":"K–R Excitation–Regulation Learning: A Stability-Driven Framework for Robust and Generalizable Vision Transformers","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"K–R Excitation–Regulation learning, stability-driven learning, Vision Transformers, representation stability, nonlinear representation learning, feature drift analysis, dynamical systems in deep learning, stability-constrained optimization, domain generalization, noise-robust learning, few-shot learning, and scaling behavior in deep learning","lastPublishedDoi":"10.21203/rs.3.rs-8918824/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8918824/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eVision Transformers (ViTs) have achieved strong performance in visual recognition tasks, yet they often exhibit unstable representation dynamics, sensitivity to perturbations, and limited generalization under distribution shifts. These limitations arise from optimization processes that prioritize predictive accuracy without explicitly controlling feature evolution and stability. To address this gap, we propose a stability-driven learning framework termed \u003cstrong\u003eK–R Excitation–Regulation Learning\u003c/strong\u003e, which introduces nonlinear excitation and regulation mechanisms to guide representation dynamics toward equilibrium.\u003c/p\u003e\n\u003cp\u003eThe proposed framework models feature evolution as a dynamical process in which excitation enhances nonlinear feature interactions while regulation constrains representation drift, enabling stable embedding formation. A stability-constrained objective inspired by equilibrium principles is integrated into standard training, promoting balanced excitation–regulation behavior during learning. Unlike conventional architecture modifications, the K–R formulation directly governs representation dynamics, improving learning consistency and robustness.\u003c/p\u003e\n\u003cp\u003eExtensive experiments demonstrate that the proposed method improves representation stability and generalization across multiple evaluation conditions. Specifically, K–R reduces feature drift, exhibits more controlled responses under noise perturbations, improves performance under distribution shifts and few-shot learning settings, and shows superior scaling behavior as training progresses. Notably, these gains are achieved while maintaining competitive predictive accuracy with standard ViT baselines.\u003c/p\u003e\n\u003cp\u003eThese findings suggest that stability-driven learning offers a principled alternative to purely optimization-based training, enabling more robust and generalizable representation learning. The K–R framework provides a new perspective on integrating dynamical systems principles into deep learning, highlighting the importance of controlled feature evolution for reliable visual recognition.\u003c/p\u003e","manuscriptTitle":"K–R Excitation–Regulation Learning: A Stability-Driven Framework for Robust and Generalizable Vision Transformers","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-20 10:39:13","doi":"10.21203/rs.3.rs-8918824/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"dafb1023-9678-48dc-92fe-bcfd0983ee2d","owner":[],"postedDate":"February 20th, 2026","published":true,"recentEditorialEvents":[{"type":"decision","content":"Rejected","date":"2026-05-14T16:53:14+00:00","index":"","fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-05-14T17:08:24+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-20 10:39:13","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8918824","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8918824","identity":"rs-8918824","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}