Data-Driven Reduced Modeling of Recurrent Neural Networks

Data-Driven Reduced Modeling of Recurrent Neural Networks | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Data-Driven Reduced Modeling of Recurrent Neural Networks Alice Marraffa, Renate Krause, Valerio Mante, George Haller This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7214752/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted You are reading this latest preprint version Abstract Vanilla Recurrent Neural Networks (RNNs) are widely used in neuroscience to model the collective activity of neurons during behavioral tasks. Their high dimensionality and lack of interpretability, however, often make the fundamental features of their dynamics unclear. In this study, we employ recent nonlinear dynamical system techniques to uncover the core dynamics of several RNNs used in contemporary neuroscience. Specifically, using a data-driven approach, we identify Spectral Submanifolds (SSMs), i.e., low-dimensional attracting invariant manifolds tangent to the eigenspaces of fixed points. The internal dynamics of SSMs serve as nonlinear models that reduce the dimension of the full RNNs by orders of magnitude. Through low-dimensional SSM-reduced models, we give mathematically precise definitions of line and ring attractors, which are intuitive concepts commonly used to explain decision making and working memory. This new level of understanding of RNNs obtained from SSM reduction enables the interpretation of mathematically well-defined and robust structures in neuronal dynamics. Biological sciences/Computational biology and bioinformatics/Computational neuroscience Biological sciences/Computational biology and bioinformatics/Machine learning Full Text Additional Declarations There is NO Competing Interest. Supplementary Files DataDrivenReducedModelingofRecurrentNeuralNetworks13048.pdf Supplementary Materials and Methods Cite Share Download PDF Status: Under Review Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7214752","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":502767235,"identity":"9a73f49b-4f72-45a1-983f-d37a310ef14c","order_by":0,"name":"Alice Marraffa","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAy0lEQVRIiWNgGAWjYPACOX4G9gYGZhDTgEgtxpINPAcYm0nUIpFApBb+aYeffa6oMZDgn/n4+OOCPwzy5oS0SNxOM5555piBBJCR2DyDh8FwZwMhPbcTjBkb2P7UMdzOMWzmkWBIMDhAQIf87fTPjA3/DCTkb57/2MxjQIQWg9s5xoyNbQYSBjd4GJt5EojQYng7p5ixsc9AwvBMmuFsngMShhsIaZG7nb6ZseGbgYTc8cMPPvP8sZEnaAs6kCBR/SgYBaNgFIwCrAAAQCU+pxoZQTgAAAAASUVORK5CYII=","orcid":"","institution":"ETH Zurich","correspondingAuthor":true,"prefix":"","firstName":"Alice","middleName":"","lastName":"Marraffa","suffix":""},{"id":502767236,"identity":"7f7f010a-05ef-4748-9091-99597df14bdf","order_by":1,"name":"Renate Krause","email":"","orcid":"","institution":"University of Zurich \u0026 ETH Zurich","correspondingAuthor":false,"prefix":"","firstName":"Renate","middleName":"","lastName":"Krause","suffix":""},{"id":502767237,"identity":"6ca8478c-e7f6-4c86-9e97-b2ba57efadca","order_by":2,"name":"Valerio Mante","email":"","orcid":"","institution":"University of Zurich \u0026 ETH Zurich","correspondingAuthor":false,"prefix":"","firstName":"Valerio","middleName":"","lastName":"Mante","suffix":""},{"id":502767238,"identity":"ee205405-c2db-4248-9a77-3f1519722494","order_by":3,"name":"George Haller","email":"","orcid":"","institution":"ETH Zurich","correspondingAuthor":false,"prefix":"","firstName":"George","middleName":"","lastName":"Haller","suffix":""}],"badges":[],"createdAt":"2025-07-25 13:26:01","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7214752/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7214752/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":92921879,"identity":"8bbfcc69-9d0f-41ef-bf65-00a03d74593d","added_by":"auto","created_at":"2025-10-07 07:09:58","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":7316757,"visible":true,"origin":"","legend":"Article File","description":"","filename":"DataDrivenReducedModelingofRecurrentNeuralNetworks2129.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7214752/v1_covered_dff7fee7-2ebb-4af4-8376-6f47de0b5503.pdf"},{"id":92921116,"identity":"02a39630-51d1-4d9b-9d61-ef0218dff8d8","added_by":"auto","created_at":"2025-10-07 07:01:24","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":3870374,"visible":true,"origin":"","legend":"Supplementary Materials and Methods","description":"","filename":"DataDrivenReducedModelingofRecurrentNeuralNetworks13048.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7214752/v1/7c4ae932f41a8b0767fe68fb.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Data-Driven Reduced Modeling of Recurrent Neural Networks","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7214752/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7214752/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Vanilla Recurrent Neural Networks (RNNs) are widely used in neuroscience to model the collective activity of neurons during behavioral tasks. Their high dimensionality and lack of interpretability, however, often make the fundamental features of their dynamics unclear. \r\n\r\n\r\nIn this study, we employ recent nonlinear dynamical system techniques to uncover the core dynamics of several RNNs used in contemporary neuroscience. Specifically, using a data-driven approach, we identify Spectral Submanifolds (SSMs), i.e., low-dimensional attracting invariant manifolds tangent to the eigenspaces of fixed points. The internal dynamics of SSMs serve as nonlinear models that reduce the dimension of the full RNNs by orders of magnitude. \r\n\r\nThrough low-dimensional SSM-reduced models, we give mathematically precise definitions of line and ring attractors, which are intuitive concepts commonly used to explain decision making and working memory. This new level of understanding of RNNs obtained from SSM reduction enables the interpretation of mathematically well-defined and robust structures in neuronal dynamics.","manuscriptTitle":"Data-Driven Reduced Modeling of Recurrent Neural Networks","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-07 07:01:20","doi":"10.21203/rs.3.rs-7214752/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"473c22d7-2b4b-460b-8dcd-841c55cba7ed","owner":[],"postedDate":"October 7th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":53407404,"name":"Biological sciences/Computational biology and bioinformatics/Computational neuroscience"},{"id":53407405,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"}],"tags":[],"updatedAt":"2025-10-07T07:01:20+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-07 07:01:20","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7214752","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7214752","identity":"rs-7214752","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}