Simple Recursive Model: Simplified, Single-State Reasoning with Skip Connections

Simple Recursive Model: Simplified, Single-State Reasoning with Skip Connections | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Short Report Simple Recursive Model: Simplified, Single-State Reasoning with Skip Connections Qianli Liao, Tomaso Poggio This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8492126/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Hierarchical Reasoning Models (HRM) and their variants (TRM) have shown impressive performance on a variety of reasoning tasks. These models maintain two separate states $z_L$ and $z_H$ that are designed to capture low-level and high-level representations for reasoning. However, the necessity of this dual-state strcuture remains unclear. We conjecture that the benefit of $z_H$ stems not necessarily from representing any distinct level of information, but rather from providing access to information from earlier timesteps, as $z_H$ is updated less frequently than $z_L$. To test this, we propose Simple Recursive Model (SRM): a model that maintains only a single state $z$ and adopts skip connections to earlier timesteps in replacement of the second-level state $z_H$. This approach eliminates the complicated nested loops in HRM/TRM that alternate between $z_L$ and $z_H$, leading to a more straightforward and parsimonious architecture. Through experiments on the Sudoku task, we demonstrate that the single-state SRM can achieve comparable performance to the TRM baseline. Our results suggest that the key benefit of HRM/TRM is not necessarily hierarchical state separation, but may come from the ability to access information across different timescales. Such ability can be equivalently achieved through other mechanisms such as skip connections, opening up new avenues for designing simpler and more efficient reasoning models. Code available at: https://github.com/liaoq/SimpleRecursiveModel Artificial Intelligence and Machine Learning deep learning reasoning transformers Sudoku Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8492126","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Short Report","associatedPublications":[],"authors":[{"id":567933313,"identity":"76f75584-fc17-4dde-8f30-06ad78d5c930","order_by":0,"name":"Qianli Liao","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAjklEQVRIiWNgGAWjYBACPgbmxgcMBiBmApFa2BgYmw0YDAxI09ImwcBAkhb2xLaqGwV/GPjZcwyI1MLzsO12DtBhkj1viNUikQjRYnCDaFuAWopBWuxJ0sIMtkWCBL80S+cYGPNInHlWQJwWfvbkg59z/sjJ8bcnbyBOCyw6eIhVjtAyCkbBKBgFowA3AABn6SPitT2BgwAAAABJRU5ErkJggg==","orcid":"","institution":"MIT","correspondingAuthor":true,"prefix":"","firstName":"Qianli","middleName":"","lastName":"Liao","suffix":""},{"id":567933314,"identity":"2a7e7061-8da1-4cae-abd7-dc65d85d7122","order_by":1,"name":"Tomaso Poggio","email":"","orcid":"","institution":"MIT","correspondingAuthor":false,"prefix":"","firstName":"Tomaso","middleName":"","lastName":"Poggio","suffix":""}],"badges":[],"createdAt":"2025-12-31 20:33:50","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8492126/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8492126/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":99795202,"identity":"26b214f8-30fc-491f-a93e-f4a0585be3bb","added_by":"auto","created_at":"2026-01-08 13:37:20","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":783500,"visible":true,"origin":"","legend":"","description":"","filename":"srmver21.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8492126/v1_covered_d9032857-49e2-41f1-88a5-51436c9485a4.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eSimple Recursive Model: Simplified, Single-State Reasoning with Skip Connections\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Massachusetts Institute of Technology","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"deep learning, reasoning, transformers, Sudoku","lastPublishedDoi":"10.21203/rs.3.rs-8492126/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8492126/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eHierarchical Reasoning Models (HRM) and their variants (TRM) have shown impressive performance on a variety of reasoning tasks. These models maintain two separate states $z_L$ and $z_H$ that are designed to capture low-level and high-level representations for reasoning. However, the necessity of this dual-state strcuture remains unclear. We conjecture that the benefit of $z_H$ stems not necessarily from representing any distinct level of information, but rather from providing access to information from earlier timesteps, as $z_H$ is updated less frequently than $z_L$. To test this, we propose Simple Recursive Model (SRM): a model that maintains only a single state $z$ and adopts skip connections to earlier timesteps in replacement of the second-level state $z_H$. This approach eliminates the complicated nested loops in HRM/TRM that alternate between $z_L$ and $z_H$, leading to a more straightforward and parsimonious architecture. Through experiments on the Sudoku task, we demonstrate that the single-state SRM can achieve comparable performance to the TRM baseline. Our results suggest that the key benefit of HRM/TRM is not necessarily hierarchical state separation, but may come from the ability to access information across different timescales. Such ability can be equivalently achieved through other mechanisms such as skip connections, opening up new avenues for designing simpler and more efficient reasoning models.\u003c/p\u003e\n\u003cp\u003eCode available at: https://github.com/liaoq/SimpleRecursiveModel\u003c/p\u003e","manuscriptTitle":"Simple Recursive Model: Simplified, Single-State Reasoning with Skip Connections","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-07 03:40:27","doi":"10.21203/rs.3.rs-8492126/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"bb87c087-ba9f-456c-98db-f80cc417cf07","owner":[],"postedDate":"January 7th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":60440834,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2026-01-07T03:40:27+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-07 03:40:27","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8492126","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8492126","identity":"rs-8492126","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}