Rational decisions in multi-step environments with few rollouts

Rational decisions in multi-step environments with few rollouts | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Rational decisions in multi-step environments with few rollouts Marcelo Mattar, Sixing Chen, Kristopher Jensen This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7367098/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted You are reading this latest preprint version Abstract People routinely make decisions by mentally simulating the potential outcomes of their actions. However, this process appears computationally intractable in real-world situations involving sequences of decisions with exponentially many possible futures. How the brain efficiently evaluates temporally extended decisions despite limited cognitive resources remains a fundamental puzzle. Here we present a mathematical theory showing that for decisions in multi-step environments, the rational strategy is to perform only a few mental simulations, formalized as rollouts. This is because the first rollouts provide substantially more information than later ones despite taking a similar amount of time, so the opportunity cost of additional simulations quickly outweighs their marginal benefit. Our framework demonstrates that this efficiency relies on the correlated reward structure of naturalistic environments, which allows information from one rollout to generalize to many related future paths. This theory also explains why, under resource constraints, many shallow rollouts are preferable to fewer deep ones; why apparently myopic decisions can arise without explicit temporal discounting; and how to relate the dynamics of planning to evidence accumulation models. We validate predictions of our theory in two behavioral experiments, which confirm that humans achieve higher reward rates with few rollouts and dynamically adjust their simulation depth based on available cognitive resources. These findings reveal how the brain balances the depth and breadth of mental simulation to make effective decisions under computational constraints, providing a unifying account of planning that bridges computationally intensive search algorithms in machine learning and the remarkable efficiency of human decision-making. Biological sciences/Psychology/Human behaviour Biological sciences/Neuroscience/Cognitive neuroscience/Decision decision-making mental simulation reinforcement learning planning Full Text Additional Declarations There is NO Competing Interest. Cite Share Download PDF Status: Under Review Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7367098","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":514208580,"identity":"2e65e483-2e6b-401c-8e53-19842982fce4","order_by":0,"name":"Marcelo Mattar","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA50lEQVRIiWNgGAWjYNCCAjjLBkLxENRiACKYGRsYGNJI13KYsBbd9rMPP3wwsAEyzh9/8HPP+TyDGwmMD9624dZidibdWHKGQRqQkczY2PPsdjFQC7PhXHxaDqSxMfMYHAYykhkbeA7cTtxwI4FNmheflvPP2Jj/gLScf8zY+OfAOZAW9t94tdwA2sIA0nIjmbGZ58ABsC3M+LU8Y5bsMUjjMbvx2HC2zIHkxJlnHjZLzjmHz2FpjB9+VNjImZ1PfPDxzQG7xL7jyQc/vCnDrQUGEBGhcAAUPyQBeVI1jIJRMApGwbAHAFr6VwRdAcKpAAAAAElFTkSuQmCC","orcid":"","institution":"NYU","correspondingAuthor":true,"prefix":"","firstName":"Marcelo","middleName":"","lastName":"Mattar","suffix":""},{"id":514208581,"identity":"aee31256-22d8-460a-84ac-cac8054c7ead","order_by":1,"name":"Sixing Chen","email":"","orcid":"","institution":"New York University","correspondingAuthor":false,"prefix":"","firstName":"Sixing","middleName":"","lastName":"Chen","suffix":""},{"id":514208582,"identity":"c0023fc9-50fb-41ca-851a-57d9101da34b","order_by":2,"name":"Kristopher Jensen","email":"","orcid":"","institution":"University College London","correspondingAuthor":false,"prefix":"","firstName":"Kristopher","middleName":"","lastName":"Jensen","suffix":""}],"badges":[],"createdAt":"2025-08-13 17:05:19","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7367098/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7367098/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":92005070,"identity":"d79bda79-541c-4fe1-9d62-ae220e9349ad","added_by":"auto","created_at":"2025-09-23 15:06:41","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1189855,"visible":true,"origin":"","legend":"Article File","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7367098/v1_covered_5e675151-ed08-4acd-a499-ee59b2a0d5cf.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Rational decisions in multi-step environments with few rollouts","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"decision-making, mental simulation, reinforcement learning, planning","lastPublishedDoi":"10.21203/rs.3.rs-7367098/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7367098/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"People routinely make decisions by mentally simulating the potential outcomes of their actions. However, this process appears computationally intractable in real-world situations involving sequences of decisions with exponentially many possible futures. How the brain efficiently evaluates temporally extended decisions despite limited cognitive resources remains a fundamental puzzle. Here we present a mathematical theory showing that for decisions in multi-step environments, the rational strategy is to perform only a few mental simulations, formalized as rollouts. This is because the first rollouts provide substantially more information than later ones despite taking a similar amount of time, so the opportunity cost of additional simulations quickly outweighs their marginal benefit. Our framework demonstrates that this efficiency relies on the correlated reward structure of naturalistic environments, which allows information from one rollout to generalize to many related future paths. This theory also explains why, under resource constraints, many shallow rollouts are preferable to fewer deep ones; why apparently myopic decisions can arise without explicit temporal discounting; and how to relate the dynamics of planning to evidence accumulation models. We validate predictions of our theory in two behavioral experiments, which confirm that humans achieve higher reward rates with few rollouts and dynamically adjust their simulation depth based on available cognitive resources. These findings reveal how the brain balances the depth and breadth of mental simulation to make effective decisions under computational constraints, providing a unifying account of planning that bridges computationally intensive search algorithms in machine learning and the remarkable efficiency of human decision-making.","manuscriptTitle":"Rational decisions in multi-step environments with few rollouts","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-23 14:58:32","doi":"10.21203/rs.3.rs-7367098/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"b17a5eee-e7b5-4fcc-b3c2-67158c6d40c6","owner":[],"postedDate":"September 23rd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":54631266,"name":"Biological sciences/Psychology/Human behaviour"},{"id":54631267,"name":"Biological sciences/Neuroscience/Cognitive neuroscience/Decision"}],"tags":[],"updatedAt":"2025-09-23T14:58:32+00:00","versionOfRecord":[],"versionCreatedAt":"2025-09-23 14:58:32","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7367098","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7367098","identity":"rs-7367098","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}