A computational model of behavioral adaptation to solve the credit assignment problem

A computational model of behavioral adaptation to solve the credit assignment problem | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A computational model of behavioral adaptation to solve the credit assignment problem Roy Clymer, Sanjeev Namjoshi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4551575/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The adaptive fitness of an organism in its ecological niche is highly reliant upon its ability to associate environmental or internal stimuli with behavioral responses through reinforcement. This simple but powerful approach has been successfully applied in computational neuroscience and reinforcement learning to model both human and animal behaviors. However, a critical challenge faced by these models is the credit assignment problem, in which the association between past behavior and a delayed reinforcement signal must be considered. In this paper, we reformulate the credit assignment problem to consider how past stimuli are linked to adaptive behavioral responses in a simple neuronal circuit. We propose a biologically plausible variant of a spiking neural network, which can model a wide variety of behavioral, learning, and evolutionary phenomena. Our model suggests one fundamental mechanism for associating a behavior with an adaptive response that may be used in the brains of both simple and complex organisms. Our results show the model's versatility and biological plausibility in a number of tasks related to classical and operant conditioning, including behavioral chaining. Additionally, we present simulations to demonstrate how adaptive behaviors such as reflexes and simple category detection may have evolved using our model. Our results indicate the potential for further modifications and extensions of our model to replicate more sophisticated and biologically plausible behavioral, learning, and intelligence phenomena found throughout the animal kingdom. Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4551575","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":312651858,"identity":"d3a32da7-c1c3-4984-b7e2-6fd0dd446ce7","order_by":0,"name":"Roy Clymer","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAz0lEQVRIiWNgGAWjYLCCByCCvQFIGFgQqSUBRPAcAGmRIEWLBIQkrNq8vcfwQ2LbNjnzmc+vbvhRIMHA396dgFeLzJkzxhKJbbeNZW7nlN3sATpM4szZDXi1SEjkmDEAtSTOkM5Ju8ED1GIgkUuclvoZkmfSbv4hRUuChAT7sdvE2cJzrFgi4dxtwxk8OWy3ZQwkeAj7hb1544cPZbflJdiPP7v55o+NHH97L34tDAwcBgyMbCAGjwGYJKAcBNgfMDD8gTFGwSgYBaNgFGABAMaRRZkA0UwwAAAAAElFTkSuQmCC","orcid":"","institution":"","correspondingAuthor":true,"prefix":"","firstName":"Roy","middleName":"","lastName":"Clymer","suffix":""},{"id":312651859,"identity":"bed53705-eff1-47d7-abc9-9268f8b84204","order_by":1,"name":"Sanjeev Namjoshi","email":"","orcid":"","institution":"KungFu.AI","correspondingAuthor":false,"prefix":"","firstName":"Sanjeev","middleName":"","lastName":"Namjoshi","suffix":""}],"badges":[],"createdAt":"2024-06-08 18:26:22","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4551575/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4551575/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":61576237,"identity":"c9942f4b-79ff-4657-8ccc-56883bc2d1df","added_by":"auto","created_at":"2024-08-01 12:19:30","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1266922,"visible":true,"origin":"","legend":"","description":"","filename":"Archive.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4551575/v1_covered_c8d5f270-5680-4829-8023-a01e1e6e5625.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A computational model of behavioral adaptation to solve the credit assignment problem","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-4551575/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4551575/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The adaptive fitness of an organism in its ecological niche is highly reliant upon its ability to associate environmental or internal stimuli with behavioral responses through reinforcement. This simple but powerful approach has been successfully applied in computational neuroscience and reinforcement learning to model both human and animal behaviors. However, a critical challenge faced by these models is the credit assignment problem, in which the association between past behavior and a delayed reinforcement signal must be considered. In this paper, we reformulate the credit assignment problem to consider how past stimuli are linked to adaptive behavioral responses in a simple neuronal circuit. We propose a biologically plausible variant of a spiking neural network, which can model a wide variety of behavioral, learning, and evolutionary phenomena. Our model suggests one fundamental mechanism for associating a behavior with an adaptive response that may be used in the brains of both simple and complex organisms. Our results show the model's versatility and biological plausibility in a number of tasks related to classical and operant conditioning, including behavioral chaining. Additionally, we present simulations to demonstrate how adaptive behaviors such as reflexes and simple category detection may have evolved using our model. Our results indicate the potential for further modifications and extensions of our model to replicate more sophisticated and biologically plausible behavioral, learning, and intelligence phenomena found throughout the animal kingdom.","manuscriptTitle":"A computational model of behavioral adaptation to solve the credit assignment problem","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-25 02:36:42","doi":"10.21203/rs.3.rs-4551575/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ed569b09-540e-46fe-b76d-21e859d7500f","owner":[],"postedDate":"June 25th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-11-16T14:23:14+00:00","versionOfRecord":[],"versionCreatedAt":"2024-06-25 02:36:42","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4551575","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4551575","identity":"rs-4551575","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}