RRH-RL: Recursive Receding Horizon Deep Reinforcement Learning for End-to-End Robot Navigation in Mapless Harsh Environments

RRH-RL: Recursive Receding Horizon Deep Reinforcement Learning for End-to-End Robot Navigation in Mapless Harsh Environments | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article RRH-RL: Recursive Receding Horizon Deep Reinforcement Learning for End-to-End Robot Navigation in Mapless Harsh Environments Eslam Mohamed, Ahmed Elliethy, Armando Sousa, Filipe Santos This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8194478/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract This paper presents RRH-RL, a Recursive Receding Horizon Reinforcement Learning approach for end-to-end mobile robot navigation in mapless, harsh environments. Navigation in such conditions poses multiple challenges, including partial observability, short-sighted decision-making, trap avoidance, and energy inefficiency. RRH-RL addresses these by predicting a horizon of future actions at each step, executing the first, and recursively embedding the rest into subsequent observations to maintain temporal consistency and to produce energy-efficient motion. A probabilistic costmap—updated via Bayesian filtering and belief propagation (BP)—serves as spatial memory to avoid revisiting dead ends and being trapped. A projection-based mechanism estimates future sensor observations, enabling accurate foresight-driven reward shaping. Experiments in high-fidelity simulations across diverse harsh environments maps show that RRH-RL significantly outperforms state-of-the-art approaches, achieving up to 7.58% higher success rate, 9.86% shorter path length, 22.06% smoother motion, and 20.67% lower energy consumption, compared to the next best approach. Additionally, ablation studies are performed to validate the contribution of horizon length, recursive action embedding, and the BP to overall performance. The full source code is available on GitHub. Robot RL Navigation reced Learn Harsh Full Text Additional Declarations No competing interests reported. Supplementary Files RRHRL.mp4 Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8194478","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":556663555,"identity":"dd23fc9d-ca01-4910-80e1-522f534adcf3","order_by":0,"name":"Eslam Mohamed","email":"","orcid":"","institution":"University of Porto","correspondingAuthor":false,"prefix":"","firstName":"Eslam","middleName":"","lastName":"Mohamed","suffix":""},{"id":556663556,"identity":"3686786d-21de-4bd3-ba8c-7f98f5b6a39a","order_by":1,"name":"Ahmed Elliethy","email":"","orcid":"","institution":"Military Technical College","correspondingAuthor":false,"prefix":"","firstName":"Ahmed","middleName":"","lastName":"Elliethy","suffix":""},{"id":556663557,"identity":"ca035eb6-cf7c-4f05-bfb4-24f075760796","order_by":2,"name":"Armando Sousa","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAtElEQVRIiWNgGAWjYPACG36StaRJNkAYBkRrOUyCFv7+w8ckf9Scl+Bn4DF7wLjjD2EtEjfS0qR5jt2WkGzgMTdgPEOMw27wmEkzsN2uMzjAYybB2EaEFvnzZ8wkf/w7J2FPtBaDAzlmErxtByQMGIjVYngjLdmaty9ZQuIwW7lB4hljwlrkzh8+ePPHNzsJ/vbmbQ8+7pAjrAUBmBnYGBIbSNEBBGwMjKRqGQWjYBSMghEBABl3MY3SNvKPAAAAAElFTkSuQmCC","orcid":"","institution":"University of Porto","correspondingAuthor":true,"prefix":"","firstName":"Armando","middleName":"","lastName":"Sousa","suffix":""},{"id":556663558,"identity":"ba9a5261-ac61-42b1-b786-7277b7fceeb3","order_by":3,"name":"Filipe Santos","email":"","orcid":"","institution":"University of Porto","correspondingAuthor":false,"prefix":"","firstName":"Filipe","middleName":"","lastName":"Santos","suffix":""}],"badges":[],"createdAt":"2025-11-24 14:53:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8194478/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8194478/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":97861590,"identity":"fa3158de-c36e-433e-bd9e-6c2cd54e42ba","added_by":"auto","created_at":"2025-12-10 08:54:36","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5729,"visible":true,"origin":"","legend":"","description":"","filename":"919ec0c12cb04f56b385cb4dee28f27f.json","url":"https://assets-eu.researchsquare.com/files/rs-8194478/v1/175d0425868fed42127e66df.json"},{"id":108803612,"identity":"de2d76eb-0055-4011-bcf9-6d220c9f9105","added_by":"auto","created_at":"2026-05-08 15:01:06","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4753802,"visible":true,"origin":"","legend":"","description":"","filename":"RRHRL.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8194478/v1_covered_4930cd46-4da3-4082-bf8d-5af04655e93a.pdf"},{"id":97861591,"identity":"419267bb-7097-49e8-b40f-237cb225107d","added_by":"auto","created_at":"2025-12-10 08:54:36","extension":"mp4","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":20196244,"visible":true,"origin":"","legend":"","description":"","filename":"RRHRL.mp4","url":"https://assets-eu.researchsquare.com/files/rs-8194478/v1/bfc53e72da4c4a743a6eb321.mp4"}],"financialInterests":"No competing interests reported.","formattedTitle":"RRH-RL: Recursive Receding Horizon Deep Reinforcement Learning for End-to-End Robot Navigation in Mapless Harsh Environments","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Robot, RL, Navigation, reced, Learn, Harsh","lastPublishedDoi":"10.21203/rs.3.rs-8194478/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8194478/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"This paper presents RRH-RL, a Recursive Receding Horizon Reinforcement Learning approach for end-to-end mobile robot navigation in mapless, harsh environments. Navigation in such conditions poses multiple challenges, including partial observability, short-sighted decision-making, trap avoidance, and energy inefficiency. RRH-RL addresses these by predicting a horizon of future actions at each step, executing the first, and recursively embedding the rest into subsequent observations to maintain temporal consistency and to produce energy-efficient motion. A probabilistic costmap—updated via Bayesian filtering and belief propagation (BP)—serves as spatial memory to avoid revisiting dead ends and being trapped. A projection-based mechanism estimates future sensor observations, enabling accurate foresight-driven reward shaping. Experiments in high-fidelity simulations across diverse harsh environments maps show that RRH-RL significantly outperforms state-of-the-art approaches, achieving up to 7.58\\% higher success rate, 9.86\\% shorter path length, 22.06\\% smoother motion, and 20.67\\% lower energy consumption, compared to the next best approach. Additionally, ablation studies are performed to validate the contribution of horizon length, recursive action embedding, and the BP to overall performance. The full source code is available on GitHub.","manuscriptTitle":"RRH-RL: Recursive Receding Horizon Deep Reinforcement Learning for End-to-End Robot Navigation in Mapless Harsh Environments","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-12-10 08:54:31","doi":"10.21203/rs.3.rs-8194478/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"47f14539-ee8b-4324-b600-9bb736ca54be","owner":[],"postedDate":"December 10th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-04-27T15:56:52+00:00","versionOfRecord":[],"versionCreatedAt":"2025-12-10 08:54:31","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8194478","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8194478","identity":"rs-8194478","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}