Mitigating Spatial Hallucination in Large Language Models for Path Planning via Prompt Engineering

Mitigating Spatial Hallucination in Large Language Models for Path Planning via Prompt Engineering | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Mitigating Spatial Hallucination in Large Language Models for Path Planning via Prompt Engineering Hongjie Zhang, Hourui Deng, Jie Ou, Chaosheng Feng This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4609889/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 14 Mar, 2025 Read the published version in Scientific Reports → Version 1 posted 10 You are reading this latest preprint version Abstract Spatial reasoning in Large Language Models (LLMs) serves as a foundation for embodied intelligence. However, even in simple maze environments, LLMs often struggle to plan correct paths due to hallucination issues. To address this, we propose S2ERS, an LLM-based technique that integrates entity and relation extraction with the on-policy reinforcement learning algorithm Sarsa for optimal path planning. We introduce three key improvements: 1) To tackle the hallucination of spatial, we extract a graph structure of entities and relations from the text-based maze description, aiding LLMs in accurately comprehending spatial relationships. 2) To prevent LLMs from getting trapped in dead ends due to context inconsistency hallucination by long-term reasoning, we insert the state-action value function Q into the prompts, guiding the LLM's path planning. 3) To reduce the token consumption of LLMs, we utilize multi-step reasoning, dynamically inserting local Q-tables into the prompt to assist the LLM in outputting multiple steps of actions at once. Our comprehensive experimental evaluation, conducted using ChatGPT 3.5 and ERNIE-Bot 4.0, demonstrates that S2ERS significantly mitigates the spatial hallucination issues in LLMs, and improves the success rate and optimal rate by approximately 29% and 19%, respectively, in comparison to the SOTA CoT methods. Physical sciences/Mathematics and computing/Computer science Physical sciences/Mathematics and computing/Software Full Text Additional Declarations No competing interests reported. Supplementary Files data.zip Cite Share Download PDF Status: Published Journal Publication published 14 Mar, 2025 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 19 Dec, 2024 Reviews received at journal 13 Dec, 2024 Reviewers agreed at journal 24 Nov, 2024 Reviews received at journal 09 Nov, 2024 Reviewers agreed at journal 23 Oct, 2024 Reviewers invited by journal 24 Aug, 2024 Editor assigned by journal 24 Aug, 2024 Editor invited by journal 24 Jun, 2024 Submission checks completed at journal 24 Jun, 2024 First submitted to journal 20 Jun, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4609889","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":326671594,"identity":"8b73815d-c614-40ef-b956-9ddac970293d","order_by":0,"name":"Hongjie Zhang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABDklEQVRIiWNgGAWjYDACCSBmbACxDjYwfABzgYCHWC2MM0jUwsDADFeJT4v87OZnD7/usMmTdzzc+Njmj4W8uUQC44O3bQzy5ji0MM45Zm4seyat2PDAwWbj3DYJw50zEpgN57YxGO5swK6FWSLBTFqy7XDixoaDbdK5DRIJBjcS2KR52xgSDA5g18Imkf4NpqX9t8UfsBb23/i08EjkmEl+BGqZz3CwjRloAtgWZnxaJCRyyqQZ29ISNzAcbJbsBfplw5mHzZJzzgEZOLTIz0jfJvmzzSZx/ozjDz/8+FMnb3A8+eCHN2U28rhsAQcBKBYMbsBVgKNJArd6kJIfIOv6G/AqGgWjYBSMghEMAIfUX08Ef/fNAAAAAElFTkSuQmCC","orcid":"","institution":"Sichuan Normal University","correspondingAuthor":true,"prefix":"","firstName":"Hongjie","middleName":"","lastName":"Zhang","suffix":""},{"id":326671595,"identity":"58c98f6e-11b8-41da-9c12-320b919c7440","order_by":1,"name":"Hourui Deng","email":"","orcid":"","institution":"Sichuan Normal University","correspondingAuthor":false,"prefix":"","firstName":"Hourui","middleName":"","lastName":"Deng","suffix":""},{"id":326671596,"identity":"21cb2a48-4919-40fa-a3d8-d4ac95884382","order_by":2,"name":"Jie Ou","email":"","orcid":"","institution":"University of Electronic Science and Technology of China","correspondingAuthor":false,"prefix":"","firstName":"Jie","middleName":"","lastName":"Ou","suffix":""},{"id":326671597,"identity":"289fb4fc-9af4-44a6-915f-2b4a46ca0cf9","order_by":3,"name":"Chaosheng Feng","email":"","orcid":"","institution":"Sichuan Normal University","correspondingAuthor":false,"prefix":"","firstName":"Chaosheng","middleName":"","lastName":"Feng","suffix":""}],"badges":[],"createdAt":"2024-06-20 07:23:30","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4609889/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4609889/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-025-93601-5","type":"published","date":"2025-03-14T15:57:51+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":78689007,"identity":"a74b2973-3bfc-4a9e-91a1-96e782547649","added_by":"auto","created_at":"2025-03-17 16:10:06","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3658121,"visible":true,"origin":"","legend":"","description":"","filename":"ScientificReportsnew.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4609889/v1_covered_769c7b7e-ca1c-4388-8fd2-b470e717dbf3.pdf"},{"id":60320450,"identity":"b7701114-db73-4e0c-97b6-5750afc20bd9","added_by":"auto","created_at":"2024-07-15 14:00:14","extension":"zip","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":39260,"visible":true,"origin":"","legend":"","description":"","filename":"data.zip","url":"https://assets-eu.researchsquare.com/files/rs-4609889/v1/00aba7e3f1cf04fd2ea3eb14.zip"}],"financialInterests":"No competing interests reported.","formattedTitle":"Mitigating Spatial Hallucination in Large Language Models for Path Planning via Prompt Engineering","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-4609889/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4609889/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Spatial reasoning in Large Language Models (LLMs) serves as a foundation for embodied intelligence. However, even in simple maze environments, LLMs often struggle to plan correct paths due to hallucination issues. To address this, we propose S2ERS, an LLM-based technique that integrates entity and relation extraction with the on-policy reinforcement learning algorithm Sarsa for optimal path planning. We introduce three key improvements: 1) To tackle the hallucination of spatial, we extract a graph structure of entities and relations from the text-based maze description, aiding LLMs in accurately comprehending spatial relationships. 2) To prevent LLMs from getting trapped in dead ends due to context inconsistency hallucination by long-term reasoning, we insert the state-action value function Q into the prompts, guiding the LLM's path planning. 3) To reduce the token consumption of LLMs, we utilize multi-step reasoning, dynamically inserting local Q-tables into the prompt to assist the LLM in outputting multiple steps of actions at once. Our comprehensive experimental evaluation, conducted using ChatGPT 3.5 and ERNIE-Bot 4.0, demonstrates that S2ERS significantly mitigates the spatial hallucination issues in LLMs, and improves the success rate and optimal rate by approximately 29% and 19%, respectively, in comparison to the SOTA CoT methods.","manuscriptTitle":"Mitigating Spatial Hallucination in Large Language Models for Path Planning via Prompt Engineering","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-07-15 14:00:10","doi":"10.21203/rs.3.rs-4609889/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-12-19T11:31:26+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-12-13T06:45:42+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"305766987312724715183608778301964910435","date":"2024-11-25T00:02:13+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-09T13:27:37+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"235169198093857452127813327764044894299","date":"2024-10-24T02:03:29+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-08-24T05:06:44+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-08-24T05:00:34+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-06-24T12:07:58+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-06-24T06:58:12+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2024-06-20T07:22:13+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e2ebd435-5870-4200-bf9b-8b0e6ecfd096","owner":[],"postedDate":"July 15th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":34564450,"name":"Physical sciences/Mathematics and computing/Computer science"},{"id":34564451,"name":"Physical sciences/Mathematics and computing/Software"}],"tags":[],"updatedAt":"2025-03-17T16:02:42+00:00","versionOfRecord":{"articleIdentity":"rs-4609889","link":"https://doi.org/10.1038/s41598-025-93601-5","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2025-03-14 15:57:51","publishedOnDateReadable":"March 14th, 2025"},"versionCreatedAt":"2024-07-15 14:00:10","video":"","vorDoi":"10.1038/s41598-025-93601-5","vorDoiUrl":"https://doi.org/10.1038/s41598-025-93601-5","workflowStages":[]},"version":"v1","identity":"rs-4609889","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4609889","identity":"rs-4609889","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}