3D path planning of unmanned ground vehicles based on improved DDQN

3D path planning of unmanned ground vehicles based on improved DDQN | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article 3D path planning of unmanned ground vehicles based on improved DDQN Can Tang, Tao Peng, Xingxing Xie, Junhu Peng This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4612963/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 11 You are reading this latest preprint version Abstract For safe and efficient path planning of unmanned ground vehicles in complex 3D environment, this paper proposes an improved deep reinforcement learning algorithm (Dual Experience Dynamic Target DDQN, DEDT DDQN) to solve the problems of sparse reward convergence and over-estimation that are difficulties for traditional DDQN algorithms in complex maps. The algorithm improves the performance of the DDQN algorithm in dealing with complex environments by dividing the input quality experience and dynamically fusing the a priori knowledge of DDQN and average DDQN for network parameter training. For unstructured 3D environments, this paper adopts a path planning strategy based on the digital elevation model (DEM) considering environmental characteristics and time cost. Simulation experiments of the DEDT DDQN algorithm in 3D maps modeled on realistic environments show that the DEDT DDQN algorithm reduces the number of inflection points and the average slope change by 40% and 16.7%, respectively, and improves the performance of optimization searching as well as the convergence speed by 5.34% and 60%, respectively. The proposed improved algorithm and adopted strategy can be applied in two different types of maps, which verifies the effectiveness and robustness of the algorithm and strategy. Deep reinforcement learning Unmanned ground vehicles Digital elevation model 3D path planning Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 05 Oct, 2024 Reviews received at journal 04 Oct, 2024 Reviews received at journal 02 Oct, 2024 Reviews received at journal 27 Sep, 2024 Reviewers agreed at journal 16 Sep, 2024 Reviewers agreed at journal 13 Sep, 2024 Reviewers agreed at journal 12 Sep, 2024 Reviewers invited by journal 11 Sep, 2024 Editor assigned by journal 21 Jun, 2024 Submission checks completed at journal 21 Jun, 2024 First submitted to journal 20 Jun, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4612963","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":320527007,"identity":"87c03d22-8864-4436-98be-0f6a3f3501ec","order_by":0,"name":"Can Tang","email":"","orcid":"","institution":"Chongqing University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Can","middleName":"","lastName":"Tang","suffix":""},{"id":320527008,"identity":"f671aa59-0607-4c7c-bd72-9c5f4380b89b","order_by":1,"name":"Tao Peng","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAoklEQVRIiWNgGAWjYFAC5gNQRgKxOtjYQEoNSNLCY0CiFnP5no8f3rb9YeBnzzFg+LmDCC2WbbybJee2GTBI9rwxYOw9Q4QWg2O825h5gVoMbuQYMDO2EaWF5xlYiz0pWtggtkgQq8WyLc1Ycs45Yx6JM88KDvYSo8Wc+fDDD2/K5OT425M3PvhJlMNABA8YMTAcIEIDQssoGAWjYBSMAtwAAETTLkJCFMSdAAAAAElFTkSuQmCC","orcid":"","institution":"Chongqing University of Technology","correspondingAuthor":true,"prefix":"","firstName":"Tao","middleName":"","lastName":"Peng","suffix":""},{"id":320527009,"identity":"a4545cd3-0703-462a-b7ca-44f8cd00c041","order_by":2,"name":"Xingxing Xie","email":"","orcid":"","institution":"Chongqing University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Xingxing","middleName":"","lastName":"Xie","suffix":""},{"id":320527010,"identity":"42ed027c-9084-43fc-bbd0-2194ac786eee","order_by":3,"name":"Junhu Peng","email":"","orcid":"","institution":"Chongqing University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Junhu","middleName":"","lastName":"Peng","suffix":""}],"badges":[],"createdAt":"2024-06-20 15:59:56","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4612963/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4612963/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":60045735,"identity":"0c620ab7-f6fd-4bcd-8f30-9f869944e98c","added_by":"auto","created_at":"2024-07-11 04:28:15","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2317898,"visible":true,"origin":"","legend":"","description":"","filename":"snarticletemplate.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4612963/v1_covered_ade054c9-8bf2-4a3f-8db3-197fe309cf86.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"3D path planning of unmanned ground vehicles based on improved DDQN","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"the-journal-of-supercomputing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [The Journal of Supercomputing](https://www.springer.com/journal/11227)","snPcode":"11227","submissionUrl":"https://submission.nature.com/new-submission/11227/3","title":"The Journal of Supercomputing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Deep reinforcement learning, Unmanned ground vehicles, Digital elevation model, 3D path planning","lastPublishedDoi":"10.21203/rs.3.rs-4612963/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4612963/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eFor safe and efficient path planning of unmanned ground vehicles in complex 3D environment, this paper proposes an improved deep reinforcement learning algorithm (Dual Experience Dynamic Target DDQN, DEDT DDQN) to solve the problems of sparse reward convergence and over-estimation that are difficulties for traditional DDQN algorithms in complex maps. The algorithm improves the performance of the DDQN algorithm in dealing with complex environments by dividing the input quality experience and dynamically fusing the a priori knowledge of DDQN and average DDQN for network parameter training. For unstructured 3D environments, this paper adopts a path planning strategy based on the digital elevation model (DEM) considering environmental characteristics and time cost. Simulation experiments of the DEDT DDQN algorithm in 3D maps modeled on realistic environments show that the DEDT DDQN algorithm reduces the number of inflection points and the average slope change by 40% and 16.7%, respectively, and improves the performance of optimization searching as well as the convergence speed by 5.34% and 60%, respectively. The proposed improved algorithm and adopted strategy can be applied in two different types of maps, which verifies the effectiveness and robustness of the algorithm and strategy.\u003c/p\u003e","manuscriptTitle":"3D path planning of unmanned ground vehicles based on improved DDQN","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-07-11 04:20:07","doi":"10.21203/rs.3.rs-4612963/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-10-05T17:30:18+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-10-05T02:39:12+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-10-02T23:56:23+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-09-27T06:25:01+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"310786255991165934889600135964926597475","date":"2024-09-16T20:19:58+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"234668153647748596705765057370480846701","date":"2024-09-13T18:36:14+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"244972650042749524626453816764496454948","date":"2024-09-12T06:50:49+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-09-11T17:20:33+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-06-21T09:26:05+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-06-21T09:25:36+00:00","index":"","fulltext":""},{"type":"submitted","content":"The Journal of Supercomputing","date":"2024-06-20T15:58:41+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"the-journal-of-supercomputing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [The Journal of Supercomputing](https://www.springer.com/journal/11227)","snPcode":"11227","submissionUrl":"https://submission.nature.com/new-submission/11227/3","title":"The Journal of Supercomputing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"df21353e-158b-4b82-9ed9-772bb283bc7c","owner":[],"postedDate":"July 11th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2024-11-04T14:53:14+00:00","versionOfRecord":[],"versionCreatedAt":"2024-07-11 04:20:07","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4612963","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4612963","identity":"rs-4612963","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}