Deep Reinforcement Learning Double-LayerEco-City Vehicle-Road Cooperative Control inLarge-Scale Road Networks

Deep Reinforcement Learning Double-LayerEco-City Vehicle-Road Cooperative Control inLarge-Scale Road Networks | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Deep Reinforcement Learning Double-LayerEco-City Vehicle-Road Cooperative Control inLarge-Scale Road Networks Liping Yan, Kanglai Wu, Renjie Tang, Jiayue Xu, Haojie Jia, Kai Song This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5882276/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 3 You are reading this latest preprint version Abstract As Connected and Automated Vehicle (CAV) technology progresses, optimizing vehicle-road cooperative control becomes critical for alleviating traffic congestion. Existing approaches often involve complex computations that are unsuitable for real-time control and are typically limited to isolated intersections, restricting their scalability. To address these limitations, this paper introduces a Mixed Platoon Dual-Layer (MPDL) model designed for large-scale road networks. The model leverages mixed platoon arrival time predictions combined with Deep Reinforcement Learning (DRL) for cooperative control at signalized intersections. To enhance decision-making, this study proposes the Multi-Distributed Proximal Policy Optimization (MDPPO) algorithm, which efficiently manages dynamic vehicle-traffic interactions. This algorithm optimizes mixed platoon trajectories and determines the optimal signal phases. Furthermore, the model incorporates eco-friendly traffic strategies aimed at reducing emissions across the network. Extensive simulations using the SUMO traffic simulator on both synthetic and real-world networks demonstrate that the MPDL model achieves higher training rewards compared to existing methods such as MA2C and IA2C. The MPDL model consistently outperforms these methods in key performance metrics, including waiting time, speed, delay, and pollutant emissions. Mixed traffic Platoon control Connected and automated vehicle Vehicle-Road cooperative control Deep reinforcement learning Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editor assigned by journal 24 Jan, 2025 Submission checks completed at journal 24 Jan, 2025 First submitted to journal 22 Jan, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5882276","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":406910900,"identity":"58dc4a57-e590-4abd-8431-ffe32d96f904","order_by":0,"name":"Liping Yan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA9klEQVRIiWNgGAWjYHACxgMJDBIMDOwNDCAKCBII64Fo4TkA1JJArBYwCVZOjBaD42cPHHi4wyJPPvLxwQ+WPw4z8LPnGDD83IFHy5m8hAOJZySKDW+nJUtIJBxmkOx5Y8DYewaPlgM5BgcS2yQSN87OMWMAaTG4kWPAzNiGR8v5N1AtM89/A2uxJ6jlBtSW+RI8bBBbJAhokbwBsuWMROIGnjRjCYm0dB6JM88KDvbi0cJ3Psfw4c8ddYnz2w8//CxhYy3H35688cFPPFoUDgAJxgZQODAwMINiFCR6ALcGBgb5BqgWEIPxAz6lo2AUjIJRMGIBAGzHV5iCsLUfAAAAAElFTkSuQmCC","orcid":"","institution":"East China Jiaotong University","correspondingAuthor":true,"prefix":"","firstName":"Liping","middleName":"","lastName":"Yan","suffix":""},{"id":406910901,"identity":"701d17d4-3356-45de-b724-8d5ed4ef5979","order_by":1,"name":"Kanglai Wu","email":"","orcid":"","institution":"East China Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Kanglai","middleName":"","lastName":"Wu","suffix":""},{"id":406910902,"identity":"76983fc6-a288-4036-b0c4-918e1bfe2b30","order_by":2,"name":"Renjie Tang","email":"","orcid":"","institution":"East China Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Renjie","middleName":"","lastName":"Tang","suffix":""},{"id":406910903,"identity":"671d099c-efcd-439c-aa09-bc81d157ad4c","order_by":3,"name":"Jiayue Xu","email":"","orcid":"","institution":"East China Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Jiayue","middleName":"","lastName":"Xu","suffix":""},{"id":406910904,"identity":"d5cadc24-f4db-4cef-a389-555668751dd5","order_by":4,"name":"Haojie Jia","email":"","orcid":"","institution":"East China Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Haojie","middleName":"","lastName":"Jia","suffix":""},{"id":406910905,"identity":"7aec22c3-9e2e-40ea-b856-0190661fbbec","order_by":5,"name":"Kai Song","email":"","orcid":"","institution":"East China Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Kai","middleName":"","lastName":"Song","suffix":""}],"badges":[],"createdAt":"2025-01-22 15:53:34","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5882276/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5882276/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":74957533,"identity":"7a99340e-b16d-4846-94cd-0e441557fbe9","added_by":"auto","created_at":"2025-01-28 17:56:55","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":12827252,"visible":true,"origin":"","legend":"","description":"","filename":"2025122LatexDeepReinforcementLearningDoubleLayerEcoCityVehicleRoadCooperativeControlinLargeScaleRoadNetworks.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5882276/v1_covered_034a5f14-fca2-49fb-9d45-0e87fa19b23c.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Deep Reinforcement Learning Double-LayerEco-City Vehicle-Road Cooperative Control inLarge-Scale Road Networks","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"the-journal-of-supercomputing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [The Journal of Supercomputing](https://www.springer.com/journal/11227)","snPcode":"11227","submissionUrl":"https://submission.nature.com/new-submission/11227/3","title":"The Journal of Supercomputing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Mixed traffic, Platoon control, Connected and automated vehicle, Vehicle-Road cooperative control, Deep reinforcement learning","lastPublishedDoi":"10.21203/rs.3.rs-5882276/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5882276/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eAs Connected and Automated Vehicle (CAV) technology progresses, optimizing vehicle-road cooperative control becomes critical for alleviating traffic congestion. Existing approaches often involve complex computations that are unsuitable for real-time control and are typically limited to isolated intersections, restricting their scalability. To address these limitations, this paper introduces a Mixed Platoon Dual-Layer (MPDL) model designed for large-scale road networks. The model leverages mixed platoon arrival time predictions combined with Deep Reinforcement Learning (DRL) for cooperative control at signalized intersections. To enhance decision-making, this study proposes the Multi-Distributed Proximal Policy Optimization (MDPPO) algorithm, which efficiently manages dynamic vehicle-traffic interactions. This algorithm optimizes mixed platoon trajectories and determines the optimal signal phases. Furthermore, the model incorporates eco-friendly traffic strategies aimed at reducing emissions across the network. Extensive simulations using the SUMO traffic simulator on both synthetic and real-world networks demonstrate that the MPDL model achieves higher training rewards compared to existing methods such as MA2C and IA2C. The MPDL model consistently outperforms these methods in key performance metrics, including waiting time, speed, delay, and pollutant emissions.\u003c/p\u003e","manuscriptTitle":"Deep Reinforcement Learning Double-LayerEco-City Vehicle-Road Cooperative Control inLarge-Scale Road Networks","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-01-28 17:24:44","doi":"10.21203/rs.3.rs-5882276/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorAssigned","content":"","date":"2025-01-25T03:35:24+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-01-25T03:34:13+00:00","index":"","fulltext":""},{"type":"submitted","content":"The Journal of Supercomputing","date":"2025-01-22T15:42:19+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"the-journal-of-supercomputing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [The Journal of Supercomputing](https://www.springer.com/journal/11227)","snPcode":"11227","submissionUrl":"https://submission.nature.com/new-submission/11227/3","title":"The Journal of Supercomputing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"cc0ab8ab-f510-430a-b0ca-b40e2a463fe1","owner":[],"postedDate":"January 28th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-01-28T17:24:44+00:00","versionOfRecord":[],"versionCreatedAt":"2025-01-28 17:24:44","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5882276","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5882276","identity":"rs-5882276","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}