Dynamic Task Offloading Strategy for Multi-Agent Deep Reinforcement Learning Based on Lyapunov

Dynamic Task Offloading Strategy for Multi-Agent Deep Reinforcement Learning Based on Lyapunov | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Dynamic Task Offloading Strategy for Multi-Agent Deep Reinforcement Learning Based on Lyapunov Yang ZheXing, Xie XiaoLan, Guo Qian, Tan ShuRu This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4447725/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Aiming at the issue of communication overload and task backlog caused by the dynamic nature of the environment and the surge in the number of mobile devices in multi-user Mobile Edge Computing scenarios, a dynamic task offloading strategy based on Lyapunov-guided multi-agent deep reinforcement learning is proposed. This strategy aims to ensure long-term system stability while minimizing the average task processing latency and energy consumption. First, the dependency relationships among subtasks are modeled using a directed acyclic graph, and this is expressed as an optimization problem to minimize task offloading costs with long-term constraints. Then, using Lyapunov optimization theory, the long-term average problem is decoupled into deterministic problems for each time slot. Finally, the issue is further transformed into an optimal policy problem under a Markov Decision Process framework, and solved using the designed Lyapunov Multi-Agent Deep Deterministic Policy Gradient (Ly-MADDPG) algorithm. The simulation experiment results indicate that compared to the alternative algorithms, our proposed method reduces the task offloading cost while ensuring queue stability. Edge Computing Task Offloading Lyapunov Deep Reinforcement Learning Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4447725","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":309320265,"identity":"7140ed14-8c00-431a-a3e8-7e98330e0274","order_by":0,"name":"Yang ZheXing","email":"","orcid":"","institution":"Guilin University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Yang","middleName":"","lastName":"ZheXing","suffix":""},{"id":309320266,"identity":"1bf54b50-ae22-44db-9f69-336bc5af545e","order_by":1,"name":"Xie XiaoLan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAyklEQVRIiWNgGAWjYBACfv7+jw8+GNjI8bM3EKlFcsYBY8MZFWnGkj0HiNRicCDBTJrnzKHEDTMSiHXZgQMJ0rxtBxI3SD7eeIOhxiaaoA7G5oYDhnPb7hhvl04rtmA4lpbbQEgLM8PBhoS3bc9kd87OMZNgbDhMWAsbQzLDAd62w4wbbp4hUgsPQxpjI8+Zw4obbvAQqUVC4gwzIySQgX5JIMYv9ud72H9AovLwxhsfamwIa0EGBhIJpCiHaCFVxygYBaNgFIwMAADQ60YSQotfBAAAAABJRU5ErkJggg==","orcid":"","institution":"Guilin University of Technology","correspondingAuthor":true,"prefix":"","firstName":"Xie","middleName":"","lastName":"XiaoLan","suffix":""},{"id":309320267,"identity":"ec0e8a8e-377e-4982-aa31-c67c9d9a3979","order_by":2,"name":"Guo Qian","email":"","orcid":"","institution":"Guilin University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Guo","middleName":"","lastName":"Qian","suffix":""},{"id":309320268,"identity":"004bfd6d-b0a2-4617-bf7b-adc71a7c71e2","order_by":3,"name":"Tan ShuRu","email":"","orcid":"","institution":"Guilin University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Tan","middleName":"","lastName":"ShuRu","suffix":""}],"badges":[],"createdAt":"2024-05-20 08:29:33","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4447725/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4447725/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":78022335,"identity":"f7760b30-0423-465a-90f6-8dab32375105","added_by":"auto","created_at":"2025-03-08 04:01:54","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4648610,"visible":true,"origin":"","legend":"","description":"","filename":"DynamicTaskOffloadingStrategyforMultiAgentDeepReinforcementLearningBasedonLyapunov.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4447725/v1_covered_92454b56-0f92-471e-b512-165479984c91.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Dynamic Task Offloading Strategy for Multi-Agent Deep Reinforcement Learning Based on Lyapunov","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Edge Computing, Task Offloading, Lyapunov, Deep Reinforcement Learning","lastPublishedDoi":"10.21203/rs.3.rs-4447725/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4447725/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Aiming at the issue of communication overload and task backlog caused by the dynamic nature of the environment and the surge in the number of mobile devices in multi-user Mobile Edge Computing scenarios, a dynamic task offloading strategy based on Lyapunov-guided multi-agent deep reinforcement learning is proposed. This strategy aims to ensure long-term system stability while minimizing the average task processing latency and energy consumption. First, the dependency relationships among subtasks are modeled using a directed acyclic graph, and this is expressed as an optimization problem to minimize task offloading costs with long-term constraints. Then, using Lyapunov optimization theory, the long-term average problem is decoupled into deterministic problems for each time slot. Finally, the issue is further transformed into an optimal policy problem under a Markov Decision Process framework, and solved using the designed Lyapunov Multi-Agent Deep Deterministic Policy Gradient (Ly-MADDPG) algorithm. The simulation experiment results indicate that compared to the alternative algorithms, our proposed method reduces the task offloading cost while ensuring queue stability.","manuscriptTitle":"Dynamic Task Offloading Strategy for Multi-Agent Deep Reinforcement Learning Based on Lyapunov","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-05 18:27:09","doi":"10.21203/rs.3.rs-4447725/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2b1b6bdc-f2d0-4735-b1c6-834bd65a040c","owner":[],"postedDate":"June 5th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-03-08T03:53:26+00:00","versionOfRecord":[],"versionCreatedAt":"2024-06-05 18:27:09","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4447725","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4447725","identity":"rs-4447725","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}