A Comprehensive Survey of Multi-Agent Reinforcement Learning for Autonomous Systems: Algorithms, Applications, and Open Challenges

A Comprehensive Survey of Multi-Agent Reinforcement Learning for Autonomous Systems: Algorithms, Applications, and Open Challenges | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Comprehensive Survey of Multi-Agent Reinforcement Learning for Autonomous Systems: Algorithms, Applications, and Open Challenges Adnan Khalid Bhatti, Muhammad Shahzad Mughal This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8751804/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Autonomous systems are ever-performing tasks in complicated and multi-agent conditions in which coordination, scalability, safety, and reliability are key demands. In these environments, non-stationarity of the climate, decentralized information, and closely interacting agent dynamics are fundamental limitations to traditional single-agent reinforcement learning. Multi-agent reinforcement learning (MARL) has become an effective framework for addressing these issues, enabling agents to adopt cooperative, competitive, or a combination of strategies through interaction with one another. Nevertheless, even with the breakthroughs in the field of algorithms, the scalability of MARL remains limited in its application to real-world autonomous systems due to scalability constraints, communication assumptions, safety considerations, and theoretical guarantees of its success. The paper is a survey of MARL in autonomous systems in a critical and detailed manner, focusing on the algorithmic and application-driven perspectives. It is based on a systematic literature review methodology that is used to gather, filter, and group recent peer-reviewed works by MARL paradigms and autonomous system areas. MARL algorithms are divided into value-style, policy-style, and a hybrid style, and their advantages and disadvantages and implementation implications are comparatively discussed. The use in autonomous vehicles, UAV swarms, multi-robot systems, and industrial autonomous environments is discussed to underscore domain-specific limitations associated with the coordination, communication, and safety. The survey reveals through cross-domain synthesis that there are still persistent issues, such as scaling to large and heterogeneous sets of agents, the use of an idealized communication model, weak sim-to-real transfer, limited interpretability, and failure to achieve general convergence and safety guarantees. Lastly, the most important gaps in research are presented, and the future direction is proposed to make MARL a safer, more interpretable, and deployable autonomous system. This survey represents a structured and critical source of guidance to researchers and practitioners who wish to build robust MARL-enabled autonomy beyond evaluation processes within simulations. Multi-agent reinforcement learning autonomous systems decentralized control cooperative learning multi-robot systems unmanned aerial vehicles autonomous vehicle decentralized training centralized execution Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8751804","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":591812954,"identity":"e5958916-251c-40d9-bcea-9ec596906a3a","order_by":0,"name":"Adnan Khalid Bhatti","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABU0lEQVRIie2SP0sDMRiHEw7ulkDXlLb2K+QouBi8r5JwEJcTdXSyx0FuKXY90Q/RsZuVDF3uAxy4tAh26XC3XfGgprZo/4mrQ54hgeR98ntDAoDB8A/xMI8/qgqjmgXDiV6AXbAZ7Z8qtq24iT8aMUKb9TiKiN7Txda3oid2oJBMMK0IStKxxDsK+EVxk4BoRSGQcXm7KFWr3+irHA4pv3fi2WRRUlBzArJ9l/Z8rcCEy1fEVOfhSdkYpoJLlLphjwlQ783JsRQLawUwxQeZbwMo1aXEAezqQ3TzOykku8q1skS2Vm5KrTxnvpV/Ke3ZNKzYEnj7yiYFoRcJ0CoF+wCvU4AbITYCBAdHG8NOGDWQuOgkmX+KeSruJArcx6bwEU7fr3eecqVUyvOUMy1KetbqJ/wtL4a0047Hk2JOz09qsT84+AMHbD8E+rvcYDAYDHt8Auedjskjlq16AAAAAElFTkSuQmCC","orcid":"","institution":"Bowie State University","correspondingAuthor":true,"prefix":"","firstName":"Adnan","middleName":"Khalid","lastName":"Bhatti","suffix":""},{"id":591812955,"identity":"f12d7139-5a57-4a75-9636-3d1e7226e902","order_by":1,"name":"Muhammad Shahzad Mughal","email":"","orcid":"","institution":"University of The Punjab","correspondingAuthor":false,"prefix":"","firstName":"Muhammad","middleName":"Shahzad","lastName":"Mughal","suffix":""}],"badges":[],"createdAt":"2026-01-31 18:08:06","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8751804/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8751804/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":104398198,"identity":"c1771790-441a-4b06-83c2-e3b156ae4433","added_by":"auto","created_at":"2026-03-11 12:00:31","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":539932,"visible":true,"origin":"","legend":"","description":"","filename":"MARLAutonomousSystemsSurvey.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8751804/v1_covered_779304c3-f528-453a-93e6-d18c16b1be27.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A Comprehensive Survey of Multi-Agent Reinforcement Learning for Autonomous Systems: Algorithms, Applications, and Open Challenges","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Multi-agent reinforcement learning, autonomous systems, decentralized control, cooperative learning, multi-robot systems, unmanned aerial vehicles, autonomous vehicle, decentralized training, centralized execution","lastPublishedDoi":"10.21203/rs.3.rs-8751804/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8751804/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eAutonomous systems are ever-performing tasks in complicated and multi-agent conditions in which coordination, scalability, safety, and reliability are key demands. In these environments, non-stationarity of the climate, decentralized information, and closely interacting agent dynamics are fundamental limitations to traditional single-agent reinforcement learning. Multi-agent reinforcement learning (MARL) has become an effective framework for addressing these issues, enabling agents to adopt cooperative, competitive, or a combination of strategies through interaction with one another. Nevertheless, even with the breakthroughs in the field of algorithms, the scalability of MARL remains limited in its application to real-world autonomous systems due to scalability constraints, communication assumptions, safety considerations, and theoretical guarantees of its success.\u003c/p\u003e\n\u003cp\u003eThe paper is a survey of MARL in autonomous systems in a critical and detailed manner, focusing on the algorithmic and application-driven perspectives. It is based on a systematic literature review methodology that is used to gather, filter, and group recent peer-reviewed works by MARL paradigms and autonomous system areas. MARL algorithms are divided into value-style, policy-style, and a hybrid style, and their advantages and disadvantages and implementation implications are comparatively discussed. The use in autonomous vehicles, UAV swarms, multi-robot systems, and industrial autonomous environments is discussed to underscore domain-specific limitations associated with the coordination, communication, and safety.\u003c/p\u003e\n\u003cp\u003eThe survey reveals through cross-domain synthesis that there are still persistent issues, such as scaling to large and heterogeneous sets of agents, the use of an idealized communication model, weak sim-to-real transfer, limited interpretability, and failure to achieve general convergence and safety guarantees. Lastly, the most important gaps in research are presented, and the future direction is proposed to make MARL a safer, more interpretable, and deployable autonomous system. This survey represents a structured and critical source of guidance to researchers and practitioners who wish to build robust MARL-enabled autonomy beyond evaluation processes within simulations.\u003c/p\u003e","manuscriptTitle":"A Comprehensive Survey of Multi-Agent Reinforcement Learning for Autonomous Systems: Algorithms, Applications, and Open Challenges","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-18 08:04:49","doi":"10.21203/rs.3.rs-8751804/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"4d53e710-e4c3-433d-9588-f67a88274569","owner":[],"postedDate":"February 18th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-02-27T00:23:53+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-18 08:04:49","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8751804","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8751804","identity":"rs-8751804","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}