Scalable multi-agent reinforcement learning based on battlefield situation transfer and curriculum learning

Scalable multi-agent reinforcement learning based on battlefield situation transfer and curriculum learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Scalable multi-agent reinforcement learning based on battlefield situation transfer and curriculum learning Yihuan Wang, Yaofei Ma, Jiangyun wang, Xutao Feng This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4648387/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Due to its potential application value, multi-UAV air combat missions have become a hot topic in the current military research field at home and abroad. With the rise of artificial intelligence and intelligent warfare, a series of UAV air combat decision-making methods based on deep reinforcement learning have been proposed by various countries. However, when the number of agents increases and the type of aircraft changes, the action space and state space of reinforcement learning will change, which will face the problem of poor stability of the training process and is extremely sensitive to hyperparameters (such as learning rate). Therefore, based on the traditional multi-agent reinforcement learning based on AC decision network, this paper improves the decision network, introduces curriculum learning and transfer learning, and proposes a multi-agent reinforcement learning decision framework that can converge quickly and improve training robustness. Aircombat Transfer learning Curriculum learning Multi-agent reinforcement learning Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4648387","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":333846822,"identity":"9b24e21c-4f1f-47ac-ab9d-f090255798b7","order_by":0,"name":"Yihuan Wang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA90lEQVRIiWNgGAWjYFAC5oMPEn7Y8PCz90AFDhDUwpZs8LEnTU6y5wzRWnjMJGewHTY2uJFDpBb5GQkG0jw8aYkbbr49JvGjhkGO70YC4+cCPFoYZyQkGPNY2CTOvJ2XJtlzjMFY8kYCs/QMPFqYJRIOJINs6budYybBw8aQuOFGAhszDx4tbBKJDYd52A4nNtw8Yyb55x9DPUEtPBLJjI0g7wvc4DGT5m1jSDAgpEWC5xkzAySQ85KtZfskDGeeedgsjU+LfHv+9x+QqDx78OabbzbyfMeTD37Gp4VBIAHVViBmbMCngYGB/wB++VEwCkbBKBgFDABXcE3Vmn3yVgAAAABJRU5ErkJggg==","orcid":"","institution":"Beihang University School of Automation Science and Electrical Engineering","correspondingAuthor":true,"prefix":"","firstName":"Yihuan","middleName":"","lastName":"Wang","suffix":""},{"id":333846823,"identity":"ff182547-5fc0-473a-97a6-266ad437503c","order_by":1,"name":"Yaofei Ma","email":"","orcid":"","institution":"Beihang University School of Automation Science and Electrical Engineering","correspondingAuthor":false,"prefix":"","firstName":"Yaofei","middleName":"","lastName":"Ma","suffix":""},{"id":333846824,"identity":"91d85949-98f3-49ab-b3d8-0d8ae53e8a43","order_by":2,"name":"Jiangyun wang","email":"","orcid":"","institution":"Beihang University School of Automation Science and Electrical Engineering","correspondingAuthor":false,"prefix":"","firstName":"Jiangyun","middleName":"","lastName":"wang","suffix":""},{"id":333846825,"identity":"b6af7c03-1963-4fa2-92a8-55885edbab49","order_by":3,"name":"Xutao Feng","email":"","orcid":"","institution":"Beihang University School of Automation Science and Electrical Engineering","correspondingAuthor":false,"prefix":"","firstName":"Xutao","middleName":"","lastName":"Feng","suffix":""}],"badges":[],"createdAt":"2024-06-27 12:02:42","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4648387/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4648387/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":61537828,"identity":"3efde65d-21ae-4747-8020-d80296ed4cbb","added_by":"auto","created_at":"2024-08-01 02:35:56","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":659297,"visible":true,"origin":"","legend":"","description":"","filename":"ACML.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4648387/v1_covered_0918a2c8-e70b-4233-8d72-894624fb1adb.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Scalable multi-agent reinforcement learning based on battlefield situation transfer and curriculum learning","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Aircombat, Transfer learning, Curriculum learning, Multi-agent reinforcement learning","lastPublishedDoi":"10.21203/rs.3.rs-4648387/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4648387/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eDue to its potential application value, multi-UAV air combat missions have become a hot topic in the current military research field at home and abroad. With the rise of artificial intelligence and intelligent warfare, a series of UAV air combat decision-making methods based on deep reinforcement learning have been proposed by various countries. However, when the number of agents increases and the type of aircraft changes, the action space and state space of reinforcement learning will change, which will face the problem of poor stability of the training process and is extremely sensitive to hyperparameters (such as learning rate). Therefore, based on the traditional multi-agent reinforcement learning based on AC decision network, this paper improves the decision network, introduces curriculum learning and transfer learning, and proposes a multi-agent reinforcement learning decision framework that can converge quickly and improve training robustness.\u003c/p\u003e","manuscriptTitle":"Scalable multi-agent reinforcement learning based on battlefield situation transfer and curriculum learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-08-01 02:27:49","doi":"10.21203/rs.3.rs-4648387/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"fed4d7d5-a8fc-4cb4-9996-a33e6ecc3c8c","owner":[],"postedDate":"August 1st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-08-01T02:27:49+00:00","versionOfRecord":[],"versionCreatedAt":"2024-08-01 02:27:49","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4648387","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4648387","identity":"rs-4648387","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}