Multi-Agent Reinforcement Learning for Cyber Defence Transferability and Scalability

Multi-Agent Reinforcement Learning for Cyber Defence Transferability and Scalability | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL Applied AI Letters This is a preprint and has not been peer reviewed. Data may be preliminary. 17 March 2025 V1 Latest version Share on Multi-Agent Reinforcement Learning for Cyber Defence Transferability and Scalability Authors : Andrew Thomas 0009-0005-7784-4345 [email protected] , Matthew Yates , and Oliver Osborne Authors Info & Affiliations https://doi.org/10.22541/au.174222566.61099769/v1 Published Applied AI Letters Version of record Peer review timeline 510 views 345 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract Reinforcement learning (RL) has shown to be effective for simple automated cyber defence (ACD) type tasks. However, there are limitations to these approaches that prevent them from being deployed onto real-world hardware. Trained policies will often have limited transferability across even small changes to the environment setup. Instability during training can prevent optimal learning, a problem that only increases as the environment scales and grows in complexity. In this work we look at addressing these limitations with a zero-shot transfer approach based on multi-agent reinforcement learning. Our approach partitions up the task into smaller network machine subtasks, where agents learn the solution to the local problem. These local agents are trained in a small-scale network, then transferred to larger networks by mapping the agents to machines in the new network. We have found that this transfer method is effective for direct application to a number of ACD tasks. We show that its performance is robust to changes in network activity, attack scenario and reduces the effects of network scale on performance. Supplementary Material File (multi-agent_reinforcement_learning_for_cyber_defence.pdf) Download 1.83 MB Information & Authors Information Version history V1 Version 1 17 March 2025 Peer review timeline Published Applied AI Letters Version of Record 22 Dec 2025 Published Copyright This work is licensed under a Non Exclusive No Reuse License. Collection Applied AI Letters Keywords autonomous cyber defense multi agent rl reinforcement learning rl transfer learning Authors Affiliations Andrew Thomas 0009-0005-7784-4345 [email protected] Raytheon Systems Ltd View all articles by this author Matthew Yates Raytheon Systems Ltd View all articles by this author Oliver Osborne Raytheon Systems Ltd View all articles by this author Metrics & Citations Metrics Article Usage 510 views 345 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Andrew Thomas, Matthew Yates, Oliver Osborne. Multi-Agent Reinforcement Learning for Cyber Defence Transferability and Scalability. Authorea . 17 March 2025. DOI: https://doi.org/10.22541/au.174222566.61099769/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.174222566.61099769/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9fe75d07089b593a',t:'MTc3OTIzODM5Ng=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();