Evaluating Adversarial Robustness of AI Intrusion Detection Systems Using Automated Traffic Generation

Evaluating Adversarial Robustness of AI Intrusion Detection Systems Using Automated Traffic Generation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Evaluating Adversarial Robustness of AI Intrusion Detection Systems Using Automated Traffic Generation Samer Aoudi, Hussain Al-Aqrabi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8316582/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Artificial intelligence (AI) has substantially advanced intrusion detection systems (IDS) by enabling scalable and adaptive analysis of network flows, yet these models remain vulnerable to adversarial manipulation. This study develops a comprehensive and reproducible framework for evaluating the adversarial robustness of machine learning-based IDS under realistic black-box threat conditions. Using CICIDS2017 as the primary benchmark, four representative IDS models, Random Forest, Logistic Regression, Multilayer Perceptron, and CNN1D, are trained and assessed under clean conditions and multiple adversarial scenarios, including surrogate-based FGSM and PGD perturbations, as well as HopSkipJump (HSJA) and Zeroth-Order Optimization (ZOO) black-box attacks. The results reveal substantial differences in robustness across model families: Random Forest remains consistently stable across all attacks, whereas the MLP exhibits severe performance degradation under PGD transfer. Logistic Regression and CNN1D show mixed susceptibility depending on attack strength and feature sensitivity. To evaluate generalization under distribution shift, the models are further tested on the CICIDS2018 Friday slice, showing limited cross-dataset transferability of adversarial examples. These findings reveal significant differences between white-box and operational black-box vulnerability and demonstrate that adversarial robustness depends strongly on model architecture and dataset alignment. The proposed evaluation methodology provides a practical basis for integrating adversarial stress testing into IDS development and deployment workflows. adversarial machine learning network intrusion detection artificial intelligence cybersecurity flow features model robustness transfer-based attacks black-box evasion Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8316582","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":565418473,"identity":"e7dee6e9-3a3c-45a7-a360-4b1b48da410f","order_by":0,"name":"Samer Aoudi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAtUlEQVRIiWNgGAWjYLACHgObBJK1pCUwsJGmheEwCVrk2w8fk3hTcD6Pf34D44cfDHZyBLUYnElLk5xjcLtY4hgDs2QPQ7IxYS0MOWbSPAa3EzcAHSbNwMCc2EDQYf1vQFrOgbQw/2ZgqK8nqIXhBtiWAyAtbEBbDicQdtiNZ8mWcwySE2ccS2yz7DE4bkiEw5IP3njzxy6xv/nw4Rs/KqrlCdqCBBgbQKExCkbBKBgFo4AaAABgWDU2zm7iPwAAAABJRU5ErkJggg==","orcid":"","institution":"Higher Colleges of Technology","correspondingAuthor":true,"prefix":"","firstName":"Samer","middleName":"","lastName":"Aoudi","suffix":""},{"id":565418474,"identity":"bc9bd439-ece8-4d18-923f-6ed153214c79","order_by":1,"name":"Hussain Al-Aqrabi","email":"","orcid":"","institution":"Higher Colleges of Technology","correspondingAuthor":false,"prefix":"","firstName":"Hussain","middleName":"","lastName":"Al-Aqrabi","suffix":""}],"badges":[],"createdAt":"2025-12-09 10:38:33","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8316582/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8316582/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":99313436,"identity":"cf9533e2-f7e6-45be-81fb-472262cea669","added_by":"auto","created_at":"2025-12-31 16:20:10","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1625470,"visible":true,"origin":"","legend":"","description":"","filename":"EvaluatingAdversarialRobustnessofAIIDSUsingAutomatedTrafficGeneration.docx","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/0d702c1c7497bc09736d9ab8.docx"},{"id":99313971,"identity":"8a10c24a-a899-459f-aeee-49deeaefd565","added_by":"auto","created_at":"2025-12-31 16:20:42","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5395,"visible":true,"origin":"","legend":"","description":"","filename":"321deea609fc44cca2fa4eba6b8637b9.json","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/60726a333f76e7fb74bb0a0c.json"},{"id":99313957,"identity":"d620325b-ca79-4515-99cc-410cdf0fdf31","added_by":"auto","created_at":"2025-12-31 16:20:40","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":157764,"visible":true,"origin":"","legend":"","description":"","filename":"321deea609fc44cca2fa4eba6b8637b91enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/b7ed7a5719f0e4fa85234328.xml"},{"id":99033275,"identity":"5df3c74e-deda-4da6-abfd-cf3b29aa84e8","added_by":"auto","created_at":"2025-12-26 09:16:28","extension":"png","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":76782,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/7b8855f2e2075d0b77a125b4.png"},{"id":99313624,"identity":"19c2b882-7a31-403d-9257-8aaec0a8e596","added_by":"auto","created_at":"2025-12-31 16:20:21","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":53570,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/d2640ff3d9d678e6333bc512.png"},{"id":99033277,"identity":"e5396454-0779-4502-94ff-1de82e179377","added_by":"auto","created_at":"2025-12-26 09:16:28","extension":"png","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":251660,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/0d9eff07dab7775ebffccee8.png"},{"id":99313890,"identity":"ede775fb-d407-4c95-bc08-1ce37832a46f","added_by":"auto","created_at":"2025-12-31 16:20:34","extension":"png","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":803154,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/a1c252f845e1aec4bd4dade8.png"},{"id":99313453,"identity":"cf85b662-1cf7-417d-86ac-45d32a6f2ac5","added_by":"auto","created_at":"2025-12-31 16:20:11","extension":"png","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":147381,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/393272605045cf24cc7ee065.png"},{"id":99033282,"identity":"7ffd5eb9-c00f-4d58-af28-d5879da0a556","added_by":"auto","created_at":"2025-12-26 09:16:29","extension":"png","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":117791,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/2e6628b22471fdfacad638a0.png"},{"id":99033284,"identity":"7e70e00e-1405-4610-9241-f88dcef57bdb","added_by":"auto","created_at":"2025-12-26 09:16:29","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":85934,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/86b69337bc470f4df42b8948.png"},{"id":99313524,"identity":"a21e9930-24da-40ad-bc11-4754c6515c2d","added_by":"auto","created_at":"2025-12-31 16:20:15","extension":"png","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":36617,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/6fbb2719e237ed079ed404f0.png"},{"id":99313213,"identity":"7e04a3ba-a902-4c22-a174-8cc25b458b06","added_by":"auto","created_at":"2025-12-31 16:19:54","extension":"png","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":43922,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage9.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/1d3cae952e638fa35d2459b7.png"},{"id":99313388,"identity":"9f30e1d2-8be4-40f8-ad1c-a8974d4fda2e","added_by":"auto","created_at":"2025-12-31 16:20:05","extension":"png","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":23906,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/b60bba0b373c6ba0f4aca867.png"},{"id":99314400,"identity":"7f6b9653-376f-4b9a-9321-fb1cbfae6bcf","added_by":"auto","created_at":"2025-12-31 16:21:21","extension":"png","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":16081,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/48b0218e6d617be4c8abcbe9.png"},{"id":99033287,"identity":"d3f8bb46-9e1d-4806-bb99-f500bb217e02","added_by":"auto","created_at":"2025-12-26 09:16:29","extension":"png","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":50858,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/e54cdd0d5bf04ef3971a9d27.png"},{"id":99313905,"identity":"ed463fb0-999d-4c88-9d55-dfa192d58042","added_by":"auto","created_at":"2025-12-31 16:20:35","extension":"png","order_by":15,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":80649,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/5ed3814d202f0cec45ca1b41.png"},{"id":99313855,"identity":"d8f34d4e-fbfb-4235-a82b-8d83f39bae6a","added_by":"auto","created_at":"2025-12-31 16:20:33","extension":"png","order_by":16,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":19800,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/1889b8ffb632187d71caa8d3.png"},{"id":99033298,"identity":"22fd7275-d086-4286-9b95-ca2e76538d52","added_by":"auto","created_at":"2025-12-26 09:16:29","extension":"png","order_by":17,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":21714,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/dc6053071038bcb30b1b74f1.png"},{"id":99033292,"identity":"0f3468a9-845a-4dc5-a13c-5c8f7229124e","added_by":"auto","created_at":"2025-12-26 09:16:29","extension":"png","order_by":18,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":20449,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/4b98f126d4f60f28662af086.png"},{"id":99033295,"identity":"58c97675-5f43-4622-a05e-1e3e294ef074","added_by":"auto","created_at":"2025-12-26 09:16:29","extension":"png","order_by":19,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":10190,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/4cb11c108144007aac5de823.png"},{"id":99313570,"identity":"1d541e7d-c85e-497e-966e-cfa354e80d41","added_by":"auto","created_at":"2025-12-31 16:20:17","extension":"png","order_by":20,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":11745,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage9.png","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/8b6b677d09e9bc04c76cdd72.png"},{"id":99033301,"identity":"b77e72c4-c5b8-4910-8da1-416aa7d0b095","added_by":"auto","created_at":"2025-12-26 09:16:29","extension":"xml","order_by":21,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":152217,"visible":true,"origin":"","legend":"","description":"","filename":"321deea609fc44cca2fa4eba6b8637b91structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/83ffd120917a837011900884.xml"},{"id":99033300,"identity":"e8ed232e-c9fc-49b9-94b0-aee725d62f40","added_by":"auto","created_at":"2025-12-26 09:16:29","extension":"html","order_by":22,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":170555,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1/7106af2da7fd7bd6168e7022.html"},{"id":105064845,"identity":"931a0079-3acf-4a80-8ab8-c2666b720dea","added_by":"auto","created_at":"2026-03-20 13:41:35","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1072287,"visible":true,"origin":"","legend":"","description":"","filename":"EvaluatingAdversarialRobustnessofAIIDSUsingAutomatedTrafficGeneration.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8316582/v1_covered_cc4db289-6556-4335-945d-8ad12e0d03e3.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Evaluating Adversarial Robustness of AI Intrusion Detection Systems Using Automated Traffic Generation","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"adversarial machine learning, network intrusion detection, artificial intelligence, cybersecurity, flow features, model robustness, transfer-based attacks, black-box evasion","lastPublishedDoi":"10.21203/rs.3.rs-8316582/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8316582/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eArtificial intelligence (AI) has substantially advanced intrusion detection systems (IDS) by enabling scalable and adaptive analysis of network flows, yet these models remain vulnerable to adversarial manipulation. This study develops a comprehensive and reproducible framework for evaluating the adversarial robustness of machine learning-based IDS under realistic black-box threat conditions. Using CICIDS2017 as the primary benchmark, four representative IDS models, Random Forest, Logistic Regression, Multilayer Perceptron, and CNN1D, are trained and assessed under clean conditions and multiple adversarial scenarios, including surrogate-based FGSM and PGD perturbations, as well as HopSkipJump (HSJA) and Zeroth-Order Optimization (ZOO) black-box attacks. The results reveal substantial differences in robustness across model families: Random Forest remains consistently stable across all attacks, whereas the MLP exhibits severe performance degradation under PGD transfer. Logistic Regression and CNN1D show mixed susceptibility depending on attack strength and feature sensitivity. To evaluate generalization under distribution shift, the models are further tested on the CICIDS2018 Friday slice, showing limited cross-dataset transferability of adversarial examples. These findings reveal significant differences between white-box and operational black-box vulnerability and demonstrate that adversarial robustness depends strongly on model architecture and dataset alignment. The proposed evaluation methodology provides a practical basis for integrating adversarial stress testing into IDS development and deployment workflows.\u003c/p\u003e","manuscriptTitle":"Evaluating Adversarial Robustness of AI Intrusion Detection Systems Using Automated Traffic Generation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-12-26 09:16:24","doi":"10.21203/rs.3.rs-8316582/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"3751cadd-0b58-45b9-8923-162474814183","owner":[],"postedDate":"December 26th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-03-20T13:41:10+00:00","versionOfRecord":[],"versionCreatedAt":"2025-12-26 09:16:24","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8316582","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8316582","identity":"rs-8316582","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}