BeamCraft: Deep Reinforcement Learning-DrivenMulti-Objective Beamforming for ISAC

BeamCraft: Deep Reinforcement Learning-DrivenMulti-Objective Beamforming for ISAC | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article BeamCraft: Deep Reinforcement Learning-DrivenMulti-Objective Beamforming for ISAC Duc Nguyen Dao, Yang Miao This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8762852/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Integrated sensing and communication (ISAC) is central to the vision of 5G-Advanced and 6G systems. Dynamic resource allocation remains one of the most challenging topics in ISAC, yet existing optimization methods struggle with fast-varying environment and inherent non-convexity of ISAC problems. This paper presents a deep reinforcement learning (DRL) framework that enables robust, real-time adaptive beamforming by jointly optimizing communication and sensing performance under dynamic environments. The proposed approach employs a Twin Delayed Deep Deterministic Policy Gradient with Prioritized Experience Replay (TD3-PER) algorithm to efficiently learn the optimal resource allocation strategy under continuous state and action spaces. A multi-objective reward formulation allows flexible adjustment of the ISAC performance weights, enabling the system to prioritize either communication or sensing performance depending on operational requirements. The training process consists of an offline phase, where the agent learns from extensive interactions with a simulated environment until convergence, and an online phase, where the trained model performs real-time inference and adapts to time-varying channel and target conditions. Simulation results demonstrate that the proposed DRL-based framework achieves fast online execution (millisecondlevel), strong robustness, and superior adaptability compared to conventional deterministic and other learning-based methods, making it a promising candidate for practical ISAC deployment in next-generation wireless networks. Electrical Engineering Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8762852","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":584179886,"identity":"32b84ddf-c790-4827-ae20-a65b391b7500","order_by":0,"name":"Duc Nguyen Dao","email":"","orcid":"","institution":"University of Twente","correspondingAuthor":false,"prefix":"","firstName":"Duc","middleName":"Nguyen","lastName":"Dao","suffix":""},{"id":584179887,"identity":"1b269566-f09e-40b6-a925-bb6579ca3023","order_by":1,"name":"Yang Miao","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAuElEQVRIiWNgGAWjYJCCAwwMNggeH5Fa0hh4oCwGNiItOkyCFoMD3IkHflScT9wvkXzs8YcKhjwitPBuONhz5nZij0RausGBMwzFRGk5wNsG1CKdYyZxsI0hsY0oW/62nQNqyf8mcfAfkVoO87YdANnCJnGwgQgtkoeBWmTOJBv33H9mJnHmmARhLXzHezd/fFNhJ9vec/iZREWNTWI/IS0Kh1H5EoQ0MDDINxBWMwpGwSgYBSMdAADihkRKoEBgjwAAAABJRU5ErkJggg==","orcid":"","institution":"University of Twente","correspondingAuthor":true,"prefix":"","firstName":"Yang","middleName":"","lastName":"Miao","suffix":""}],"badges":[],"createdAt":"2026-02-02 09:14:23","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8762852/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8762852/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":101754633,"identity":"9ebf0537-df23-456b-bd5b-2b8c43704950","added_by":"auto","created_at":"2026-02-03 10:43:07","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":10946362,"visible":true,"origin":"","legend":"","description":"","filename":"NatureBeamforming.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8762852/v1_covered_298cbbd5-75c1-4661-a4b9-1734e2745809.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eBeamCraft: Deep Reinforcement Learning-DrivenMulti-Objective Beamforming for ISAC\u003c/strong\u003e\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[{"identity":"abf1a14f-bdf7-4c52-b472-4297ada626bb","identifier":"10.13039/501100003246","name":"Nederlandse Organisatie voor Wetenschappelijk Onderzoek","awardNumber":"19751","order_by":0}],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"University of Twente","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-8762852/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8762852/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIntegrated sensing and communication (ISAC) is central to the vision of 5G-Advanced and 6G systems. Dynamic resource allocation remains one of the most challenging topics in ISAC, yet existing optimization methods struggle with fast-varying environment and inherent non-convexity of ISAC problems. This paper presents a deep reinforcement learning (DRL) framework that enables robust, real-time adaptive beamforming by jointly optimizing communication and sensing performance under dynamic environments. The proposed approach employs a Twin Delayed Deep Deterministic Policy Gradient with Prioritized Experience Replay (TD3-PER) algorithm to efficiently learn the optimal resource allocation strategy under continuous state and action spaces. A multi-objective reward formulation allows flexible adjustment of the ISAC performance weights, enabling the system to prioritize either communication or sensing performance depending on operational requirements. The training process consists of an offline phase, where the agent learns from extensive interactions with a simulated environment until convergence, and an online phase, where the trained model performs real-time inference and adapts to time-varying channel and target conditions. Simulation results demonstrate that the proposed DRL-based framework achieves fast online execution (millisecondlevel), strong robustness, and superior adaptability compared to conventional deterministic and other learning-based methods, making it a promising candidate for practical ISAC deployment in next-generation wireless networks.\u003c/p\u003e","manuscriptTitle":"BeamCraft: Deep Reinforcement Learning-DrivenMulti-Objective Beamforming for ISAC","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-03 08:33:51","doi":"10.21203/rs.3.rs-8762852/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"d9745778-90dd-4ca6-bba5-5caaa433b286","owner":[],"postedDate":"February 3rd, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":62143894,"name":"Electrical Engineering"}],"tags":[],"updatedAt":"2026-02-03T08:33:51+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-03 08:33:51","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8762852","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8762852","identity":"rs-8762852","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}