Queue-Aware Admission Control for Safe RL-Based Computation Offloading: Cross-Policy Evaluation and Zero-Shot Retrofit

Queue-Aware Admission Control for Safe RL-Based Computation Offloading: Cross-Policy Evaluation and Zero-Shot Retrofit | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Queue-Aware Admission Control for Safe RL-Based Computation Offloading: Cross-Policy Evaluation and Zero-Shot Retrofit Bingchi Sun This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9533648/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 10 You are reading this latest preprint version Abstract Reinforcement learning (RL) enables adaptive computation offloading in mobile edge computing (MEC), yet learned policies may keep offloading into a congested shared edge queue, degrading service quality for all co-located devices. This article proposes Queue-Aware Admission Control (QAAC), a plug-and-play admission layer that translates real-time edge-queue occupancy into a per-step offloading budget and can wrap any pre-trained RL policy without modifying its architecture or retraining. We derive Lyapunov-style drift bounds showing that the controlled edge queue remains positive recurrent under both linear and sigmoid budget mappings. Empirically, we evaluate QAAC under a unified 20-seed × 30-episode protocol across three RL families. At moderate edge capacity (Ce=4), where the edge retains residual service headroom, QAAC reduces the violation metric by 10–14% (absolute Δ=0.032–0.036) for DQN, PPO, and Lagrangian DQN (p<0.001, Cohen's d=2.06–2.61). At tight capacity (Ce=2), specialized baselines such as Lyapunov control remain stronger on mean performance; QAAC's contribution in this regime is variance compression, halving PPO's cross-seed standard deviation without improving the mean. A zero-shot retrofit experiment shows that attaching QAAC to a frozen PPO controller achieves 0.430±0.038, matching retrained Lagrangian DQN (0.433±0.040) without any retraining cost. Under capacity-drop stress (train at Ce=4, evaluate at Ce=2), QAAC reduces violations by 32–36% across all three RL policies (p<0.001), an effect that is partly expected under severe capacity mismatch. We further validate the mechanism in a heterogeneous 20-device environment with mixed device types, Markov-modulated arrivals, and Rayleigh fading channels, where admission-controlled policies reduce violations by 17–27% compared to ungated baselines (p2.7). The analysis is limited to queue-level behavior and does not establish per-task delay guarantees; extending the analytical scope represents future work. Physical sciences/Engineering Physical sciences/Mathematics and computing Mobile edge computing Computation offloading Reinforcement learning Admission control Queue management Lyapunov analysis Safe reinforcement learning Full Text Additional Declarations No competing interests reported. Supplementary Files supplementarycode.zip Cite Share Download PDF Status: Under Review Version 1 posted Reviewers agreed at journal 18 May, 2026 Reviews received at journal 17 May, 2026 Reviews received at journal 09 May, 2026 Reviewers agreed at journal 09 May, 2026 Reviewers agreed at journal 09 May, 2026 Reviewers invited by journal 07 May, 2026 Editor invited by journal 29 Apr, 2026 Editor assigned by journal 27 Apr, 2026 Submission checks completed at journal 27 Apr, 2026 First submitted to journal 26 Apr, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9533648","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":641463712,"identity":"e4f83632-940b-42d6-9ca6-dc291830ee47","order_by":0,"name":"Bingchi Sun","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7klEQVRIiWNgGAWjYNACg388/MzMBw58qJCQkyesnBmIKw7ISba3JT6cccbC2LCBKC1nDhgbnDljbMzZVpHIcICABnP2/oOPK9vuJG64kZYmzThPIoGxgfnhoxt4tFj2HGY2PNv2LHHmjeRj0oXbJPLYGdiMjXPwaDG4kcwm2djGnNgHsmXmNolixgYeNmm8Wu4/Zv8J0tJwI8dMmneORGLDAUJabjCzMTacOWwsAPI+bwMRWix7ko0lGyrSoIF8TMLYsJmAX8zZDz782GBgA43Kmjo5efbmh4/xOgxTiBmPchxaRsEoGAWjYBSgAQABX1LiZpDnrAAAAABJRU5ErkJggg==","orcid":"","institution":"Zhejiang University of Technology","correspondingAuthor":true,"prefix":"","firstName":"Bingchi","middleName":"","lastName":"Sun","suffix":""}],"badges":[],"createdAt":"2026-04-26 17:08:28","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9533648/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9533648/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":109799408,"identity":"34a6f09e-6871-4c42-8b39-948ba8b7f8f3","added_by":"auto","created_at":"2026-05-22 15:28:05","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1745670,"visible":true,"origin":"","legend":"","description":"","filename":"manuscriptsubmission.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9533648/v1_covered_74b4d555-4660-4a16-8635-7d95b7a87bea.pdf"},{"id":109435542,"identity":"ac60e210-3d65-4b14-8649-5e19697e73c1","added_by":"auto","created_at":"2026-05-18 06:06:26","extension":"zip","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":1350252,"visible":true,"origin":"","legend":"","description":"","filename":"supplementarycode.zip","url":"https://assets-eu.researchsquare.com/files/rs-9533648/v1/3264ce79fd811f19bc74e27a.zip"}],"financialInterests":"No competing interests reported.","formattedTitle":"Queue-Aware Admission Control for Safe RL-Based Computation Offloading: Cross-Policy Evaluation and Zero-Shot Retrofit","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Mobile edge computing, Computation offloading, Reinforcement learning, Admission control, Queue management, Lyapunov analysis, Safe reinforcement learning","lastPublishedDoi":"10.21203/rs.3.rs-9533648/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9533648/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Reinforcement learning (RL) enables adaptive computation offloading in mobile edge computing (MEC), yet learned policies may keep offloading into a congested shared edge queue, degrading service quality for all co-located devices. This article proposes Queue-Aware Admission Control (QAAC), a plug-and-play admission layer that translates real-time edge-queue occupancy into a per-step offloading budget and can wrap any pre-trained RL policy without modifying its architecture or retraining. We derive Lyapunov-style drift bounds showing that the controlled edge queue remains positive recurrent under both linear and sigmoid budget mappings. Empirically, we evaluate QAAC under a unified 20-seed × 30-episode protocol across three RL families. At moderate edge capacity (Ce=4), where the edge retains residual service headroom, QAAC reduces the violation metric by 10–14% (absolute Δ=0.032–0.036) for DQN, PPO, and Lagrangian DQN (p\u003c0.001, Cohen's d=2.06–2.61). At tight capacity (Ce=2), specialized baselines such as Lyapunov control remain stronger on mean performance; QAAC's contribution in this regime is variance compression, halving PPO's cross-seed standard deviation without improving the mean. A zero-shot retrofit experiment shows that attaching QAAC to a frozen PPO controller achieves 0.430±0.038, matching retrained Lagrangian DQN (0.433±0.040) without any retraining cost. Under capacity-drop stress (train at Ce=4, evaluate at Ce=2), QAAC reduces violations by 32–36% across all three RL policies (p\u003c0.001), an effect that is partly expected under severe capacity mismatch. We further validate the mechanism in a heterogeneous 20-device environment with mixed device types, Markov-modulated arrivals, and Rayleigh fading channels, where admission-controlled policies reduce violations by 17–27% compared to ungated baselines (p\u003c0.001, d\u003e2.7). The analysis is limited to queue-level behavior and does not establish per-task delay guarantees; extending the analytical scope represents future work.","manuscriptTitle":"Queue-Aware Admission Control for Safe RL-Based Computation Offloading: Cross-Policy Evaluation and Zero-Shot Retrofit","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-05-18 06:06:21","doi":"10.21203/rs.3.rs-9533648/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewerAgreed","content":"280222714066256605785805313903376873357","date":"2026-05-18T12:13:29+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-18T01:16:42+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-09T09:37:44+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"122417303665611405228836132710488532881","date":"2026-05-09T09:11:12+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"64500530525024276680517874617710193834","date":"2026-05-09T08:27:10+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-05-07T08:21:44+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2026-04-29T15:06:58+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-04-27T05:52:39+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-04-27T05:52:19+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2026-04-26T17:05:48+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"c1eafa3b-8054-485f-a3d0-2f7d91e80cd2","owner":[],"postedDate":"May 18th, 2026","published":true,"recentEditorialEvents":[{"type":"reviewerAgreed","content":"280222714066256605785805313903376873357","date":"2026-05-18T12:13:29+00:00","index":49,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-18T01:16:42+00:00","index":45,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-09T09:37:44+00:00","index":44,"fulltext":""},{"type":"reviewerAgreed","content":"122417303665611405228836132710488532881","date":"2026-05-09T09:11:12+00:00","index":43,"fulltext":""},{"type":"reviewerAgreed","content":"64500530525024276680517874617710193834","date":"2026-05-09T08:27:10+00:00","index":42,"fulltext":""},{"type":"reviewersInvited","content":"9","date":"2026-05-07T08:21:44+00:00","index":"","fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":68234809,"name":"Physical sciences/Engineering"},{"id":68234810,"name":"Physical sciences/Mathematics and computing"}],"tags":[],"updatedAt":"2026-05-18T06:06:22+00:00","versionOfRecord":[],"versionCreatedAt":"2026-05-18 06:06:21","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9533648","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9533648","identity":"rs-9533648","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}