Characterization of a Fixed Reinforcement Learning Policy for Aerial Robot with Suspended Payload under Variable Flight Conditions

Characterization of a Fixed Reinforcement Learning Policy for Aerial Robot with Suspended Payload under Variable Flight Conditions | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Characterization of a Fixed Reinforcement Learning Policy for Aerial Robot with Suspended Payload under Variable Flight Conditions Ali Tahir Karasahin, Ziniu Wu, Basaran Bahadir Kocer This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8603104/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 10 You are reading this latest preprint version Abstract Flights with suspended payloads are particularly challenging because of their coupled dynamics, which lead to instability and increased sensitivity to disturbances. Although reinforcement learning (RL) has successfully achieved controller performance, the generalization and robustness of a single policy remain significant areas of investigation. In this study, we characterized the performance and robustness of a single RL policy for an aerial robot with different trajectory profiles, including a smooth, feasible lemniscate curve and a sharp-turning, infeasible pentagram, under varying velocity references (0.5 m/s and 1.0 m/s) and crosswind disturbances (1.0 m/s). We trained a single RL policy using Proximal Policy Optimization (PPO) with collective thrust and body-rate (CTBR) control using a high-fidelity physics simulator based on the SimpleFlight framework. Real-world experimental results on the Crazyflie 2.1 platform show that the single RL policy successfully generalizes to different trajectory profiles and velocity references and maintains stability under a crosswind disturbance of up to 1.0 m/s which is a substantial challenge for this small class platform and even smaller payload underneath, where such aerodynamic forces are significant compared to the available control authority and system mass. Furthermore, the single RL policy was systematically evaluated using the Mean Euclidean distance (MED) error, cable length transitions, and swing angle distributions. Although the single RL policy maintained a robust control performance, the experimental results indicated performance degradation at higher velocities owing to increased dynamic challenges such as nonlinear aerodynamic drag and actuator saturation. This study provides a detailed performance characterization that highlights the generalization capability of a single-payload-aware RL policy in real-world applications and the limitations arising from the hybrid dynamics of the system. Physical sciences/Engineering Physical sciences/Mathematics and computing Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 10 Mar, 2026 Reviews received at journal 08 Mar, 2026 Reviewers agreed at journal 05 Mar, 2026 Reviews received at journal 02 Mar, 2026 Reviewers agreed at journal 28 Feb, 2026 Reviewers agreed at journal 26 Feb, 2026 Reviewers invited by journal 26 Feb, 2026 Editor assigned by journal 16 Jan, 2026 Submission checks completed at journal 16 Jan, 2026 First submitted to journal 14 Jan, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8603104","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":599527843,"identity":"5e739d89-6da3-40ae-924d-ead2cc47f7dd","order_by":0,"name":"Ali Tahir Karasahin","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABTElEQVRIie3RPWvCQBjA8ec4OJekWS+o9SskBGwL4meJBNqlL4NQKILeIZyL7hb78iGE4mjJ4BI7Z+hgKbSLgtKl0hR6poUSTZ073H/KXe7Hc3AAKtV/LIMYIEYBtO+1SwAP5Q5Y8VJLIRhxtkaImyB0g8jf8sAvkV/WVmI0cWuyHOyfgea/zN+j8ulOtr14vhiUHaPH0GQmoL7HEoT6iPNOQA+YLpzLjvCqJDfu2+PAK9LHIbavBdDcMDlGEnmaWmCAAzrDFUFP7kwucAlCl2R1SdZuVlhN+YxJ5g1FUUOS49clF41SIXQzHynEkqQZT9HbDtaIvyIEceEXLTkFpxBbkl5eEqIFVZwXI0kOHZMHI8cOK03z6oGa3STZHbUmi5moW4Z21EfTqFa57XpPCz6o2Tehdz+fnpeMjYf5iaTsxe/1F1CpVCrVlr4AUQhvtKgG6QUAAAAASUVORK5CYII=","orcid":"","institution":"Necmettin Erbakan University","correspondingAuthor":true,"prefix":"","firstName":"Ali","middleName":"Tahir","lastName":"Karasahin","suffix":""},{"id":599527844,"identity":"6d1d3a61-6b8a-4808-8444-86e8ad1a9627","order_by":1,"name":"Ziniu Wu","email":"","orcid":"","institution":"University of Bristol","correspondingAuthor":false,"prefix":"","firstName":"Ziniu","middleName":"","lastName":"Wu","suffix":""},{"id":599527845,"identity":"cb83fb50-6c69-458e-b762-e97ff3e2c29f","order_by":2,"name":"Basaran Bahadir Kocer","email":"","orcid":"","institution":"University of Bristol","correspondingAuthor":false,"prefix":"","firstName":"Basaran","middleName":"Bahadir","lastName":"Kocer","suffix":""}],"badges":[],"createdAt":"2026-01-14 15:08:08","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8603104/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8603104/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":104402105,"identity":"2c17d725-d530-4521-a9eb-002949575a23","added_by":"auto","created_at":"2026-03-11 12:14:19","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":6274927,"visible":true,"origin":"","legend":"","description":"","filename":"2025ScientificReportsPerformanceCharacterizationofaFixedRLPolicyforAerialRobotwithSuspendedPayloadunderVariableFlightConditionsv1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8603104/v1_covered_39734002-9e0e-48b6-952d-ec821df7839e.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Characterization of a Fixed Reinforcement Learning Policy for Aerial Robot with Suspended Payload under Variable Flight Conditions","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-8603104/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8603104/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Flights with suspended payloads are particularly challenging because of their coupled dynamics, which lead to instability and increased sensitivity to disturbances. Although reinforcement learning (RL) has successfully achieved controller performance, the generalization and robustness of a single policy remain significant areas of investigation. In this study, we characterized the performance and robustness of a single RL policy for an aerial robot with different trajectory profiles, including a smooth, feasible lemniscate curve and a sharp-turning, infeasible pentagram, under varying velocity references (0.5 m/s and 1.0 m/s) and crosswind disturbances (1.0 m/s). We trained a single RL policy using Proximal Policy Optimization (PPO) with collective thrust and body-rate (CTBR) control using a high-fidelity physics simulator based on the SimpleFlight framework. Real-world experimental results on the Crazyflie 2.1 platform show that the single RL policy successfully generalizes to different trajectory profiles and velocity references and maintains stability under a crosswind disturbance of up to 1.0 m/s which is a substantial challenge for this small class platform and even smaller payload underneath, where such aerodynamic forces are significant compared to the available control authority and system mass. Furthermore, the single RL policy was systematically evaluated using the Mean Euclidean distance (MED) error, cable length transitions, and swing angle distributions. Although the single RL policy maintained a robust control performance, the experimental results indicated performance degradation at higher velocities owing to increased dynamic challenges such as nonlinear aerodynamic drag and actuator saturation. This study provides a detailed performance characterization that highlights the generalization capability of a single-payload-aware RL policy in real-world applications and the limitations arising from the hybrid dynamics of the system.","manuscriptTitle":"Characterization of a Fixed Reinforcement Learning Policy for Aerial Robot with Suspended Payload under Variable Flight Conditions","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-04 12:50:58","doi":"10.21203/rs.3.rs-8603104/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-03-10T05:27:22+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-08T18:13:03+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"183323024132451231343843599862157442222","date":"2026-03-06T02:54:43+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-02T10:48:35+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"175578008808255090182667133703496069308","date":"2026-02-28T10:43:25+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"135659171749997754256719241604252618927","date":"2026-02-26T22:02:20+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-02-26T17:07:08+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-01-16T12:21:23+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-01-16T12:15:18+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2026-01-14T14:53:25+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e0a24606-093b-4ed6-b46e-fbe8d04d193a","owner":[],"postedDate":"March 4th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":63795485,"name":"Physical sciences/Engineering"},{"id":63795486,"name":"Physical sciences/Mathematics and computing"}],"tags":[],"updatedAt":"2026-03-24T05:55:46+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-04 12:50:58","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8603104","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8603104","identity":"rs-8603104","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}