Development of Perception and Control Applications for Collaborative Robots Using ROS2

Development of Perception and Control Applications for Collaborative Robots Using ROS2 | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Development of Perception and Control Applications for Collaborative Robots Using ROS2 Anonione da Silva Mascarenhas Pinho, Cassius Zanetti Resende, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5664100/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract This work explores the integration of perception systems in behavior-based control applications using ROS2, with a focus on safe and effective interactions between humans and collaborative robots. Unlike conventional industrial robots, collaborative robots are designed to interact directly with human operators without physical barriers, making automation more accessible, particularly for small and medium-sized enterprises. A system integrating the UR3e robot and the Intel RealSense D435i camera was developed, enabling the robot to recognize objects and interpret gestures. YOLOv8 was used for object detection and segmentation, MediaPipe for gesture recognition and face detection, and Detectron2 for human detection, empowering the robot to manipulate objects and respond to gesture commands. The system's efficiency was validated through practical tests by moving objects between known points. The study highlights opportunities for improvement, such as replacing Detectron2 with YOLOv8, expanding the dataset, and adopting low-latency kernels. This research contributes to collaborative robotics by demonstrating the feasibility of integrating perception and behavior-based control, laying the foundation for future investigations and applications. Collaborative Robotics Behavior-Based Control Perception Systems ROS2 Human-Robot Interaction Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5664100","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":391957302,"identity":"5dc394d9-f828-4455-b9f1-93bdceefacae","order_by":0,"name":"Anonione da Silva Mascarenhas Pinho","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAzElEQVRIiWNgGAWjYNACAyBmbwAxLIhRzgzVwnMAxJAgVgsISCSAScIa+Gf3H3xcUGCTxy/5/OqGHwUSDPzt3Ql4tUjcOcxsPMMgrVhydk7ZzR6gwyTOnN2A35obyWzSPAaHEzfczkm7wQPUYiCRi1+LPEzL/ptn0m7+IUaLAdwWCfZjt4myxfBGsrExj0Fa4owzOWy3ZQwkeAj6Re5G4sPHPH9sEvvbjz+7+eaPjRx/ey8B7yMAjwGYJFY5CLA/IEX1KBgFo2AUjCAAAMSuRAWDIjkyAAAAAElFTkSuQmCC","orcid":"","institution":"Federal Institute for Education, Science and Technology of Espírito Santo (IFES)","correspondingAuthor":true,"prefix":"","firstName":"Anonione","middleName":"da Silva Mascarenhas","lastName":"Pinho","suffix":""},{"id":391957303,"identity":"e786cc11-fa5c-430a-a0cb-a4dd79aff0ed","order_by":1,"name":"Cassius Zanetti Resende","email":"","orcid":"","institution":"Federal Institute for Education, Science and Technology of Espírito Santo (IFES)","correspondingAuthor":false,"prefix":"","firstName":"Cassius","middleName":"Zanetti","lastName":"Resende","suffix":""},{"id":391957304,"identity":"7a8e2c26-0752-4bcd-a274-9c641ba400e6","order_by":2,"name":"Daniel Cruz Cavalieri","email":"","orcid":"","institution":"Federal Institute for Education, Science and Technology of Espírito Santo (IFES)","correspondingAuthor":false,"prefix":"","firstName":"Daniel","middleName":"Cruz","lastName":"Cavalieri","suffix":""}],"badges":[],"createdAt":"2024-12-17 18:23:13","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5664100/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5664100/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":72928351,"identity":"20c5875e-5352-4e34-833e-dd69b8dbc062","added_by":"auto","created_at":"2025-01-03 21:01:24","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1747901,"visible":true,"origin":"","legend":"","description":"","filename":"ArtigoAntonioneen.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5664100/v1_covered_518494bd-27bc-404c-94bf-b1d8a066220b.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Development of Perception and Control Applications for Collaborative Robots Using ROS2","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Collaborative Robotics, Behavior-Based Control, Perception Systems, ROS2, Human-Robot Interaction","lastPublishedDoi":"10.21203/rs.3.rs-5664100/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5664100/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis work explores the integration of perception systems in behavior-based control applications using ROS2, with a focus on safe and effective interactions between humans and collaborative robots. Unlike conventional industrial robots, collaborative robots are designed to interact directly with human operators without physical barriers, making automation more accessible, particularly for small and medium-sized enterprises. A system integrating the UR3e robot and the Intel RealSense D435i camera was developed, enabling the robot to recognize objects and interpret gestures. YOLOv8 was used for object detection and segmentation, MediaPipe for gesture recognition and face detection, and Detectron2 for human detection, empowering the robot to manipulate objects and respond to gesture commands. The system's efficiency was validated through practical tests by moving objects between known points. The study highlights opportunities for improvement, such as replacing Detectron2 with YOLOv8, expanding the dataset, and adopting low-latency kernels. This research contributes to collaborative robotics by demonstrating the feasibility of integrating perception and behavior-based control, laying the foundation for future investigations and applications.\u003c/p\u003e","manuscriptTitle":"Development of Perception and Control Applications for Collaborative Robots Using ROS2","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-21 00:35:35","doi":"10.21203/rs.3.rs-5664100/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"db65e74c-42dc-4f9d-9e70-baf0cdfa2b69","owner":[],"postedDate":"December 21st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-01-03T20:53:13+00:00","versionOfRecord":[],"versionCreatedAt":"2024-12-21 00:35:35","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5664100","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5664100","identity":"rs-5664100","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}