Light-Perception-Based Interactive Control of an Underwater Digital Twin Hand

Light-Perception-Based Interactive Control of an Underwater Digital Twin Hand | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Light-Perception-Based Interactive Control of an Underwater Digital Twin Hand Yang Yu, Jinlong Lu, Chao Zhang, nong chang, Dongying Wang, Hongyu Zhou, and 7 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7096659/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract With increasing demand for ocean exploration, underwater tasks become more complex. Soft robotic hands leverage flexible structures and biomimetic properties to provide a strong alternative to rigid robotic hands. However, most existing studies on underwater soft robotic hands focus on novel materials and improved biomimetic structural designs. These systems often suffer from low sensing accuracy and poor human–machine interaction performance, making it difficult to achieve real-time and precise control in complex environments. Here, we propose a Light-Perception-Based Underwater Digital Twin Hand Interactive Control System (LP-UDTHS) that integrates a multi-degree-of-freedom (multi-DOF) biomimetic pneumatic soft robotic hand with a wearable data glove based on fiber Bragg grating (FBG) sensors. By fusing data from FBGs, RGB cameras, and inertial measurement units (IMUs), and optimizing control strategies through reinforcement learning, we established a light-dominated perception–control–feedback closed-loop system that significantly enhances the control performance of the system in complex and dynamic underwater environments. With light as the information carrier, the system integrates a data glove with high strain sensitivity (1.148 pm/µε, linearity = 0.9996) and high bending sensitivity (26.667 pm/°, linearity = 0.9791). It achieves four-dimensional collaborative interaction and control between the human operator, the machine, the virtual space, and the environment. The system achieves posture monitoring and precise grasping control across various underwater scenarios. It demonstrates advantages such as low cost, high precision, rapid response, strong interference resistance, and stable interaction. This establishes a new paradigm for underwater intelligent equipment by integrating light sensing with digital twin technology, offering broad prospects for future underwater operations. Physical sciences/Optics and photonics/Applied optics/Fibre optics and optical communications Physical sciences/Optics and photonics/Applied optics/Optical sensors Physical sciences/Optics and photonics/Applied optics/Optoelectronic devices and components Full Text Additional Declarations There is no conflict of interest Supplementary Files XXXXXXXXXXXX1.mp4 Supplementary Movie 1 XXXXXXXXXXXX2.mp4 Supplementary Movie 2 SupplementaryInformationforLightPerceptionBasedInteractiveControlofanUnderwaterDigitalTwinHand.docx Supplementary Information for Light-Perception-Based Interactive Control of an Underwater Digital Twin Hand Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7096659","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":501582214,"identity":"63f890f7-3258-4843-b457-0bfdd5d70909","order_by":0,"name":"Yang Yu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA60lEQVRIiWNgGAWjYPACCTkG9gYGBh4Q+wBxWmyMGXiASnkSiNeSltggkUCkFoMbOYafC34dNuaf+faYxNsfDHJ8NxIYPxfg12IsPbPvsJzE7bw0yTkJDMaSNxKYpWfg12Igzdtz2Jjhdo6ZNNBhiRtuJLAx8xCw5TdQS+L8m2fAWuqJ0QJU+SMNaDgPWEuCASEtkmeelVnzNtgYG57JMbackyZhOPPMw2ZpfFr4jidvvs3zR0JO7vgZwxtvbGzkgSIHP+PTonCAw4CBsQ3OlwBixgY8GhgY5BvYHzAw/MGrZhSMglEwCkY6AAAwr06ZegzjawAAAABJRU5ErkJggg==","orcid":"","institution":"College of Science, National University of Defense Technology","correspondingAuthor":true,"prefix":"","firstName":"Yang","middleName":"","lastName":"Yu","suffix":""},{"id":501582215,"identity":"c0907f9e-a01c-4f8a-a1d9-7eb0bb7f1afb","order_by":1,"name":"Jinlong Lu","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Jinlong","middleName":"","lastName":"Lu","suffix":""},{"id":501582216,"identity":"ce884ee0-c17a-4eba-986f-4f6dbc53e6cc","order_by":2,"name":"Chao Zhang","email":"","orcid":"","institution":"School of Automation, Central South University","correspondingAuthor":false,"prefix":"","firstName":"Chao","middleName":"","lastName":"Zhang","suffix":""},{"id":501582217,"identity":"dcd999ef-2c67-40c1-a750-8cb59a495959","order_by":3,"name":"nong chang","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"nong","middleName":"","lastName":"chang","suffix":""},{"id":501582218,"identity":"8fecfac4-57ca-4236-bed2-b1ba8c958666","order_by":4,"name":"Dongying Wang","email":"","orcid":"","institution":"College of Meteorology and Oceanography, National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Dongying","middleName":"","lastName":"Wang","suffix":""},{"id":501582219,"identity":"af2da762-7435-40eb-8d71-b2c88b4c82a6","order_by":5,"name":"Hongyu Zhou","email":"","orcid":"","institution":"College of Science","correspondingAuthor":false,"prefix":"","firstName":"Hongyu","middleName":"","lastName":"Zhou","suffix":""},{"id":501582220,"identity":"cfa86413-bb2e-4dba-8633-09410497d4b4","order_by":6,"name":"Junjie Weng","email":"","orcid":"","institution":"College of Intelligence Science and Technology, National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Junjie","middleName":"","lastName":"Weng","suffix":""},{"id":501582221,"identity":"bf7a81fd-ce07-4823-8483-947ec4f79add","order_by":7,"name":"Yuehua Deng","email":"","orcid":"","institution":"College of Science, National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Yuehua","middleName":"","lastName":"Deng","suffix":""},{"id":501582222,"identity":"ed24ac67-be85-4b9b-8057-330d7581c24d","order_by":8,"name":"Qiang Bian","email":"","orcid":"","institution":"College of Meteorology and Oceanography, National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Qiang","middleName":"","lastName":"Bian","suffix":""},{"id":501582223,"identity":"52518ba7-20d2-4ba2-b6ef-23c4ee98690c","order_by":9,"name":"Jianfa Zhang","email":"","orcid":"https://orcid.org/0000-0001-5879-5633","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Jianfa","middleName":"","lastName":"Zhang","suffix":""},{"id":501582224,"identity":"3298619d-9575-4345-808f-e95fac694807","order_by":10,"name":"Chaofan Zhang","email":"","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Chaofan","middleName":"","lastName":"Zhang","suffix":""},{"id":501582225,"identity":"69f500f5-4e79-4178-a257-a793b67ee258","order_by":11,"name":"Zhenrong Zhang","email":"","orcid":"","institution":"Guangxi University","correspondingAuthor":false,"prefix":"","firstName":"Zhenrong","middleName":"","lastName":"Zhang","suffix":""},{"id":501582226,"identity":"cb512c4f-c190-460c-b3d4-f0c0f0ead04b","order_by":12,"name":"Junbo Yang","email":"","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Junbo","middleName":"","lastName":"Yang","suffix":""}],"badges":[],"createdAt":"2025-07-11 01:21:16","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7096659/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7096659/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":91501393,"identity":"23beea3e-a306-4172-b357-f13620e7797b","added_by":"auto","created_at":"2025-09-17 07:28:43","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1525204,"visible":true,"origin":"","legend":"","description":"","filename":"LightPerceptionBasedInteractiveControlofanUnderwaterDigitalTwinHand.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7096659/v1_covered_bf917921-e7e0-4c0b-b518-74c2b81d6b6e.pdf"},{"id":89840428,"identity":"ab151762-eb1b-4cf5-9830-0a5f4ead64bd","added_by":"auto","created_at":"2025-08-25 15:16:15","extension":"mp4","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":52332335,"visible":true,"origin":"","legend":"Supplementary Movie 1","description":"","filename":"XXXXXXXXXXXX1.mp4","url":"https://assets-eu.researchsquare.com/files/rs-7096659/v1/06de1c4f837ce1df8e77aa39.mp4"},{"id":89840429,"identity":"99c931ba-7c6f-471f-b933-6a0762b9f1fa","added_by":"auto","created_at":"2025-08-25 15:16:15","extension":"mp4","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":92461532,"visible":true,"origin":"","legend":"Supplementary Movie 2","description":"","filename":"XXXXXXXXXXXX2.mp4","url":"https://assets-eu.researchsquare.com/files/rs-7096659/v1/1c2de2cad5c1d30a969f035d.mp4"},{"id":89840410,"identity":"ab224dad-17dd-44ec-8864-f78acc4ed5f6","added_by":"auto","created_at":"2025-08-25 15:16:14","extension":"docx","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":1326216,"visible":true,"origin":"","legend":"Supplementary Information for Light-Perception-Based Interactive Control of an Underwater Digital Twin Hand","description":"","filename":"SupplementaryInformationforLightPerceptionBasedInteractiveControlofanUnderwaterDigitalTwinHand.docx","url":"https://assets-eu.researchsquare.com/files/rs-7096659/v1/b62cf00c9ff88a76d9ae9094.docx"}],"financialInterests":"There is no conflict of interest","formattedTitle":"Light-Perception-Based Interactive Control of an Underwater Digital Twin Hand","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7096659/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7096659/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eWith increasing demand for ocean exploration, underwater tasks become more complex. Soft robotic hands leverage flexible structures and biomimetic properties to provide a strong alternative to rigid robotic hands. However, most existing studies on underwater soft robotic hands focus on novel materials and improved biomimetic structural designs. These systems often suffer from low sensing accuracy and poor human\u0026ndash;machine interaction performance, making it difficult to achieve real-time and precise control in complex environments. Here, we propose a Light-Perception-Based Underwater Digital Twin Hand Interactive Control System (LP-UDTHS) that integrates a multi-degree-of-freedom (multi-DOF) biomimetic pneumatic soft robotic hand with a wearable data glove based on fiber Bragg grating (FBG) sensors. By fusing data from FBGs, RGB cameras, and inertial measurement units (IMUs), and optimizing control strategies through reinforcement learning, we established a light-dominated perception\u0026ndash;control\u0026ndash;feedback closed-loop system that significantly enhances the control performance of the system in complex and dynamic underwater environments. With light as the information carrier, the system integrates a data glove with high strain sensitivity (1.148 pm/\u0026micro;ε, linearity\u0026thinsp;=\u0026thinsp;0.9996) and high bending sensitivity (26.667 pm/\u0026deg;, linearity\u0026thinsp;=\u0026thinsp;0.9791). It achieves four-dimensional collaborative interaction and control between the human operator, the machine, the virtual space, and the environment. The system achieves posture monitoring and precise grasping control across various underwater scenarios. It demonstrates advantages such as low cost, high precision, rapid response, strong interference resistance, and stable interaction. This establishes a new paradigm for underwater intelligent equipment by integrating light sensing with digital twin technology, offering broad prospects for future underwater operations.\u003c/p\u003e","manuscriptTitle":"Light-Perception-Based Interactive Control of an Underwater Digital Twin Hand","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-08-25 15:16:09","doi":"10.21203/rs.3.rs-7096659/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2c32bfcd-8111-49a0-b07f-8e0d97e2a9fd","owner":[],"postedDate":"August 25th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":53269591,"name":"Physical sciences/Optics and photonics/Applied optics/Fibre optics and optical communications"},{"id":53269592,"name":"Physical sciences/Optics and photonics/Applied optics/Optical sensors"},{"id":53269593,"name":"Physical sciences/Optics and photonics/Applied optics/Optoelectronic devices and components"}],"tags":[],"updatedAt":"2025-09-17T07:20:32+00:00","versionOfRecord":[],"versionCreatedAt":"2025-08-25 15:16:09","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7096659","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7096659","identity":"rs-7096659","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}