Cost-Effective 18 DOF Dexterous Hand: Fusing Force-Position Multimodal Sensing of Whole Hand with a Monocular Camera

Cost-Effective 18 DOF Dexterous Hand: Fusing Force-Position Multimodal Sensing of Whole Hand with a Monocular Camera | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Cost-Effective 18 DOF Dexterous Hand: Fusing Force-Position Multimodal Sensing of Whole Hand with a Monocular Camera Cheng wei, Shiwei Chen, Jiapeng Li, Zhiming Deng, Peiji Wang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5410108/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 23 Jul, 2025 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Abstract The anthropomorphic hand plays a crucial role in human-machine interaction tasks. However, there are very few hands that realize multimodal perception with high DOF in a low-cost way. We present a dexterous hand that achieves multimodal sensing solely through a camera. The hand has 18 DOF but does not require any position or force sensors, making it low-cost and easy to manufacture. We developed an integrated base for the hand that provides both actuation and multimodal sensing information simultaneously. This includes the angles of 18 joints, 5 fingertip positions and contact forces, as well as information on object softness and texture. The core principle of sensing is that the camera can track the displacement and tension of all tendons simultaneously. By characterizing tendon properties and coupling them with the hand dynamics, we developed a multimodal sensing model. Experiments indicate that our hand has potential in multimodal sensing and dexterity. Physical sciences/Engineering/Mechanical engineering Physical sciences/Engineering/Electrical and electronic engineering Full Text Additional Declarations Yes there is potential Competing Interest. Shiwei Chen, Peiji Wang, and Cheng Wei are the inventors of two patent applications, submitted to Harbin Institute of Technology. The patents involve the principles of visual perception measurement and the dexterous hand design. Supplementary Files SupplementaryMaterials.pdf Supplementary Materials movie.zip Video files of supplementary materials Cite Share Download PDF Status: Published Journal Publication published 23 Jul, 2025 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5410108","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":387387589,"identity":"d0d3d8bf-30e4-4275-afa4-26044946aef5","order_by":0,"name":"Cheng wei","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA2klEQVRIie3OPQuCQBjAcQ+hlgtXQ/oOQtAi6FfxcHCJaHRweETQsfW+R9CcHNhy7UJCtTQH7tId1HpcW9D94V6G5wePZZlMPxkCWz6Oi0A89hdkTlEBmuQ95ncW0iP+qSyGbdaH+0tZ36wsIDA9H9WEN6VH+SM59I1YjKcE8CZWklVHKntWsUR8CkAVI+BiX02ud0FGliypJKMO6ZAgwELflQQ0SMRJ6eGWxa5YjMZtuqzwWk3m9ek+4JxFDk1vz2ceLHZTriafCMg7FmeiNS8X1B00mUymP+wFSmVLZoYlAG8AAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0002-7025-3279","institution":"Harbin Institute of Technology","correspondingAuthor":true,"prefix":"","firstName":"Cheng","middleName":"","lastName":"wei","suffix":""},{"id":387387590,"identity":"10d5a105-659a-4382-935c-bb6f9185b02a","order_by":1,"name":"Shiwei Chen","email":"","orcid":"","institution":"Harbin Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Shiwei","middleName":"","lastName":"Chen","suffix":""},{"id":387387591,"identity":"a90d57b8-59fe-459d-8f8c-13f8045eead0","order_by":2,"name":"Jiapeng Li","email":"","orcid":"","institution":"Harbin Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Jiapeng","middleName":"","lastName":"Li","suffix":""},{"id":387387592,"identity":"1d852b8b-215d-4ea0-b505-f548c4b11b7c","order_by":3,"name":"Zhiming Deng","email":"","orcid":"","institution":"Harbin Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Zhiming","middleName":"","lastName":"Deng","suffix":""},{"id":387387593,"identity":"15e5e408-9b67-4425-b34f-a18d122d0bc6","order_by":4,"name":"Peiji Wang","email":"","orcid":"","institution":"Harbin Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Peiji","middleName":"","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2024-11-07 13:06:12","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5410108/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5410108/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41467-025-62122-0","type":"published","date":"2025-07-23T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":87465203,"identity":"5cfecbe5-9760-4d5e-9124-4cd4227a05ae","added_by":"auto","created_at":"2025-07-24 07:09:23","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2236629,"visible":true,"origin":"","legend":"","description":"","filename":"Article.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5410108/v1_covered_8394e711-32d1-4867-8040-fdf6e146807a.pdf"},{"id":70991767,"identity":"597edf8e-7186-4168-a715-e5f333b60c75","added_by":"auto","created_at":"2024-12-10 03:27:32","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":133778,"visible":true,"origin":"","legend":"Supplementary Materials","description":"","filename":"SupplementaryMaterials.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5410108/v1/46ab20416765883cf06a0609.pdf"},{"id":70991770,"identity":"b8d8ec4a-3800-46a7-9559-3b982824d5a5","added_by":"auto","created_at":"2024-12-10 03:27:33","extension":"zip","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":49481015,"visible":true,"origin":"","legend":"Video files of supplementary materials","description":"","filename":"movie.zip","url":"https://assets-eu.researchsquare.com/files/rs-5410108/v1/e93fbb97505a807748b0d9d1.zip"}],"financialInterests":"\u003cb\u003eYes\u003c/b\u003e there is potential Competing Interest.\nShiwei Chen, Peiji Wang, and Cheng Wei are the inventors of two patent applications, submitted to Harbin Institute of Technology. The patents involve the principles of visual perception measurement and the dexterous hand design.","formattedTitle":"Cost-Effective 18 DOF Dexterous Hand: Fusing Force-Position Multimodal Sensing of Whole Hand with a Monocular Camera","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-5410108/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5410108/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The anthropomorphic hand plays a crucial role in human-machine interaction tasks. However, there are very few hands that realize multimodal perception with high DOF in a low-cost way. We present a dexterous hand that achieves multimodal sensing solely through a camera. The hand has 18 DOF but does not require any position or force sensors, making it low-cost and easy to manufacture. We developed an integrated base for the hand that provides both actuation and multimodal sensing information simultaneously. This includes the angles of 18 joints, 5 fingertip positions and contact forces, as well as information on object softness and texture. The core principle of sensing is that the camera can track the displacement and tension of all tendons simultaneously. By characterizing tendon properties and coupling them with the hand dynamics, we developed a multimodal sensing model. Experiments indicate that our hand has potential in multimodal sensing and dexterity.","manuscriptTitle":"Cost-Effective 18 DOF Dexterous Hand: Fusing Force-Position Multimodal Sensing of Whole Hand with a Monocular Camera","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-10 03:27:27","doi":"10.21203/rs.3.rs-5410108/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"92b13db9-f7dc-4ff0-88f4-33d115553b9f","owner":[],"postedDate":"December 10th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":41286081,"name":"Physical sciences/Engineering/Mechanical engineering"},{"id":41286082,"name":"Physical sciences/Engineering/Electrical and electronic engineering"}],"tags":[],"updatedAt":"2025-07-24T07:09:15+00:00","versionOfRecord":{"articleIdentity":"rs-5410108","link":"https://doi.org/10.1038/s41467-025-62122-0","journal":{"identity":"nature-communications","isVorOnly":false,"title":"Nature Communications"},"publishedOn":"2025-07-23 04:00:00","publishedOnDateReadable":"July 23rd, 2025"},"versionCreatedAt":"2024-12-10 03:27:27","video":"","vorDoi":"10.1038/s41467-025-62122-0","vorDoiUrl":"https://doi.org/10.1038/s41467-025-62122-0","workflowStages":[]},"version":"v1","identity":"rs-5410108","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5410108","identity":"rs-5410108","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}