Emergent Vision Technology: 3D Human Pose Estimation for Single-Pixel Imaging (SPI)

Emergent Vision Technology: 3D Human Pose Estimation for Single-Pixel Imaging (SPI) | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Emergent Vision Technology: 3D Human Pose Estimation for Single-Pixel Imaging (SPI) Carlos Osorio Quero, Jose Martinez-Carranza This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4837829/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Applying 3D human pose and body shape details from a single monocular image presents a significant challenge in computer vision. Traditional methods that rely on RGB images often face constraints due to varying lighting conditions and occlusions. However, advancements in imaging technologies have introduced new techniques, such as single-pixel imaging (SPI), which can overcome these limitations. SPI is particularly effective in capturing 3D human pose in the Near-Infrared (NIR) spectrum. This wavelength can penetrate clothing and is less affected by lighting variations than visible light, providing a reliable means to accurately capture body shape and pose data, even in challenging environments. In this work, we explore using an SPI camera operating in the NIR range, with Time-of-Flight (TOF) technology at wavelengths of 850-1550 nm, to detect humans in night-time environments. Our proposed system employs SPI for depth estimation and feature extraction in humans. These features generate point clouds integrated into a 3D body model (SMPLX) via 3D body shape regression. This process utilizes deep learning techniques based on self-supervised 3D human mesh methodologies. We constructed a laboratory scenario simulating night-time conditions to evaluate the efficacy of NIR-SPI 3D image reconstruction. This setup allowed us to test the feasibility of using NIR-SPI as a vision sensor in outdoor environments. By assessing the results obtained from this setup, we aim to demonstrate the potential of NIR-SPI as an effective tool for detecting humans in night-time scenarios and accurately capturing their 3D body pose and shape, with future applications in environmental rescue. Single-pixel imaging (SPI) Deep Learning depth perception Human 34 Pose Estimate (HPE) 3D body model point clouds Near-Infrared (NIR). Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4837829","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":338096948,"identity":"0afd7ba4-6f68-44a0-a62b-c0890f5df3f5","order_by":0,"name":"Carlos Osorio Quero","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAArUlEQVRIiWNgGAWjYLCCDwak6mCcQbIWZh6SlPO3Hz5426bgjjw/e+8BppttRGiROJOWbJ1j8MxwZs+5BOacM0RoMWDIMZPOMTjMuOFGjgFzTgUxWvjfmElbGBy233D/DVALMeFgIAG0hcHgcOKGGzxE2iJx41myZY/B4eSZPXkJh4nyC39/8sEbP/4ctu1nP3vwcS4xIQa2CULxMBwgUgOSllEwCkbBKBgFWAEAH/Qz4C7jYjIAAAAASUVORK5CYII=","orcid":"","institution":"Instituto Nacional de Astrofisica Optica y\nElectronica","correspondingAuthor":true,"prefix":"","firstName":"Carlos","middleName":"Osorio","lastName":"Quero","suffix":""},{"id":338096949,"identity":"17c698ba-635f-4509-ae1d-354169846c69","order_by":1,"name":"Jose Martinez-Carranza","email":"","orcid":"","institution":"Instituto Nacional de Astrofisica Optica y\nElectronica","correspondingAuthor":false,"prefix":"","firstName":"Jose","middleName":"","lastName":"Martinez-Carranza","suffix":""}],"badges":[],"createdAt":"2024-07-31 20:03:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4837829/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4837829/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":64002394,"identity":"1b0232c4-e0c1-4da8-97dd-56ae09f8efaf","added_by":"auto","created_at":"2024-09-04 20:22:28","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1078095,"visible":true,"origin":"","legend":"","description":"","filename":"NIRHPEP.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4837829/v1_covered_815b7db7-a71c-419d-b4b4-b28a9139bac8.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Emergent Vision Technology: 3D Human Pose Estimation for Single-Pixel Imaging (SPI)","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Single-pixel imaging (SPI), Deep Learning, depth perception, Human 34 Pose Estimate (HPE), 3D body model, point clouds, Near-Infrared (NIR).","lastPublishedDoi":"10.21203/rs.3.rs-4837829/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4837829/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Applying 3D human pose and body shape details from a single monocular image presents a significant challenge in computer vision. Traditional methods that rely on RGB images often face constraints due to varying lighting conditions and occlusions. However, advancements in imaging technologies have introduced new techniques, such as single-pixel imaging (SPI), which can overcome these limitations. SPI is particularly effective in capturing 3D human pose in the Near-Infrared (NIR) spectrum. This wavelength can penetrate clothing and is less affected by lighting variations than visible light, providing a reliable means to accurately capture body shape and pose data, even in challenging environments. In this work, we explore using an SPI camera operating in the NIR range, with Time-of-Flight (TOF) technology at wavelengths of 850-1550 nm, to detect humans in night-time environments. Our proposed system employs SPI for depth estimation and feature extraction in humans. These features generate point clouds integrated into a 3D body model (SMPLX) via 3D body shape regression. This process utilizes deep learning techniques based on self-supervised 3D human mesh methodologies. We constructed a laboratory scenario simulating night-time conditions to evaluate the efficacy of NIR-SPI 3D image reconstruction. This setup allowed us to test the feasibility of using NIR-SPI as a vision sensor in outdoor environments. By assessing the results obtained from this setup, we aim to demonstrate the potential of NIR-SPI as an effective tool for detecting humans in night-time scenarios and accurately capturing their 3D body pose and shape, with future applications in environmental rescue.","manuscriptTitle":"Emergent Vision Technology: 3D Human Pose Estimation for Single-Pixel Imaging (SPI)","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-08-28 06:28:44","doi":"10.21203/rs.3.rs-4837829/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"f9042160-12de-456b-9133-2c586b133931","owner":[],"postedDate":"August 28th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-09-04T20:14:21+00:00","versionOfRecord":[],"versionCreatedAt":"2024-08-28 06:28:44","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4837829","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4837829","identity":"rs-4837829","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}