Modality Discrepancy Reduction for Visible-Infrared Person Re-Identification under Complete Modality Missing

Modality Discrepancy Reduction for Visible-Infrared Person Re-Identification under Complete Modality Missing | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Modality Discrepancy Reduction for Visible-Infrared Person Re-Identification under Complete Modality Missing Ning Zhang, Wenhang Ge, Hong Tang, Juan Wu This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4142107/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Person re-identification (Re-ID) aims to match person images across non-overlapping camera views. Most of existing methods focus on short-time and small-scale surveillance systems in which each person is captured in multiple camera views of adjacent scenes in a short time span. However, in long-term and large-scale surveillance systems covering larger areas and spanning different time duration, existing methods tend to fail. Specifically, when light condition is extremely dark, the camera automatically switches to infrared mode. Matching across infrared images and RGB images is difficult due to the large appearance difference. Meanwhile, since most pedestrians appear in limited local areas and fixed time, it is difficult to collect training images of the same person under both infrared and RGB mode. Hence, a model is required to match person images cross modality under a complete modality missing condition. In this work, we study intra-modality supervised person re-identification under complete modality missing, which uses cross-modality unpaired data with intra-modality identity labels for training. It is challenging as cross-modality paired data plays an important role for learning modality-invariant representation in most existing Re-ID methods. To learn modality-invariant representation from cross-modality unpaired training data, we first introduce a strong baseline with a dual-head cross-entropy loss and a multi-modality negative loss, aiming to alleviate cross-modality contrast and enhance intra-modality contrast. Then, we propose a residual modality alleviation network and a shape-guided consistency learning loss to further alleviate cross-modality representation discrepancy. The experiments are conducted in the complete modality missing setting on SYSY-MM01 and RegDB datasets. The evaluation results demonstrate the superiority of our method. Person re-identification complete modality missing cross-modality matching cross-modality representation Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4142107","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":282501206,"identity":"4ae9b32a-672b-4370-be70-64639442bec9","order_by":0,"name":"Ning Zhang","email":"","orcid":"","institution":"Research Institute of China Telecom","correspondingAuthor":false,"prefix":"","firstName":"Ning","middleName":"","lastName":"Zhang","suffix":""},{"id":282501208,"identity":"37cf85a4-b319-4a3b-adaf-1b46a74be0ff","order_by":1,"name":"Wenhang Ge","email":"","orcid":"","institution":"Hong Kong University of Science and Technology, Guangzhou, China","correspondingAuthor":false,"prefix":"","firstName":"Wenhang","middleName":"","lastName":"Ge","suffix":""},{"id":282501210,"identity":"d326f0fb-11a1-489f-bfc6-ad3dff9d03d5","order_by":2,"name":"Hong Tang","email":"","orcid":"","institution":"Research Institute of China Telecom","correspondingAuthor":false,"prefix":"","firstName":"Hong","middleName":"","lastName":"Tang","suffix":""},{"id":282501212,"identity":"fd18d7dc-fbbf-4b7a-93a9-ecf489e8c6e8","order_by":3,"name":"Juan Wu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA5ElEQVRIiWNgGAWjYHCC9M8/KhgS2MDsAuK0pDEznAFrYWxgMCBOCxszYxtDAgPRWgxuJDx7XDjPLo9Puvn5gw8GNvL8YgcYP+NzIFBLuvHMbcnFbDLHDBtnGKQZzpydwCw9A48WsxsJCRK82w4ktkkkGDbzGBxOMLidwMbMQ1DLHJCW9I/Nfwz+E6UlTZq3AaQlx7CZweAAYS32Zx4kG844lgzSUjizxyAZ6JfEZml8WiTbcxIffKixS5w/I33Dhx8VdvL80skHP+PTwsDAk4AuAowf/ID9AAEFo2AUjIJRMOIBANwGTn2ZqyPRAAAAAElFTkSuQmCC","orcid":"","institution":"Research Institute of China Telecom","correspondingAuthor":true,"prefix":"","firstName":"Juan","middleName":"","lastName":"Wu","suffix":""}],"badges":[],"createdAt":"2024-03-21 08:59:37","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4142107/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4142107/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":53845680,"identity":"958995c3-804a-425e-a0e9-17f897c3ebd1","added_by":"auto","created_at":"2024-04-01 08:32:56","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":284972,"visible":true,"origin":"","legend":"","description":"","filename":"submission.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4142107/v1_covered_b1390f86-395b-44fe-933f-741dd7a12448.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Modality Discrepancy Reduction for Visible-Infrared Person Re-Identification under Complete Modality Missing","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Person re-identification, complete modality missing, cross-modality matching, cross-modality representation","lastPublishedDoi":"10.21203/rs.3.rs-4142107/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4142107/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Person re-identification (Re-ID) aims to match person images across non-overlapping camera views. Most of existing methods focus on short-time and small-scale surveillance systems in which each person is captured in multiple camera views of adjacent scenes in a short time span. However, in long-term and large-scale surveillance systems covering larger areas and spanning different time duration, existing methods tend to fail. Specifically, when light condition is extremely dark, the camera automatically switches to infrared mode. Matching across infrared images and RGB images is difficult due to the large appearance difference. Meanwhile, since most pedestrians appear in limited local areas and fixed time, it is difficult to collect training images of the same person under both infrared and RGB mode.\nHence, a model is required to match person images cross modality under a complete modality missing condition.\nIn this work, we study intra-modality supervised person re-identification under complete modality missing, which uses cross-modality unpaired data with intra-modality identity labels for training. \nIt is challenging as cross-modality paired data plays an important role for learning modality-invariant representation in most existing Re-ID methods. \nTo learn modality-invariant representation from cross-modality unpaired training data, we first introduce a strong baseline with a dual-head cross-entropy loss and a multi-modality negative loss, aiming to alleviate cross-modality contrast and enhance intra-modality contrast. Then, we propose a residual modality alleviation network and a shape-guided consistency learning loss to further alleviate cross-modality representation discrepancy. The experiments are conducted in the complete modality missing setting on SYSY-MM01 and RegDB datasets. The evaluation results demonstrate the superiority of our method.","manuscriptTitle":"Modality Discrepancy Reduction for Visible-Infrared Person Re-Identification under Complete Modality Missing","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-26 02:12:08","doi":"10.21203/rs.3.rs-4142107/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"7a294738-d888-4436-9826-e5b446193065","owner":[],"postedDate":"March 26th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-04-01T08:24:49+00:00","versionOfRecord":[],"versionCreatedAt":"2024-03-26 02:12:08","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4142107","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4142107","identity":"rs-4142107","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}