A meta-upscale based end-to-end depth framework for infrared and visible image fusion

A meta-upscale based end-to-end depth framework for infrared and visible image fusion | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A meta-upscale based end-to-end depth framework for infrared and visible image fusion Wemrui Niu, Mingwei Tang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1742199/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract In the recent years, infrared and visible image fusion has been actively explored due to its advantages in various vision-based applications. However,the existing fusion methods only use the output of the last layer in the coded network. The results of fusion are not good as most of the important information in the intermediate layers is lost. In this paper, a meta upscale based end-to-end depth framework for infrared and visible image fusion is proposed. The meta-upscale module can increase the resolution of the multi-scale features without repeatedly training the whole model. Firstly, in the proposed Multi-scale Feature Extraction Module (MFEM), the multi-scale deep features of each source image are extracted and up-scaled by a meta upscale module. Secondly, in the Multi-scale Feature fusion Module (MFM), the L1-Norm strategy-based feature fusion method is developed to fuse the features based on the scaling. Thirdly, the Multi-scale Feature Compensation Module (MFCM), it has dense skip connections, which can preserve signifificant amounts of information from the input data in a multi-scale perspective. In addition, a new content retention loss is proposed to further improve the contrast enhancement approach. Experiments demonstrate that the proposed method can achieve signifificant results compared tothe state-of-the-art algorithms. Image Fusion Convolutional Neural Network Infrared and Visible Image Multi-scale Feature Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1742199","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":113367725,"identity":"e6098838-50fd-4760-870a-bc5a6eb4782d","order_by":0,"name":"Wemrui Niu","email":"","orcid":"","institution":"Xihua University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Wemrui","middleName":"","lastName":"Niu","suffix":""},{"id":113367726,"identity":"939455b7-2a30-4918-bbd6-e2e7196ba912","order_by":1,"name":"Mingwei Tang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAkklEQVRIiWNgGAWjYLCCDxU2JKjmAWLGGWfSSNTCzNt2iAQt9uxnD7+2YTvAwN/enUCkLTx5adY5PHcYJM6c3UCsw3LMjHMknjEYSOQSq4X/jZmxhcFhUrRI5Bg/ZkggScuNN2aMPQfSeIj3C3t/jvGHn/9s5Pjbe4nUAgRsEgyQ+CEeMH8gSfkoGAWjYBSMPAAA7fIoagDJ/2wAAAAASUVORK5CYII=","orcid":"","institution":"Xihua University","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Mingwei","middleName":"","lastName":"Tang","suffix":""}],"badges":[],"createdAt":"2022-06-09 13:59:25","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1742199/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1742199/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":22736716,"identity":"65b76919-ff0f-41af-ba58-d02bad012988","added_by":"auto","created_at":"2022-06-16 16:01:10","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":955941,"visible":true,"origin":"","legend":"","description":"","filename":"template.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1742199/v1_covered.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A meta-upscale based end-to-end depth framework for infrared and visible image fusion ","fulltext":[{"header":"Full Text","content":"This preprint is available for \u003ca href='/article/rs-1742199/latest.pdf' target='_blank'\u003edownload as a PDF\u003c/a\u003e."}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Image Fusion, Convolutional Neural Network, Infrared and Visible Image, Multi-scale Feature","lastPublishedDoi":"10.21203/rs.3.rs-1742199/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1742199/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"In the recent years, infrared and visible image fusion has been actively explored due to its advantages in various vision-based applications. However,the existing fusion methods only use the output of the last layer in the coded network. The results of fusion are not good as most of the important information in the intermediate layers is lost. In this paper, a meta upscale based end-to-end depth framework for infrared and visible image fusion is proposed. The meta-upscale module can increase the resolution of the multi-scale features without repeatedly training the whole model. Firstly, in the proposed Multi-scale Feature Extraction Module (MFEM), the multi-scale deep features of each source image are extracted and up-scaled by a meta upscale module. Secondly, in the Multi-scale Feature fusion Module (MFM), the L1-Norm strategy-based feature fusion method is developed to fuse the features based on the scaling. Thirdly, the Multi-scale Feature Compensation Module (MFCM), it has dense skip connections, which can preserve signifificant amounts of information from the input data in a multi-scale perspective. In addition, a new content retention loss is proposed to further improve the contrast enhancement approach. Experiments demonstrate that the proposed method can achieve signifificant results compared tothe state-of-the-art algorithms.","manuscriptTitle":"A meta-upscale based end-to-end depth framework for infrared and visible image fusion","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-06-16 16:01:05","doi":"10.21203/rs.3.rs-1742199/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"25c8b349-11be-4b83-a5c1-928db7f746e9","owner":[],"postedDate":"June 16th, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2022-08-16T22:59:09+00:00","versionOfRecord":[],"versionCreatedAt":"2022-06-16 16:01:05","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-1742199","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1742199","identity":"rs-1742199","version":["v1"]},"buildId":"WrCJVZZCHTDjtuVLN7oU0","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}