Infrared and visible image fusion based on modal feature enhancement with global attention and dynamic convolutional reconstruction

Infrared and visible image fusion based on modal feature enhancement with global attention and dynamic convolutional reconstruction | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Infrared and visible image fusion based on modal feature enhancement with global attention and dynamic convolutional reconstruction wang meng, Guo Xia, Liu Haipeng This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3752092/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 5 You are reading this latest preprint version Abstract Visible and infrared image fusion (VIF) for obtaining foreground salient information has strong application potential and made substantial progress based on deep neural networks. However, it remains difficult to resolve the feature degradation and spatial detail loss in the feed-forward process of the existing deep networks. In this paper, we propose an input modality-independent feature analysis-reconstruction fusion network to solve the above problems. In the feature extraction stage, a feed-forward feature enhancement module (DFEM) is embedded to explicitly enhance the infrared and visible modal salient features, respectively.Also, an attention template based on global correlation is constructed for converging different channel feature mappings to obtain a consistent fusion representation. Afterwards,dynamic convolution is used to adaptively construct a convolutional kernels in terms of the current input to generate the fused image.Additionally , a perceptual loss function is added into the encoder training to further preserve the semantic information in the fused features for reference-free image scenarios. Subjective and multiple objective evaluations Additionally,using the TNO and RoadScene datasets show that the proposed method outperforms existing fusion baseline models, with the greater average measurements of EN, MI, Q AB/F and SCD. Moreover, the fusion results maintain the visible background texture as well as the infrared salient target contrast better. Infrared and Visible Image Fusion Feature Extraction Feature Enhancement Dynamic Convolutional Reconstruction Full Text Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Major revision 10 Jun, 2025 Reviewers agreed at journal 03 Mar, 2024 Reviewers invited by journal 03 Mar, 2024 Editor assigned by journal 13 Dec, 2023 First submitted to journal 12 Dec, 2023 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3752092","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":276083948,"identity":"606e44c2-bce5-4471-a1a9-af533d9e4676","order_by":0,"name":"wang meng","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7UlEQVRIiWNgGAWjYDADfh4Y6wCxWiR7QGQCKVoMzhCrxeD42cOvblTcsdt85nSa5M8fDHJ8NxIYPxfgNTwvzTrnzLPkbWd7t0nzJDAYS95IYJaegUeL2YEcM+PctsPJZud5t0kDHZa44UYCGzMPPi3n3wC1/DucbNzPu03yRwJDPWEtN3KMH+c2HLYz4O3dJgF0WIIBIS32N96YMeccO5wgcebsZmueNAnDmWceNkvj0yLZn2P8OafmsD1/T+7Gmz9sbOT5jicf/IxPCxCwSQCJxAYIB8RmbMCvgYGB+QPIgYRUjYJRMApGwQgGACcwUFBJMachAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0009-0003-8083-5410","institution":"Kunming University of Science and Technology","correspondingAuthor":true,"prefix":"","firstName":"wang","middleName":"","lastName":"meng","suffix":""},{"id":276083949,"identity":"4b8fa908-f443-44b5-b41e-7f1695a88968","order_by":1,"name":"Guo Xia","email":"","orcid":"","institution":"Kunming University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Guo","middleName":"","lastName":"Xia","suffix":""},{"id":276083950,"identity":"7f4db898-fa44-4451-a2fc-ed208589bae6","order_by":2,"name":"Liu Haipeng","email":"","orcid":"","institution":"Kunming University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Liu","middleName":"","lastName":"Haipeng","suffix":""}],"badges":[],"createdAt":"2023-12-14 07:34:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3752092/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3752092/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":52108429,"identity":"be78816b-b2c3-444a-98b4-a21aa7565678","added_by":"auto","created_at":"2024-03-06 20:16:22","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1923330,"visible":true,"origin":"","legend":"","description":"","filename":"VIFManuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3752092/v1_covered_bdd453a7-2953-452b-9570-cd08f57477a2.pdf"}],"financialInterests":"","formattedTitle":"Infrared and visible image fusion based on modal feature enhancement with global attention and dynamic convolutional reconstruction","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"eurasip-journal-on-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jivp","sideBox":"Learn more about [EURASIP Journal on Image and Video Processing](http://jivp-eurasipjournals.springeropen.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/jivp/default.aspx","title":"EURASIP Journal on Image and Video Processing","twitterHandle":"@SpringerEng","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Infrared and Visible Image Fusion, Feature Extraction, Feature Enhancement, Dynamic Convolutional Reconstruction","lastPublishedDoi":"10.21203/rs.3.rs-3752092/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3752092/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eVisible and infrared image fusion (VIF) for obtaining foreground salient information has strong application potential and made substantial progress based on deep neural networks. However, it remains difficult to resolve the feature degradation and spatial detail loss in the feed-forward process of the existing deep networks. In this paper, we propose an input modality-independent feature analysis-reconstruction fusion network to solve the above problems. In the feature extraction stage, a feed-forward feature enhancement module (DFEM) is embedded to explicitly enhance the infrared and visible modal salient features, respectively.Also, an attention template based on global correlation is constructed for converging different channel feature mappings to obtain a consistent fusion representation. Afterwards,dynamic convolution is used to adaptively construct a convolutional kernels in terms of the current input to generate the fused image.Additionally , a perceptual loss function is added into the encoder training to further preserve the semantic information in the fused features for reference-free image scenarios. Subjective and multiple objective evaluations Additionally,using the TNO and RoadScene datasets show that the proposed method outperforms existing fusion baseline models, with the greater average measurements of EN, MI, Q\u003csup\u003e\u003csup\u003eAB/F\u003c/sup\u003e\u003c/sup\u003e and SCD. Moreover, the fusion results maintain the visible background texture as well as the infrared salient target contrast better.\u003c/p\u003e","manuscriptTitle":"Infrared and visible image fusion based on modal feature enhancement with global attention and dynamic convolutional reconstruction","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-06 20:08:07","doi":"10.21203/rs.3.rs-3752092/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Major revision","date":"2025-06-10T17:35:31+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"","date":"2024-03-04T02:42:51+00:00","index":0,"fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-03-03T19:50:00+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2023-12-14T02:19:29+00:00","index":"","fulltext":""},{"type":"submitted","content":"EURASIP Journal on Image and Video Processing","date":"2023-12-12T23:27:36+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"eurasip-journal-on-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jivp","sideBox":"Learn more about [EURASIP Journal on Image and Video Processing](http://jivp-eurasipjournals.springeropen.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/jivp/default.aspx","title":"EURASIP Journal on Image and Video Processing","twitterHandle":"@SpringerEng","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"4a4d12bc-e11f-435a-be59-63ed6f126d44","owner":[],"postedDate":"March 6th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"in-revision","subjectAreas":[],"tags":[],"updatedAt":"2025-06-10T21:36:46+00:00","versionOfRecord":[],"versionCreatedAt":"2024-03-06 20:08:07","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3752092","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3752092","identity":"rs-3752092","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}