Single Image Highlight Removal via Innovative Pseudo Image Bases Fusion with a Dual-Network

Single Image Highlight Removal via Innovative Pseudo Image Bases Fusion with a Dual-Network | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Systematic Review Single Image Highlight Removal via Innovative Pseudo Image Bases Fusion with a Dual-Network Xufang PANG, Xiansheng CHEN, Chao YANG, Zhenliang ZHENG, Shengbo LIU, and 5 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5003957/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Specular highlights in images can damage or completely obliterate the color and texture details of objects, posing significant challenges to various visual tasks. Traditional highlight removal methods often struggle with handling specular highlights on surfaces with complex textures or rely on stringent and complicated shooting conditions. On the other hand, deep learning-based methods excel at highlight removal on single images with intricate surfaces due to their robust encoding capabilities. However, these methods still face issues with texture distortion in highlight regions. In this paper, we propose a novel method for highlight removal inspired by the observation that specular highlights typically result in increased brightness and decreased saturation. Our method relies on generating pseudo-SV (saturation-value) modulated image bases, effectively constructing a discrete color space that closely approximates the brightness, saturation, and hues of highlight-free pixels. We employ a dual-network architecture, jointly training a highlight detection sub-network and a highlight removal sub-network. By leveraging the generated image bases and highlight positional priors from the highlight detection network, our method enables the highlight removal network to learn the nuances of texture alterations across different highlight levels, thereby producing high-quality highlight-free images via a weighted fusion process. Our experiments demonstrate that our approach effectively restores texture and color details in highlighted regions, significantly outperforming existing methods, as evidenced by superior PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index) scores. To encourage future research and collaboration, we have made our source code publicly available at https://github.com/XufangPANG/Highlight-Removal-based-on-Pesudo-image-bases-fusion . Specular highlights highlight removal highlight detection Pseudo-SV modulated image bases Fusion Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5003957","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Systematic Review","associatedPublications":[],"authors":[{"id":356384193,"identity":"4d7fbffd-61fb-4962-a7e6-4a6fb159d4a2","order_by":0,"name":"Xufang PANG","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/0lEQVRIiWNgGAWjYFACxgdQBvMBBsaGBCAjgZAWZgMogy2BZC08BsRpMbiRzPi54NcdeXP+NR8/8+5IY+BnzzFg+LkDtxbJGcnM0jP7nhnunPF2szTvmRwGyZ43Boy9Z3Br4ZfIPyDN23OYccONs9uYedsqgPbmGDAztuHWwiaRzPwbqMV+w40zz8Ba7Alp4ZdIZpPm+XE4ccP5HjaglhwGAwkCWiR7HrNZ8zY8S95wg81Ycm5bGo/EmWcFB3vxaDE4nsx8m+fPHdsN5w8//PC2LVmOvz1544OfeLSAAWPbAQYGiQQwmwdEHCCgAQj+ANXwE6FuFIyCUTAKRiYAAGHlVRdaxeaWAAAAAElFTkSuQmCC","orcid":"","institution":"Shenzhen Institute of Artificial Intelligence and Robotics for Society","correspondingAuthor":true,"prefix":"","firstName":"Xufang","middleName":"","lastName":"PANG","suffix":""},{"id":356384194,"identity":"7abe8765-0d06-4071-98b2-38e30ba5a3bd","order_by":1,"name":"Xiansheng CHEN","email":"","orcid":"","institution":"Shenzhen University","correspondingAuthor":false,"prefix":"","firstName":"Xiansheng","middleName":"","lastName":"CHEN","suffix":""},{"id":356384195,"identity":"0152c0df-e4eb-41ff-a955-5a602627ebed","order_by":2,"name":"Chao YANG","email":"","orcid":"","institution":"Shenzhen Institute of Artificial Intelligence and Robotics for Society","correspondingAuthor":false,"prefix":"","firstName":"Chao","middleName":"","lastName":"YANG","suffix":""},{"id":356384196,"identity":"8861c86e-7eca-47f1-9360-e91fcd9ca885","order_by":3,"name":"Zhenliang ZHENG","email":"","orcid":"","institution":"Shenzhen Institute of Artificial Intelligence and Robotics for Society","correspondingAuthor":false,"prefix":"","firstName":"Zhenliang","middleName":"","lastName":"ZHENG","suffix":""},{"id":356384197,"identity":"448cde3e-b71a-408d-8f03-196e713ed6c5","order_by":4,"name":"Shengbo LIU","email":"","orcid":"","institution":"Shenzhen Institute of Artificial Intelligence and Robotics for Society","correspondingAuthor":false,"prefix":"","firstName":"Shengbo","middleName":"","lastName":"LIU","suffix":""},{"id":356384198,"identity":"45017414-74ee-49a0-b707-1a6a1691a5e8","order_by":5,"name":"Hongjun ZHOU","email":"","orcid":"","institution":"Shenzhen Institute of Artificial Intelligence and Robotics for Society","correspondingAuthor":false,"prefix":"","firstName":"Hongjun","middleName":"","lastName":"ZHOU","suffix":""},{"id":356384199,"identity":"e3d3973f-08a7-4010-946c-a58586a31cb7","order_by":6,"name":"Xuan XIA","email":"","orcid":"","institution":"Shenzhen Institute of Artificial Intelligence and Robotics for Society","correspondingAuthor":false,"prefix":"","firstName":"Xuan","middleName":"","lastName":"XIA","suffix":""},{"id":356384200,"identity":"ce66a55c-9e2c-450c-a18b-d38cf11852c6","order_by":7,"name":"Bo JIANG","email":"","orcid":"","institution":"Shenzhen Haixing Intelligent Driving Technology Co., Ltd.","correspondingAuthor":false,"prefix":"","firstName":"Bo","middleName":"","lastName":"JIANG","suffix":""},{"id":356384201,"identity":"0cf3e001-8421-4285-8330-27889d2afd58","order_by":8,"name":"Ning DING","email":"","orcid":"","institution":"Shenzhen Institute of Artificial Intelligence and Robotics for Society","correspondingAuthor":false,"prefix":"","firstName":"Ning","middleName":"","lastName":"DING","suffix":""},{"id":356384202,"identity":"48c1e8fa-af28-408f-adc3-935a7010b90f","order_by":9,"name":"Xiaoping ZHONG","email":"","orcid":"","institution":"Shenzhen University","correspondingAuthor":false,"prefix":"","firstName":"Xiaoping","middleName":"","lastName":"ZHONG","suffix":""}],"badges":[],"createdAt":"2024-08-30 12:36:02","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5003957/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5003957/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":69519505,"identity":"13bd4e96-4a20-44dc-a4f0-91668258610c","added_by":"auto","created_at":"2024-11-21 08:39:23","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1402421,"visible":true,"origin":"","legend":"","description":"","filename":"SingleImageHighlightRemovalviaInnovativePseudoImageBasesFusionwithaDualNetworkC.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5003957/v1_covered_7fe6a945-0693-4841-a0f6-bbfeea90c8dc.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Single Image Highlight Removal via Innovative Pseudo Image Bases Fusion with a Dual-Network","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Specular highlights, highlight removal, highlight detection, Pseudo-SV modulated image bases, Fusion","lastPublishedDoi":"10.21203/rs.3.rs-5003957/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5003957/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Specular highlights in images can damage or completely obliterate the color and texture details of objects, posing significant challenges to various visual tasks. Traditional highlight removal methods often struggle with handling specular highlights on surfaces with complex textures or rely on stringent and complicated shooting conditions. On the other hand, deep learning-based methods excel at highlight removal on single images with intricate surfaces due to their robust encoding capabilities. However, these methods still face issues with texture distortion in highlight regions. In this paper, we propose a novel method for highlight removal inspired by the observation that specular highlights typically result in increased brightness and decreased saturation. Our method relies on generating pseudo-SV (saturation-value) modulated image bases, effectively constructing a discrete color space that closely approximates the brightness, saturation, and hues of highlight-free pixels. We employ a dual-network architecture, jointly training a highlight detection sub-network and a highlight removal sub-network. By leveraging the generated image bases and highlight positional priors from the highlight detection network, our method enables the highlight removal network to learn the nuances of texture alterations across different highlight levels, thereby producing high-quality highlight-free images via a weighted fusion process. Our experiments demonstrate that our approach effectively restores texture and color details in highlighted regions, significantly outperforming existing methods, as evidenced by superior PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index) scores. To encourage future research and collaboration, we have made our source code publicly available at https://github.com/XufangPANG/Highlight-Removal-based-on-Pesudo-image-bases-fusion.","manuscriptTitle":"Single Image Highlight Removal via Innovative Pseudo Image Bases Fusion with a Dual-Network","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-10-01 05:39:20","doi":"10.21203/rs.3.rs-5003957/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"4667bad8-2652-4e87-a3a3-374f3fa8301f","owner":[],"postedDate":"October 1st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-11-21T08:39:03+00:00","versionOfRecord":[],"versionCreatedAt":"2024-10-01 05:39:20","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5003957","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5003957","identity":"rs-5003957","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}