Weakly Supervised Semantic Segmentation of Remote Sensing Images Based on SD-CAM with Improved Mumford-Shah Loss

Weakly Supervised Semantic Segmentation of Remote Sensing Images Based on SD-CAM with Improved Mumford-Shah Loss | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Weakly Supervised Semantic Segmentation of Remote Sensing Images Based on SD-CAM with Improved Mumford-Shah Loss Jiaming Fan, Dali Chen, Jun Fu This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6207713/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract Weakly supervised semantic segmentation (WSSS) based on image-level annotation has received wide attention in the field of multi-class remote sensing images(RSIs). However, there is a lack of background information and there are a wide variety of targets in RSIs, which makes WSSS of RSIs suffer from incomplete information on small targets in class activation maps (CAMs). The WSSS of RSIs based on SD-CAM with improved mumford-shah loss (IMS-Loss) is proposed to solve the above challenge. The multi-stage WSSS method is adopted to improve the segmentation accuracy in novel method. Firstly, the self-attention mechanism (SAM) module is introduced into different stages of classification network to dig into the detail information of the RSI in the generation CAMs stage. Secondly, the feature maps are processed by the SAM module and are fed to the multi-dilated rate convolution (MDRC) module to obtain high quality CAMs, which enlarges the area of the localization target of the CAMs to improve the pseudo-labeling accuracy. Then, the improved Mumford-Shah Loss (IMS-loss) function is introduced to provide the segmentation network with supervised information of the original images to produce high quality segmentation results in the semantic segmentation network. Finally, simulation results show that the improved method is capable of obtaining high-precision segmentation results using image-level annotation information. The mIOU of the novel method on Vaihingen, Potsdam and WHDLD datasets reach 85.58%, 86.39% and 70.03% of the semantic segmentation methods for fully supervised semantic segmentation (FSSS) of RSIs. Self-attention mechanism multi-dilated rate convolution convolution mumford-shah Loss class activation map (CAM) remote sensing images (RSI) weakly supervised semantic segmentation (WSSS) Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 22 Mar, 2025 Reviewers invited by journal 22 Mar, 2025 Editor assigned by journal 12 Mar, 2025 Submission checks completed at journal 12 Mar, 2025 First submitted to journal 11 Mar, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6207713","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":428073512,"identity":"1a2bcb49-c2c0-41b5-9092-2cf84bdd747b","order_by":0,"name":"Jiaming Fan","email":"","orcid":"","institution":"Northeastern University","correspondingAuthor":false,"prefix":"","firstName":"Jiaming","middleName":"","lastName":"Fan","suffix":""},{"id":428073516,"identity":"3d2eaf25-86b9-4832-8ad7-a884c8bf0a35","order_by":1,"name":"Dali Chen","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7klEQVRIiWNgGAWjYDACCSDmYbDg4WdGiBkQo0WCR7KZVC0MBgeI1cI/u/nYgzc1EjLGx9kfPmDcYZPYwN68TYKh5g5uS+4cSzecc0yCx+wwj7EB45m0xAaeY2USDMee4dRiIJFjJs3DBtbCJsHYdjixASgiwdhwGI+W/G/SPP8keIyb2Z//YGz7n9gg/4aQlhw2ad42CR4DZgYzBsa2A0BbePBrkbiRZiY5t0+CRwLoF4nEtmTjNp60YouEY7i18M9Ifibx5puNPX//8YcfPrbZyfazH95440MNbi2oIAGI2WCMUTAKRsEoGAXkAwDxVEitqrFe7QAAAABJRU5ErkJggg==","orcid":"","institution":"Northeastern University","correspondingAuthor":true,"prefix":"","firstName":"Dali","middleName":"","lastName":"Chen","suffix":""},{"id":428073519,"identity":"807ab858-9403-4cf1-8911-c3fab83e27fc","order_by":2,"name":"Jun Fu","email":"","orcid":"","institution":"Northeastern University","correspondingAuthor":false,"prefix":"","firstName":"Jun","middleName":"","lastName":"Fu","suffix":""}],"badges":[],"createdAt":"2025-03-12 02:08:02","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6207713/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6207713/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":78635584,"identity":"df9ecc2b-54fe-4aee-9096-86d04617a38d","added_by":"auto","created_at":"2025-03-17 05:06:52","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":850499,"visible":true,"origin":"","legend":"","description":"","filename":"letex.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6207713/v1_covered_5d5a981b-264e-4a23-9aca-4d16628a94fc.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Weakly Supervised Semantic Segmentation of Remote Sensing Images Based on SD-CAM with Improved Mumford-Shah Loss","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"signal-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"sivp","sideBox":"Learn more about [Signal, Image and Video Processing](http://link.springer.com/journal/11760)","snPcode":"11760","submissionUrl":"https://submission.nature.com/new-submission/11760/3","title":"Signal, Image and Video Processing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Self-attention mechanism, multi-dilated rate convolution convolution, mumford-shah Loss, class activation map (CAM), remote sensing images (RSI), weakly supervised semantic segmentation (WSSS)","lastPublishedDoi":"10.21203/rs.3.rs-6207713/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6207713/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Weakly supervised semantic segmentation (WSSS) based on image-level annotation has received wide attention in the field of multi-class remote sensing images(RSIs). However, there is a lack of background information and there are a wide variety of targets in RSIs, which makes WSSS of RSIs suffer from incomplete information on small targets in class activation maps (CAMs). The WSSS of RSIs based on SD-CAM with improved mumford-shah loss (IMS-Loss) is proposed to solve the above challenge. The multi-stage WSSS method is adopted to improve the segmentation accuracy in novel method. Firstly, the self-attention mechanism (SAM) module is introduced into different stages of classification network to dig into the detail information of the RSI in the generation CAMs stage. Secondly, the feature maps are processed by the SAM module and are fed to the multi-dilated rate convolution (MDRC) module to obtain high quality CAMs, which enlarges the area of the localization target of the CAMs to improve the pseudo-labeling accuracy. Then, the improved Mumford-Shah Loss (IMS-loss) function is introduced to provide the segmentation network with supervised information of the original images to produce high quality segmentation results in the semantic segmentation network. Finally, simulation results show that the improved method is capable of obtaining high-precision segmentation results using image-level annotation information. The mIOU of the novel method on Vaihingen, Potsdam and WHDLD datasets reach 85.58%, 86.39% and 70.03% of the semantic segmentation methods for fully supervised semantic segmentation (FSSS) of RSIs.","manuscriptTitle":"Weakly Supervised Semantic Segmentation of Remote Sensing Images Based on SD-CAM with Improved Mumford-Shah Loss","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-03-17 04:58:46","doi":"10.21203/rs.3.rs-6207713/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-03-22T22:24:19+00:00","index":"","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-03-22T22:23:42+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-03-13T01:34:05+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-03-13T01:32:10+00:00","index":"","fulltext":""},{"type":"submitted","content":"Signal, Image and Video Processing","date":"2025-03-12T01:52:03+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"signal-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"sivp","sideBox":"Learn more about [Signal, Image and Video Processing](http://link.springer.com/journal/11760)","snPcode":"11760","submissionUrl":"https://submission.nature.com/new-submission/11760/3","title":"Signal, Image and Video Processing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"a13df960-be37-4eec-b943-465ba7bffb19","owner":[],"postedDate":"March 17th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-04-13T01:23:06+00:00","versionOfRecord":[],"versionCreatedAt":"2025-03-17 04:58:46","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6207713","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6207713","identity":"rs-6207713","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}