DAMMGA-Net: Direction-Aware Attention with Genetic Algorithm-Optimized Loss for Retinal Vessel Segmentation

DAMMGA-Net: Direction-Aware Attention with Genetic Algorithm-Optimized Loss for Retinal Vessel Segmentation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article DAMMGA-Net: Direction-Aware Attention with Genetic Algorithm-Optimized Loss for Retinal Vessel Segmentation Yintao Hong, Yajun Xie, Yanyan Wu, Jinwei Wang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8934073/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract Retinal vessel segmentation is fundamental for early screening and quantitative analysis of fundus diseases, yet frequent fragmentation of fine vessels, pronounced scale variation, and severe foreground--background imbalance still limit structural continuity and detail preservation. To address these challenges, we propose DAMMGA-Net, a lightweight retinal vessel segmentation framework built on an HFEM encoder and decoder-side Refinement Blocks. The proposed framework improves segmentation through three complementary components. First, the Direction-Aware Mixed Attention Module (DAMM) enhances directional continuity modeling for thin, tortuous, and branching vessels. Second, the Multi-scale Fixed Receptive-field Parallel Pyramid (MFRPP) aggregates cross-receptive-field information at the bottleneck to improve representation consistency across vessel scales. Third, GA-Loss adaptively optimizes the weights and internal parameters of a bounded composite-loss family to alleviate optimization bias caused by class imbalance. Experiments on DRIVE, CHASE_DB1, and STARE show that DAMMGA-Net achieves stable and competitive performance while better preserving fine vessels, reaching AUC 0.9945, F1 0.8721, and Acc 0.9804 on STARE, and AUC 0.9879, F1 0.8413, and Acc 0.9732 on DRIVE. Qualitative results further show improved connectivity and integrity in fine branches, tortuous vessels, and low-contrast regions. These findings indicate that the synergy of directional continuity modeling, stable cross-scale aggregation, and dataset-adaptive loss optimization is effective for retinal vessel segmentation under severe class imbalance. Biological sciences/Computational biology and bioinformatics Physical sciences/Engineering Physical sciences/Mathematics and computing retinal vessel segmentation direction-aware mixed attention multi-scale fixed receptive-field pyramid genetic algorithm loss function search class imbalance Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviewers invited by journal 17 Apr, 2026 Editor assigned by journal 17 Apr, 2026 Editor invited by journal 07 Apr, 2026 Submission checks completed at journal 02 Apr, 2026 First submitted to journal 02 Apr, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8934073","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":627857127,"identity":"8a1da000-b687-4ef6-b93b-eee71bd27503","order_by":0,"name":"Yintao Hong","email":"","orcid":"","institution":"City University of Macau","correspondingAuthor":false,"prefix":"","firstName":"Yintao","middleName":"","lastName":"Hong","suffix":""},{"id":627857128,"identity":"3dc4f440-e04e-4bd5-ad2a-ec3f622a2ea9","order_by":1,"name":"Yajun Xie","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAuUlEQVRIiWNgGAWjYHCCxAcVDHIghgHRWpINzjAYk6aFTYI0LQY3Ep5VHPhjkNjA3rxNgqHmDmEtkjMS0m4c4AFq4TlWJsFw7BlhLfwSCWm3P0j8SWyQyDGTYGw4TFgLG1BLwQEDoC3yb4jUArKF4UACUIsED5FaJHseJEscOGBg3MaTVmyRcIwILQbHcxI/AENMtp/98MYbH2qI0MIgkJMAptlARAIRGoCeOX6AKHWjYBSMglEwggEAOYU6MCKy/2YAAAAASUVORK5CYII=","orcid":"","institution":"Fuzhou University of International Studies and Trade","correspondingAuthor":true,"prefix":"","firstName":"Yajun","middleName":"","lastName":"Xie","suffix":""},{"id":627857129,"identity":"2ca41ae9-5e0f-468e-93a8-27f2eee77d8f","order_by":2,"name":"Yanyan Wu","email":"","orcid":"","institution":"City University of Macau","correspondingAuthor":false,"prefix":"","firstName":"Yanyan","middleName":"","lastName":"Wu","suffix":""},{"id":627857136,"identity":"d8d2058f-ede0-4430-9391-dc801759e5eb","order_by":3,"name":"Jinwei Wang","email":"","orcid":"","institution":"City University of Macau","correspondingAuthor":false,"prefix":"","firstName":"Jinwei","middleName":"","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2026-02-21 14:23:27","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8934073/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8934073/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":107707886,"identity":"381579c6-8584-4097-83ec-43428fe52087","added_by":"auto","created_at":"2026-04-24 09:21:21","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":9816636,"visible":true,"origin":"","legend":"","description":"","filename":"paperlatex.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8934073/v1_covered_d3db772b-4f4d-4bf7-81b1-8b85000bfee5.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"\u003cp\u003eDAMMGA-Net: Direction-Aware Attention with Genetic Algorithm-Optimized Loss for Retinal Vessel Segmentation\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"retinal vessel segmentation, direction-aware mixed attention, multi-scale fixed receptive-field pyramid, genetic algorithm, loss function search, class imbalance","lastPublishedDoi":"10.21203/rs.3.rs-8934073/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8934073/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eRetinal vessel segmentation is fundamental for early screening and quantitative analysis of fundus diseases, yet frequent fragmentation of fine vessels, pronounced scale variation, and severe foreground--background imbalance still limit structural continuity and detail preservation. To address these challenges, we propose DAMMGA-Net, a lightweight retinal vessel segmentation framework built on an HFEM encoder and decoder-side Refinement Blocks. The proposed framework improves segmentation through three complementary components. First, the Direction-Aware Mixed Attention Module (DAMM) enhances directional continuity modeling for thin, tortuous, and branching vessels. Second, the Multi-scale Fixed Receptive-field Parallel Pyramid (MFRPP) aggregates cross-receptive-field information at the bottleneck to improve representation consistency across vessel scales. Third, GA-Loss adaptively optimizes the weights and internal parameters of a bounded composite-loss family to alleviate optimization bias caused by class imbalance. Experiments on DRIVE, CHASE_DB1, and STARE show that DAMMGA-Net achieves stable and competitive performance while better preserving fine vessels, reaching AUC 0.9945, F1 0.8721, and Acc 0.9804 on STARE, and AUC 0.9879, F1 0.8413, and Acc 0.9732 on DRIVE. Qualitative results further show improved connectivity and integrity in fine branches, tortuous vessels, and low-contrast regions. These findings indicate that the synergy of directional continuity modeling, stable cross-scale aggregation, and dataset-adaptive loss optimization is effective for retinal vessel segmentation under severe class imbalance.\u003c/p\u003e","manuscriptTitle":"DAMMGA-Net: Direction-Aware Attention with Genetic Algorithm-Optimized Loss for Retinal Vessel Segmentation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-24 04:01:31","doi":"10.21203/rs.3.rs-8934073/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewersInvited","content":"","date":"2026-04-17T09:32:29+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-04-17T09:23:42+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2026-04-07T11:31:37+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-04-02T04:08:06+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2026-04-02T04:05:35+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"5e479b33-5c5b-41f3-8695-8af8c48c4afa","owner":[],"postedDate":"April 24th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":66910879,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":66910880,"name":"Physical sciences/Engineering"},{"id":66910881,"name":"Physical sciences/Mathematics and computing"}],"tags":[],"updatedAt":"2026-04-24T04:01:32+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-24 04:01:31","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8934073","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8934073","identity":"rs-8934073","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}