Bridging Scale, Semantics, and Boundaries: A Hybrid CNN-Transformer Architecture with Bidirectional Spatial-Channel Fusion for Medical Image Segmentation

Bridging Scale, Semantics, and Boundaries: A Hybrid CNN-Transformer Architecture with Bidirectional Spatial-Channel Fusion for Medical Image Segmentation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Bridging Scale, Semantics, and Boundaries: A Hybrid CNN-Transformer Architecture with Bidirectional Spatial-Channel Fusion for Medical Image Segmentation Lanxiang Ma, Zongjian Yang, Jinghua Zhu, Jiquan Ma This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9178737/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract Accurate segmentation of anatomical structures in medical images is fundamental to a wide range of clinical applications, from disease diagnosis to treatment planning. However, this task remains persistently challenging due to substantial variations in anatomical scale, ambiguous tissue boundaries, and heterogeneous image appearances. Existing approaches, whether convolutional neural networks or vision transformers, often struggle to simultaneously capture long-range dependencies, preserve fine structural details, and adapt to diverse morphological contexts. To address these limitations, we introduce BRF-Net, a hybrid CNN-Transformer framework that unifies adaptive multi-scale feature aggregation, bidirectional spatial-channel refinement, and frequency-domain detail preservation within a single architecture. Specifically, we propose an Adaptive Gated Multi-Scale (AGMS) block that dynamically selects receptive fields based on image content; a Bidirectional Refinement and Fusion (BRF) Attention Block that enforces reciprocal conditioning between spatial and semantic features; and a Patch-wise Fourier Feed-Forward Network (PF-FFN) that explicitly preserves high-frequency boundary information through learnable spectral filtering. Here we show that BRF-Net achieves state-of-the-art performance across eight diverse public benchmarks covering abdominal organs, cardiac structures, polyps, skin lesions, breast lesions, and nuclei. It surpasses the strongest competing methods by an average of 0.87 points in Dice and 1.45 points in IoU on six binary datasets, while reducing the Hausdorff distance by 4.43. On the multi-organ Synapse dataset, it improves average Dice and IoU by 3.44 and 4.05 points, respectively. These results demonstrate that explicitly coupling scale adaptivity, spatial-semantic consistency, and boundary awareness yields substantial and robust improvements in segmentation fidelity, offering a more reliable tool for clinical image analysis. The source code is publicly available at GitHub and archived in Zenodo with DOI: \url{ https://doi.org/10.5281/zenodo.19129179} . Medical Image Segmentation Hybrid CNN-Transformer Bidirectional Attention Frequency Domain Learning Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 27 Apr, 2026 Reviews received at journal 19 Apr, 2026 Reviews received at journal 13 Apr, 2026 Reviewers agreed at journal 30 Mar, 2026 Reviewers agreed at journal 29 Mar, 2026 Reviewers invited by journal 29 Mar, 2026 Editor assigned by journal 21 Mar, 2026 Submission checks completed at journal 21 Mar, 2026 First submitted to journal 20 Mar, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9178737","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":614381668,"identity":"cf4e3526-b49a-4b13-9bf4-28d9bb76b101","order_by":0,"name":"Lanxiang Ma","email":"","orcid":"","institution":"Heilongjiang University","correspondingAuthor":false,"prefix":"","firstName":"Lanxiang","middleName":"","lastName":"Ma","suffix":""},{"id":614381669,"identity":"bea3b3f6-a7e7-4e44-8b32-7574c9000e7e","order_by":1,"name":"Zongjian Yang","email":"","orcid":"","institution":"Heilongjiang University","correspondingAuthor":false,"prefix":"","firstName":"Zongjian","middleName":"","lastName":"Yang","suffix":""},{"id":614381670,"identity":"4df6087e-d19c-4ccc-9cf7-dac1ec6fded7","order_by":2,"name":"Jinghua Zhu","email":"","orcid":"","institution":"Heilongjiang University","correspondingAuthor":false,"prefix":"","firstName":"Jinghua","middleName":"","lastName":"Zhu","suffix":""},{"id":614381671,"identity":"0a3fcb2e-e12d-49ca-9121-292e0d084143","order_by":3,"name":"Jiquan Ma","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA4klEQVRIiWNgGAWjYHCChAMgkl+CucEASDM2EK1FcgYj8VogwOAGRDFhLbrtBx4eeNtml2d8u7GhmIfBRnbDAeZnD/BpMTuTkHBwbltysdmdgw3GPAxpxhsOsJkb4NVyICHhMG8bc+K2G4kgLYcTNxzgYZPAq+X8A5CW+sTNM8Ba/hOh5QbYFqDhEmAtB4jR8iDh4JxzxxNnAB1mOMcg2XjmYTYzAg7LSf7wpqw6sX9G8jGDNxV2sn3Hm5/h1cLAwJPAwANhsRkwgIKKGb96IGA/ANPC/ICg4lEwCkbBKBiRAABFmlHrqX877AAAAABJRU5ErkJggg==","orcid":"","institution":"Heilongjiang University","correspondingAuthor":true,"prefix":"","firstName":"Jiquan","middleName":"","lastName":"Ma","suffix":""}],"badges":[],"createdAt":"2026-03-20 12:08:41","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9178737/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9178737/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":106093464,"identity":"f35ecbb8-21aa-40de-a02a-5f6671247500","added_by":"auto","created_at":"2026-04-03 11:37:28","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":5001096,"visible":true,"origin":"","legend":"","description":"","filename":"BRFNetforMedicalImageSegmentation.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9178737/v1_covered_82ab351a-a205-40c7-9f21-1c6d3a79b580.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Bridging Scale, Semantics, and Boundaries: A Hybrid CNN-Transformer Architecture with Bidirectional Spatial-Channel Fusion for Medical Image Segmentation","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"the-visual-computer","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"tvcj","sideBox":"Learn more about [The Visual Computer](http://link.springer.com/journal/371)","snPcode":"371","submissionUrl":"https://submission.nature.com/new-submission/371/3","title":"The Visual Computer","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Medical Image Segmentation, Hybrid CNN-Transformer, Bidirectional Attention, Frequency Domain Learning","lastPublishedDoi":"10.21203/rs.3.rs-9178737/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9178737/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Accurate segmentation of anatomical structures in medical images is fundamental to a wide range of clinical applications, from disease diagnosis to treatment planning. However, this task remains persistently challenging due to substantial variations in anatomical scale, ambiguous tissue boundaries, and heterogeneous image appearances. Existing approaches, whether convolutional neural networks or vision transformers, often struggle to simultaneously capture long-range dependencies, preserve fine structural details, and adapt to diverse morphological contexts. To address these limitations, we introduce BRF-Net, a hybrid CNN-Transformer framework that unifies adaptive multi-scale feature aggregation, bidirectional spatial-channel refinement, and frequency-domain detail preservation within a single architecture. Specifically, we propose an Adaptive Gated Multi-Scale (AGMS) block that dynamically selects receptive fields based on image content; a Bidirectional Refinement and Fusion (BRF) Attention Block that enforces reciprocal conditioning between spatial and semantic features; and a Patch-wise Fourier Feed-Forward Network (PF-FFN) that explicitly preserves high-frequency boundary information through learnable spectral filtering. Here we show that BRF-Net achieves state-of-the-art performance across eight diverse public benchmarks covering abdominal organs, cardiac structures, polyps, skin lesions, breast lesions, and nuclei. It surpasses the strongest competing methods by an average of 0.87 points in Dice and 1.45 points in IoU on six binary datasets, while reducing the Hausdorff distance by 4.43. On the multi-organ Synapse dataset, it improves average Dice and IoU by 3.44 and 4.05 points, respectively. These results demonstrate that explicitly coupling scale adaptivity, spatial-semantic consistency, and boundary awareness yields substantial and robust improvements in segmentation fidelity, offering a more reliable tool for clinical image analysis. The source code is publicly available at GitHub and archived in Zenodo with DOI: \\url{https://doi.org/10.5281/zenodo.19129179}.","manuscriptTitle":"Bridging Scale, Semantics, and Boundaries: A Hybrid CNN-Transformer Architecture with Bidirectional Spatial-Channel Fusion for Medical Image Segmentation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-01 20:26:37","doi":"10.21203/rs.3.rs-9178737/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-04-27T21:19:48+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-19T16:10:43+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-13T08:40:23+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"339122883436133331464523226647803502345","date":"2026-03-30T08:07:53+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"252389660814310192530699174364424834429","date":"2026-03-30T02:26:47+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-03-30T02:24:16+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-03-21T13:27:15+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-03-21T10:29:25+00:00","index":"","fulltext":""},{"type":"submitted","content":"The Visual Computer","date":"2026-03-20T12:02:55+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"the-visual-computer","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"tvcj","sideBox":"Learn more about [The Visual Computer](http://link.springer.com/journal/371)","snPcode":"371","submissionUrl":"https://submission.nature.com/new-submission/371/3","title":"The Visual Computer","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"56b3c6dc-a288-4d7c-818f-5c7109dd4f0f","owner":[],"postedDate":"April 1st, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-05-11T06:41:33+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-01 20:26:37","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9178737","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9178737","identity":"rs-9178737","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}