Tuning Vision Foundation Models for Rectal Cancer Segmentation from CT Scans: Development and Validation of U-SAM

Tuning Vision Foundation Models for Rectal Cancer Segmentation from CT Scans: Development and Validation of U-SAM | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Tuning Vision Foundation Models for Rectal Cancer Segmentation from CT Scans: Development and Validation of U-SAM Shouhong Wan, hantao zhang, Weidong Guo, Bingbing Zou, Wanqin Wang, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4476094/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 01 Jul, 2025 Read the published version in Communications Medicine → Version 1 posted You are reading this latest preprint version Abstract Background: Rectal cancer segmentation in CT is crucial for timely diagnosis. Despite promising methods, challenges remain due to the rectum's complex anatomy and the lack of a comprehensive annotated dataset. Methods: A total of 33,024 slice pairs from 398 rectal cancer patients in a new source center are enrolled into our dataset, named CARE Dataset, with pixel-level annotations for both normal and cancerous rectum tissue. We split it into 317 cases for training and 81 for testing. Additionally, we introduce a novel segmentation model, U-SAM, designed to handle the complex anatomy of the rectum by incorporating prompt information. Segmentation performance for normal and cancerous rectum is evaluated using Intersection-over-Union (IoU) and Dice Coefficient (Dice). With the assistance of 46 clinical practitioners, an observer study is conducted to benchmark the U-SAM with human performance and evaluate its clinical applicability. The original new source 398 CT scans and our code are openly available for research. Findings: Our method achieves Dice of 71.23% for normal rectum and 76.38% for rectal tumor, with IoU of 55.32% and 61.78%, respectively, surpassing state-of-the-art methods. The observer study validates that U-SAM can produce diagnostic results comparable to those of highly experienced doctors in just 3 seconds of inference time in clinical settings. Conclusions: The proposed U-SAM offers an efficient and reliable method for segmenting rectal cancer and normal tissue, significantly reducing time in clinical settings and effectively assisting radiologists. We believe this initial exploration in CT-based rectal cancer segmentation will be instrumental for future diagnosis. Health sciences/Gastroenterology/Gastrointestinal diseases/Gastrointestinal cancer/Colorectal cancer/Rectal cancer Health sciences/Health care/Medical imaging/Tomography/Computed tomography Rectal cancer CT scans Deep learning Segmentation Full Text Additional Declarations There is NO Competing Interest. Cite Share Download PDF Status: Published Journal Publication published 01 Jul, 2025 Read the published version in Communications Medicine → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4476094","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":314589206,"identity":"0063312f-9d21-4446-903c-dc8ec660cef6","order_by":0,"name":"Shouhong Wan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAwElEQVRIiWNgGAWjYJACZoYKKIuHeC1noKqJ18LYRooWg+NnD38unHfY3p79AOODt20M8uYEtZzJS5Oeue1wYg9PArPh3DYGw50NBLSYHcgxY+bddjiBR4KBTZq3jSHB4AAhLeffGH/mnXPYHqiF/TdxWm7kGEjzNhxm7AHawkyUFvsbb8ykeY6lJ/acSWyWnHNOwnADIS2S/TnGn3lqrO3Z2w8f/PCmzEaeoC1IgLEBSEgQr34UjIJRMApGAW4AAF9gOWcomJ8YAAAAAElFTkSuQmCC","orcid":"","institution":"School of Computer Science and Technology, University of Science and Technology of China, Hefei, China","correspondingAuthor":true,"prefix":"","firstName":"Shouhong","middleName":"","lastName":"Wan","suffix":""},{"id":314589207,"identity":"89d051f6-1dc8-4861-baa3-c8e0db8229d7","order_by":1,"name":"hantao zhang","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"hantao","middleName":"","lastName":"zhang","suffix":""},{"id":314589208,"identity":"f5c70af6-b4cf-4c52-a30f-27f11703ab59","order_by":2,"name":"Weidong Guo","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Weidong","middleName":"","lastName":"Guo","suffix":""},{"id":314589209,"identity":"1feadf2c-8f15-425a-b994-5440e641786b","order_by":3,"name":"Bingbing Zou","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Bingbing","middleName":"","lastName":"Zou","suffix":""},{"id":314589210,"identity":"3327303d-629d-4032-9114-d872b11d996c","order_by":4,"name":"Wanqin Wang","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Wanqin","middleName":"","lastName":"Wang","suffix":""},{"id":314589211,"identity":"cf2b11ff-e443-463f-b8a9-7e2fc5bcce23","order_by":5,"name":"Chenyang Qiu","email":"","orcid":"","institution":"Department of General Surgery, The First Affiliated Hospital of Anhui Medical University, Hefei, China","correspondingAuthor":false,"prefix":"","firstName":"Chenyang","middleName":"","lastName":"Qiu","suffix":""},{"id":314589212,"identity":"d262c5c9-1bb8-468f-909e-4fecc1221fa4","order_by":6,"name":"Kaige Liu","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Kaige","middleName":"","lastName":"Liu","suffix":""},{"id":314589213,"identity":"e4e32c21-6876-4f57-a948-397c5bc6991b","order_by":7,"name":"Peiquan Jin","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Peiquan","middleName":"","lastName":"Jin","suffix":""},{"id":314589214,"identity":"593d04b1-5368-4d95-a428-f2a2affb5891","order_by":8,"name":"jiancheng yang","email":"","orcid":"https://orcid.org/0000-0003-4455-7145","institution":"","correspondingAuthor":false,"prefix":"","firstName":"jiancheng","middleName":"","lastName":"yang","suffix":""}],"badges":[],"createdAt":"2024-05-25 09:15:13","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4476094/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4476094/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s43856-025-00953-0","type":"published","date":"2025-07-01T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":85828148,"identity":"7adc7817-5a78-4e6c-809b-cbc339f60adf","added_by":"auto","created_at":"2025-07-02 07:26:19","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":5378810,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4476094/v1_covered_eef24d07-8ff4-4422-8e17-b0ad4b65b984.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Tuning Vision Foundation Models for Rectal Cancer Segmentation from CT Scans: Development and Validation of U-SAM","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Rectal cancer, CT scans, Deep learning, Segmentation","lastPublishedDoi":"10.21203/rs.3.rs-4476094/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4476094/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eBackground: Rectal cancer segmentation in CT is crucial for timely diagnosis. Despite promising methods, challenges remain due to the rectum's complex anatomy and the lack of a comprehensive annotated dataset.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eMethods: A total of 33,024 slice pairs from 398 rectal cancer patients in a new source center are enrolled into our dataset, named CARE Dataset, with pixel-level annotations for both normal and cancerous rectum tissue. We split it into 317 cases for training and 81 for testing. Additionally, we introduce a novel segmentation model, U-SAM, designed to handle the complex anatomy of the rectum by incorporating prompt information. Segmentation performance for normal and cancerous rectum is evaluated using Intersection-over-Union (IoU) and Dice Coefficient (Dice). With the assistance of 46 clinical practitioners, an observer study is conducted to benchmark the U-SAM with human performance and evaluate its clinical applicability. The original new source 398 CT scans and our code are openly available for research.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFindings: Our method achieves Dice of 71.23% for normal rectum and 76.38% for rectal tumor, with IoU of 55.32% and 61.78%, respectively, surpassing state-of-the-art methods. The observer study validates that U-SAM can produce diagnostic results comparable to those of highly experienced doctors in just 3 seconds of inference time in clinical settings.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eConclusions: The proposed U-SAM offers an efficient and reliable method for segmenting rectal cancer and normal tissue, significantly reducing time in clinical settings and effectively assisting radiologists. We believe this initial exploration in CT-based rectal cancer segmentation will be instrumental for future diagnosis.\u003c/p\u003e","manuscriptTitle":"Tuning Vision Foundation Models for Rectal Cancer Segmentation from CT Scans: Development and Validation of U-SAM","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-07-02 06:11:16","doi":"10.21203/rs.3.rs-4476094/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"communications-medicine","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"commsmed","sideBox":"Learn more about [Communications Medicine](http://www.nature.com/commsmed)","snPcode":"43856","submissionUrl":"https://mts-commsmed.nature.com/cgi-bin/main.plex","title":"Communications Medicine","twitterHandle":"@commsmedicine","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Communications Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"f17c7a46-f049-4e2a-90de-802a75a82579","owner":[],"postedDate":"July 2nd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":33274967,"name":"Health sciences/Gastroenterology/Gastrointestinal diseases/Gastrointestinal cancer/Colorectal cancer/Rectal cancer"},{"id":33274968,"name":"Health sciences/Health care/Medical imaging/Tomography/Computed tomography"}],"tags":[],"updatedAt":"2025-07-02T07:26:11+00:00","versionOfRecord":{"articleIdentity":"rs-4476094","link":"https://doi.org/10.1038/s43856-025-00953-0","journal":{"identity":"communications-medicine","isVorOnly":false,"title":"Communications Medicine"},"publishedOn":"2025-07-01 04:00:00","publishedOnDateReadable":"July 1st, 2025"},"versionCreatedAt":"2024-07-02 06:11:16","video":"","vorDoi":"10.1038/s43856-025-00953-0","vorDoiUrl":"https://doi.org/10.1038/s43856-025-00953-0","workflowStages":[]},"version":"v1","identity":"rs-4476094","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4476094","identity":"rs-4476094","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}