A multimodal deep learning framework with contrastive learning and multi-instance learning for endometrial cancer preoperative risk stratification

A multimodal deep learning framework with contrastive learning and multi-instance learning for endometrial cancer preoperative risk stratification | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article A multimodal deep learning framework with contrastive learning and multi-instance learning for endometrial cancer preoperative risk stratification Shenghua Cheng, Hongtao Kang, Yu Wang, Haoqiang He, Gengxi Cai, and 13 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7435825/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted You are reading this latest preprint version Abstract Endometrial cancer (EC) risk stratification currently relies on postoperative pathology, limiting preoperative surgical planning. Magnetic resonance imaging (MRI) is the preferred modality for preoperative EC evaluation. However, current research on EC preoperative risk stratification has certain limitations: traditional radiomics demonstrate deficient performance while relying on manual segmentation, and unimodal designs that fail to fully leverage clinical data. To address this, we developed an automated deep learning framework integrating multiparametric MRI and MRI reports from 2,662 patients across six centers. Key innovations include: contrastive learning leverages MRI reports to supervise and refine the image encoder, aligning visual features with diagnostic semantics; and multi-instance learning dynamically aggregates features from multi-parametric MRI sequences, even in the presence of missing data. Our multimodal model significantly outperformed unimodal approaches, achieving AUCs of 0.827 (internal validation) and 0.768-0.863 (four external cohorts), improving AUC by 4.7%-10.2% versus image-only and 4.1%-8.5% versus text-only models. This demonstrates strong generalizability and offers potential to optimize surgical planning, improve prognosis, and reduce complications. Health sciences/Oncology/Cancer/Gynaecological cancer/Endometrial cancer Health sciences/Risk factors Physical sciences/Mathematics and computing/Information technology Physical sciences/Engineering/Biomedical engineering Endometrial cancer preoperative risk stratification multimodal deep learning multi-instance learning contrastive learning Full Text Additional Declarations There is NO Competing Interest. Supplementary Files Supplementaryinformation.pdf Supplementary information Cite Share Download PDF Status: Under Review Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7435825","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":509604082,"identity":"f7b719eb-6779-45cc-9bd3-3640fd841439","order_by":0,"name":"Shenghua Cheng","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6klEQVRIiWNgGAWjYDACCTB5gIGBvQcqcoBoLTxngHQCSVokcojUwj+7+djDL3/uyJlLvj0m+fMHgxzfjQTGzwX4LLlzLN1Ytu2ZseXsvDRpngQGY8kbCczSM/BoMZDIMZOWbDicuOE2kAF0WOKGGwlszDx4teR/k5b4c7h+w80zZpI/EhjqidCSwyb5ge1wgsENHjMJoMOADAJaJG6kmUkzth023HAmx9iaJ03CcOaZh83S+LTwz0h+Jvnjz2F5g+NnDG/+sLGR5zuefPAzPi0ggOwMUDQxNhDQAFTyg6CSUTAKRsEoGNEAAGB4THAEVDXnAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0003-3527-3845","institution":"Southern Medical University","correspondingAuthor":true,"prefix":"","firstName":"Shenghua","middleName":"","lastName":"Cheng","suffix":""},{"id":509604083,"identity":"e02b27d8-7397-4e1c-8908-ac60255a86bc","order_by":1,"name":"Hongtao Kang","email":"","orcid":"https://orcid.org/0000-0001-8194-5548","institution":"Southern Medical University","correspondingAuthor":false,"prefix":"","firstName":"Hongtao","middleName":"","lastName":"Kang","suffix":""},{"id":509604084,"identity":"cdbb4eaf-3f6d-474a-93c9-a29fad8e6b51","order_by":2,"name":"Yu Wang","email":"","orcid":"https://orcid.org/0000-0001-6731-6518","institution":"Sun Yat-Sen University","correspondingAuthor":false,"prefix":"","firstName":"Yu","middleName":"","lastName":"Wang","suffix":""},{"id":509604085,"identity":"79d64a85-8493-45fb-b08a-5a5b44a225c0","order_by":3,"name":"Haoqiang He","email":"","orcid":"","institution":"Sun Yat-sen University Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Haoqiang","middleName":"","lastName":"He","suffix":""},{"id":509604086,"identity":"62e7240f-6db1-4d2a-8c05-87a280217d5a","order_by":4,"name":"Gengxi Cai","email":"","orcid":"","institution":"Department of Breast Surgery, The First People’s Hospital of Foshan, Foshan, China","correspondingAuthor":false,"prefix":"","firstName":"Gengxi","middleName":"","lastName":"Cai","suffix":""},{"id":509604087,"identity":"2d98108a-5dfd-4f1a-b58c-9e00f988afaa","order_by":5,"name":"Yujian Zou","email":"","orcid":"","institution":"Affiliated Dongguan Hospital, Southern Medical University","correspondingAuthor":false,"prefix":"","firstName":"Yujian","middleName":"","lastName":"Zou","suffix":""},{"id":509604088,"identity":"bd720870-96e1-47c3-8f39-41f465053d17","order_by":6,"name":"Jianpeng Li","email":"","orcid":"","institution":"Affiliated Dongguan People's Hospital, Southern Medical University","correspondingAuthor":false,"prefix":"","firstName":"Jianpeng","middleName":"","lastName":"Li","suffix":""},{"id":509604089,"identity":"d31b4ed1-aac2-44e8-ae6e-3700cba7d677","order_by":7,"name":"Yi Shen","email":"","orcid":"","institution":"Shantou Central Hospital","correspondingAuthor":false,"prefix":"","firstName":"Yi","middleName":"","lastName":"Shen","suffix":""},{"id":509604090,"identity":"b6c49196-9689-4650-bf38-74b902cd6f3f","order_by":8,"name":"Jiayi Zhuo","email":"","orcid":"","institution":"Shenshan Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Jiayi","middleName":"","lastName":"Zhuo","suffix":""},{"id":509604091,"identity":"26475c2c-634f-4ffb-8e39-8acaa0a94b20","order_by":9,"name":"Abuduxukuer Aili","email":"","orcid":"","institution":"Sun Yat-sen Memorial Hospital","correspondingAuthor":false,"prefix":"","firstName":"Abuduxukuer","middleName":"","lastName":"Aili","suffix":""},{"id":509604092,"identity":"ee3d8dbf-42c0-488d-a1d1-5c20308abc74","order_by":10,"name":"Lingjie Yang","email":"","orcid":"","institution":"Sun Yat-sen Memorial Hospital, Sun Yat-sen University","correspondingAuthor":false,"prefix":"","firstName":"Lingjie","middleName":"","lastName":"Yang","suffix":""},{"id":509604093,"identity":"67174258-87c0-448f-b1a0-5e4b85a47f07","order_by":11,"name":"Longfeng Yang","email":"","orcid":"https://orcid.org/0009-0007-5940-6303","institution":"Sun Yat-sen Memorial Hospital","correspondingAuthor":false,"prefix":"","firstName":"Longfeng","middleName":"","lastName":"Yang","suffix":""},{"id":509604094,"identity":"d72f0afe-1b80-470b-8453-f930725b3f0a","order_by":12,"name":"Wenjun Zheng","email":"","orcid":"","institution":"School of Biomedical Engineering and Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, Guangzhou, China","correspondingAuthor":false,"prefix":"","firstName":"Wenjun","middleName":"","lastName":"Zheng","suffix":""},{"id":509604095,"identity":"ceb76cfd-b214-484e-a467-257e8c8ebe02","order_by":13,"name":"Lan Zhang","email":"","orcid":"","institution":"School of Biomedical Engineering and Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, Guangzhou, China","correspondingAuthor":false,"prefix":"","firstName":"Lan","middleName":"","lastName":"Zhang","suffix":""},{"id":509604096,"identity":"5e971220-69d3-4b64-ba47-e765c5751a87","order_by":14,"name":"Peiqi Zhuang","email":"","orcid":"https://orcid.org/0009-0007-9072-0923","institution":"School of Biomedical Engineering and Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, Guangzhou, China","correspondingAuthor":false,"prefix":"","firstName":"Peiqi","middleName":"","lastName":"Zhuang","suffix":""},{"id":509604097,"identity":"0b3a3c6c-5294-46b3-928f-dc5444cb9142","order_by":15,"name":"Daiying Lin","email":"","orcid":"","institution":"Shantou Central Hospital","correspondingAuthor":false,"prefix":"","firstName":"Daiying","middleName":"","lastName":"Lin","suffix":""},{"id":509604098,"identity":"1084c0dd-d825-4ea0-8e00-01b8439e4d5d","order_by":16,"name":"Xiaohua Ban","email":"","orcid":"https://orcid.org/0009-0000-2322-9092","institution":"Sun Yat-sen University Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Xiaohua","middleName":"","lastName":"Ban","suffix":""},{"id":509604099,"identity":"12ad7b06-3439-4288-909b-89d8a9a6757e","order_by":17,"name":"Xiaohui Duan","email":"","orcid":"https://orcid.org/0000-0002-2224-0887","institution":"Department of Radiology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University","correspondingAuthor":false,"prefix":"","firstName":"Xiaohui","middleName":"","lastName":"Duan","suffix":""}],"badges":[],"createdAt":"2025-08-22 15:25:32","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7435825/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7435825/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":90570344,"identity":"e7f8ca1f-a7bc-40c8-949e-03ff0e091bd5","added_by":"auto","created_at":"2025-09-04 08:16:00","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":5153716,"visible":true,"origin":"","legend":"Article File","description":"","filename":"main.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7435825/v1_covered_7de5a2c5-99cb-461c-972e-eb1300ba5dba.pdf"},{"id":90569208,"identity":"2a7f027d-0a74-43d3-9bb2-13cca3836530","added_by":"auto","created_at":"2025-09-04 08:07:58","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":2345158,"visible":true,"origin":"","legend":"Supplementary information","description":"","filename":"Supplementaryinformation.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7435825/v1/45edce27fb901cd87b4a5523.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"A multimodal deep learning framework with contrastive learning and multi-instance learning for endometrial cancer preoperative risk stratification","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Endometrial cancer, preoperative risk stratification, multimodal deep learning, multi-instance learning, contrastive learning","lastPublishedDoi":"10.21203/rs.3.rs-7435825/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7435825/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Endometrial cancer (EC) risk stratification currently relies on postoperative pathology, limiting preoperative surgical planning. Magnetic resonance imaging (MRI) is the preferred modality for preoperative EC evaluation. However, current research on EC preoperative risk stratification has certain limitations: traditional radiomics demonstrate deficient performance while relying on manual segmentation, and unimodal designs that fail to fully leverage clinical data. To address this, we developed an automated deep learning framework integrating multiparametric MRI and MRI reports from 2,662 patients across six centers. Key innovations include: contrastive learning leverages MRI reports to supervise and refine the image encoder, aligning visual features with diagnostic semantics; and multi-instance learning dynamically aggregates features from multi-parametric MRI sequences, even in the presence of missing data. Our multimodal model significantly outperformed unimodal approaches, achieving AUCs of 0.827 (internal validation) and 0.768-0.863 (four external cohorts), improving AUC by 4.7%-10.2% versus image-only and 4.1%-8.5% versus text-only models. This demonstrates strong generalizability and offers potential to optimize surgical planning, improve prognosis, and reduce complications.","manuscriptTitle":"A multimodal deep learning framework with contrastive learning and multi-instance learning for endometrial cancer preoperative risk stratification","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-04 08:07:53","doi":"10.21203/rs.3.rs-7435825/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"ec8ac0e7-a81e-4e08-9c1c-40ffa6dd88b4","owner":[],"postedDate":"September 4th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":54125087,"name":"Health sciences/Oncology/Cancer/Gynaecological cancer/Endometrial cancer"},{"id":54125088,"name":"Health sciences/Risk factors"},{"id":54125089,"name":"Physical sciences/Mathematics and computing/Information technology"},{"id":54125090,"name":"Physical sciences/Engineering/Biomedical engineering"}],"tags":[],"updatedAt":"2026-04-17T17:06:28+00:00","versionOfRecord":[],"versionCreatedAt":"2025-09-04 08:07:53","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7435825","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7435825","identity":"rs-7435825","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}