Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning

Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning Chunfeng Lian, Shen Chen, Wanqing Zhang, Fan Wang, Jianhua Zhang, and 9 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4618703/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 23 Jul, 2025 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Abstract Forensic pathology is critical in determining the cause and manner of death through post-mortem examinations, both macroscopic and microscopic. The field, however, grapples with issues such as outcome variability, laborious processes, and a scarcity of trained professionals. This paper presents SongCi, an innovative visual-language model (VLM) designed specifically for forensic pathology. SongCi utilizes advanced prototypical cross-modal self-supervised contrastive learning to enhance the accuracy, efficiency, and generalizability of forensic analyses. It was pre-trained and evaluated on a comprehensive multi-center dataset, which includes over 16 million high-resolution image patches, 2,228 vision-language pairs of post-mortem whole slide images (WSIs), and corresponding gross key findings, along with 471 distinct diagnostic outcomes. Our findings indicate that SongCi surpasses existing multi-modal AI models in many forensic pathology tasks, performs comparably to experienced forensic pathologists and significantly better than less experienced ones, and provides detailed multi-modal explainability, offering critical assistance in forensic investigations. To the best of our knowledge, SongCi is the first VLM specifically developed for forensic pathological analysis and the first large-vocabulary computational pathology (CPath) model that directly processes gigapixel WSIs in forensic science. Biological sciences/Computational biology and bioinformatics/Image processing Biological sciences/Computational biology and bioinformatics/Machine learning Biological sciences/Computational biology and bioinformatics/Data mining Full Text Additional Declarations There is NO Competing Interest. Supplementary Files supplementaryinformation.docx Cite Share Download PDF Status: Published Journal Publication published 23 Jul, 2025 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4618703","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":328112840,"identity":"7d880493-7309-45a6-ae5d-4d35db45eb10","order_by":0,"name":"Chunfeng Lian","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABCElEQVRIiWNgGAWjYDACCSCuADHYGRtALGawKA8hLWdADGaQljMMzDwkaAFixjaoanxa+Gf3mD04UHHHroGZufFz4bzD7PYSCYwP3rYxyJvjsuTOGXODA2eeJTcwMzZLz9yWxswjkcBsOLeNwXBnA3YtBhI5ZtIf2w4ng/wizbvNBqSFTZq3jSHB4ABuLRIH/4G1NP/mnSMB0sL+m7CWhsN2QC1t0rwNEFuY8WmRuJFWJnHg2GGgMsY2a55jQL+cedgsOeechOEGHFr4ZyRvkzhQc9ien7398W2emsPJ7O3JBz+8KbORx2ULDCS2QRnJwNhpYIDEF35gD2PYEVQ6CkbBKBgFIw4AAEKGUMEduXnrAAAAAElFTkSuQmCC","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":true,"prefix":"","firstName":"Chunfeng","middleName":"","lastName":"Lian","suffix":""},{"id":328112841,"identity":"851a868e-2b98-4a6c-9471-2fdb303ff67d","order_by":1,"name":"Shen Chen","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Shen","middleName":"","lastName":"Chen","suffix":""},{"id":328112842,"identity":"2e50e04e-676b-4529-891c-650cdb826806","order_by":2,"name":"Wanqing Zhang","email":"","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Wanqing","middleName":"","lastName":"Zhang","suffix":""},{"id":328112843,"identity":"eb21106a-275f-4272-bc1d-c30826bdccd7","order_by":3,"name":"Fan Wang","email":"","orcid":"","institution":"University of North Carolina at Chapel Hill","correspondingAuthor":false,"prefix":"","firstName":"Fan","middleName":"","lastName":"Wang","suffix":""},{"id":328112844,"identity":"3cedf6ca-f96c-4d0c-a383-9ed4031f595e","order_by":4,"name":"Jianhua Zhang","email":"","orcid":"","institution":"Shanghai Forensic Service Platform","correspondingAuthor":false,"prefix":"","firstName":"Jianhua","middleName":"","lastName":"Zhang","suffix":""},{"id":328112845,"identity":"6706be44-f014-45ec-868b-4def29de6c8b","order_by":5,"name":"Shuanliang Fan","email":"","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Shuanliang","middleName":"","lastName":"Fan","suffix":""},{"id":328112846,"identity":"823981af-fa01-42bd-86d8-b007ffd7c972","order_by":6,"name":"Xin Wei","email":"","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Xin","middleName":"","lastName":"Wei","suffix":""},{"id":328112847,"identity":"43ec9558-6356-4275-9eb6-ec97247320fd","order_by":7,"name":"Gongji Wang","email":"","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Gongji","middleName":"","lastName":"Wang","suffix":""},{"id":328112848,"identity":"44c07ad4-5979-4d65-af94-530927fb62eb","order_by":8,"name":"Kehan Li","email":"","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Kehan","middleName":"","lastName":"Li","suffix":""},{"id":328112849,"identity":"96b13a52-3132-4b4b-8175-f9200def2470","order_by":9,"name":"Hongshu Mu","email":"","orcid":"","institution":"Xian'yang Public Security Bureau","correspondingAuthor":false,"prefix":"","firstName":"Hongshu","middleName":"","lastName":"Mu","suffix":""},{"id":328112850,"identity":"c34c6104-4b54-4958-81e5-bb36ef315360","order_by":10,"name":"Hao Wu","email":"","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Hao","middleName":"","lastName":"Wu","suffix":""},{"id":328112851,"identity":"82190ff7-1dc4-4ab0-ab61-337992572e7c","order_by":11,"name":"Xinggong Liang","email":"","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Xinggong","middleName":"","lastName":"Liang","suffix":""},{"id":328112852,"identity":"7e0f62b0-fbfc-4fb7-9398-57f6279adad5","order_by":12,"name":"Jianhua Ma","email":"","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Jianhua","middleName":"","lastName":"Ma","suffix":""},{"id":328112853,"identity":"ec9120b4-35f8-49ac-9fbf-ebafec8b627f","order_by":13,"name":"Zhenyuan Wang","email":"","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Zhenyuan","middleName":"","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2024-06-21 17:29:16","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4618703/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4618703/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41467-025-62060-x","type":"published","date":"2025-07-23T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":87464663,"identity":"7b3e3a61-a5e6-4036-a473-4ca8b16da5f4","added_by":"auto","created_at":"2025-07-24 07:06:49","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":10577377,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4618703/v1_covered_b8f0b67f-e51d-4a14-adae-308078fccd14.pdf"},{"id":60638159,"identity":"b75d8802-f555-4976-b298-1639887722be","added_by":"auto","created_at":"2024-07-19 02:51:26","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":1555282,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"supplementaryinformation.docx","url":"https://assets-eu.researchsquare.com/files/rs-4618703/v1/867c382cc72188bb28f4102c.docx"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-4618703/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4618703/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eForensic pathology is critical in determining the cause and manner of death through post-mortem examinations, both macroscopic and microscopic. The field, however, grapples with issues such as outcome variability, laborious processes, and a scarcity of trained professionals. This paper presents SongCi, an innovative visual-language model (VLM) designed specifically for forensic pathology. SongCi utilizes advanced prototypical cross-modal self-supervised contrastive learning to enhance the accuracy, efficiency, and generalizability of forensic analyses. It was pre-trained and evaluated on a comprehensive multi-center dataset, which includes over 16\u0026nbsp;million high-resolution image patches, 2,228 vision-language pairs of post-mortem whole slide images (WSIs), and corresponding gross key findings, along with 471 distinct diagnostic outcomes. Our findings indicate that SongCi surpasses existing multi-modal AI models in many forensic pathology tasks, performs comparably to experienced forensic pathologists and significantly better than less experienced ones, and provides detailed multi-modal explainability, offering critical assistance in forensic investigations. To the best of our knowledge, SongCi is the first VLM specifically developed for forensic pathological analysis and the first large-vocabulary computational pathology (CPath) model that directly processes gigapixel WSIs in forensic science.\u003c/p\u003e","manuscriptTitle":"Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-07-19 02:51:21","doi":"10.21203/rs.3.rs-4618703/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"21eee3eb-9156-4030-96ff-7ae311ee9a32","owner":[],"postedDate":"July 19th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":34726242,"name":"Biological sciences/Computational biology and bioinformatics/Image processing"},{"id":34726243,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"},{"id":34726244,"name":"Biological sciences/Computational biology and bioinformatics/Data mining"}],"tags":[],"updatedAt":"2025-07-24T07:06:39+00:00","versionOfRecord":{"articleIdentity":"rs-4618703","link":"https://doi.org/10.1038/s41467-025-62060-x","journal":{"identity":"nature-communications","isVorOnly":false,"title":"Nature Communications"},"publishedOn":"2025-07-23 04:00:00","publishedOnDateReadable":"July 23rd, 2025"},"versionCreatedAt":"2024-07-19 02:51:21","video":"","vorDoi":"10.1038/s41467-025-62060-x","vorDoiUrl":"https://doi.org/10.1038/s41467-025-62060-x","workflowStages":[]},"version":"v1","identity":"rs-4618703","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4618703","identity":"rs-4618703","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}