A Knowledge-driven Deep Learning Framework for Organoid Morphological Segmentation and Characterisation

A Knowledge-driven Deep Learning Framework for Organoid Morphological Segmentation and Characterisation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Knowledge-driven Deep Learning Framework for Organoid Morphological Segmentation and Characterisation Yiming Qin, Jiajia Li, Heng Yin, Dezhi Wu, Xuejiao Liu, Tobias Plötz, and 6 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5380950/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 7 You are reading this latest preprint version Abstract Background Organoids have great potential to revolutionize various aspects of biomedical research and healthcare. Researchers typically use the fluorescence-based approach to analyse their dynamics, which requires specialized equipment and may interfere with their growth. Therefore, it is an open challenge to develop a general framework to analyse organoid dynamics under non-invasive and low-resource settings. Results In this paper, we present a knowledge-driven deep learning system named TransOrga-plus to automatically analyse organoid dynamics in a non-invasive manner. Given a bright-field microscopic image, TransOrga-plus detects organoids through a multi-modal transformer-based segmentation module. To provide customized and robust organoid analysis, a biological knowledge-driven branch is embedded into the segmentation module which integrates biological knowledge, e.g., the morphological characteristics of organoids, into the analysis process. Then, based on the detection results, a lightweight multi-object tracking module based on the decoupling of visual and identity features is introduced to track organoids over time. Finally, TransOrga-plus outputs the dynamics analysis to assist biologists for further research. To train and validate our framework, we curate a large-scale organoid dataset encompassing diverse tissue types and various microscopic imaging settings. Extensive experimental results demonstrate that our method outperforms all baselines in organoid analysis. The results show that TransOrga-plus provides comparable analytical results to biologists and significantly accelerates organoid work process. Conclusions In conclusion, TransOrga-plus integrates the biological expertise with cutting-edge deep learning-based model and enables the non-invasive analysis of various organoids from complex, low-resource, and time-lapse situations. Organoid Deep learning Knowledge-driven Full Text Additional Declarations No competing interests reported. Supplementary Video not available with this version. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 18 Dec, 2024 Reviewers agreed at journal 13 Dec, 2024 Reviewers invited by journal 05 Nov, 2024 Editor invited by journal 05 Nov, 2024 Editor assigned by journal 05 Nov, 2024 Submission checks completed at journal 04 Nov, 2024 First submitted to journal 03 Nov, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5380950","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":374453782,"identity":"e97ea733-0f80-433e-8bd8-0dccb9852eab","order_by":0,"name":"Yiming Qin","email":"","orcid":"","institution":"Tsinghua University","correspondingAuthor":false,"prefix":"","firstName":"Yiming","middleName":"","lastName":"Qin","suffix":""},{"id":374453783,"identity":"24e3c0b6-d87d-4c41-9e02-3f231ba1b21b","order_by":1,"name":"Jiajia Li","email":"","orcid":"","institution":"Shanghai Jiao Tong University","correspondingAuthor":false,"prefix":"","firstName":"Jiajia","middleName":"","lastName":"Li","suffix":""},{"id":374453784,"identity":"47788b41-e344-40aa-a110-da673b044044","order_by":2,"name":"Heng Yin","email":"","orcid":"","institution":"Merck KGaA","correspondingAuthor":false,"prefix":"","firstName":"Heng","middleName":"","lastName":"Yin","suffix":""},{"id":374453785,"identity":"d76d0a33-27f8-4998-a48d-94d791b425ad","order_by":3,"name":"Dezhi Wu","email":"","orcid":"","institution":"Shanghai Jiao Tong University","correspondingAuthor":false,"prefix":"","firstName":"Dezhi","middleName":"","lastName":"Wu","suffix":""},{"id":374453786,"identity":"d60d0864-8e7c-44d9-97f7-7e6a37ea78d2","order_by":4,"name":"Xuejiao Liu","email":"","orcid":"","institution":"Fudan University","correspondingAuthor":false,"prefix":"","firstName":"Xuejiao","middleName":"","lastName":"Liu","suffix":""},{"id":374453787,"identity":"825a9cd8-eed8-48ee-b4c9-a3a4f4c1b729","order_by":5,"name":"Tobias Plötz","email":"","orcid":"","institution":"Merck KGaA","correspondingAuthor":false,"prefix":"","firstName":"Tobias","middleName":"","lastName":"Plötz","suffix":""},{"id":374453788,"identity":"39f9c53f-66f7-4387-a21c-9fec6727acbf","order_by":6,"name":"Alexander Hopp","email":"","orcid":"","institution":"Merck KGaA","correspondingAuthor":false,"prefix":"","firstName":"Alexander","middleName":"","lastName":"Hopp","suffix":""},{"id":374453790,"identity":"e7f8fd6b-2934-4bf1-bb8c-8736dd2f4c8f","order_by":7,"name":"Nicholas Kurniawan","email":"","orcid":"","institution":"Eindhoven University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Nicholas","middleName":"","lastName":"Kurniawan","suffix":""},{"id":374453792,"identity":"27ca76ca-4743-43a5-be8d-a06dfc709f5c","order_by":8,"name":"Mathias Winkel","email":"","orcid":"","institution":"Merck KGaA","correspondingAuthor":false,"prefix":"","firstName":"Mathias","middleName":"","lastName":"Winkel","suffix":""},{"id":374453795,"identity":"80854083-aaac-4531-bbc6-4a51c067c6e0","order_by":9,"name":"Philipp Harbach","email":"","orcid":"","institution":"Merck KGaA","correspondingAuthor":false,"prefix":"","firstName":"Philipp","middleName":"","lastName":"Harbach","suffix":""},{"id":374453796,"identity":"0cb61a68-a837-4a5e-8bb8-9ade0897897e","order_by":10,"name":"Chunling Tang","email":"","orcid":"","institution":"Eindhoven University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Chunling","middleName":"","lastName":"Tang","suffix":""},{"id":374453797,"identity":"f525e716-2206-4830-8e88-2130b5b79157","order_by":11,"name":"Feng Tan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA8ElEQVRIiWNgGAWjYBAC9gYogx9MHiBCCw9YUQIDg2QDyVoMDhCthb338AvGH3Z5xtcOH/vw40xtPr9E7gGGHxXbcGvhOZdmwZCQXGx2Oy15Zs+N45YzZ+QlMPacuY1Ti71EjpkBQwJz4rbbOcbMDB+OGRjcyDFgZmzDrYVH/g1IS33i5tn5n4nUIsFj/IAh4XDiBukcZmaGGzVEaOHJMWNISDueOON2mjHQCwcMJHveJRzE5xce9jPGHz7YVCf2z05+zPDjWJ0BP3vuwQc/KnBrAQI2iQQE5zDIGILRw/wBiVMH1jIKRsEoGAWjABkAAAkrWNq1xNW4AAAAAElFTkSuQmCC","orcid":"","institution":"Merck KGaA","correspondingAuthor":true,"prefix":"","firstName":"Feng","middleName":"","lastName":"Tan","suffix":""}],"badges":[],"createdAt":"2024-11-03 07:23:32","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5380950/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5380950/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":69109062,"identity":"81ab6740-16fe-46f3-a2c6-3433d6646451","added_by":"auto","created_at":"2024-11-15 18:20:43","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1200729,"visible":true,"origin":"","legend":"","description":"","filename":"Manuscripts.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5380950/v1_covered_9112e2bf-4419-4ff0-9840-890a516ef7b5.pdf"}],"financialInterests":"\u003cp\u003eNo competing interests reported.\u003c/p\u003e\n\u003cp\u003eSupplementary Video not available with this version.\u003c/p\u003e","formattedTitle":"A Knowledge-driven Deep Learning Framework for Organoid Morphological Segmentation and Characterisation","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-biology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [BMC Biology](https://bmcbiol.biomedcentral.com/)","snPcode":"12915","submissionUrl":"https://submission.springernature.com/new-submission/12915/3","title":"BMC Biology","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Organoid, Deep learning, Knowledge-driven","lastPublishedDoi":"10.21203/rs.3.rs-5380950/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5380950/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eOrganoids have great potential to revolutionize various aspects of biomedical research and healthcare. Researchers typically use the fluorescence-based approach to analyse their dynamics, which requires specialized equipment and may interfere with their growth. Therefore, it is an open challenge to develop a general framework to analyse organoid dynamics under non-invasive and low-resource settings.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eIn this paper, we present a knowledge-driven deep learning system named TransOrga-plus to automatically analyse organoid dynamics in a non-invasive manner. Given a bright-field microscopic image, TransOrga-plus detects organoids through a multi-modal transformer-based segmentation module. To provide customized and robust organoid analysis, a biological knowledge-driven branch is embedded into the segmentation module which integrates biological knowledge, e.g., the morphological characteristics of organoids, into the analysis process. Then, based on the detection results, a lightweight multi-object tracking module based on the decoupling of visual and identity features is introduced to track organoids over time. Finally, TransOrga-plus outputs the dynamics analysis to assist biologists for further research. To train and validate our framework, we curate a large-scale organoid dataset encompassing diverse tissue types and various microscopic imaging settings. Extensive experimental results demonstrate that our method outperforms all baselines in organoid analysis. The results show that TransOrga-plus provides comparable analytical results to biologists and significantly accelerates organoid work process.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eIn conclusion, TransOrga-plus integrates the biological expertise with cutting-edge deep learning-based model and enables the non-invasive analysis of various organoids from complex, low-resource, and time-lapse situations.\u003c/p\u003e","manuscriptTitle":"A Knowledge-driven Deep Learning Framework for Organoid Morphological Segmentation and Characterisation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-11-15 18:12:33","doi":"10.21203/rs.3.rs-5380950/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-12-18T14:07:13+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"67678155070200236280502944368499476139","date":"2024-12-13T09:31:37+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-11-05T15:05:54+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-11-05T10:59:02+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-11-05T07:55:48+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-11-04T08:51:38+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Biology","date":"2024-11-03T07:15:09+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-biology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [BMC Biology](https://bmcbiol.biomedcentral.com/)","snPcode":"12915","submissionUrl":"https://submission.springernature.com/new-submission/12915/3","title":"BMC Biology","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"5c2c9c0e-1130-46d9-91bb-fc75660dd777","owner":[],"postedDate":"November 15th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-09-05T13:23:29+00:00","versionOfRecord":[],"versionCreatedAt":"2024-11-15 18:12:33","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5380950","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5380950","identity":"rs-5380950","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}