AI-enabled live-dead cell viability classification and motion forecasting

AI-enabled live-dead cell viability classification and motion forecasting | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article AI-enabled live-dead cell viability classification and motion forecasting Paul Bogdan, Anzhe Cheng, Chenzhong Yin, Michael Lamba, Mathieu Sertorio, and 8 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8197080/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted You are reading this latest preprint version Abstract Distinguishing live from dead cells is crucial in a wide variety of research fields, including regenerative medicine, toxicology, pharmacology, and cellular product manufacturing, because it allows researchers to evaluate the efficacy and toxicity of molecules, materials, and therapies and ensures the quality of manufactured cell products. The cost of failure or uncertainty for live-dead analysis can be particularly high for therapeutic compound screening and evaluating medical therapies. Here, we present a novel deep learning framework that integrates a self-attention UNet for segmentation and a transformer network for dynamic tracking of cell movements. Our proposed model achieves state-of-the-art performance, with a high intersection-over-union (IoU) score of 96% and an area-under-curve (AUC) score of 99% for cell segmentation and over 65% IoU of full image cell motion forecasting, highlighting its ability to predict cell dynamics accurately. The self-attention mechanism significantly enhances the model’s ability to differentiate live and dead cells, even in densely packed or morphologically diverse environments. Additionally, the transformer network effectively captures temporal dependencies, enabling precise predictions of future cell movements. This integrated framework demonstrates robust performance across diverse datasets, consistently outperforming existing methods. By offering high-accuracy segmentation and predictive modeling, our approach provides a transformative tool for advancing cellular analysis in research and clinical applications, including cell therapeutics, cancer diagnostics, drug development, and regenerative medicine. Biological sciences/Cell biology Biological sciences/Computational biology and bioinformatics/Machine learning Full Text Additional Declarations There is NO Competing Interest. Supplementary Files Appendix.pdf Appendix machinelearningchecklistBogdan.pdf Machine Learning Checklist nrreportingsummaryBogdan.pdf Reporting Summary nrsoftwarepolicyBogdan.pdf Software Policy Checklist Cite Share Download PDF Status: Under Review Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8197080","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":581378207,"identity":"ca28039d-3e42-415b-b917-4ddc92fef9d5","order_by":0,"name":"Paul Bogdan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAsElEQVRIiWNgGAWjYDACCSDmYWCQg/DYSNBiTLqWxAaitZhLNx9+8abmTvqG84cfMHwoO0xYi+WcY2lA/Cx3w400A8YZ54jQYnAjx8yYh+0wUAsPAzNvG9Fa/h1ONzh/hoH5L5FajB8DDU8wOJDDwMxIjBaQXxjn9h02nAn0y8Gec+mEtYBC7MObb4fl+c4ffvjgR5k1EQ4DxoUEjHOAsHqIFuYPRKkcBaNgFIyCkQsAZsA/qfale/QAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0003-2118-0816","institution":"University of Southern California","correspondingAuthor":true,"prefix":"","firstName":"Paul","middleName":"","lastName":"Bogdan","suffix":""},{"id":581378208,"identity":"6f7e38e4-ab2e-4699-aa72-27496614002d","order_by":1,"name":"Anzhe Cheng","email":"","orcid":"https://orcid.org/0009-0007-4647-8631","institution":"Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California","correspondingAuthor":false,"prefix":"","firstName":"Anzhe","middleName":"","lastName":"Cheng","suffix":""},{"id":581378209,"identity":"a6b3d2e1-adad-4d35-9c01-8ea3e07e55b9","order_by":2,"name":"Chenzhong Yin","email":"","orcid":"","institution":"University of Southern California","correspondingAuthor":false,"prefix":"","firstName":"Chenzhong","middleName":"","lastName":"Yin","suffix":""},{"id":581378210,"identity":"37ee5ca4-2ec9-4eb9-8121-aa4a552b3309","order_by":3,"name":"Michael Lamba","email":"","orcid":"","institution":"Department of Radiation Oncology, College of Medicine, University of Cincinnati","correspondingAuthor":false,"prefix":"","firstName":"Michael","middleName":"","lastName":"Lamba","suffix":""},{"id":581378211,"identity":"313087bd-1005-4fcb-904f-22d511fd17dc","order_by":4,"name":"Mathieu Sertorio","email":"","orcid":"","institution":"Department of Radiation Oncology, College of Medicine, University of Cincinnati","correspondingAuthor":false,"prefix":"","firstName":"Mathieu","middleName":"","lastName":"Sertorio","suffix":""},{"id":581378212,"identity":"ad0b8ca8-617d-4626-9408-33501503efce","order_by":5,"name":"Jorge Maldonado","email":"","orcid":"https://orcid.org/0000-0003-1960-3541","institution":"Neuroscience Program, University of Illinois at Urbana-Champaign","correspondingAuthor":false,"prefix":"","firstName":"Jorge","middleName":"","lastName":"Maldonado","suffix":""},{"id":581378213,"identity":"f0c2a809-5b3d-49e3-9da5-671ade408b1c","order_by":6,"name":"Alexandre Sathler","email":"","orcid":"https://orcid.org/0000-0002-6800-1208","institution":"National Institute of Neurological Disorders and Stroke, National Institutes of Health","correspondingAuthor":false,"prefix":"","firstName":"Alexandre","middleName":"","lastName":"Sathler","suffix":""},{"id":581378214,"identity":"8e393149-0811-45df-94dc-525ebab150b2","order_by":7,"name":"Yu Chang","email":"","orcid":"","institution":"Department of Statistics, The University of British Columbia","correspondingAuthor":false,"prefix":"","firstName":"Yu","middleName":"","lastName":"Chang","suffix":""},{"id":581378215,"identity":"4b59c31b-0505-40ee-a86c-96447f0e5981","order_by":8,"name":"Catalin Chiritescu","email":"","orcid":"","institution":"Phi Optics, Inc.","correspondingAuthor":false,"prefix":"","firstName":"Catalin","middleName":"","lastName":"Chiritescu","suffix":""},{"id":581378216,"identity":"fd552b41-49d8-4c89-8743-21ccd9dbf930","order_by":9,"name":"Catherine Best-Popescu","email":"","orcid":"https://orcid.org/0000-0003-3857-5953","institution":"Department of Bioengineering, University of Illinois at Urbana-Champaign","correspondingAuthor":false,"prefix":"","firstName":"Catherine","middleName":"","lastName":"Best-Popescu","suffix":""},{"id":581378217,"identity":"b7dd5e55-52cc-4f5d-a195-b05a16111be1","order_by":10,"name":"Dan Ionascu","email":"","orcid":"","institution":"Department of Radiation Oncology, College of Medicine, University of Cincinnati","correspondingAuthor":false,"prefix":"","firstName":"Dan","middleName":"","lastName":"Ionascu","suffix":""},{"id":581378218,"identity":"763c5214-a343-4dcc-914f-5ec23faa85d8","order_by":11,"name":"Nicholas Kotov","email":"","orcid":"https://orcid.org/0000-0002-6864-5804","institution":"University of Michigan-Ann Arbor","correspondingAuthor":false,"prefix":"","firstName":"Nicholas","middleName":"","lastName":"Kotov","suffix":""},{"id":581378219,"identity":"75aebe26-13b2-4b90-973c-c203f851121e","order_by":12,"name":"Shahin Nazarian","email":"","orcid":"","institution":"University of Southern California","correspondingAuthor":false,"prefix":"","firstName":"Shahin","middleName":"","lastName":"Nazarian","suffix":""}],"badges":[],"createdAt":"2025-11-24 22:05:22","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8197080/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8197080/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":101398548,"identity":"68b05ad0-ff71-4633-afd1-b8a1ac4c0f24","added_by":"auto","created_at":"2026-01-29 09:42:13","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3618536,"visible":true,"origin":"","legend":"Article File","description":"","filename":"CellSegmentation.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8197080/v1_covered_df8d32d9-51d3-4342-abc7-5b2971f88ea8.pdf"},{"id":101373116,"identity":"da40e759-9c39-47e5-bb56-f5e13ccd7a7b","added_by":"auto","created_at":"2026-01-29 03:41:45","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":386618,"visible":true,"origin":"","legend":"Appendix","description":"","filename":"Appendix.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8197080/v1/038862d7cb6ac9c73193ccf8.pdf"},{"id":101373117,"identity":"6adca5e9-f52e-49d3-a60b-3d10a27433a7","added_by":"auto","created_at":"2026-01-29 03:41:45","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":286759,"visible":true,"origin":"","legend":"Machine Learning Checklist","description":"","filename":"machinelearningchecklistBogdan.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8197080/v1/3266d1e9c6b1af7a5f29cc96.pdf"},{"id":101373119,"identity":"d3739535-b8ba-4c7d-b61c-7964a0727c60","added_by":"auto","created_at":"2026-01-29 03:41:47","extension":"pdf","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":1665818,"visible":true,"origin":"","legend":"Reporting Summary","description":"","filename":"nrreportingsummaryBogdan.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8197080/v1/2fe304a5a6965ad502f043be.pdf"},{"id":101373118,"identity":"a6c3b76a-e592-4986-aa88-1a9d78995b68","added_by":"auto","created_at":"2026-01-29 03:41:45","extension":"pdf","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":1316971,"visible":true,"origin":"","legend":"Software Policy Checklist","description":"","filename":"nrsoftwarepolicyBogdan.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8197080/v1/c725d0c8ac3668f352186ff5.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"AI-enabled live-dead cell viability classification and motion forecasting","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-8197080/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8197080/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Distinguishing live from dead cells is crucial in a wide variety of research fields, including regenerative medicine, toxicology, pharmacology, and cellular product manufacturing, because it allows researchers to evaluate the efficacy and toxicity of molecules, materials, and therapies and ensures the quality of manufactured cell products. The cost of failure or uncertainty for live-dead analysis can be particularly high for therapeutic compound screening and evaluating medical therapies. Here, we present a novel deep learning framework that integrates a self-attention UNet for segmentation and a transformer network for dynamic tracking of cell movements. Our proposed model achieves state-of-the-art performance, with a high intersection-over-union (IoU) score of 96% and an area-under-curve (AUC) score of 99% for cell segmentation and over 65% IoU of full image cell motion forecasting, highlighting its ability to predict cell dynamics accurately. The self-attention mechanism significantly enhances the model’s ability to differentiate live and dead cells, even in densely packed or morphologically diverse environments. Additionally, the transformer network effectively captures temporal dependencies, enabling precise predictions of future cell movements. This integrated framework demonstrates robust performance across diverse datasets, consistently outperforming existing methods. By offering high-accuracy segmentation and predictive modeling, our approach provides a transformative tool for advancing cellular analysis in research and clinical applications, including cell therapeutics, cancer diagnostics, drug development, and regenerative medicine.","manuscriptTitle":"AI-enabled live-dead cell viability classification and motion forecasting","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-29 03:41:41","doi":"10.21203/rs.3.rs-8197080/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"950e229c-8613-4d70-950e-2f74f0b28dd9","owner":[],"postedDate":"January 29th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":61851884,"name":"Biological sciences/Cell biology"},{"id":61851885,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"}],"tags":[],"updatedAt":"2026-03-18T17:01:30+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-29 03:41:41","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8197080","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8197080","identity":"rs-8197080","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}