scMEDAL for the interpretable analysis of single-cell transcriptomics data with batch effect visualization using a deep mixed effects autoencoder

scMEDAL for the interpretable analysis of single-cell transcriptomics data with batch effect visualization using a deep mixed effects autoencoder | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article scMEDAL for the interpretable analysis of single-cell transcriptomics data with batch effect visualization using a deep mixed effects autoencoder Aixa X. Andrade, Son Nguyen, Albert Montillo This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6081478/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted You are reading this latest preprint version Abstract scRNA-seq data has the potential to provide new insights into cellular heterogeneity and data acquisition; however, a major challenge is unraveling confounding from technical and biological batch effects. Existing batch correction algorithms suppress and discard these effects, rather than quantifying and modeling them. Here, we present scMEDAL, a framework for single-cell Mixed Effects Deep Autoencoder Learning, which separately models batch-invariant and batch-specific effects using two complementary autoencoder networks. One network is trained through adversarial learning to capture a batch-invariant representation, while a Bayesian autoencoder learns a batch-specific representation. Comprehensive evaluations spanning conditions (e.g., autism, leukemia, and cardiovascular), cell types, and technical and biological effects demonstrate that scMEDAL suppresses batch effects while modeling batch-specific variation, enhancing accuracy and interpretability. Unlike prior approaches, the framework’s fixed- and random-effects autoencoders enable retrospective analyses, including predicting a cell's expression as if it had been acquired in a different batch via genomap projections at the cellular level, revealing the impact of biological (e.g., diagnosis) and technical (e.g., acquisition) effects. By combining scMEDAL’s batch-agnostic and batch-specific latent spaces, it enables more accurate predictions of disease status, donor group, and cell type, making scMEDAL a valuable framework for gaining deeper insight into data acquisition and cellular heterogeneity. Biological sciences/Computational biology and bioinformatics/Machine learning Biological sciences/Computational biology and bioinformatics/Data integration Biological sciences/Computational biology and bioinformatics/Statistical methods Biological sciences/Computational biology and bioinformatics/Data processing Biological sciences/Computational biology and bioinformatics/Computational models Full Text Additional Declarations There is NO Competing Interest. Ethics Approval This study did not involve the collection of data from human participants, animal subjects, or biological samples. All datasets used in this work were previously published as part of their respective studies. As the study is purely computational, no ethical approval was required." Roma confirmed: "Hi Mark, the author has included links and/or citations to the datasets analyzed in the study; no ethics statement needed. Thanks for following up. Supplementary Files scMEDALsupplementaryinfoSubmitted.pdf Supplementary information machinelearningchecklistfilledoutAAM.pdf Machine Learning Reporting Summary nrsoftwarepolicyfilledoutAAM.pdf Software Policy Checklist nreditorialpolicychecklistfilledoutAAM.pdf Editorial Policy Checklist Cite Share Download PDF Status: Under Review Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6081478","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":425743789,"identity":"9b241762-78c6-473d-a8a3-76e9f8159724","order_by":0,"name":"Aixa X. Andrade","email":"","orcid":"","institution":"University of Texas Southwestern Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Aixa","middleName":"X.","lastName":"Andrade","suffix":""},{"id":425743790,"identity":"bc267379-2872-45f3-a62d-04d90ae88e75","order_by":1,"name":"Son Nguyen","email":"","orcid":"","institution":"University of Texas Southwestern Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Son","middleName":"","lastName":"Nguyen","suffix":""},{"id":425743788,"identity":"f5463eb7-ad8d-45e7-b396-5d8094c13fe7","order_by":2,"name":"Albert Montillo","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABPUlEQVRIie3PMWvCQBQH8BcCyXIma4KgX+FCwC6BfpU7AjoFCl3ayYjgVrIa8ENk7NaEG1wsrhkKjQTapYNuuvWdrcVUpWuH+w93x7v78d4BqKj8wxgm2e88xiWvbqBDX98reT56lDeIrUvC4DYGHXJGwaeCUKzIZ/pZ4o6/yFASQMIzsa9cJlS0ivp+C1HSTgrsEgyQaHVr+8ITe4lXj9CxStYkVugtGETpTMjB+lEmzNon7I2n01D3nhfgu78J6bkxkqwMJRFIoNcmDA+lbrijCY7aJNeCXO2Q8AMZ0AN5Wgpzh2SYnXbRjgn7IRmEhoaE0dO/uHHfkZNT+RcvHRPfm/XFcFqGPg7meOmiapD5Q7GJg4AnDq/X67uga+vmqvoIhG8nxWozmgRda97o8h3nTO3vKxUVFRWVS/kEim6IubKnRj0AAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0002-4353-290X","institution":"University of Texas Southwestern Medical Center","correspondingAuthor":true,"prefix":"","firstName":"Albert","middleName":"","lastName":"Montillo","suffix":""}],"badges":[],"createdAt":"2025-02-21 17:55:20","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6081478/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6081478/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":79132167,"identity":"be45311d-ab94-4e66-b416-32e48106c939","added_by":"auto","created_at":"2025-03-24 19:37:23","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2925588,"visible":true,"origin":"","legend":"Article File","description":"","filename":"scMEDALMainSubmitted.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6081478/v1_covered_a526c27d-c86a-41f0-b222-bd5c1936baae.pdf"},{"id":78819250,"identity":"9d96ac31-ba6a-4063-b19d-7025e50bef52","added_by":"auto","created_at":"2025-03-19 11:12:38","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":1224219,"visible":true,"origin":"","legend":"Supplementary information","description":"","filename":"scMEDALsupplementaryinfoSubmitted.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6081478/v1/6edf8ac9e0f6c334435ae1f5.pdf"},{"id":78819662,"identity":"8b74386b-6a5a-4908-8de5-6ad1f011c117","added_by":"auto","created_at":"2025-03-19 11:20:38","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":200756,"visible":true,"origin":"","legend":"Machine Learning Reporting Summary","description":"","filename":"machinelearningchecklistfilledoutAAM.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6081478/v1/f0fba4efc8001a44f121ef52.pdf"},{"id":78819251,"identity":"c2c1f094-6bc0-4657-9f10-02041d1c08a9","added_by":"auto","created_at":"2025-03-19 11:12:38","extension":"pdf","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":1337459,"visible":true,"origin":"","legend":"Software Policy Checklist","description":"","filename":"nrsoftwarepolicyfilledoutAAM.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6081478/v1/70e28b19feb131b0fbd93f8d.pdf"},{"id":78819249,"identity":"2a3b21de-e11f-4222-a858-0835c632df0a","added_by":"auto","created_at":"2025-03-19 11:12:38","extension":"pdf","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":1697004,"visible":true,"origin":"","legend":"Editorial Policy Checklist","description":"","filename":"nreditorialpolicychecklistfilledoutAAM.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6081478/v1/ddb081d49a15d785f02badac.pdf"}],"financialInterests":"\u003cp\u003eThere is \u003cstrong\u003eNO\u003c/strong\u003e Competing Interest.\u003c/p\u003e\n\u003cp\u003eEthics Approval This study did not involve the collection of data from human participants, animal subjects, or biological samples. All datasets used in this work were previously published as part of their respective studies. As the study is purely computational, no ethical approval was required.\"\u003c/p\u003e\n\u003cp\u003eRoma confirmed: \"Hi Mark, the author has included links and/or citations to the datasets analyzed in the study; no ethics statement needed. Thanks for following up.\u003c/p\u003e","formattedTitle":"scMEDAL for the interpretable analysis of single-cell transcriptomics data with batch effect visualization using a deep mixed effects autoencoder","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-6081478/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6081478/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"scRNA-seq data has the potential to provide new insights into cellular heterogeneity and data acquisition; however, a major challenge is unraveling confounding from technical and biological batch effects. Existing batch correction algorithms suppress and discard these effects, rather than quantifying and modeling them. Here, we present scMEDAL, a framework for single-cell Mixed Effects Deep Autoencoder Learning, which separately models batch-invariant and batch-specific effects using two complementary autoencoder networks. One network is trained through adversarial learning to capture a batch-invariant representation, while a Bayesian autoencoder learns a batch-specific representation. Comprehensive evaluations spanning conditions (e.g., autism, leukemia, and cardiovascular), cell types, and technical and biological effects demonstrate that scMEDAL suppresses batch effects while modeling batch-specific variation, enhancing accuracy and interpretability. Unlike prior approaches, the framework’s fixed- and random-effects autoencoders enable retrospective analyses, including predicting a cell's expression as if it had been acquired in a different batch via genomap projections at the cellular level, revealing the impact of biological (e.g., diagnosis) and technical (e.g., acquisition) effects. By combining scMEDAL’s batch-agnostic and batch-specific latent spaces, it enables more accurate predictions of disease status, donor group, and cell type, making scMEDAL a valuable framework for gaining deeper insight into data acquisition and cellular heterogeneity.","manuscriptTitle":"scMEDAL for the interpretable analysis of single-cell transcriptomics data with batch effect visualization using a deep mixed effects autoencoder","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-03-19 11:12:33","doi":"10.21203/rs.3.rs-6081478/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"7182e624-bc4f-4e76-9571-1db0fba41208","owner":[],"postedDate":"March 19th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":45377995,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"},{"id":45377996,"name":"Biological sciences/Computational biology and bioinformatics/Data integration"},{"id":45377997,"name":"Biological sciences/Computational biology and bioinformatics/Statistical methods"},{"id":45377998,"name":"Biological sciences/Computational biology and bioinformatics/Data processing"},{"id":45377999,"name":"Biological sciences/Computational biology and bioinformatics/Computational models"}],"tags":[],"updatedAt":"2026-04-13T14:50:49+00:00","versionOfRecord":[],"versionCreatedAt":"2025-03-19 11:12:33","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6081478","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6081478","identity":"rs-6081478","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}