{"paper_id":"e64ac00b-de5e-4bd0-a552-9edfb4904e20","body_text":"Defining and benchmarking open problems in single-cell analysis | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Brief Communication Defining and benchmarking open problems in single-cell analysis Malte Luecken, Scott Gigante, Daniel Burkhardt, Robrecht Cannoodt, and 46 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4181617/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 01 Jul, 2025 Read the published version in Nature Biotechnology → Version 1 posted You are reading this latest preprint version Abstract With the growing number of single-cell analysis tools, benchmarks are increasingly important to guide analysis and method development. However, a lack of standardisation and extensibility in current benchmarks limits their usability, longevity, and relevance to the community. We present Open Problems, a living, extensible, community-guided benchmarking platform including 10 current single-cell tasks that we envision will raise standards for the selection, evaluation, and development of methods in single-cell analysis. Biological sciences/Computational biology and bioinformatics/Computational platforms and environments Biological sciences/Computational biology and bioinformatics/Standards Biological sciences/Computational biology and bioinformatics/Machine learning Biological sciences/Computational biology and bioinformatics/Quality control Biological sciences/Computational biology and bioinformatics/Software Full Text Additional Declarations Yes there is potential Competing Interest. M.D.L. contracted for the Chan Zuckerberg Initiative and received speaker fees from Pfizer and Janssen Pharmaceuticals. S.G. has equity interest in Immunai Inc. D.B.B. is a paid employee of and has equity interest in Cellarity Inc. L.Z. has consulted for Lamin Labs GmbH. W.L. contracted for Protein Evolution Incorporated. From 2019 to 2022 A.A. was a consultant for 10X Genomics. From October 2023 E.D. has been a consultant for EnsoCell Therapeutics. O.B.B is currently an employee of Bridge Bio Pharma. J.B. has equity interest in Cellarity, Inc. J.S.R. reports funding from GSK, Pfizer and Sanofi and fees/honoraria from Travere Therapeutics, Stadapharm, Astex, Owkin, Pfizer and Grunenthal. D.W. has equity interest in Immunai Inc. F.J.T. consults for Immunai Inc., Singularity Bio B.V., CytoReason Ltd, Cellarity, and has ownership interest in Dermagnostix GmbH and Cellarity. S.K. is a visiting professor at Meta and scientific advisor at Ascent Bio, Inc. E.d.V.B has ownership interest in Retro Biosciences and ImYoo Inc and is employed by ImYoo Inc. A.T.C. is an employee of Orion Medicines. B.D. is a paid employee of and has equity interest in Cellarity Inc. A.G. is currently an employee of Google DeepMind. Google DeepMind has not directed any aspect of this study nor exerts any commercial rights over the results. R.L. is an employee of Genentech. V.S. has ownership interest in Altos Labs and Vesalius Therapeutics. A.T. has an ownership interest in Dreamfold. Supplementary Files SupplementaryInformation.pdf Supplementary Information: Defining and benchmarking open problems in single-cell analysis Cite Share Download PDF Status: Published Journal Publication published 01 Jul, 2025 Read the published version in Nature Biotechnology → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {\"props\":{\"pageProps\":{\"initialData\":{\"identity\":\"rs-4181617\",\"acceptedTermsAndConditions\":true,\"allowDirectSubmit\":false,\"archivedVersions\":[],\"articleType\":\"Brief Communication\",\"associatedPublications\":[],\"authors\":[{\"id\":287191966,\"identity\":\"ada50bae-0de4-4bed-be55-299ebe009536\",\"order_by\":0,\"name\":\"Malte Luecken\",\"email\":\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABVUlEQVRIie2QMUvDQBiGv1C4LKluchLUX1BICASH4m+54yAuoUVcOkibUEiWlq4tiL/BqbiZEohLxDWDQ4qQOSBIi4J+xQqpinR0uAe+O7iXh/fuACSSf0nNw+UYh+B0ItgBDSD/SpVVCmxTUXw8pWsljXBDhbFtFSXYQmmEszAvgfKRmvCns6vHQ7J3P8v5oguNsUjy5U2zBaqIKoqdct8fozIZOLE1mRYW0VvCYIyAnTmn5jB1zkErqjV2xP2+hsp1pgZ6fRrzQHdtypjWm2aurStBzD3qGlXlYe7331C5zdTwtX4Z99YKxZb2CyrvqLTLqpJhC6xaKElqdS9m5FMxMHIJKtGqBTaUuT8ZGNQap47QtSQ2scUymMPwmYW9PwwED7Ri82IiKhed5sEoTMxn7SI+2tVdMy+bXbDvREGXwQl+psjhO8aPkyrkz1QikUgkv/ABjgqAbVxTxX8AAAAASUVORK5CYII=\",\"orcid\":\"https://orcid.org/0000-0001-7464-7921\",\"institution\":\"Institute of Computational Biology, Helmholtz Munich\",\"correspondingAuthor\":true,\"prefix\":\"\",\"firstName\":\"Malte\",\"middleName\":\"\",\"lastName\":\"Luecken\",\"suffix\":\"\"},{\"id\":287191967,\"identity\":\"d8c563bb-0c7a-4e07-bf62-e6d01c39df50\",\"order_by\":1,\"name\":\"Scott Gigante\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0002-4544-2764\",\"institution\":\"The Walter and Eliza Hall Institute of Medical Research\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Scott\",\"middleName\":\"\",\"lastName\":\"Gigante\",\"suffix\":\"\"},{\"id\":287191968,\"identity\":\"4e36268a-1087-47dd-95c8-15915f987c1e\",\"order_by\":2,\"name\":\"Daniel Burkhardt\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Cellarity, Inc.\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Daniel\",\"middleName\":\"\",\"lastName\":\"Burkhardt\",\"suffix\":\"\"},{\"id\":287191969,\"identity\":\"959f724b-3ff2-44e6-849f-cada848d657d\",\"order_by\":3,\"name\":\"Robrecht Cannoodt\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Data Intuitive\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Robrecht\",\"middleName\":\"\",\"lastName\":\"Cannoodt\",\"suffix\":\"\"},{\"id\":287191970,\"identity\":\"f4d00f3b-c612-473e-a6fe-ec811e4d2017\",\"order_by\":4,\"name\":\"Daniel Strobl\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Institute of Computational Biology, Helmholtz Munich\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Daniel\",\"middleName\":\"\",\"lastName\":\"Strobl\",\"suffix\":\"\"},{\"id\":287191971,\"identity\":\"aca92241-fe8d-4ea3-bebb-beb4b8d16d27\",\"order_by\":5,\"name\":\"Nikolay Markov\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Division of Pulmonary and Critical Care Medicine, Feinberg School of Medicine, Northwestern University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Nikolay\",\"middleName\":\"\",\"lastName\":\"Markov\",\"suffix\":\"\"},{\"id\":287191972,\"identity\":\"e6b5702c-02de-456d-bd81-9d8fc3a2ac64\",\"order_by\":6,\"name\":\"Luke Zappia\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0001-7744-8565\",\"institution\":\"Helmholtz Center Munich\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Luke\",\"middleName\":\"\",\"lastName\":\"Zappia\",\"suffix\":\"\"},{\"id\":287191973,\"identity\":\"8888cc88-4547-4be7-b24e-93d5b64dd1b1\",\"order_by\":7,\"name\":\"Giovanni Palla\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0002-8004-4462\",\"institution\":\"Helmholtz Center Munich\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Giovanni\",\"middleName\":\"\",\"lastName\":\"Palla\",\"suffix\":\"\"},{\"id\":287191974,\"identity\":\"a1f717e1-57ac-4fb4-a35a-10665d014629\",\"order_by\":8,\"name\":\"Wesley Lewis\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Interdepartmental Program in Computational Biology and Bioinformatics, Yale University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Wesley\",\"middleName\":\"\",\"lastName\":\"Lewis\",\"suffix\":\"\"},{\"id\":287191975,\"identity\":\"a2c1a90d-7d94-4c5d-b848-881895ff11ab\",\"order_by\":9,\"name\":\"Daniel Dimitrov\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Heidelberg University, Faculty of Medicine, and Heidelberg University Hospital, Institute for Computational Biomedicine\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Daniel\",\"middleName\":\"\",\"lastName\":\"Dimitrov\",\"suffix\":\"\"},{\"id\":287191976,\"identity\":\"c3180157-1b37-42a1-a28f-e1b8c02ebaef\",\"order_by\":10,\"name\":\"Michael Vinyard\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Department of Chemistry and Chemical Biology, Harvard University, Cambridge, MA, USA\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Michael\",\"middleName\":\"\",\"lastName\":\"Vinyard\",\"suffix\":\"\"},{\"id\":287191977,\"identity\":\"2eb05ce0-f6f0-4141-969f-6e0028a24f75\",\"order_by\":11,\"name\":\"Daniel Magruder\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Department of Computer Science, Yale University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Daniel\",\"middleName\":\"\",\"lastName\":\"Magruder\",\"suffix\":\"\"},{\"id\":287191978,\"identity\":\"4ce13dbc-6c28-43b4-86f0-57ac277c6866\",\"order_by\":12,\"name\":\"Alma Andersson\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Genentech Inc\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Alma\",\"middleName\":\"\",\"lastName\":\"Andersson\",\"suffix\":\"\"},{\"id\":287191979,\"identity\":\"94d1829a-2e76-4093-8663-da736a0212ee\",\"order_by\":13,\"name\":\"Emma Dann\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0002-7400-7438\",\"institution\":\"Wellcome Sanger Institute\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Emma\",\"middleName\":\"\",\"lastName\":\"Dann\",\"suffix\":\"\"},{\"id\":287191980,\"identity\":\"de6b8a9e-801e-4e3b-99da-2a1757cdc1d3\",\"order_by\":14,\"name\":\"Qian Qin\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0002-2119-6263\",\"institution\":\"Broad Institute\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Qian\",\"middleName\":\"\",\"lastName\":\"Qin\",\"suffix\":\"\"},{\"id\":287191981,\"identity\":\"871d7ec5-80f4-4e5c-a1d5-373eed017f14\",\"order_by\":15,\"name\":\"Dominik Otto\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0002-6116-053X\",\"institution\":\"Fred Hutchinson Cancer Center\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Dominik\",\"middleName\":\"\",\"lastName\":\"Otto\",\"suffix\":\"\"},{\"id\":287191982,\"identity\":\"13e43917-733f-4119-b960-5de23ada3cf8\",\"order_by\":16,\"name\":\"Michal Klein\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Apple\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Michal\",\"middleName\":\"\",\"lastName\":\"Klein\",\"suffix\":\"\"},{\"id\":287191983,\"identity\":\"127f2cf1-54e6-46c2-ba24-55cf09fd1842\",\"order_by\":17,\"name\":\"Olga Botvinnik\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Data Sciences Platform, Chan Zuckerberg Biohub\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Olga\",\"middleName\":\"\",\"lastName\":\"Botvinnik\",\"suffix\":\"\"},{\"id\":287191984,\"identity\":\"a6edc834-18ce-4799-9cc0-a7186ff2e29f\",\"order_by\":18,\"name\":\"Louise Deconinck\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Data Mining and Modelling for Biomedicine group, VIB Center for Inflammation Research\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Louise\",\"middleName\":\"\",\"lastName\":\"Deconinck\",\"suffix\":\"\"},{\"id\":287191985,\"identity\":\"19715fad-ee8f-4859-bee0-1a5a80f62e12\",\"order_by\":19,\"name\":\"Kai Waldrant\",\"email\":\"\",\"orcid\":\"https://orcid.org/0009-0003-8555-1361\",\"institution\":\"Data Intuitive\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Kai\",\"middleName\":\"\",\"lastName\":\"Waldrant\",\"suffix\":\"\"},{\"id\":287191986,\"identity\":\"5f989a7a-3769-4f4f-9e43-8ed7d805ec5a\",\"order_by\":20,\"name\":\"Bastian Rieck\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Helmholtz Munich\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Bastian\",\"middleName\":\"\",\"lastName\":\"Rieck\",\"suffix\":\"\"},{\"id\":287191987,\"identity\":\"6936f988-adf2-4bce-b5ca-4b4d2d6c2f56\",\"order_by\":21,\"name\":\"Constantin Ahlmann-Eltze\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Genome Biology Unit, EMBL\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Constantin\",\"middleName\":\"\",\"lastName\":\"Ahlmann-Eltze\",\"suffix\":\"\"},{\"id\":287191988,\"identity\":\"22a53f0b-f811-401b-87cc-3ec402e2656b\",\"order_by\":22,\"name\":\"Eduardo da Veiga Beltrame\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"ImYoo\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Eduardo\",\"middleName\":\"da Veiga\",\"lastName\":\"Beltrame\",\"suffix\":\"\"},{\"id\":287191989,\"identity\":\"ef30e518-03c2-4c7e-8a39-f1b311dd6fbf\",\"order_by\":23,\"name\":\"Andrew Benz\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Cellarity, Inc.\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Andrew\",\"middleName\":\"\",\"lastName\":\"Benz\",\"suffix\":\"\"},{\"id\":287191990,\"identity\":\"7bfd9f92-7d3c-4d3a-a1f3-524bed69d477\",\"order_by\":24,\"name\":\"Carmen Bravo González-Blas\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"VIB-KU Leuven Center for Brain \\u0026 Disease Research\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Carmen\",\"middleName\":\"Bravo\",\"lastName\":\"González-Blas\",\"suffix\":\"\"},{\"id\":287191991,\"identity\":\"3b7120cf-df64-4e7b-bf58-dad24b66cf8f\",\"order_by\":25,\"name\":\"Ann Chen\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Chan Zuckerberg Biohub\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Ann\",\"middleName\":\"\",\"lastName\":\"Chen\",\"suffix\":\"\"},{\"id\":287191992,\"identity\":\"e0e03a69-c1c2-48fe-a0b5-4cf15039313b\",\"order_by\":26,\"name\":\"Benjamin DeMeo\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Cellarity, Inc.\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Benjamin\",\"middleName\":\"\",\"lastName\":\"DeMeo\",\"suffix\":\"\"},{\"id\":287191993,\"identity\":\"84c640fe-5fef-48d9-afd4-c24f0925e843\",\"order_by\":27,\"name\":\"Can Ergen\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"UC Berkeley, EECS\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Can\",\"middleName\":\"\",\"lastName\":\"Ergen\",\"suffix\":\"\"},{\"id\":287191994,\"identity\":\"dfe9151e-8295-44d9-a1d2-88b9c1de9f44\",\"order_by\":28,\"name\":\"Swann Floc'hlay\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"VIB Center for Brain \\u0026 Disease Research\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Swann\",\"middleName\":\"\",\"lastName\":\"Floc'hlay\",\"suffix\":\"\"},{\"id\":287191995,\"identity\":\"852e2410-7cee-40b9-8a09-f836ed4dd57d\",\"order_by\":29,\"name\":\"Adam Gayoso\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0001-9537-0845\",\"institution\":\"University of California, Berkeley\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Adam\",\"middleName\":\"\",\"lastName\":\"Gayoso\",\"suffix\":\"\"},{\"id\":287191996,\"identity\":\"81e6d104-9267-48a6-8eaa-c161db7ba258\",\"order_by\":30,\"name\":\"Stephanie Hicks\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0002-7858-0231\",\"institution\":\"Johns Hopkins University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Stephanie\",\"middleName\":\"\",\"lastName\":\"Hicks\",\"suffix\":\"\"},{\"id\":287191997,\"identity\":\"a8545770-f9b3-4409-a83b-58d4208a5336\",\"order_by\":31,\"name\":\"yuge ji\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Helmholtz Zentrum München\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"yuge\",\"middleName\":\"\",\"lastName\":\"ji\",\"suffix\":\"\"},{\"id\":287191998,\"identity\":\"2133bb42-2f3f-41e4-835f-bf5dfe87985e\",\"order_by\":32,\"name\":\"Vitalii Kleshchevnikov\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Wellcome Sanger Institute\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Vitalii\",\"middleName\":\"\",\"lastName\":\"Kleshchevnikov\",\"suffix\":\"\"},{\"id\":287191999,\"identity\":\"b75e38ba-67c6-4b7a-938b-51f357917cfa\",\"order_by\":33,\"name\":\"Gioele La Manno\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0003-1428-8757\",\"institution\":\"École Polytechnique Fédérale de Lausanne (EPFL)\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Gioele\",\"middleName\":\"La\",\"lastName\":\"Manno\",\"suffix\":\"\"},{\"id\":287192000,\"identity\":\"93f8018c-a043-4481-9518-83cb6079d316\",\"order_by\":34,\"name\":\"Maximilian Lombardo\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Chan Zuckerberg Initiative Foundation\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Maximilian\",\"middleName\":\"\",\"lastName\":\"Lombardo\",\"suffix\":\"\"},{\"id\":287192001,\"identity\":\"400f1ccd-ee79-4a7d-8682-9cf28f86975f\",\"order_by\":35,\"name\":\"Romain Lopez\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Research \\u0026 Early Development, Genentech\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Romain\",\"middleName\":\"\",\"lastName\":\"Lopez\",\"suffix\":\"\"},{\"id\":287192002,\"identity\":\"285788e1-ed0c-4728-a9f2-5c7bc9153957\",\"order_by\":36,\"name\":\"Dario Righelli\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Department of Statistical Sciences, University of Padova\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Dario\",\"middleName\":\"\",\"lastName\":\"Righelli\",\"suffix\":\"\"},{\"id\":287192003,\"identity\":\"ff1c6a2f-e0c8-478b-8b95-e26892f1fadc\",\"order_by\":37,\"name\":\"Hirak Sarkar\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Princeton University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Hirak\",\"middleName\":\"\",\"lastName\":\"Sarkar\",\"suffix\":\"\"},{\"id\":287192004,\"identity\":\"59e9ee18-ee37-407b-a7a7-386029d416e7\",\"order_by\":38,\"name\":\"Valentine Svensson\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0002-9217-2330\",\"institution\":\"FL60 Inc\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Valentine\",\"middleName\":\"\",\"lastName\":\"Svensson\",\"suffix\":\"\"},{\"id\":287192005,\"identity\":\"6fd31bad-ad44-428f-877c-b2a03d2e7389\",\"order_by\":39,\"name\":\"Alexander Tong\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Mila -- Quebec AI Institute\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Alexander\",\"middleName\":\"\",\"lastName\":\"Tong\",\"suffix\":\"\"},{\"id\":287192006,\"identity\":\"4c4b2f7d-038e-42e2-b005-540cd866d0c8\",\"order_by\":40,\"name\":\"Galen Xing\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"University of California, Berkeley\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Galen\",\"middleName\":\"\",\"lastName\":\"Xing\",\"suffix\":\"\"},{\"id\":287192007,\"identity\":\"ec6aaecd-17be-4981-898e-8f4661524858\",\"order_by\":41,\"name\":\"Chenling Xu\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Triplebar\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Chenling\",\"middleName\":\"\",\"lastName\":\"Xu\",\"suffix\":\"\"},{\"id\":287192008,\"identity\":\"33bd120c-46de-4871-9964-1412c76753c8\",\"order_by\":42,\"name\":\"Jonathan Bloom\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Massachusetts Institute of Technology\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Jonathan\",\"middleName\":\"\",\"lastName\":\"Bloom\",\"suffix\":\"\"},{\"id\":287192009,\"identity\":\"c88f0cf3-f78d-403a-b608-7e1b7cca942c\",\"order_by\":43,\"name\":\"Angela Pisco\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Chan Zuckerberg Biohub\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Angela\",\"middleName\":\"\",\"lastName\":\"Pisco\",\"suffix\":\"\"},{\"id\":287192010,\"identity\":\"ad698b85-1e0a-48ab-b097-4b80de022e9d\",\"order_by\":44,\"name\":\"Julio Saez-Rodriguez\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Heidelberg University, Faculty of Medicine, and Heidelberg University Hospital, Institute for Computational Biomedicine\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Julio\",\"middleName\":\"\",\"lastName\":\"Saez-Rodriguez\",\"suffix\":\"\"},{\"id\":287192011,\"identity\":\"07c896b4-74bd-41fd-b91a-45ebdd290dd9\",\"order_by\":45,\"name\":\"Drausin Wulsin\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Immunai\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Drausin\",\"middleName\":\"\",\"lastName\":\"Wulsin\",\"suffix\":\"\"},{\"id\":287192012,\"identity\":\"abaf2583-8929-4399-8d4b-373361b35cb4\",\"order_by\":46,\"name\":\"Luca Pinello\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0003-1109-3823\",\"institution\":\"Massachusetts General Hospital, Harvard Medical School, Broad Institute of Harvard and MIT\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Luca\",\"middleName\":\"\",\"lastName\":\"Pinello\",\"suffix\":\"\"},{\"id\":287192013,\"identity\":\"d1b5dd2e-6e70-4dd7-9d5d-f1562c7282d0\",\"order_by\":47,\"name\":\"Yvan Saeys\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0002-0415-1506\",\"institution\":\"Ghent University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Yvan\",\"middleName\":\"\",\"lastName\":\"Saeys\",\"suffix\":\"\"},{\"id\":287192014,\"identity\":\"0dab3018-373d-4ddc-a87a-04b0b2c816dc\",\"order_by\":48,\"name\":\"Fabian Theis\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0002-2419-1943\",\"institution\":\"Helmholtz Centre Munich GmbH\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Fabian\",\"middleName\":\"\",\"lastName\":\"Theis\",\"suffix\":\"\"},{\"id\":287192015,\"identity\":\"6ba7343a-72d5-43cd-91a6-1f30e3f056b9\",\"order_by\":49,\"name\":\"Smita Krishnaswamy\",\"email\":\"\",\"orcid\":\"https://orcid.org/0000-0001-5823-1985\",\"institution\":\"Yale University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Smita\",\"middleName\":\"\",\"lastName\":\"Krishnaswamy\",\"suffix\":\"\"}],\"badges\":[],\"createdAt\":\"2024-03-28 10:46:59\",\"currentVersionCode\":1,\"declarations\":\"\",\"doi\":\"10.21203/rs.3.rs-4181617/v1\",\"doiUrl\":\"https://doi.org/10.21203/rs.3.rs-4181617/v1\",\"draftVersion\":[],\"editorialEvents\":[{\"content\":\"https://doi.org/10.1038/s41587-025-02694-w\",\"type\":\"published\",\"date\":\"2025-07-01T04:00:00+00:00\"}],\"editorialNote\":\"\",\"failedWorkflow\":false,\"files\":[{\"id\":85826036,\"identity\":\"be0f2877-fc31-44bd-8aa0-93d85c8b7c9b\",\"added_by\":\"auto\",\"created_at\":\"2025-07-02 07:11:38\",\"extension\":\"pdf\",\"order_by\":1,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"manuscript-pdf\",\"size\":811765,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"OpenProblemsbriefcommunicationsmanuscriptsubmissionversion.pdf\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-4181617/v1_covered_6190fe14-e668-4270-ac2b-f65129e50725.pdf\"},{\"id\":54061289,\"identity\":\"d596f306-2856-4f26-8972-8a31dc3f46db\",\"added_by\":\"auto\",\"created_at\":\"2024-04-04 04:07:25\",\"extension\":\"pdf\",\"order_by\":1,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"supplement\",\"size\":4717481,\"visible\":true,\"origin\":\"\",\"legend\":\"\\u003cp\\u003eSupplementary Information: Defining and benchmarking open problems in single-cell analysis\\u003c/p\\u003e\",\"description\":\"\",\"filename\":\"SupplementaryInformation.pdf\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-4181617/v1/c2c98a9c0c08ff1b225c53df.pdf\"}],\"financialInterests\":\"\\u003cb\\u003eYes\\u003c/b\\u003e there is potential Competing Interest.\\nM.D.L. contracted for the Chan Zuckerberg Initiative and received speaker fees from Pfizer and Janssen Pharmaceuticals. S.G. has equity interest in Immunai Inc. D.B.B. is a paid employee of and has equity interest in Cellarity Inc. L.Z. has consulted for Lamin Labs GmbH. W.L. contracted for Protein Evolution Incorporated. From 2019 to 2022 A.A. was a consultant for 10X Genomics. From October 2023 E.D. has been a consultant for EnsoCell Therapeutics. O.B.B is currently an employee of Bridge Bio Pharma. J.B. has equity interest in Cellarity, Inc. J.S.R. reports funding from GSK, Pfizer and Sanofi and fees/honoraria from Travere Therapeutics, Stadapharm, Astex, Owkin, Pfizer and Grunenthal. D.W. has equity interest in Immunai Inc. F.J.T. consults for Immunai Inc., Singularity Bio B.V., CytoReason Ltd, Cellarity, and has ownership interest in Dermagnostix GmbH and Cellarity. S.K. is a visiting professor at Meta and scientific advisor at Ascent Bio, Inc. E.d.V.B has ownership interest in Retro Biosciences and ImYoo Inc and is employed by ImYoo Inc. A.T.C. is an employee of Orion Medicines. B.D. is a paid employee of and has equity interest in Cellarity Inc. A.G. is currently an employee of Google DeepMind. Google DeepMind has not directed any aspect of this study nor exerts any commercial rights over the results. R.L. is an employee of Genentech. V.S. has ownership interest in Altos Labs and Vesalius Therapeutics. A.T. has an ownership interest in Dreamfold.\",\"formattedTitle\":\"Defining and benchmarking open problems in single-cell analysis\",\"fulltext\":[],\"fulltextSource\":\"\",\"fullText\":\"\",\"funders\":[],\"hasAdminPriorityOnWorkflow\":false,\"hasManuscriptDocX\":false,\"hasOptedInToPreprint\":true,\"hasPassedJournalQc\":\"\",\"hasAnyPriority\":true,\"hideJournal\":false,\"highlight\":\"\",\"institution\":\"\",\"isAcceptedByJournal\":true,\"isAuthorSuppliedPdf\":true,\"isDeskRejected\":\"\",\"isHiddenFromSearch\":false,\"isInQc\":false,\"isInWorkflow\":false,\"isPdf\":true,\"isPdfUpToDate\":true,\"isWithdrawnOrRetracted\":false,\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"nature-portfolio\",\"isNatureJournal\":true,\"hasQc\":false,\"allowDirectSubmit\":false,\"externalIdentity\":\"\",\"sideBox\":\"\",\"snPcode\":\"\",\"submissionUrl\":\"\",\"title\":\"Nature Portfolio\",\"twitterHandle\":\"\",\"acdcEnabled\":false,\"dfaEnabled\":false,\"editorialSystem\":\"ejp\",\"reportingPortfolio\":\"\",\"inReviewEnabled\":true,\"inReviewRevisionsEnabled\":false},\"keywords\":\"\",\"lastPublishedDoi\":\"10.21203/rs.3.rs-4181617/v1\",\"lastPublishedDoiUrl\":\"https://doi.org/10.21203/rs.3.rs-4181617/v1\",\"license\":{\"name\":\"CC BY 4.0\",\"url\":\"https://creativecommons.org/licenses/by/4.0/\"},\"manuscriptAbstract\":\"With the growing number of single-cell analysis tools, benchmarks are increasingly important to guide analysis and method development. However, a lack of standardisation and extensibility in current benchmarks limits their usability, longevity, and relevance to the community. We present Open Problems, a living, extensible, community-guided benchmarking platform including 10 current single-cell tasks that we envision will raise standards for the selection, evaluation, and development of methods in single-cell analysis.\",\"manuscriptTitle\":\"Defining and benchmarking open problems in single-cell analysis\",\"msid\":\"\",\"msnumber\":\"\",\"nonDraftVersions\":[{\"code\":1,\"date\":\"2024-04-04 04:07:20\",\"doi\":\"10.21203/rs.3.rs-4181617/v1\",\"editorialEvents\":[],\"status\":\"published\",\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"nature-biotechnology\",\"isNatureJournal\":true,\"hasQc\":false,\"allowDirectSubmit\":false,\"externalIdentity\":\"nbt\",\"sideBox\":\"Learn more about [Nature Biotechnology](http://www.nature.com/nbt/)\",\"snPcode\":\"\",\"submissionUrl\":\"\",\"title\":\"Nature Biotechnology\",\"twitterHandle\":\"\",\"acdcEnabled\":true,\"dfaEnabled\":true,\"editorialSystem\":\"ejp\",\"reportingPortfolio\":\"Nature Research\",\"inReviewEnabled\":true,\"inReviewRevisionsEnabled\":false}}],\"origin\":\"\",\"ownerIdentity\":\"0d551ad5-926b-425e-94df-76b7f3e9348f\",\"owner\":[],\"postedDate\":\"April 4th, 2024\",\"published\":true,\"recentEditorialEvents\":[],\"rejectedJournal\":[],\"revision\":\"\",\"amendment\":\"\",\"status\":\"published-in-journal\",\"subjectAreas\":[{\"id\":30231358,\"name\":\"Biological sciences/Computational biology and bioinformatics/Computational platforms and environments\"},{\"id\":30231359,\"name\":\"Biological sciences/Computational biology and bioinformatics/Standards\"},{\"id\":30231360,\"name\":\"Biological sciences/Computational biology and bioinformatics/Machine learning\"},{\"id\":30231361,\"name\":\"Biological sciences/Computational biology and bioinformatics/Quality control\"},{\"id\":30231362,\"name\":\"Biological sciences/Computational biology and bioinformatics/Software\"}],\"tags\":[],\"updatedAt\":\"2025-07-02T07:11:30+00:00\",\"versionOfRecord\":{\"articleIdentity\":\"rs-4181617\",\"link\":\"https://doi.org/10.1038/s41587-025-02694-w\",\"journal\":{\"identity\":\"nature-biotechnology\",\"isVorOnly\":false,\"title\":\"Nature Biotechnology\"},\"publishedOn\":\"2025-07-01 04:00:00\",\"publishedOnDateReadable\":\"July 1st, 2025\"},\"versionCreatedAt\":\"2024-04-04 04:07:20\",\"video\":\"\",\"vorDoi\":\"10.1038/s41587-025-02694-w\",\"vorDoiUrl\":\"https://doi.org/10.1038/s41587-025-02694-w\",\"workflowStages\":[]},\"version\":\"v1\",\"identity\":\"rs-4181617\",\"journalConfig\":\"researchsquare\"},\"__N_SSP\":true},\"page\":\"/article/[identity]/[[...version]]\",\"query\":{\"redirect\":\"/article/rs-4181617\",\"identity\":\"rs-4181617\",\"version\":[\"v1\"]},\"buildId\":\"qtupq5eGEP_6zYnWcrvyt\",\"isFallback\":false,\"isExperimentalCompile\":false,\"dynamicIds\":[84888],\"gssp\":true,\"scriptLoader\":[]}","source_license":"CC-BY-4.0","license_restricted":false}