Classpose: foundation model-driven whole slide image-scale cell phenotyping in H&E

Abstract Cell segmentation and phenotyping in histopathology samples are essential techniques applied across diagnostic and research workflows. However, annotation by human experts requires significant time and domain expertise and is affected by inter-observer variability. While multiple artificial intelligence methods have been developed for this task, their performance is oftentimes sub-par and rarely assessed using external cohorts. Here, we present Classpose, an easily trainable analog of the Cellpose-SAM model for semantic segmentation. We extensively benchmark Classpose against 3 other state-of-the-art methods (Semantic Cellpose-SAM, CellViT++, and Stardist) across 6 datasets (CoNIC, ConSep, GlySAC, MoNuSAC, NuCLS, and PUMA), showing that Classpose consistently outperforms all other methods. We show that training only subsets of parameters is insufficient to reach well-performing models, and that specific additions to the training protocol lead to improved performance. We also show that Classpose outperforms other methods in pairwise cross-dataset evaluation. We provide pre-trained models for cell phenotyping, and provide a command-line tool and a QuPath extension for whole slide image-wide cell phenotyping. Classpose provides accessible, fast and reliable cell phenotyping to the research community. Competing Interest Statement TG is named as a coinventor on patent applications that describe a method for TCR sequencing (GB2305655.9), a method to measure evolutionary dynamics in cancers using DNA methylation (GB2317139.0), and a method to infer drug resistance mechanisms from barcoding data (GB2501439.0). TG has received honorarium from Genentech and consultancy fees from DAiNA therapeutics. TG acknowledges funding from Cancer Research UK (DRCNPG-May21_100001) and the CRUK Convergence Science Centre (CTRQQR-2021\100009).