Scaling Large Language Models for Next-Generation Single-Cell Analysis

Abstract Single-cell RNA sequencing has transformed our understanding of cellular diversity, yet current single-cell foundation models (scFMs) remain limited in their scalability, flexibility across diverse tasks, and ability to natively integrate textual information. In this work, we build upon the Cell2Sentence (C2S) framework, which represents scRNA-seq profiles as textual “cell sentences,” to train Large Language Models (LLMs) on a corpus comprising over one billion tokens of transcriptomic data, biological text, and metadata. Scaling the model to 27 billion parameters yields consistent improvements in predictive and generative capabilities and supports advanced downstream tasks that require synthesis of information across multi-cellular contexts. Targeted fine-tuning with modern reinforcement learning techniques produces strong performance in perturbation response prediction, natural language interpretation, and complex biological reasoning. This predictive strength enabled a dual-context virtual screen that nominated the kinase inhibitor silmitasertib (CX-4945) as a candidate for context-selective upregulation of antigen presentation. Experimental assessment in human cell models unseen during training supported this prediction, demonstrating that C2S-Scale can effectively guide the discovery of context-conditioned biology. C2S-Scale unifies transcriptomic and textual data at unprecedented scales, surpassing both specialized single-cell models and general-purpose LLMs to provide a platform for next-generation single-cell analysis and the development of “virtual cells.” Competing Interest Statement The authors have declared no competing interest. Footnotes ↵† Work partially done during internship at Google Research This version of the manuscript has been revised to improve figure presentation, benchmarking, and experimental validation. The main figures have been reorganized with a consistent layout and color scheme to improve clarity and readability. We expanded the benchmarking analysis to include comparisons with linear models and non foundation model baselines, and added additional detail on computational efficiency by reporting FLOPs usage for C2S-Scale and comparator models. The virtual screening results were strengthened with new experimental data, including validation using a positive control compound, additional immune context negative screening across multiple cell lines, and validation in a primary tumor model system. The corresponding section was revised to reflect these additions.