Adding layers of information to scRNA-seq data using pre-trained language models

Abstract Pre-trained language models promise to enrich analyses of single-cell data with additional layers of information leveraging large text corpora. Yet, it is still unclear how to achieve optimal alignment with the primary quantitative single-cell data. To address this, we construct text-based training datasets from both scRNA-seq data and biomedical literature targeted to the experimental setting at hand. We then jointly train language models on both information sources to learn a common, literature-enriched representation. Our examples on functionality, disease associations, and temporal trajectories show the potential of knowledge-augmented embeddings as a generalizable and interpretable strategy for enriching single-cell analysis pipelines. Competing Interest Statement The authors have declared no competing interest. Footnotes Streamlining story line and revision of positioning within current literature. Updated Results section incorporating further results on integration of disease and temporal meta-data as well as updated corresponding Methods section.