Accurate cell type deconvolution in spatial transcriptomics using a batch effect-free strategy

ABSTRACT Sequencing-based spatial transcriptomics (ST) techniques have been groundbreaking in dissecting cell-cell communications within tissues by profiling positional gene expression. However, the most widely used ST technique, Visium Spatial Gene Expression by 10x Genomics (Visium), does not provide single-cell resolution, making it difficult to profile cell type-level information. Many reference-based deconvolution methods have been developed to increase its resolution, but the platform and batch effects between the reference and ST data compromise their accuracy. Here, we propose a new approach, Re gion-based cell Sort ing (ReSort), that generates a pseudo-internal-reference to reduce these platform effects. By simulating ST datasets under various scenarios, we demonstrate that ReSort significantly improves the accuracy of six state-of-the-art reference-based deconvolution methods. Moreover, applying ReSort to a mouse breast cancer tumor bearing both epithelial and mesenchymal clones identifies the spatial differences of immune cells between the clones, providing important insights for understanding the relationship between epithelial-mesenchymal transition and immune infiltration in breast cancer.