SPICEiST: Subcellular RNA Pattern Enhances Cell Clustering of Imaging-Based Spatial Transcriptomics

SPICEiST: Subcellular RNA Pattern Enhances Cell Clustering of Imaging-Based Spatial Transcriptomics | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Method Article SPICEiST: Subcellular RNA Pattern Enhances Cell Clustering of Imaging-Based Spatial Transcriptomics Sungwoo Bae, Yuchang Seong, Dongjoo Lee, Hongyoon Choi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7135777/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract Background: Imaging-based spatial transcriptomics (ST) enables the quantification of gene expression at single-cell resolution while preserving spatial context, but its utility is limited by small gene panels and challenges in accurate cell segmentation. To address these limitations, we present a graph autoencoder framework that integrates subcellular transcript distribution patterns with cell-level gene expression profiles for enhanced cell clustering in imaging-based ST (SPICEiST). Results: We systematically evaluated the clustering performance of SPICEiST across several cancer datasets and gene panel sizes. Our results demonstrate that the developed method consistently outperforms the conventional cell-level gene expression-based method in distinguishing subtle cell types, as measured by clustering indices, including CHI and DBI. Notably, SPICEiST reveals more spatially intermixed and less compartmentalized cell clusters, reflecting the complex and heterogeneous nature of tumor microenvironments. The improvement in cell clustering indices over the conventional approach was more pronounced in datasets with small gene panels of around 300 genes, in contrast to those with large panels containing over a thousand genes. Conclusions: These findings highlight the value of leveraging subcellular transcript patterns to overcome the inherent limitations of imaging-based ST, particularly for small gene panels, and may provide new insights into tumor heterogeneity. imaging-based spatial transcriptomics gene panel graph autoencoder subcellular gene expression clustering tumor microenvironment Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Background Spatially resolved transcriptomics (ST) provides a deeper understanding of the spatial context of biological phenomena within the tissue at single-cell resolution [1, 2]. One notable technology, imaging-based ST, captures the locations of predefined RNA types and delineates cell boundaries using immunofluorescence images [1]. This enables the quantification of transcript abundance at the single-cell level while maintaining the spatial location of the cells. Imaging-based ST, such as Xenium, MERSCOPE, and CosMx SMI, achieve high efficiency in capturing transcripts and can obtain subcellular-scale transcript distributions [3–5]. However, the types of genes that can be targeted simultaneously, referred to as a “plex”, typically ranges from a few hundred to a few thousand. This limitation makes imaging-based ST less suitable for unbiased screening of the tissue microenvironment [6]. In most studies, imaging-based ST is analyzed similarly to single-cell RNA sequencing (scRNA-seq) datasets, starting with a cell-by-gene transcript count matrix [4]. However, imaging-based ST typically encompasses a significantly smaller number of genes compared to scRNA-seq, which captures transcriptome-wide expression profiles. Furthermore, in imaging-based ST, transcripts are allocated to cells by analyzing the images of cell membranes or the expansion of nuclear regions, instead of applying cell dissociation methods [4]. This technique can result in inaccuracies when assigning transcripts to cells, mainly because of the substantial overlap between closely situated cells in a 2D space. Consequently, in imaging-based ST, the combination of a limited plex and potential errors in cell segmentation leads to ambiguous distinctions between cell types [7]. Numerous strategies have been explored to address the challenges of limited plex and cell segmentation, aiming to enhance the clustering of cell types in imaging-based ST. Recent innovations in multiplexed imaging technologies have significantly expanded the gene panel in imaging-based ST, with some methods capable of analyzing up to approximately 5,000 genes [8]. Additionally, other approaches utilize computational techniques to impute missing genes, thereby broadening the transcriptomic profile captured by imaging-based ST [9–12]. To address segmentation issues, newly introduced transcript density-based approaches provide more accurate delineation of cell boundaries [7]. Furthermore, one of the methods estimate the distribution of transcripts that are intermixed from adjacent cells and backgrounds, allowing for corrections in the cell-by-gene count matrix [13]. In addition to gene panels and cell segmentation, incorporating the spatial coordinates of each cell alongside gene expression can enhance cell clustering performance, highlighting the importance of the location of a cell within the tissue for accurate characterization [14]. As another approach to improve cell clustering using imaging-based ST, it is essential to examine subcellular transcript distribution patterns along with cell-level gene expression. Subcellular transcript distribution can vary within the same cell type, depending on the state of the cell [15]. Previous studies have leveraged these subcellular patterns and RNA colocalization to finely characterize subcellular domains and their functional implications [16, 17]. Besides, one study enhanced the processes of cell segmentation and annotation by employing a multi-scale topology-based approach [18]. However, to the best of our knowledge, no method exists that uses a graph neural network to enhance clustering of cells in imaging-based ST analysis. Additionally, a systematic evaluation of how subcellular expression patterns can further aid in distinguishing cell types based on panel size and granularity has yet to be conducted. Furthermore, the capabilities of subcellular transcript patterns have not been thoroughly investigated in heterogeneous tumor tissues. The objective of this study is to utilize a graph autoencoder for imaging-based ST of tumor tissues to extract subcellular gene expression patterns and integrate individual cellular gene expression profiles. By leveraging this approach, named SPICEiST ( S ubcellular P attern I ntegration with C ellular E xpression in I maging-based ST ), to extract subcellular patterns, it is expected to overcome the limitations associated with imaging-based ST, thereby facilitating the identification of biologically significant yet subtle variations among cells. Furthermore, this study will quantitatively assess the enhancements in clustering performance within tumors according to the gene plex and the granularity of the cell type clustering. Methods Composition of the publicly available datasets Xenium Human Lung Cancer: Version 1 and Prime 5K The Xenium dataset was derived from formalin-fixed paraffin-embedded (FFPE) tissues of a lung adenocarcinoma patient (https://www.10xgenomics.com/datasets/xenium-human-lung-cancer-post-xenium-technote). The tissue slide underwent analysis using two distinct platforms: version 1 (v1) and Prime 5K. This approach allowed for a direct comparison of the panel sizes on the same tissue slide. The total number of detected transcripts reached 32,073,729 for v1 and 177,464,221 for 5K. The types of transcripts identified, the gene panel size, were 289 and 5,001, respectively. Additionally, the number of cells with a total count exceeding 10 was 268,072 for v1 and 275,556 for 5K. Each slide was split into a 4x4 patches, with each individual patch serving as a separate dataset, and the patch with no assigned transcript in the cell was excluded from the downstream analysis. Xenium Human Colorectal Cancer The Xenium dataset was derived from FFPE tissues of a colon adenocarcinoma patient (https://www.10xgenomics.com/datasets/human-colon-preview-data-xenium-human-colon-gene-expression-panel-1-standard). This dataset encompasses a total of 32,073,729 transcripts across 325 unique transcript types. Additionally, the number of cells with a total count exceeding 10 was 630,998. Each slide was split into a 4x4 patches, with each individual patch serving as a separate dataset, and the patch with no assigned transcript in the cell was excluded from the downstream analysis. CosMx SMI - Lung Adenocarcinoma The CosMx SMI dataset was created from a FFPE tissue slide sourced from primary tumors of a treatment-naive lung adenocarcinoma patients (Lung 5-1) [18, 19]. The tissue slide contains a total of 37,226,610 transcripts, representing 960 unique transcript types. The number of cells with a total count exceeding 10 was 99,181. Each slide was split into a 3x3 patches, with each individual patch serving as a separate dataset, and the patch with no assigned transcript in the cell was removed from the downstream analysis. Dataset Preparation The whole slide of imaging-based ST was segmented into a NxN patches, each containing an average of over 10,000 cells. This enabled a comprehensive comparison of the gene expression-based cell clustering method (gex) and the enhanced approach (integ) in a large number of cases. The cases in which there was no assigned transcript to cells were excluded from the downstream analysis. Additionally, this analysis included a comparison between v1 and the 5K Xenium dataset obtained from the same tissue, which has 289 and 5001 genes in the panel, respectively. The cells that had a total count exceeding 10 were filtered for the subsequent analysis. Building Cell Graphs and Constructing Graph Autoencoders The transcript coordinates for M types of genes (where M denotes the total number of genes in the panel) from the imaging-based ST dataset were utilized to assign each transcript to a distinct 2 mm grid location. Subsequently, based on the staining images of cell nuclei or cell membranes, cells were segmented using the vendor-provided method, and transcripts along with grids were allocated to each cell. The grid count matrix underwent smoothing with a Gaussian kernel for each cell, utilizing a sigma value of 3 mm, followed by L 2 normalization for input to the neural network ( Fig. 1 ). The count matrix of individual cells was normalized to a total sum of 100, log-transformed (log1p), and then scaled with a mean of 0 and a standard deviation of 1 [4]. Principal component analysis was performed to extract the top 64 components that explain the variance of cell-level gene expression the most. For each cell, a graph was defined that connects the four neighboring grids (up, down, right, left) with edges. Node features were assigned in two layers: the first layer represented grid-level subcellular gene expression values, while the second layer represented cell-level gene expression values. This approach effectively transforms the transcripts within the cell into a graph that integrates both subcellular and cell-level gene expression profiles ( Fig. 1 ). The graph autoencoder was developed with a structure that includes an encoder and a decoder. The encoder consists of two convolutional blocks, each utilizing a graph convolutional neural network [20], followed by batch normalization, rectified linear unit (ReLU) activation, and dropout layers set at a probability of 0.2. This process results in a latent feature representation with dimensions of 128 and 64 after each convolutional block. In contrast, the decoder is structured with a fully connected layer, batch normalization, ReLU activation, a dropout layer, and concludes with another fully connected layer as the final output ( Fig. 1 ). Training the Graph Autoencoder The loss function is divided into two components. The first component (L 1 ) calculates the mean squared error (MSE) loss for the reconstruction of subcellular node features that have passed through both the encoder and decoder, compared to the original subcellular features. The second component (L 2 ) defines the MSE loss between the averaged node features across the nodes, with the final feature number of 64, and the PCA-derived cell-level gene expression feature. The weight term, alpha (a), was applied to the second loss component and then combined with the first loss (Total Loss = L 1 +aL 2 ). A higher value of alpha signifies that the cell-level expression profiles exert a greater influence on the training loss ( Fig. 1 ). Conversely, a lower alpha indicates that the subcellular expression profiles have a more pronounced effect on the training loss. The optimization process employs the Adam optimizer, with the learning rate configured at 10 -3 . The datasets were partitioned into training and validation sets at an 80:20 ratio and trained for 50 epochs. During each epoch, training involved randomly selecting 32 batches and computing the average loss. After each epoch, the trained model was utilized to assess validation loss by selecting 32 batches without shuffling, followed by the calculation of the average loss for that epoch. An early stopping strategy was employed, enabling the training process to persist until the validation loss failed to decrease by more than 10 -4 over three consecutive iterations compared to the lowest loss observed. The model with the lowest validation loss was selected. Assessment of Cell Clustering Performance To perform cell clustering using the integrated features of cell and grid-level subcellular expression (referred to as "integ"), we utilized a trained model to calculate a 64D latent representation of the graph structure for each cell ( Fig. 1 ). This latent feature was subsequently utilized to compute the 16-nearest neighbor graphs, after which the Louvain algorithm was applied to cluster the cells into distinct cell groups. The resolution of these clusters was varied from 0.3 to 1.8 (specifically at 0.3, 0.6, 0.9, 1.2, 1.5, and 1.8) to adjust the granularity of the clusters. We repeated this process for the cell-level gene expression features (referred to as "gex"), which were represented by 64 principal components (PCs), and defined the cell clusters in a similar manner. The clustering performance was evaluated using three primary indices: the Average Silhouette Width (ASW), the Calinski-Harabasz index (CHI), and the Davies-Bouldin index (DBI). They assess how closely the cells within the same clusters are grouped together and how well the cells in different clusters are separated in the feature space [21–24]. ASW and CHI are indices where higher values indicate better performance, while DBI is an index where lower values signify superior clustering effectiveness. These indices were computed based on the latent representations derived from the "integ" and "gex" features, utilizing cell cluster labels generated by the Louvain algorithm. This approach serves as a performance measure, employing embeddings to evaluate unsupervised clustering. Furthermore, to assess the spatial coherence of the cell clustering results, we computed the node assortativity coefficient based on a spatial-proximity network considering the 16 nearest neighbors as connected. In this framework, a high coefficient demonstrates strong spatial assortativity, meaning that data points belonging to the same cell clusters are significantly more likely to be spatial neighbors than would be expected by chance [25]. In other words, a lower coefficient indicates less organized cell clusters within the tissue. This provides a single, robust measure of how well the identified clusters are segregated into spatially compact regions. Clustering performance was evaluated across two distinct imaging-based platforms, Xenium and CosMx SMI, with a focus on two types of cancer: lung and colon adenocarcinoma. Initially, the paired Xenium dataset for human lung cancer, derived from the v1 and 5K platforms, was utilized. This dataset comprises 289 and 5001 genes, respectively, and was employed to assess the enhancement of clustering performance through subcellular gene expression patterns, while also comparing the effects based on different panel sizes. Next, datasets from Xenium for human colon cancer and CosMx for human lung cancer were analyzed to evaluate the scalability of the method in improving clustering outcomes. Results A graph autoencoder was constructed that represents each cell as a spatial graph of its subcellular expression patterns. In these graphs, nodes correspond to 2 µm grids and are featured with both local (subcellular-level) and global (cell-level) gene expression data. Using graph convolutional layers, the autoencoder learns a 64D latent embedding for each cell. Training is guided by a dual-objective loss function that reconstructs fine subcellular details while ensuring the latent space reflects the overall cellular expression profile. A hyperparameter, alpha, balances these two objectives ( Fig. 1 ). Performance Evaluation Based on Panel Size and Cell Cluster Granularity in the Xenium Lung Cancer Dataset The clusters were visualized on a UMAP plot, and the clustering performance indices (ASW, CHI, and DBI) were compared between the integrated method (integ) and the gene expression-based method (gex) for a v1 gene panel. This analysis shows the clustering performance according to cluster granularity, as well as the effect of varying alpha weights assigned to loss functions derived from subcellular- to cell-level gene expression. Visualization of the cell clusters in the low-dimensional UMAP plot revealed that the “integ” provided a clearer distinction between the cell clusters than the “gex” did at a cluster resolution of 0.3 ( Fig. 2A, B and Supplementary Fig. S1A, C ). To quantitatively evaluate the clustering performance, we employed several indices that describe the clustering patterns of cells both within and between clusters. Overall, both CHI and DBI demonstrated consistently superior performance in the “integ” model when compared to the “gex” model, irrespective of the cell granularity indicated by the resolution of the Louvain clusters and alpha weight ( Fig. 2C and Supplementary Fig. S2 ). Conversely, ASW exhibited a trend of enhanced clustering performance in the “integ” model relative to the “gex” model. However, this difference was statistically significant at resolutions of 0.3 and 0.6 with alpha weights of 0.25 and 2.00, and up to 0.9 with alpha weights of 0.50 and 1.00. ( Fig. 2C and Supplementary Fig. S2) . The “integ” model exhibited improved performance on standard clustering indices. However, this improvement was accompanied by a compromise in spatial organization ( Fig. 2A, B and Supplementary Fig. S1B, D ). Notably, the assortativity coefficient was significantly lower in the integ model ( Fig. 2C and Supplementary Fig. S2, 3 ). While the “integ” model generated distinct clusters in gene expression, it identified cell types that exhibited a more spatially dispersed and less compartmentalized arrangement compared to those identified by the “gex” model. This finding may more accurately represent the disorganized and heterogeneous nature of the tumor than the “gex” model does. For instance, cell clusters from patch number 11, selected from a total of 15 patches, were mapped to the corresponding tissue and analyzed across both methods ( Fig. 2C and Supplementary Fig. S3 ). The visual examination of the spatial distribution of the clusters revealed that the “integ” model exhibited a globally well-structured pattern. In contrast, it also displayed a more intricate local intermixture of cell clusters compared to the “gex” model, highlighting the complexity of the cell organization ( red-circled regions in Fig. 2C ). This trend of differences was observed across multiple different patches, with the local intermixture of cells being more prominent in the “integ” model ( Supplementary Fig. S1B, D ). Then, the ASW, CHI, and DBI were compared for a 5K gene panel between the “integ” and “gex” models. This analysis shows the clustering performance according to cluster granularity along with varying alpha weights. Both CHI and DBI consistently outperformed the “integ” when compared to the “gex” model, regardless of the alpha weight and cell granularity ( Fig. 3A and Supplementary Fig. S4 ). However, it is important to note that as the cluster resolution in the DBI index increases to 1.5 the DBI for the “integ” model significantly surpasses that of the “gex” model in one of the outliers, irrespective of the alpha values. The outlier dataset is derived from the same sample, patch number 3 (fourth patch among 15 patches), which features a patch with the fewest number of cells, encompassing less than half of the patch area, as depicted in the Supplementary Fig. S5 . This specific configuration may have made the sample more susceptible to noise in a high-resolution setting, resulting in a distinctly different behavior of the DBI when compared to other data points. Meanwhile, ASW displayed a trend of improved clustering performance in the “integ” model relative to the “gex” model at resolutions below 1.0, although this improvement was not statistically significant, except at a resolution of 0.3 with an alpha weight of 0.25, 0.50, or 1.00 ( Fig. 3A and Supplementary Fig. S4 ). Regarding the assortativity coefficient, no significant changes were observed at lower resolutions up to 0.6. However, a significant decrease was noted in the “integ” model when compared to the “gex” model at higher resolutions ( Fig. 3A and Supplementary Fig. S4 ). The cell clusters from patch number 11, the same patch used in the v1 platform for visualization, were mapped to the corresponding tissue and analyzed across “integ” and “gex” methods. The visual assessment of the spatial distribution of the clusters at a resolution of 0.3 revealed that the “integ” model did not show significant differences in the global and local patterns of clusters across multiple patches, including the patch 11 ( Fig. 3B and Supplementary Fig. S5 ). Furthermore, no patch exhibited a higher number of cell clusters in "integ" than in "gex," as compared to 8 out of 15 patches in the v1 gene panel. This suggests that the “integ” model is more useful for describing the intratumoral heterogeneity of cells within tissue, particularly with a small gene panel size. With a large panel size of over a thousand genes, however, the merit of the “integ” model diminishes. To evaluate the effect size of the difference between the “integ” and the “gex” models across two distinct panel sizes (5K and v1 platforms), we analyzed the ratio of the median index for the “integ” model relative to the “gex” model according to the alpha value. Although the results did not reach statistical significance, all clustering indices demonstrated a trend suggesting a higher ratio on the v1 platform compared to the 5K in ASW and CHI and a lower ratio in the DBI ( Fig. 3C ). Additionally, the assortativity coefficient demonstrated a reduced decline in the “integ” model compared to the “gex” model as the panel size increased in the 5K dataset ( Fig. 3C ). Performance Assessment Based on Cell Cluster Granularity in the Xenium Colon Cancer Dataset Given that a smaller alpha value correlates with greater significance in the training loss related to subcellular gene expression, we set the alpha weight to be 0.25 for further analysis. In the Xenium colon cancer dataset, we compared the ASW, CHI, DBI, and assortativity coefficients between the “integ” and “gex” methods, utilizing a panel size of 325. When quantitatively assessing clustering performance, the ASW indicated no significant differences between the “integ” and “gex” models at cluster resolutions up to 1.2 ( Fig. 4A ). However, at higher resolutions, the “integ” model demonstrated a notably lower ASW. In contrast, the CHI increased in the “integ” model, while both the DBI and assortativity coefficients decreased when compared to the “gex” model ( Fig. 4A ). These trends in CHI, DBI, and assortativity coefficients between “integ” and “gex” were consistent with those observed in the human lung cancer Xenium v1 dataset, which featured a similarly sized gene panel. Then, the UMAP visualization was performed at a cluster resolution of 0.6. This resolution revealed significant differences in the CHI, DBI, and assortativity coefficient between the two models ( Fig. 4A ). UMAP plots revealed a clearer separation among clusters in the "integ" model than in the "gex" model across 16 different patches ( Fig. 4B, C and Supplementary Fig. S6A, C ). Furthermore, cells and their cluster identities in the 16 patches were mapped to their respective locations and compared between the two methods ( Fig. 4B, C and Supplementary Fig. 6B, D ). In comparison to the “gex” model, the “integ” model offered a more comprehensive depiction of the tumor microenvironment, showing the intricate intermixture between distinct cell types. Performance Assessment Based on Cell Cluster Granularity in the CosMx SMI Lung Cancer Dataset The performance of the “integ” and “gex” models was evaluated in lung adenocarcinoma tissue using another imaging-based ST platform, specifically the CosMx SMI, which has a panel size of 1,001 genes. The ASW, CHI, DBI, and assortativity coefficients between the integ and gex methods were compared ( Fig. 5A ). Notably, the ASW was significantly higher in the integ model at a resolution of 0.6, but it decreased at higher resolutions of 1.2 and 1.5. In contrast, the CHI showed an increase in the “integ” model, while both the DBI and assortativity coefficients decreased when compared to the “gex” model. The observed patterns in CHI, DBI, and assortativity coefficients exhibited congruence with those described in the human lung cancer Xenium dataset for v1 and 5K panels. Cell clusters from representative patch number 3, chosen from a total of 9 patches, were mapped to the corresponding tissue and compared across “integ” and “gex” models ( Fig. 5B, C ). The cluster resolution of 0.6 was selected on the basis of its demonstration of significantly different values between the two models across all indices. Visual inspection revealed that the “integ” model effectively represented the highly intermixed regions of cell infiltration that the “gex” model failed to capture ( red-circled regions in Fig. 5B, C ). However, many of other patches did not show a clear distinction between the two models, which suggests that the “integ” model may be less beneficial in a dataset with a large panel size ( Supplementary Fig. S7 ). Discussion Imaging-based ST has emerged as a vital platform for capturing single-cell gene expression profiles while preserving spatial context. However, its smaller coverage of genes along with the difficulties in cell segmentation, compared to single-cell RNA sequencing, limits its ability to accurately characterize cell states within tissues [7]. In this study, we employed a graph autoencoder to integrate cell-level gene expression profiles with subcellular transcript patterns at a resolution of 2 mm. This approach aimed to identify an optimal representation of cellular states that effectively elucidates the tumor microenvironment. The integrated model (integ) consistently outperformed the conventional cell-level gene expression-based model (gex) in clustering performance, as indicated by the CHI and DBI. Conversely, the clustering index ASW, which reflects the compactness of cells within the feature space, did not exhibit a consistent trend across the dataset. However, there was a noticeable trend where the median value of the integ model was relatively lower than that of the gex model as the cluster resolution increased. This suggests that as the resolution of the cell clusters rises, subcellular gene expression patterns may lead to over-clustering, resulting in less compact clusters for the “integ” model, while the “gex” model maintains better compactness in the feature space. The spatially clustered patterns, as measured by the assortativity coefficient, consistently demonstrated a trend of lower value in the “integ” model compared to the “gex” model, especially in the small panel size with approximately 300 genes. This observation suggests that the “integ” model may offer a more nuanced representation of the cellular organization of the tissue compared to the “gex” model, which is characterized by a more dispersed arrangement of cells. Therefore, it can be inferred that in the case of a small gene panel size, the “integ” model effectively represents the disorganization of cells, possibly influenced by tumor heterogeneity. The performance of the “integ” model was quantitatively compared based on gene panel size within the same lung cancer tissue. The results indicated that both ASW and CHI demonstrated a greater increase in index ratios compared to “gex” when using a smaller panel. Conversely, DBI exhibited a greater decrease in ratios under the same conditions. These findings suggest that the additive effects of integrating subcellular expression patterns are more pronounced in imaging-based ST when a smaller gene panel is utilized. Furthermore, an examination of the spatial clustering pattern as indicated by the assortativity coefficient suggests that the “integ” model accentuates spatial disorganization to a greater extent in the smaller panel relative to the larger one. This enhancement allows the model to better explain the spatial intermixture of cells and the heterogeneous nature of tumors in the small panel dataset, encompassing approximately 300 genes. There are additional considerations to keep in mind when interpreting the trends in index differences and their implications. The observed variations in trends for feature space-based indices, such as CHI, DBI, and ASW, can be attributed to the unique local patterns of the clusters. For example, non-convex and elongated cluster shapes, along with the presence of outliers, can significantly affect ASW and CHI more than DBI, thereby influencing overall outcomes [26, 27]. Consequently, the trends of relatively lower ASW in the “integ” model at higher resolutions may stem from the elongated distribution patterns of fine clusters within the feature space. Conclusions The limitations of imaging-based ST can be addressed by utilizing subcellular gene expression patterns alongside cell-level gene expression profiles through a graph autoencoder, SPICEiST. This approach proves especially effective in distinguishing subtle cell types within small gene panel-based imaging ST platforms, thereby enhancing the understanding of tumor heterogeneity. List of Abbreviations ASW: Average silhouette width CHI: Calinski-Harabasz index DBI: Davies-Bouldin index Gex: Conventional method which uses cell-level gene expression profiles for cell clustering Integ: Integrated model which uses both subcellular- and cell-level expression profiles for cell clustering, named as SPICEiST ST: Spatial Transcriptomics Declarations Data Availability Imaging-based ST datasets were utilized for the training and evaluation of the SPICIEiST model. Xenium ST datasets for lung adenocarcinoma tissue were acquired from both the v1 and Prime 5K platforms. Both datasets were downloaded from the 10x Genomics dataset repository (https://www.10xgenomics.com/datasets/xenium-human-lung-cancer-post-xenium-technote). Additionally, the Xenium v1 dataset from a colon adenocarcinoma patient was downloaded (https://www.10xgenomics.com/datasets/human-colon-preview-data-xenium-human-colon-gene-expression-panel-1-standard). Furthermore, the CosMx SMI dataset from a lung adenocarcinoma patient, designated Lung 5–1, was retrieved from the NanoString dataset repository (https://nanostring.com/products/cosmx-spatial-molecular-imager/ffpe-dataset/) [19]. Source codes (in Python) for SPICIEiST are accessible at https://github.com/portrai-io/SPICEiST. Acknowledgements We appreciate all the members of the Portrai Inc. for their support of the research. Funding This study was supported by the National Research Foundation of Korea (NRF-2023R1A2C2006636 and NRF-2022M3A9D3016848). Ethics declarations Ethics approval and consent to participate Not Applicable Competing interests S.B., Y.S., and D.L. are currently researchers at Portrai, Inc. H.C. is one of the co-founders of Portrai, Inc. References Rao A, Barkley D, França GS, Yanai I. Exploring tissue architecture using spatial transcriptomics. Nature. 2021;596:211–20. Zhang L, Chen D, Song D, Liu X, Zhang Y, Xu X, et al. Clinical and translational values of spatial transcriptomics. Signal Transduct Target Ther. 2022;7. Qian X, Coleman K, Jiang S, Kriz AJ, Marciano JH, Luo C, et al. Spatial transcriptomics reveals human cortical layer and area specification. Nature. 2025. https://doi.org/10.1038/s41586-025-09010-1. Marco Salas S, Kuemmerle LB, Mattsson-Langseth C, Tismeyer S, Avenel C, Hu T, et al. Optimizing Xenium In Situ data utility by quality assessment and best-practice analysis workflows. Nat Methods. 2025;22:813–23. Pei G, Min J, Rajapakshe KI, Branchi V, Liu Y, Selvanesan BC, et al. Spatial mapping of transcriptomic plasticity in metastatic pancreatic cancer. Nature. 2025;642:212–21. Lim HJ, Wang Y, Buzdin A, Li X. A practical guide for choosing an optimal spatial transcriptomics technology from seven major commercially available options. BMC Genomics. 2025;26. Jones DC, Elz AE, Hadadianpour A, Ryu H, Glass DR, Newell EW. Cell simulation as cell segmentation. Nat Methods. 2025;22:1331–42. Liu Y, Sinjab A, Min J, Han G, Paradiso F, Zhang Y, et al. Conserved spatial subtypes and cellular neighborhoods of cancer-associated fibroblasts revealed by single-cell spatial multi-omics. Cancer Cell. 2025;43:905-924.e6. Lopez R, Nazaret A, Langevin M, Samaran J, Regier J, Jordan MI, et al. A joint model of unpaired data from scRNA-seq and spatial transcriptomics for imputing missing gene expression measurements. 2019. Abdelaal T, Mourragui S, Mahfouz A, Reinders MJT. SpaGE: Spatial Gene Enhancement using scRNA-seq. Nucleic Acids Res. 2020;48:e107–e107. Shengquan C, Boheng Z, Xiaoyang C, Xuegong Z, Rui J. stPlus: a reference-based method for the accurate enhancement of spatial transcriptomics. Bioinformatics. 2021;37 Supplement_1:i299–307. Biancalani T, Scalia G, Buffoni L, Avasthi R, Lu Z, Sanger A, et al. Deep learning and alignment of spatially resolved single-cell transcriptomes with Tangram. Nat Methods. 2021;18:1352–62. Ergen C, Yosef N. ResolVI - addressing noise and bias in spatial transcriptomics. 2025. Hu Y, Xie M, Li Y, Rao M, Shen W, Luo C, et al. Benchmarking clustering, alignment, and integration methods for spatial transcriptomics. Genome Biol. 2024;25. Wang J, Horlacher M, Cheng L, Winther O. RNA trafficking and subcellular localization—a review of mechanisms, experimental and predictive methodologies. Brief Bioinform. 2023;24. Mah CK, Ahmed N, Lopez NA, Lam DC, Pong A, Monell A, et al. Bento: a toolkit for subcellular analysis of spatial transcriptomics data. Genome Biol. 2024;25. Kumar A, Schrader AW, Aggarwal B, Boroojeny AE, Asadian M, Lee J, et al. Intracellular spatial transcriptomic analysis toolkit (InSTAnT). Nat Commun. 2024;15. Benjamin K, Bhandari A, Kepple JD, Qi R, Shang Z, Xing Y, et al. Multiscale topology classifies cells in subcellular spatial transcriptomics. Nature. 2024;630:943–9. He S, Bhatt R, Brown C, Brown EA, Buhr DL, Chantranuvatana K, et al. High-plex imaging of RNA and proteins at subcellular resolution in fixed tissue by spatial molecular imaging. Nat Biotechnol. 2022;40:1794–806. Kipf TN, Welling M. Semi-Supervised Classification with Graph Convolutional Networks. 2017. Wani AA. Comprehensive analysis of clustering algorithms: exploring limitations and innovative solutions. PeerJ Comput Sci. 2024;10:e2286. Rousseeuw PJ. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math. 1987;20:53–65. Calinski T, Harabasz J. A dendrite method for cluster analysis. Commun Stat - Theory Methods. 1974;3:1–27. Davies DL, Bouldin DW. A Cluster Separation Measure. IEEE Trans Pattern Anal Mach Intell. 1979;PAMI-1:224–7. Newman MEJ. Mixing patterns in networks. Phys Rev E. 2003;67. Monshizadeh M, Khatri V, Kantola R, Yan Z. A deep density based and self-determining clustering approach to label unknown traffic. J Netw Comput Appl. 2022;207:103513. Rautenstrauch P, Ohler U. Metrics Matter: Why We Need to Stop Using Silhouette in Single-Cell Benchmarking. 2025. Additional Declarations Competing interest reported. S.B., Y.S., and D.L. are currently researchers at Portrai, Inc. H.C. is one of the co-founders of Portrai, Inc. Supplementary Files SupplementaryFigures.docx Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 20 Aug, 2025 Reviews received at journal 20 Aug, 2025 Reviews received at journal 19 Aug, 2025 Reviewers agreed at journal 11 Aug, 2025 Reviewers agreed at journal 23 Jul, 2025 Reviewers invited by journal 23 Jul, 2025 Editor assigned by journal 22 Jul, 2025 Submission checks completed at journal 18 Jul, 2025 First submitted to journal 16 Jul, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7135777","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Method Article","associatedPublications":[],"authors":[{"id":489655433,"identity":"78cfbb9c-6c00-40de-9ffd-da02291ae361","order_by":0,"name":"Sungwoo Bae","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6klEQVRIiWNgGAWjYDADAwYGxgcQJmMDQwKRWpiBmEGCJC1sEhAtBID8jNzDr3kqbBjM2Q8fqy6ouFdn3n64geFBBR7Db+SlWfOcSWOw7ElLuz3jTLGEzJlEoMPO4NEikWNmnNt2mMHgQI7Zbd62BAkJBqCWxDZ8DoNpOf/+WzFYC/9D/FoYbuQYPwZruZHDxgzWIkHAFoMzb8yY/5xJ47Gc8cxYesaZBMkZEg8bDuDzi3x7jvHHGRU2cub8yQ8/F1Qk8Evwpz98+ANPiDFAo4MHxGKGCR3AqwGo8AOcRUDlKBgFo2AUjFAAAKrNTerTIi2+AAAAAElFTkSuQmCC","orcid":"","institution":"Portrai, Inc","correspondingAuthor":true,"prefix":"","firstName":"Sungwoo","middleName":"","lastName":"Bae","suffix":""},{"id":489655435,"identity":"4dbada23-3426-4556-950c-17d75de060e4","order_by":1,"name":"Yuchang Seong","email":"","orcid":"","institution":"Portrai, Inc","correspondingAuthor":false,"prefix":"","firstName":"Yuchang","middleName":"","lastName":"Seong","suffix":""},{"id":489655437,"identity":"2a0ffec6-ad5d-4a73-a4bc-cecd0f148681","order_by":2,"name":"Dongjoo Lee","email":"","orcid":"","institution":"Portrai, Inc","correspondingAuthor":false,"prefix":"","firstName":"Dongjoo","middleName":"","lastName":"Lee","suffix":""},{"id":489655441,"identity":"bfe13c7f-66e6-41a8-802f-0a08e45b03c9","order_by":3,"name":"Hongyoon Choi","email":"","orcid":"","institution":"Portrai, Inc","correspondingAuthor":false,"prefix":"","firstName":"Hongyoon","middleName":"","lastName":"Choi","suffix":""}],"badges":[],"createdAt":"2025-07-16 04:53:13","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7135777/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7135777/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":87728065,"identity":"58a544c2-5b9c-4d56-9591-4d19c3948bda","added_by":"auto","created_at":"2025-07-28 11:06:50","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":1235888,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSchematic image for SPICEiST. \u003c/strong\u003eTo address the limitations of imaging-based ST, subcellular patterns of gene expression, along with cell-level gene expression, were utilized to extract core features that most accurately represent cellular states. For preprocessing, Gaussian smoothing was applied to the 2 µm subcellular-level count matrix, followed by L\u003csub\u003e2\u003c/sub\u003e normalization. Graphs were constructed for each cell by designating the 2 µm grid as nodes, the 1-hop neighbor grid as connected edges, and the node features as subcellular-level gene expression. A graph autoencoder consists of an encoder, built with graph convolutional blocks, and a decoder for node feature reconstruction. Its training is guided by a weighted, dual-objective loss function. This function simultaneously minimizes the reconstruction error of subcellular features while enforcing similarity between the 64D latent embedding of the model and the cellular expression profile. Finally, cell clustering was performed using the integrated features, applying the Louvain algorithm to identify distinct cell clusters.\u003c/p\u003e","description":"","filename":"Fig.1.tiff.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7135777/v1/4f886d2c1843d93f3b2fa034.jpg"},{"id":87728066,"identity":"6a06f981-5d54-4253-ba82-bf2d2d442b3f","added_by":"auto","created_at":"2025-07-28 11:06:50","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":2068645,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eComparison of the SPICEiST (integ) clustering results with those obtained from cell-level gene expression-based (gex) clustering in the Xenium human lung cancer dataset with v1 panel. \u003c/strong\u003eThe alpha weight for the loss was set to 0.25 during training. UMAP plots (left panel in each subplot) and the spatial distribution of cells (right panel in each subplot) illustrate the cell-level embedding and spatial distribution derived from (A) integ and (B) gex-based clustering, respectively. The four numerical values at the top of the plot represent ASW, CHI, DBI, and assortativity coefficient. The tissue sample was obtained from patch number 11, and the cluster resolution was set to 0.3. Each dot in the plot represents cells, with the color indicating the identity of the clusters. (C) The boxplot compares three distinct clustering performance indices: ASW, CHI, DBI, and the assortativity coefficient, across varying resolutions of cell clusters.\u003c/p\u003e","description":"","filename":"Fig.2.tiff.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7135777/v1/3cafecae4d738867077d527e.jpg"},{"id":87728068,"identity":"19b04008-4de6-49a3-8b12-d8843cbbd30c","added_by":"auto","created_at":"2025-07-28 11:06:50","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":1886861,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eComparison of the SPICEiST (integ) clustering results with those obtained from cell-level gene expression-based (gex) clustering in the Xenium human lung cancer dataset with 5K panel.\u003c/strong\u003e (A) The boxplot compares three distinct clustering performance indices: ASW, CHI, DBI, and the assortativity coefficient across different resolutions of cell clusters. The alpha weight for the loss was set to 0.25 during training. UMAP plots (left panel in each subplot) and the spatial distribution of cells (right panel in each subplot) illustrate the cell-level embedding and spatial distribution derived from (B) integ and (C) gex-based clustering, respectively. The tissue sample was obtained from patch number 11, and the cluster resolution was set to 0.3. The four numerical values at the top of the plot represent ASW, CHI, DBI, and assortativity coefficient. Each dot in the plot represents cells, with the color indicating the identity of the clusters.\u003c/p\u003e","description":"","filename":"Fig.3.tiff.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7135777/v1/117833881cbe4e82e06fed0b.jpg"},{"id":87728070,"identity":"3df1b574-7f99-4739-96c2-d0847ec67fd5","added_by":"auto","created_at":"2025-07-28 11:06:50","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":2382274,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eComparison of the SPICEiST (integ) clustering results with those obtained from cell-level gene expression-based (gex) clustering in the Xenium human colorectal cancer dataset with v1 panel.\u003c/strong\u003e (A) The boxplot compares three distinct clustering performance indices: ASW, CHI, DBI, and the assortativity coefficient across varying resolutions of cell clusters. The alpha weight for the loss was set to 0.25 during training. UMAP plots (left panel in each subplot) and the spatial distribution of cells (right panel in each subplot) illustrate the cell-level embedding and spatial distribution derived from (B) integ and (C) gex-based clustering, respectively. The tissue sample was obtained from patch number 4, and the cluster resolution was set to 0.6. The four numerical values at the top of the plot represent ASW, CHI, DBI, and assortativity coefficient. Each dot in the plot represents cells, with the color indicating the identity of the clusters.\u003c/p\u003e","description":"","filename":"Fig.4.tiff.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7135777/v1/0617b4e02d8fc9b52ddff015.jpg"},{"id":87728913,"identity":"2501130f-667d-49ee-83fa-db24e4124d26","added_by":"auto","created_at":"2025-07-28 11:14:50","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":1936078,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eComparison of the SPICEiST (integ) clustering results with those obtained from cell-level gene expression-based (gex) clustering in the CosMx SMI human lung cancer dataset.\u003c/strong\u003e (A) The boxplot compares three distinct clustering performance indices: ASW, CHI, DBI, and the assortativity coefficient across varying resolutions of cell clusters. The alpha weight for the loss was set to 0.25 during training. UMAP plots (left panel in each subplot) and the spatial distribution of cells (right panel in each subplot) illustrate the cell-level embedding and spatial distribution derived from (B) integ and (C) gex-based clustering, respectively. The tissue sample was obtained from patch number 4, and the cluster resolution was set to 0.6. The four numerical values at the top of the plot represent ASW, CHI, DBI, and assortativity coefficient. Each dot in the plot represents cells, with the color indicating the identity of the clusters.\u003c/p\u003e","description":"","filename":"Fig.5.tiff.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7135777/v1/f32cf0d012e9ec46c98aa1ae.jpg"},{"id":87729433,"identity":"10bd2333-7893-4df1-aa42-029e6a21f8dd","added_by":"auto","created_at":"2025-07-28 11:22:53","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":10711332,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7135777/v1/980457e1-be71-417d-b97d-55dc2c58bd8e.pdf"},{"id":87728071,"identity":"c0d11686-1eef-4b40-963a-b791e3a3f3f0","added_by":"auto","created_at":"2025-07-28 11:06:52","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":67917988,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryFigures.docx","url":"https://assets-eu.researchsquare.com/files/rs-7135777/v1/b7f11388c56f50e40659bfb4.docx"}],"financialInterests":"Competing interest reported. S.B., Y.S., and D.L. are currently researchers at Portrai, Inc. H.C. is one of the co-founders of Portrai, Inc.","formattedTitle":"SPICEiST: Subcellular RNA Pattern Enhances Cell Clustering of Imaging-Based Spatial Transcriptomics","fulltext":[{"header":"Background","content":"\u003cp\u003eSpatially resolved transcriptomics (ST) provides a deeper understanding of the spatial context of biological phenomena within the tissue at single-cell resolution [1, 2]. One notable technology, imaging-based ST, captures the locations of predefined RNA types and delineates cell boundaries using immunofluorescence images [1]. This enables the quantification of transcript abundance at the single-cell level while maintaining the spatial location of the cells. Imaging-based ST, such as Xenium, MERSCOPE, and CosMx SMI, achieve high efficiency in capturing transcripts and can obtain subcellular-scale transcript distributions [3\u0026ndash;5]. However, the types of genes that can be targeted simultaneously, referred to as a \u0026ldquo;plex\u0026rdquo;, typically ranges from a few hundred to a few thousand. This limitation makes imaging-based ST less suitable for unbiased screening of the tissue microenvironment [6].\u003c/p\u003e\n\u003cp\u003eIn most studies, imaging-based ST is analyzed similarly to single-cell RNA sequencing (scRNA-seq) datasets, starting with a cell-by-gene transcript count matrix [4]. However, imaging-based ST typically encompasses a significantly smaller number of genes compared to scRNA-seq, which captures transcriptome-wide expression profiles. Furthermore, in imaging-based ST, transcripts are allocated to cells by analyzing the images of cell membranes or the expansion of nuclear regions, instead of applying cell dissociation methods [4]. This technique can result in inaccuracies when assigning transcripts to cells, mainly because of the substantial overlap between closely situated cells in a 2D space. Consequently, in imaging-based ST, the combination of a limited plex and potential errors in cell segmentation leads to ambiguous distinctions between cell types [7].\u003c/p\u003e\n\u003cp\u003eNumerous strategies have been explored to address the challenges of limited plex and cell segmentation, aiming to enhance the clustering of cell types in imaging-based ST. Recent innovations in multiplexed imaging technologies have significantly expanded the gene panel in imaging-based ST, with some methods capable of analyzing up to approximately 5,000 genes [8]. Additionally, other approaches utilize computational techniques to impute missing genes, thereby broadening the transcriptomic profile captured by imaging-based ST [9\u0026ndash;12]. To address segmentation issues, newly introduced transcript density-based approaches provide more accurate delineation of cell boundaries [7]. Furthermore, one of the methods estimate the distribution of transcripts that are intermixed from adjacent cells and backgrounds, allowing for corrections in the cell-by-gene count matrix [13]. In addition to gene panels and cell segmentation, incorporating the spatial coordinates of each cell alongside gene expression can enhance cell clustering performance, highlighting the importance of the location of a cell within the tissue for accurate characterization [14].\u003c/p\u003e\n\u003cp\u003eAs another approach to improve cell clustering using imaging-based ST, it is essential to examine subcellular transcript distribution patterns along with cell-level gene expression. Subcellular transcript distribution can vary within the same cell type, depending on the state of the cell [15]. Previous studies have leveraged these subcellular patterns and RNA colocalization to finely characterize subcellular domains and their functional implications [16, 17]. Besides, one study enhanced the processes of cell segmentation and annotation by employing a multi-scale topology-based approach [18]. However, to the best of our knowledge, no method exists that uses a graph neural network to enhance clustering of cells in imaging-based ST analysis. Additionally, a systematic evaluation of how subcellular expression patterns can further aid in distinguishing cell types based on panel size and granularity has yet to be conducted. Furthermore, the capabilities of subcellular transcript patterns have not been thoroughly investigated in heterogeneous tumor tissues.\u003c/p\u003e\n\u003cp\u003eThe objective of this study is to utilize a graph autoencoder for imaging-based ST of tumor tissues to extract subcellular gene expression patterns and integrate individual cellular gene expression profiles. By leveraging this approach, named SPICEiST (\u003cstrong\u003eS\u003c/strong\u003eubcellular \u003cstrong\u003eP\u003c/strong\u003eattern \u003cstrong\u003eI\u003c/strong\u003entegration with \u003cstrong\u003eC\u003c/strong\u003eellular \u003cstrong\u003eE\u003c/strong\u003expression in \u003cstrong\u003eI\u003c/strong\u003emaging-based \u003cstrong\u003eST\u003c/strong\u003e), to extract subcellular patterns, it is expected to overcome the limitations associated with imaging-based ST, thereby facilitating the identification of biologically significant yet subtle variations among cells. Furthermore, this study will quantitatively assess the enhancements in clustering performance within tumors according to the gene plex and the granularity of the cell type clustering.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003e\u003cstrong\u003eComposition of the publicly available datasets\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eXenium Human Lung Cancer: Version 1 and Prime 5K\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe Xenium dataset was derived from formalin-fixed paraffin-embedded (FFPE) tissues of a lung adenocarcinoma patient (https://www.10xgenomics.com/datasets/xenium-human-lung-cancer-post-xenium-technote). The tissue slide underwent analysis using two distinct platforms: version 1 (v1) and Prime 5K. This approach allowed for a direct comparison of the panel sizes on the same tissue slide. The total number of detected transcripts reached 32,073,729 for v1 and 177,464,221 for 5K. The types of transcripts identified, the gene panel size, were 289 and 5,001, respectively. Additionally, the number of cells with a total count exceeding 10 was 268,072 for v1 and 275,556 for 5K. Each slide was split into a 4x4 patches, with each individual patch serving as a separate dataset, and the patch with no assigned transcript in the cell was excluded from the downstream analysis.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eXenium Human Colorectal Cancer\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe Xenium dataset was derived from FFPE tissues of a colon adenocarcinoma patient (https://www.10xgenomics.com/datasets/human-colon-preview-data-xenium-human-colon-gene-expression-panel-1-standard). This dataset encompasses a total of 32,073,729 transcripts across 325 unique transcript types. Additionally, the number of cells with a total count exceeding 10 was 630,998. Each slide was split into a 4x4 patches, with each individual patch serving as a separate dataset, and the patch with no assigned transcript in the cell was excluded from the downstream analysis.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eCosMx SMI - Lung Adenocarcinoma\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe CosMx SMI dataset was created from a FFPE tissue slide sourced from primary tumors of a treatment-naive lung adenocarcinoma patients (Lung 5-1) [18, 19]. The tissue slide contains a total of 37,226,610 transcripts, representing 960 unique transcript types. The number of cells with a total count exceeding 10 was 99,181. Each slide was split into a 3x3 patches, with each individual patch serving as a separate dataset, and the patch with no assigned transcript in the cell was removed from the downstream analysis.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDataset Preparation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe whole slide of imaging-based ST was segmented into a NxN patches, each containing an average of over 10,000 cells. This enabled a comprehensive comparison of the gene expression-based cell clustering method (gex) and the enhanced approach (integ) in a large number of cases. The cases in which there was no assigned transcript to cells were excluded from the downstream analysis. Additionally, this analysis included a comparison between v1 and the 5K Xenium dataset obtained from the same tissue, which has 289 and 5001 genes in the panel, respectively. The cells that had a total count exceeding 10 were filtered for the subsequent analysis.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eBuilding Cell Graphs and Constructing Graph Autoencoders\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe transcript coordinates for M types of genes (where M denotes the total number of genes in the panel) from the imaging-based ST dataset were utilized to assign each transcript to a distinct 2\u0026nbsp;mm grid location. Subsequently, based on the staining images of cell nuclei or cell membranes, cells were segmented using the vendor-provided method, and transcripts along with grids were allocated to each cell. The grid count matrix underwent smoothing with a Gaussian kernel for each cell, utilizing a sigma value of 3\u0026nbsp;mm, followed by L\u003csub\u003e2\u003c/sub\u003e normalization for input to the neural network (\u003cstrong\u003eFig. 1\u003c/strong\u003e). The count matrix of individual cells was normalized to a total sum of 100, log-transformed (log1p), and then scaled with a mean of 0 and a standard deviation of 1 [4]. Principal component analysis was performed to extract the top 64 components that explain the variance of cell-level gene expression the most.\u003c/p\u003e\n\u003cp\u003eFor each cell, a graph was defined that connects the four neighboring grids (up, down, right, left) with edges. Node features were assigned in two layers: the first layer represented grid-level subcellular gene expression values, while the second layer represented cell-level gene expression values. This approach effectively transforms the transcripts within the cell into a graph that integrates both subcellular and cell-level gene expression profiles (\u003cstrong\u003eFig. 1\u003c/strong\u003e).\u003c/p\u003e\n\u003cp\u003eThe graph autoencoder was developed with a structure that includes an encoder and a decoder. The encoder consists of two convolutional blocks, each utilizing a graph convolutional neural network [20], followed by batch normalization, rectified linear unit (ReLU) activation, and dropout layers set at a probability of 0.2. This process results in a latent feature representation with dimensions of 128 and 64 after each convolutional block. In contrast, the decoder is structured with a fully connected layer, batch normalization, ReLU activation, a dropout layer, and concludes with another fully connected layer as the final output (\u003cstrong\u003eFig. 1\u003c/strong\u003e).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTraining the Graph Autoencoder\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe loss function is divided into two components. The first component (L\u003csub\u003e1\u003c/sub\u003e) calculates the mean squared error (MSE) loss for the reconstruction of subcellular node features that have passed through both the encoder and decoder, compared to the original subcellular features. The second component (L\u003csub\u003e2\u003c/sub\u003e) defines the MSE loss between the averaged node features across the nodes, with the final feature number of 64, and the PCA-derived cell-level gene expression feature. The weight term, alpha (a), was applied to the second loss component and then combined with the first loss (Total Loss = L\u003csub\u003e1\u003c/sub\u003e+aL\u003csub\u003e2\u003c/sub\u003e). A higher value of alpha signifies that the cell-level expression profiles exert a greater influence on the training loss (\u003cstrong\u003eFig. 1\u003c/strong\u003e). Conversely, a lower alpha indicates that the subcellular expression profiles have a more pronounced effect on the training loss. The optimization process employs the Adam optimizer, with the learning rate configured at 10\u003csup\u003e-3\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eThe datasets were partitioned into training and validation sets at an 80:20 ratio and trained for 50 epochs. During each epoch, training involved randomly selecting 32 batches and computing the average loss. After each epoch, the trained model was utilized to assess validation loss by selecting 32 batches without shuffling, followed by the calculation of the average loss for that epoch. An early stopping strategy was employed, enabling the training process to persist until the validation loss failed to decrease by more than 10\u003csup\u003e-4\u003c/sup\u003e over three consecutive iterations compared to the lowest loss observed. The model with the lowest validation loss was selected.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAssessment of Cell Clustering Performance\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo perform cell clustering using the integrated features of cell and grid-level subcellular expression (referred to as \u0026quot;integ\u0026quot;), we utilized a trained model to calculate a 64D latent representation of the graph structure for each cell (\u003cstrong\u003eFig. 1\u003c/strong\u003e). This latent feature was subsequently utilized to compute the 16-nearest neighbor graphs, after which the Louvain algorithm was applied to cluster the cells into distinct cell groups. The resolution of these clusters was varied from 0.3 to 1.8 (specifically at 0.3, 0.6, 0.9, 1.2, 1.5, and 1.8) to adjust the granularity of the clusters. We repeated this process for the cell-level gene expression features (referred to as \u0026quot;gex\u0026quot;), which were represented by 64 principal components (PCs), and defined the cell clusters in a similar manner.\u003c/p\u003e\n\u003cp\u003eThe clustering performance was evaluated using three primary indices: the Average Silhouette Width (ASW), the Calinski-Harabasz index (CHI), and the Davies-Bouldin index (DBI). They assess how closely the cells within the same clusters are grouped together and how well the cells in different clusters are separated in the feature space [21\u0026ndash;24]. ASW and CHI are indices where higher values indicate better performance, while DBI is an index where lower values signify superior clustering effectiveness. These indices were computed based on the latent representations derived from the \u0026quot;integ\u0026quot; and \u0026quot;gex\u0026quot; features, utilizing cell cluster labels generated by the Louvain algorithm. This approach serves as a performance measure, employing embeddings to evaluate unsupervised clustering.\u003c/p\u003e\n\u003cp\u003eFurthermore, to assess the spatial coherence of the cell clustering results, we computed the node assortativity coefficient based on a spatial-proximity network considering the 16 nearest neighbors as connected. In this framework, a high coefficient demonstrates strong spatial assortativity, meaning that data points belonging to the same cell clusters are significantly more likely to be spatial neighbors than would be expected by chance [25]. In other words, a lower coefficient indicates less organized cell clusters within the tissue. This provides a single, robust measure of how well the identified clusters are segregated into spatially compact regions.\u003c/p\u003e\n\u003cp\u003eClustering performance was evaluated across two distinct imaging-based platforms, Xenium and CosMx SMI, with a focus on two types of cancer: lung and colon adenocarcinoma. Initially, the paired Xenium dataset for human lung cancer, derived from the v1 and 5K platforms, was utilized. This dataset comprises 289 and 5001 genes, respectively, and was employed to assess the enhancement of clustering performance through subcellular gene expression patterns, while also comparing the effects based on different panel sizes. Next, datasets from Xenium for human colon cancer and CosMx for human lung cancer were analyzed to evaluate the scalability of the method in improving clustering outcomes.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eA graph autoencoder was constructed that represents each cell as a spatial graph of its subcellular expression patterns. In these graphs, nodes correspond to 2 µm grids and are featured with both local (subcellular-level) and global (cell-level) gene expression data. Using graph convolutional layers, the autoencoder learns a 64D latent embedding for each cell. Training is guided by a dual-objective loss function that reconstructs fine subcellular details while ensuring the latent space reflects the overall cellular expression profile. A hyperparameter, alpha, balances these two objectives (\u003cstrong\u003eFig. 1\u003c/strong\u003e).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePerformance Evaluation Based on Panel Size and Cell Cluster Granularity in the Xenium Lung Cancer Dataset\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe clusters were visualized on a UMAP plot, and the clustering performance indices (ASW, CHI, and DBI) were compared between the integrated method (integ) and the gene expression-based method (gex) for a v1 gene panel. This analysis shows the clustering performance according to cluster granularity, as well as the effect of varying alpha weights assigned to loss functions derived from subcellular- to cell-level gene expression. Visualization of the cell clusters in the low-dimensional UMAP plot revealed that the “integ” provided a clearer distinction between the cell clusters than the “gex” did at a cluster resolution of 0.3 (\u003cstrong\u003eFig. 2A, B\u003c/strong\u003e \u003cstrong\u003eand\u003c/strong\u003e \u003cstrong\u003eSupplementary Fig. S1A, C\u003c/strong\u003e).\u003c/p\u003e\n\u003cp\u003eTo quantitatively evaluate the clustering performance, we employed several indices that describe the clustering patterns of cells both within and between clusters. Overall, both CHI and DBI demonstrated consistently superior performance in the “integ” model when compared to the “gex” model, irrespective of the cell granularity indicated by the resolution of the Louvain clusters and alpha weight (\u003cstrong\u003eFig. 2C and Supplementary Fig. S2\u003c/strong\u003e). Conversely, ASW exhibited a trend of enhanced clustering performance in the “integ” model relative to the “gex” model. However, this difference was statistically significant at resolutions of 0.3 and 0.6 with alpha weights of 0.25 and 2.00, and up to 0.9 with alpha weights of 0.50 and 1.00. (\u003cstrong\u003eFig. 2C and Supplementary Fig. S2)\u003c/strong\u003e.\u003c/p\u003e\n\u003cp\u003eThe “integ” model exhibited improved performance on standard clustering indices. However, this improvement was accompanied by a compromise in spatial organization (\u003cstrong\u003eFig. 2A, B and Supplementary Fig. S1B, D\u003c/strong\u003e). Notably, the assortativity coefficient was significantly lower in the integ model (\u003cstrong\u003eFig. 2C and Supplementary Fig. S2, 3\u003c/strong\u003e). While the “integ” model generated distinct clusters in gene expression, it identified cell types that exhibited a more spatially dispersed and less compartmentalized arrangement compared to those identified by the “gex” model. This finding may more accurately represent the disorganized and heterogeneous nature of the tumor than the “gex” model does. For instance, cell clusters from patch number 11, selected from a total of 15 patches, were mapped to the corresponding tissue and analyzed across both methods (\u003cstrong\u003eFig. 2C and Supplementary Fig. S3\u003c/strong\u003e). The visual examination of the spatial distribution of the clusters revealed that the “integ” model exhibited a globally well-structured pattern. In contrast, it also displayed a more intricate local intermixture of cell clusters compared to the “gex” model, highlighting the complexity of the cell organization (\u003cstrong\u003ered-circled regions in Fig. 2C\u003c/strong\u003e). This trend of differences was observed across multiple different patches, with the local intermixture of cells being more prominent in the “integ” model (\u003cstrong\u003eSupplementary Fig. S1B, D\u003c/strong\u003e).\u003c/p\u003e\n\u003cp\u003eThen, the ASW, CHI, and DBI were compared for a 5K gene panel between the “integ” and “gex” models. This analysis shows the clustering performance according to cluster granularity along with varying alpha weights. Both CHI and DBI consistently outperformed the “integ” when compared to the “gex” model, regardless of the alpha weight and cell granularity (\u003cstrong\u003eFig. 3A and Supplementary Fig. S4\u003c/strong\u003e). However, it is important to note that as the cluster resolution in the DBI index increases to 1.5 the DBI for the “integ” model significantly surpasses that of the “gex” model in one of the outliers, irrespective of the alpha values. The outlier dataset is derived from the same sample, patch number 3 (fourth patch among 15 patches), which features a patch with the fewest number of cells, encompassing less than half of the patch area, as depicted in the \u003cstrong\u003eSupplementary Fig. S5\u003c/strong\u003e. This specific configuration may have made the sample more susceptible to noise in a high-resolution setting, resulting in a distinctly different behavior of the DBI when compared to other data points. Meanwhile, ASW displayed a trend of improved clustering performance in the “integ” model relative to the “gex” model at resolutions below 1.0, although this improvement was not statistically significant, except at a resolution of 0.3 with an alpha weight of 0.25, 0.50, or 1.00 (\u003cstrong\u003eFig. 3A and Supplementary Fig. S4\u003c/strong\u003e).\u003c/p\u003e\n\u003cp\u003eRegarding the assortativity coefficient, no significant changes were observed at lower resolutions up to 0.6. However, a significant decrease was noted in the “integ” model when compared to the “gex” model at higher resolutions (\u003cstrong\u003eFig. 3A and Supplementary Fig. S4\u003c/strong\u003e). The cell clusters from patch number 11, the same patch used in the v1 platform for visualization, were mapped to the corresponding tissue and analyzed across “integ” and “gex” methods. The visual assessment of the spatial distribution of the clusters at a resolution of 0.3 revealed that the “integ” model did not show significant differences in the global and local patterns of clusters across multiple patches, including the patch 11 (\u003cstrong\u003eFig. 3B and Supplementary Fig. S5\u003c/strong\u003e). Furthermore, no patch exhibited a higher number of cell clusters in \"integ\" than in \"gex,\" as compared to 8 out of 15 patches in the v1 gene panel. This suggests that the “integ” model is more useful for describing the intratumoral heterogeneity of cells within tissue, particularly with a small gene panel size. With a large panel size of over a thousand genes, however, the merit of the “integ” model diminishes.\u003c/p\u003e\n\u003cp\u003eTo evaluate the effect size of the difference between the “integ” and the “gex” models across two distinct panel sizes (5K and v1 platforms), we analyzed the ratio of the median index for the “integ” model relative to the “gex” model according to the alpha value. Although the results did not reach statistical significance, all clustering indices demonstrated a trend suggesting a higher ratio on the v1 platform compared to the 5K in ASW and CHI and a lower ratio in the DBI (\u003cstrong\u003eFig. 3C\u003c/strong\u003e). Additionally, the assortativity coefficient demonstrated a reduced decline in the “integ” model compared to the “gex” model as the panel size increased in the 5K dataset (\u003cstrong\u003eFig. 3C\u003c/strong\u003e).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePerformance Assessment Based on Cell Cluster Granularity in the Xenium Colon Cancer Dataset\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eGiven that a smaller alpha value correlates with greater significance in the training loss related to subcellular gene expression, we set the alpha weight to be 0.25 for further analysis. In the Xenium colon cancer dataset, we compared the ASW, CHI, DBI, and assortativity coefficients between the “integ” and “gex” methods, utilizing a panel size of 325. When quantitatively assessing clustering performance, the ASW indicated no significant differences between the “integ” and “gex” models at cluster resolutions up to 1.2 (\u003cstrong\u003eFig. 4A\u003c/strong\u003e). However, at higher resolutions, the “integ” model demonstrated a notably lower ASW. In contrast, the CHI increased in the “integ” model, while both the DBI and assortativity coefficients decreased when compared to the “gex” model (\u003cstrong\u003eFig. 4A\u003c/strong\u003e). These trends in CHI, DBI, and assortativity coefficients between “integ” and “gex” were consistent with those observed in the human lung cancer Xenium v1 dataset, which featured a similarly sized gene panel.\u003c/p\u003e\n\u003cp\u003eThen, the UMAP visualization was performed at a cluster resolution of 0.6. This resolution revealed significant differences in the CHI, DBI, and assortativity coefficient between the two models (\u003cstrong\u003eFig. 4A\u003c/strong\u003e). UMAP plots revealed a clearer separation among clusters in the \"integ\" model than in the \"gex\" model across 16 different patches (\u003cstrong\u003eFig. 4B, C and Supplementary Fig. S6A, C\u003c/strong\u003e). Furthermore, cells and their cluster identities in the 16 patches were mapped to their respective locations and compared between the two methods (\u003cstrong\u003eFig. 4B, C and Supplementary Fig. 6B, D\u003c/strong\u003e). In comparison to the “gex” model, the “integ” model offered a more comprehensive depiction of the tumor microenvironment, showing the intricate intermixture between distinct cell types.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePerformance Assessment Based on Cell Cluster Granularity in the CosMx SMI Lung Cancer Dataset\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe performance of the “integ” and “gex” models was evaluated in lung adenocarcinoma tissue using another imaging-based ST platform, specifically the CosMx SMI, which has a panel size of 1,001 genes. The ASW, CHI, DBI, and assortativity coefficients between the integ and gex methods were compared (\u003cstrong\u003eFig. 5A\u003c/strong\u003e). Notably, the ASW was significantly higher in the integ model at a resolution of 0.6, but it decreased at higher resolutions of 1.2 and 1.5. In contrast, the CHI showed an increase in the “integ” model, while both the DBI and assortativity coefficients decreased when compared to the “gex” model. The observed patterns in CHI, DBI, and assortativity coefficients exhibited congruence with those described in the human lung cancer Xenium dataset for v1 and 5K panels.\u003c/p\u003e\n\u003cp\u003eCell clusters from representative patch number 3, chosen from a total of 9 patches, were mapped to the corresponding tissue and compared across “integ” and “gex” models (\u003cstrong\u003eFig. 5B, C\u003c/strong\u003e). The cluster resolution of 0.6 was selected on the basis of its demonstration of significantly different values between the two models across all indices. Visual inspection revealed that the “integ” model effectively represented the highly intermixed regions of cell infiltration that the “gex” model failed to capture (\u003cstrong\u003ered-circled regions in Fig. 5B, C\u003c/strong\u003e). However, many of other patches did not show a clear distinction between the two models, which suggests that the “integ” model may be less beneficial in a dataset with a large panel size (\u003cstrong\u003eSupplementary Fig. S7\u003c/strong\u003e).\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eImaging-based ST has emerged as a vital platform for capturing single-cell gene expression profiles while preserving spatial context. However, its smaller coverage of genes along with the difficulties in cell segmentation, compared to single-cell RNA sequencing, limits its ability to accurately characterize cell states within tissues [7].\u003c/p\u003e\n\u003cp\u003eIn this study, we employed a graph autoencoder to integrate cell-level gene expression profiles with subcellular transcript patterns at a resolution of 2\u0026nbsp;mm. This approach aimed to identify an optimal representation of cellular states that effectively elucidates the tumor microenvironment. The integrated model (integ) consistently outperformed the conventional cell-level gene expression-based model (gex) in clustering performance, as indicated by the CHI and DBI. Conversely, the clustering index ASW, which reflects the compactness of cells within the feature space, did not exhibit a consistent trend across the dataset. However, there was a noticeable trend where the median value of the integ model was relatively lower than that of the gex model as the cluster resolution increased. This suggests that as the resolution of the cell clusters rises, subcellular gene expression patterns may lead to over-clustering, resulting in less compact clusters for the \u0026ldquo;integ\u0026rdquo; model, while the \u0026ldquo;gex\u0026rdquo; model maintains better compactness in the feature space.\u003c/p\u003e\n\u003cp\u003eThe spatially clustered patterns, as measured by the assortativity coefficient, consistently demonstrated a trend of lower value in the \u0026ldquo;integ\u0026rdquo; model compared to the \u0026ldquo;gex\u0026rdquo; model, especially in the small panel size with approximately 300 genes. This observation suggests that the \u0026ldquo;integ\u0026rdquo; model may offer a more nuanced representation of the cellular organization of the tissue compared to the \u0026ldquo;gex\u0026rdquo; model, which is characterized by a more dispersed arrangement of cells. Therefore, it can be inferred that in the case of a small gene panel size, the \u0026ldquo;integ\u0026rdquo; model effectively represents the disorganization of cells, possibly influenced by tumor heterogeneity.\u003c/p\u003e\n\u003cp\u003eThe performance of the \u0026ldquo;integ\u0026rdquo; model was quantitatively compared based on gene panel size within the same lung cancer tissue. The results indicated that both ASW and CHI demonstrated a greater increase in index ratios compared to \u0026ldquo;gex\u0026rdquo; when using a smaller panel. Conversely, DBI exhibited a greater decrease in ratios under the same conditions. These findings suggest that the additive effects of integrating subcellular expression patterns are more pronounced in imaging-based ST when a smaller gene panel is utilized. Furthermore, an examination of the spatial clustering pattern as indicated by the assortativity coefficient suggests that the \u0026ldquo;integ\u0026rdquo; model accentuates spatial disorganization to a greater extent in the smaller panel relative to the larger one. This enhancement allows the model to better explain the spatial intermixture of cells and the heterogeneous nature of tumors in the small panel dataset, encompassing approximately 300 genes.\u003c/p\u003e\n\u003cp\u003eThere are additional considerations to keep in mind when interpreting the trends in index differences and their implications. The observed variations in trends for feature space-based indices, such as CHI, DBI, and ASW, can be attributed to the unique local patterns of the clusters. For example, non-convex and elongated cluster shapes, along with the presence of outliers, can significantly affect ASW and CHI more than DBI, thereby influencing overall outcomes [26, 27]. Consequently, the trends of relatively lower ASW in the \u0026ldquo;integ\u0026rdquo; model at higher resolutions may stem from the elongated distribution patterns of fine clusters within the feature space.\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003eThe limitations of imaging-based ST can be addressed by utilizing subcellular gene expression patterns alongside cell-level gene expression profiles through a graph autoencoder, SPICEiST. This approach proves especially effective in distinguishing subtle cell types within small gene panel-based imaging ST platforms, thereby enhancing the understanding of tumor heterogeneity.\u003c/p\u003e"},{"header":"List of Abbreviations","content":"\u003cp\u003e\u003cstrong\u003eASW:\u0026nbsp;\u003c/strong\u003eAverage silhouette width\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCHI:\u003c/strong\u003e Calinski-Harabasz index\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDBI:\u003c/strong\u003e Davies-Bouldin index\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eGex:\u0026nbsp;\u003c/strong\u003eConventional method which uses cell-level gene expression profiles for cell clustering\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eInteg:\u0026nbsp;\u003c/strong\u003eIntegrated model which uses both subcellular- and cell-level expression profiles for cell clustering, named as SPICEiST\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eST:\u0026nbsp;\u003c/strong\u003eSpatial Transcriptomics\u0026nbsp;\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eData Availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eImaging-based ST datasets were utilized for the training and evaluation of the SPICIEiST model. Xenium ST datasets for lung adenocarcinoma tissue were acquired from both the v1 and Prime 5K platforms. Both datasets were downloaded from the 10x Genomics dataset repository (https://www.10xgenomics.com/datasets/xenium-human-lung-cancer-post-xenium-technote). Additionally, the Xenium v1 dataset from a colon adenocarcinoma patient was downloaded (https://www.10xgenomics.com/datasets/human-colon-preview-data-xenium-human-colon-gene-expression-panel-1-standard). Furthermore, the CosMx SMI dataset from a lung adenocarcinoma patient, designated Lung 5\u0026ndash;1, was retrieved from the NanoString dataset repository (https://nanostring.com/products/cosmx-spatial-molecular-imager/ffpe-dataset/) [19]. Source codes (in Python) for SPICIEiST are accessible at https://github.com/portrai-io/SPICEiST.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe appreciate all the members of the Portrai Inc. for their support of the research.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study was supported by the National Research Foundation of Korea (NRF-2023R1A2C2006636 and NRF-2022M3A9D3016848).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics declarations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot Applicable\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eS.B., Y.S., and D.L. are currently researchers at Portrai, Inc. H.C. is one of the co-founders of Portrai, Inc.\u0026nbsp;\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eRao A, Barkley D, Fran\u0026ccedil;a GS, Yanai I. Exploring tissue architecture using spatial transcriptomics. Nature. 2021;596:211\u0026ndash;20.\u003c/li\u003e\n\u003cli\u003eZhang L, Chen D, Song D, Liu X, Zhang Y, Xu X, et al. Clinical and translational values of spatial transcriptomics. Signal Transduct Target Ther. 2022;7.\u003c/li\u003e\n\u003cli\u003eQian X, Coleman K, Jiang S, Kriz AJ, Marciano JH, Luo C, et al. Spatial transcriptomics reveals human cortical layer and area specification. Nature. 2025. https://doi.org/10.1038/s41586-025-09010-1.\u003c/li\u003e\n\u003cli\u003eMarco Salas S, Kuemmerle LB, Mattsson-Langseth C, Tismeyer S, Avenel C, Hu T, et al. Optimizing Xenium In Situ data utility by quality assessment and best-practice analysis workflows. Nat Methods. 2025;22:813\u0026ndash;23.\u003c/li\u003e\n\u003cli\u003ePei G, Min J, Rajapakshe KI, Branchi V, Liu Y, Selvanesan BC, et al. Spatial mapping of transcriptomic plasticity in metastatic pancreatic cancer. Nature. 2025;642:212\u0026ndash;21.\u003c/li\u003e\n\u003cli\u003eLim HJ, Wang Y, Buzdin A, Li X. A practical guide for choosing an optimal spatial transcriptomics technology from seven major commercially available options. BMC Genomics. 2025;26.\u003c/li\u003e\n\u003cli\u003eJones DC, Elz AE, Hadadianpour A, Ryu H, Glass DR, Newell EW. Cell simulation as cell segmentation. Nat Methods. 2025;22:1331\u0026ndash;42.\u003c/li\u003e\n\u003cli\u003eLiu Y, Sinjab A, Min J, Han G, Paradiso F, Zhang Y, et al. Conserved spatial subtypes and cellular neighborhoods of cancer-associated fibroblasts revealed by single-cell spatial multi-omics. Cancer Cell. 2025;43:905-924.e6.\u003c/li\u003e\n\u003cli\u003eLopez R, Nazaret A, Langevin M, Samaran J, Regier J, Jordan MI, et al. A joint model of unpaired data from scRNA-seq and spatial transcriptomics for imputing missing gene expression measurements. 2019.\u003c/li\u003e\n\u003cli\u003eAbdelaal T, Mourragui S, Mahfouz A, Reinders MJT. SpaGE: Spatial Gene Enhancement using scRNA-seq. Nucleic Acids Res. 2020;48:e107\u0026ndash;e107.\u003c/li\u003e\n\u003cli\u003eShengquan C, Boheng Z, Xiaoyang C, Xuegong Z, Rui J. stPlus: a reference-based method for the accurate enhancement of spatial transcriptomics. Bioinformatics. 2021;37 Supplement_1:i299\u0026ndash;307.\u003c/li\u003e\n\u003cli\u003eBiancalani T, Scalia G, Buffoni L, Avasthi R, Lu Z, Sanger A, et al. Deep learning and alignment of spatially resolved single-cell transcriptomes with Tangram. Nat Methods. 2021;18:1352\u0026ndash;62.\u003c/li\u003e\n\u003cli\u003eErgen C, Yosef N. ResolVI - addressing noise and bias in spatial transcriptomics. 2025.\u003c/li\u003e\n\u003cli\u003eHu Y, Xie M, Li Y, Rao M, Shen W, Luo C, et al. Benchmarking clustering, alignment, and integration methods for spatial transcriptomics. Genome Biol. 2024;25.\u003c/li\u003e\n\u003cli\u003eWang J, Horlacher M, Cheng L, Winther O. RNA trafficking and subcellular localization\u0026mdash;a review of mechanisms, experimental and predictive methodologies. Brief Bioinform. 2023;24.\u003c/li\u003e\n\u003cli\u003eMah CK, Ahmed N, Lopez NA, Lam DC, Pong A, Monell A, et al. Bento: a toolkit for subcellular analysis of spatial transcriptomics data. Genome Biol. 2024;25.\u003c/li\u003e\n\u003cli\u003eKumar A, Schrader AW, Aggarwal B, Boroojeny AE, Asadian M, Lee J, et al. Intracellular spatial transcriptomic analysis toolkit (InSTAnT). Nat Commun. 2024;15.\u003c/li\u003e\n\u003cli\u003eBenjamin K, Bhandari A, Kepple JD, Qi R, Shang Z, Xing Y, et al. Multiscale topology classifies cells in subcellular spatial transcriptomics. Nature. 2024;630:943\u0026ndash;9.\u003c/li\u003e\n\u003cli\u003eHe S, Bhatt R, Brown C, Brown EA, Buhr DL, Chantranuvatana K, et al. High-plex imaging of RNA and proteins at subcellular resolution in fixed tissue by spatial molecular imaging. Nat Biotechnol. 2022;40:1794\u0026ndash;806.\u003c/li\u003e\n\u003cli\u003eKipf TN, Welling M. Semi-Supervised Classification with Graph Convolutional Networks. 2017.\u003c/li\u003e\n\u003cli\u003eWani AA. Comprehensive analysis of clustering algorithms: exploring limitations and innovative solutions. PeerJ Comput Sci. 2024;10:e2286.\u003c/li\u003e\n\u003cli\u003eRousseeuw PJ. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math. 1987;20:53\u0026ndash;65.\u003c/li\u003e\n\u003cli\u003eCalinski T, Harabasz J. A dendrite method for cluster analysis. Commun Stat - Theory Methods. 1974;3:1\u0026ndash;27.\u003c/li\u003e\n\u003cli\u003eDavies DL, Bouldin DW. A Cluster Separation Measure. IEEE Trans Pattern Anal Mach Intell. 1979;PAMI-1:224\u0026ndash;7.\u003c/li\u003e\n\u003cli\u003eNewman MEJ. Mixing patterns in networks. Phys Rev E. 2003;67.\u003c/li\u003e\n\u003cli\u003eMonshizadeh M, Khatri V, Kantola R, Yan Z. A deep density based and self-determining clustering approach to label unknown traffic. J Netw Comput Appl. 2022;207:103513.\u003c/li\u003e\n\u003cli\u003eRautenstrauch P, Ohler U. Metrics Matter: Why We Need to Stop Using Silhouette in Single-Cell Benchmarking. 2025.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"genomics-and-informatics","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [Genomics \u0026 Informatics](https://genomicsinform.biomedcentral.com/)","snPcode":"44342","submissionUrl":"https://submission.springernature.com/new-submission/44342/3","title":"Genomics \u0026 Informatics","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"imaging-based spatial transcriptomics, gene panel, graph autoencoder, subcellular gene expression, clustering, tumor microenvironment","lastPublishedDoi":"10.21203/rs.3.rs-7135777/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7135777/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground:\u003c/strong\u003e Imaging-based spatial transcriptomics (ST) enables the quantification of gene expression at single-cell resolution while preserving spatial context, but its utility is limited by small gene panels and challenges in accurate cell segmentation. To address these limitations, we present a graph autoencoder framework that integrates subcellular transcript distribution patterns with cell-level gene expression profiles for enhanced cell clustering in imaging-based ST (SPICEiST).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e We systematically evaluated the clustering performance of SPICEiST across several cancer datasets and gene panel sizes. Our results demonstrate that the developed method consistently outperforms the conventional cell-level gene expression-based method in distinguishing subtle cell types, as measured by clustering indices, including CHI and DBI. Notably, SPICEiST reveals more spatially intermixed and less compartmentalized cell clusters, reflecting the complex and heterogeneous nature of tumor microenvironments. The improvement in cell clustering indices over the conventional approach was more pronounced in datasets with small gene panels of around 300 genes, in contrast to those with large panels containing over a thousand genes.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions:\u003c/strong\u003e These findings highlight the value of leveraging subcellular transcript patterns to overcome the inherent limitations of imaging-based ST, particularly for small gene panels, and may provide new insights into tumor heterogeneity.\u003c/p\u003e","manuscriptTitle":"SPICEiST: Subcellular RNA Pattern Enhances Cell Clustering of Imaging-Based Spatial Transcriptomics","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-07-28 11:06:45","doi":"10.21203/rs.3.rs-7135777/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-08-20T23:39:23+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-20T17:45:22+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-19T12:25:14+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"98811256071119111384186541613647208602","date":"2025-08-11T13:22:55+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"196671223019919541771963101956152441506","date":"2025-07-23T09:30:44+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-07-23T09:20:54+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-07-23T03:38:51+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-07-18T16:35:56+00:00","index":"","fulltext":""},{"type":"submitted","content":"Genomics \u0026 Informatics","date":"2025-07-16T04:39:57+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"genomics-and-informatics","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [Genomics \u0026 Informatics](https://genomicsinform.biomedcentral.com/)","snPcode":"44342","submissionUrl":"https://submission.springernature.com/new-submission/44342/3","title":"Genomics \u0026 Informatics","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"5297500e-c535-47b8-92bb-20abc7f6c456","owner":[],"postedDate":"July 28th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-10-31T10:08:48+00:00","versionOfRecord":[],"versionCreatedAt":"2025-07-28 11:06:45","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7135777","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7135777","identity":"rs-7135777","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}