GeneBag: training a cell foundation model for broad-spectrum cancer diagnosis and prognosis with bulk RNA-seq data

GeneBag: training a cell foundation model for broad-spectrum cancer diagnosis and prognosis with bulk RNA-seq data | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article GeneBag: training a cell foundation model for broad-spectrum cancer diagnosis and prognosis with bulk RNA-seq data Kun Tang, Yuhu Liang, Dan Li, Dong Luo, Augix Xu, Pengchao Luo, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5720342/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Numerous Pre-trained cell foundation models (CFM) have been developed to encapsulate the comprehensive gene-gene interaction network within cells, leveraging extensive single-cell sequencing data. These models have shown promise in various cell biology applications, including cell type annotation, perturbation inference, and cell state embedding, etc. However, their clinical utility, particularly in cancer diagnosis and prognosis, remains an open question. We introduce the GeneBag model, a novel CFM that represents a cell as “a bag of unordered genes” with continuous expression values and a full-length gene list. Pre-trained on single-cell data and fine-tuned on bulk RNA-seq datasets, GeneBag achieves superior performance across cancer diagnosis and prognosis scenarios. In a zero-shot learning setting, GeneBag can classify cancer and non-cancer tissues with approximately 96.2% accuracy. With fine-tuning, it can annotate 40 different types of cancers and corresponding normal biopsies with an overall accuracy of ~ 97.2%. It notably excels in classifying challenging cancers such as bladder (93%) and stomach (90%). Furthermore, GeneBag is capable of cancer staging with 68.5% accuracy and 1 to 5 year survival prediction with an AUC of 76.98% − 82.81%. This study marks the first to demonstrate the potential of CFMs in RNA-based cancer diagnostics and prognostics, indicating a promising avenue for AI-assisted molecular diagnosis. Biological sciences/Computational biology and bioinformatics/Computational models Biological sciences/Cancer/Cancer models Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Introduction Traditionally, systems biology has aimed to fully understand the entirety of multi-omics data and to map the intricate relationships between countless biomolecules. The aspiration is that a complete analysis of biological systems will facilitate an in-depth understanding of the underlying mechanisms, thereby significantly advancing disease diagnostics and the innovation of pharmaceuticals and therapeutics (G. Li et al. 2022 ; X. Wang et al. 2021 ; Kitano 2002 ). However, efforts in large-scale systems biology research were significantly impeded by the constraints of existing computational algorithms and analytical techniques, which struggle to keep pace with the intricate complexity and vast scales inherent in living organisms. Consequently, the findings from these studies often remain descriptive, and their models usually differ significantly from in vivo conditions, limiting their practical applicability in real-world scenarios (Gomez-Ramirez and Sanz 2013; Bartocci and Lió 2016; Atta and Fan 2021). Recent artificial intelligence research has seen remarkable progress, particularly with the advent of foundation models based on transformer architecture. Models like BERT, GPT employ self-attention to weigh and distill the complex interplays among thousands of words within the sentence contexts (Floridi and Chiriatti 2020; Devlin et al. 2018 ). This development is highly inspirational systems biology, where biomolecules, akin to words, engage in intricate communications. The potential to learn comprehensive molecular interactome within a foundation model framework suggests that downstream biological inferences could be conducted, analogous to the fine-tuning or prompting techniques employed in large language models. Indeed, a multitude of cell foundation models (CFMs) have recently emerged, trained on extensive single-cell RNA sequencing (scRNA-seq) datasets. Models such as scBERT (F. Yang et al. 2022 ), Geneformer (Theodoris et al. 2023 ), GeneCompass (X. Yang et al. 2023 ), and scFoundation (Hao et al. 2023 ) are primarily based on the BERT-like bidirectional transformer architecture, while scGPT adopts a GPT-like generative transformer approach (Cui et al. 2024 ). These CFMs have demonstrated a variety of applications in downstream tasks, including cell type annotation, single-cell perturbation inference, target gene prediction, and drug response forecasting, showcasing their versatility and potential in advancing single-cell biology research. However, the potential CFMs in clinical scenarios, particularly in cancer diagnosis and prognosis, has not yet been fully explored. Theoretically CFMs could apply the intergenic interaction patterns learned from one data modality, such as scRNA-seq, to identify diseases across various other data modalities, such as bulk RNA sequencing data (bulk RNA-seq), using transfer learning. Traditional bulk RNA-seq has been a cornerstone in cancer research, with different RNA components showing distinct signatures associated with cancer development (H. Wang et al. 2022 ). Protein-coding mRNAs have shown aberrant expression profiles across various cancers, prompting the proposal of mRNA expression profiling panels for diagnostic purposes (Bareche et al. 2018 ; Golub et al. 1999 ; Ma et al. 2020 ; Teresa Agulló-Ortuño, López-Ríos, and Paz-Ares 2010). Circular RNAs (circRNAs), noted for their differential expression in different cancer types (Shang et al. 2019 ; Feng et al. 2022 ; Xia et al. 2018 ), are considered promising biomarkers (J. Li et al. 2020 ; Meng et al. 2017 ; S. Wang et al. 2021 ; Wen, Zhou, and Gu 2020; H. Zhang et al. 2017 ). Other RNA species, such as long noncoding RNAs (lncRNAs) and micro-RNAs (miRNAs), have also been implicated in signaling specific cancers (Shen 2020 ; Hua, Chen, and He 2019; Loewen et al. 2014 ; Y. Wang et al. 2015 ; Xu et al. 2020 ; Zhan et al. 2020 ; Giulietti et al. 2018 ; Wu et al. 2021 ; Iorio and Croce 2012; Lu et al. 2005 ; Valihrach, Androvic, and Kubista 2020; Verduci et al. 2019 ). However, the traditional focus has been on identifying a limited set of biomarkers representing only a fraction of cancer subtypes. These marker-gene-based approaches do not enable a comprehensive assessment of the entire spectrum of cancers. The heterogeneous nature of oncogenesis further complicates the representation of all cancer subtypes by biomarker panels. Recently, efforts have also been paid to employ machine learning or deep learning techniques to analyze bulk RNA data, in order to improve tumor classification and prognosis (L. Zhang et al. 2018; Bostanci et al. 2023 ). However, these applications are currently confined to specific cancer types and rely on end-to-end training with limited datasets. In this study, we explore the utilization of CFM for the first time in the context of cancer diagnostics and prognostics, leveraging bulk RNA-seq data. We constructed a novel foundation model, the GeneBag, based on transformer encoder architecture, capable of concurrently processing all genes with an assumption of random gene order and continuous expression values. Initially trained on an extensive dataset of single-cell transcriptomics, the model was subsequently retrained using bulk RNA-seq data. We then evaluated its performance across a series of downstream tasks of cancer diagnosis and prognosis. Results In general, cell foundation models aim to capitalize on transformer structures to discern gene-gene interactions from transcriptomic data, in a manner akin to language learning. However, unlike the language sentences that depend on strict sequences of discrete word tokens for conveying precise meaning, transcriptomic data typically regard genes as an unordered set, with gene expressions represented as continuous scalar values (F. Yang et al. 2022 ). In this study, we intentionally designed the GeneBag model, a CFM that adapts to these unique characteristics of transcriptomic data. We employed a bidirectional encoder, substituting the positional embeddings with gene id embeddings (Fig. 1 ). The model's indifference to the sequence of gene tokens was reinforced through iterative shuffling during both pretraining and fine-tuning stages (Methods). Continuous expression values, via logarithmic transformation of the raw read counts, were embedded for each gene to mirror the quantitative spectrum of transcriptome (methods). GeneBag is constructed based on the Longformer architecture, which supports longer context length (Beltagy, Peters, and Cohan 2020). This enabled the comprehensive analysis of the complete gene list, encompassing 17,930 genes across all datasets in this study, in a single operation (methods). Pretraining and Evaluation with Single-Cell RNA Sequencing Data GeneBag underwent initial pretraining on 1.3 million human single-cell RNA sequencing (scRNA-seq) data points from PanglaoDB (Franzén, Gan, and Björkegren 2019 ), employing a strategy of random masking of either gene identifiers or expression values (Fig. 1 ). This pre-training phase endowed the model with the capability to impute expression values for any random genes with an accuracy of 94.9% (Pearson correlation coefficient), indicative of the model's adept learning of gene-gene interactions at the single-cell level. Our initial evaluation of GeneBag focused on the task of single-cell annotation, utilizing the Zheng68K dataset as a benchmark (Zheng et al. 2017 ). This dataset, comprising blood mononuclear cells of highly similar subtypes, presents a great challenge to cell annotation (F. Yang et al. 2022 ). Post fine-tuning, GeneBag achieved an overall accuracy of 71.33% and an F1-score of 0.61. The cell type annotation closely mirrored the ground truth, as visualized on the T-SNE plot (Fig. 2 A). The confusion matrix (Fig. 2 B) revealed that the majority of cell types attained a high degree of accuracy, ranging from 65.95 to 96.94%, with the exception of CD4+/CD45RA+/CD25- Naïve T, which registered a lower accuracy of 40.57%. Notably, the model demonstrated a significantly elevated accuracy of 88.94% for CD4+/CD25 T Reg, significantly outperforming the previous models(F. Yang et al. 2022 ). GeneBag also separates CD8 + Cytotoxic T cells and CD8+/CD45RA + Naive Cytotoxic cells with relatively high accuracies (65.95% and 88.42% respectively). This is noteworthy as these cell types have traditionally been considered challenging to discern due to their marked similarities(Zheng et al. 2017 )(F. Yang et al. 2022 ). Transfer Learning of the GeneBag Model on Bulk RNA-seq Data In clinical practice, tissue-level bulk RNA sequencing is a widely utilized and well-established diagnostic method. Should the single cell foundation model successfully integrate bulk RNA-seq data, it would pave the way for numerous downstream applications in diagnostics and prognostics. Attention based foundation models have demonstrated remarkable capabilities of multimodal inferences through transfer learning, in domains such as natural language processing and computer vision (Zhu et al. 2020 ; Vaswani et al. 2017 ). Motivated by this, we retrained GeneBag on a mixed dataset, integrating 100k single cell samples from PanglaoDB and 19k bulk RNA-seq samples from the Genotype-Tissue Expression (GTEx) project (Lonsdale et al. 2013 ; Wilks et al. 2021 ) (see Methods). This hybrid retraining strategy facilitated the model's adaptation to the nuances of bulk RNA-seq data, while minimally impacting its proficiency in the single cell context, as evidenced by the overall mask prediction accuracy of 90.51%. Tissue and Tumor Type Classification utilizing Bulk RNA-seq The retrained GeneBag model was first evaluated for its capability to classify various tissues and tumor types using bulk RNA-seq data. The primary inquiry was whether the model could learn to tell the intrinsic differences between normal and malignant stages, even in previously unseen tissue types. To examine this, we selected a diverse range of ten tissue types—encompassing breast, colon, esophagus, kidney, liver, lung, prostate, stomach, thyroid, and uterus—where both normal (from GTEx) and tumorous (from TCGA (Wilks et al. 2021 )) samples were available. These samples were thus combined into a single training set, labeled as either normal (5,397 instances) or tumorous (6,163 instances), respectively. GeneBag underwent fine-tuning on this training set. Subsequently, the model's efficacy of separating tumor from normal tissues was evaluated on tissue types (from GTEx or TCGA) not represented in the training set. The zero-shot learning prediction accuracy (on unseen tissue types) achieved an overall accuracy of 96.17%, with a classification specificity of 99.76% and a sensitivity of 85.53% for identifying tumors (Supplementary 2 Tables: Table S1 ). These results suggest that GeneBag has learned to generalize the fundamental characteristics that distinguish between normal and tumorous tissues, irrespective of the considerable variations across different tissue profiles. The only tissue marking a poor performance in this test is the Bone Marrow, with an accuracy of 15.69% (Supplementary 2 Tables: Table S2 ). This may be mainly because there are a large number of stem cells in the bone marrow, and the cell division characteristics of these stem cells are similar to those of tumor cells. Since most cancer types have been extensively studied, a more critical question is whether a CFM, after thorough fine-tuning, can accurately classify a comprehensive range of tissue and tumor types. To address this, we curated a broad bulk RNA-seq dataset, consisting of 28 distinct normal tissue (NT) types, 7 paracancerous tissue (PT) types, and 31 tumor (TM) types. We divided the samples in each type into training and testing sets with an 8:2 ratio. GeneBag was then subjected to fine-tuning on the training dataset, followed by evaluation on the reserved test set. The output embeddings from the classification network can be visualized using UMAP (Fig. 3 A). It can be seen that the data points of the 66 NT, PT, and TM types formed compact clusters within the same type, while being almost distinctly separated across different types (Supplementary 1 Figures: Fig. S2 ). The overall classification accuracy is 97.23%. Upon closer examination, NT types tend to gravitate towards the upper right direction, separated from the TM types distributed at the lower left corner, and PT samples are interspersed in between (Fig. 3 A). Notably, NT and TM from the same tissue type (in the same color-code, in Fig. 3 A) are distantly separeted, exhibiting a similar diagonal projection, contrasting with previous studies where NT and TM of the same tissue were in closer proximity (Prada-Luengo et al. 2023 ). When examining the relationships among related tissues, we found that in normal tissues, the colon and small intestine are closely aligned, mirroring their anatomical proximity, yet they are notably distant from the esophagus and stomach, indicating significant developmental divergence among these clusters (Fig. 3 B). Interestingly, breast samples partially intermingle with adipose tissue, hinting at a shared developmental lineage (Fig. 3 B). In the realm of tumor tissues, three subtypes of kidney tumors: chromophobe, renal papillary cell carcinoma, and renal clear cell carcinoma, although converging into the same subspace, are clearly demarcated from one another and are also far removed from the kidney paracancerous tissue (Fig. 3 C). The model's capacity for subtype classification is further exemplified in Fig. 3 D, where a total of 15 diverse tumor types are largely distinguishable within a small area of the 2D UMAP. Notably, testicular germ cell tumor, cervical squamous cell carcinoma and endocervical adenocarcinoma (TGCT and CESC) are observed to be in close proximity, suggesting a shared biological signature (Fig. 3 D). In summary, the developmental relatedness is reflected in the classification embeddings, yet even highly similar subtypes can be effectively segregated. To analytically evaluate the classification performance, we generated a confusion matrix encompassing the 40 tumor types (Fig. 4 ). The predicted assignments were found to align closely with the actual labels, achieving an overall accuracy of 93.62%. This is a state-of-art (SOTA) performance across nearly all commonly studied cancer types (Supplementary 1 Figures: Fig. S3 ), when compared to previous studies. Particularly in Bladder Urothelial Carcinoma (BLAC) and Stomach adenocarcinoma (STAD), which have been challenging to discern with bulk RNA-seq in previous studies (< 0.45 for BLAC and < 0.35 for STAD) (Prada-Luengo et al. 2023 ; Vivian et al. 2020 ). GeneBag attained accuracies of 0.93 for BLAC and 0.90 for STAD, respectively. The overall results suggest that foundation model based approaches, such as GeneBag, have great potential in simultaneous classification of a wide range of tumor types. On the other hand, it seems that the current version of GeneBag has poor ability to recognize metastatic or recurrent tumors. All the metastatic or recurrent cases in BRCA, GBM, LGG, OV, TGCT, and THCA were predominantly categorized under the corresponding primary tumor categories, with the only exception of skin cutaneous melanoma (SKCM), whose metastatic samples were correctly labeled as the given type in 86% of cases (Fig. 4 ). This observation may imply either a lack of distinct molecular markers for tumor recurrences and metastasis, or there is a limitation of resolution in the current model to differentiate between primary and recurrent tumors. Staging and Survival Prediction Utilizing Bulk RNA-Seq Beyond facilitating cancer diagnosis, there is a significant interest in leveraging bulk RNA-seq data for prognostic inferences such as cancer staging and survival prediction. Accurate staging is essential for assessing disease severity and formulating treatment plans. However, the subjectivity inherent in cancer staging can lead to variability in results, influenced by factors such as inter-observer variation, tumor characteristics, and diagnostic methodologies. It has been reported that staging concordance rates can range widely, from 58–86% (Dolly et al. 2020 ) (Gwon et al. 2024 ; Plichta et al. 2019). In this study, we explored the potential of using bulk RNA-seq to classify cancer staging across various cancer types. We included all tumor samples with available staging information and utilized paracancerous samples as a baseline stage 0 for our fine-tuning experiments. As depicted in Fig. 5 A, a clear correlation was observed between the predicted and actual stages, with an overall Pearson correlation coefficient of 0.685. This correlation is notably high, especially considering the inherent variability in staging assessments and the diverse range of cancer types included in this analysis. This result underscores the promise of the foundation model in objective and precise cancer stage estimation through the analysis of bulk RNA-seq data. We further assessed the model’s capability to predict the survival rate through varying time spans of 1, 3 and 5 years. The TCGA samples were categorized based on two conditions: 1, those who succumbed to the disease within a given time span, post-initial cancer diagnosis; 2, those who survived beyond the given time span or were confirmed alive after the given time span post the initial diagnosis. The fine-tuned model achieved an overall accuracy of 74.77% − 75.09%, in predicting survival rates (Fig. 5 B, Supplementary 2 Tables: Table S3 ). Adjusting the prediction threshold, defined by the ratio of softmax outputs from the binary classifier network, the model demonstrated an overall AUC (Area Under the Curve) of 0.7698, 0.8281 and 0.8042 for 1-, 3- and 5-year, respectively (Supplementary 1 Figures: Fig. S4A). To gain further insights in individual disease, we focused on the analysis of 5-year survival. It is found that the accuracy of survival rate predictions varied significantly across different cancer types. Notably, Esophageal Adenocarcinoma (ESCA, 93.75%), Glioblastoma Multiforme (GBM, 100%), Kidney Clear Cell Carcinoma (KICH, 90%), Mesothelioma (MESO, 100%), Pancreatic Adenocarcinoma (PAAD, 90%), Stomach Adenocarcinoma (STAD, 92.59%), and Testicular Germ Cell Tumors (TGCT, 92.31%) showed accuracies of 5-year survival prediction exceeding 90% (Supplementary 1 Figures: Fig. S4B). In contrast, certain cancer types such as Head and Neck Squamous Cell Carcinoma (HNSC, 64.58%), Kidney Renal Clear Cell Carcinoma (KIRC, 66.18%), Lung Squamous Cell Carcinoma (LUSC, 66.38%), and Ovarian Cancer (OV, 57.63%) were found to have comparatively poor survival predictions (Supplementary 2 Tables: Table S4). Discussion In the realm of leveraging AI technologies for large-scale omics data analysis, previous studies have predominantly employed end-to-end training network approaches. These methods, while valuable, encounter limitations due to constraints such as limited labeled data and a narrow scope of application. This study, to our knowledge, is the first to make use of a Cell Foundation Model for addressing pivotal questions in cancer diagnosis and prognosis using bulk RNA-seq data. Despite having a modest number of parameters, GeneBag has demonstrated exceptional capabilities in classifying a diverse array of tissue and cancer types with high accuracy, as well as in forecasting cancer stages and 5-year survival rates. Larger models trained on more expansive single-cell data corpora would certainly further amplify performance. Moreover, with fine-tuning, foundation models may be able to address more complex questions, such as forecasting the effectiveness of diverse treatments or customizing personalized therapeutic strategies based on the analyses of bulk RNA-seq data. To accomplish this, it is essential to have cohort data that comprehensively records treatments and their associated outcomes, enabling CFMs to discern the relationship between therapeutic interventions and RNA profiles. Traditionally, pathological testing through standard biopsy involves sectioning, staining, and microscopic examination of tissue by pathologists to ascertain cancer type and grade, a process prone to inter-observer variability. Furthermore, subtle distinctions between tumor types might not be apparent through morphological changes alone. Conversely, a CFM-based approach with biopsy bulk RNA-seq offers high-throughput, objective screening across a wide range of cancer types, and potentially with greater sensitivity and specificity due to the utilization of comprehensive transcriptomic data. Additionally, CFMs could enable the inclusion of new inferences in a single examination, such as prognostic assessments and precision medicine strategies. Ultimately, it is anticipated that such novel, foundation model-driven and RNA profiling based diagnostic approaches will complement traditional pathology, offering supplementary information to bolster clinical decision-making. However, the integration of CFMs into clinical practice, like any AI tool, necessitates stringent validation and regulatory approval to ensure safety and efficacy. Methods Overview The GeneBag Model is a modified BERT (Bidirectional Encoder Representations from Transformers) model. It processes input sequences from single-cell or bulk tissue RNA-Seq data through an encoder with multi-head attention mechanism. The encoder output is then processed differently during the pre-training and fine-tuning stage. In pre-training (and also in retraining), it performs a fill-mask task, using a decoder to predict expression values for masked genes. This trains both the encoder and decoder. During fine-tuning, a classifier is trained by applying it to the entire encoder output, adapting the model for specific prediction about each sample. While this process shares similarities with other language models used in gene expression analysis, GeneBag introduces three key innovations. 1. GeneBag doesn't rely on fixed gene positions or position encoding, only requiring alignment between gene tokens and expression values. 2. Unlike binned expression, GeneBag directly encodes and decodes raw expression data, preserving the full granularity of the biological information. 3. By incorporating the Longformer attention module, GeneBag can process input sequences containing full gene lists. Input Embedding We collected a huge RNA-Seq dataset and organized it into paired sequences. In the dataset, each sample consists of a gene ID sequence and its corresponding expression value sequence. We incorporated special tokens ( , , and ) at the beginning and end of sequences, similar to other language models. All gene IDs and special tokens were tokenized as integers using a common vocabulary. Unlike other models, genes don't have fixed positions in our sequences. However, gene ID tokens remain aligned with their corresponding expression values in the paired sequences, maintaining crucial gene-expression relationships. Gene ID tokens were converted to gene embeddings of dimension d_model using PyTorch torch.nn.embedding. Expression values were directly encoded into expression embeddings of the same dimension d_model using a series of cosine and sine functions. This deterministic encoding allows for precise representation of continuous data. The gene and expression embeddings were then added together to form the input (1) for the Longformer encoder. Encoder with attention mechanism Usually the input embeddings were processed with a transformer encoder. However, standard self-attention struggles with very long sequences due to its quadratic complexity. This is problematic for transcriptome data, which can exceed 15,000 genes. To solve this, we implement Longformer attention, which uses a sparse attention pattern. This allows the model to handle longer input sequences efficiently and potentially capture broader biological contexts and interactions. Our sequence length is set to 17,932. Our Longformer encoder includes six attention heads and six encoder layers. The model's dimension ( d_model ) is set to 42. Pre-training of GeneBag Model Based on Single Cell RNA-seq Data The pre-training data consists of approximately 1.3 million single-cell RNA-Seq sequences. Each cell sequence was firstly processed with masking, where 15% of the gene expression values in the non-padding regions are randomly replaced. These masked sequences are then processed through input embedding and the Longformer encoder. The encoder output for each masked gene is fed into a multi-layer perceptron to predict the expression value. The predicted expressions are compared against the actual values using Mean Squared Error (MSE) as the loss function. Prediction accuracy is evaluated with the Pearson correlation coefficient. Other configurations include a batch size of 32, a learning rate of 1e-4, and the Adam optimizer. Cell Type Annotation We benchmarked the GeneBag model's performance in cell type annotation using the Zheng68K dataset. In this dataset, the occurrence of some cell types is sparse, leading to an imbalanced distribution. To address this, we set a threshold of 6,185 cells per cell type. For types exceeding this threshold, we randomly select a subset to meet this limit. Cell types with fewer than 6,185 cells are fully included, creating a more balanced dataset. This process results in a dataset with a total of 40,828 cells (Supplementary 3 Tables: Table S1 ). This was further divided as a train and test dataset with 8:2 ratio. A classifier with a single convolutional layer followed by three multilayer perceptrons (MLPs) was used to predict the cell type from the entire encoder output. The loss function is CrossEntropy. Re-training of GeneBag Model on Bulk RNA-seq For training on bulk RNA-seq data, the same gene ID embedding and expression value embedding methods was used for the scRNA-seq. The token was used to distinguish the datatypes of scRNA and bulk RNA. Re-training is similar to pre-training while including the bulk tissue data. We used the entire GTEx data (19,081 samples) and randomly sampled 100,000 cells from the pre-training dataset, labeled as “bulk” and “scRNA”, respectively. We merged them to create a balanced dataset. We further divided it into training/test/validation subsets in an 8/1/1 ratio. Zero-shot Tumor Recognition To assess the GeneBag model's capacity to accurately identify tumor samples among previously unseen tissue types, we selected ten tissues as training set in this fine-tune task. These 10 tissues were chosen based on two criteria: (a.) their presence in both the GTEx and TCGA databases; (b.) the availability of at least 50 samples from the GTEx and TCGA-tumor datasets, as well as at least 10 samples from the TCGA-normal datasets. For testing, we included 13,684 normal and 3,798 primary tumor samples from other tissue types. Similar to cell type annotation, we used the same classifier and CrossEntropy loss function to distinguish normal and tumor samples (Supplementary 3 Tables: Table S2 ). Tissue and Cancer Type Classification The whole dataset was composed of three distinct categories: normal tissues from GTEx, tumor tissues from TCGA-tumor and paracancerous tissues from TCGA-normal. We included only types of sample size exceeding 50. We evenly divided these samples into training and testing sets, comprising 23,654 and 5,945 samples, respectively (Supplementary 3 Tables: Table S3 ). For confusion table analysis of tumor subtypes, 40 tumor types were analyzed. Tumor types with 5 or fewer samples were excluded. The total samples were evenly divided within each tumor type according to a 8:2 ratio, resulting in a training set of 7,987 and a test set of 2,525 samples (Supplementary 3 Tables: Table S4). The same classifier network was used except the number of output neurons were matched to the number of types. Tumor Staging For tumor staging, we focused solely on primary tumor samples for stage prediction purposes. Stage labels I, II, III and IV were used, and subtypes such as Ia, Ib, were merged into the major type. Samples using non-standard labels were excluded for clarity. Solid normal tissues, which are paracancerous in TCGA, were labeled as stage 0. The final dataset comprised a total of 7,611 samples (Supplementary 3 Tables: Table S5). We assumed the cancer stages as continuous values. The decoder to predict stages was similar to the cell type classifier, except the final layer of the MLP goes to a single neuron to estimate the tumor stage, and MSE loss was used. Survival Prediction In the TCGA dataset, we identified 966, 2,146 and 2,559 samples who did not survive beyond one, three and five years, respectively and 7,804, 3,281 and 1,590 individuals who lived longer than the corresponding years. For the binary classification task, we applied the same classifier architecture as mentioned before (Supplementary 3 Tables: Table S6). Data Availability: Panglao dataset The Single Cell RNA-seq (scRNA) gene expression dataset was from the Panglao dataset ( https://panglaodb.se/ ). This dataset contains 1,16,580 cells with 77 tissues, which were collected from 209 single cell datasets. In our study, only the gene IDs and expression values from the Panglao dataset were used.. Zheng68K dataset Zheng68K (Zheng et al. 2017 ) is a well annotated PBMC dataset generally used for cell type annotation evaluation, and it consists of 11 cell types, totaling 68,450 cells (Supplementary 1 Figures: Fig. S1 ). Among them, there are 20,757 CD8 + Cytotoxic T cells, 16,645 CD8+/CD45RA + Naive Cytotoxic cells, 8,775 CD56 + NK cells, 6,185 CD4+/CD25 T Reg cells, 5,877 CD19 + B cells, 3,059 CD4+/CD45RO + Memory cells, 2,847 CD14 + Monocyte cells, 2,095 Dendritic cells, 1,871 CD4+/CD45RA+/CD25- Naive T cells, 242 CD34 + cells and 97 CD4 + T Helper2 cells, accounting for 30.3%, 24.3%, 12.8%, 9.0%, 8.6%, 4.5%, 4.2%, 3.1%, 2.7%, 0.4%, and 0.1%, respectively. Genotype-Tissue Expression (GTEx) GTEx is a public database containing transcriptome gene expression data from large scale human normal tissues. We downloaded the raw gene counts from Recount3 (Wilks et al. 2021 ) platform via the build-in R packages. The total data of 31 tissue types with 19,081 samples were used in our study. The Cancer Genome Atlas Program (TCGA) The TCGA database is composed of RNA sequencing data of tumor biopsies, with a total of 11,348 samples. As above, the raw gene counts and meta-data files both were obtained from Recount3 (Wilks et al. 2021 ). Data pre-processing Raw expression values are all read counts. For GTEx data, we removed low-expression genes and non-protein-coding genes. This results in a list of 16,874 genes. The same genes were selected for TCGA data. Log transformation was done only for GTEx and TCGA data, because the single cell data was already processed. Lastly, we performed a gene-wised normalization by calculating z-scores for all datasets (2). Declarations Code availability: All code was implemented in Python using PyTorch as the primary deep learning package. The source codes for the GeneBag model’s pre-training and subsequent fine-tuning tasks are available at Github (https://github.com/transomics/GeneBag). The Fine-tuning models are available at Zenodo (https://zenodo.org/records/14499146) Ethics approval and consent to participate: Not applicable Consent for publication: Not applicable References Atta, Lyla, and Jean Fan. 2021. “Computational Challenges and Opportunities in Spatially Resolved Transcriptomic Data Analysis.” Nature Communications 12 (1): 5283. Bareche, Y., D. Venet, M. Ignatiadis, P. Aftimos, M. Piccart, F. Rothe, and C. Sotiriou. 2018. “Unravelling Triple-Negative Breast Cancer Molecular Heterogeneity Using an Integrative Multiomic Analysis.” Annals of Oncology: Official Journal of the European Society for Medical Oncology / ESMO 29 (4): 895–902. Bartocci, Ezio, and Pietro Lió. 2016. “Computational Modeling, Formal Analysis, and Tools for Systems Biology.” PLoS Computational Biology 12 (1): e1004591. Beltagy, Iz, Matthew E. Peters, and Arman Cohan. 2020. “Longformer: The Long-Document Transformer.” http://arxiv.org/abs/2004.05150. Bostanci, Erkan, Engin Kocak, Metehan Unal, Mehmet Serdar Guzel, Koray Acici, and Tunc Asuroglu. 2023. “Machine Learning Analysis of RNA-Seq Data for Diagnostic and Prognostic Prediction of Colon Cancer.” Sensors 23 (6). https://doi.org/10.3390/s23063080. Cui, Haotian, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang. 2024. “scGPT: Toward Building a Foundation Model for Single-Cell Multi-Omics Using Generative AI.” Nature Methods , February, 1–11. Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.” https://doi.org/10.48550/ARXIV.1810.04805. Dolly, Y. Wu, Ann E. Spangler, Dat T. Vo, Alberto de Hoyos, and Stephen J. Seiler. 2020. “Simplified, Standardized Methods to Assess the Accuracy of Clinical Cancer Staging.” Cancer Treatment and Research Communications 25 (January): 100253. Feng, Jing, Wenbo Chen, Xin Dong, Jun Wang, Xiangfei Mei, Jin Deng, Siqi Yang, et al. 2022. “CSCD2: An Integrated Interactional Database of Cancer-Specific Circular RNAs.” Nucleic Acids Research 50 (D1): D1179–83. Floridi, Luciano, and Massimo Chiriatti. 2020. “GPT-3: Its Nature, Scope, Limits, and Consequences.” Minds and Machines 30 (4): 681–94. Franzén, Oscar, Li-Ming Gan, and Johan L. M. Björkegren. 2019. “PanglaoDB: A Web Server for Exploration of Mouse and Human Single-Cell RNA Sequencing Data.” Database: The Journal of Biological Databases and Curation 2019 (April): baz046. Giulietti, Matteo, Alessandra Righetti, Giovanni Principato, and Francesco Piva. 2018. “LncRNA Co-Expression Network Analysis Reveals Novel Biomarkers for Pancreatic Cancer.” Carcinogenesis 39 (8): 1016–25. Golub, T. R., D. K. Slonim, P. Tamayo, C. Huard, M. Gaasenbeek, J. P. Mesirov, H. Coller, et al. 1999. “Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring.” Science 286 (5439): 531–37. Gomez-Ramirez, Jaime, and Ricardo Sanz. 2013. “On the Limitations of Standard Statistical Modeling in Biological Systems: A Full Bayesian Approach for Biology.” Progress in Biophysics and Molecular Biology 113 (1): 80–91. Gwon, Hye Ran, A. La Woo, Seung Hyun Yong, Youngmok Park, Song Yee Kim, Eun Young Kim, Ji Ye Jung, et al. 2024. “Factors Affecting Accuracy of Clinical Staging in Resectable Non-Small Cell Lung Cancer in a Real-World Study.” Thoracic Cancer 15 (9): 730–37. Hao, Minsheng, Jing Gong, Xin Zeng, Chiming Liu, Yucheng Guo, Xingyi Cheng, Taifeng Wang, Jianzhu Ma, Le Song, and Xuegong Zhang. 2023. “Large Scale Foundation Model on Single-Cell Transcriptomics.” bioRxiv . https://doi.org/10.1101/2023.05.29.542705. Hua, Junjie T., Sujun Chen, and Housheng H. He. 2019. “Landscape of Noncoding RNA in Prostate Cancer.” Trends in Genetics: TIG 35 (11): 840–51. Iorio, Marilena V., and Carlo M. Croce. 2012. “MicroRNA Dysregulation in Cancer: Diagnostics, Monitoring and Therapeutics. A Comprehensive Review.” EMBO Molecular Medicine , February. https://doi.org/10.1002/emmm.201100209. Kitano, Hiroaki. 2002. “Systems Biology: A Brief Overview.” Science 295 (5560): 1662–64. Li, Guanqiao, Yang Liu, Hongxi Hu, Shuona Yuan, Liyun Zhou, and Xiaoyuan Chen. 2022. “Evolution of Innovative Drug R&D in China.” Nature Reviews. Drug Discovery 21 (8): 553–54. Li, Jiao, Dan Sun, Wenchen Pu, Jin Wang, and Yong Peng. 2020. “Circular RNAs in Cancer: Biogenesis, Function, and Clinical Significance.” Trends in Cancer Research 6 (4): 319–36. Loewen, Gregory, Janarthanan Jayawickramarajah, Ying Zhuo, and Bin Shan. 2014. “Functions of lncRNA HOTAIR in Lung Cancer.” Journal of Hematology & Oncology 7 (1): 1–10. Lonsdale, John, Jeffrey Thomas, Mike Salvatore, Rebecca Phillips, Edmund Lo, Saboor Shad, Richard Hasz, et al. 2013. “The Genotype-Tissue Expression (GTEx) Project.” Nature Genetics 45 (6): 580–85. Lu, Jun, Gad Getz, Eric A. Miska, Ezequiel Alvarez-Saavedra, Justin Lamb, David Peck, Alejandro Sweet-Cordero, et al. 2005. “MicroRNA Expression Profiles Classify Human Cancers.” Nature 435 (7043): 834–38. Ma, Xiaoshi, Jinan Guo, Kaisheng Liu, Lipeng Chen, Dale Liu, Shaowei Dong, Jinquan Xia, et al. 2020. “Identification of a Distinct Luminal Subgroup Diagnosing and Stratifying Early Stage Prostate Cancer by Tissue-Based Single-Cell RNA Sequencing.” Molecular Cancer 19 (1): 147. Meng, Shujuan, Hecheng Zhou, Ziyang Feng, Zihao Xu, Ying Tang, Peiyao Li, and Minghua Wu. 2017. “CircRNA: Functions and Properties of a Novel Potential Biomarker for Cancer.” Molecular Cancer 16 (1): 1–8. Plichta, Jennifer K., Samantha M. Thomas, Amanda R. Sergesketter, Rachel A. Greenup, Oluwadamilola M. Fayanju, Laura H. Rosenberger, Nina Tamirisa, Terry Hyslop, and E. Shelley Hwang. 2019. “Clinical and Pathological Stage Discordance among 433,514 Breast Cancer Patients.” American Journal of Surgery 218 (4): 669–76. Prada-Luengo, Iñigo, Viktoria Schuster, Yuhu Liang, Thilde Terkelsen, Valentina Sora, and Anders Krogh. 2023. “N-of-One Differential Gene Expression without Control Samples Using a Deep Generative Model.” Genome Biology 24 (1): 263. Shang, Qingfeng, Zhi Yang, Renbing Jia, and Shengfang Ge. 2019. “The Novel Roles of circRNAs in Human Cancer.” Molecular Cancer 18 (1): 1–10. Shen, Yong Shen Xiaowei Peng. 2020. “Identification and Validation of Immune-Related lncRNA Prognostic Signature for Breast Cancer.” Genomics 112 (3): 2640–46. Teresa Agulló-Ortuño, M., Fernando López-Ríos, and Luis Paz-Ares. 2010. “Lung Cancer Genomic Signatures.” Journal of Thoracic Oncology: Official Publication of the International Association for the Study of Lung Cancer 5 (10): 1673–91. Theodoris, Christina V., Ling Xiao, Anant Chopra, Mark D. Chaffin, Zeina R. Al Sayed, Matthew C. Hill, Helene Mantineo, et al. 2023. “Transfer Learning Enables Predictions in Network Biology.” Nature 618 (7965): 616–24. Valihrach, Lukas, Peter Androvic, and Mikael Kubista. 2020. “Circulating miRNA Analysis for Cancer Diagnostics and Therapy.” Molecular Aspects of Medicine 72 (April): 100825. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. “Attention Is All You Need.” https://doi.org/10.48550/ARXIV.1706.03762. Verduci, Lorena, Sabrina Strano, Yosef Yarden, and Giovanni Blandino. 2019. “The circRNA-microRNA Code: Emerging Implications for Cancer Diagnosis and Treatment.” Molecular Oncology 13 (4): 669–80. Vivian, John, Jordan M. Eizenga, Holly C. Beale, Olena M. Vaske, and Benedict Paten. 2020. “Bayesian Framework for Detecting Gene Expression Outliers in Individual Samples.” JCO Clinical Cancer Informatics 4 (February): 160–70. Wang, Hongbo, Qinghai Meng, Jinjun Qian, Muxi Li, Chunyan Gu, and Ye Yang. 2022. “Review: RNA-Based Diagnostic Markers Discovery and Therapeutic Targets Development in Cancer.” Pharmacology & Therapeutics 234 (June): 108123. Wang, Sumeng, Ke Zhang, Shanyue Tan, Junyi Xin, Qianyu Yuan, Huanhuan Xu, Xian Xu, et al. 2021. “Circular RNAs in Body Fluids as Cancer Biomarkers: The New Frontier of Liquid Biopsies.” Molecular Cancer 20 (1): 1–10. Wang, Xuefeng, Shuo Zhang, Yuqin Liu, Jian Du, and Heng Huang. 2021. “How Pharmaceutical Innovation Evolves: The Path from Science to Technological Development to Marketable Drugs.” Technological Forecasting and Social Change 167 (June): 120698. Wang, Yanying, Lei He, Ying Du, Pingping Zhu, Guanling Huang, Jianjun Luo, Xinlong Yan, et al. 2015. “The Long Noncoding RNA lncTCF7 Promotes Self-Renewal of Human Liver Cancer Stem Cells through Activation of Wnt Signaling.” Cell Stem Cell 16 (4): 413–25. Wen, Guoxia, Tong Zhou, and Wanjun Gu. 2020. “The Potential of Using Blood Circular RNA as Liquid Biopsy Biomarker for Human Diseases.” Protein & Cell 12 (12): 911–46. Wilks, Christopher, Shijie C. Zheng, Feng Yong Chen, Rone Charles, Brad Solomon, Jonathan P. Ling, Eddie Luidy Imada, et al. 2021. “recount3: Summaries and Queries for Large-Scale RNA-Seq Expression and Splicing.” Genome Biology 22 (1): 323. Wu, Qiong, Jiali Ma, Jue Wei, Wenying Meng, Yugang Wang, and Min Shi. 2021. “lncRNA SNHG11 Promotes Gastric Cancer Progression by Activating the Wnt/β-Catenin Pathway and Oncogenic Autophagy.” Molecular Therapy: The Journal of the American Society of Gene Therapy 29 (3): 1258–78. Xia, Qianlin, Tao Ding, Guihong Zhang, Zehuan Li, Ling Zeng, Yanjun Zhu, Jianming Guo, et al. 2018. “Circular RNA Expression Profiling Identifies Prostate Cancer- Specific circRNAs in Prostate Cancer.” Cellular Physiology and Biochemistry: International Journal of Experimental Cellular Physiology, Biochemistry, and Pharmacology 50 (5): 1903–15. Xu, Wei, Gai Zhou, Huizhi Wang, Yawen Liu, Baoding Chen, Wei Chen, Chen Lin, Shuhui Wu, Aihua Gong, and Min Xu. 2020. “Circulating lncRNA SNHG11 as a Novel Biomarker for Early Diagnosis and Prognosis of Colorectal Cancer.” International Journal of Cancer. Journal International Du Cancer 146 (10): 2901–12. Yang, Fan, Wenchuan Wang, Fang Wang, Yuan Fang, Duyu Tang, Junzhou Huang, Hui Lu, and Jianhua Yao. 2022. “scBERT as a Large-Scale Pretrained Deep Language Model for Cell Type Annotation of Single-Cell RNA-Seq Data.” Nature Machine Intelligence 4 (10): 852–66. Yang, Xiaodong, Guole Liu, Guihai Feng, Dechao Bu, Pengfei Wang, Jie Jiang, Shubai Chen, et al. 2023. “GeneCompass: Deciphering Universal Gene Regulatory Mechanisms with Knowledge-Informed Cross-Species Foundation Model.” bioRxiv . https://doi.org/10.1101/2023.09.26.559542. Zhang, He-da, Lin-Hong Jiang, Da-Wei Sun, Jun-Chen Hou, and Zhen-Ling Ji. 2017. “CircRNA: A Novel Type of Biomarker for Cancer.” Breast Cancer 25 (1): 1–7. Zhang, Li, Chenkai Lv, Yaqiong Jin, Ganqi Cheng, Yibao Fu, Dongsheng Yuan, Yiran Tao, Yongli Guo, Xin Ni, and Tieliu Shi. 2018. “Deep Learning-Based Multi-Omics Data Integration Reveals Two Prognostic Subtypes in High-Risk Neuroblastoma.” Frontiers in Genetics 9 (October): 477. Zhan, Yonghao, Zhicong Chen, Shiming He, Yanqing Gong, Anbang He, Yifan Li, Lianghao Zhang, et al. 2020. “Long Non-Coding RNA SOX2OT Promotes the Stemness Phenotype of Bladder Cancer Cells by Modulating SOX2.” Molecular Cancer 19 (1): 1–13. Zheng, Grace X. Y., Jessica M. Terry, Phillip Belgrader, Paul Ryvkin, Zachary W. Bent, Ryan Wilson, Solongo B. Ziraldo, et al. 2017. “Massively Parallel Digital Transcriptional Profiling of Single Cells.” Nature Communications 8 (1): 1–12. Zhu, Hu, Ze Wang, Yu Shi, Yingying Hua, Guoxia Xu, and Lizhen Deng. 2020. “Multimodal Fusion Method Based on Self-Attention Mechanism.” Proceedings of the ... International Wireless Communications & Mobile Computing Conference / Association for Computing Machinery. International Wireless Communications & Mobile Computing Conference 2020 (September): 1–8. Additional Declarations There is NO Competing Interest. Supplementary Files Supplementary1Figures.docx Supplementary 1 Figures Supplementary2Tables.xlsx Supplementary 2 Tables Supplementary3Tables.xlsx Supplementary 3 Tables Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5720342","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":400485539,"identity":"2b272719-fb82-4e41-ade8-1ae8423e776e","order_by":0,"name":"Kun Tang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAvklEQVRIiWNgGAWjYBACCQhlIQciDzwgQYuEMVhLAilaEhtAFFFaJGfkHvzwc49E+vywww+BttjJ6TYQ0CItkZcs2fNMInfj7TQDoJZkY7MDBLTISeSYMfAcAGqZnQDSciBxGzFaGP8ckEg3nJ3+gTgt0kAtzEBbEuSlc4i0RbLnjbG0zAEJww3SOQUHEgyI8IvE8RzDj28O2MjLz07f/OFDhZ0cQS1wYABWaUCschCQbyBF9SgYBaNgFIwoAACI5kE1W593/gAAAABJRU5ErkJggg==","orcid":"","institution":"Zhejiang Lab","correspondingAuthor":true,"prefix":"","firstName":"Kun","middleName":"","lastName":"Tang","suffix":""},{"id":400485540,"identity":"caa423f9-cacb-4328-b63c-9c3acb25316e","order_by":1,"name":"Yuhu Liang","email":"","orcid":"https://orcid.org/0000-0001-7489-6928","institution":"Zhejiang Lab","correspondingAuthor":false,"prefix":"","firstName":"Yuhu","middleName":"","lastName":"Liang","suffix":""},{"id":400485541,"identity":"4fb6cdc5-1296-4746-86be-6a40e0e33e7c","order_by":2,"name":"Dan Li","email":"","orcid":"","institution":"Wenzhou Medical University","correspondingAuthor":false,"prefix":"","firstName":"Dan","middleName":"","lastName":"Li","suffix":""},{"id":400485542,"identity":"65bd0133-d580-4a64-ac82-be21ccb04440","order_by":3,"name":"Dong Luo","email":"","orcid":"","institution":"Xiangya Hospital","correspondingAuthor":false,"prefix":"","firstName":"Dong","middleName":"","lastName":"Luo","suffix":""},{"id":400485543,"identity":"41088f95-ed84-43b5-968d-9ed20988a672","order_by":4,"name":"Augix Xu","email":"","orcid":"","institution":"Zhejiang University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Augix","middleName":"","lastName":"Xu","suffix":""},{"id":400485544,"identity":"3377c3e7-ce92-4961-acca-e32cb5529cb3","order_by":5,"name":"Pengchao Luo","email":"","orcid":"","institution":"University of Chinese Academy of Sciences","correspondingAuthor":false,"prefix":"","firstName":"Pengchao","middleName":"","lastName":"Luo","suffix":""},{"id":400485545,"identity":"85c63b0a-601b-4eea-b90b-7771f84cc02a","order_by":6,"name":"Yan Shao","email":"","orcid":"","institution":"China Mobile Hangzhou Research and Development Center","correspondingAuthor":false,"prefix":"","firstName":"Yan","middleName":"","lastName":"Shao","suffix":""},{"id":400485546,"identity":"4d10b626-69ae-4df7-9e47-c459e23eef11","order_by":7,"name":"Jianbo Yang","email":"","orcid":"","institution":"Chinese Academy of Sciences","correspondingAuthor":false,"prefix":"","firstName":"Jianbo","middleName":"","lastName":"Yang","suffix":""},{"id":400485547,"identity":"248cad16-9e82-4d9e-851a-79590b35cdbe","order_by":8,"name":"Xuejun Gong","email":"","orcid":"","institution":"Xiangya Hospital","correspondingAuthor":false,"prefix":"","firstName":"Xuejun","middleName":"","lastName":"Gong","suffix":""}],"badges":[],"createdAt":"2024-12-27 09:05:13","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5720342/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5720342/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":78818198,"identity":"df779b31-be9a-4ce6-812a-1b01f51b3008","added_by":"auto","created_at":"2025-03-19 10:58:09","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":106172,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eModel structure\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eEach cell consists of a pair of gene and expression sequences. Gene IDs were tokenized, paired with the corresponding expression value embeddings to generate gene embeddings. In pre-training and fine-tuning, these embeddings, representing 17,930 genes, are fully shuffled to enhance model generalization before being fed into the encoder. The encoder consists of 6 layer transformers with multi-head attention. The output from the encoder is subsequently processed by task-specific decoder networks for fine-tune tasks.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-5720342/v1/d598100b4357efb30f974710.png"},{"id":78818617,"identity":"8ba632cc-baf4-4856-824a-57c6dd592065","added_by":"auto","created_at":"2025-03-19 11:06:09","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":240150,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSingle cell annotation.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA. In the scatter plots of Zheng68K cells, X and Y positions were obtained from T-SNE analysis of raw expression. Colors represent the different cell type labels.\u003c/p\u003e\n\u003cp\u003eB. Confusion matrix on the Zheng68k test cells. Columns represent ground truth and rows represent the predictions.\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-5720342/v1/9c2af3507a06108ef38982c5.png"},{"id":78818202,"identity":"a2f36459-c658-427f-a732-f123b52bb5c6","added_by":"auto","created_at":"2025-03-19 10:58:09","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":186049,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTissue classification. \u003c/strong\u003eInference on normal and tumor samples was done with a classifier. The last layer of the classifier was used for producing UMAP. Colors represent the ground truth of the sample types.\u003c/p\u003e\n\u003cp\u003eA. Only 12 tissue colors are labeled, in which tumorous and normal types are both present. Different marker shapes represent different tissue types, with solid dots, crosses and solid squares representing normal, tumorous and paracancerous tissues respectively.\u003c/p\u003e\n\u003cp\u003eB, C, D. zoom-in views of panel A.\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-5720342/v1/7f558e7853c69e1111829f26.png"},{"id":78818206,"identity":"6d3475d2-4b9f-4776-9619-919ebac4b51f","added_by":"auto","created_at":"2025-03-19 10:58:09","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":376509,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTumor subtyping.\u003c/strong\u003e Confusion matrix of tumor subtypes. Columns represent ground truth and rows represent the predictions.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-5720342/v1/362cc71bb4240b91ac767712.png"},{"id":78818618,"identity":"6210bb92-32c9-4842-a2b2-f4b1de62caa0","added_by":"auto","created_at":"2025-03-19 11:06:09","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":53913,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTumor staging and survival prediction.\u003c/strong\u003e A. Violin plot of tumor stage, X axis shows the tumor stage collected from TCGA metadata. The Y axis presents the predicted tumor stages of the GeneBag model. B. 1-, 3- and 5-year survival predictions. The dash-line indicates the percentage of patients alive at the time node. The bar-plots show the accuracy, F1-score and AUC of the survival prediction of the GeneBag model for the three time nodes of 1, 3 and 5 years.\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-5720342/v1/b4d6401d0cfc28340267971c.png"},{"id":78819261,"identity":"babe13a3-e48a-4a5f-b46d-718f5b7ef249","added_by":"auto","created_at":"2025-03-19 11:14:15","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1629000,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5720342/v1/609d7865-038b-4d9e-8f82-eff2a6d5e37b.pdf"},{"id":78819259,"identity":"12af7e6e-bdc4-494e-898e-47d7a69888f0","added_by":"auto","created_at":"2025-03-19 11:14:09","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":510579,"visible":true,"origin":"","legend":"Supplementary 1 Figures","description":"","filename":"Supplementary1Figures.docx","url":"https://assets-eu.researchsquare.com/files/rs-5720342/v1/2451dcddcd30d2cb34f5aa45.docx"},{"id":78818199,"identity":"8963c0ae-4153-4650-a594-5fb53219e7f0","added_by":"auto","created_at":"2025-03-19 10:58:09","extension":"xlsx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":16406,"visible":true,"origin":"","legend":"\u003cp\u003eSupplementary 2 Tables\u003c/p\u003e","description":"","filename":"Supplementary2Tables.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-5720342/v1/204ded8deb4d3cfe91d45114.xlsx"},{"id":78818626,"identity":"63e78f48-28ba-4fca-b1f3-3e727d176d57","added_by":"auto","created_at":"2025-03-19 11:06:10","extension":"xlsx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":3147621,"visible":true,"origin":"","legend":"\u003cp\u003eSupplementary 3 Tables\u003c/p\u003e","description":"","filename":"Supplementary3Tables.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-5720342/v1/fbcf347edefd908cace3b7bd.xlsx"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"GeneBag: training a cell foundation model for broad-spectrum cancer diagnosis and prognosis with bulk RNA-seq data","fulltext":[{"header":"Introduction","content":"\u003cp\u003eTraditionally, systems biology has aimed to fully understand the entirety of multi-omics data and to map the intricate relationships between countless biomolecules. The aspiration is that a complete analysis of biological systems will facilitate an in-depth understanding of the underlying mechanisms, thereby significantly advancing disease diagnostics and the innovation of pharmaceuticals and therapeutics (G. Li et al. \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2022\u003c/span\u003e; X. Wang et al. \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e2021\u003c/span\u003e; Kitano \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e2002\u003c/span\u003e). However, efforts in large-scale systems biology research were significantly impeded by the constraints of existing computational algorithms and analytical techniques, which struggle to keep pace with the intricate complexity and vast scales inherent in living organisms. Consequently, the findings from these studies often remain descriptive, and their models usually differ significantly from in vivo conditions, limiting their practical applicability in real-world scenarios (Gomez-Ramirez and Sanz 2013; Bartocci and Li\u0026oacute; 2016; Atta and Fan 2021).\u003c/p\u003e \u003cp\u003eRecent artificial intelligence research has seen remarkable progress, particularly with the advent of foundation models based on transformer architecture. Models like BERT, GPT employ self-attention to weigh and distill the complex interplays among thousands of words within the sentence contexts (Floridi and Chiriatti 2020; Devlin et al. \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). This development is highly inspirational systems biology, where biomolecules, akin to words, engage in intricate communications. The potential to learn comprehensive molecular interactome within a foundation model framework suggests that downstream biological inferences could be conducted, analogous to the fine-tuning or prompting techniques employed in large language models. Indeed, a multitude of cell foundation models (CFMs) have recently emerged, trained on extensive single-cell RNA sequencing (scRNA-seq) datasets. Models such as scBERT (F. Yang et al. \u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e2022\u003c/span\u003e), Geneformer (Theodoris et al. \u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e2023\u003c/span\u003e), GeneCompass (X. Yang et al. \u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e2023\u003c/span\u003e), and scFoundation (Hao et al. \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e2023\u003c/span\u003e) are primarily based on the BERT-like bidirectional transformer architecture, while scGPT adopts a GPT-like generative transformer approach (Cui et al. \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). These CFMs have demonstrated a variety of applications in downstream tasks, including cell type annotation, single-cell perturbation inference, target gene prediction, and drug response forecasting, showcasing their versatility and potential in advancing single-cell biology research.\u003c/p\u003e \u003cp\u003eHowever, the potential CFMs in clinical scenarios, particularly in cancer diagnosis and prognosis, has not yet been fully explored. Theoretically CFMs could apply the intergenic interaction patterns learned from one data modality, such as scRNA-seq, to identify diseases across various other data modalities, such as bulk RNA sequencing data (bulk RNA-seq), using transfer learning. Traditional bulk RNA-seq has been a cornerstone in cancer research, with different RNA components showing distinct signatures associated with cancer development (H. Wang et al. \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Protein-coding mRNAs have shown aberrant expression profiles across various cancers, prompting the proposal of mRNA expression profiling panels for diagnostic purposes (Bareche et al. \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2018\u003c/span\u003e; Golub et al. \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e1999\u003c/span\u003e; Ma et al. \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e2020\u003c/span\u003e; Teresa Agull\u0026oacute;-Ortu\u0026ntilde;o, L\u0026oacute;pez-R\u0026iacute;os, and Paz-Ares 2010). Circular RNAs (circRNAs), noted for their differential expression in different cancer types (Shang et al. \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e2019\u003c/span\u003e; Feng et al. \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e2022\u003c/span\u003e; Xia et al. \u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e2018\u003c/span\u003e), are considered promising biomarkers (J. Li et al. \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e2020\u003c/span\u003e; Meng et al. \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e2017\u003c/span\u003e; S. Wang et al. \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e2021\u003c/span\u003e; Wen, Zhou, and Gu 2020; H. Zhang et al. \u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). Other RNA species, such as long noncoding RNAs (lncRNAs) and micro-RNAs (miRNAs), have also been implicated in signaling specific cancers (Shen \u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e2020\u003c/span\u003e; Hua, Chen, and He 2019; Loewen et al. \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e2014\u003c/span\u003e; Y. Wang et al. \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e2015\u003c/span\u003e; Xu et al. \u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e2020\u003c/span\u003e; Zhan et al. \u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e2020\u003c/span\u003e; Giulietti et al. \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2018\u003c/span\u003e; Wu et al. \u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e2021\u003c/span\u003e; Iorio and Croce 2012; Lu et al. \u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e2005\u003c/span\u003e; Valihrach, Androvic, and Kubista 2020; Verduci et al. \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e2019\u003c/span\u003e). However, the traditional focus has been on identifying a limited set of biomarkers representing only a fraction of cancer subtypes. These marker-gene-based approaches do not enable a comprehensive assessment of the entire spectrum of cancers. The heterogeneous nature of oncogenesis further complicates the representation of all cancer subtypes by biomarker panels. Recently, efforts have also been paid to employ machine learning or deep learning techniques to analyze bulk RNA data, in order to improve tumor classification and prognosis (L. Zhang et al. 2018; Bostanci et al. \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2023\u003c/span\u003e). However, these applications are currently confined to specific cancer types and rely on end-to-end training with limited datasets.\u003c/p\u003e \u003cp\u003eIn this study, we explore the utilization of CFM for the first time in the context of cancer diagnostics and prognostics, leveraging bulk RNA-seq data. We constructed a novel foundation model, the GeneBag, based on transformer encoder architecture, capable of concurrently processing all genes with an assumption of random gene order and continuous expression values. Initially trained on an extensive dataset of single-cell transcriptomics, the model was subsequently retrained using bulk RNA-seq data. We then evaluated its performance across a series of downstream tasks of cancer diagnosis and prognosis.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eIn general, cell foundation models aim to capitalize on transformer structures to discern gene-gene interactions from transcriptomic data, in a manner akin to language learning. However, unlike the language sentences that depend on strict sequences of discrete word tokens for conveying precise meaning, transcriptomic data typically regard genes as an unordered set, with gene expressions represented as continuous scalar values (F. Yang et al. \u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). In this study, we intentionally designed the GeneBag model, a CFM that adapts to these unique characteristics of transcriptomic data. We employed a bidirectional encoder, substituting the positional embeddings with gene id embeddings (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). The model's indifference to the sequence of gene tokens was reinforced through iterative shuffling during both pretraining and fine-tuning stages (Methods). Continuous expression values, via logarithmic transformation of the raw read counts, were embedded for each gene to mirror the quantitative spectrum of transcriptome (methods). GeneBag is constructed based on the Longformer architecture, which supports longer context length (Beltagy, Peters, and Cohan 2020). This enabled the comprehensive analysis of the complete gene list, encompassing 17,930 genes across all datasets in this study, in a single operation (methods).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003ePretraining and Evaluation with Single-Cell RNA Sequencing Data\u003c/h2\u003e \u003cp\u003eGeneBag underwent initial pretraining on 1.3\u0026nbsp;million human single-cell RNA sequencing (scRNA-seq) data points from PanglaoDB (Franz\u0026eacute;n, Gan, and Bj\u0026ouml;rkegren \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e2019\u003c/span\u003e), employing a strategy of random masking of either gene identifiers or expression values (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). This pre-training phase endowed the model with the capability to impute expression values for any random genes with an accuracy of 94.9% (Pearson correlation coefficient), indicative of the model's adept learning of gene-gene interactions at the single-cell level.\u003c/p\u003e \u003cp\u003eOur initial evaluation of GeneBag focused on the task of single-cell annotation, utilizing the Zheng68K dataset as a benchmark (Zheng et al. \u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). This dataset, comprising blood mononuclear cells of highly similar subtypes, presents a great challenge to cell annotation (F. Yang et al. \u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Post fine-tuning, GeneBag achieved an overall accuracy of 71.33% and an F1-score of 0.61. The cell type annotation closely mirrored the ground truth, as visualized on the T-SNE plot (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eA). The confusion matrix (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eB) revealed that the majority of cell types attained a high degree of accuracy, ranging from 65.95 to 96.94%, with the exception of CD4+/CD45RA+/CD25- Na\u0026iuml;ve T, which registered a lower accuracy of 40.57%. Notably, the model demonstrated a significantly elevated accuracy of 88.94% for CD4+/CD25 T Reg, significantly outperforming the previous models(F. Yang et al. \u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). GeneBag also separates CD8\u0026thinsp;+\u0026thinsp;Cytotoxic T cells and CD8+/CD45RA\u0026thinsp;+\u0026thinsp;Naive Cytotoxic cells with relatively high accuracies (65.95% and 88.42% respectively). This is noteworthy as these cell types have traditionally been considered challenging to discern due to their marked similarities(Zheng et al. \u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e2017\u003c/span\u003e)(F. Yang et al. \u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e2022\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eTransfer Learning of the GeneBag Model on Bulk RNA-seq Data\u003c/h3\u003e\n\u003cp\u003eIn clinical practice, tissue-level bulk RNA sequencing is a widely utilized and well-established diagnostic method. Should the single cell foundation model successfully integrate bulk RNA-seq data, it would pave the way for numerous downstream applications in diagnostics and prognostics. Attention based foundation models have demonstrated remarkable capabilities of multimodal inferences through transfer learning, in domains such as natural language processing and computer vision (Zhu et al. \u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e2020\u003c/span\u003e; Vaswani et al. \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). Motivated by this, we retrained GeneBag on a mixed dataset, integrating 100k single cell samples from PanglaoDB and 19k bulk RNA-seq samples from the Genotype-Tissue Expression (GTEx) project (Lonsdale et al. \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e2013\u003c/span\u003e; Wilks et al. \u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e2021\u003c/span\u003e) (see Methods). This hybrid retraining strategy facilitated the model's adaptation to the nuances of bulk RNA-seq data, while minimally impacting its proficiency in the single cell context, as evidenced by the overall mask prediction accuracy of 90.51%.\u003c/p\u003e\n\u003ch3\u003eTissue and Tumor Type Classification utilizing Bulk RNA-seq\u003c/h3\u003e\n\u003cp\u003eThe retrained GeneBag model was first evaluated for its capability to classify various tissues and tumor types using bulk RNA-seq data. The primary inquiry was whether the model could learn to tell the intrinsic differences between normal and malignant stages, even in previously unseen tissue types. To examine this, we selected a diverse range of ten tissue types\u0026mdash;encompassing breast, colon, esophagus, kidney, liver, lung, prostate, stomach, thyroid, and uterus\u0026mdash;where both normal (from GTEx) and tumorous (from TCGA (Wilks et al. \u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e2021\u003c/span\u003e)) samples were available. These samples were thus combined into a single training set, labeled as either normal (5,397 instances) or tumorous (6,163 instances), respectively. GeneBag underwent fine-tuning on this training set. Subsequently, the model's efficacy of separating tumor from normal tissues was evaluated on tissue types (from GTEx or TCGA) not represented in the training set. The zero-shot learning prediction accuracy (on unseen tissue types) achieved an overall accuracy of 96.17%, with a classification specificity of 99.76% and a sensitivity of 85.53% for identifying tumors (Supplementary 2 Tables: Table \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e). These results suggest that GeneBag has learned to generalize the fundamental characteristics that distinguish between normal and tumorous tissues, irrespective of the considerable variations across different tissue profiles. The only tissue marking a poor performance in this test is the Bone Marrow, with an accuracy of 15.69% (Supplementary 2 Tables: Table \u003cspan refid=\"MOESM2\" class=\"InternalRef\"\u003eS2\u003c/span\u003e). This may be mainly because there are a large number of stem cells in the bone marrow, and the cell division characteristics of these stem cells are similar to those of tumor cells.\u003c/p\u003e \u003cp\u003eSince most cancer types have been extensively studied, a more critical question is whether a CFM, after thorough fine-tuning, can accurately classify a comprehensive range of tissue and tumor types. To address this, we curated a broad bulk RNA-seq dataset, consisting of 28 distinct normal tissue (NT) types, 7 paracancerous tissue (PT) types, and 31 tumor (TM) types. We divided the samples in each type into training and testing sets with an 8:2 ratio. GeneBag was then subjected to fine-tuning on the training dataset, followed by evaluation on the reserved test set. The output embeddings from the classification network can be visualized using UMAP (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eA). It can be seen that the data points of the 66 NT, PT, and TM types formed compact clusters within the same type, while being almost distinctly separated across different types (Supplementary 1 Figures: Fig. \u003cspan refid=\"MOESM2\" class=\"InternalRef\"\u003eS2\u003c/span\u003e). The overall classification accuracy is 97.23%. Upon closer examination, NT types tend to gravitate towards the upper right direction, separated from the TM types distributed at the lower left corner, and PT samples are interspersed in between (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eA). Notably, NT and TM from the same tissue type (in the same color-code, in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eA) are distantly separeted, exhibiting a similar diagonal projection, contrasting with previous studies where NT and TM of the same tissue were in closer proximity (Prada-Luengo et al. \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e2023\u003c/span\u003e). When examining the relationships among related tissues, we found that in normal tissues, the colon and small intestine are closely aligned, mirroring their anatomical proximity, yet they are notably distant from the esophagus and stomach, indicating significant developmental divergence among these clusters (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eB). Interestingly, breast samples partially intermingle with adipose tissue, hinting at a shared developmental lineage (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eB). In the realm of tumor tissues, three subtypes of kidney tumors: chromophobe, renal papillary cell carcinoma, and renal clear cell carcinoma, although converging into the same subspace, are clearly demarcated from one another and are also far removed from the kidney paracancerous tissue (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eC). The model's capacity for subtype classification is further exemplified in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eD, where a total of 15 diverse tumor types are largely distinguishable within a small area of the 2D UMAP. Notably, testicular germ cell tumor, cervical squamous cell carcinoma and endocervical adenocarcinoma (TGCT and CESC) are observed to be in close proximity, suggesting a shared biological signature (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eD). In summary, the developmental relatedness is reflected in the classification embeddings, yet even highly similar subtypes can be effectively segregated.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eTo analytically evaluate the classification performance, we generated a confusion matrix encompassing the 40 tumor types (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e). The predicted assignments were found to align closely with the actual labels, achieving an overall accuracy of 93.62%. This is a state-of-art (SOTA) performance across nearly all commonly studied cancer types (Supplementary 1 Figures: Fig. \u003cspan refid=\"MOESM3\" class=\"InternalRef\"\u003eS3\u003c/span\u003e), when compared to previous studies. Particularly in Bladder Urothelial Carcinoma (BLAC) and Stomach adenocarcinoma (STAD), which have been challenging to discern with bulk RNA-seq in previous studies (\u0026lt;\u0026thinsp;0.45 for BLAC and \u0026lt;\u0026thinsp;0.35 for STAD) (Prada-Luengo et al. \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e2023\u003c/span\u003e; Vivian et al. \u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). GeneBag attained accuracies of 0.93 for BLAC and 0.90 for STAD, respectively. The overall results suggest that foundation model based approaches, such as GeneBag, have great potential in simultaneous classification of a wide range of tumor types. On the other hand, it seems that the current version of GeneBag has poor ability to recognize metastatic or recurrent tumors. All the metastatic or recurrent cases in BRCA, GBM, LGG, OV, TGCT, and THCA were predominantly categorized under the corresponding primary tumor categories, with the only exception of skin cutaneous melanoma (SKCM), whose metastatic samples were correctly labeled as the given type in 86% of cases (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e). This observation may imply either a lack of distinct molecular markers for tumor recurrences and metastasis, or there is a limitation of resolution in the current model to differentiate between primary and recurrent tumors.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e\n\u003ch3\u003eStaging and Survival Prediction Utilizing Bulk RNA-Seq\u003c/h3\u003e\n\u003cp\u003eBeyond facilitating cancer diagnosis, there is a significant interest in leveraging bulk RNA-seq data for prognostic inferences such as cancer staging and survival prediction. Accurate staging is essential for assessing disease severity and formulating treatment plans. However, the subjectivity inherent in cancer staging can lead to variability in results, influenced by factors such as inter-observer variation, tumor characteristics, and diagnostic methodologies. It has been reported that staging concordance rates can range widely, from 58\u0026ndash;86% (Dolly et al. \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2020\u003c/span\u003e) (Gwon et al. \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2024\u003c/span\u003e; Plichta et al. 2019). In this study, we explored the potential of using bulk RNA-seq to classify cancer staging across various cancer types. We included all tumor samples with available staging information and utilized paracancerous samples as a baseline stage 0 for our fine-tuning experiments. As depicted in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eA, a clear correlation was observed between the predicted and actual stages, with an overall Pearson correlation coefficient of 0.685. This correlation is notably high, especially considering the inherent variability in staging assessments and the diverse range of cancer types included in this analysis. This result underscores the promise of the foundation model in objective and precise cancer stage estimation through the analysis of bulk RNA-seq data.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eWe further assessed the model\u0026rsquo;s capability to predict the survival rate through varying time spans of 1, 3 and 5 years. The TCGA samples were categorized based on two conditions: 1, those who succumbed to the disease within a given time span, post-initial cancer diagnosis; 2, those who survived beyond the given time span or were confirmed alive after the given time span post the initial diagnosis. The fine-tuned model achieved an overall accuracy of 74.77% \u0026minus;\u0026thinsp;75.09%, in predicting survival rates (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eB, Supplementary 2 Tables: Table \u003cspan refid=\"MOESM3\" class=\"InternalRef\"\u003eS3\u003c/span\u003e). Adjusting the prediction threshold, defined by the ratio of softmax outputs from the binary classifier network, the model demonstrated an overall AUC (Area Under the Curve) of 0.7698, 0.8281 and 0.8042 for 1-, 3- and 5-year, respectively (Supplementary 1 Figures: Fig. S4A). To gain further insights in individual disease, we focused on the analysis of 5-year survival. It is found that the accuracy of survival rate predictions varied significantly across different cancer types. Notably, Esophageal Adenocarcinoma (ESCA, 93.75%), Glioblastoma Multiforme (GBM, 100%), Kidney Clear Cell Carcinoma (KICH, 90%), Mesothelioma (MESO, 100%), Pancreatic Adenocarcinoma (PAAD, 90%), Stomach Adenocarcinoma (STAD, 92.59%), and Testicular Germ Cell Tumors (TGCT, 92.31%) showed accuracies of 5-year survival prediction exceeding 90% (Supplementary 1 Figures: Fig. S4B). In contrast, certain cancer types such as Head and Neck Squamous Cell Carcinoma (HNSC, 64.58%), Kidney Renal Clear Cell Carcinoma (KIRC, 66.18%), Lung Squamous Cell Carcinoma (LUSC, 66.38%), and Ovarian Cancer (OV, 57.63%) were found to have comparatively poor survival predictions (Supplementary 2 Tables: Table S4).\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn the realm of leveraging AI technologies for large-scale omics data analysis, previous studies have predominantly employed end-to-end training network approaches. These methods, while valuable, encounter limitations due to constraints such as limited labeled data and a narrow scope of application. This study, to our knowledge, is the first to make use of a Cell Foundation Model for addressing pivotal questions in cancer diagnosis and prognosis using bulk RNA-seq data. Despite having a modest number of parameters, GeneBag has demonstrated exceptional capabilities in classifying a diverse array of tissue and cancer types with high accuracy, as well as in forecasting cancer stages and 5-year survival rates. Larger models trained on more expansive single-cell data corpora would certainly further amplify performance. Moreover, with fine-tuning, foundation models may be able to address more complex questions, such as forecasting the effectiveness of diverse treatments or customizing personalized therapeutic strategies based on the analyses of bulk RNA-seq data. To accomplish this, it is essential to have cohort data that comprehensively records treatments and their associated outcomes, enabling CFMs to discern the relationship between therapeutic interventions and RNA profiles.\u003c/p\u003e \u003cp\u003eTraditionally, pathological testing through standard biopsy involves sectioning, staining, and microscopic examination of tissue by pathologists to ascertain cancer type and grade, a process prone to inter-observer variability. Furthermore, subtle distinctions between tumor types might not be apparent through morphological changes alone. Conversely, a CFM-based approach with biopsy bulk RNA-seq offers high-throughput, objective screening across a wide range of cancer types, and potentially with greater sensitivity and specificity due to the utilization of comprehensive transcriptomic data. Additionally, CFMs could enable the inclusion of new inferences in a single examination, such as prognostic assessments and precision medicine strategies. Ultimately, it is anticipated that such novel, foundation model-driven and RNA profiling based diagnostic approaches will complement traditional pathology, offering supplementary information to bolster clinical decision-making. However, the integration of CFMs into clinical practice, like any AI tool, necessitates stringent validation and regulatory approval to ensure safety and efficacy.\u003c/p\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003cdiv id=\"Sec9\" class=\"Section3\"\u003e \u003c/div\u003e \u003c/div\u003e\n\n \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003cdiv id=\"Sec12\" class=\"Section3\"\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec20\" class=\"Section2\"\u003e \u003cdiv id=\"Sec21\" class=\"Section3\"\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec22\" class=\"Section2\"\u003e \u003cdiv id=\"Sec23\" class=\"Section3\"\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec24\" class=\"Section2\"\u003e \u003cdiv id=\"Sec25\" class=\"Section3\"\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec26\" class=\"Section3\"\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Methods","content":"\u003ch2\u003eOverview\u003c/h2\u003e\n\u003cp\u003eThe GeneBag Model is a modified BERT (Bidirectional Encoder Representations from Transformers) model. It processes input sequences from single-cell or bulk tissue RNA-Seq data through an encoder with multi-head attention mechanism. The encoder output is then processed differently during the pre-training and fine-tuning stage. In pre-training (and also in retraining), it performs a fill-mask task, using a decoder to predict expression values for masked genes. This trains both the encoder and decoder. During fine-tuning, a classifier is trained by applying it to the entire encoder output, adapting the model for specific prediction about each sample.\u003c/p\u003e\n\u003cp\u003eWhile this process shares similarities with other language models used in gene expression analysis, GeneBag introduces three key innovations. 1. GeneBag doesn\u0026apos;t rely on fixed gene positions or position encoding, only requiring alignment between gene tokens and expression values. 2. Unlike binned expression, GeneBag directly encodes and decodes raw expression data, preserving the full granularity of the biological information. 3. By incorporating the Longformer attention module, GeneBag can process input sequences containing full gene lists.\u003c/p\u003e\n\u003ch3\u003eInput Embedding\u003c/h3\u003e\n\u003cp\u003eWe collected a huge RNA-Seq dataset and organized it into paired sequences. In the dataset, each sample consists of a gene ID sequence and its corresponding expression value sequence. We incorporated special tokens (\u003cem\u003e\u0026lt;\u0026thinsp;cls\u0026gt;, \u0026lt;pad\u0026gt;\u003c/em\u003e, and \u0026lt;\u0026thinsp;\u003cem\u003esep\u003c/em\u003e\u0026gt;) at the beginning and end of sequences, similar to other language models. All gene IDs and special tokens were tokenized as integers using a common vocabulary. Unlike other models, genes don\u0026apos;t have fixed positions in our sequences. However, gene ID tokens remain aligned with their corresponding expression values in the paired sequences, maintaining crucial gene-expression relationships. Gene ID tokens were converted to gene embeddings of dimension \u003cem\u003ed_model\u003c/em\u003e using PyTorch torch.nn.embedding. Expression values were directly encoded into expression embeddings of the same dimension \u003cem\u003ed_model\u003c/em\u003e using a series of cosine and sine functions. This deterministic encoding allows for precise representation of continuous data. The gene and expression embeddings were then added together to form the input (1) for the Longformer encoder.\u003c/p\u003e\n\u003ch2\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAu8AAAB+CAYAAACUJv+eAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAFiUAABYlAUlSJPAAAC62SURBVHhe7d15XFRl/z/+1wygLIKIsoMioIKgoCAq7uKCmqam5cPSNg21svtute5vqWV3e7Sofcz0Y+VeuaagYqggIrIr+67sMOwDwzbX748PnJ8zYGl3tzn6ej4e83jIdZ05M86cmXmd67zPdWRCCAEiIiIiIrrnybUbiIiIiIjo3sTwTkRERESkIxjeiYiIiIh0BMM7EREREZGOYHgnIiIiItIRDO9ERERERDqC4Z2IiIiISEcwvBMRERER6QiGdyIiIiIiHcHwTkRERESkIxjeiYiIiIh0BMM7EREREZGOYHgnIiIiItIRDO9ERERERDqC4Z2IiIiISEcwvBMRERER6QiGdyIiIiIiHcHwTkRERESkIxjeiYiIiIh0BMM7EREREZGOYHgnIiIiItIRDO9ERERERDqC4Z2IiIiISEcwvBMRERER6QiGdyIiIiIiHcHwTkRERESkIxjeiYiIiIh0BMM7EREREZGOYHgnIiIiItIRDO9ERERERDqC4Z2IiIiISEcwvBMRERER6QiGdyIiIiIiHSETQgjtRl1x7NgxhIeHo6CgAE1NTRg4cCDMzMzwz3/+E9bW1tqLE/0pzc3NOHXqFE6fPo26ujro6enB2NgYzs7OWLZsGaqrq7Fv3z68/fbb0NPT0747PeCSkpJw8OBB5Ofno76+Hs7Ozujduzc6v3rb2tpQUlKCkpISrFixAo888ghkMpn2aoiI6C5QKBQIDAxEcHAwxo8fr919T9Dp8K5UKlFdXY2ZM2dCqVTil19+wcCBA2Fubg65/N45qCCEwP79+2Fvb4+JEydqd9M9LC8vD2+++SZUKhVWrVqFcePGwdTUFEIIXLhwAb/++ivOnTuHefPm4e2339a+O90HhBD/UZhWKpWorKzEsmXLEB8fj9OnT8PR0VEjvBcXF+OVV17BokWL8Oqrr/5Hj0dERH/emjVr0KdPH7z//vvaXfeMeyfh/gkmJiZQq9WoqqrC5MmT4ePjAwsLi3squANAWVkZ3nzzTWRlZWl30T0sLi4OgYGB6N27N3788UcEBgbC1NQUACCTyTBp0iS4uLggNjYW/v7+2nen+8T27dtRUFCg3XzbTExMYGdnh8LCQvj7+2Ps2LFwdHRE//790b9/fzg7O2P8+PEICgrCwIEDte9ORER3SWRkJPbv34+XX35Zu0uiUCiwZs0ayGQyREZGanffFfdWyv0TEhISoFAoMGrUKO2ue0Z+fj7Ky8sxevRo7S66R+Xm5uKJJ57AwIED8cUXX0ihXdvUqVPh4eHB0HUfi4yMRFVVlXbzHUlKSsL169e77OS1tbVJ/+7ZsydsbW056k5E9DfZvHkz1q1bh759+2p3AR3f5Y8//jj69Omj3XVX6Xx4j4uLg1wux5gxY7S7oFKpUFJSotFWUlKC4uJijbZO1dXVUCgU0t9KpbLbEbfm5mYUFhZqN9/SxYsXMWjQIAY8HdHS0oJ169ahqKgIH374IYyMjLQXkRgYGMDLywt2dnbaXQCAyspK5OXlaTejpqZGIxAKIVBQUIDKykqN5bpTWlra7XZJ/x36+vr/caCOioqCTCaDj4+P1FZdXY0zZ85IAd7FxQX29vY33ev/tsWCggKpxEatVqOgoAAtLS0ay5WVlUGpVEp/V1dXd/sdVVxcjNbWVo2/6+rqNJYhInoQKRQKHDhwAM8++6x2F9AxkPPII4/go48+wqxZs7S77yqdrnlvaWnBQw89hKysLKSnp6Nnz55SX3V1NQ4ePIiCggL0798fjzzyCA4fPoxevXohPz8fjY2NWL9+PQwMDAAAOTk5CAkJQXZ2NgICAmBhYYFr165BT08PWVlZeO2119CvXz8UFRUhNDQUxcXF6NGjB15//XXph/348eOIjo7Gpk2boFQqERwcjJKSEoSGhqJHjx6YOnUqBgwYgLVr1/5uIKS/19mzZxEYGIhly5Zhx44dvxvcFAoF4uLiMGPGDI32jIwM/Pjjj7CxsUGfPn1QWFiIlStXwsLCAjk5OQgNDUVGRgYCAwMxZMgQhIaGwtLSEleuXIGzszNWrVrV5XETExNx4MABDBgwAEZGRqiqqsLKlSvRq1cvjeXor/XMM89g7dq18Pb21u66bcuXL8eZM2dw8eJFODs7AwB++uknNDY2Yvny5V3ea3S831FRUTA2NkZqaiqeffZZREdHQwiBy5cvIygoCN7e3oiOjkZCQgKSk5Px1FNPobKyEmVlZWhoaEBdXR1effVVGBoa4tSpU8jJyUFaWhpefPFFxMTEoLW1FYmJiZg/fz6mTJmi/RSIiB4Y+/fvx65duxAaGqrdBQDYtm0bxowZAy8vL0RGRmLChAmIiIj4e05qFTqspKREmJubi8cff1y7S+zbt0/ExMSIo0ePCnt7e7F+/XqRl5cn2tvbRWZmphgyZIgICwuTlv/yyy9FSUmJ2LRpk3B0dBQ7d+4UKpVKtLW1iccff1z84x//EEIIsXnzZpGbmyt+/fVXYWdnJxoaGqR1zJs3T8yfP1+o1WqhVqtFfX29yMvLE05OTuLzzz8XSqVSY/l7TVtbm3bTXaVWq7Wb/lB0dLQIDg4WX3311R/evvzyS/HVV1+JnJwc7dVoWL58uQAgTp8+rd11W8LDw4Wvr684dOiQ9Jru2rVL7N69W4iOba20tFRs2LBBODs7i23btomKigqhVqtFSEiIGDRokLh+/brGOo8cOSJ8fX1FWFiYaG9vl9Zz4sQJjeXuFZ2fgb9T5+v0n3r66adFQkKCdvNtq6ysFCNHjhS2trbiww8/FB9//LFYs2aNcHFxERcuXNBeXAghRG5urggODhaVlZVCCCGCgoKEp6enSEpKEiEhIQKA+P7770VTU5MIDg4WSqVSLFmyRHh6eoqQkBDR1tYmamtrhb+/v9i1a5eoqqoSX3zxhWhvbxfDhg0TAQEBIiUlRSQmJgq5XC7ee++9v/396k57e/vf/rz+qu2IiO5tb731lvjoo4+0m7sVEREhAIiIiAjtrrtCp8tmUlJSUFNTgwkTJmi0Nzc3o66uDm5ubrh+/Tra2towd+5cODk5QS6XQyaTQQghHS5WKBQwMDCAjY0NcnNz4erqiiVLlqBnz57Q09ODvb09wsPDkZ2dDX19fQwcOBARERFwcHCQRu6rq6uRn5+PMWPGQCaTQSaTSaP8crkc/v7+MDY2homJicZzbWpqQm5uLvLy8pCTkyPdsrOzkZ6eLj3/P6uxsREXL17Er7/+ivr6eu1uAEBrayt++uknXL9+XbvrjlVVVSE0NBTnz5+X2srKyhASEoKoqCiNZbVVVFRgz549aGho0O66JQMDA/Ts2RM9evT4w1vPnj1haGgIfX197dVIVCoVYmJiYG5ujiFDhmh3/6Hi4mI888wzmDNnDhYsWAA9PT2oVCqUl5fD0dERCoUCpqamsLS0RGFhIYyNjfHwww+jX79+kMlkkMvlaG5uhkqlktaZmZmJ5557Dk8//TQCAgIgl8uhVCpRU1MDKysrjcevrq5GTk4O8vPzkZ2dLW1PWVlZyMjI6FJGdqcqKytx+vRpnD9//pbbZXV1Nfbu3XvL7e1O5Ofn4/jx40hKSpLacnJycOTIEaSnp2ssqy0zMxNHjhxBe3u7dtcd0dPTQ48ePbSbb1thYSHS09Mxd+5cTJ8+HQEBAXjooYfg6ekpjcJrO3fuHKZPny7VXTY0NMDS0hLu7u5wd3fHjh07MH/+fGRkZGDAgAEwNjZGSUkJRo8ejRkzZkBPTw8mJiawsbFBaGgoYmNj4ePjg4aGBlRVVWHKlClwd3eHnZ0dvv32W43R/85Sne6+kzIyMpCbm4vm5matZ3xnSktLceLECelIQndKS0tx4MABjc/Cn5WVlYVjx44hIyMD6Pg/XrlyBceOHUNNTY324hpCQ0ORkJBwy+dJRPeHuLg4eHp6ajffk3Q6vJ87dw4GBgYYOXKkdhdmzZqFXr16ITo6Gr6+vhq1prm5uaioqJDeJAMDAyxYsADNzc2Ij4/H5MmTNcpaysvL0djYCAMDAyxevBgtLS04dOgQFi1aJP2op6Wloaqqqsvhk5iYGPTo0QPDhg3TaO9UVVWFnTt3ws/PD5988gnCwsJw6tQpnDhxAl9++SXmzp2L0tJS7bvdNqVSiSNHjuD111/XqIm92e7du9He3g4nJyftrjtWVVWFjz76CJs3b5Z+7MrLy7Fp0yb88MMPv/sDaGVlBTs7O2zbtu2WwVDbyJEjsXr1agQFBd3WbeXKlejfv7/2aiTl5eWorq7G4MGD0bt3b+1upKSk4NNPP8X777+PTZs2Yf369di+fTtqa2sBADt37kReXh5sbW1x7tw5HD58GJs3b4alpSUmTpwIAwMDzJo1C0qlEsnJyZg3b55GAE9JSUGPHj00ap+3bNkChUKB3r17Izw8HL/88gs2b94Md3d3+Pr6SssBQEFBATZt2oQRI0Zg//79OH36NEJDQ3H8+HG88cYbePLJJ3/3PfgjNTU1+PTTT7Fly5Zu19PW1oYtW7bA1dUVZmZm2t13rLCwEK+88gpOnTolPV5mZibWrFmDy5cvd/scOrm5uUnlc7+3HDrOOTh69CjefvttbNiwARs2bMDGjRuxceNGxMbG4quvvsK7774r9a1fvx4ffPABKioqtFfVRWJiIhobG/Hwww9jxIgRGDlyJKZOnYrRo0fDxsYG6BhwyM7Olu4zfvx4eHh4AB117klJSZgwYQL09fUxYMAAPPPMMzAzM4OdnR0mTZqE7OxsFBUVITAwUArhzc3NKC0tRW1tLTw8PODj4yN9H3UuZ2lpiWeffVbjM9HU1IRTp05h7NixeO2113DmzBmcOnUKISEh2LVrFwICAhAbGyst/2dUVlbinXfewZ49e7S7AAD19fXYunUrvL29/5ISw85Sobi4OAghoFKp8Ntvv2HVqlV/eDKyn58fjhw5wtnCiO5zCoXiL/nduht0NrwLIXDp0iU4Ozt3CWM9e/aURjlTUlK6BOoLFy5g4MCBUlg1MzODjY0NMjIyUF9frxGI6urqkJqaChsbGwwYMAAWFhaIjIxEY2MjJk2aJC139epVAMCIESOkts4f3WHDhsHY2Fhqv5m9vT2mTZsGmUyGt956C0FBQVizZg1eeuklbNmyBVOnTu0yWn8nLC0t4ejoCC8vL1haWmp3IzExEUlJSVi4cGG3dbd3ytXVFY6Ojhg7dqy0Pk9PT5ibm2PcuHF/+BhTpkxBU1PTLWvO/tsMDQ0hl8vRu3fvbkdb7ezsMHPmTJiYmODtt99GTk4OJkyYIAWMM2fOwMPDAy4uLjAzM4OTkxMWL16Mp556CrhpW8vLy8ONGzcwceJE6TVpb29HWFgYxo8fL62vvb0d4eHh8PPzg6OjI8zMzODq6orHH38cixcvvumZ/R9vb294eXnB3t4e//rXv7B69Wo8//zzePnll7F9+3YMHz5c+y53xNXVFVZWVhgxYoR01OlmP/30E4yNjf+ymZU8PDxgbW0Nf39/6XUaM2YMLC0tMXz48D/cnh599FFERUVJI66/x97eHh4eHhg6dKh08/DwgLm5OZydnTXahw4dCnd3d43zbG4lIiICRkZG8PT0lJ6vXC6XjswAQHR0NNLS0qSdjEGDBkn3z8rKQm1tbbc195aWlrCwsEBqairq6uqkI3/o2NFKTU3FoEGDYGtrCyMjI0RHR6NXr16/e1Spd+/eWLhwIVpbW6Wd3jVr1uDFF1/EBx98gKeffvo/DtSenp6wsrLqduAFAL7//nu4uLjAzc1Nu+tP8fT0hI2NDXx9fSGTyaTPpre3N2xtbbUX19CvXz/Mnj0b27dv73KiMBHdP/7TQYm7SWfDe1FREbKysuDt7d1tKEXHqF12drZGyFapVDh06BAWLlwIAwMDlJSUQK1WAx3TTra2tmpMO3n9+nUkJSVh/vz5Utvp06fh4OAgjaYLIRAdHY2RI0fC2NhYms2muroasbGxmDx5MtAxKlleXi49Xqfo6Gg4ODhIOyGFhYWoqKiAXC6Hm5vbLfcEtUcTtf/uFBUVhREjRnR79c89e/Zg6tSp3QbVP9Ld45WVlSEjI0Nj9h+FQoHr169LI4l/ZPr06Th27NhtHS4/deoU1q5de9u3f/zjH9KOVncsLS1hZ2eHyspKjVk5OvXp0wfDhg2TwsvSpUvh5uYmvX6VlZXw8vJCQEAARo4ciREjRmDAgAFaa/m/EfbW1laNnb3CwkJERERgwYIFkMlkKC8vR0tLCxQKBUaPHo0JEybAx8cHXl5ecHBw0FhfJ7VajfPnz2tcDCw3Nxd1dXXo06cP3Nzc/jDwduru/a2vr8f169fh6+vbpb+hoQHHjx/X+Kxo077PzbrrS0tLQ319vcaRq8TERJiYmMDR0VFj2e6YmJjAx8cHP/30U7fr7ySTyeDr64slS5bg0UcfxaOPPorFixdj0aJFcHFxwZw5c7Bo0SKp77HHHsP8+fNv+dns1NLSgsjISPj4+Gh8TxkYGEjBtLW1FefOndPY4b3Z5cuX0aNHD3h5eUn9N39voeNwr729vUYQjYuLQ01NDWbPng2ZTIa2tjbExcVh6NChfzggEBkZKZWOyWQy1NfXIycnB+g4ovGfTpNWVFSE6upqje2/U0lJCaKioqTn3Z3fey+760tJSYGenp7GQE94eDg8PT1haGiosWx3/Pz8UFtbi5iYGO0uIqK7TmfDe1paGsrKyjB69OhbXpQpKSlJGqnsFBMTg6qqKsyYMUOq3xVCQAiB2NhYODo6avzIHjp0CAMGDMDjjz8utRUUFMDBwUEKcE1NTYiKisLEiRPR1taGX3/9FQBw48YNFBcXw8/PD+gYnT9//rzGD5LouFKnl5cXysrKcP36dezduxfl5eUAgCVLlkBPTw9FRUUIDg7GJ598gpycHPz888/46quv8Nlnn6G4uBgnTpzA1q1bsXHjRty4cUNaf1VVFQoKClBdXY1vv/0WH374oVTbXl5ejuzs7G5/QBsbG3Hw4EF8/fXX+Prrr3H06FHpR7G+vh779+/HN998g82bN+Ps2bNSkOish715xCw+Ph7m5ua3PVWmu7s7ysrKNP4ftzJ48GDMmzfvtm9z58793ZE2mUyG5cuXIzU1FZmZmdrdQMd7FhISAhsbG40RUgCwtrbudvaXiooKJCYmSq/hpUuXMGzYMFhYWEjLHDt2DPb29hgzZgzi4+MRFxcHQ0NDWFpadhu2CgsLkZKSotFWW1uLy5cvY9iwYSgvL0dycjIOHDiA9vZ26Onp4dFHHwU6PhsbNmzAjz/+iPT0dPz444/44IMPsGfPHty4cQMHDhxAcHAwgoODNcqtUlNTUVVVhczMTHz99dfYvHmzdI5CZ0DqbtrMsrIyfPfdd9i6dSuCg4M1QlBRURF27NiBb775Blu2bNHYuUpMTISLi4vGPPsXLlyAu7u7xmv3e0aMGIGrV6+isbFRu+u2qNXqbnfkbkdWVhYKCwvh5+d3y5C4d+9e9O7dW6pvb25uRmhoqFSmERUVBQsLCyl41tbW4uDBg9L929vbce7cOUycOFH6LhRC4Pvvv8ecOXOkI48KhQLXrl27rVllwsPDYW9vD7lcjuLiYuzbtw+5ubkAgNmzZ0s7TpGRkfh//+//4ZdffsG1a9fwww8/4N1338XRo0eRl5eHffv24eOPP8b//M//aNTJx8fHQ6lUIiEhAcHBwfjuu++k/vj4eJiZmaFfv37S8p0KCgqwbds2fPPNNwgODtbY/nNycvDtt99K29HNZUjR0dHw9vaWjpS0tLQgJydHo5zy98hkMnh6euK3337T7iKi+4SLi4t20z2r+9R7D1Or1Whra8Nvv/2G5uZmDB8+/JYnpJ05cwbe3t4wNzeX2q5evYr+/fvD3d0dkZGRcHJygp6eHhobG3Hp0iU0NTVJP/JXrlyRapatra2ldVhbW2vUZB8/fhy5ubnw8PDA1atXpVGpmpoa9OrVC7a2tlAqlbh48aLGYW10/BBfu3YNPXv2RFhYGD777DNERERIQbez7vrMmTNYsGABfvjhB3z66acYO3YsXnrpJZw4cQJvvvkmXF1d8fzzz+PGjRsaP+xZWVmorKzE6NGj8dxzz8HY2Bgffvgh0DHC1dbWpvH6oCO4v/HGG8jMzMTKlSsxePBg/Pvf/4ZarUZDQwNeeOEFlJeXY/Xq1Vi4cCHOnTsnnZyYkJAAFxcXjXrxyMhIuLi4dHkcIUS3752ZmRnMzMyQmpqq3dXFwIEDMW3atNu+BQQEdBsKbvb0009j8uTJeO2111BUVKTRp1ar8fPPPyMxMRFDhw7tUrK1dOlSZGdnS4FWCIG8vDzs2rULra2tkHWcLH327FmMHTtW42hIYmIihg8fDisrK8TGxsLV1RUymQyPPfYYUlJSpEP2Qgikp6dj9+7d0n07paenQ6VSSTumGzduRHl5OczNzaVygaamJkRHR2PWrFl45513EBoaiocffhjLly/He++9h+3bt2Py5MlYvXo1fv75Z42gffXqVajVakyaNAlr165FcnKyVLeclZWF3r17dyklKSwsxOrVq9G7d288//zzUKlU2Lp1K9BxQmpQUBBsbW2xZs0ajBgxQqNk6tKlSxg1apTGDnpiYiKGDRvWZVRWrVZ3uz3Z2dlBqVTeVn36X6XzuZw+fRpKpVL6nuq8tbW1obS0FF988QU+++wzTJ48Wfr/REVFYc6cOTh16hQqKiqQnJwMCwsL6OnpQa1W49ChQxqDFpWVlUhOTpZ2+AHgwIEDKCwsxCeffCLt+OXm5qKgoOAPy9fa29uRnJwMQ0NDREREYMeOHdixY4d09KNXr17Q19dHTU0NMjMzMWbMGLz66qtISEjAo48+igULFuDll1/G4cOHMXPmTKxYsQLbtm3TOME4KSkJRkZGmDhxIl588UXp3AwAuHbtWrdHltLS0vDCCy/AxcUFa9asQV5eHvbu3QshBBITE7F27Vp4enpizZo1cHBwwNmzZ6WBmStXrmgcDczJyUFtbS1GjhzZ5bW41fk2rq6ut1V+RUS6ydXVVWeue6Fz4X3btm1Ys2YN4uPj4e/vj++++06aV/1mQggolUrMmTNHozZ32rRpGDp0KH755Re0tbVJVzwsLi5GVlYWlixZgv3792P//v344Ycf8PXXX2P69Ok3rRl48cUXYW5ujj179mD37t0wMjLChg0bEBMTg5iYGIwdOxboqD+eM2cOTp48iX379mHSpEldDvUnJCRALpfj2WefxaJFi7By5UpMmTIFxsbGaGhoQEVFBYQQcHV1RXNzM4QQWL58Oezt7dHS0oLa2lpMmTJFqmFVKBQa9fWJiYkYPHiwVMrQu3dvJCYmAh07Dvr6+l1ql/fu3Yu4uDi88sorMDQ0xODBg/HFF19AT08Pu3fvRlxcHPz9/ZGUlITjx4/D19dXGhmNiorCmDFjpFCqVquRkpICHx+fLj+SZ86cwbfffqvR1qlXr14oKyvTbr4rTE1NsW/fPgwfPhxBQUH48ssvceTIERw8eBD//ve/0djYiD179mDp0qVdgury5csREBCAbdu2ISwsDLt27cKpU6ewaNEiqRyrrq4OZmZmmDp1qsZr8thjj8HU1BR79+6Fs7OzNArw0ksvwdPTE9u3b8eZM2fw3Xff4eLFi1i+fHmXUqTw8HAMGTIEzz33HBYvXoygoCCpHKPzQj7Nzc0YO3Ysrl+/DisrKyxduhRmZmaora2FTCbDww8/DGtra6hUKjQ2NmqUVF2+fBmLFy+WTvY2MTGR6gQrKiq6HHUQQuC9996DpaUlFi1aBACYOXMm3njjDQDAhg0bIJfLMXjwYERFReHy5ctSyY9KpUJaWppGXXRpaSlKSkq6LTHZsWMHwsLCpKMbnYyMjKBWq+/ql3JISAhWrVqF48ePY8SIEThz5gxef/11vPrqq3j11VexatUqBAUFYffu3QgMDNQoC3JwcMCyZctgbm6Ow4cP4+uvv4aHhwd2796N77//HjY2NtLRPHQcTTQzM8PIkSOxf/9+7Ny5E9HR0di1axfc3d2l5UpKSjBp0qQuO5za8vLyUFxcjBUrVmDRokVYtWoV5s6dC0tLSzQ3N0tlgUqlElOmTEFOTg4GDhyIRYsWwdDQEAqFAiYmJliwYAEsLCygVCrR1tYmfc+0trYiJiYGzz33HJydnWFgYAB9fX0kJCQAtzhprLW1FW+88QZGjRqFgIAACCGwdOlSrFixAkIIrFu3Dra2trCyssL58+eRlZWFCRMmQCaToaKiAmVlZfDw8JC2meTkZBgbG3eZ7aehoQEbN27sdkevV69eGhfxI6L7i7OzM65du6bdfG/SnjvyXtfc3CxUKpU0925LS4tQqVTaiwnRMQ98Y2OjdrOoqKgQaWlpGm179+4V9vb2oqCgQJSVlYmkpCTR1NSksczNmpqaRFJSksZ83JmZmdK8zJ1UKpVISkoS1dXVGu2d3nvvPTFmzBhRX18vhBCisbFR3LhxQwghxKFDh8T58+elZffu3SsCAgKkeY+vXr0q/Pz8RF5enhBCiPLycuHl5SUuXbok3ScoKEisX79e+nvdunVixowZQgghzp07J+bPn6/x/2xvbxfTpk0TGzZskNputmDBArF48WJx+vRpce7cOZGfny/1NTc3C19fXxESEiK1FRYWCj8/PxETEyO1daqtrRVVVVXazUIIIVatWiW2bdum3XzXpaeni59//ln8+OOP4uTJk6KkpER7kW7l5eWJ+Ph4UV5ert0l2tvbRVFRUbfz6hcUFIjc3FztZiGEENnZ2SIhIUEoFArtLsn8+fPFk08+KW0jlZWVQqFQCLVaLbZu3aoxx/1rr70mVq9eLS37v//7v2L27NmipaVFCCHE+fPnhZ+fnygrKxNCCFFXVyfGjx8vTp48KUTHZ2/mzJninXfeEUII8fnnn4t169ZpzMutUCiEvb29xjbRqbGxUQwfPly89dZbIiwsTERGRmp8fhITE4Wvr6/GNnb27Fnh4+PT5XoJarVaVFZWCqVSqdEuhBD19fUiMDBQJCUlaXfdlieffPKO53lvbW0VTU1NoqWlRbS3t4vm5mbR1NSkcWtubu52GxBCiIaGBpGcnCzq6uqE6Hitk5KSRGlpqfai4uWXXxb+/v5CqVSKvLw8kZKS0u3c6PX19d1uj9qOHTsmbG1tRVFRkRAdj11QUCCEECI+Pl788MMPGut/8sknxaZNm6S2999/XzzxxBPS33v27BHTpk2T3pvS0lLh6ekpvaY1NTVi1KhR4rvvvhOiY7v86quvNB4jLy9PWFpadvs9cuPGDeHm5iY+/vhjERYWJi5duiS9bkIIERoaKsaNG6fxHfzaa6+JF154ocvr1NLSIkpLS7u0CyHEhQsXRGBgoHYzEd0n9u3bJ2bOnKnd3C3O836HOufr7jxk3DnPd3dsbGy6nRWhX79+XWYxCA8Ph4eHB6ysrGBlZYXhw4ffskYVHbOSDB8+XGMkfdCgQVLdaqeePXti+PDhXUpGOsXHx2Po0KHSiKWRkREcHBxQWFiIxMREjTlH4+PjNUaPYmJipNlk0HEirbW1NVxdXXH27Fm0t7ejqqoKQ4cOBTpq1S9cuIBly5YBHSdftre3a8ygIDqmUdOuTy8tLUVbWxv09fXh4+OD6dOnY9KkSRgwYADKy8vR3NyMlJQUyOVyuLq6Ijw8HOiY1k+tVqN///64ePEi1Go1VCoVTpw4gW+//bbbOmQhBOrr67tcKv7vMGTIEDzyyCN44oknMGvWLGlqvz/i5OSEESNGdHsytVwuh52dXbcnEPfv37/La9/JxcUF3t7et6z1VigUyM/P15iZpW/fvrCwsEB0dDQaGhqkGZZUKhWSk5Mxfvx4adkrV67Azc1Nmgf/8OHDGD16NAwMDBAVFQW1Wg0hhDRym5mZifz8fDzyyCNAx1Sf2nO7q1QqGBoadnndioqKIISAsbExpk6dioCAAIwbNw59+/ZFaWkpRMdJ4A4ODjAzM0NERARw05GkyspKqTa+srISBw8evOW0g42NjdDT07vlZ/CPPPHEE3e8Lerr68PQ0BAGBgaQy+Xo0aMHDA0NNW49evTodhtAxxGNYcOGSUe0DAwMMHz4cI3yPXQc2er8njAyMoKTkxOGDh3a5agEOkaOu9setcXHx8PJyUl6zwwMDNC/f3+0tLTgyJEjGttM5/kPnducEAJJSUnSzDhCCJw4cQJTpkxBfX094uLi0NraClNTU+m7Mjk5GQ0NDdJViq2srLrMva5UKmFubt6l5K3zKECvXr0wc+ZMBAQEYMyYMTA1NZWm2L148SIGDx6M9vZ2xMfHo62tDQkJCZg8eTKSk5Ol0rhr165h69atiIuL03iMTnV1dbf1+hGRbvLz88OpU6d04gibzoX3v5parUZGRgbOnz+Pfv363dYMJ3+F+vp6nD17FnFxcdLFobKyspCWlobDhw/jueeeg7W1tRTUmpqapHKVTpcvX8aQIUOkAJCSkgIPDw9kZ2dDqVRCT08Pfn5+uH79OpqamrBz5074+PhI5Qu2traQy+UaP5R6enqYO3cuUlNT0dTUhKamJly4cEGarWPhwoXS1HUqlQpRUVHYv38/2traoFAo0KtXLyiVSumHs6amBjY2NkhLS0NdXR3kcjny8/PRt29fpKenSyfB3ay2thaNjY0ah/zp91VWVuLw4cPIz89Hnz59kJ2djaysLCQnJ2Pnzp1444034O/vL+30FhUV4caNG9K0qI2NjUhNTdUoR8nPz4ePjw/CwsJgamoKU1NTeHp6Ij8/HzU1Ndi2bRteeOEFafrJwYMHo7q6WuPERCsrK4wbNw7JyclQqVSoqanBL7/8gvPnz8PIyAjTpk1DbGwsVCoV6uvrcfToUZw5cwZCCFRXV8Pa2hqJiYkaYdHR0RHx8fHSSaRpaWlwcXFBeHh4t5/f4uLiW54AeTumTZt2T4a2lpYWRERE4Nq1a7C1tb2ji5t1p6mpCVevXsXRo0fh5OQkfSdlZGQgLCwMTz31FJRKpcb1IPLy8lBbWysNEHSeBN8Z5lUqFUpLS+Hl5YWTJ0/CysoKffv2hbOzM4qKilBaWoqdO3di/fr1Up27p6cnbty4IZ0Ej45zWzw9PZGamgqVSoWqqirs3r0bsbGxsLe3h5+fH2JjY9HU1ISamhrs3bsXly5dAjpCt729vTRlZ0tLC1QqFXr06IGUlBT07NkTbW1tKCoqgoWFxS1PSs3KypL+n0R0/3F2dsZjjz2GHTt2aHcBHSeud946Lw7aWZ4nk8kQGRmpfZf/Gr0NGzZs0G58kBQWFmLLli3o168fDA0NUVhYKNWs/zdlZ2cjOTkZvr6+sLKyksJUYWEhqqurMWjQICxbtkw6qtDc3Iz6+nrMnTtXGqUvKSnBzJkzpdk9+vfvjxs3bsDExARz5syBXC6Hl5cX8vPzER0dDXt7e6xdu1Zap4mJCZKSkiCE0PhR8vX1RV1dHaKjo5GamgoTExM89NBDMDY2xtChQyGXyxEREYH09HQYGBhg7ty5MDU1hZ2dHVpbW1FcXIz58+fD0NBQOllQpVJh+vTpMDQ0hLm5OQoLC3HlyhU89dRTkMvlGiOQcXFxyMvLw+LFi3/3aqj0/4uNjUVJSYk048iNGzek2Y7q6+sxceJEjan36uvrYWJigoCAAOjr66OpqQlKpRLz5s2TTnC0s7NDRkYGPDw8pPmxvb29ERMTg+TkZEycOBGPPfaY9BzMzc1x7NgxeHt7S0FZLpdj/PjxSEtLQ0JCArKzs+Hk5IRp06ZBX18fo0aNQmFhIWJiYpCZmQlHR0dMnz4dBgYGcHFxQWFhIQwMDDBjxgzI5XIMHDgQeXl5sLa2xrhx46CnpwcHBweEhITAxMQEkyZNgqzjSrWdTp48CVNTU42TQu8HcXFx+Omnn+Do6AiVSoXW1laNI3N3qqKiAhEREXBzc8OgQYNQXFwsbUeVlZXo06cPnn32WY0jGNXV1bCxscH48eMhl8ulHYiHHnoIBgYGMDAwQN++faUTW93c3GBgYABPT0/pO2Tu3LkaF5cyMzPDsWPHMH78eI0jD/7+/oiPj0dSUhLy8vLg7u6OCRMmQE9PD2PGjEF6ejoSEhKQk5ODIUOGYNKkSdDT04OrqyuysrJgb2+PsWPHokePHnBwcEBqaipGjx4NZ2dnyGQy2NvbY+vWrVi4cCGsra01zvVoa2vD999/jwULFnR7Mi0R3R9sbGzw/PPPY8WKFV2uz9N5kb5b3f7ofKK/kkxon91FD5TU1FRs2bIFn3/++S3Lj/4b3n//fZiYmMDd3R0jR47UGNl89913MWrUKMyaNUvjPnTvO3LkCFJSUvCvf/1Lu+u/pq2tDStXrpR2bCdNmiRtyw0NDVi3bh3++c9/6tQ0YA+67du3o729HUFBQX96Z+ROZWRkYP369QgKCoKRkZHG7DRRUVE4duwYNm7ceFe/J4no7luyZAlGjhyJ119/XbvrnvHAj7w/6CwtLVFfXy/Vzd48YvnfZGZmBoVCAXd3dzg5OUHWUS975swZqFQqLF26VPsupAMGDx6MmJgYtLS03LJ2/68mk8lgbGyMmpoa+Pn5SVO1trS0YO/evRg+fHiXKVrp3ubm5oazZ8/C1NT0d6/L8FcyNDSUrofg7+8vHfUrLy/HgQMHsGLFij9dekVEumPKlClYvXo1Ro4ceVdH0+8ER94Jra2tOHnyJDw9Pf/W0cnKykqEhYVh3rx5XQ5Xke6ora1FSEgIZs+e3WXKv7spIyMDWVlZmD179l3bKaW/TkVFBc6ePfu3fh8IIRAaGgoHBwd4enpyB5CI7gkM70REREREOoLDUUREREREOoLhnYiIiIhIRzC8ExERERHpCIZ3IiIiIiIdwfBORERERKQjGN6JiIiIiHQEwzsRERERkY5geCciIiIi0hEM70REREREOoLhnYiIiIhIRzC8ExERERHpCIZ3IiIiIiIdwfBORERERKQjGN6JiIiIiHQEwzsRERERkY5geCciIiIi0hEM70REREREOoLhnYiIiIhIRzC8ExERERHpCIZ3IiIiIiIdwfBORERERKQjdCK8BwYGYtu2bdrNREREREQPFJkQQmg33mtyc3Ph6+uL2NhYODs7a3cTERERET0QdCK8A8DHH3+M3377DaGhodpdREREREQPBJ0J7wqFAoMGDUJ4eDi8vLy0u4mIiIiI7ns6UfMOAH379sWSJUtw8OBB7S4iIiIiogeCzoR3AHjooYdw4MAB7WYiIiIiogeCzpTNdJLJZKisrETfvn21u4iIiIiI7ms6NfIOAC4uLkhLS9NuJiIiIiK67+lceHd1dUVdXZ12MxERERHRfU/nwjsAXLt2TbuJiIiIiOi+p5PhnYiIiIjoQaST4b1///7aTURERERE9z2dC+8KhQIODg7azURERERE9z2dnCoyJycHzs7O2l1ERERERPc1nRp5P3nyJFxcXBjciYiIiOiBpFPh/ddff8WMGTO0m4mIiIiIHgg6UzajUCgwaNAghIeHw8vLS7ubiIiIiOi+pzMj7zt27ICfnx+DOxERERE9sHRi5J2j7kREREREOhLeAwMDsWDBAgQFBWl3ERERERE9MHQivBMRERERkQ7VvBMRERERPegY3omIiIiIdATDOxERERGRjmB4JyIiIiLSEQzvREREREQ6guGdiIiIiEhHMLwTEREREekIhnciIiIiIh3B8E5EREREpCMY3omIiIiIdATDOxERERGRjmB4JyIiIiLSEQzvREREREQ6guGdiIiIiEhHMLwTEREREekIhnciIiIiIh3B8E5EREREpCMY3omIiIiIdATDOxERERGRjmB4JyIiIiLSEQzvREREREQ6guGdiIiIiEhHMLwTEREREekIhnciIiIiIh3B8E5EREREpCMY3omIiIiIdATDOxERERGRjmB4JyIiIiLSEf8fHOauQpDTpLwAAAAASUVORK5CYII=\"\u003e\u003c/h2\u003e\n\u003ch2\u003eEncoder with attention mechanism\u003c/h2\u003e\n\u003cp\u003eUsually the input embeddings were processed with a transformer encoder. However, standard self-attention struggles with very long sequences due to its quadratic complexity. This is problematic for transcriptome data, which can exceed 15,000 genes. To solve this, we implement Longformer attention, which uses a sparse attention pattern. This allows the model to handle longer input sequences efficiently and potentially capture broader biological contexts and interactions. Our sequence length is set to 17,932. Our Longformer encoder includes six attention heads and six encoder layers. The model\u0026apos;s dimension (\u003cem\u003ed_model\u003c/em\u003e) is set to 42.\u003c/p\u003e\n\u003ch2\u003ePre-training of GeneBag Model Based on Single Cell RNA-seq Data\u003c/h2\u003e\n\u003cp\u003eThe pre-training data consists of approximately 1.3\u0026nbsp;million single-cell RNA-Seq sequences. Each cell sequence was firstly processed with masking, where 15% of the gene expression values in the non-padding regions are randomly replaced. These masked sequences are then processed through input embedding and the Longformer encoder. The encoder output for each masked gene is fed into a multi-layer perceptron to predict the expression value. The predicted expressions are compared against the actual values using Mean Squared Error (MSE) as the loss function. Prediction accuracy is evaluated with the Pearson correlation coefficient. Other configurations include a batch size of 32, a learning rate of 1e-4, and the Adam optimizer.\u003c/p\u003e\n\u003ch2\u003eCell Type Annotation\u003c/h2\u003e\n\u003cp\u003eWe benchmarked the GeneBag model\u0026apos;s performance in cell type annotation using the Zheng68K dataset. In this dataset, the occurrence of some cell types is sparse, leading to an imbalanced distribution. To address this, we set a threshold of 6,185 cells per cell type. For types exceeding this threshold, we randomly select a subset to meet this limit. Cell types with fewer than 6,185 cells are fully included, creating a more balanced dataset. This process results in a dataset with a total of 40,828 cells (Supplementary 3 Tables: Table \u003cspan class=\"InternalRef\"\u003eS1\u003c/span\u003e). This was further divided as a train and test dataset with 8:2 ratio. A classifier with a single convolutional layer followed by three multilayer perceptrons (MLPs) was used to predict the cell type from the entire encoder output. The loss function is CrossEntropy.\u003c/p\u003e\n\u003ch2\u003eRe-training of GeneBag Model on Bulk RNA-seq\u003c/h2\u003e\n\u003cp\u003eFor training on bulk RNA-seq data, the same gene ID embedding and expression value embedding methods was used for the scRNA-seq.\u0026nbsp;The \u0026lt;\u0026thinsp;\u003cem\u003ecls\u003c/em\u003e\u0026thinsp;\u0026gt;\u0026thinsp;token was used to distinguish the datatypes of scRNA and bulk RNA. Re-training is similar to pre-training while including the bulk tissue data. We used the entire GTEx data (19,081 samples) and randomly sampled 100,000 cells from the pre-training dataset, labeled as \u0026ldquo;bulk\u0026rdquo; and \u0026ldquo;scRNA\u0026rdquo;, respectively. We merged them to create a balanced dataset. We further divided it into training/test/validation subsets in an 8/1/1 ratio.\u003c/p\u003e\n\u003ch2\u003eZero-shot Tumor Recognition\u003c/h2\u003e\n\u003cp\u003eTo assess the GeneBag model\u0026apos;s capacity to accurately identify tumor samples among previously unseen tissue types, we selected ten tissues as training set in this fine-tune task. These 10 tissues were chosen based on two criteria: (a.) their presence in both the GTEx and TCGA databases; (b.) the availability of at least 50 samples from the GTEx and TCGA-tumor datasets, as well as at least 10 samples from the TCGA-normal datasets. For testing, we included 13,684 normal and 3,798 primary tumor samples from other tissue types. Similar to cell type annotation, we used the same classifier and CrossEntropy loss function to distinguish normal and tumor samples (Supplementary 3 Tables: Table \u003cspan class=\"InternalRef\"\u003eS2\u003c/span\u003e).\u003c/p\u003e\n\u003ch2\u003eTissue and Cancer Type Classification\u003c/h2\u003e\n\u003cp\u003eThe whole dataset was composed of three distinct categories: normal tissues from GTEx, tumor tissues from TCGA-tumor and paracancerous tissues from TCGA-normal. We included only types of sample size exceeding 50. We evenly divided these samples into training and testing sets, comprising 23,654 and 5,945 samples, respectively (Supplementary 3 Tables: Table \u003cspan class=\"InternalRef\"\u003eS3\u003c/span\u003e). For confusion table analysis of tumor subtypes, 40 tumor types were analyzed. Tumor types with 5 or fewer samples were excluded. The total samples were evenly divided within each tumor type according to a 8:2 ratio, resulting in a training set of 7,987 and a test set of 2,525 samples (Supplementary 3 Tables: Table S4). The same classifier network was used except the number of output neurons were matched to the number of types.\u003c/p\u003e\n\u003ch2\u003eTumor Staging\u003c/h2\u003e\n\u003cp\u003eFor tumor staging, we focused solely on primary tumor samples for stage prediction purposes. Stage labels I, II, III and IV were used, and subtypes such as Ia, Ib, were merged into the major type. Samples using non-standard labels were excluded for clarity. Solid normal tissues, which are paracancerous in TCGA, were labeled as stage 0. The final dataset comprised a total of 7,611 samples (Supplementary 3 Tables: Table S5). We assumed the cancer stages as continuous values. The decoder to predict stages was similar to the cell type classifier, except the final layer of the MLP goes to a single neuron to estimate the tumor stage, and MSE loss was used.\u003c/p\u003e\n\u003ch2\u003eSurvival Prediction\u003c/h2\u003e\n\u003cp\u003eIn the TCGA dataset, we identified 966, 2,146 and 2,559 samples who did not survive beyond one, three and five years, respectively and 7,804, 3,281 and 1,590 individuals who lived longer than the corresponding years. For the binary classification task, we applied the same classifier architecture as mentioned before (Supplementary 3 Tables: Table S6).\u003c/p\u003e\n\u003ch2\u003eData Availability:\u003c/h2\u003e\n\u003ch2\u003ePanglao dataset\u003c/h2\u003e\n\u003cp\u003eThe Single Cell RNA-seq (scRNA) gene expression dataset was from the Panglao dataset (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://panglaodb.se/\u003c/span\u003e\u003c/span\u003e). This dataset contains 1,16,580 cells with 77 tissues, which were collected from 209 single cell datasets. In our study, only the gene IDs and expression values from the Panglao dataset were used..\u003c/p\u003e\n\u003ch2\u003eZheng68K dataset\u003c/h2\u003e\n\u003cp\u003eZheng68K (Zheng et al. \u003cspan class=\"CitationRef\"\u003e2017\u003c/span\u003e) is a well annotated PBMC dataset generally used for cell type annotation evaluation, and it consists of 11 cell types, totaling 68,450 cells (Supplementary 1 Figures: Fig. \u003cspan class=\"InternalRef\"\u003eS1\u003c/span\u003e). Among them, there are 20,757 CD8\u0026thinsp;+\u0026thinsp;Cytotoxic T cells, 16,645 CD8+/CD45RA\u0026thinsp;+\u0026thinsp;Naive Cytotoxic cells, 8,775 CD56\u0026thinsp;+\u0026thinsp;NK cells, 6,185 CD4+/CD25 T Reg cells, 5,877 CD19\u0026thinsp;+\u0026thinsp;B cells, 3,059 CD4+/CD45RO\u0026thinsp;+\u0026thinsp;Memory cells, 2,847 CD14\u0026thinsp;+\u0026thinsp;Monocyte cells, 2,095 Dendritic cells, 1,871 CD4+/CD45RA+/CD25- Naive T cells, 242 CD34\u0026thinsp;+\u0026thinsp;cells and 97 CD4\u0026thinsp;+\u0026thinsp;T Helper2 cells, accounting for 30.3%, 24.3%, 12.8%, 9.0%, 8.6%, 4.5%, 4.2%, 3.1%, 2.7%, 0.4%, and 0.1%, respectively.\u003c/p\u003e\n\u003ch2\u003eGenotype-Tissue Expression (GTEx)\u003c/h2\u003e\n\u003cp\u003eGTEx is a public database containing transcriptome gene expression data from large scale human normal tissues. We downloaded the raw gene counts from Recount3 (Wilks et al. \u003cspan class=\"CitationRef\"\u003e2021\u003c/span\u003e) platform via the build-in R packages. The total data of 31 tissue types with 19,081 samples were used in our study.\u003c/p\u003e\n\u003ch2\u003eThe Cancer Genome Atlas Program (TCGA)\u003c/h2\u003e\n\u003cp\u003eThe TCGA database is composed of RNA sequencing data of tumor biopsies, with a total of 11,348 samples. As above, the raw gene counts and meta-data files both were obtained from Recount3 (Wilks et al. \u003cspan class=\"CitationRef\"\u003e2021\u003c/span\u003e).\u003c/p\u003e\n\u003ch2\u003eData pre-processing\u003c/h2\u003e\n\u003cp\u003eRaw expression values are all read counts. For GTEx data, we removed low-expression genes and non-protein-coding genes. This results in a list of 16,874 genes. The same genes were selected for TCGA data. Log transformation was done only for GTEx and TCGA data, because the single cell data was already processed. Lastly, we performed a gene-wised normalization by calculating z-scores for all datasets (2).\u003c/p\u003e\n\u003ch2\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAo8AAAERCAYAAAAaFE3VAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAFiUAABYlAUlSJPAAAG18SURBVHhe7d13eJRV/vfx92TSO0lIAiEhgdBCE6QLSJMOIgIioCCoKLqW3QUsuyx2RVTAwqIIgkoNoNKbSAkdQk8gvZHe+7T7+eOX3A8zCTC6ggS+r+uaazfnnPueScH5zKkaRVEUhBBCCCGEsIKNZYEQQgghhBDXI+FRCCGEEEJYTcKjEEIIIYSwmoRHIYQQQghhNQmPQgghhBDCahIehRBCCCGE1SQ8CiGEEEIIq0l4FEIIIYQQVpPwKIQQQgghrCbhUQghhBBCWE3CoxBCCCGEsJqERyGEEEIIYTUJj0IIIYQQwmoSHoUQQgghhNUkPAohhBBCCKtJeBRCCCGEEFaT8CiEEEIIIawm4VEIIYQQQlhNwqMQQgghhLCahEchhBBCCGE1CY9CCCGEEMJqEh6FEEIIIYTVJDwKIYQQQgirSXgUQgghhBBWk/AohBBCCCGsJuFRCCGEEEJYTcKjEEIIIYSwmoRHIYQQQghhNQmPQgghhBDCahIehRBCCCGE1SQ8CiGEEEIIq0l4FEIIIYQQVpPwKIQQQgghrCbhUQghhBBCWE3CoxBCCCGEsJqERyGEEEIIYTUJj0IIIYQQwmoSHoUQQgghhNUkPAohhBBCCKtJeBRCCCGEEFaT8CiEEEIIIawm4VEIIYQQQlhNwqMQQgghhLCahEchhBBCCGE1CY9CCCGEEMJqEh6FEEIIIYTVJDwKIYQQQgirSXgU4gb0ej3x8fHExsaSkpJCZWWlZROVwWAgLS2N2NhYrly5csO2t8OJEydYs2YNRqPRskoVHx/P8uXLKS0ttawSQgghaiXhUYgbKC8v5+TJk7z99tsMHDiQvXv3WjZR/frrrwwZMoT33nuPI0eOUFFRYdnktomLi+OXX37hoYceQqvVmtV9/fXX7NixA4AmTZrg7e3NqlWrzNoIIYQQ1yPhUYgbcHd35+GHH6ZHjx5kZ2dz7tw5yyYA5OTkEBERwZUrV3juueeYPHkyHh4els1uC6PRyH//+18eeughvL29zeoqKyvZvHkzWVlZatmwYcO4cuUKR44cMWsrhBBC1EbCoxA3kZ+fj62tLT169CA5OdmyGr1ez9mzZ7GxsSEgIIBWrVpZNrmtTpw4QWFhId26dbOswsHBgR9//JHHH39cLdNqtQwdOpSVK1eiKIpZeyGEEMKShEchbiI/Px9XV1datmzJ1atXMRgMZvXnz5/H29ubqKgoevTogZubm1n9tcrLy687v1BRFEpKSmrcn6r5lNcOgxsMBoqLi83aVNuxYwcdOnTA3t7erNxkMlFWVoabmxt2dnZmde3atSM1NZXLly+blQshhBCWJDwKcRMXL16kXbt2hISEkJ6ebhba8vLySE9Pp0mTJhw/fpxu3bqh0WjMrgfIyMjgiy++4Mcff2Tt2rVs3LjRrD4xMZFPPvmEjRs3sm7dOtauXcvx48cB0Ol0rF+/nvfff5+dO3dy/vx5Vq9eTXh4OHPmzCEvL0+9j8Fg4Pz587Rt2/aau/9f+ZYtW/jqq6/45ptvzOoAvLy88PHxITo62rJKCCGEMCPhUdxWOTk5xMXFkZKSQlpaGlTN0bt2RbPJZLK87C8VHx9Ps2bNaNq0qVl4NJlMHD16lK5duxIdHY3RaKwR2qhavDJ58mT8/Px4+umneeqpp4iIiKCkpASq7j9jxgzCwsJ48sknadmyJS+++CJRUVEAHD9+nEaNGhEYGMgrr7xCYmIiEydOZPz48Rw9epTdu3erz5WZmUl+fj6NGzdWywB2796Nn58fbdu2ZfXq1RQVFZnVazQagoODrzunUwghhKgm4VHcVoWFhXz55ZcMGzaMM2fOQFV4/Oijj5g/fz4pKSmWl/xuer2erKwsMjIybvjIzMysdYj4Wjk5Oeowb0BAACUlJeTn5wMQHR2Nm5sbPj4+HDhwADc3N9q0aWN2vcFg4B//+Af+/v6MHTuW4uJijh07hq+vLy4uLhgMBubOnYu/vz9Dhw4FwM3NDX9/fzWIZmVl0blzZy5dukSzZs0YPHgwNjY2GI1GiouLsbH5//+Ms7KyUBQFLy8vtQygtLSU9u3bs3v3bho0aIC7u7tZPVW9j9WBXgghhLgeCY/itmratCkvv/wyBQUFlJWVARAZGUnfvn359NNP6dGjh1kY+iOys7P55ptv+PTTT1mwYEGtj88++4zPP//cbMi3NlFRUQQFBQHg7e2Nra0tWVlZ6HQ6rly5QteuXQE4efIkTZs2rRHaoqKi2L59O4GBgezatYs9e/YQHx/P1KlT0Wg0REdHs379eiZOnKhec/jwYby8vGjZsiUADz/8MAAXLlzgoYceUucrZmVlcfXqVbMFOuXl5SiKYjanUVEUxowZQ2FhIQcPHmTs2LFq3bW0Wi3l5eWWxUIIIYSZ/+1dWog/oHHjxkyZMoUFCxawb98+MjMzefTRR3F2drZs+of4+vrywgsv8NprrzFr1qxaH7Nnz+Yf//hHja1sLGVnZ1O/fn0AnJ2dCQwMJC4ujuPHj9O2bVvs7e3JzMzk0qVL9OnTx/JyoqKi8Pf3Z/jw4XTv3p1BgwYxYcIE9Z5nzpzBzc2NFi1aQFXQqw6izs7OVFZWotVqiY+PJy0tjd69e6v33rx5My1atCAkJETtQXVyckKj0aDT6dR21XMwjxw5QklJCf369UOn09VYWW0wGHBycjIrE0IIISxJeBR/iSlTppCRkcG2bdsYPnx4jdW//wtbW1s8PT3x8vK64aNevXo1NtC+lk6no7S0lICAAKgKZsHBwWzbtg29Xk/Tpk0BuHr1KgkJCbWGR2dnZ7y9vQkKCsLNzU0NyJmZmRQWFlJSUkLTpk3VFdoFBQWcPn2aBx98kMTERM6fPw/A5cuXsbGxoXnz5lA133Lbtm30798fk8nErl27APD398fGxobc3Fz1NVAVSsPDwxk2bBj29vb8+uuv6PV6s/rc3Fz1exVCCCGuR8Kj+Evk5OTQpk0bTp06dd2ta3bu3El6erpZWUpKitkCkVulqKiI3377jZ07d6LX69Wh4IYNG6LT6ejZsyd6vZ68vDy2bNmCo6Mj9vb2ZoEMoGvXrnh7e5OQkABV8zEjIyPZvHkzDg4O3H///Wg0GioqKlAUhS1bthAXF8d9993HkSNH8PHxgapeww4dOqg9g5mZmZSUlDBgwABOnz6Nq6srVPW6enp6kpiYeM2r+L8gHBMTQ79+/Th79ix2dnZmW/koikJSUhIdOnQwu04IIYSwpJ07d+5cy0Jxe8XHx5OTk3PTIdT/1cmTJ3F2dsbR0dGy6raKiIggJyeHyZMns2DBAho1alRjlXL1noe+vr5mr7esrAw7Ozt12PdWqO7V27dvH3q9nsLCQlq1aoWjoyNarZaHH36YgIAAEhIS2LBhA6mpqYSGhlJQUED9+vXVwAfg4uJCaGgoR44cISUlRT3zetCgQbi6utKgQQPs7e2JjIwkKSmJkJAQ7rvvPs6fP0+jRo3UcLlv3z769etHWFiYet/CwkJSU1Px9PSkX79+aDQaNBoNFy5coKSkRJ2PSVVvrKIoJCYm4unpSc+ePc16e3Nzc1m7di2PP/74Lf87FEIIUbdpFMuJT3XQgQMHiIqKIicnh969e9OrVy+1Lj8/n61bt5KZmYm9vT1t27ald+/e//OijD/LlStX+Pnnn3n66aepV6+eZfWf6tKlS2zfvp1nnnmm1tW2t8OBAwfIz89n+PDhaLVa3n//fX7++Wf279+vhsTCwkJ+++03ysvLGT9+PFSFxtOnT3PlyhUGDBigLmK5VcrLy9FqtWi1WnQ6HQ4ODjX+ZgwGA5WVlTg4OKDRaKisrMTe3h5bW1uzdgAlJSVUVFRgb29f688+Pz8frVaLu7s7iqJQWFiIs7Oz2jtYWVmJra2t2TC7yWSioKAAd3d3s+c8evQoS5YsYfHixWbBu3p19rX3rbZz5062bNnC559/blYuhBBCWLozEtT/KDg4mNLSUj788EP+9a9/UVhYqNY5OTnh7+/P1q1bcXR0JDAwsNZNnP8KJSUlfP311wwdOvSWB0eAsLAwGjVqxKpVqyyrbrmLFy8yd+5cPvnkEzp37qyGIH9/f+Li4vj444+5evUqVAUpOzs7Dh06pF6fnp5OUFAQqampnD17Vi2/VZycnLC3t0er1eLk5FQjOFLVm+fi4qKGOmdn51qDI4Crqys+Pj61BkeAevXqqXUajQZPT0+zgOfg4FBjfqaNjQ1eXl41nrNLly74+flx+PBhs3KtVlvjvlQNaW/fvp0pU6aYlQshhBC1qfmOWAcFBQXRpEkT/vWvf3H06FGOHTum1jk6OtKqVSueeuoppkyZQtOmTe+Y8LhlyxY8PDxo3bq1ZdUtM3LkSE6ePHnbTxLx8PBg5MiRvPvuu3h6ekLV0HTv3r3Zv38/I0eOVHvJgoODuXjxIt26dSMzMxOqtvipX78+V69eVYduRe1sbGyYMWMGv/76q/rzu5FNmzbRrl077r//fssqIYQQooa7IjyWlJSQl5fH+PHjad++fY3j13JycnBycvpTV/T+r0wmE7t372bAgAGWVbeUk5MT9913H+vXr7esuqUaNWpEx44dadu2rbriWKPREBoaSuvWrWnfvr26R6KiKGg0GgoKCsjJyVHvkZCQgI2NDQ0aNFDLRO2CgoIYO3Yshw8fxmg0WlarEhMTMZlMPP7445ZVQgghRK3uivBYVFREeXk5jRs3ZsKECezZs8fsmLWrV6/SsGFDs2uq6XQ6Dh06xO7du/ntt9/Ulb86nY6DBw+qQ38XL15k//796pFxJpOJM2fOsGfPHkpLSyksLCQiIoIdO3Zw6dIlqHpj/u2339i+fTsZGRnXPOv/zT/MycmhY8eOZuW3Q8+ePTl58uR1Vzn/1TQaDS+//DJPPfUUrVu35sKFC/z6668cOXKEsLCwP20/yLtd+/btGT58eK1D7tUaNWrEuHHjZH9HIYQQVrv+u0odcvXqVfz9/QEYNWoULi4urFmzRq2Pj483O4Wj2tWrV3nttddITEykQYMGREdHU734/NChQ5SVlbFo0SL+/e9/k5GRwf79+5kyZQrFxcVcvnyZ6Ohodu7cydy5c9m5c6caat544w1++uknzpw5g4+PD0lJScyePZuKigr1uSMjI/Hx8cHBwUEtu10CAgIoLS294VF0JSUlxMXF3fQRGxtLenp6jQ2n/1d2dna4uLhA1Vy906dP4+fnJ/Pyfic7O7sbTtOwXIQjhBBC3Mxdsdp66dKl9O3bV920+fnnn2f37t1ERETg7u7OkiVLeOWVV8yuKSsrY+zYsfTs2ZPXX38dqhZljBo1irfffpuEhASee+45HnzwQcLCwli8eDHh4eGcPHmSt99+m40bN9KzZ0+++OIL9u3bx9atW/Hx8aGoqIgBAwbQq1cvPv74Y2xsbDhz5gyPPfYYv/76q7oJ83vvvUdRUREfffSR2euiamXt8ePHSU1Nxd7eni5duhAYGEhSUhIHDhzA1taWHj160LhxY8tLrVJaWsro0aP5z3/+Q48ePSyroWpbnzVr1mAymW4YPgwGAy1btmTq1Kl/SRAWQgghxO11V4THjz76iJdeekkdejt+/Dg9evTghx9+oFWrViQnJzNixAizaxYuXMjSpUvZuXOnOqSdnp7OwIEDeeaZZxg7diyVlZWMHj2aTz/91Oz0EJPJRExMDM2aNaNXr15MmjSJ559/HoDz588zbtw4Vq1apW64vGzZMlauXMnmzZvVk0Rmz56Ns7Mz//nPf9T7VjMajcTGxvLiiy9iMpnYsGEDnp6enDx5kvnz5/PEE0/QrVu3P7wfX0VFBSNGjGD27Nm3Zc5ldHQ069atw2Aw3DCI3ohGo/nTezdF7QwGA1OnTlU/jAkhhBDXqvPhMT4+ngMHDvDkk0+qc7sqKysZNWoUlZWVPPnkk3Tp0sVsha5Op2PQoEE0bNiQH3/8US0/dOgQ48aN49tvv2XIkCH89NNPzJkzh8OHD6sneFwrNTWV/v37891339G9e3cA1q1bx/vvv8+xY8fUnrhx48bRqFEjPv30U/XaWbNm4eLiUmt4rLZjxw6ee+45du3aRb169di2bRtDhgzB19fXsunvUllZyYgRI5g1a9ZtCY8pKSkcOHAAo9H4h8OjuH2MRiMDBw687jxhIYQQ97Y6Hx537tyJnZ0d/fr1Myv/4YcfeOmllxgxYgSffPKJ2akfWVlZDB48mMcff5yZM2eq5e+//z6rVq3i6NGjuLq6MmfOHM6dO8dPP/2ktrnWmjVrWLRoEdu2bVO3n5k5cyZZWVmsWLECqhbNjBo1ik8++YSuXbtSXFxMgwYN+PDDD8nNzeXjjz+2uOv/ZzQaefTRR3FwcGD06NE8+OCD6tzO/0VJSQmjR4/mrbfeUkOvpWPHjvHDDz9YNWwdFhbGs88++5efXCOEEEKIW6/OL5hJT0/Hw8PDspgRI0ZQr1497OzszIIjVRsyN2jQwGwVamJiIhs3buTtt9/G1dUVnU7HyZMn6du3r9m11zp27BiBgYFqcKxeuX3tCTeHDh3Czs6O/v37s3XrVsrKygAIDAw024amNlqtlieeeIKff/6Zhg0b1hoc9Xo92dnZNYZ0CwoKKCkpMSurVl5ejslkqvFzudZ9993HnDlzmDt3Lv/5z3+u+3j77beZPHmyzHe8Q5SXl2MwGMzKFEUhIyODq1ev1vg7sUZ0dDRz5swhJibGskoIIcQ9qM6Gx4SEBL744gu++OILdu/eTXp6ulm9h4cHzzzzDN26dTMrp2oF6ssvv0xUVBSXLl0iMjKSRYsW8c9//pPRo0cDkJeXR0JCQq3XUxUUr1y5Qu/evdWy3NxccnJy1LmOAG5ubjRt2lQdxq6eR9ahQweys7NvuF1OWloapaWl9O7d22x4/VpZWVls3bq1RmDYt28fcXFxZmXVkpOTcXd3p1GjRpZVKgcHB+rXr3/Th6+vLx4eHjfsnbwevV5PYmIiMTExFBcXQ9WQuuX3cjN6vZ6MjAxSUlJqBOaCggLCw8PNTh2621SfWb1q1SqmTZtGbm6uWmcymViyZAl79+7l1KlTLFiwQP1ZA2zfvl3dfup6zp49y7x589iyZYtllRBCiHuQdm713jR1THWv3fDhw/H19aVevXpqD2C1jh070rx581r3sGvatClNmjRRg8vDDz9Mz5491XpFUWjZsiVdunSpcZwbVW/Kvr6+PPTQQ+oWPYqi0KZNG7p27aoeGdekSRO8vLzQ6/X07dtX3ajc29ubHTt20KBBg1pXTaemprJnzx4eeeQRmjRpwvz58xkwYAB+fn5qm/LyctLT0+nRo4f6PRqNRq5cuULjxo1p0aLFNXf8/9auXYu/v/8Ne1VvtcuXL/Pdd9+RkZFBRkYGJ0+epKioiK1bt9KmTZvf1ZOZlZXFsmXLmDlzJvfddx8hISFQNaT++eef07ZtW0JDQ82uiYyM5MKFCzRp0uQPBd/fIysri+joaBo0aHBLnstkMpGamkppaSnbtm1jzJgx6sKsHTt2sH//ft544w1atGjB3r17uXz5Ml26dIGqXvjly5fTpEmTWnvwAZo3b06XLl1wcXGhZcuWt+R7EEIIUXfU2Z7HZs2aMWTIEB588EEGDhxYawBzdna+4ZnRbdq0YdSoUYwcObLG9W5ubgwdOlTda9CSnZ0dgwYNMhv6dXd3Z8iQIWbBx97enoceeoj+/fubhVgbGxtGjRrF3r171bJqkZGRbN26lWHDhuHu7k6/fv1o27Yt8+fPV9uYTCZOnTrF7t27Wb16tVp+/PhxMjMzWbhwYa3D4kVFReqK8L/KpUuXeO+99+jbty9Tp05l0qRJPPLII8yfP5+oqCiz85/37NnDW2+9ZXa9pQYNGjBw4EAMBoPZ0P7atWtxdHSsdV7nkiVL2LhxIyaTybKqVhUVFRQUFFgWX5eiKBw7doy1a9cyceJEFi9efMtCl1arpV27drRp08ZsX0dFUdizZw+BgYFq22bNmvHrr7+qX/v4+NCvXz+WLVumllmyt7enSZMmuLi43LLvQQghRN1RZ8Pj3WDo0KGUlZURGRkJVQtZ1q5dy8cff0xpaSn169cHIDMzk5CQEDIzM1m+fDnl5eUoikLr1q3JzMykefPmUBUWQkNDadGiBbm5uWrv07U2bNhAt27daNKkiWXVbaEoCvPmzaNHjx507txZDSP+/v706tWrxjSBDRs2kJSUZFZWmzNnzhASEqL2OhYUFLB7926GDRtm2RSAOXPm8O6771q9QfalS5euO3WgNoqioNfrCQoKwsvLy2yD+FultiCcmZlp9mHGycmJ/Px8s6HrXr16kZGRwaFDh9SyaxkMBk6fPk2bNm0sq4QQQtyDJDz+hZycnHjhhRfYvXs3WVlZuLi4MGDAAL766iueeeYZdXGDp6cn77zzDuvXr2fkyJHY29uj1WopKSkhIyODsrIysrOz0Wg01K9fnyNHjtC8efMaQ7+nTp2iqKiISZMmmZXfTomJiRw/frzWPQSbNGlitml5YWEhZ86cYeTIkWbtqimKogamiIgIWrZsqfbuHjlyBEdHxxrD1VSFrAYNGvyufTLLy8vJy8uzLL4uGxsbevbsSbdu3a7be30tRVFqnEGdlJTEzz//XOtj06ZNREdHm7WvjdFoNFskc22v5LVlPXr0YOPGjWrZtYqKiujcubO6wb0QQoh7m4THv1hwcDDjx4+nvLwcjUaDt7c3np6euLm5qW/0Dg4Oapm3t7faW+bl5cXYsWMJCAjAy8tLDTeHDx+mc+fOZs9D1X2mTp36l54NbWNjg8Fg4LPPPuPixYvo9Xq1rn///rRo0YK8vDy++OILPv74Y6Kjozl8+DDLly83W1wUExPDkiVLWLduHdu3b6+xuOm3336jdevW6tfVIiMjWbZsGStWrKi1p+56NBrNDc+IvpGbrXA+fvw4X3/9NWvWrGHNmjXqinyj0UhFRUWtj/LycrOf3fV4e3uj0+nUr8vLy3FxcanRK92pUycuX75c69C8l5dXrSFcCCHEvemPvRuKP1VQUFCNOZfWcHFxYejQobRr1w6tVsvixYtZunQp9evXr/XYwTZt2tQIDbdb48aNmTRpEtu3b+e+++6jU6dOzJw5k+joaHWY3s3NjdGjR+Pq6kqrVq2YMmUKgwcPVnsVIyIimDlzJl27dmX06NGcOXOGqKgos5Xvly9frhF44uLiuHz5Mg8++CBLlizh8uXLZvU3cyvm+33++ed88cUXDB48mHHjxqHRaNi9ezdU9cQ+9thjtT4mTJhA27Ztze6l0WgwmUxqWNVoNPTu3ZuUlBS1TVxcHD169KjxvdSvX5+KigoyMzPNyoUQQghLEh7vIlOnTqVz587MmDHDqqHSv8prr73Gli1bmDFjBkajUT1y8erVq1C1GKlhw4bExMTQqVMnwsLC1H05CwsLmTlzJoMGDaJDhw7Y29urK9arV6KXlJRQWlpa4ySes2fP0r17d6Kjo9Hr9dfd59JoNFJZWak+qucvGgwGdDqdWZ01vX/Xs2XLFhYtWsRrr71G48aN0ev15OfnqyHaWoqisH//flasWEFxcTHffvstJ06cAODhhx+mUaNGrFu3jp07d1JUVMTTTz9teQscHR1RFOWu3tJICCHEn6POnzAj6o7qOX3XLlIxGo3qiuRNmzYxatQoqNpHs2/fvjz77LNMnjxZbf/f//6Xzz77jN27dxMUFATA9OnTcXV15ZNPPgEgOzubsWPH8tVXX5kdS1lWVoajoyPTpk3DxcWFL774Qq271tmzZ9WV2BqNBq1WS1JSEomJifTq1Uvt2TMajTRr1owJEybUup0TVcFu6tSp6PV6fvjhB7Vcp9Px2GOPkZqayoIFCygtLSUxMZGgoCAGDx5sdg9rVFRUoNPpsLW1xWAw4ODgoM551ev1JCUlYTAYCA4OrvUkoLKyMgYPHsx7771ntsm9EEIIYUl6HsVtExERQXJyslmZVqtl+PDhBAcHm80pjIqKory8XN2PsDqwHTx4kMDAQDU4VlZWcvz4cbp27apea2dnh1arrbHZuLOzM6mpqRw8eJDx48eb1V0rJCSECRMmMGnSJCZOnMgTTzzBwIEDad++PRMnTlQfTzzxBP3791f39Pw9qrdMGjp0KAEBAbRq1YrJkyf/oeBIVc+hu7s7zs7OuLu7my2WsrOzIzQ0lJYtW9YaHKlaRKTVav/Q9yKEEOLeIuFR3DZ79uyhqKjIspiMjAz8/Pxo3769Wnbq1CkcHR1p1aoVRUVFREREUF5eTmlpqdlCmCtXrqDT6WjdujXR0dGkpaXh6emJg4MD2dnZartqe/bswdfXl44dOxIREVHrAhF3d3datGihPpo0aULjxo3x9fWlefPmannLli0JDAy86UIajUZTY45hdXmnTp0IDg4mMDAQBwcHKisrzU6IuV10Oh0ajQZXV1fLKiGEEMLMjd/1hPiTlJaWcvjwYXbt2mVWXlZWxldffcW0adPMFg0lJyere1GePn2a8vJynJycaNasmTr8XVFRwcqVK/H09KR+/focP35cDT9BQUG17g+5a9cuhg0bRllZGYmJiVavPL92WyBrGQwGSkpKKCkpoaioiNLSUnWOpJeXFz179jQ7QrK0tJStW7f+7oU8f4aioiLs7Ox+93xLIYQQ9546ezyhqFtiY2OJjY2lYcOGHDlyhKSkJC5cuMBPP/1Eq1atmDZtmll7W1tbYmNjMRgM6PV6Bg0ahEajoXnz5kRGRlJeXk5UVBQtW7aktLSUkpISAgMDadWqFVQdCXju3Lkaw8DFxcVkZGRgNBrp3LnzdRfNWEpJSSEhIcFsRfeNKIrC7t272bNnDwkJCQDk5+eTm5tLixYt0Gg0tGnThsOHD1NcXExSUhLnzp0jICCg1pXyt9qhQ4dIS0tj4sSJllVCCCGEGVkwI26LoqIi9Ho9np6exMTEkJmZiVarpVGjRjRu3LjWYd20tDQqKipo3Lix2Vy8wsJC0tPT8fHxwcfHh/z8fPLy8szOqU5MTGT27NksWrTI7Dzw6sUjHh4ev6uX7dixYxw4cICZM2daVl1Xbm4uer1enX9YUVGBnZ2dWWAtLCwkIyMDW1tb/Pz8/pJhY0VRmD17Np06dfpLj60UQghRN0h4FHetDz74QN0r8X9VWFhIXl6eevzh3eTSpUt8/vnnfPLJJ1YP4wshhLh3yZxHcdeaOnUqZ86cqbHC+4/w8PC4K4OjwWDgp59+YuLEiRIchRBCWEV6HsVd7dKlS0REREg4uo5t27ahKArDhg2zrBJCCCFqJeFR3PUKCwtxdnbGzs7Osuqel5eXh6en5023GxJCCCGqSXgUQgghhBBWk+4GIYQQQghhNQmPQgghhBDCahIehRBCCCGE1SQ8CiGEEEIIq0l4FEIIIYQQVpPwKIQQQgghrCbhUQghhBBCWE3CoxBCCCGEsJqERyGEEEIIYTUJj0IIIYQQwmoSHoUQQgghhNUkPAohhBBCCKtJeBRCCCGEEFaT8CiEEEIIIawm4VEIIYQQQlhNwqMQQgghhLCahEchhBBCCGE1CY9CCCGEEMJqGkVRFMvCu1FcXBxbt24lNzeXF154AV9fX8smQgghhBDiJu6ZnkcfHx8cHBxYuHAhOTk5ltVCCCGEEMIK90x49PDwoE+fPvj6+nKPdLYKIYQQQvzp7pnwKIQQQggh/nd1ds5jamoqWq0WRVFwdHTEYDBgMBhQFAUXFxdcXV3JzMxEo9Gg1Wrx8/Pj8uXLjBgxgk2bNuHr60tSUhLu7u40a9YMjUaj3ttoNJKSkkJOTg7u7u6EhIRgZ2dHYWEh8fHxGAwGWrduTWpqKgaDgZYtW2Jj8385PCcnh9TUVDQaDcHBwXh4eFzzqoUQQggh6rY62/O4fv16BgwYwJgxYzh37hwHDhxgxIgRjBs3jkuXLlFeXs57771H37592bJli3qdRqPhyJEjHD58GL1ez5w5c1i8eLFabzAYmDdvHitWrMBkMrFp0ybefvttTCYTqampLFy4kEcffZSffvqJ/fv3M3DgQH777TcAdu/ezb/+9S+ys7OJj4/n73//O5cvX1bvLYQQQghR12nnzp0717KwLujevTuJiYnodDpee+01wsLCyM7OprKykldffRUHBwcaNmyIr68vzz//PAB5eXksXbqUDh068NRTTxEYGEhubi7r1q1j3Lhx2NnZsWTJEjZv3szChQtp0qQJ7dq14+OPP8bDw4PevXvTokULli1bRp8+fXjyySexs7OjZ8+e5OXlMXXqVGbPnk2/fv1o1aoVFy5cYPPmzTz88MOWL18IIYQQok6qsz2PAOPHj+fMmTNcuHABAJPJRGRkJDExMQBcuXKFESNGmF2j0Wjo2rWr+rW7uzvl5eWUl5djNBpZvnw5oaGhaLVasrOzMZlMNGnShF27dgGg1WqxtbWlQ4cO2NnZ8corrxAYGMimTZvIzs6madOm5OTkkJ+fT5s2bTh9+jTp6enXvAIhhBBCiLqrTofHNm3a4O/vz8GDB8nMzKRZs2a0b9+ePXv2YDAYyMvLo0mTJmbXaLVanJyc1K9NJhMmkwkbGxuys7MpLCyksrKSEydOcPr0aU6cOEGvXr0YM2YMgDrH0t3d/Zq7QmxsLC4uLpw9e1a9zmQy8Y9//AM3NzeztkIIIYQQdVWdDo8ODg48+uijhIeHc+rUKe677z4eeeQRdu3axeHDh2nVqpXlJVAVAC2ZTCbc3d2xt7enQYMG9OnTh0GDBvHQQw8xefJk+vfvr7bVaDRmC2wA6tevj42NDf3792fgwIEMHDiQRx55hCeeeAJXV1eztkIIIYQQdVWdDo8A/fv3Jy4ujkOHDhEQEEC/fv3Izs5m8+bNNG/e3LI5ALa2tur/t7GxwcbGBkVRcHZ2ZsSIEZw+fZqKigq1TVpamjpsXU2r1Zp9PXToUCoqKrh48aJaptPp2L59O7m5uWZthRBCCCHqqjofHjt06EDz5s2xs7PD3d0dPz8/mjRpQlFREQ0bNlTbZWdns3fvXnJycti2bRtXr14lPj6egwcPkpaWxs6dOyktLeX111/Hz8+PTz/9lIKCAlJTU/nll18IDQ0lJyeHXbt2kZ6ezoYNG7h06ZLai9m1a1dee+01PvroI6KioigpKWHr1q0YDAa8vb2vecVCCCGEEHVXnd3n8VoRERE0bNiQkJAQAC5evEhFRQX333+/2iYlJYUzZ85gNBpRFIX77ruPsrIyrly5gkajwc7Ojm7duuHt7U1ZWRkHDhwgKSkJHx8funXrRkBAAElJSZw9exaTyQRAYGAgHTt2NBvCjoyMJDIyEhsbG9q3b0/79u3VPSCFEEIIIeq6uyI8CiGEEEKI20O6xIQQQgghhNUkPAohhBBCCKtJeBRCCCGEEFaT8CiEEEIIIawm4VEIIYQQQlhNwqMQQgghhLCahEchhBBCCGE1CY9CCCGEEMJqEh6FEEIIIYTVJDwKIYQQQgirSXgUQgghhBBWk/AohBBCCCGsJuFRCCGEEEJYTcKjEEIIIYSwmoRHIYQQQghhNQmPQgghhBDCahIehRBCCCGE1SQ8CiGEEEIIq0l4FLdFYmIiW7Zsobi4mCtXrrBp0yYuXryITqcjIiKCn376iezsbMvLhBBCiDvamjVrGDx4sGXx7zJ48GCWLFliWXzH0iiKolgWCgFgMpnYt28faWlp2NjYYDAYLJuoTCYTiqKg1WrRaDR4e3szdOhQbGxsKC0tZevWrURHR3PlyhWef/55/Pz8+Mc//sGAAQMYNmwYJ0+eJCIigoULF1reWgghhLgjxcfH06lTJ06ePEmTJk0sq632Z93ndpGeR3FdNjY27Nixg8mTJ7Nv3z4cHR3RaDQ1HoqiUFxczMWLF1m8eDFTpkzhqaee4syZMwCUlpZy//33k5OTQ0BAAA888ABubm7k5eXRrl07mjRpgq2tLXl5eZYvQQghhLhjzZgxg9dee63WwJebm8u8efMIDQ1Fo9Hg5eXFjBkziI+Pt2xKkyZNeO2115gxY4Zl1R1Jeh7FDRUWFjJ8+HBKS0vZvXs33t7elk3MKIrCyZMnmT17Nr1792bu3LkAFBcX89hjj/Huu+/SsWNHDh06xEcffUR4eDgODg7885//pFGjRrzyyiuYTCZsbORzjRBCiDvX2bNn6du3LzExMbW+N3bu3BmApUuX0r59e+Lj45k/fz5r1qyp9Zrc3FyaNWvGvn37aN++vVndnUbeocUNeXh48Omnn5KamsqsWbNuOHQNoNFo6Ny5Mz/88AM6nY7MzEyomvOo1+vVT2fnzp0jJCQEBwcHCgoKOHfuHF26dGHfvn3o9XqLu4q6TlEUUlNTLYvvKAUFBRQVFVkW3xJ6vZ6MjAzks7sQdde6det4/vnna4RAquZBnjx5kh07dqhBsEmTJnz11Vd4eXkRHh5ueQne3t6MHz+edevWWVbdcSQ8ipvq3Lkz7733HitXrmTp0qWW1bVq2LAho0aNwmQyAZCWlkazZs3w9PSEqjDZq1cvqAoWgYGBxMTE4OTkhIODg9m9RN1XUlLCW2+9ZVl8R1m3bh379u2zLL4lUlNTWbBggXxQEqIOW7t2LUOGDLEsBuD8+fMMGjSo1mAZGhrKpk2bLIsBGD58OGvXrrUsvuPIsLWwiqIoTJkyhc2bN7Nnzx46duxo2aRWiqKg0WgoLCxEp9NRv359ADIyMnBzc8PFxQWAzMxMSktLa5038mcwGAykp6fj4OCAr6+vZTUmk4mcnBzKy8vx8PBQQ25lZSU5OTnUq1cPZ2dniouLyc/Px9HRkfr166PRaCxvRUFBAYWFhTg5OV23zb2msLCQl19+me+++86y6o7x1Vdf0aBBAx555BHLqj9dXFwcixcv5v3338fe3t6yWghxh8vNzcXHx+cPjR4MHjwYT09P1qxZY1kFVSN4OTk5tQbPO4X0PAqraDQa3n//ffz8/HjxxRetXtxSHZw8PDzU4Ajg7++vBkcAPz+/WxYc09LS+Oijj9i7dy/79+9n4cKFfPPNN2p9cXExX375JeHh4Zw6dYoVK1aoQ6z79+9nw4YNvPXWW5w+fZodO3Zw+vRp3n//fbZv337Ns4DRaGTdunV88803nDx5kvDwcI4cOWLW5l5VvbjqTnY7X19d+HkIIa4vKiqKTp06WRbfVG5uLsePH6dv376WVaqmTZsSFRVlWXxHkfAorBYQEMCXX35JZGQk7777rmX1HamgoIDnnnsOf39/pkyZwpAhQ9iwYQNHjx6FquHUGTNmkJmZydNPP83o0aNp2LAhZ86cIS8vj6tXrzJu3DiWLVvGrl27GDZsGKNGjSI4OJivv/7a7Lnmz5/P2rVrefLJJ3n00Ufp3LkzkZGRZm2KioooLS01K7udSkpKSEhIIC0tjdTUVHXYND09nYSEBBITE286r/VW0uv16utLSkqivLwcgJycHJKSkoiPj/9Lf363W3l5OYmJiaSlpZGSkkJFRQUAWVlZJCYmEh8fr5YJIW6foqKiP9Qz+O9//xuAMWPGWFapQkNDb9v86z9KwqP4Xfr168e///1vPv/88zoxL2PRokXk5+czduxYuGY/yv79+wPw3XffceDAAV555RVsbW05d+4c0dHRdOjQgfLyctq3b09MTAyenp489thjODs7Q9WnRzs7O/V5jh49ysKFC3nuuefw8/MjNTWVAwcO0KZNG7UNwIIFCzhw4IBZ2e1UUlJCeHg4gwcPZtOmTeqQy6pVq/jHP/7B8ePH/+fwqCgKlZWVVFRUUFFRof5/o9Goll1bdy2DwcDBgwcZPXo08+bNQ6fTQdXP9+mnn2bv3r2UlJSYXfNH6PV6s9dhMBjQ6/VqmL62zmg0Wl7+uyiKUuO5Kisr1f+99mel0+nMhsEqKyvZs2cPw4YNY8mSJervZs+ePcyYMYMDBw5QIeFRiNvuwoULlkU3tWbNGhYvXswPP/xw0+D5R+5/O8mcR/G7FRYWMnjwYEaPHs3MmTMtq+8YRUVFdO/enVGjRvHee+8BEB0dzcCBA/n1118JDQ2lc+fOuLi48PTTT6PVanF1deW+++4jMDBQ3TLozTff5MqVK6xfvx6qeoMefvhhRo4cyYsvvgjAzJkz+fHHH5k7dy5ubm7Y2toSFhZGq1atzLYdysjIwNXVFVdXV7XsdisuLmb48OE88MADvP/++6SlpbFz504GDBhAUFCQZfPfLSMjg3fffRe9Xq8Ozep0Og4fPkyfPn3Udoqi4Obmxptvvkm9evXMyqdPn056ejqbNm2irKyMLVu20KJFC+6//3613R9lMplYtmwZR44cUT8A2NjYcOHCBZycnAgNDVUDo8lkYvTo0f/T6RFXrlzh888/R6fTodFosLGxoaCggEuXLtGlSxf178NoNNKwYUNeffVVdc5tdfmECROwtbXlxx9/pKCggJ9++okuXboQFhZ2zTMJIW6XefPm8euvv7Jjxw7LqlotWbKE5557jtWrVzN+/HjLajODBw+mX79+zJo1y7LqzqEI8Ttt3rxZ+fvf/65UVFRYVt0WJpNJMRgMSmxsrJKenm5ZrTpz5owSEhKibNq0SS1bu3at0rVrV6W4uFgxmUyKr6+vsnjxYrPrLA0ZMkR555131K8vXbqkNGrUSImKilL0er1iMpmUxx57THnkkUfMrruWXq9X9u7dq1y+fNmy6i+xbt06pVmzZsr+/fuV77//XsnOzrZs8oeZTCalrKxMKS0tVUpLS5WysjIlPT1dmTRpklp2bZ3JZLK8hRIVFaUEBAQo69evV9asWfOn/9wqKyvNXkdlZaWyYMECZfXq1YrRaDSr0+v1lpf/LpY/j8rKSuXChQvKK6+8ohQUFJjVlZeX1/rzOHz4sBIcHKxs2bJFWbVqlZKYmGjZRAhxG61evVoZNGiQZXGtnn/+eQVQVq9ebVlVq0GDBlnd9q8iw9bid4mMjCQ8PJy///3vf9mWOpcvX2bZsmU8+eSTXLp0ybJa5ezsjIeHBw0aNFDLNmzYQOfOnXF1dUWj0eDn51ejFzA3N1e9b0pKCpmZmfTo0UOt//nnn+nQoQPBwcFs3ryZoqIifHx8zHrPqBr6PHv2LCaTia1bt1JWVsZbb731pwy7/q8efvhhAgICePvttxkxYgQ+Pj6WTf4wjUaDk5MTzs7OODs7q//f1tZWLbu2rraFIy1atGDMmDHMnDmT+++/n+bNm1s2qVX11lA3Y29vb/Y67O3tsbe3x8HBARsbG7M6W1tb9brs7GyzxWImk4m0tDSzoWOdTkdaWpr6Wix/Hvb29jg5OWFra4uTk5NZXfUpTpa6detG7969mT17Nj179qRx48aWTYQQt1GjRo3Izc21LK5hxowZLF682Koex2q5ubk0atTIsviOIuHxLxYZGUlBQYFlsaqiooJTp07dEfvBpaens2jRIl544QUCAgIsq2tVPSvCZDKZDQVazu36PQIDAxk/fjxeXl43nJ/XrFkzevTowZkzZygvL2ft2rXs27fPbIXcxIkTOXv2rDrfLSYmhg0bNqhv/OfOnaO4uNhsa6LY2Fg6duyoBgQPDw8eeeQR8vLyyMvLw2AwkJOTQ3h4OLm5uVRWVhIYGIiiKJSVleHo6Kje66+Sm5tLq1atSExMpLCw0LIagMOHD6tHTOp0Oo4dO8Z///vfG/69/lnKyspo1KgRBoOBmJgYy2qoOgt2z549UDVFYceOHaxevZrPPvvslmxIXlZWxtSpU/nb3/6mzsWMjY2ld+/eZvufrlmzhj59+nDs2LFrrv7fFBUVERwcTG5uLlevXrWshqo5UocOHbIsFkLcAg0bNuTkyZOWxWaq5zj+nuAIcPLkSRo2bGhZfEe5K8LjkSNHWLp0KR999JH6ZlIbvV7Pxo0bmT9/Pt999x0XL160bHJb7d+/n+PHj9fo+SovL+fq1asoioKjoyPp6el/+Y7zlZWVzJs3j+HDh6tHLt3M6dOnycjIAODUqVN88803nDlzhiNHjrB+/XrCw8PVQFm9n+KNHtUB2sXFBeeqhSu19dJc67333sPFxYXNmzdjY2NDgwYN6NChg1r/t7/9jdatW7Ny5UrWrVtHVFQUo0aNUhe6ODk5MW3aNLM5aC+88AK2tracP3+egQMHAtC/f3+effZZVq9eTXh4OHv27KFHjx707dsXJycnOnTowPbt2xk0aJBZT9ZfITExkT179vCvf/2L4OBgFi1aZNkEAF9fX/z8/KDqb7KkpIRdu3b9oZ5TRVGs7hXMy8tjw4YNjBs3jkcffZSFCxfWWFhD1fZP1XM0t27dyuXLl3nsscdwc3P7QxuS3+zDjL29Pe3atSMsLAytVguAm5sbXbp0ITg4WG0XGBhIly5d8PLyuuZqc4qi3PT5quXk5LBp0yaefvppBg4cyPz58y2bAFCvXr07vrdCiLtFkyZNaNq06XU/sMXHxzNjxgw++uij3xUct23bRtOmTW/Z1nV/Gstx7LooISFB+fTTT5UGDRoojzzyiKLT6SybKIqiKCdPnlRCQ0OV7t27K4cOHVLS0tIsm9w2iYmJyquvvqrk5ORYVilffvmlMnDgQKW8vFxRFEUxGo3KO++8o+zfv9+y6W3z8ccfK5988oll8XXl5uYqr732mpKRkaEUFhYqGzduVObOnasMGTJEuXLlipKVlaUMHDhQiYuLUxRFUc6ePau88847ynvvvVfr491331Xi4+PV+xsMBmX48OHKrl27rnlWc6WlpWZ/C998843StWvXWueUmUymWsuv53ptTSaTYjQaLYuV1NRUpV+/fsrp06eVAwcOWFbfNomJicp3332n/t1t3rxZCQ4ONptTaDAYlL179yrLly9X8vPz1fKysjJl0qRJSkpKilpmraKiIuW5556zLK4hPz9fWbFihRIbG6soiqLExcUpgYGBypYtW8zanTlzRlm8eLGSlJSkKIqixMbGKhcvXlSUqvmckydPNmtvjaVLlyqbN2+2LL4lEhMTlX/961/X/W9VtaysLGXFihVKcnKyoiiKcvr0aaVx48Y1/ltw5MgRZenSpUpqaqqiKIqSkZGhfPPNN8rOnTuVffv2KT/++KOyf/9+Zf/+/cry5cuVhIQEs+uFEL/f888/r7zxxhuWxYpSVde0aVPL4pt6/vnnleeff96y+I5zV/Q8BgcH07JlS4YOHUpGRkat+yPl5+cTGRmJnZ0dEyZM4IEHHvhLu4WXLl1Kz549a12u3759e2bMmKEOb9rY2DBmzBhWrFjxl+xxt2HDBnJycnj55Zctq2pQFIVz584xceJETCYTfn5+KIpC7969SUpKYtiwYTRr1oyioiKMRqPag9imTRtmzZrFP//5z1ofM2fO/F3zvHJzc3nggQfUFdLp6emsWbOGZ555ptbeyt+7afP12lavprWk1+sJCAggISHhLxm2zsjIYNWqVUyfPp2wsDD1765hw4YYjUY+++wzYmNjoeq0HycnJw4ePGg2RG1tz2FtXF1dmTNnjmWxqqSkhG3btqmr3ps2bQpVvX2hoaF88sknnDlzBoPBgMFgoLy8nKioKJKSkqBqU92wsDCuXLnC2bNn1b3Ufo8xY8bw4IMPWhbfEgEBAbz00kvX7YXOz8/nl19+4amnnsLLy4vAwECo6l309PRkwYIFnD9/Xm1rY2NDZGQk6enpUNVbaTKZWLVqFffffz8Gg4Fly5bRvXt3CgoKiIiIMHs+IcTvN336dBYvXlzr3MfFixcTFxenvrfU9rCUm5vLmjVrmD59umXVHafmu1wdVFlZSX5+Pg888ECt4dFoNBIZGYmvry+pqan07NnTrP52S05OJjY2lgceeMCyCoAHHniAhx9+2KysWbNmaLVaDh8+bFZ+q0VERPDWW29Rv359wsPD+fHHH2s8li1bxqJFi5g5cyZDhw6lZ8+eHDx4kEGDBkHV8KKiKKSlpanDvNVzOlxcXNDpdKSkpLBhwwY2btx43UdWVpbZa6vtH181rVZLs2bNSElJYf369fz3v/9lypQpTJ482bLpbdG4cWPmzJlDWFiY1cP+f7aAgABmz55tNrTp5ubGjz/+yPjx49Vh1IYNG1JaWoq/v7/ZXpbVQ7W1heOb0Wg0ZguXauPu7s5zzz1Hly5d1DJbW1veeust5s6dqz6vra0t9erVU7e2qd5I/MyZM/z222888cQTFBQU/O6w6+HhgZubm2XxLWFra3vToyu9vLx49dVXad26tVqm1Wr5/PPPefnll7GxscFkMlGvXj0cHR2xsbHB29sbRVFo3bo1GRkZPPLII7i5uZGQkMCjjz6KnZ0dcXFxVk/2F0JcX/v27enSpQvffvutZZU6NeVGD0vffvstXbp0oX379pZVd5y7Yp/HrKwsfvrpJ/r06UOnTp04ePCg2Q///Pnz6PV6Tpw4wccff8zp06dxd3c3uwdVk9JPnz6NwWDAy8urxvnNmZmZnDt3DkdHRzp06IDRaMTd3R2NRkNubi4nT54kODiYFi1acPHiRdLT07GxsaFbt25qDxtVPXm7du1iyZIlZvevfn5nZ2c6d+5c443lyy+/JC0tjffff9+s/Fb6+uuv+fXXX3Fycrrum7FSNZ/NaDSqcxhDQkJ466231F62Y8eOMWfOHDZu3IiLiwvPPfccPXr0oGHDhoSGhuLi4sLp06drfM/VFEWhU6dOao9ZeXk5AwcO5O9///t1zyI2Go0UFRWh1+txcXExOw5R3NjGjRtJTEykS5cu9OzZk9LSUg4dOsQ333zDhAkT6N+/Px4eHpaX3TYnTpxg69atPPDAAwwYMIDz588za9YsPD091T0233jjDcvL7loHDhzgwIEDdO/enf79+1NZWcnHH3/MhAkTaNKkCW+++SavvPIK9vb2LFy4kDZt2tCxY0ezuZpCiN/v7Nmz9O3bl5iYmFpHEq2Vm5tLs2bN2Ldvn4TH2+XSpUtERkYyfPhwmjdvzvLlyxk6dChUBbL9+/czbNgwxowZg6urKytWrKgRUk6fPs2SJUsYPnw4ISEhHDx4kI4dO9K1a1cAjh8/zvLlyxk3bhzOzs4sX74co9HIl19+iVarZe3atbi5ubFkyRLGjRuHv78/jRo1YtmyZXh7e/P666+rzzVr1iz8/Pz4xz/+oZYZjUY2bNiAv78/8+bN46233qqxIfLOnTv55ptvCA8PNyu/Vnp6OhUVFTW+P0uKouDr63vbAtVvv/3G8ePH1U1PV61aRUJCAn379jXbBscaUVFR7Nu3j+joaPz9/Wnfvv0dsRDlz6TT6cjIyLhuYL+WVqulQYMGf+r3X1lZSWpqKoGBgdjb21NeXk5MTAy2trYYDAZCQkJuWy9dbQwGA6mpqfj4+ODq6kpOTo66CllRFBo0aICvr6/lZXcto9FIUlISDRo0UD/oFRQU4OHhgVarJTs7Wz1bPisrC4PB8JdO2xHibrJkyRI2bdpk9YbhtRk8eDCPPPJInRiy5m4Jj2vXriUsLIzQ0FB69+7NE088wUsvvQRVgSssLIz69evTokULZs2axQsvvGB2fWxsLGPGjOE///mP2ou1Zs0aXF1dGT58OElJSTz++OPMnj1bHU4eOXIkgYGBfPnll6SkpHDgwAEGDhxIixYtmDdvHk8//TRU/VGFh4eze/du9fkmTJjAsGHDmDhxolp27NgxsrOz6dWrF926deO///1vjflXZ86cYebMmWzatKnGCm2q3jQ///xz4uPj1SHG6zEajTz11FN14hPOvejq1assXryYkpKSGw4TV28V9NJLL91wda8QQgjxZ7krwuNHH33Eiy++iKOjI+PGjSMgIIBFixYRExNDSkoK/fr1IyIiggkTJvD999/Tu3dvs+tfeukl9u7dy44dO9Dr9Zw6dYqMjAyefvppnJycmD17NgcPHiQiIgKNRoPJZKJnz55Mnz6dyZMnk5mZia2tLfv27WP+/Pls3rxZ/ZT/yiuvUFZWxtdff60+34gRI5g2bRqjRo1Sy2JjYwkKCmLlypWsXLmSrVu31ujZiY6O5plnniE8PFzdQsXS9eZS1OZGoYSq3sGYmJibBlHx+yiKgq2tLaNGjaJVq1aW1Spreh2r3eh3mZubyw8//EBhYeEN24l717/+9S/LIiGEuK46Hx5zcnLYsGGDukrz1Vdf5fLly/zyyy9s3ryZQYMG4ezszLx581i5ciURERFmc7UqKipo27Yt3bp14/HHH8fOzo6goCBCQ0PRarXk5eXRu3dvHn30UXXvuOjoaMaNG8eKFSvM9gx84YUXKC4uZuXKlVB1hvCDDz7I66+/ztixY9V2w4cP55lnnqmxKEZRFB5++GE6dOhQ6z51ly9f5plnnmH9+vXXDY9/pp07d5KSkiKB40+mKAparZY+ffrcljlnhYWFbN++/aa9mOLeNXXqVMsiIYS4rjofHg8cOEBZWRmDBw8G4LPPPmP58uUsWLCAgIAAWrRoAcDjjz9ORUUFmzZtMru+pKSEkJAQlixZwujRo9VyRVHQaDQkJyfTrl07Vq9ezZAhQwBYtmwZy5cvZ+fOnWRmZhISEoJOp2Po0KGMGjWKF198Eape2/Tp09m1axd2dnY4Ozvj7u7OxIkTGTJkCJMmTVKfj6ph6bFjx7J582bc3NyoV6+e2UKbyMhIZs2adcNh6wULFhAXF3fT3kKDwcC0adNqLAoSd4arV6/y+eef3zTwmUwmPD09eeWVV/6nydpCCCGEtep8ePz+++9p2rSpuuhi/fr1TJ8+na+//poxY8ZA1R53/fv35/nnn1eDXTWDwcCDDz7I3/72N7Nd4M+dO0dJSQktWrSga9eu/PDDD3Tr1g29Xs+4cePw9PTk66+/ZunSpTz//PNERUXxyCOPsHbtWnUe4SuvvEJxcTFfffUV69atY+jQoXh7ezNr1ix8fX355z//ec0rgQULFrBjxw62bNnCsmXLGDNmjNk8tp07d/L111+zYcMGs+uulZuba9WCGaq2Avkr9hwUN1d9xKE1Q9darZb69evfMGQKIYQQfxbt3Llz51oW1gWpqamsX7+exYsXY29vT5MmTXB3d6egoID8/Hz+9re/odVqOXHiBJs3b2b79u106NABb29vs/3VbGxs8PHx4eDBg4SEhFBQUMDJkydJSEigZ8+e1KtXj4KCAtLS0mjQoAERERGkpaXh7u6O0WikRYsWNGzYkCNHjrBv3z5ef/11dW+8Y8eOodVqcXd3x8XFhbCwMAAKCgo4fvw4I0eONPueTp06hcFgwM3NDQ8PD7P93agKjw4ODvTv39+s/FrVvZtubm43ffyZq3Nvp4qKCoqLizGZTGg0mhv2sppMJsrKyigrK8NoNGJvb2/Z5I5kY2ODq6trjd9ZbQ9XV1erPizci0pKSigvL8doNGJra3vDn5NOp1P/VmxsbOrsvw8hhLjV6mx4zMnJobS0lCFDhlC/fn3c3d3x9vbG29ubhx56CC8vL3Q6HbGxsfj4+DBp0iR8fHyws7OjYcOGZm8iLVu2xM/Pj/j4eAoLCwkICKBfv344OTkB0K1bNyoqKsjIyKBt27aMHz8ejUaDu7s7nTt3xsbGBhsbG3r16kWzZs3U+7Zv3x6TyYSbmxu9evVSy11cXNiwYQP9+vUzG35u2bIl9vb2ODk58cADD5iFIoPBwLfffsvw4cPv/DMvb7HTp0+zdu1a5s+fz4ULF+jbt2+tvW4Gg4EPPviAzz77jOTkZBwcHG7LHENxZzAajWzfvp0ffviBzz77DEdHR/XMckvp6em8/PLLbNq0iaysLPz8/GQagBBCXEedH7auq+bMmUO7du3UofWbuXjxIp999hmLFi0ymwd5L9Lr9Zw/f55//OMfxMbGcunSJdxq2XPw1KlTTJ06lYCAAJYvX46npycODg6WzcRdrLKykuXLl/PFF19w//33s2LFCssm6PV6Nm7cyNSpU5k1axazZs3C1tbW7HQdIYQQ/1/N7hpxW0ybNo2IiAirjggzmUysXbuWJ5544p4PjgB2dnakpKQwevRoDAYDeXl5lk3IysoiMTERk8lE9+7d8fPzk+B4D6r+nQ8ZMoQrV65YVkPVpvNZWVn4+/vTp08fnJycJDgKIcQNSHj8izRu3JjRo0ezevVqdDqdZbWZTZs2ERYWVmPT8HtZRkYGrVu3xtHRkZSUFLM6o9HImTNncHd3p7y8XD0lSNx7cnJycHNzo3nz5mRmZlJRUWFWn5WVRXZ2Nnq9HkdHR9q1a2dWL4QQoiYJj3+hXr160bt37xpvaNfS6XSEhoaabSN0r8vOzsbJyYnGjRtTr1494uPjzeojIyNp1KgRV69eRafT0aVLF7N6ce+4fPkyfn5+6lzXzMxMtc5oNHLq1CnatGnDuXPnCA0NpV69etdcLYQQojYSHv9i7dq1w93d3bJYZW9vT/v27evMKuHbISUlBWdnZ0JCQqhXrx4xMTFqXU5ODpmZmbRo0YIjR47QqlUrPD09za4X947s7Gzq1atHo0aNsLe3JykpSa2LiorC29sbW1tbTp8+Td++fc2uFUIIUTsJj6LOyc3NxdPTExsbG+rXr09ycrJad+LECe6//34qKio4fPiwBIJ7mF6vJz8/H39/f+rXr4+DgwOJiYkAFBUVER8fT5cuXUhJSSE2NpaePXta3kIIIUQtJDyKOsVkMpGcnEzLli0B1OFpgPPnz+Pu7o6/vz8pKSnExcXRvXt3izuIe0VxcTFFRUUEBATg7e2Ni4uL2vN49OhRdX7j0aNH8fPzIyQkxOIOQgghaiPhUdQplZWVFBQU0KhRIwBCQkIoLi4mJSWFpKQk9azxo0eP0rBhQwkE97CcnByzc+z9/PzIyMhQz2sPDg5GURT27NlDly5dZL6jEEJYScKjqFNSUlJo0KCB+nXTpk0pLCxk06ZNtGvXTt3KaO/evYSFhREQEHDN1eJecv78efXDBEBwcDBRUVEcOXJEHaKurKzk3LlzdO3atdaN5oUQQtQk/7UUdYKiKOj1ejZt2mS2IXhISAjZ2dm4u7sTFBSEoijk5OQQGRlJ8+bNb3h0obg7mUwmCgoK2LVrl9qbqNFoaNasGQkJCeoWTwCHDh2irKxMnQYhhBDi5urs8YTi3pKYmMi3337L3r17yczMxNvbm6CgIDUcTps2DXt7ezZv3syGDRvIycnB09OTsrIywsLCbnimsbi7HD58mK+//poLFy6Ql5dHSEgInp6emEwmgoODGTZsGDk5OYSHh7Nz5071zHM7OzuCgoIsbyeEEMKCHE8o6oTS0lLy8/NxcnKisrISR0dHvLy8UBQFnU6Hg4MDiqKQmZmJ0WjE0dERnU6Hoig0bNjQ8nbiLpaXl0dFRQUODg6Ul5fj5eWFs7MzJpMJk8mEra0tlZWVZGdnq8cQlpeX4+zsjJeXl+XthBBCWJDwKIQQQgghrCZzHoUQQgghhNUkPAohhBBCCKtJeBRCCCGEEFaT8CiEEEIIIawm4VEIIYQQQlhNwqMQQgghhLCahEchhBBCCGE1CY9CCCGEEMJqEh6FEEIIIYTVJDwKIYQQQgirSXgUQgghhBBWk/AohBBCCCGsJuFRCCGEEEJYTcKjEEIIIYSwmoRHIYQQQghhNQmPQgghhBDCahIehRBCCCGE1SQ8CiGEEEIIq2kURVEsC8W9R1EUDh06RFFRETY28plCCEuKomBnZ0fPnj1xcnKyrBZCiHuGhEcBwJ49e1i8eDENGza0rBJCACaTCVdXV2bNmoW3t7dltRBC3DMkPAoAPv74Y9q1a8egQYMsq4QQQgghVDI+KQDIzc2VXkchhBBC3JSER0F+fj6KoshQnBBCCCFuSsKjIDc3F0VR8PHxsawSQgghhDAj4VGQnp6Oi4sL9vb2llVCCCGEEGYkPApSUlJo3LixZfGfLisri/LycstiIYQQQtQhEh4FCQkJNG/e3LL4T1VZWcnHH3/MpUuXLKuEEEIIUYdIeLzHKYpCWloaoaGhllV/KgcHB958803atWtnWSWEEEKIOkTC4z0uKysLGxubW7rSOj09nV9//ZWCggLs7Owsq4UQQghRh8gm4fe4w4cPs2/fPt588021zGg0EhUVRWFhISaTCaPRaHaNJY1Gg0ajQavV4u7uTtu2bdW6+Ph4jh8/jpOTE+vXr2fp0qU4OjqaXX8rVFRUUFZWhpeXl2WVuAsUFxejKAru7u6WVVA1TaK0tLTW339BQQH29vY4OztbVmEymTh69CgpKSn4+/vTvXt3WUgmhBAWpOfxHhcXF0dQUJBZmUaj4YsvvmDo0KGsWbOGyMhIjh8/XuNx5MgRtm3bxhdffMHLL7/M4MGDGT9+PBkZGeq9KisrGTRoEM7OzlRUVKDVas2e61YoKChgxYoVlJWVWVaJa+Tk5HDmzBkuXLhAYmKiZbWZnJwcLl68yOnTp0lOTuav/MyZkpLCqlWran0N1WX29vbs3LmTEydOWDahpKSE77//npKSErNyk8nEunXrqKiooE2bNmzevJnXXnvtph+ehBDinqOIe9p7772n7N2717JYuXDhglK/fn1l9OjRil6vt6yuITk5WVmwYIESFBSkLFmyxLJaef3115UPPvjAsvhPZzKZlI8//ljZvn27ZZWwcP78eeXdd99V/Pz8lHbt2inl5eWWTRRFUZSSkhJl4sSJioeHhzJnzhxl586dislksmx2WxQXFytvvvmmEhkZaVmlbNu2TZk9e7ZSXFysKIqi5OXlKW+88YYSFxdn2VTZtm2bsmDBArOyuLg4ZfDgwUpMTIyiKIpy6dIlpVmzZsqpU6fM2gkhxL1Oeh7vYTqdjpKSEho0aGBZRevWrVm4cCG//PILCxcutKyuITAwkJdffpk1a9YQExNDaWmpWqcoChcuXKBDhw5m19wKBw4cICMjQ87otkKbNm2YPn06PXr0oKKigqysLMsmKIrCiRMnSE5OZsCAAfz73/9m4MCBaDQay6a3xc8//4y7uzv33XefZRVJSUmUl5erw8z16tXjwQcf5LvvvrNsyuDBg8nIyOC3335Ty/z8/BgzZgyurq4AaLVatFqt9DwKIYQFCY/3sPz8fAwGA76+vpZVADz22GNMnz6dd999l4MHD1pW16p79+4MGDCAvLw8tSwlJYXi4uLbstJ67dq1DBs27C8LN3XNyZMnGTRoEHZ2drUOXcfGxqLT6cjKyqJr167Y2tpaNrltysvL2bp1K2PGjLGsAuC5555j4cKFZnMUH3zwQWJjYzl//rxZW41Gw4ABA1i9ejUmkwkAFxcXpk2bhr+/PwAbN27k/vvv5/777ze7Vggh7nUSHu9h1eHxeiutbWxsePvtt2nRogUvvfRSrT1Ttenfvz/+/v5cunQJg8HAvn37aNy4ca09nH+my5cvk5GRQefOnS2rVEVFRVy8eJGioiLLKqia95aSksKFCxfM5kyWlJQQExODTqeDqp/dhQsXyMnJueZqc9nZ2Zw/f97qn9tfITo6mn79+mFnZ0dCQoJZXVlZGXFxcbi4uFBZWVlrb1+19PR0Lly4QG5urmUVRqORuLi4WsOpwWAw66kuLi7m0qVLpKamWjbl0KFD2NnZ0aRJE8sq4uPja/1dODg4qPMXLbVp04a0tDSSkpIsq9i8eTP5+fksWrQIGxv5z6QQQlxLO3fu3LmWheLecOrUKYqKiujVq5dllcrJyYnWrVuzePFiMjIyGDp06E3fTG1sbCgsLOTFF1/Ew8ODAwcOMH36dBo2bGjZ9E+1b98+0tLSGDt2rGUVAPv37yc8PBwXFxcuXrzIpk2bsLe3p1GjRlC1bdEXX3yhnvW9Z88eWrZsiY2NDTt37uT06dNqgDlz5gyVlZUsXboUPz8/s2BcUVHBd999x9mzZ3FyciIiIgInJ6fr9vD+VQoLC4mOjqZXr16sXbsWX19fevfurdYfOnSIli1bcujQIU6dOsXrr7+Oi4uL2T2Ki4v5+uuviYuLw9bWlgMHDlC/fn08PT2h6jm++OILCgoKKCwsZO/evURERNClSxc0Gg179uwhMjKSX375BT8/P44cOYKiKKxZs4aysjJatGihPte6detwd3enT58+17wCOHjwIFeuXOH777+nTZs2eHh4mNUXFRWxe/fuGj2Wzs7ObNu2DX9/f7NN8vfv309GRgZ///vfyc7OprS09LqruoUQ4l504xQg7mqxsbG0bNnSsriG7t2788477/Ddd9+xfPlyy+pa1atXjzfeeAOTycRrr712W4b+4uLi8PPzsywG4Ndff2Xu3LmMGjWKwYMHExQUxAcffKD2JGZkZDB58mQ8PT154oknGDhwIAaDgYSEBFJTU6msrKR37958+OGHXLx4kVGjRqnzKlesWKE+j6IozJo1i9OnT/PUU08xYMAAAgMDOXfunNqmut1fLSoqigYNGuDs7Ez9+vVJTk5W6xITEzEajQQGBnL8+HEaN25M/fr1za4vKytjxowZZGVl8dRTT/HQQw/h5uZGVFQUVIXoF198EYPBwKRJkxg8eDA7d+5k586d2NjYkJKSQkFBAb169SI8PJwtW7YwePBg+vbtS0BAAMuWLVOHlBVFISYmhpCQELPXEBsbS2pqKn379uXUqVPExsaa1QM0adKErKysGr2iGo2GRo0acfHiRbVs9erVLFq0iLKyMr777jvef/999Hq92XVCCHGvk/B4D4uPj7f6WMLp06fz6KOP8tprrxEZGWlZXYNGo6Fbt26MHDlS7dm71TIyMvDx8bEspry8nH//+988+OCDtG7dGqrCiL+/P61atQJg3rx5ZGVlMX78eHQ6HadOnaK4uJimTZtSUFBAjx49OHnyJM2bN+eRRx5R5/4VFhaa7Re4fv16fv75Z6ZOnYqTkxPJycmcO3dOfV6qhmrffffdGoGSqtdx6NAhy2JVcXExSUlJJCcnX/eRlJRkNuf0ejIzM9V9EBs3bkxmZiZGoxGj0ci5c+fo1q0beXl5nDt3rtbe6W+++YYTJ04wefJkNBoNMTExJCYmqr2Fq1at4vTp00ydOhWq/iZsbW3VnsOKigrCwsIoKChAp9MxYcIE3N3dURSFrKwsHB0d1V5uvV5PTk5OjQCr0+lo3749ly9fJjMzk7CwMLN6AE9PT0pKSiguLjYr12g0uLu7c/XqVaj62V6+fFntLd69ezfOzs63fLqFEELUNRIe71GVlZXk5eXV6Mm5Hq1Wy4cffoi3tzfbt2+3rL4jVFRU1HqCzd69e0lOTmbIkCFqWUREBG3atMHLy4vi4mLWrl1LixYt2LNnD+vWrSMpKYkZM2bg6upK+/btCQwMZO/evfTs2VMdks3MzOTkyZP07dtXve/atWtxdXUlNjaWNWvWcPDgQR577DGzxUK2trb069eP4OBgtaxar169bhjof/vtN9555x0++OCD6z7effddwsPDMRgMlperDAYDRUVFarBv3rw5GRkZFBcXc+LECYKDg3F1dSU1NZXo6Giz75GqQL5p0ya8vb2JjIxkzZo1nDp1iqlTp9KkSROMRiNff/01PXv2VHuDr169yvnz59XwGBoaSps2bdTfRWBgoHr/48ePm82xNJlM6HS6Ght2t2rVilatWvHjjz8yYMCAWnueNRrNdTe712q1au+zq6src+bM4YcffuD7779n9erVfPbZZzg4OFheJoQQ9zQJj/eoxMREvL29f9dpL2lpafTp04fp06dbVt0WiqJQUlLCxYsXqaiosKzG3d3dbIugatHR0djb25udfLN//346d+6MVqslKysLnU7HpEmTGDNmDE888QSjR49WFxLZ2NhQUlJCfHw8HTt2VO8RGRmJyWSiY8eOGI1GCgsLyc7OZtiwYYwfP54JEyYwceJEswUeBoOBtLQ0OnXqZDaPrry8nJSUFDp27HjDuZEjRoxg6dKlLF68+LqPb775hmefffaGK6NLS0spKCigcePGADRt2pTs7Gyio6MpKipSe/AiIyPx8PCocfZ5Xl4eeXl5jB8/nrFjxzJp0iTGjx9PQEAAVK2wz8nJMft5xcfHoygKoaGharBVFIX9+/fTs2dPtZfx3LlzpKSkMHToULWdra0tLi4uNTb21mg05ObmcujQIR5++GGo6o28ll6vx97evtYQqNfrcXNzg6p73Ww+rxBCCAmP96y4uDgCAgKsfrOMj49nyZIlvPLKK9ddnX2rHT16lG+++Ybnn3+egoICy2qCgoLMTrep5ujoSP369dXh5aSkJOLi4tRg4+Ligre3d43FIFevXiUmJgaAixcvUlZWRteuXdX6devWMXDgQFxcXPjll1+gaq6n5ZF4hYWFXLhwAYA9e/awevVq/vOf/5jVb926lV27djFr1ix1nt+tlJOTg5ubm/r7DwoKIj8/n927d9OlSxdsbW1RFIVff/2V7t271/ieXF1da/1es7Oz1XmHrq6uanBWFIUtW7bQqVMnTCYT27dvR6/XU1xcTHx8vNkeoPv27cPX15d27dqxevVqSkpK0Gq1BAQEqEPM19q6dSseHh707NmT3bt311g9nZWVhbu7e42FNIqikJOTU2sPsBBCiOuzLjkIldFoJD09/YZDgsXFxbVuG3IniY+PNxsmvJGioiLmz5/P448/rs4RtNa1w4VGo7FGr9DvERoaykMPPYRWq611wUmzZs3UeXvXGjhwIL6+vkRFRXH16lXmzZuHyWSiffv2APj7+9OvXz/OnDmDwWCgpKSECxcusHnzZpycnKBqZbq9vb1ZL2JGRgZt27bl0qVLODs74+HhwciRI9WtZ0pLS0lJSSE8PBy9Xk9mZiZOTk40atTILARduHCB9u3b07ZtW65cuXJLw6NOpyMhIYFPP/2UlJQUdcui6iDYsWNHPD09ycnJISoqir179+Lr66uec17Nw8ODIUOGcPr0aSorK9We2Y0bN2IwGGjcuDFt27YlISEBo9HI4cOHOXXqFA0bNiQ5ORmNRoO9vT0nT57EwcFBXbilKApJSUl0796d6OhoXFxccHV1RaPR0LJly1oXxCQmJtKkSRPKyspITU2ladOmZvUxMTEEBgaqPYzVdDodaWlpt2X/USGEuJvcNVv1mEwmjh8/zqFDh7h06RI6nQ5PT0+uXLmCp6fn7zpTOSUlhc2bN3P48GEaNWqknjgBsH37dmJiYmjbtu11N6KuqKhg5cqV+Pv7q/Pj7jTbtm2jY8eOVi1mef/99wkNDWX8+PGWVbVKSEhAo9Hg4ODArl27OHLkCM7Ozpw/f55jx46ZDZfq9Xp0Oh0Gg6HWh9FoRKvVotFocHFxoby8nJ9++olHH320RhhwcXEhPDyc/v37m9V5e3vTunVrjhw5gk6nIzk5Gb1ez/PPP6+26datG9HR0aSkpBAbG0tJSQnDhw9XF2jEx8fTvn17s2FYPz8/UlNTcXZ2pn///mi1Wjp06EBpaSkXL17k6tWrJCYm0q9fP5o3b46rqysBAQHMmzePESNG0KZNGwAaNWqEj48P4eHhaLVahg4dqj7Hn+3q1ausXLkSRVHQ6/UYDAZatmyJRqOhQ4cO9OnTB1tbW7Zs2cLevXtp06YNzs7OpKenExYWZjantEuXLmRkZBATE0NycjJXr15l6NChBAUFodFo6NixIxcuXCA1NRUHBwcmTJhAeno6JpOJPn364OTkREpKCvXq1WPYsGFQNXTs7+9PVlYWRqORvn37qlMrtFotP//8M2PHjjXrMW/UqBH5+fkUFBQwZMgQsx5kRVFYuXIlnTp1qrFPZXJyMtu3b+epp56q0esshBDi+jRKbV04dUxFRQULFy7E1dWV7t27o9Fo1KDi6enJe++9p7bNycmhrKyMoKAgs3tcKycnh++++44PPviAiIgItVfk5MmTbN68mTfeeKPG/KlXX32VgIAA/vnPf0LVNiirVq3izTff/F3zCm+HkpIS3nnnHZ577rmbLphZvnw58fHxzJ0716oAXlpayhdffMGUKVNIT08nLy+PRYsW0axZM15//XUiIyP56quvWLVqFQ4ODmzatImIiIha5+eZTCbq1avHjBkz1CHHxMREnnrqKVatWlXrKti33nqLxo0bM2XKFKjq7SwoKFA/QBiNRh577DHatGlDbZ+bSkpKcHR0rPF6qv+ZWH5gqKysxM7Orsbwf3l5ORqNpsbv/tKlS7z66qt89tln6HQ6NdAoisKUKVMYPHgwjz/+uNk1d7qysjK0Wm2NfxNUzfHU6/VqD65er8fGxuamf0uVlZXY2tqatVMUhRdeeIFx48bV2OvRYDCgKEqNBVNXr15l1qxZfPbZZzVWan///fckJCQwZ84cs3IhhBA3dlcMW3/66acUFxfzwgsv0LFjRzp06MDEiRO5cuWK2sNV7d133+WTTz4xK7Pk4+ODr68vHTp0UId2KyoqWLZsGQ8//HCNN0mTyYSHh4fZhsatWrXCx8eHVatWmbW9HUwmU61zw6oVFBRQUVGhHsN2Pb/99hsnT57k9ddfv+mbPVWnrvzzn/8kKysLPz8/vL29CQ4Opri4mMceewwvLy90Op1ZeHjkkUeYP38+H374YY3HvHnzeP3112vMVbuRyZMnc/z4cXWrmm3bttG/f3/1xJJdu3aRkJDAk08+aXHl/3F1da0RHKkKjZbBkaoTTCyDI1Wbq1sGR4DU1FS8vLxISkoyG8LPzMxUF8zUNc7OzjX+TVSztbVVgyOAnZ2dVX9LDg4ONdppNBqeeOIJtmzZUmPvRVtb2xrBEeCXX36hb9++NYJjXl4eR48eZfLkyWblQgghbq7mu14dk5CQwM8//8xjjz1mVq7Vaunbt6/ZUXWFhYWcPn2aLl26mLWtzb59+2jXrp06nHXixAlMJlOt86NsbGyYO3cuI0aMMCsfPnw4O3fuJD8/36z8Vjt48CDffvutZbGqeu7dtW/qli5fvsyHH37IwIEDSUtL49KlSzUe58+f59SpU/zyyy+8+eabdO3alR9//JHBgwcDEBgYSG5uLs7Ozmrv7Z49e+jRoweKomAwGDh//jwbNmxg06ZNNR4bN25k586dlJeXq6+rOsDVFuQAgoODGTx4MBs2bICqYNO4cWMuXLjAjz/+yL59+1i6dGmtR9zdDj179uTZZ58lMDCQLl26UFRUhKIoxMXF4ejoaNU0gntZ9+7dadmyJVu2bLGsquHs2bPk5eXV+G+DyWRi7dq1DB8+vMaHSyGEEDdX58NjSkoKycnJZGZmWlbRuXNnwsLC0Ov1FBUVceHCBRITE2nWrBnFxcU1Fl3ExMRw7tw5SktLuXz5stkK0N27d9OuXbsavVJlZWVERkbW2ICYqrlYNjY2tW4GfasYDAYWLFhww42m4+Pjb/qm+dNPP5Gfn8/ixYuZMWMGL7/8co3H3/72N2bMmMFbb73Fjh07cHJyYujQoWYbSh8/flzdM7CsrIyLFy/SuXNnNm3ahNFoxGQyqb16tT1MJpPZ76m0tJTKykqzQGlp5MiRBAUFERsbS//+/Vm9ejUtW7akb9++fPjhh2a/19vN2dmZvn370qZNG/R6PbNmzeLYsWP89NNPPP744zL3zgqTJ09GW7XF0vXodDqSkpJ45plnzOYsc80Cm+oPOUIIIX6fOj/nMTk5mf79+1NSUsLUqVPp378/rVq1okGDBphMJmxsbEhKSmLDhg0cPXqUkydPMm3aNEJCQhg3bhy2trYUFRXx+eefq2fc7tixgy1btrBhwwZCQ0NRFIVHH32Ul156yWyuVVlZGT///DOKohAREcGnn35aY/ju9ddfx8/Pj1deecWs/FqlpaVWr7B1dnauMZx3raioKCZPnozBYODXX3+tdcHOBx98QLdu3Wps/FxNURSKi4tvuKK8muVwrq2trdmb9VdffUVQUBDDhw9Hr9fz6aef4ubmRr9+/aw6GvFau3fv5sSJEyQlJdG0aVN69OhBz549LZtB1fdgMBhqHcq8kyxfvpySkhLCwsLo27dvrUPgoiaTyYTJZKrxYa5a9Yr72v6t6PX6O/7vQggh7mR1PjwC/Pzzz7z++uvqmbrBwcF8+OGHNYarpk+fjqOjIwsXLjQrf/bZZ3F3d2f+/PkAvPHGG2zfvl09hi83N5cxY8bw9ddf06xZM/W68PBwWrRoQW5uLq+//jo///xzjQ2ev/rqK6Kiovj888/NyquZTCY+/fRTMjIyan2jszRt2rTrnkCi0+nYtm0biYmJfPbZZ2zdulVd0XutF154gdmzZ99w0dCtcr2FJ0IIIYSoG+6K8EjVfMa4uDj27t3LN998Q0VFBQcPHlSHZ41GI127duWVV15h0qRJ6nW//PILr776Klu2bFH3MHzxxRfRarVqyLxy5QrPPvss69atMwuH+/fvp1u3bsycOZOCggJWrlyp1lX7/vvv2bx5M+vWrbOsUmVlZVnVy0fVtjOWvZvVDFVHzl26dImRI0fy448/mh3JR1UQ/ve//80nn3xy3TmPy5cvJzk52aowK8TdTlEUHB0defHFF6/7b0YIIe4ldTo8Vq9crT45pFpERAR9+vTh0KFD6okgUVFRPP7446xcudJs0cvYsWMpKipi586dULVVS+/evXnzzTd59NFH1Wufe+45Nm7cWON0lfT0dIYPH17rghmAH374gU2bNqkLOG6HrKwsQkND+eijj8z2MuSa7Ybmzp173d6/zZs3k5GRIUOoQlSFRwcHB8aNG3fdD25CCHEvqdPh8csvv2TMmDH4+fmZlcfGxjJ06FC2bdumnsn77bff8t1337Fv3z40Go16Ru6gQYPo27cvH3zwAVStMh46dChbtmyhcePG2Nvbk5mZyaRJk1i5cmWNU1k2bNjAW2+9xeHDh6msrKyxIfm3337LoUOHWL58udl11RRFYdWqVeTk5FgV1kaNGlXjNVgyGAy0a9eOgQMHsmDBArO68PBwsrOza4RKIYQQQghr3Dyt3KEqKyvZt28fubm5llXs2LGD/v37mx1TFhkZSYsWLbC1teXSpUscOXIEBwcHnJyczOb+bdu2jcaNG+Pj48Pu3bspLi7G19dXDZGW1q9fz8iRI9FoNOzatavG/nNZWVk3DXtBQUG0aNGC5s2b3/RhzbCZra0tYWFhXLx4scbrSU1NpWHDhmZlQgghhBDWqrPh8cqVKyQmJrJu3ToSExMxmUyUlZURHh7O+fPnmTNnjtmwrIODA76+vmRkZHD27Fk6dOiAo6MjgwYNIiYmhvLyciIiIjh79iz16tUjNTWViooK6tWrh52dHY0bN671XN2ioiJatmzJkSNHCAkJqbExdEJCQo1j0a6l0Wjo1asXgwcPZtCgQTd9+Pj4WN6iVu3atSMqKspsSxuDwUBubu5Nw6y4/QwGA5WVlepJKTdjMBioqKio8eFACCGEuNXq7NnWycnJdOnShY4dO7J37171NBStVstzzz1X4/SUFi1akJOTQ3FxMd27d1d73zp06EBZWRkxMTG4u7vz+OOPY29vT3FxMX379lXnU2ZnZ3Pu3DkGDhxodt/Q0FDS0tIICgqiU6dOZkPPubm5rFu3jieeeAIvLy+z6261zMxMVqxYwbRp06hXrx5UBd3du3czZMiQGnvfib/Wb7/9xrfffsvHH39MYmIiDzzwQK3TGMrLy/nPf/7Dp59+yuXLl7G3tyc4ONiymRBCCHHL1Nk5j9du+aLT6aioqECr1d5wk+XqDactVxGbTCYMBgP29vbq11SdHFPt6tWrzJw5k/nz59c4U9lgMKDVamssQPnll1+IiIjgo48+Miu/HY4dO8ZDDz3E6tWrGTZsGFQF7vfff5///ve/ls3FX8xkMrF//37eeecd8vPz2b9/P+7u7pbNiIiIYMqUKYSFhfHDDz/g4OCg/t0KIYQQt0PNro064trNqe3t7XF3d79hcKQqDFoGx+rya9+AbWxsavT6NGzYkH79+tV6LJqtrW2N4FhcXMzevXuZNm2aWfnt4uvrS/369Tl//rxalpCQUKNHVtwZbGxsyMnJYdiwYWRkZFBZWWnZhKtXrxIfH6+eUuPm5ibBUQghxG1XZ8PjX2Hy5Mnk5eVx7NgxyyoziqKwZs0aBgwYcN0NvW81Pz8//P39zcJjTEwMLVq0MGsn7gxGo5GysjJat25NZWUl2dnZZvUVFRWcP38eLy8vcnJyzI6AFEIIIW4nCY+/g62tLdOnT6ekpASdTmdZrcrMzKR58+YMHTrUsuq2cXZ2JigoSF1MhITHO1p6ejparZamTZvi5eVFYmKiWf25c+cICQkhJSUFe3t7wsLCzOqFEEKI20XC4+/k6elJv379bjhc6Ofnx4MPPljrEPnt1Lx5c3JyckhPT0dRFHJzcwkJCbFsJu4Aubm52Nra0qhRI3x9fYmJiVHrMjMzycnJITQ0lN9++43u3btbtWWTEEIIcStIePwDLOc3WrpZ/e3SunVr8vPzycjIICMjAw8Pjxqn8Yg7Q2pqKoGBgTg6OuLn56eGR6PRyMmTJ+natau6nVTv3r0tLxdCCCFuGwmPd7GwsDCKi4vJyMggOTkZHx8fOV7tDhUTE0Pbtm0B8Pf3JyEhAaqGq729vfH29ubSpUuUlJTccN9QIYQQ4laT8HgXa9y4Ma6ursTExBAfHy8ny9yhysrKMBqNuLu7o9FoaNSoEYWFhWRnZ5Oamsr9998PVXtBBgcHy9QDIYQQfykJj3cxZ2dnmjVrxpkzZ+RYwjvY+fPnadWqlfp1SEgIhYWF/Pzzz3Tq1Ak7OzsUReHo0aM0a9asxlnuQgghxO0k4fEuptVqad++PSdOnLDqjG1x+xmNRn755RezDcFDQkLIy8vD09NT3ZA+OTmZ6OhoOnTocM3VQgghxO1XZ48nFNZJTk5mxYoVBAYG8sQTT9xwlbi4vaKiovjyyy/Zv38/aWlpeHt7ExgYiJ2dHSaTiaeffhqDwUB4eDgbN24kMzMTR0dHdT/IO2VhlhBCiHtLnT2eUFhn586dDB48mKeeeoply5ZZVou/UFlZGUVFRTg6OqLT6XB0dFR7IPV6vRoic3NzMZlMajuNRoOPj4/l7YQQQojbQoat73J+fn74+PjI5uB3IGdnZ/z9/fH09MTX19ds6NrOzg6qji2sX78+fn5+eHh4UL9+fQmOQggh/lISHu9y/v7+hISE0Lp1a8sqIYQQQojfTcLjXc7X15cOHTrQsmVLyyohhBBCiN9N5jwKIYQQQgirSc+jEEIIIYSwmoRHIYQQQghhNQmPQgghhBDCahIehRBCCCGE1SQ8CiGEEEIIq0l4FEIIIYQQVpPwKIQQQgghrCbhUQghhBBCWE3CoxBCCCGEsJqERyGEEEIIYTUJj0IIIYQQwmoSHoUQQgghhNUkPAohhBBCCKtJeBRCCCGEEFaT8CiEEEIIIawm4VEIIYQQQlhNwqMQQgghhLCahEchhBBCCGE1CY9CCCGEEMJqEh6FEEIIIYTVJDwKIYQQQgir/T8niohu2miSegAAAABJRU5ErkJggg==\"\u003e\u003c/h2\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eCode availability:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll code was implemented in Python using PyTorch as the primary deep learning package. The source codes for the GeneBag model\u0026rsquo;s pre-training and subsequent fine-tuning tasks are available at Github (https://github.com/transomics/GeneBag).\u003c/p\u003e\n\u003cp\u003eThe Fine-tuning models are available at Zenodo (https://zenodo.org/records/14499146)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eAtta, Lyla, and Jean Fan. 2021. \u0026ldquo;Computational Challenges and Opportunities in Spatially Resolved Transcriptomic Data Analysis.\u0026rdquo; \u003cem\u003eNature Communications\u003c/em\u003e 12 (1): 5283.\u003c/li\u003e\n\u003cli\u003eBareche, Y., D. Venet, M. Ignatiadis, P. Aftimos, M. Piccart, F. Rothe, and C. Sotiriou. 2018. \u0026ldquo;Unravelling Triple-Negative Breast Cancer Molecular Heterogeneity Using an Integrative Multiomic Analysis.\u0026rdquo; \u003cem\u003eAnnals of Oncology: Official Journal of the European Society for Medical Oncology / ESMO\u003c/em\u003e 29 (4): 895\u0026ndash;902.\u003c/li\u003e\n\u003cli\u003eBartocci, Ezio, and Pietro Li\u0026oacute;. 2016. \u0026ldquo;Computational Modeling, Formal Analysis, and Tools for Systems Biology.\u0026rdquo; \u003cem\u003ePLoS Computational Biology\u003c/em\u003e 12 (1): e1004591.\u003c/li\u003e\n\u003cli\u003eBeltagy, Iz, Matthew E. Peters, and Arman Cohan. 2020. \u0026ldquo;Longformer: The Long-Document Transformer.\u0026rdquo; http://arxiv.org/abs/2004.05150.\u003c/li\u003e\n\u003cli\u003eBostanci, Erkan, Engin Kocak, Metehan Unal, Mehmet Serdar Guzel, Koray Acici, and Tunc Asuroglu. 2023. \u0026ldquo;Machine Learning Analysis of RNA-Seq Data for Diagnostic and Prognostic Prediction of Colon Cancer.\u0026rdquo; \u003cem\u003eSensors \u003c/em\u003e 23 (6). https://doi.org/10.3390/s23063080.\u003c/li\u003e\n\u003cli\u003eCui, Haotian, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang. 2024. \u0026ldquo;scGPT: Toward Building a Foundation Model for Single-Cell Multi-Omics Using Generative AI.\u0026rdquo; \u003cem\u003eNature Methods\u003c/em\u003e, February, 1\u0026ndash;11.\u003c/li\u003e\n\u003cli\u003eDevlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. \u0026ldquo;BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.\u0026rdquo; https://doi.org/10.48550/ARXIV.1810.04805.\u003c/li\u003e\n\u003cli\u003eDolly, Y. Wu, Ann E. Spangler, Dat T. Vo, Alberto de Hoyos, and Stephen J. Seiler. 2020. \u0026ldquo;Simplified, Standardized Methods to Assess the Accuracy of Clinical Cancer Staging.\u0026rdquo; \u003cem\u003eCancer Treatment and Research Communications\u003c/em\u003e 25 (January): 100253.\u003c/li\u003e\n\u003cli\u003eFeng, Jing, Wenbo Chen, Xin Dong, Jun Wang, Xiangfei Mei, Jin Deng, Siqi Yang, et al. 2022. \u0026ldquo;CSCD2: An Integrated Interactional Database of Cancer-Specific Circular RNAs.\u0026rdquo; \u003cem\u003eNucleic Acids Research\u003c/em\u003e 50 (D1): D1179\u0026ndash;83.\u003c/li\u003e\n\u003cli\u003eFloridi, Luciano, and Massimo Chiriatti. 2020. \u0026ldquo;GPT-3: Its Nature, Scope, Limits, and Consequences.\u0026rdquo; \u003cem\u003eMinds and Machines\u003c/em\u003e 30 (4): 681\u0026ndash;94.\u003c/li\u003e\n\u003cli\u003eFranz\u0026eacute;n, Oscar, Li-Ming Gan, and Johan L. M. Bj\u0026ouml;rkegren. 2019. \u0026ldquo;PanglaoDB: A Web Server for Exploration of Mouse and Human Single-Cell RNA Sequencing Data.\u0026rdquo; \u003cem\u003eDatabase: The Journal of Biological Databases and Curation\u003c/em\u003e 2019 (April): baz046.\u003c/li\u003e\n\u003cli\u003eGiulietti, Matteo, Alessandra Righetti, Giovanni Principato, and Francesco Piva. 2018. \u0026ldquo;LncRNA Co-Expression Network Analysis Reveals Novel Biomarkers for Pancreatic Cancer.\u0026rdquo; \u003cem\u003eCarcinogenesis\u003c/em\u003e 39 (8): 1016\u0026ndash;25.\u003c/li\u003e\n\u003cli\u003eGolub, T. R., D. K. Slonim, P. Tamayo, C. Huard, M. Gaasenbeek, J. P. Mesirov, H. Coller, et al. 1999. \u0026ldquo;Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring.\u0026rdquo; \u003cem\u003eScience\u003c/em\u003e 286 (5439): 531\u0026ndash;37.\u003c/li\u003e\n\u003cli\u003eGomez-Ramirez, Jaime, and Ricardo Sanz. 2013. \u0026ldquo;On the Limitations of Standard Statistical Modeling in Biological Systems: A Full Bayesian Approach for Biology.\u0026rdquo; \u003cem\u003eProgress in Biophysics and Molecular Biology\u003c/em\u003e 113 (1): 80\u0026ndash;91.\u003c/li\u003e\n\u003cli\u003eGwon, Hye Ran, A. La Woo, Seung Hyun Yong, Youngmok Park, Song Yee Kim, Eun Young Kim, Ji Ye Jung, et al. 2024. \u0026ldquo;Factors Affecting Accuracy of Clinical Staging in Resectable Non-Small Cell Lung Cancer in a Real-World Study.\u0026rdquo; \u003cem\u003eThoracic Cancer\u003c/em\u003e 15 (9): 730\u0026ndash;37.\u003c/li\u003e\n\u003cli\u003eHao, Minsheng, Jing Gong, Xin Zeng, Chiming Liu, Yucheng Guo, Xingyi Cheng, Taifeng Wang, Jianzhu Ma, Le Song, and Xuegong Zhang. 2023. \u0026ldquo;Large Scale Foundation Model on Single-Cell Transcriptomics.\u0026rdquo; \u003cem\u003ebioRxiv\u003c/em\u003e. https://doi.org/10.1101/2023.05.29.542705.\u003c/li\u003e\n\u003cli\u003eHua, Junjie T., Sujun Chen, and Housheng H. He. 2019. \u0026ldquo;Landscape of Noncoding RNA in Prostate Cancer.\u0026rdquo; \u003cem\u003eTrends in Genetics: TIG\u003c/em\u003e 35 (11): 840\u0026ndash;51.\u003c/li\u003e\n\u003cli\u003eIorio, Marilena V., and Carlo M. Croce. 2012. \u0026ldquo;MicroRNA Dysregulation in Cancer: Diagnostics, Monitoring and Therapeutics. A Comprehensive Review.\u0026rdquo; \u003cem\u003eEMBO Molecular Medicine\u003c/em\u003e, February. https://doi.org/10.1002/emmm.201100209.\u003c/li\u003e\n\u003cli\u003eKitano, Hiroaki. 2002. \u0026ldquo;Systems Biology: A Brief Overview.\u0026rdquo; \u003cem\u003eScience\u003c/em\u003e 295 (5560): 1662\u0026ndash;64.\u003c/li\u003e\n\u003cli\u003eLi, Guanqiao, Yang Liu, Hongxi Hu, Shuona Yuan, Liyun Zhou, and Xiaoyuan Chen. 2022. \u0026ldquo;Evolution of Innovative Drug R\u0026amp;D in China.\u0026rdquo; \u003cem\u003eNature Reviews. Drug Discovery\u003c/em\u003e 21 (8): 553\u0026ndash;54.\u003c/li\u003e\n\u003cli\u003eLi, Jiao, Dan Sun, Wenchen Pu, Jin Wang, and Yong Peng. 2020. \u0026ldquo;Circular RNAs in Cancer: Biogenesis, Function, and Clinical Significance.\u0026rdquo; \u003cem\u003eTrends in Cancer Research\u003c/em\u003e 6 (4): 319\u0026ndash;36.\u003c/li\u003e\n\u003cli\u003eLoewen, Gregory, Janarthanan Jayawickramarajah, Ying Zhuo, and Bin Shan. 2014. \u0026ldquo;Functions of lncRNA HOTAIR in Lung Cancer.\u0026rdquo; \u003cem\u003eJournal of Hematology \u0026amp; Oncology\u003c/em\u003e 7 (1): 1\u0026ndash;10.\u003c/li\u003e\n\u003cli\u003eLonsdale, John, Jeffrey Thomas, Mike Salvatore, Rebecca Phillips, Edmund Lo, Saboor Shad, Richard Hasz, et al. 2013. \u0026ldquo;The Genotype-Tissue Expression (GTEx) Project.\u0026rdquo; \u003cem\u003eNature Genetics\u003c/em\u003e 45 (6): 580\u0026ndash;85.\u003c/li\u003e\n\u003cli\u003eLu, Jun, Gad Getz, Eric A. Miska, Ezequiel Alvarez-Saavedra, Justin Lamb, David Peck, Alejandro Sweet-Cordero, et al. 2005. \u0026ldquo;MicroRNA Expression Profiles Classify Human Cancers.\u0026rdquo; \u003cem\u003eNature\u003c/em\u003e 435 (7043): 834\u0026ndash;38.\u003c/li\u003e\n\u003cli\u003eMa, Xiaoshi, Jinan Guo, Kaisheng Liu, Lipeng Chen, Dale Liu, Shaowei Dong, Jinquan Xia, et al. 2020. \u0026ldquo;Identification of a Distinct Luminal Subgroup Diagnosing and Stratifying Early Stage Prostate Cancer by Tissue-Based Single-Cell RNA Sequencing.\u0026rdquo; \u003cem\u003eMolecular Cancer\u003c/em\u003e 19 (1): 147.\u003c/li\u003e\n\u003cli\u003eMeng, Shujuan, Hecheng Zhou, Ziyang Feng, Zihao Xu, Ying Tang, Peiyao Li, and Minghua Wu. 2017. \u0026ldquo;CircRNA: Functions and Properties of a Novel Potential Biomarker for Cancer.\u0026rdquo; \u003cem\u003eMolecular Cancer\u003c/em\u003e 16 (1): 1\u0026ndash;8.\u003c/li\u003e\n\u003cli\u003ePlichta, Jennifer K., Samantha M. Thomas, Amanda R. Sergesketter, Rachel A. Greenup, Oluwadamilola M. Fayanju, Laura H. Rosenberger, Nina Tamirisa, Terry Hyslop, and E. Shelley Hwang. 2019. \u0026ldquo;Clinical and Pathological Stage Discordance among 433,514 Breast Cancer Patients.\u0026rdquo; \u003cem\u003eAmerican Journal of Surgery\u003c/em\u003e 218 (4): 669\u0026ndash;76.\u003c/li\u003e\n\u003cli\u003ePrada-Luengo, I\u0026ntilde;igo, Viktoria Schuster, Yuhu Liang, Thilde Terkelsen, Valentina Sora, and Anders Krogh. 2023. \u0026ldquo;N-of-One Differential Gene Expression without Control Samples Using a Deep Generative Model.\u0026rdquo; \u003cem\u003eGenome Biology\u003c/em\u003e 24 (1): 263.\u003c/li\u003e\n\u003cli\u003eShang, Qingfeng, Zhi Yang, Renbing Jia, and Shengfang Ge. 2019. \u0026ldquo;The Novel Roles of circRNAs in Human Cancer.\u0026rdquo; \u003cem\u003eMolecular Cancer\u003c/em\u003e 18 (1): 1\u0026ndash;10.\u003c/li\u003e\n\u003cli\u003eShen, Yong Shen Xiaowei Peng. 2020. \u0026ldquo;Identification and Validation of Immune-Related lncRNA Prognostic Signature for Breast Cancer.\u0026rdquo; \u003cem\u003eGenomics\u003c/em\u003e 112 (3): 2640\u0026ndash;46.\u003c/li\u003e\n\u003cli\u003eTeresa Agull\u0026oacute;-Ortu\u0026ntilde;o, M., Fernando L\u0026oacute;pez-R\u0026iacute;os, and Luis Paz-Ares. 2010. \u0026ldquo;Lung Cancer Genomic Signatures.\u0026rdquo; \u003cem\u003eJournal of Thoracic Oncology: Official Publication of the International Association for the Study of Lung Cancer\u003c/em\u003e 5 (10): 1673\u0026ndash;91.\u003c/li\u003e\n\u003cli\u003eTheodoris, Christina V., Ling Xiao, Anant Chopra, Mark D. Chaffin, Zeina R. Al Sayed, Matthew C. Hill, Helene Mantineo, et al. 2023. \u0026ldquo;Transfer Learning Enables Predictions in Network Biology.\u0026rdquo; \u003cem\u003eNature\u003c/em\u003e 618 (7965): 616\u0026ndash;24.\u003c/li\u003e\n\u003cli\u003eValihrach, Lukas, Peter Androvic, and Mikael Kubista. 2020. \u0026ldquo;Circulating miRNA Analysis for Cancer Diagnostics and Therapy.\u0026rdquo; \u003cem\u003eMolecular Aspects of Medicine\u003c/em\u003e 72 (April): 100825.\u003c/li\u003e\n\u003cli\u003eVaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. \u0026ldquo;Attention Is All You Need.\u0026rdquo; https://doi.org/10.48550/ARXIV.1706.03762.\u003c/li\u003e\n\u003cli\u003eVerduci, Lorena, Sabrina Strano, Yosef Yarden, and Giovanni Blandino. 2019. \u0026ldquo;The circRNA-microRNA Code: Emerging Implications for Cancer Diagnosis and Treatment.\u0026rdquo; \u003cem\u003eMolecular Oncology\u003c/em\u003e 13 (4): 669\u0026ndash;80.\u003c/li\u003e\n\u003cli\u003eVivian, John, Jordan M. Eizenga, Holly C. Beale, Olena M. Vaske, and Benedict Paten. 2020. \u0026ldquo;Bayesian Framework for Detecting Gene Expression Outliers in Individual Samples.\u0026rdquo; \u003cem\u003eJCO Clinical Cancer Informatics\u003c/em\u003e 4 (February): 160\u0026ndash;70.\u003c/li\u003e\n\u003cli\u003eWang, Hongbo, Qinghai Meng, Jinjun Qian, Muxi Li, Chunyan Gu, and Ye Yang. 2022. \u0026ldquo;Review: RNA-Based Diagnostic Markers Discovery and Therapeutic Targets Development in Cancer.\u0026rdquo; \u003cem\u003ePharmacology \u0026amp; Therapeutics\u003c/em\u003e 234 (June): 108123.\u003c/li\u003e\n\u003cli\u003eWang, Sumeng, Ke Zhang, Shanyue Tan, Junyi Xin, Qianyu Yuan, Huanhuan Xu, Xian Xu, et al. 2021. \u0026ldquo;Circular RNAs in Body Fluids as Cancer Biomarkers: The New Frontier of Liquid Biopsies.\u0026rdquo; \u003cem\u003eMolecular Cancer\u003c/em\u003e 20 (1): 1\u0026ndash;10.\u003c/li\u003e\n\u003cli\u003eWang, Xuefeng, Shuo Zhang, Yuqin Liu, Jian Du, and Heng Huang. 2021. \u0026ldquo;How Pharmaceutical Innovation Evolves: The Path from Science to Technological Development to Marketable Drugs.\u0026rdquo; \u003cem\u003eTechnological Forecasting and Social Change\u003c/em\u003e 167 (June): 120698.\u003c/li\u003e\n\u003cli\u003eWang, Yanying, Lei He, Ying Du, Pingping Zhu, Guanling Huang, Jianjun Luo, Xinlong Yan, et al. 2015. \u0026ldquo;The Long Noncoding RNA lncTCF7 Promotes Self-Renewal of Human Liver Cancer Stem Cells through Activation of Wnt Signaling.\u0026rdquo; \u003cem\u003eCell Stem Cell\u003c/em\u003e 16 (4): 413\u0026ndash;25.\u003c/li\u003e\n\u003cli\u003eWen, Guoxia, Tong Zhou, and Wanjun Gu. 2020. \u0026ldquo;The Potential of Using Blood Circular RNA as Liquid Biopsy Biomarker for Human Diseases.\u0026rdquo; \u003cem\u003eProtein \u0026amp; Cell\u003c/em\u003e 12 (12): 911\u0026ndash;46.\u003c/li\u003e\n\u003cli\u003eWilks, Christopher, Shijie C. Zheng, Feng Yong Chen, Rone Charles, Brad Solomon, Jonathan P. Ling, Eddie Luidy Imada, et al. 2021. \u0026ldquo;recount3: Summaries and Queries for Large-Scale RNA-Seq Expression and Splicing.\u0026rdquo; \u003cem\u003eGenome Biology\u003c/em\u003e 22 (1): 323.\u003c/li\u003e\n\u003cli\u003eWu, Qiong, Jiali Ma, Jue Wei, Wenying Meng, Yugang Wang, and Min Shi. 2021. \u0026ldquo;lncRNA SNHG11 Promotes Gastric Cancer Progression by Activating the Wnt/\u0026beta;-Catenin Pathway and Oncogenic Autophagy.\u0026rdquo; \u003cem\u003eMolecular Therapy: The Journal of the American Society of Gene Therapy\u003c/em\u003e 29 (3): 1258\u0026ndash;78.\u003c/li\u003e\n\u003cli\u003eXia, Qianlin, Tao Ding, Guihong Zhang, Zehuan Li, Ling Zeng, Yanjun Zhu, Jianming Guo, et al. 2018. \u0026ldquo;Circular RNA Expression Profiling Identifies Prostate Cancer- Specific circRNAs in Prostate Cancer.\u0026rdquo; \u003cem\u003eCellular Physiology and Biochemistry: International Journal of Experimental Cellular Physiology, Biochemistry, and Pharmacology\u003c/em\u003e 50 (5): 1903\u0026ndash;15.\u003c/li\u003e\n\u003cli\u003eXu, Wei, Gai Zhou, Huizhi Wang, Yawen Liu, Baoding Chen, Wei Chen, Chen Lin, Shuhui Wu, Aihua Gong, and Min Xu. 2020. \u0026ldquo;Circulating lncRNA SNHG11 as a Novel Biomarker for Early Diagnosis and Prognosis of Colorectal Cancer.\u0026rdquo; \u003cem\u003eInternational Journal of Cancer. Journal International Du Cancer\u003c/em\u003e 146 (10): 2901\u0026ndash;12.\u003c/li\u003e\n\u003cli\u003eYang, Fan, Wenchuan Wang, Fang Wang, Yuan Fang, Duyu Tang, Junzhou Huang, Hui Lu, and Jianhua Yao. 2022. \u0026ldquo;scBERT as a Large-Scale Pretrained Deep Language Model for Cell Type Annotation of Single-Cell RNA-Seq Data.\u0026rdquo; \u003cem\u003eNature Machine Intelligence\u003c/em\u003e 4 (10): 852\u0026ndash;66.\u003c/li\u003e\n\u003cli\u003eYang, Xiaodong, Guole Liu, Guihai Feng, Dechao Bu, Pengfei Wang, Jie Jiang, Shubai Chen, et al. 2023. \u0026ldquo;GeneCompass: Deciphering Universal Gene Regulatory Mechanisms with Knowledge-Informed Cross-Species Foundation Model.\u0026rdquo; \u003cem\u003ebioRxiv\u003c/em\u003e. https://doi.org/10.1101/2023.09.26.559542.\u003c/li\u003e\n\u003cli\u003eZhang, He-da, Lin-Hong Jiang, Da-Wei Sun, Jun-Chen Hou, and Zhen-Ling Ji. 2017. \u0026ldquo;CircRNA: A Novel Type of Biomarker for Cancer.\u0026rdquo; \u003cem\u003eBreast Cancer \u003c/em\u003e 25 (1): 1\u0026ndash;7.\u003c/li\u003e\n\u003cli\u003eZhang, Li, Chenkai Lv, Yaqiong Jin, Ganqi Cheng, Yibao Fu, Dongsheng Yuan, Yiran Tao, Yongli Guo, Xin Ni, and Tieliu Shi. 2018. \u0026ldquo;Deep Learning-Based Multi-Omics Data Integration Reveals Two Prognostic Subtypes in High-Risk Neuroblastoma.\u0026rdquo; \u003cem\u003eFrontiers in Genetics\u003c/em\u003e 9 (October): 477.\u003c/li\u003e\n\u003cli\u003eZhan, Yonghao, Zhicong Chen, Shiming He, Yanqing Gong, Anbang He, Yifan Li, Lianghao Zhang, et al. 2020. \u0026ldquo;Long Non-Coding RNA SOX2OT Promotes the Stemness Phenotype of Bladder Cancer Cells by Modulating SOX2.\u0026rdquo; \u003cem\u003eMolecular Cancer\u003c/em\u003e 19 (1): 1\u0026ndash;13.\u003c/li\u003e\n\u003cli\u003eZheng, Grace X. Y., Jessica M. Terry, Phillip Belgrader, Paul Ryvkin, Zachary W. Bent, Ryan Wilson, Solongo B. Ziraldo, et al. 2017. \u0026ldquo;Massively Parallel Digital Transcriptional Profiling of Single Cells.\u0026rdquo; \u003cem\u003eNature Communications\u003c/em\u003e 8 (1): 1\u0026ndash;12.\u003c/li\u003e\n\u003cli\u003eZhu, Hu, Ze Wang, Yu Shi, Yingying Hua, Guoxia Xu, and Lizhen Deng. 2020. \u0026ldquo;Multimodal Fusion Method Based on Self-Attention Mechanism.\u0026rdquo; \u003cem\u003eProceedings of the ... International Wireless Communications \u0026amp; Mobile Computing Conference / Association for Computing Machinery. International Wireless Communications \u0026amp; Mobile Computing Conference\u003c/em\u003e 2020 (September): 1\u0026ndash;8.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-5720342/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5720342/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eNumerous Pre-trained cell foundation models (CFM) have been developed to encapsulate the comprehensive gene-gene interaction network within cells, leveraging extensive single-cell sequencing data. These models have shown promise in various cell biology applications, including cell type annotation, perturbation inference, and cell state embedding, etc. However, their clinical utility, particularly in cancer diagnosis and prognosis, remains an open question. We introduce the GeneBag model, a novel CFM that represents a cell as \u0026ldquo;a bag of unordered genes\u0026rdquo; with continuous expression values and a full-length gene list. Pre-trained on single-cell data and fine-tuned on bulk RNA-seq datasets, GeneBag achieves superior performance across cancer diagnosis and prognosis scenarios. In a zero-shot learning setting, GeneBag can classify cancer and non-cancer tissues with approximately 96.2% accuracy. With fine-tuning, it can annotate 40 different types of cancers and corresponding normal biopsies with an overall accuracy of ~\u0026thinsp;97.2%. It notably excels in classifying challenging cancers such as bladder (93%) and stomach (90%). Furthermore, GeneBag is capable of cancer staging with 68.5% accuracy and 1 to 5 year survival prediction with an AUC of 76.98% \u0026minus;\u0026thinsp;82.81%. This study marks the first to demonstrate the potential of CFMs in RNA-based cancer diagnostics and prognostics, indicating a promising avenue for AI-assisted molecular diagnosis.\u003c/p\u003e","manuscriptTitle":"GeneBag: training a cell foundation model for broad-spectrum cancer diagnosis and prognosis with bulk RNA-seq data","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-03-19 10:58:04","doi":"10.21203/rs.3.rs-5720342/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"f057775f-7b64-414e-b0ed-342687aab871","owner":[],"postedDate":"March 19th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":42684951,"name":"Biological sciences/Computational biology and bioinformatics/Computational models"},{"id":42684952,"name":"Biological sciences/Cancer/Cancer models"}],"tags":[],"updatedAt":"2025-03-19T10:58:07+00:00","versionOfRecord":[],"versionCreatedAt":"2025-03-19 10:58:04","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5720342","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5720342","identity":"rs-5720342","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}