DNA-Methyaltion-Based Deep Learning for Precision Classification of Central Nervous System Tumors: A Comparative Study

DNA-Methyaltion-Based Deep Learning for Precision Classification of Central Nervous System Tumors: A Comparative Study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article DNA-Methyaltion-Based Deep Learning for Precision Classification of Central Nervous System Tumors: A Comparative Study Brent Orr, Quynh Tran, Alex Breuer, Tong Lin, Ruth Tatevossian, and 13 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3897766/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 02 Oct, 2024 Read the published version in npj Precision Oncology → Version 1 posted 10 You are reading this latest preprint version Abstract As part of the advancement in therapeutic decision-making for brain tumor patients at St. Jude Children’s Research Hospital (SJCRH), we develop and compare the performance of three classification models: a deep learning neural network (NN), an exact bootstrap k-nearest neighbor (kNN), and a random forest classifier (RF) model to predict the 82 molecularly distinct central nervous system (CNS) tumor classes based on DNA-methylation profiles of 2,801 patients. We validate their classification accuracy, precision, and recall with 2,054 samples from two independent cohorts. Although all models perform robustly to missing data, the NN model achieves the highest classification accuracy and maintains better balance between precision and recall than kNN and RF. Average precision and recall of NN reduce to that of RF and kNN only when tumor purity was less than 50%. In conclusion, DNA-methylation based deep learning approach provides the most potential advancement toward precision medicine for brain tumors. Biological sciences/Computational biology and bioinformatics Health sciences/Oncology/Cancer/CNS cancer deep learning machine learning neural network central nervous system tumor DNA methylation Figures Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 INTRODUCTION Brain tumor classification has historically relied on morphologic examination of tumor specimens under a light microscopy 1 . Refinement of the process has occurred through recognition of additional tumor-specific histologic patterns and by integrating testing modalities such as cytogenetics, immunohistochemistry, and nucleic acid sequencing findings into the classification schemes. More recently, it has been recognized that normal and neoplastic tissues have inherent epigenetic signatures encoded in their methylome 2–5 . The pattern of CpG methylation has been established as a stable and reliable biomarker for tumors and normal tissues 2,6 . The DNA methylation signature is considered a combined surrogate of the cell of origin and genomic driver abnormality. It is retained even after tumor recurrence or passage of tumors as an orthotopic xenograft 7–9 . Histologically-defined tumor types often consist of heterogeneous molecular subtypes with distinct biological and clinical behavior 10,11 . Due to the lack of recurrent defining mutations, some of these molecular subtypes may only be recognizable by their distinct methylation or transcriptomic signatures 5,12 . The adoption of high-density methylation arrays (Illumina Infinium 450K and 850K EPIC arrays) has allowed for genome-wide evaluation of DNA methylation from large cohorts of human tumors. These arrays have favorable characteristics, including relatively low cost, comparable performance on both fresh or formalin-fixed and paraffin-embedded (FFPE) tissues, and stability of the methylation mark even in material stored for multiple decades 13 . In addition, the ability to assay FFPE has facilitated the accumulation of large tumor cohorts and allows for easy integration into standard clinical workflows 5,6,14 . The initial utility of methylation profiling to refine the classification of brain tumors relied on an unsupervised analysis comparing specific tumor cohorts of interest to reference brain tumor types 5,12,14,15 . However, the introduction of supervised classification models based on methylation data has significantly improved clinical diagnostics. In addition, these models are amenable to the types of performance metrics typically utilized for clinical tests 6,16 . The current state-of-the-art model for methylation-based classification relies on a random forest (RF) classifier trained on a reference cohort containing all tumor entities represented in the 2016 WHO Classification of Tumours of the Central Nervous System 6 . While showing relatively good performance, certain model features remain suboptimal, yielding either misclassification or a substantial number of subthreshold scores for selected tumor types 6,17 . Several models utilizing simple and advanced machine learning techniques have been proposed to improve classification accuracy. For example, the well-received k-nearest neighbor (kNN) algorithm was adapted and showed high capability and notable sensitivity in the acceptable discrimination of classes 18–21 . The kNN approach was also employed in a hybrid classification model combining genetic algorithms and artificial neural networks 22 . This hybrid model showed more effectiveness in classifying performance than expectation and maximization classifiers. Recently, artificial neural networks (NN) have been considered by researchers as one of the most useful and applicable constructs in artificial intelligence 23–25 . Specifically, deep NNs have been widely adopted in the diagnostic process of various biomedical fields and provided many opportunities to improve health care and heighten precision in the oncologic pathology 23,26–28 . Despite their effectiveness and popularity, neither k-NN nor deep learning NN has been explored in the epigenetic field to classify the complex molecular entities among central nervous system (CNS) tumors. Currently, no commercial models are available for CNS tumor classification based on DNA methylation data. As part of our clinical implementation of DNA methylation profiling at St. Jude Children’s Research Hospital (SJCRH), we developed a multi-layer perceptron NN model (NNmod) based on DNA methylation profiles of 2,801 samples from a reference tumor cohort to predict the 82 histologically and/or molecularly distinct CNS tumor classes and 9 normal controls. We also compared the performance characteristics of this NNmod to the current state-of-the-art RF classifier 6 (RFmod) and another alternative model using an exact bootstrap k-nearest neighbor (kNNmod) algorithm. We validated the performance of these models with two independent brain tumor cohorts consisting of 1104 samples from GSE109379 and 950 samples from the St. Jude Children’s Research Hospital. Our results showed that although all models performed robustly to missing data, the deep NN model had the highest CNS classification accuracy and the most favorable performance characteristics, especially in minimizing the proportion of subthreshold score during testing and validation. Average precision and recall of the NNmod started reduce to similar levels of kNNmod and RFmod when tumor purity was less than 50%. This suggests that a deep NN model can be implemented in clinical laboratories as a reliable and essential diagnostic tool to assist in precision therapy for brain tumors. RESULTS Model performance on train and test set : We developed three models, i.e., a k-nearest neighbors model (kNNmod), a random forest model (RFmod), and a multi-layer perceptron neural network model (NNmod) (Fig. S1 ), to classify human CNS tumors based on methylation signatures of the comprehensive reference set (GSE90496, n = 2801). This set comprises 91 methylation classes grouped into 75 methylation class families based on their histological and biological closeness (Capper et al., 2008). The RF model represented a recapitulation of the previous random forest produced by Capper et al., representing the best in the current model. Here, we compared the performance of kNNmod and NNmod to the RFmod. The three models were evaluated with 1000 leave-out-25% cross-validations in predicting methylation classes and families (Fig. S2 ). All models produced accuracies above 0.95 for both class and family prediction (Table 1 ). Among the three models, classification accuracy and its Kappa statistic were highest in NNmod (above 0.98) and lowest in kNNmod (0.90 and 0.95 for class and family prediction) (Table 1 ). These accuracies were statistically significantly different from the null accuracy, i.e. the accuracy could be achieved by predicting the most frequent class (McNeMar's p-values < 10 –16 ). These results suggest that all models produced useful predictions with high accuracy. Table 1 Overall performance of leave-out-25% train-test process for each classifier on the GSE90496 RF kNN NN Class Family Class Family Class Family Accuracy 0.98 0.99 0.95 0.96 0.99 0.99 Accuracy Null 0.05 0.11 0.06 0.11 0.06 0.12 Kappa 0.96 0.99 0.90 0.95 0.98 0.99 Recall 0.97 0.98 0.86 0.88 0.98 0.99 Precision 0.96 0.985 0.90 0.92 0.98 0.995 Specificity 0.99 0.99 0.99 0.99 0.99 0.99 Cross-validation misclassifications by RFmod and NNmod focused on a few methylation classes while miss-classifications by kNNmod spread into many classes. Cross-validation of RFmod, kNNmod, and NNmod resulted in an average accuracy of 98%, 95%, and 99% for class prediction, respectively (Table 1 ). Notably, NNmod produced the best accuracy in predicting methylation class in all 1000 cross-validation rounds (Table S1 ). kNNmod, compared to RFmod and NNmod, had the lowest precision (90% vs. 96% and 98%, respectively) and recall (86% vs. 97% and 98%). All models had comparable specificity (around 99%). The majority of miss-classifications among three models occurred within the six histologically and biologically closely related tumor classes (pituitary adenomas - PITAD and PITUI) and myxopapillary ependymomas (EPN-MPE) (Fig. 1 A). However, kNNmod misclassification expanded to other methylation classes such as ependymomas (EPN), supratentorial subependymomas (SUBEPN), low-grade gliomas (LGG), melanomas (MELAN), melanocytomas (MELCYT), and plexus tumors (PLEX) (Fig. 1 Aii). On the other hand, NNmod had the narrowest ranges in accuracy, precision, and recall of predicting 91 subclasses with a median value around 0.98 for each metric (Fig. 1 B-C and Table S1 ). Minimal F1 scores for RFmod, kNNmod, and NNmod were 0.729, 0.359, and 0.761, respectively (Fig. 1 D and Table S1 ), suggesting NNmod had the best balance between precision and recall. All models, in general, performed better at predicting methylation families. The classification metrics of these 75 methylation families are shown in Table 1 and Fig. S3 . The cross-validation accuracies for the clinically relevant groupings were improved in all models. Compared to kNNmod, NNmod showed higher accuracy (99% vs 96%), precision (99% vs 88%), and recall (99.5% vs 93%). Compared to RFmod, the NNmod showed higher recall (99% vs. 98%) and comparable accuracy (~ 99%), precision (~ 98%), and specificity (~ 99.9%) (Table 1 ). Among the 1000 cross-validation rounds in predicting methylation family, NNmod produced the best accuracy 604 times, while RFmod produced the best accuracy 280 times. The rest of the cross-validation rounds, NNmod and RFmod had the same accuracy that was higher than kNNmod (Table S2 ). The misclassification of RFmod and NNmod among the CNS tumor classes shown in Fig. 1 A appeared to be dissolved but retained in kNNmod (Fig. S3 A). Although accuracy was improved for all models, the gap between precision (88%) and recall (93%) for kNNmod (Fig. S3 B-D and Table S1 ) remained the same in predicting methylation families. Minimal F1 scores for RFmod and NNmod were increased to 0.878 and 0.883, while this score was reduced to 0.318 by kNNmod (Table S2 ). In conclusion, these results indicated that although RFmod and NNmod had very comparable performance, NNmod still had the highest accuracy and the best balance between precision and recall among the three models, suggesting that it had the highest discriminating power for both methylation class and family. Model performance on two independent validation sets : The classification performance of the three models was additionally tested on two independent data sets (GSE 109379) and the SJCRH data sets. To objectively assign each independent test sample to the reference methylation class group, we performed a semi-supervised learning approach 29 to assign labels to the two validation data sets. The 1,104 samples were assigned to 65 methylation classes and 50 families (Table S3 ), while the 950 SJCRH samples were grouped into 49 methylation classes (Table S4 ). This result was then used as the ground truth to measure the accuracy of the prediction results from our classifiers. We evaluated the performance of each classifier at multiple probabilistic prediction cutoffs ranging from 0 to 0.9 with an 0.1 increment. Figure 2 shows the overall average precision and recall at each cutoff for each classifier when validating on GSE109379 and SJCRH data sets. Although all models had their prediction precision increase as the threshold increased for both class (red line) and family (blue line) prediction, the recalls that met the cutoff dropped quickly to around 65% in RFmod and kNNmod, but it stayed above 0.75 in NNmod (Fig. 2 and Table S5 - S8 ). For application to diagnostic tumor samples, an optimal calibrated score threshold of ≥ 0.9 was selected 6 . For subclasses within methylation class families, a threshold value of ≥ 0.5 was defined as sufficient for a valid prediction, as long as all family member scores add up to a total score of ≥ 0.9. Single class specificity and sensitivity are provided in Table S5 - S8 . While RFmod and kNNmod had average balanced accuracies below 90% (82% and 86%, respectively), NNmod achieved 91% accuracy for methylation class prediction of GSE109379. At the 0.9 threshold for methylation class prediction, NNmod maintained good recalls (> 82% for class and > 84% for family) with > 90% precision, while RFmod and kNNmod produced recalls of less than 60% (Fig. 2 A and Table 2 ). When predicting SJCRH at the 0.9 threshold, NNmod and RFmod achieved comparable results with balanced accuracies of 94% and 93%, recalls of 89% and 87%, and precisions of 96% and 92%, respectively. Meanwhile, kNNmod got 84% accuracy with 70% recall and 80% precision, respectively (Fig. 2 B and Table 2 ). These results suggested that NNmod could identify the most positive calls with higher accuracy and precision. Table 2 Performance of each classifier when predicting methylation class in the independent test sets at 0.9 threshold Data set RF kNN NN Precision Recall Accuracy Precision Recall Accuracy Precision Recall Accuracy GSE109379 0.87 0.73 0.86 0.76 0.64 0.82 0.91 0.82 0.91 SJCRH 0.92 0.87 0.93 0.8 0.7 0.84 0.96 0.89 0.94 Model robustness CNS tumor classification of our classifiers is based on features that measure DNA methylation at different CpG sites in the human genome using probes on Illumina BeadChip arrays. This microarray technology is easy to use, time-efficient and cost-effective. However, it keeps evolving, and in each new release, more probes are printed to cover more diverse genomic regions, and some probes are purposely removed for efficiency. Other potential applications, such as detection of tumors by cell free DNA testing, may also have uneven or missing values. Because the missing probes could differentially affect model performance, we investigated whether the performance of the three classifiers was robust in producing consistent outputs in class labels and their corresponding prediction scores even when a proportion of input probes were not present. We performed an experiment in which we randomly dropped 10% of the probes in the independent test data sets GSE109379 and SJCRH. We repeated this process 10 times to create 10 different missing probes scenarios. The robustness of each classifier was accessed based on the Theil’s U uncertainty coefficients between the two sets of predicted labels and Spearman’s correlation coefficients between prediction scores with and without missing probes. Table 3 shows that RFmod and NNmod have Theil’s U uncertainty coefficients greater than 0.94, suggesting that the predicted labels by RFmod and NNmod before probe drop-out were as similar as those produced after probe drop-out. In contrast, kNNmod has the lowest uncertainty coefficient among the three classifiers with Theil’s U ranging from 0.889 to 0.908 for methylation family and class prediction. These results indicate that the two sets of predicted labels are strongly associated. All models have Pearson correlation coefficients > 0.928 with p-values < 2.2e -16 , suggesting a strong and statistically significant linear correlation between prediction scores produced when 10% of probes were missing and when there were no missing probes (Table 4 ). Figure 3 shows the regression analysis of the two sets of classification scores. Scores produced by NNmod and kNNmod with drop-out data set were generally higher than those that were output using all probes as indicated with positive y-intercepts (Fig. 3 ). RFmod, when using data with missing probes, produced lower classification scores (negative y-intercepts). All models had the goodness-of-fit R-squared of at least 86%, indicating a strong correlation between the two sets of prediction scores (Fig. 3 ). These results suggest that missing probes do not affect the prediction outcomes of any classifiers. Table 3 Theil’s U uncertainty coefficient with a 95% confidence interval of each classifier with and without dropping 10% of probes in the GSE109379 and SJCRH independent test sets Data set RFmod kNNmod NNmod Class Family Class Family Class Family GSE109379 0.969 (0.967, 0.972) 0.974 (0.970, 0.978) 0.892 (0.887, 0.898) 0.908 (0.902, 0.914) 0.973 (0.970, 0.9796) 0.980 (0.976, 0.983) SJCRH 0.945 (0.940, 0.949) 0.964 (0.960, 0.968) 0.889 (0.883, 0.894) 0.899 (0.894, 0.905) 0.970 (0.966, 0.974) 0.979 (0.975, 0.982) Table 4 Pearson's correlation coefficients of prediction scores with and without 10% missing probes Data set RFmod kNNmod NNmod Class Family Class Family Class Family GSE109379 0.98 0.99 0.93 0.94 0.94 0.94 SJCRH 0.99 0.99 0.95 0.96 0.99 0.99 Model assessment based on sample purity Infiltrating of normal cells such as epithelial, stromal, and immune cells in tumor tissue can perturb the tumor signal in molecular studies. In our application, this contamination can affect the methylation level measured by microarray chips, leading to possible degradation in the performance of a classifier. Therefore, we developed an in silico experiment in which different fractions of the normal control cells were mixed with the tumor tissue to answer these questions: (1) whether a classifier produces unexpected methylation class/family prediction (2) if yes, would the prediction have a suprathreshold score, and (3) approximately at what percentage of control contamination, the classification accuracy starts to degrade. We first observed the overall performance of the three classifiers based on their average recall and precision for methylation class and family prediction at different thresholds and purity fractions (Fig. 4 ). NNmod started to perform the best, as seen in previous sections. RFmod degraded at a comparable rate with kNNmod after the sample purity was less than 65%. As the purity of tumor samples was less than 40%, NNmod started to yield lower precision and recall compared to the other two classifiers (Fig. 4 , threshold = 0). At the 0.9 clinical threshold and 0.95 purity, NNmod had twice the average recalls and a much higher average precision than RFmod and kNNmod. The performance of NNmod did not start to degrade at a similar rate to RFmod and kNNmod until the purity of tumor samples was less than 50%. As the contamination increased, RFmod had the lowest performance among the three classifiers (Fig. 4 , threshold = 0.9). NNmod maintained the highest average precision and recall among the three classifiers. Its performance reduced to a comparable level with RFmod and kNNmod only when the tumor purity was less than 50%. Next, we observed the prediction results of each classifier for each methylation class and family. Figures 5 and 6 show the performance of RFmod, kNNmod, and NNmod at different control fractions in the tumor sample for methylation class diffused midline glioma H3 K-27 mutant (DMG, K27) and glioblastoma, IDH wildtype, H3.3 G34 mutant (GBM, G34), respectively. When the high-grade DMG, K27 tumors got contaminated with control, RFmod and NNmod did not produce unexpected methylation classes besides DMG, K27 and its corresponding mixed control cerebella hemisphere (CONTR, CEBM) class (Fig. 5 A, B, G, and H) or family (Fig. S4 A, C, D, and F). On the other hand, kNNmod unexpectedly predicted these high-grade gliomas to be low-grade pilocytic astrocytoma (LGG, PA PF) with scores above the clinical threshold (0.9) (Fig. 5 D, E and Fig. S4 B, E). kNNmod and RFmod could not accurately predict the methylation class of the DMG, K27 tumors if the purity of these samples was less than 70% (Fig. 5 C, F and Fig. S4 G, H). Meanwhile, NNmod was able to maintain its prediction accuracy prediction for DMG, K27 samples unless the sample’s purity dropped below 40% (Fig. 6 I). Figure 6 shows that when predicting the GBM, G34 methylation class, RFmod did not provide any suprathreshold results if more than 30% of control tissues were present in the mixture (Fig. 6 A, B, and C). On the contrary, kNNmod accurately predicted these samples until the contamination was up to 60%. At this fraction, kNNmod unexpectedly classified these grade IV glioblastomas as grade I dysembryoplastic neuroepithelial gliomas (LGG, DNT) (Fig. 6 D, E and F). NNmod did not provide any suprathreshold classification to GBM, G34 samples besides their corresponding normal hemispheric cortex (CONTR, HEMI) starting at 70% contamination (Fig. 6 G, H and I). Similar results were shown in Fig. S5 for GBM, G34 samples at the methylation family. DISCUSSION We developed a deep neural network model to predict CNS tumor classification based on a large DNA-methylation data set from 2801 patients of 82 distinct CNS tumors and 9 controls. Our multilayer perceptron neural network classifier achieved high performance, as demonstrated in 3 different evaluation settings. Compared with RFmod, a current-state-of-the-art CNS tumor classifier based on DNA-methylation 6 , our NNmod showed higher overall accuracy (99% vs. 98%), precision (98% vs. 97%) and recall (98% vs. 96%) and comparable specificity (~ 99%) in methylation class prediction (Table 1 ). Among the three developed models, the kNN model produced the lowest accuracy (95%), precision (86%), and sensitivity (90%) (Table 1 ). In addition, we showed that our DNN model is highly robust and generalizable as evaluated in an independent testing dataset of 1104 GSE109379 samples (65 tumor classes) and 700 classifiable SJCRH samples (45 tumor classes), with an overall accuracy of 91% and 94%. Among these results, NNmod showed the highest accuracy and the best balance between precision and recall compared to RFmod and kNNmod (Table 1 , Fig. 1 – 2 ). All classifiers were trained on the reference data (GSE90496) generated from the Illumina Human Methylation 450K chips. These chips featured 485,577 CpG sites throughout the human genome, but they became obsolete and have been replaced by the Ilumina HumanMehtylationEPIC BeadChip (EPIC). EPIC measures methylation at > 850,000 CpG sites and covers approximately 90% of the same sites represented on the 450K chip. EPIC eliminates sites reported to be poorly performed 30 and features more CpGs that cover more regulatory elements. When using classifiers trained on data produced by 450K chips to predict samples ran on EPIC chips, it is possible that some probes used for prediction are no longer present on EPIC chips and could hinder the classifier performance. As such, we performed a random probes drop-out experiment to evaluate the classification performance of RFmod, kNNmod, and NNmod (Tables 3 and 4 ). Although having the probes dropped out randomly may be adequate, it would be additionally useful to know in the future whether dropping all the poorly performed probes in the 450K training data set would enhance the performance and increase the robustness of all classifiers. Diagnosis of CNS tumors is a complex multiclass classification problem as the number of diagnostic classes in which patients are stratified is not limited to a few selected classes but rather to a very high list of entities represented in the 5th edition of the WHO Classification of CNS Tumors 6 . It has been shown that diagnostic accuracy can be improved by utilizing a robust machine-learning classification algorithm based on DNA-methylation profiles obtained from formalin-fixed, paraffin embedded (FFPE) or frozen tissue samples 6 .The preparation of FFPE samples is one of the most widely used procedures to preserve and archive specimens in clinical oncology. This workflow requires an invasive tissue biopsy to be performed on patients. Recently, the use of liquid biopsies, a less invasive method for cancer detection has rapidly gained prominence 31 . Particularly, plasma cell-free DNA methylation profiles have been shown to be highly sensitive, cost effective, and accurate in early tumor detection for cancer interception, and for multi-cancer classification 32,33 . Our study demonstrated that NNmod was the top stand-alone classifier among the three developed classifiers using DNA-methylation signatures from FFPE samples. The 11-layer perceptron NNmod maintained high recalls (> 82% for GSE109379 and > 90% for SJCRH) above an 0.9 clinical threshold with > 0.92 precision when validated with two independent data sets (Fig. 2 and Table S5 - S8 ). With these improvements over RFmod, NNmod represents a viable method that could be used in conjunction with clinical, histopathologic, and molecular data to aid in the diagnosis and classification of CNS tumors. Future study would be to apply this machine learning modeling with the DNA-methylation profiles from plasma cell-free DNA obtained through the less invasive liquid biopsy procedure. MATERIALS and METHODS Patient material. FFPE or frozen tumor samples representing pediatric patient samples encountered on the typical pathology service were evaluated. The samples represented 650 samples expected to be present in the reference series and 300, true negative samples representative of non-brain solid tumors known to be absent from the reference series (Table S9 ). Training and independent testing data sets All supervised models were trained on the genome-wide DNA methylation profiles from the CNS tumor reference cohort (GSE90496), consisting of 2,801 samples from 91 methylation classes 6 . All classifiers were independently validated with two methylation data sets, including the 950 CNS tumor samples from the St. Jude Children's Research Hospital (SJCRH) and 1,104 CNS tumor samples from GSE109379 6 . Data generation and methylation array processing. We analyzed the 950 independent test samples using Illumina Methylation BeadChip (EPIC) arrays according to the manufacturer's instructions. In summary, DNA was isolated from formalin-fixed paraffin-embedded (FFPE) tumor tissue using the Maxwell® Clinical Sample Concentrator system (Promega, Madison, WI). Following extraction, DNA was quantified using a Qubit fluorometer and quantitation reagents (Thermo Fisher Scientific, Waltham, MA), and bisulfite converted using the Zymo EZ DNA methylation kit (Zymo Research, Irvine, CA). The overall DNA input amount was approximately 250 ng. DNA methylation profiling was carried out with the Infinium HumanMethylationEPIC BeadChip (850K) array (Illumina Inc., San Diego, CA) on the Illumina iScan platform. All methylation data analyses, including those from GSE90496 and GSE109379, were performed in R ( http://www.r-project.org , version 3.5.3), using several packages from Bioconductor and other repositories. Specifically, array data were preprocessed using the minfi package (v.1.28.4) 34 . Background correction with dye-bias normalization was performed for all samples using noob (normal-exponential out-of-band) with the "single" dye method 35 . Batch effects such as hybridization time and other technical variables were removed using removeBatchEffect from the limma package (v.3.38.3) 36 . Probe filtering was performed after normalization. Specifically, we removed probes located on sex chromosomes, probes containing nucleotide polymorphism (dbSNP132 Common) within five base pairs, including the targeted CpG-site, or mapping to multiple sites on hg19 (allowing for one mismatch), as well as cross-reactive probes. Semi-supervised analysis. We developed a combination approach including a self-training with editing using a support vector machine (SETRED-SVM) as the base learner model with an L2-penalized, multinomial logistic regression model to obtain high confidence labels from a few reference instances 29 . We applied this approach on GSE109379 and the SJ samples to get labels for the independent validation purpose of the supervised models. The ssc R package (v2.1-0) was used to build and train the SETRED-SMV semi-supervised model. First, the standard deviation for each probe across all 2,801 samples from GSE90496 was calculated. Input features for SSL models were the 5072 probes with a standard deviation greater than 0.3 across all 2801 samples. We used the best SETRED-SVM model to predict the methylation class for 1104 GSE109379 and 950 SJ samples. The SSL scores were calibrated with an L2-penalized, multinomial logistic regression. Scores above the 0.8 threshold were considered correctly classifiable 29 . The random forest algorithm and development. The random forest algorithm was reconstructed from Capper's algorithm 6 using the randomForest R package (v.4.6–14) 37,38 . This model was trained based on the 408,862 overlapping probes of the 450K and 850K array probes. First, the 10,000 features (or probes) with the highest importance scores were selected by splitting the 408,862 intersecting probes into 43 sets of approximately 9500 probes. Next, one hundred trees were fitted for each set using 639 randomly sampled candidate features at each split (mtry = square root of 408,862). The subclass labels, stratified subsampling methods, and the number of trees in the forest were followed as in 6 . This framework can produce a model that either predicts the methylation class or the methylation "family" scores 6 that represent clinically-equivalent families on which Capper et al. witnessed their best error rates. Next, a multinomial logistic regression was used to calibrate the prediction scores from all cross-validation splits as previously described 6 . The family scores were then generated as the sum of all methylation class scores from the trained random forest. The k-nearest neighbor algorithm and development (kNNmod) An exact bootstrap k-nearest neighbor model (kNNmod) was built as described in 39 . The model was trained on score vectors constructed based on the difference in median beta values of the top 100 hyper- and hypo-methylated probes. Each set of 100 top probes was selected based on the mean ß values in a methylation group and the absolute z-scores computed by taking the differences between mean beta values of two CNS methylation groups divided by the square root of the sum of the variance in each group. Hence, each methylation group had a list of 200 probes that were either most hypo- or hypermethylated based on the absolute z-scores. Each sample had a vector of scores, i.e. one score per methylation group. Each score was computed by taking the median ß values of the top 100 hypermethylated probes and subtracting that from the top 100 hypomethylated probes. Euclidean distance on these vectors was used to measure the distance between each pair of samples. The entire Euclidean distance matrix on the methylation group score vectors was computed for all pairwise samples. To classify a new sample, kNNMod ordered all other samples by their distance from the new observation and derived the probability that those neighbors would be included among the k nearest neighbors in the binomial distribution. We used k = 5 neighbors for classification because some subgroups were very rare. For each new sample, the exact bootstrap probability of assignment to each methylation group can be conditionally computed on the training data set and the resulting probe selection and group score definition. The multilayer sparse perceptron architecture and development (NNmod) The design of the multilayer sparse perceptron is shown in Figure S1 . This design is based on two primary assumptions, (i) the methylation data from central nervous system tumors and normal brain is embedded in low dimensional space, and (ii) random combinations of important probes can predict methylation class. The first assumption is typical of high dimensional data and is supported by examination of the singular value decomposition (SVD) of previously published reference data 6 (data not shown). In addition, the ability of combined methylation probes to predict methylation class is supported by previous implementations of random forest classifiers 6 . We constructed an 11-layer perceptron neural net. The input dimension is 51,108, composed of probes selected with feature extraction described in the network training section immediately following this section. The first layer is sparse, while the remaining ten layers are fully connected. The sparse layer maps 139,264 uniformly random sets of 256 features to the space [0,1] 512 . In other words, the sparse layer computes cosines of angles of vectors of length 256 drawn with uniform probability without replacement from the 51,108 input probes, which were selected by a LASSO model. This sparse feature layer can be considered a forest of random decision stumps 40 , the output of which is fed through a standard perceptron. Stochastic gradient descent was performed with a batch size of 32 on logarithms of output scores from the network using a learning rate of 0.001. These gradient descent parameters were obtained via a random search of the parameter space. The log-likelihood loss was minimized over the three-fold cross-validation. Using the evaluation partitions from the cross-validation splits, model calibration was performed with a multinomial logistic regressor. The final model was trained on the complete 2,801 samples using identical parameters following cross-validation. Classifier cross-validation. To reduce the overfitting problem when training classifiers on high-dimensional data, all classifiers were cross-validated based on 1000 leave-out-25% cross-validations. We randomly selected 75% of the data used to train the classifiers (GSE90496), while the remaining 25% of the data were used for predictions. Stratified random sampling was performed for each methylation class or family to ensure the number of categories remained the same in each iteration. This validation process was repeated 1000 times (Fig. S2 ). Model calibration . Calibration of machine learning methods may be necessary because the scores output by the classifier may have different scales when broken down by class, even when the scores are normalized so that they sum to 1. This poses problems for comparing the uncertainty in class or family calls between cases or even in the same case. Thus, the scores must be rescaled to form a well-calibrated multinomial distribution with minimal differences between expected values and variances between the class call groups. Both RF and NN models were calibrated with the same multinomial logistic regression approach described by 6 . The glmnet package (v-4.1-3) 41 was used with R bindings for the random forest and python bindings for the neural net. Model robustness . To test whether missing methylation probes (features) affect our machine learning models, we randomly dropped 10% of the probes from the testing data (GSE109379 and SJCRH) and calculated the accuracy. The same probes at each round were used for all models. This process was repeated ten times to create 10 different missing sets of probes. Pearson’s correlation and Theil’s U uncertainty coefficients were computed using the ggpubr R package (v.0.4.0) and the DescTools R package (v.99.44), respectively. Pearson’s correlation coefficients with p-values were calculated to examine the linear relationship between the two sets of prediction scores (with and without missing probes). Theil’s U uncertainty coefficients were calculated to measure the nominal association between the two sets of labels predicted by the three classifiers on samples of GSE109379 and SJCRH data with and without missing probes. Purity analysis . We performed an in silico simulated impurity experiment using different fractions of control and positive samples in GSE109379 and SJCRH test sets. The experiment was performed based on m -values. The in silico mixed m -values (m mixed ) for each positive sample were computed as follows $${m}_{\text{m}\text{i}\text{x}\text{e}\text{d}}=(1-p){m}_{\text{t}\text{e}\text{s}\text{t}}+p{m}_{\text{c}\text{o}\text{n}\text{t}\text{r}\text{o}\text{l}}$$ where \({m}_{\text{t}\text{e}\text{s}\text{t}}\) is the input m-value from the positive samples, and \({m}_{\text{c}\text{o}\text{n}\text{t}\text{r}\text{o}\text{l}}\) is the average m-values of up to 19 appropriate control (i.e. normal) tissue samples in the test sets, p is the proportion of normal control tissues contaminated in a tumor sample (ranging from 0 to 1 with 0.05 increment). The control samples were selected based on their control methylation class corresponding to the methylation class tumor as described in Table S10 . The final measurement of the mixed sample was then converted back to beta values for classifier inputs. Model performance metrics . All models were evaluated based on accuracy, precision, specificity, recall, and F1 score. Classification accuracy is the number of correct predictions (true positives and true negatives) divided by the total number of predictions. Precision is the ratio of true positives to all the total positives predicted by a classifier. Specificity measures the proportion of true negatives correctly identified by a classification model. Recall or sensitivity is the ratio of true positives to all the ground truth positives. The F1-score is the harmonic mean of precision and recall and a good metric to measure the results in imbalanced classification problems. The higher the F1 score, the better the performance of a model. All measurements were computed using the caret R package (v.6.0–90). Abbreviations CNS central nervous system kNN k–nearest neighbors RF–random forest NN–multi–layer perceptron neural net Declarations SOFTWARE AVAILABILITY: The generated code is available from the corresponding authors upon reasonable request for non-commercial use. AUTHOR CONTRIBUTIONS: AB developed the MLPNet framework and implemented the RF model. QTT modified and maintained the models, analyzed the results, produced figures and tables, and drafted the manuscript. BAO conceptualized the project, interpreted the results and drafted the manuscript. TL and SP implemented the KNN model. RT, SJA extracted the DNA and produced methylation data. MC, LVF, GR, PN, AG, EA, SS, and DWE provided samples and interpreted the results. MC, DH, and TM modified and maintained the MLPNet. All authors reviewed and edited the manuscript. COMPETING INTERESTS: All authors declare no competing interests. MATERIALS and CORRESPONDENCE: All correspondence and material requests should be addressed to Dr. Brent Orr at [email protected] . References Ferguson, S. & Lesniak, M. S. Percival Bailey and the classification of brain tumors. Neurosurg Focus 18 , e7 (2005). https://doi.org:10.3171/foc.2005.18.4.8 Kumar, R., Liu, A. P. Y., Orr, B. A., Northcott, P. A. & Robinson, G. W. Advances in the classification of pediatric brain tumors through DNA methylation profiling: From research tool to frontline diagnostic. Cancer 124 , 4168–4180 (2018). https://doi.org:10.1002/cncr.31583 Moran, S. et al. Epigenetic profiling to classify cancer of unknown primary: a multicentre, retrospective analysis. Lancet Oncol 17 , 1386–1395 (2016). https://doi.org:10.1016/S1470-2045(16)30297-2 Pajtler, K. W. et al. Molecular Classification of Ependymal Tumors across All CNS Compartments, Histopathological Grades, and Age Groups. Cancer Cell 27 , 728–743 (2015). https://doi.org:10.1016/j.ccell.2015.04.002 Sturm, D. et al. New Brain Tumor Entities Emerge from Molecular Classification of CNS-PNETs. Cell 164 , 1060–1072 (2016). https://doi.org:10.1016/j.cell.2016.01.015 Capper, D. et al. DNA methylation-based classification of central nervous system tumours. Nature 555 , 469–474 (2018). https://doi.org:10.1038/nature26000 Smith, K. S. et al. Patient-derived orthotopic xenografts of pediatric brain tumors: a St. Jude resource. Acta Neuropathol 140 , 209–225 (2020). https://doi.org:10.1007/s00401-020-02171-5 He, C. et al. Patient-derived models recapitulate heterogeneity of molecular signatures and drug response in pediatric high-grade glioma. Nat Commun 12 , 4089 (2021). https://doi.org:10.1038/s41467-021-24168-8 Kumar, R. et al. Clinical Outcomes and Patient-Matched Molecular Composition of Relapsed Medulloblastoma. J Clin Oncol 39 , 807–821 (2021). https://doi.org:10.1200/JCO.20.01359 Pugh, T. J. et al. Medulloblastoma exome sequencing uncovers subtype-specific somatic mutations. Nature 488 , 106–110 (2012). https://doi.org:10.1038/nature11329 Robinson, G. et al. Novel mutations target distinct subgroups of medulloblastoma. Nature 488 , 43–48 (2012). https://doi.org:10.1038/nature11213 Northcott, P. A. et al. Medulloblastoma comprises four distinct molecular variants. J Clin Oncol 29 , 1408–1414 (2011). https://doi.org:10.1200/JCO.2009.27.4324 Hovestadt, V. et al. Robust molecular subgrouping and copy-number profiling of medulloblastoma from small amounts of archival tumour material using high-density DNA methylation arrays. Acta Neuropathol 125 , 913–916 (2013). https://doi.org:10.1007/s00401-013-1126-5 Northcott, P. A. et al. The whole-genome landscape of medulloblastoma subtypes. Nature 547 , 311–317 (2017). https://doi.org:10.1038/nature22973 Broniscer, A. et al. Gliomatosis cerebri in children shares molecular characteristics with other pediatric gliomas. Acta Neuropathol 131 , 299–307 (2016). https://doi.org:10.1007/s00401-015-1532-y Danielsson, A. et al. MethPed: a DNA methylation classifier tool for the identification of pediatric brain tumor subtypes. Clin Epigenetics 7 , 62 (2015). https://doi.org:10.1186/s13148-015-0103-3 Fukuoka, K. et al. Clinical impact of combined epigenetic and molecular analysis of pediatric low-grade gliomas. Neuro Oncol 22 , 1474–1483 (2020). https://doi.org:10.1093/neuonc/noaa077 Weinberger, K. Q. & Saul, L. K. Distance Metric Learning for Large Margin Nearest Neighbor Classification. J. Mach. Learn. Res. 10 , 207–244 (2009). Zeng, Y., Yang, Y. & Zhao, L. Nonparametric classification based on local mean and class statistics. Expert Syst. Appl. 36 , 8443–8448 (2009). https://doi.org:10.1016/j.eswa.2008.10.041 Mylonas, P., Wallace, M. & Kollias, S. in Methods and Applications of Artificial Intelligence Vol. 3025 Lecture Notes in Computer Science (eds G.A. Vouros & T. Panayiotopoulos) (Springer, Berlin, Heidelberg, 2004). Sakr, S. et al. Comparison of machine learning techniques to predict all-cause mortality using fitness data: the Henry ford exercIse testing (FIT) project. BMC Med Inform Decis Mak 17 , 174 (2017). https://doi.org:10.1186/s12911-017-0566-6 Aci, M., İnan, C. & Avci, M. A hybrid classification method of k nearest neighbor, Bayesian methods and genetic algorithm. Expert Syst. Appl. 37 , 5061–5067 (2010). https://doi.org:10.1016/j.eswa.2009.12.004 Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H. & Aerts, H. Artificial intelligence in radiology. Nat Rev Cancer 18 , 500–510 (2018). https://doi.org:10.1038/s41568-018-0016-5 Kotsiantis, S. B., Zaharakis, I. D. & Pintelas, P. E. Machine learning: a review of classification and combining techniques. Artificial Intelligence Review 26 , 159–190 (2007). https://doi.org:10.1007/s10462-007-9052-3 Libbrecht, M. W. & Noble, W. S. Machine learning applications in genetics and genomics. Nat Rev Genet 16 , 321–332 (2015). https://doi.org:10.1038/nrg3920 Balyen, L. & Peto, T. Promising Artificial Intelligence-Machine Learning-Deep Learning Algorithms in Ophthalmology. Asia Pac J Ophthalmol (Phila) 8 , 264–272 (2019). https://doi.org:10.22608/APO.2018479 Currie, G., Hawk, K. E., Rohren, E., Vial, A. & Klein, R. Machine Learning and Deep Learning in Medical Imaging: Intelligent Imaging. J Med Imaging Radiat Sci 50 , 477–487 (2019). https://doi.org:10.1016/j.jmir.2019.09.005 Peiffer-Smadja, N. et al. Machine learning for clinical decision support in infectious diseases: a narrative review of current applications. Clin Microbiol Infect 26 , 584–595 (2020). https://doi.org:10.1016/j.cmi.2019.09.009 Tran, Q. T., Alom, M. Z. & Orr, B. A. Comprehensive study of semi-supervised learning for DNA methylation-based supervised classification of central nervous system tumors. BMC Bioinformatics 23 , 223 (2022). https://doi.org:10.1186/s12859-022-04764-1 Pidsley, R. et al. Critical evaluation of the Illumina MethylationEPIC BeadChip microarray for whole-genome DNA methylation profiling. Genome Biol 17 , 208 (2016). https://doi.org:10.1186/s13059-016-1066-1 Diaz, L. A., Jr. & Bardelli, A. Liquid biopsies: genotyping circulating tumor DNA. J Clin Oncol 32 , 579–586 (2014). https://doi.org:10.1200/JCO.2012.45.2011 Shen, S. Y. et al. Sensitive tumour detection and classification using plasma cell-free DNA methylomes. Nature 563 , 579–583 (2018). https://doi.org:10.1038/s41586-018-0703-0 Stackpole, M. L. et al. Cost-effective methylome sequencing of cell-free DNA for accurately detecting and locating cancer. Nature Communications 13 , 5566 (2022). https://doi.org:10.1038/s41467-022-32995-6 Aryee, M. J. et al. Minfi: a flexible and comprehensive Bioconductor package for the analysis of Infinium DNA methylation microarrays. Bioinformatics 30 , 1363–1369 (2014). https://doi.org:10.1093/bioinformatics/btu049 Triche, T. J., Jr., Weisenberger, D. J., Van Den Berg, D., Laird, P. W. & Siegmund, K. D. Low-level processing of Illumina Infinium DNA Methylation BeadArrays. Nucleic Acids Res 41 , e90 (2013). https://doi.org:10.1093/nar/gkt090 Smyth, G. K. & Speed, T. Normalization of cDNA microarray data. Methods 31 , 265–273 (2003). https://doi.org:10.1016/s1046-2023(03)00155-5 Ho, T. K. in IEEE. 278–282. Liaw, A. & Wiener, M. Classification and Regression by randomForest. R news 2 , 18–22 (2002). Steele, B. M. Exact bootstrap k-nearest neighbor learners. Machine Learning 74 , 235–255 (2009). https://doi.org:10.1007/s10994-008-5096-0 in Encyclopedia of Machine Learning (eds Claude Sammut & Geoffrey I. Webb) 262–263 (Springer US, 2010). Friedman, J., Hastie, T. & Tibshirani, R. Regularization Paths for Generalized Linear Models via Coordinate Descent. J Stat Softw 33 , 1–22 (2010). Additional Declarations (Not answered) Supplementary Files FigS1MLPNetArchitecture.png Fig. S1. Architecture of the multi-layer perceptron neural network. We constructed an 11-layer perceptron neural net. The input dimension is 51,108, composed of probes selected with feature extraction described in the Methods section. The first layer is sparse, while the remaining ten layers are fully connected. The sparse layer maps 139,264 uniformly random sets of 256 features to the space [0,1] 512 . This layer is a forest of random decision stumps 40 that computes cosines of angles of vectors of length 256 drawn with uniform probability without replacement from the 51,108 input probes, which were selected by a LASSO model. The output of this layer is then fed through a standard perceptron. FigS2TrainTestValidationscheme.gif Fig. S2. Training, testing, and validation scheme of all classifiers. To reduce the overfitting problem when training classifiers on high-dimensional data, all classifiers were cross-validated based on 1000 leave-out-25% samplings. We randomly selected 75% of the data used to train the classifiers (GSE90496), while the remaining 25% of the data were used for predictions. Stratified random sampling was performed for each methylation class or family to ensure the number of categories remained the same in each iteration. This training and testing process was repeated 1000 times. The final models were validated with two independent data sets: GSE109379 and the St. Jude data set. FigS3HeatmapAccuracyPRF1family.png Fig. S3. Leave-out-25% testing results of each methylation family. (A) Heat map showing methylation family prediction results after 1000 stratified random samplings (i) RF, (ii) kNN, and (iii) NN classifier incorporating information of n = 2,801 reference tumor samples allocated to 75 methylation class families (GSE90496). Deviations from the bisecting line represent misclassification errors (using the maximum calibrated score for class prediction). Boxplots showing (B) the accuracy, (C)precision and recall, and (D) F1-score for each classifier with outliers. FigS4PurityDMGK27family.tif Fig. S4. Classification results of RF, kNN, and NN model for high-grade diffused midline glioblastoma with K-27 mutant (DMG, K27) methylation family at different contamination levels. (A-C) Density plots of all calls (blue curve) and calls over the 0.9 clinical threshold (orange curve) at each possible methylation family predicted by RF, kNN, and NN when the ground truth is DMG, K27 at different fractions of control tissue contamination. (D-F)Box plots show the score distribution for each methylation family predicted by RF, kNN, and NN models. (G-I) Accuracy of each classifier at each purity level. FigS5PurityGBMG34family.tif Fig. S5. Classification results of RF, kNN, and NN model for grade IV glioblastoma, IDH wildtype, H3.3 G34 mutant (GBM, G34)methylation family at different levels of contamination. (A-C) Density plots of all calls (blue curve) and calls over the 0.9 clinical threshold (orange curve) at each possible methylation family predicted by RF, kNN, and NN when the ground truth is GBM, G34 at different fractions of control tissue contamination. (D-F) Box plots show the score distribution for each methylation family predicted by RF, kNN, and NN models. (G-I)Accuracy of each classifier at each purity level. TableS1GSE90496caretmetricssubclass.xls Table S1. Cross-validation performance metrics per methylation class for kNN, RF, and NN TableS2GSE90496caretmetricsfamily.xls Table S2. Cross-validation performance metrics per methylation family for kNN, RF, and NN TableS3GSE109379SSL.xls Table S3. Semi-supervised labeling results with top calibration scores for GSE109379 (n=1104) TableS4SJ950casesSSL.xls Table S4. Semi-supervised labeling results with top calibration scores for 950 SJCRH samples TableS5ACCclassGSE109379.xls Table S5. Performance of kNN, RF, and NN when predicting methylation class in GSE109379 validation data set at 0.9 classification probabilistic threshold TableS6ACCfamilyGSE109379.xls Table S6. Performance of kNN, RF, and NN when predicting methylation family in GSE109379 validation data set at 0.5 classification probabilistic threshold TableS7ACCsubClasssjindep.xls Table S7. Performance of kNN, RF, and NN when predicting methylation class in SJCRH validation cohort at 0.9 classification probabilistic threshold TableS8ACCfamilysjindep.xls Table S8. Performance of kNN, RF, and NN when predicting methylation family in SJCRH validation cohort at 0.5 classification probabilistic threshold TableS9clinicaldataSJCRH.xlsx Table S9. St. Jude Children’s Research Hospital (SJCRH) validation cohort (n=950) characteristics TableS10.docx Table S10. Control tissues used for the in silico mixing experiment Cite Share Download PDF Status: Published Journal Publication published 02 Oct, 2024 Read the published version in npj Precision Oncology → Version 1 posted Editorial decision: revise 19 Mar, 2024 Review # 1 received at journal 24 Feb, 2024 Review # 3 received at journal 20 Feb, 2024 Reviewer # 3 agreed at journal 19 Feb, 2024 Reviewer # 2 agreed at journal 19 Feb, 2024 Reviewer # 1 agreed at journal 11 Feb, 2024 Reviewers invited by journal 07 Feb, 2024 Editor assigned by journal 26 Jan, 2024 Submission checks completed at journal 26 Jan, 2024 First submitted to journal 25 Jan, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3897766","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":271634700,"identity":"490b7a04-39bc-4ca3-9391-9a738d4824ec","order_by":0,"name":"Brent Orr","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA40lEQVRIie3PsQrCMBCA4QuBZAm4ngj6CilCa9GH0SVdCm7SzU6d1FnBhykU7CK+gEtFsI4FwdmWFsdYN8H8Q9Lly10BTKYfTL6/eAiAQFuSuDpETMIvCU5JOacFcXrJ4VrAZO7s8jxzo3Ef+Pp61xF3ozwZg3L3Z98Ku5E3BJHaI+1iR2FjDInEnk9KksxCVEzqSedZk256qciyBRGsJgjVYskUUNFM+y8rZuNRKonCt7Z48qxIHJhOgCPoDYNgIpGn2QMX40GHR7TQmnq9+qJYHuUIhp9JE2lebzPFZDKZ/qgXYLNA0EZc9rEAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0003-0997-4728","institution":"St. Jude Children's Research Hospital","correspondingAuthor":true,"prefix":"","firstName":"Brent","middleName":"","lastName":"Orr","suffix":""},{"id":271634701,"identity":"b350592b-ab3a-41de-8e7d-760d938fdea0","order_by":1,"name":"Quynh Tran","email":"","orcid":"","institution":"St. Jude Children's Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"Quynh","middleName":"","lastName":"Tran","suffix":""},{"id":271634702,"identity":"27a7e184-e97e-46bc-96b3-e08f4c3d4935","order_by":2,"name":"Alex Breuer","email":"","orcid":"","institution":"St. Jude Children’s Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"Alex","middleName":"","lastName":"Breuer","suffix":""},{"id":271634703,"identity":"63084a09-178b-474d-b430-e138c0bc143c","order_by":3,"name":"Tong Lin","email":"","orcid":"","institution":"St. Jude Children’s Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"Tong","middleName":"","lastName":"Lin","suffix":""},{"id":271634704,"identity":"601e908e-ddd6-4bff-87f6-b1aa9cdabb05","order_by":4,"name":"Ruth Tatevossian","email":"","orcid":"","institution":"St. Jude Children’s Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"Ruth","middleName":"","lastName":"Tatevossian","suffix":""},{"id":271634705,"identity":"6a0bbee8-9a68-4487-b7a2-05d8d00b6f4e","order_by":5,"name":"Sariah Allen","email":"","orcid":"","institution":"St. Jude Children's Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"Sariah","middleName":"","lastName":"Allen","suffix":""},{"id":271634706,"identity":"e65c342d-09fb-4199-a6be-906a79c30ff0","order_by":6,"name":"Michael Clay","email":"","orcid":"","institution":"University of Colorado School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Michael","middleName":"","lastName":"Clay","suffix":""},{"id":271634707,"identity":"b4d6f465-986a-47be-a4f9-69925cf83b96","order_by":7,"name":"Larissa Furtado","email":"","orcid":"","institution":"St. Jude Children's Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"Larissa","middleName":"","lastName":"Furtado","suffix":""},{"id":271634708,"identity":"27912629-c991-4a8e-81a1-00ca9b65b1d9","order_by":8,"name":"Mark Chen","email":"","orcid":"https://orcid.org/0000-0003-1169-563X","institution":"Cleveland Clinic","correspondingAuthor":false,"prefix":"","firstName":"Mark","middleName":"","lastName":"Chen","suffix":""},{"id":271634709,"identity":"eef69cc7-e76b-45d9-a3ba-e0037f970a19","order_by":9,"name":"Dale Hedges","email":"","orcid":"","institution":"Aster Insights","correspondingAuthor":false,"prefix":"","firstName":"Dale","middleName":"","lastName":"Hedges","suffix":""},{"id":271634710,"identity":"895975a9-eeaa-48b7-b9cf-cfcfd01a6e43","order_by":10,"name":"Tylman Michael","email":"","orcid":"","institution":"St. Jude Children's Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"Tylman","middleName":"","lastName":"Michael","suffix":""},{"id":271634711,"identity":"b53daae8-d9ea-4d84-96f3-9e1718f984ff","order_by":11,"name":"Giles Robinson","email":"","orcid":"https://orcid.org/0000-0001-7441-9486","institution":"St. Jude Children's Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"Giles","middleName":"","lastName":"Robinson","suffix":""},{"id":271634712,"identity":"27285c84-31b8-4efa-8410-359e7afae6d3","order_by":12,"name":"Paul Northcott","email":"","orcid":"","institution":"St. Jude Children's Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"Paul","middleName":"","lastName":"Northcott","suffix":""},{"id":271634713,"identity":"9d4e34b4-2dc9-460b-9ce8-f1ecce9c4633","order_by":13,"name":"Amar Gajjar","email":"","orcid":"","institution":"St Jude Children's Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"Amar","middleName":"","lastName":"Gajjar","suffix":""},{"id":271634714,"identity":"e90093d0-9063-41a7-9b42-e3afa875f543","order_by":14,"name":"Elizabeth Azzato","email":"","orcid":"","institution":"Cleveland Clinic","correspondingAuthor":false,"prefix":"","firstName":"Elizabeth","middleName":"","lastName":"Azzato","suffix":""},{"id":271634715,"identity":"1fd1b299-44a9-445c-a412-bf46697cd710","order_by":15,"name":"Sheila Shurtleff","email":"","orcid":"","institution":"Cleveland Clinic","correspondingAuthor":false,"prefix":"","firstName":"Sheila","middleName":"","lastName":"Shurtleff","suffix":""},{"id":271634716,"identity":"3d47a37a-b6dc-4e38-b8ba-be37a3db651d","order_by":16,"name":"David Ellison","email":"","orcid":"","institution":"St. Jude Children’s Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"David","middleName":"","lastName":"Ellison","suffix":""},{"id":271634717,"identity":"8ca3a828-05b5-4401-8d94-2559a54dfec1","order_by":17,"name":"Stanley Pounds","email":"","orcid":"","institution":"St. Jude Children’s Research Hospital","correspondingAuthor":false,"prefix":"","firstName":"Stanley","middleName":"","lastName":"Pounds","suffix":""}],"badges":[],"createdAt":"2024-01-25 16:17:22","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3897766/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3897766/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41698-024-00718-3","type":"published","date":"2024-10-02T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":50927869,"identity":"69c03f49-6a50-40bc-967d-96ac92a850e9","added_by":"auto","created_at":"2024-02-09 17:17:30","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":82944,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePrecision and recall above a classification probabilistic threshold for methylation class and family of each classifier\u003c/strong\u003e. \u003cstrong\u003e(A)\u003c/strong\u003e Precision and recall when predicting samples in GSE109379. \u003cstrong\u003e(B)\u003c/strong\u003e Precision and recall when predicting SJCRH samples. Validation results for subclass calls are in red. Validation results for family calls are in blue. Each point shows the precision and proportion of calls at each classification probabilistic threshold ranging from 0 to 0.9 with 0.1 increments.\u003c/p\u003e","description":"","filename":"OnlineFig2PRGSE109379SJCRH.png","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/0accacd625596d49086a71d8.png"},{"id":50927860,"identity":"701d5ceb-ce91-4146-b4da-6c441644c04c","added_by":"auto","created_at":"2024-02-09 17:17:28","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":146816,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eRFmod, kNNmod, and NNmod classification scores when predicting independent testing samples having all the probes versus samples having 10% of probes randomly dropped\u003c/strong\u003e. Line plots showing prediction scores for \u003cstrong\u003e(A)\u003c/strong\u003emethylation family and \u003cstrong\u003e(B)\u003c/strong\u003e methylation class of GSE109379 data set. Line plots showing \u003cstrong\u003e(C)\u003c/strong\u003e methylation family prediction scores and \u003cstrong\u003e(D)\u003c/strong\u003emethylation class prediction scores of SJCRH data set. Linear regression lines and the R-squared goodness-of-fit measures were estimated using the scores produced from kNNmod (red), NNmod (green), and RFmod (blue).\u003c/p\u003e","description":"","filename":"OnlineFig3regressionrobustness.png","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/fff666d590fa098b3a69c021.png"},{"id":50927899,"identity":"56691e3e-8cd8-4f94-98ac-55e74d13c98a","added_by":"auto","created_at":"2024-02-09 17:17:32","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":56259,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eAverage precision and recall of each classifier at different purity fractions per 0 and 0.9 threshold\u003c/strong\u003e. Tumor samples from GSE109379 were mixed with control samples (as indicated in Table S2) to create different fractions of normal vs tumor mixture. The average precision and recall for predicting methylation classes and families by RFmod (green), kNNmod (purple), and NNmod (orange) were computed for different mixed fractions of GSE109379 (0 to 0.95 purity – points on the lines) at 0 and 0.9 threshold.\u003c/p\u003e","description":"","filename":"OnlineFig4PRcurvepurityMethvsThres.png","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/396e3d9f4e8d6679da10687d.png"},{"id":50927865,"identity":"999a4450-6272-4261-a955-2cc2280d4b89","added_by":"auto","created_at":"2024-02-09 17:17:29","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":116718,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eClassification results of RF, kNN, and NN model for high-grade diffused midline glioblastoma with K-27 mutant (DMG, K27) methylation class at different contamination levels.\u003c/strong\u003e \u003cstrong\u003e(A, D, G)\u003c/strong\u003e Density plots of all calls (blue curve) and calls over the 0.9 clinical threshold (orange curve) at each possible methylation family predicted by RF, kNN, and NN when the ground truth is DMG, K27 at different fractions of control tissue contamination. \u003cstrong\u003e(B, E, H)\u003c/strong\u003e Box plots show the score distribution for each methylation family predicted by RF, kNN, and NN models. \u003cstrong\u003e(C, F, I)\u003c/strong\u003e Prediction accuracy of each classifier at each purity level.\u003c/p\u003e","description":"","filename":"OnlineFig5PurityDMGK27class.png","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/26de2b608e4dd2d462c8e7db.png"},{"id":50927866,"identity":"9a1e8dcd-a879-47c0-8627-e77d0ce31a22","added_by":"auto","created_at":"2024-02-09 17:17:29","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":114898,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eClassification results of RF, kNN, and NN model for grade IV glioblastoma, IDH wildtype, H3.3 G34 mutant (GBM, G34)methylation class at different contamination levels.\u003c/strong\u003e \u003cstrong\u003e(A, D, G)\u003c/strong\u003e Density plots of all calls (blue curve) and calls over the 0.9 clinical threshold (orange curve) at each possible methylation family predicted by RF, kNN, and NN when the ground truth is GBM, G34 at different fractions of control tissue contamination. \u003cstrong\u003e(B, E, H)\u003c/strong\u003e Box plots show the score distribution for each methylation family predicted by RF, kNN, and NN models. \u003cstrong\u003e(C, F, I)\u003c/strong\u003e Prediction accuracy of each classifier at each purity level.\u003c/p\u003e","description":"","filename":"OnlineFig6PurityGBMG34class.png","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/98a74e4f2044a3f979b5e18c.png"},{"id":66639079,"identity":"12fd1cf6-7d50-4c61-a120-24f740842b8c","added_by":"auto","created_at":"2024-10-15 05:57:51","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1834427,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/6f11a940-944f-4cd3-8660-a2961ffb67ec.pdf"},{"id":50927894,"identity":"251aa33a-948a-4f89-aff7-ed8bfa50a622","added_by":"auto","created_at":"2024-02-09 17:17:31","extension":"png","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":806288,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eFig. S1. Architecture of the multi-layer perceptron neural network\u003c/strong\u003e. We constructed an 11-layer perceptron neural net. The input dimension is 51,108, composed of probes selected with feature extraction described in the Methods section. The first layer is sparse, while the remaining ten layers are fully connected. The sparse layer maps 139,264 uniformly random sets of 256 features to the space [0,1]\u003csup\u003e512\u003c/sup\u003e. This layer is a forest of random decision stumps \u003csup\u003e40\u003c/sup\u003e that computes cosines of angles of vectors of length 256 drawn with uniform probability without replacement from the 51,108 input probes, which were selected by a LASSO model. The output of this layer is then fed through a standard perceptron.\u003c/p\u003e","description":"","filename":"FigS1MLPNetArchitecture.png","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/30f305b166aeca2d76d1e126.png"},{"id":50927853,"identity":"882fedf5-eb7e-4ab8-a32d-565fc95c6381","added_by":"auto","created_at":"2024-02-09 17:17:25","extension":"gif","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":310156,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eFig. S2. Training, testing, and validation scheme of all classifiers\u003c/strong\u003e. To reduce the overfitting problem when training classifiers on high-dimensional data, all classifiers were cross-validated based on 1000 leave-out-25% samplings. We randomly selected 75% of the data used to train the classifiers (GSE90496), while the remaining 25% of the data were used for predictions. Stratified random sampling was performed for each methylation class or family to ensure the number of categories remained the same in each iteration. This training and testing process was repeated 1000 times. The final models were validated with two independent data sets: GSE109379 and the St. Jude data set.\u003c/p\u003e","description":"","filename":"FigS2TrainTestValidationscheme.gif","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/ab291b89870048a3b94678f9.gif"},{"id":50927863,"identity":"cd1ab91a-c151-4edb-8f2c-670af954422c","added_by":"auto","created_at":"2024-02-09 17:17:29","extension":"png","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":294829,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eFig. S3.\u003c/strong\u003e \u003cstrong\u003eLeave-out-25% testing results of each methylation family\u003c/strong\u003e. \u003cstrong\u003e(A)\u003c/strong\u003e Heat map showing methylation family prediction results after 1000 stratified random samplings \u003cem\u003e\u003cstrong\u003e(i)\u003c/strong\u003e\u003c/em\u003eRF, \u003cem\u003e\u003cstrong\u003e(ii)\u003c/strong\u003e\u003c/em\u003e kNN, and \u003cem\u003e\u003cstrong\u003e(iii)\u003c/strong\u003e\u003c/em\u003e NN classifier incorporating information of \u003cem\u003en \u003c/em\u003e= 2,801 reference tumor samples allocated to 75 methylation class families (GSE90496). Deviations from the bisecting line represent misclassification errors (using the maximum calibrated score for class prediction). Boxplots showing \u003cstrong\u003e(B)\u003c/strong\u003e the accuracy, \u003cstrong\u003e(C)\u003c/strong\u003eprecision and recall, and \u003cstrong\u003e(D)\u003c/strong\u003e F1-score for each classifier with outliers.\u003c/p\u003e","description":"","filename":"FigS3HeatmapAccuracyPRF1family.png","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/3776d727e827362e933475ef.png"},{"id":50928626,"identity":"e85c531a-21c5-4cc4-86d7-13f27c1f57d2","added_by":"auto","created_at":"2024-02-09 17:25:31","extension":"tif","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":14511140,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eFig. S4. Classification results of RF, kNN, and NN model for high-grade diffused midline glioblastoma with K-27 mutant (DMG, K27) methylation family at different contamination levels.\u003c/strong\u003e \u0026nbsp;\u003cstrong\u003e(A-C)\u003c/strong\u003e Density plots of all calls (blue curve) and calls over the 0.9 clinical threshold (orange curve) at each possible methylation family predicted by RF, kNN, and NN when the ground truth is DMG, K27 at different fractions of control tissue contamination. \u003cstrong\u003e(D-F)\u003c/strong\u003eBox plots show the score distribution for each methylation family predicted by RF, kNN, and NN models. \u003cstrong\u003e(G-I)\u003c/strong\u003e Accuracy of each classifier at each purity level.\u003c/p\u003e","description":"","filename":"FigS4PurityDMGK27family.tif","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/508e914173c5a377eff3059a.tif"},{"id":50927897,"identity":"9488d7f3-72a6-408c-89d1-8eabc97a944d","added_by":"auto","created_at":"2024-02-09 17:17:31","extension":"tif","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":15374760,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eFig. S5\u003c/strong\u003e. \u003cstrong\u003eClassification results of RF, kNN, and NN model for grade IV glioblastoma, IDH wildtype, H3.3 G34 mutant (GBM, G34)methylation family at different levels of contamination.\u003c/strong\u003e \u003cstrong\u003e(A-C)\u003c/strong\u003e Density plots of all calls (blue curve) and calls over the 0.9 clinical threshold (orange curve) at each possible methylation family predicted by RF, kNN, and NN when the ground truth is GBM, G34 at different fractions of control tissue contamination. \u003cstrong\u003e(D-F)\u003c/strong\u003e Box plots show the score distribution for each methylation family predicted by RF, kNN, and NN models. \u003cstrong\u003e(G-I)\u003c/strong\u003eAccuracy of each classifier at each purity level.\u003c/p\u003e","description":"","filename":"FigS5PurityGBMG34family.tif","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/a0ab6d6e57666d7caffec1d5.tif"},{"id":50927893,"identity":"a5821296-42dc-4de1-956e-1ef8e672d4cb","added_by":"auto","created_at":"2024-02-09 17:17:31","extension":"xls","order_by":6,"title":"","display":"","copyAsset":false,"role":"supplement","size":181248,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable S1\u003c/strong\u003e. Cross-validation performance metrics per methylation class for kNN, RF, and NN\u003c/p\u003e","description":"","filename":"TableS1GSE90496caretmetricssubclass.xls","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/86e0a847a08f39a87e418e2f.xls"},{"id":50927900,"identity":"36012c36-3b8a-447f-98c3-044c17acfecf","added_by":"auto","created_at":"2024-02-09 17:17:32","extension":"xls","order_by":7,"title":"","display":"","copyAsset":false,"role":"supplement","size":168448,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable S2\u003c/strong\u003e. Cross-validation performance metrics per methylation family for kNN, RF, and NN\u003c/p\u003e","description":"","filename":"TableS2GSE90496caretmetricsfamily.xls","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/53b4c5092c3c99b2b99539bc.xls"},{"id":50927854,"identity":"b29920e3-d2d4-4a9e-9a99-c99516c4102c","added_by":"auto","created_at":"2024-02-09 17:17:26","extension":"xls","order_by":8,"title":"","display":"","copyAsset":false,"role":"supplement","size":215040,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable S3\u003c/strong\u003e. Semi-supervised labeling results with top calibration scores for GSE109379 (n=1104)\u003c/p\u003e","description":"","filename":"TableS3GSE109379SSL.xls","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/1fef3f14911d2855bd2f7754.xls"},{"id":50927851,"identity":"88772320-ca3c-4d64-87af-84d228e620fc","added_by":"auto","created_at":"2024-02-09 17:17:25","extension":"xls","order_by":9,"title":"","display":"","copyAsset":false,"role":"supplement","size":179712,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable S4\u003c/strong\u003e. Semi-supervised labeling results with top calibration scores for 950 SJCRH samples\u003c/p\u003e","description":"","filename":"TableS4SJ950casesSSL.xls","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/def9bb1a1f9eb04a67eb0ab1.xls"},{"id":50927892,"identity":"8e8e74ac-6285-4635-b360-26116e5e06a0","added_by":"auto","created_at":"2024-02-09 17:17:31","extension":"xls","order_by":10,"title":"","display":"","copyAsset":false,"role":"supplement","size":295424,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable S5\u003c/strong\u003e. Performance of kNN, RF, and NN when predicting methylation class in GSE109379 validation data set at 0.9 classification probabilistic threshold\u003c/p\u003e","description":"","filename":"TableS5ACCclassGSE109379.xls","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/2b9784890fe0f5da45d1d50e.xls"},{"id":50927867,"identity":"7cf7b35d-aab2-4eac-a013-d260793a853a","added_by":"auto","created_at":"2024-02-09 17:17:30","extension":"xls","order_by":11,"title":"","display":"","copyAsset":false,"role":"supplement","size":235008,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable S6\u003c/strong\u003e. Performance of kNN, RF, and NN when predicting methylation family in GSE109379 validation data set at 0.5 classification probabilistic threshold\u003c/p\u003e","description":"","filename":"TableS6ACCfamilyGSE109379.xls","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/3a140e5e60e9affb8970730b.xls"},{"id":50927902,"identity":"9d754fce-d7b3-4bd8-a5f7-ae7cc2f36f48","added_by":"auto","created_at":"2024-02-09 17:17:32","extension":"xls","order_by":12,"title":"","display":"","copyAsset":false,"role":"supplement","size":207360,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable S7\u003c/strong\u003e. Performance of kNN, RF, and NN when predicting methylation class in SJCRH validation cohort at 0.9 classification probabilistic threshold\u003c/p\u003e","description":"","filename":"TableS7ACCsubClasssjindep.xls","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/282d002ca25960895dec3581.xls"},{"id":50927862,"identity":"3608dd89-88f9-4243-a892-40fc47b75b23","added_by":"auto","created_at":"2024-02-09 17:17:28","extension":"xls","order_by":13,"title":"","display":"","copyAsset":false,"role":"supplement","size":159744,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable S8\u003c/strong\u003e. Performance of kNN, RF, and NN when predicting methylation family in SJCRH validation cohort at 0.5 classification probabilistic threshold\u003c/p\u003e","description":"","filename":"TableS8ACCfamilysjindep.xls","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/977fad570f65b03931fe743c.xls"},{"id":50927858,"identity":"6fa4225d-1c2a-4907-979b-6da19f59863b","added_by":"auto","created_at":"2024-02-09 17:17:28","extension":"xlsx","order_by":14,"title":"","display":"","copyAsset":false,"role":"supplement","size":52288,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable S9. \u003c/strong\u003eSt. Jude Children’s Research Hospital (SJCRH) validation cohort (n=950) characteristics\u003c/p\u003e","description":"","filename":"TableS9clinicaldataSJCRH.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/02af17eec7f367b4251b066a.xlsx"},{"id":50927895,"identity":"751f925f-43d9-47d0-8ec3-bf4d364f7731","added_by":"auto","created_at":"2024-02-09 17:17:31","extension":"docx","order_by":15,"title":"","display":"","copyAsset":false,"role":"supplement","size":14739,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable S10.\u003c/strong\u003e Control tissues used for the \u003cem\u003ein silico\u003c/em\u003e mixing experiment\u003c/p\u003e","description":"","filename":"TableS10.docx","url":"https://assets-eu.researchsquare.com/files/rs-3897766/v1/6167fc1a5273d962fd66b5e3.docx"}],"financialInterests":"(Not answered)","formattedTitle":"DNA-Methyaltion-Based Deep Learning for Precision Classification of Central Nervous System Tumors: A Comparative Study","fulltext":[{"header":"INTRODUCTION","content":"\u003cp\u003eBrain tumor classification has historically relied on morphologic examination of tumor specimens under a light microscopy \u003csup\u003e1\u003c/sup\u003e. Refinement of the process has occurred through recognition of additional tumor-specific histologic patterns and by integrating testing modalities such as cytogenetics, immunohistochemistry, and nucleic acid sequencing findings into the classification schemes. More recently, it has been recognized that normal and neoplastic tissues have inherent epigenetic signatures encoded in their methylome \u003csup\u003e2\u0026ndash;5\u003c/sup\u003e. The pattern of CpG methylation has been established as a stable and reliable biomarker for tumors and normal tissues \u003csup\u003e2,6\u003c/sup\u003e. The DNA methylation signature is considered a combined surrogate of the cell of origin and genomic driver abnormality. It is retained even after tumor recurrence or passage of tumors as an orthotopic xenograft \u003csup\u003e7\u0026ndash;9\u003c/sup\u003e. Histologically-defined tumor types often consist of heterogeneous molecular subtypes with distinct biological and clinical behavior \u003csup\u003e10,11\u003c/sup\u003e. Due to the lack of recurrent defining mutations, some of these molecular subtypes may only be recognizable by their distinct methylation or transcriptomic signatures \u003csup\u003e5,12\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eThe adoption of high-density methylation arrays (Illumina Infinium 450K and 850K EPIC arrays) has allowed for genome-wide evaluation of DNA methylation from large cohorts of human tumors. These arrays have favorable characteristics, including relatively low cost, comparable performance on both fresh or formalin-fixed and paraffin-embedded (FFPE) tissues, and stability of the methylation mark even in material stored for multiple decades \u003csup\u003e13\u003c/sup\u003e. In addition, the ability to assay FFPE has facilitated the accumulation of large tumor cohorts and allows for easy integration into standard clinical workflows \u003csup\u003e5,6,14\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eThe initial utility of methylation profiling to refine the classification of brain tumors relied on an unsupervised analysis comparing specific tumor cohorts of interest to reference brain tumor types \u003csup\u003e5,12,14,15\u003c/sup\u003e. However, the introduction of supervised classification models based on methylation data has significantly improved clinical diagnostics. In addition, these models are amenable to the types of performance metrics typically utilized for clinical tests \u003csup\u003e6,16\u003c/sup\u003e. The current state-of-the-art model for methylation-based classification relies on a random forest (RF) classifier trained on a reference cohort containing all tumor entities represented in the 2016 \u003cem\u003eWHO Classification of Tumours of the Central Nervous System\u003c/em\u003e \u003csup\u003e6\u003c/sup\u003e. While showing relatively good performance, certain model features remain suboptimal, yielding either misclassification or a substantial number of subthreshold scores for selected tumor types \u003csup\u003e6,17\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eSeveral models utilizing simple and advanced machine learning techniques have been proposed to improve classification accuracy. For example, the well-received k-nearest neighbor (kNN) algorithm was adapted and showed high capability and notable sensitivity in the acceptable discrimination of classes \u003csup\u003e18\u0026ndash;21\u003c/sup\u003e. The kNN approach was also employed in a hybrid classification model combining genetic algorithms and artificial neural networks \u003csup\u003e22\u003c/sup\u003e. This hybrid model showed more effectiveness in classifying performance than expectation and maximization classifiers. Recently, artificial neural networks (NN) have been considered by researchers as one of the most useful and applicable constructs in artificial intelligence \u003csup\u003e23\u0026ndash;25\u003c/sup\u003e. Specifically, deep NNs have been widely adopted in the diagnostic process of various biomedical fields and provided many opportunities to improve health care and heighten precision in the oncologic pathology \u003csup\u003e23,26\u0026ndash;28\u003c/sup\u003e. Despite their effectiveness and popularity, neither k-NN nor deep learning NN has been explored in the epigenetic field to classify the complex molecular entities among central nervous system (CNS) tumors. Currently, no commercial models are available for CNS tumor classification based on DNA methylation data.\u003c/p\u003e \u003cp\u003eAs part of our clinical implementation of DNA methylation profiling at St. Jude Children\u0026rsquo;s Research Hospital (SJCRH), we developed a multi-layer perceptron NN model (NNmod) based on DNA methylation profiles of 2,801 samples from a reference tumor cohort to predict the 82 histologically and/or molecularly distinct CNS tumor classes and 9 normal controls. We also compared the performance characteristics of this NNmod to the current state-of-the-art RF classifier \u003csup\u003e6\u003c/sup\u003e (RFmod) and another alternative model using an exact bootstrap k-nearest neighbor (kNNmod) algorithm. We validated the performance of these models with two independent brain tumor cohorts consisting of 1104 samples from GSE109379 and 950 samples from the St. Jude Children\u0026rsquo;s Research Hospital. Our results showed that although all models performed robustly to missing data, the deep NN model had the highest CNS classification accuracy and the most favorable performance characteristics, especially in minimizing the proportion of subthreshold score during testing and validation. Average precision and recall of the NNmod started reduce to similar levels of kNNmod and RFmod when tumor purity was less than 50%. This suggests that a deep NN model can be implemented in clinical laboratories as a reliable and essential diagnostic tool to assist in precision therapy for brain tumors.\u003c/p\u003e"},{"header":"RESULTS","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e\u003cem\u003eModel performance on train and test set\u003c/em\u003e:\u003c/h2\u003e \u003cp\u003eWe developed three models, i.e., a k-nearest neighbors model (kNNmod), a random forest model (RFmod), and a multi-layer perceptron neural network model (NNmod) (Fig. \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e), to classify human CNS tumors based on methylation signatures of the comprehensive reference set (GSE90496, n\u0026thinsp;=\u0026thinsp;2801). This set comprises 91 methylation classes grouped into 75 methylation class families based on their histological and biological closeness (Capper et al., 2008). The RF model represented a recapitulation of the previous random forest produced by Capper et al., representing the best in the current model. Here, we compared the performance of kNNmod and NNmod to the RFmod. The three models were evaluated with 1000 leave-out-25% cross-validations in predicting methylation classes and families (Fig. \u003cspan refid=\"MOESM2\" class=\"InternalRef\"\u003eS2\u003c/span\u003e). All models produced accuracies above 0.95 for both class and family prediction (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). Among the three models, classification accuracy and its Kappa statistic were highest in NNmod (above 0.98) and lowest in kNNmod (0.90 and 0.95 for class and family prediction) (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). These accuracies were statistically significantly different from the null accuracy, i.e. the accuracy could be achieved by predicting the most frequent class (McNeMar's p-values\u0026thinsp;\u0026lt;\u0026thinsp;10\u003csup\u003e\u0026ndash;16\u003c/sup\u003e). These results suggest that all models produced useful predictions with high accuracy.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eOverall performance of leave-out-25% train-test process for each classifier on the GSE90496\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003eRF\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003ekNN\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c7\" namest=\"c6\"\u003e \u003cp\u003eNN\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eClass\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFamily\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eClass\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eFamily\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eClass\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eFamily\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eAccuracy\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.98\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.95\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.96\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eAccuracy Null\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.05\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.11\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.06\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.11\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.06\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.12\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eKappa\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.96\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.90\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.95\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.98\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eRecall\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.97\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.98\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.86\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.88\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.98\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePrecision\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.96\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.985\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.90\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.92\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.98\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.995\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eSpecificity\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eCross-validation misclassifications by RFmod and NNmod focused on a few methylation classes while miss-classifications by kNNmod spread into many classes. Cross-validation of RFmod, kNNmod, and NNmod resulted in an average accuracy of 98%, 95%, and 99% for class prediction, respectively (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). Notably, NNmod produced the best accuracy in predicting methylation class in all 1000 cross-validation rounds (Table \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e). kNNmod, compared to RFmod and NNmod, had the lowest precision (90% vs. 96% and 98%, respectively) and recall (86% vs. 97% and 98%). All models had comparable specificity (around 99%). The majority of miss-classifications among three models occurred within the six histologically and biologically closely related tumor classes (pituitary adenomas - PITAD and PITUI) and myxopapillary ependymomas (EPN-MPE) (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e1\u003c/span\u003eA). However, kNNmod misclassification expanded to other methylation classes such as ependymomas (EPN), supratentorial subependymomas (SUBEPN), low-grade gliomas (LGG), melanomas (MELAN), melanocytomas (MELCYT), and plexus tumors (PLEX) (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e1\u003c/span\u003eAii). On the other hand, NNmod had the narrowest ranges in accuracy, precision, and recall of predicting 91 subclasses with a median value around 0.98 for each metric (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e1\u003c/span\u003eB-C and Table \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e). Minimal F1 scores for RFmod, kNNmod, and NNmod were 0.729, 0.359, and 0.761, respectively (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e1\u003c/span\u003eD and Table \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e), suggesting NNmod had the best balance between precision and recall.\u003c/p\u003e \u003cp\u003eAll models, in general, performed better at predicting methylation families. The classification metrics of these 75 methylation families are shown in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e and Fig. \u003cspan refid=\"MOESM3\" class=\"InternalRef\"\u003eS3\u003c/span\u003e. The cross-validation accuracies for the clinically relevant groupings were improved in all models. Compared to kNNmod, NNmod showed higher accuracy (99% vs 96%), precision (99% vs 88%), and recall (99.5% vs 93%). Compared to RFmod, the NNmod showed higher recall (99% vs. 98%) and comparable accuracy (~\u0026thinsp;99%), precision (~\u0026thinsp;98%), and specificity (~\u0026thinsp;99.9%) (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). Among the 1000 cross-validation rounds in predicting methylation family, NNmod produced the best accuracy 604 times, while RFmod produced the best accuracy 280 times. The rest of the cross-validation rounds, NNmod and RFmod had the same accuracy that was higher than kNNmod (Table \u003cspan refid=\"MOESM2\" class=\"InternalRef\"\u003eS2\u003c/span\u003e). The misclassification of RFmod and NNmod among the CNS tumor classes shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e1\u003c/span\u003eA appeared to be dissolved but retained in kNNmod (Fig. \u003cspan refid=\"MOESM3\" class=\"InternalRef\"\u003eS3\u003c/span\u003eA). Although accuracy was improved for all models, the gap between precision (88%) and recall (93%) for kNNmod (Fig. \u003cspan refid=\"MOESM3\" class=\"InternalRef\"\u003eS3\u003c/span\u003eB-D and Table \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e) remained the same in predicting methylation families. Minimal F1 scores for RFmod and NNmod were increased to 0.878 and 0.883, while this score was reduced to 0.318 by kNNmod (Table \u003cspan refid=\"MOESM2\" class=\"InternalRef\"\u003eS2\u003c/span\u003e). In conclusion, these results indicated that although RFmod and NNmod had very comparable performance, NNmod still had the highest accuracy and the best balance between precision and recall among the three models, suggesting that it had the highest discriminating power for both methylation class and family.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e\u003cem\u003eModel performance on two independent validation sets\u003c/em\u003e:\u003c/h2\u003e \u003cp\u003eThe classification performance of the three models was additionally tested on two independent data sets (GSE 109379) and the SJCRH data sets. To objectively assign each independent test sample to the reference methylation class group, we performed a semi-supervised learning approach \u003csup\u003e29\u003c/sup\u003e to assign labels to the two validation data sets. The 1,104 samples were assigned to 65 methylation classes and 50 families (Table \u003cspan refid=\"MOESM3\" class=\"InternalRef\"\u003eS3\u003c/span\u003e), while the 950 SJCRH samples were grouped into 49 methylation classes (Table \u003cspan refid=\"MOESM4\" class=\"InternalRef\"\u003eS4\u003c/span\u003e). This result was then used as the ground truth to measure the accuracy of the prediction results from our classifiers. We evaluated the performance of each classifier at multiple probabilistic prediction cutoffs ranging from 0 to 0.9 with an 0.1 increment. Figure\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e2\u003c/span\u003e shows the overall average precision and recall at each cutoff for each classifier when validating on GSE109379 and SJCRH data sets. Although all models had their prediction precision increase as the threshold increased for both class (red line) and family (blue line) prediction, the recalls that met the cutoff dropped quickly to around 65% in RFmod and kNNmod, but it stayed above 0.75 in NNmod (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e2\u003c/span\u003e and Table \u003cspan refid=\"MOESM5\" class=\"InternalRef\"\u003eS5\u003c/span\u003e-\u003cspan refid=\"MOESM8\" class=\"InternalRef\"\u003eS8\u003c/span\u003e). For application to diagnostic tumor samples, an optimal calibrated score threshold of \u0026ge;\u0026thinsp;0.9 was selected \u003csup\u003e6\u003c/sup\u003e. For subclasses within methylation class families, a threshold value of \u0026ge;\u0026thinsp;0.5 was defined as sufficient for a valid prediction, as long as all family member scores add up to a total score of \u0026ge;\u0026thinsp;0.9. Single class specificity and sensitivity are provided in Table \u003cspan refid=\"MOESM5\" class=\"InternalRef\"\u003eS5\u003c/span\u003e-\u003cspan refid=\"MOESM8\" class=\"InternalRef\"\u003eS8\u003c/span\u003e. While RFmod and kNNmod had average balanced accuracies below 90% (82% and 86%, respectively), NNmod achieved 91% accuracy for methylation class prediction of GSE109379. At the 0.9 threshold for methylation class prediction, NNmod maintained good recalls (\u0026gt;\u0026thinsp;82% for class and \u0026gt;\u0026thinsp;84% for family) with \u0026gt;\u0026thinsp;90% precision, while RFmod and kNNmod produced recalls of less than 60% (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e2\u003c/span\u003eA and Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). When predicting SJCRH at the 0.9 threshold, NNmod and RFmod achieved comparable results with balanced accuracies of 94% and 93%, recalls of 89% and 87%, and precisions of 96% and 92%, respectively. Meanwhile, kNNmod got 84% accuracy with 70% recall and 80% precision, respectively (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e2\u003c/span\u003eB and Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). These results suggested that NNmod could identify the most positive calls with higher accuracy and precision.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ePerformance of each classifier when predicting methylation class in the independent test sets at 0.9 threshold\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"10\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c10\" colnum=\"10\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eData set\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eRF\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003ekNN\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c10\" namest=\"c8\"\u003e \u003cp\u003eNN\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePrecision\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eRecall\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eAccuracy\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePrecision\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eRecall\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eAccuracy\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003ePrecision\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c9\"\u003e \u003cp\u003eRecall\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c10\"\u003e \u003cp\u003eAccuracy\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eGSE109379\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.87\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.73\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.86\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.76\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.64\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.82\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.91\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.82\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.91\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eSJCRH\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.92\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.87\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.93\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.84\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.96\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.89\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.94\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eModel robustness\u003c/h2\u003e \u003cp\u003eCNS tumor classification of our classifiers is based on features that measure DNA methylation at different CpG sites in the human genome using probes on Illumina BeadChip arrays. This microarray technology is easy to use, time-efficient and cost-effective. However, it keeps evolving, and in each new release, more probes are printed to cover more diverse genomic regions, and some probes are purposely removed for efficiency. Other potential applications, such as detection of tumors by cell free DNA testing, may also have uneven or missing values. Because the missing probes could differentially affect model performance, we investigated whether the performance of the three classifiers was robust in producing consistent outputs in class labels and their corresponding prediction scores even when a proportion of input probes were not present. We performed an experiment in which we randomly dropped 10% of the probes in the independent test data sets GSE109379 and SJCRH. We repeated this process 10 times to create 10 different missing probes scenarios. The robustness of each classifier was accessed based on the Theil\u0026rsquo;s U uncertainty coefficients between the two sets of predicted labels and Spearman\u0026rsquo;s correlation coefficients between prediction scores with and without missing probes. Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e shows that RFmod and NNmod have Theil\u0026rsquo;s U uncertainty coefficients greater than 0.94, suggesting that the predicted labels by RFmod and NNmod before probe drop-out were as similar as those produced after probe drop-out. In contrast, kNNmod has the lowest uncertainty coefficient among the three classifiers with Theil\u0026rsquo;s U ranging from 0.889 to 0.908 for methylation family and class prediction. These results indicate that the two sets of predicted labels are strongly associated. All models have Pearson correlation coefficients\u0026thinsp;\u0026gt;\u0026thinsp;0.928 with p-values\u0026thinsp;\u0026lt;\u0026thinsp;2.2e\u003csup\u003e-16\u003c/sup\u003e, suggesting a strong and statistically significant linear correlation between prediction scores produced when 10% of probes were missing and when there were no missing probes (Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e). Figure\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e3\u003c/span\u003e shows the regression analysis of the two sets of classification scores. Scores produced by NNmod and kNNmod with drop-out data set were generally higher than those that were output using all probes as indicated with positive y-intercepts (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e3\u003c/span\u003e). RFmod, when using data with missing probes, produced lower classification scores (negative y-intercepts). All models had the goodness-of-fit R-squared of at least 86%, indicating a strong correlation between the two sets of prediction scores (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e3\u003c/span\u003e). These results suggest that missing probes do not affect the prediction outcomes of any classifiers.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eTheil\u0026rsquo;s U uncertainty coefficient with a 95% confidence interval of each classifier with and without dropping 10% of probes in the GSE109379 and SJCRH independent test sets\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eData set\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003eRFmod\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003ekNNmod\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c7\" namest=\"c6\"\u003e \u003cp\u003eNNmod\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eClass\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFamily\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eClass\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eFamily\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eClass\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eFamily\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eGSE109379\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.969\u003c/p\u003e \u003cp\u003e(0.967, 0.972)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.974\u003c/p\u003e \u003cp\u003e(0.970, 0.978)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.892\u003c/p\u003e \u003cp\u003e(0.887, 0.898)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.908\u003c/p\u003e \u003cp\u003e(0.902, 0.914)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.973\u003c/p\u003e \u003cp\u003e(0.970, 0.9796)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0.980\u003c/p\u003e \u003cp\u003e(0.976, 0.983)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eSJCRH\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.945\u003c/p\u003e \u003cp\u003e(0.940, 0.949)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.964\u003c/p\u003e \u003cp\u003e(0.960, 0.968)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.889\u003c/p\u003e \u003cp\u003e(0.883, 0.894)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.899\u003c/p\u003e \u003cp\u003e(0.894, 0.905)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.970\u003c/p\u003e \u003cp\u003e(0.966, 0.974)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0.979\u003c/p\u003e \u003cp\u003e(0.975, 0.982)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ePearson's correlation coefficients of prediction scores with and without 10% missing probes\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eData set\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003eRFmod\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003ekNNmod\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c7\" namest=\"c6\"\u003e \u003cp\u003eNNmod\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eClass\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFamily\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eClass\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eFamily\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eClass\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eFamily\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eGSE109379\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.98\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.93\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.94\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.94\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.94\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eSJCRH\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.95\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.96\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eModel assessment based on sample purity\u003c/h2\u003e \u003cp\u003eInfiltrating of normal cells such as epithelial, stromal, and immune cells in tumor tissue can perturb the tumor signal in molecular studies. In our application, this contamination can affect the methylation level measured by microarray chips, leading to possible degradation in the performance of a classifier. Therefore, we developed an \u003cem\u003ein silico\u003c/em\u003e experiment in which different fractions of the normal control cells were mixed with the tumor tissue to answer these questions: (1) whether a classifier produces unexpected methylation class/family prediction (2) if yes, would the prediction have a suprathreshold score, and (3) approximately at what percentage of control contamination, the classification accuracy starts to degrade. We first observed the overall performance of the three classifiers based on their average recall and precision for methylation class and family prediction at different thresholds and purity fractions (Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e4\u003c/span\u003e). NNmod started to perform the best, as seen in previous sections. RFmod degraded at a comparable rate with kNNmod after the sample purity was less than 65%. As the purity of tumor samples was less than 40%, NNmod started to yield lower precision and recall compared to the other two classifiers (Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e4\u003c/span\u003e, threshold\u0026thinsp;=\u0026thinsp;0). At the 0.9 clinical threshold and 0.95 purity, NNmod had twice the average recalls and a much higher average precision than RFmod and kNNmod. The performance of NNmod did not start to degrade at a similar rate to RFmod and kNNmod until the purity of tumor samples was less than 50%. As the contamination increased, RFmod had the lowest performance among the three classifiers (Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e4\u003c/span\u003e, threshold\u0026thinsp;=\u0026thinsp;0.9). NNmod maintained the highest average precision and recall among the three classifiers. Its performance reduced to a comparable level with RFmod and kNNmod only when the tumor purity was less than 50%.\u003c/p\u003e \u003cp\u003eNext, we observed the prediction results of each classifier for each methylation class and family. Figures\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e5\u003c/span\u003e and \u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e6\u003c/span\u003e show the performance of RFmod, kNNmod, and NNmod at different control fractions in the tumor sample for methylation class diffused midline glioma H3 K-27 mutant (DMG, K27) and glioblastoma, IDH wildtype, H3.3 G34 mutant (GBM, G34), respectively. When the high-grade DMG, K27 tumors got contaminated with control, RFmod and NNmod did not produce unexpected methylation classes besides DMG, K27 and its corresponding mixed control cerebella hemisphere (CONTR, CEBM) class (Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e5\u003c/span\u003eA, B, G, and H) or family (Fig. \u003cspan refid=\"MOESM4\" class=\"InternalRef\"\u003eS4\u003c/span\u003eA, C, D, and F). On the other hand, kNNmod unexpectedly predicted these high-grade gliomas to be low-grade pilocytic astrocytoma (LGG, PA PF) with scores above the clinical threshold (0.9) (Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e5\u003c/span\u003eD, E and Fig. \u003cspan refid=\"MOESM4\" class=\"InternalRef\"\u003eS4\u003c/span\u003eB, E). kNNmod and RFmod could not accurately predict the methylation class of the DMG, K27 tumors if the purity of these samples was less than 70% (Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e5\u003c/span\u003eC, F and Fig. \u003cspan refid=\"MOESM4\" class=\"InternalRef\"\u003eS4\u003c/span\u003eG, H). Meanwhile, NNmod was able to maintain its prediction accuracy prediction for DMG, K27 samples unless the sample\u0026rsquo;s purity dropped below 40% (Fig.\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e6\u003c/span\u003eI). Figure\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e6\u003c/span\u003e shows that when predicting the GBM, G34 methylation class, RFmod did not provide any suprathreshold results if more than 30% of control tissues were present in the mixture (Fig.\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e6\u003c/span\u003eA, B, and C). On the contrary, kNNmod accurately predicted these samples until the contamination was up to 60%. At this fraction, kNNmod unexpectedly classified these grade IV glioblastomas as grade I dysembryoplastic neuroepithelial gliomas (LGG, DNT) (Fig.\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e6\u003c/span\u003eD, E and F). NNmod did not provide any suprathreshold classification to GBM, G34 samples besides their corresponding normal hemispheric cortex (CONTR, HEMI) starting at 70% contamination (Fig.\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e6\u003c/span\u003eG, H and I). Similar results were shown in Fig. \u003cspan refid=\"MOESM5\" class=\"InternalRef\"\u003eS5\u003c/span\u003e for GBM, G34 samples at the methylation family.\u003c/p\u003e \u003c/div\u003e"},{"header":"DISCUSSION","content":" \u003cp\u003eWe developed a deep neural network model to predict CNS tumor classification based on a large DNA-methylation data set from 2801 patients of 82 distinct CNS tumors and 9 controls. Our multilayer perceptron neural network classifier achieved high performance, as demonstrated in 3 different evaluation settings. Compared with RFmod, a current-state-of-the-art CNS tumor classifier based on DNA-methylation \u003csup\u003e6\u003c/sup\u003e, our NNmod showed higher overall accuracy (99% vs. 98%), precision (98% vs. 97%) and recall (98% vs. 96%) and comparable specificity (~\u0026thinsp;99%) in methylation class prediction (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). Among the three developed models, the kNN model produced the lowest accuracy (95%), precision (86%), and sensitivity (90%) (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). In addition, we showed that our DNN model is highly robust and generalizable as evaluated in an independent testing dataset of 1104 GSE109379 samples (65 tumor classes) and 700 classifiable SJCRH samples (45 tumor classes), with an overall accuracy of 91% and 94%. Among these results, NNmod showed the highest accuracy and the best balance between precision and recall compared to RFmod and kNNmod (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eAll classifiers were trained on the reference data (GSE90496) generated from the Illumina Human Methylation 450K chips. These chips featured 485,577 CpG sites throughout the human genome, but they became obsolete and have been replaced by the Ilumina HumanMehtylationEPIC BeadChip (EPIC). EPIC measures methylation at \u0026gt;\u0026thinsp;850,000 CpG sites and covers approximately 90% of the same sites represented on the 450K chip. EPIC eliminates sites reported to be poorly performed \u003csup\u003e30\u003c/sup\u003e and features more CpGs that cover more regulatory elements. When using classifiers trained on data produced by 450K chips to predict samples ran on EPIC chips, it is possible that some probes used for prediction are no longer present on EPIC chips and could hinder the classifier performance. As such, we performed a random probes drop-out experiment to evaluate the classification performance of RFmod, kNNmod, and NNmod (Tables\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e and \u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e). Although having the probes dropped out randomly may be adequate, it would be additionally useful to know in the future whether dropping all the poorly performed probes in the 450K training data set would enhance the performance and increase the robustness of all classifiers.\u003c/p\u003e \u003cp\u003eDiagnosis of CNS tumors is a complex multiclass classification problem as the number of diagnostic classes in which patients are stratified is not limited to a few selected classes but rather to a very high list of entities represented in the 5th edition of the \u003cem\u003eWHO Classification of CNS Tumors\u003c/em\u003e \u003csup\u003e6\u003c/sup\u003e. It has been shown that diagnostic accuracy can be improved by utilizing a robust machine-learning classification algorithm based on DNA-methylation profiles obtained from formalin-fixed, paraffin embedded (FFPE) or frozen tissue samples \u003csup\u003e6\u003c/sup\u003e.The preparation of FFPE samples is one of the most widely used procedures to preserve and archive specimens in clinical oncology. This workflow requires an invasive tissue biopsy to be performed on patients. Recently, the use of liquid biopsies, a less invasive method for cancer detection has rapidly gained prominence \u003csup\u003e31\u003c/sup\u003e. Particularly, plasma cell-free DNA methylation profiles have been shown to be highly sensitive, cost effective, and accurate in early tumor detection for cancer interception, and for multi-cancer classification \u003csup\u003e32,33\u003c/sup\u003e. Our study demonstrated that NNmod was the top stand-alone classifier among the three developed classifiers using DNA-methylation signatures from FFPE samples. The 11-layer perceptron NNmod maintained high recalls (\u0026gt;\u0026thinsp;82% for GSE109379 and \u0026gt;\u0026thinsp;90% for SJCRH) above an 0.9 clinical threshold with \u0026gt;\u0026thinsp;0.92 precision when validated with two independent data sets (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e2\u003c/span\u003e and Table \u003cspan refid=\"MOESM5\" class=\"InternalRef\"\u003eS5\u003c/span\u003e-\u003cspan refid=\"MOESM8\" class=\"InternalRef\"\u003eS8\u003c/span\u003e). With these improvements over RFmod, NNmod represents a viable method that could be used in conjunction with clinical, histopathologic, and molecular data to aid in the diagnosis and classification of CNS tumors. Future study would be to apply this machine learning modeling with the DNA-methylation profiles from plasma cell-free DNA obtained through the less invasive liquid biopsy procedure.\u003c/p\u003e"},{"header":"MATERIALS and METHODS","content":"\u003cp\u003e \u003cb\u003ePatient material.\u003c/b\u003e FFPE or frozen tumor samples representing pediatric patient samples encountered on the typical pathology service were evaluated. The samples represented 650 samples expected to be present in the reference series and 300, true negative samples representative of non-brain solid tumors known to be absent from the reference series (Table \u003cspan refid=\"MOESM9\" class=\"InternalRef\"\u003eS9\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eTraining and independent testing data sets\u003c/strong\u003e \u003cp\u003eAll supervised models were trained on the genome-wide DNA methylation profiles from the CNS tumor reference cohort (GSE90496), consisting of 2,801 samples from 91 methylation classes \u003csup\u003e6\u003c/sup\u003e. All classifiers were independently validated with two methylation data sets, including the 950 CNS tumor samples from the St. Jude Children's Research Hospital (SJCRH) and 1,104 CNS tumor samples from GSE109379 \u003csup\u003e6\u003c/sup\u003e.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003eData generation and methylation array processing.\u003c/b\u003e We analyzed the 950 independent test samples using Illumina Methylation BeadChip (EPIC) arrays according to the manufacturer's instructions. In summary, DNA was isolated from formalin-fixed paraffin-embedded (FFPE) tumor tissue using the Maxwell\u0026reg; Clinical Sample Concentrator system (Promega, Madison, WI). Following extraction, DNA was quantified using a Qubit fluorometer and quantitation reagents (Thermo Fisher Scientific, Waltham, MA), and bisulfite converted using the Zymo EZ DNA methylation kit (Zymo Research, Irvine, CA). The overall DNA input amount was approximately 250 ng. DNA methylation profiling was carried out with the Infinium HumanMethylationEPIC BeadChip (850K) array (Illumina Inc., San Diego, CA) on the Illumina iScan platform.\u003c/p\u003e \u003cp\u003eAll methylation data analyses, including those from GSE90496 and GSE109379, were performed in R (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.r-project.org\u003c/span\u003e\u003cspan address=\"http://www.r-project.org\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e, version 3.5.3), using several packages from Bioconductor and other repositories. Specifically, array data were preprocessed using the \u003cem\u003eminfi\u003c/em\u003e package (v.1.28.4) \u003csup\u003e34\u003c/sup\u003e. Background correction with dye-bias normalization was performed for all samples using noob (normal-exponential out-of-band) with the \"single\" dye method \u003csup\u003e35\u003c/sup\u003e. Batch effects such as hybridization time and other technical variables were removed using removeBatchEffect from the \u003cem\u003elimma\u003c/em\u003e package (v.3.38.3) \u003csup\u003e36\u003c/sup\u003e. Probe filtering was performed after normalization. Specifically, we removed probes located on sex chromosomes, probes containing nucleotide polymorphism (dbSNP132 Common) within five base pairs, including the targeted CpG-site, or mapping to multiple sites on hg19 (allowing for one mismatch), as well as cross-reactive probes.\u003c/p\u003e \u003cp\u003e \u003cb\u003eSemi-supervised analysis.\u003c/b\u003e We developed a combination approach including a self-training with editing using a support vector machine (SETRED-SVM) as the base learner model with an L2-penalized, multinomial logistic regression model to obtain high confidence labels from a few reference instances \u003csup\u003e29\u003c/sup\u003e. We applied this approach on GSE109379 and the SJ samples to get labels for the independent validation purpose of the supervised models. The \u003cem\u003essc\u003c/em\u003e R package (v2.1-0) was used to build and train the SETRED-SMV semi-supervised model. First, the standard deviation for each probe across all 2,801 samples from GSE90496 was calculated. Input features for SSL models were the 5072 probes with a standard deviation greater than 0.3 across all 2801 samples. We used the best SETRED-SVM model to predict the methylation class for 1104 GSE109379 and 950 SJ samples. The SSL scores were calibrated with an L2-penalized, multinomial logistic regression. Scores above the 0.8 threshold were considered correctly classifiable \u003csup\u003e29\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003e \u003cb\u003eThe random forest algorithm and development.\u003c/b\u003e The random forest algorithm was reconstructed from Capper's algorithm \u003csup\u003e6\u003c/sup\u003e using the \u003cem\u003erandomForest\u003c/em\u003e R package (v.4.6\u0026ndash;14) \u003csup\u003e37,38\u003c/sup\u003e. This model was trained based on the 408,862 overlapping probes of the 450K and 850K array probes. First, the 10,000 features (or probes) with the highest importance scores were selected by splitting the 408,862 intersecting probes into 43 sets of approximately 9500 probes. Next, one hundred trees were fitted for each set using 639 randomly sampled candidate features at each split (mtry\u0026thinsp;=\u0026thinsp;square root of 408,862). The subclass labels, stratified subsampling methods, and the number of trees in the forest were followed as in \u003csup\u003e6\u003c/sup\u003e. This framework can produce a model that either predicts the methylation class or the methylation \"family\" scores \u003csup\u003e6\u003c/sup\u003e that represent clinically-equivalent families on which Capper et al. witnessed their best error rates. Next, a multinomial logistic regression was used to calibrate the prediction scores from all cross-validation splits as previously described \u003csup\u003e6\u003c/sup\u003e. The family scores were then generated as the sum of all methylation class scores from the trained random forest.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eThe k-nearest neighbor algorithm and development (kNNmod)\u003c/strong\u003e \u003cp\u003eAn exact bootstrap k-nearest neighbor model (kNNmod) was built as described in \u003csup\u003e39\u003c/sup\u003e. The model was trained on score vectors constructed based on the difference in median beta values of the top 100 hyper- and hypo-methylated probes. Each set of 100 top probes was selected based on the mean \u0026szlig; values in a methylation group and the absolute z-scores computed by taking the differences between mean beta values of two CNS methylation groups divided by the square root of the sum of the variance in each group. Hence, each methylation group had a list of 200 probes that were either most hypo- or hypermethylated based on the absolute z-scores. Each sample had a vector of scores, i.e. one score per methylation group. Each score was computed by taking the median \u0026szlig; values of the top 100 hypermethylated probes and subtracting that from the top 100 hypomethylated probes. Euclidean distance on these vectors was used to measure the distance between each pair of samples. The entire Euclidean distance matrix on the methylation group score vectors was computed for all pairwise samples.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eTo classify a new sample, kNNMod ordered all other samples by their distance from the new observation and derived the probability that those neighbors would be included among the k nearest neighbors in the binomial distribution. We used k\u0026thinsp;=\u0026thinsp;5 neighbors for classification because some subgroups were very rare. For each new sample, the exact bootstrap probability of assignment to each methylation group can be conditionally computed on the training data set and the resulting probe selection and group score definition.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eThe multilayer sparse perceptron architecture and development (NNmod)\u003c/strong\u003e \u003cp\u003eThe design of the multilayer sparse perceptron is shown in Figure \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e. This design is based on two primary assumptions, (i) the methylation data from central nervous system tumors and normal brain is embedded in low dimensional space, and (ii) random combinations of important probes can predict methylation class. The first assumption is typical of high dimensional data and is supported by examination of the singular value decomposition (SVD) of previously published reference data \u003csup\u003e6\u003c/sup\u003e (data not shown). In addition, the ability of combined methylation probes to predict methylation class is supported by previous implementations of random forest classifiers \u003csup\u003e6\u003c/sup\u003e.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eWe constructed an 11-layer perceptron neural net. The input dimension is 51,108, composed of probes selected with feature extraction described in the network training section immediately following this section. The first layer is sparse, while the remaining ten layers are fully connected. The sparse layer maps 139,264 uniformly random sets of 256 features to the space [0,1]\u003csup\u003e512\u003c/sup\u003e. In other words, the sparse layer computes cosines of angles of vectors of length 256 drawn with uniform probability without replacement from the 51,108 input probes, which were selected by a LASSO model. This sparse feature layer can be considered a forest of random decision stumps \u003csup\u003e40\u003c/sup\u003e, the output of which is fed through a standard perceptron.\u003c/p\u003e \u003cp\u003eStochastic gradient descent was performed with a batch size of 32 on logarithms of output scores from the network using a learning rate of 0.001. These gradient descent parameters were obtained via a random search of the parameter space. The log-likelihood loss was minimized over the three-fold cross-validation. Using the evaluation partitions from the cross-validation splits, model calibration was performed with a multinomial logistic regressor. The final model was trained on the complete 2,801 samples using identical parameters following cross-validation.\u003c/p\u003e \u003cp\u003e \u003cb\u003eClassifier cross-validation.\u003c/b\u003e To reduce the overfitting problem when training classifiers on high-dimensional data, all classifiers were cross-validated based on 1000 leave-out-25% cross-validations. We randomly selected 75% of the data used to train the classifiers (GSE90496), while the remaining 25% of the data were used for predictions. Stratified random sampling was performed for each methylation class or family to ensure the number of categories remained the same in each iteration. This validation process was repeated 1000 times (Fig. \u003cspan refid=\"MOESM2\" class=\"InternalRef\"\u003eS2\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cb\u003eModel calibration\u003c/b\u003e. Calibration of machine learning methods may be necessary because the scores output by the classifier may have different scales when broken down by class, even when the scores are normalized so that they sum to 1. This poses problems for comparing the uncertainty in class or family calls between cases or even in the same case. Thus, the scores must be rescaled to form a well-calibrated multinomial distribution with minimal differences between expected values and variances between the class call groups.\u003c/p\u003e \u003cp\u003eBoth RF and NN models were calibrated with the same multinomial logistic regression approach described by \u003csup\u003e6\u003c/sup\u003e. The \u003cem\u003eglmnet\u003c/em\u003e package (v-4.1-3) \u003csup\u003e41\u003c/sup\u003e was used with R bindings for the random forest and python bindings for the neural net.\u003c/p\u003e \u003cp\u003e \u003cb\u003eModel robustness\u003c/b\u003e. To test whether missing methylation probes (features) affect our machine learning models, we randomly dropped 10% of the probes from the testing data (GSE109379 and SJCRH) and calculated the accuracy. The same probes at each round were used for all models. This process was repeated ten times to create 10 different missing sets of probes. Pearson\u0026rsquo;s correlation and Theil\u0026rsquo;s U uncertainty coefficients were computed using the \u003cem\u003eggpubr\u003c/em\u003e R package (v.0.4.0) and the \u003cem\u003eDescTools\u003c/em\u003e R package (v.99.44), respectively. Pearson\u0026rsquo;s correlation coefficients with p-values were calculated to examine the linear relationship between the two sets of prediction scores (with and without missing probes). Theil\u0026rsquo;s U uncertainty coefficients were calculated to measure the nominal association between the two sets of labels predicted by the three classifiers on samples of GSE109379 and SJCRH data with and without missing probes.\u003c/p\u003e \u003cp\u003e \u003cb\u003ePurity analysis\u003c/b\u003e. We performed an \u003cem\u003ein silico\u003c/em\u003e simulated impurity experiment using different fractions of control and positive samples in GSE109379 and SJCRH test sets. The experiment was performed based on \u003cem\u003em\u003c/em\u003e-values. The \u003cem\u003ein silico\u003c/em\u003e mixed \u003cem\u003em\u003c/em\u003e-values (m\u003csub\u003emixed\u003c/sub\u003e) for each positive sample were computed as follows\u003cdiv id=\"Equa\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e\n$${m}_{\\text{m}\\text{i}\\text{x}\\text{e}\\text{d}}=(1-p){m}_{\\text{t}\\text{e}\\text{s}\\text{t}}+p{m}_{\\text{c}\\text{o}\\text{n}\\text{t}\\text{r}\\text{o}\\text{l}}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({m}_{\\text{t}\\text{e}\\text{s}\\text{t}}\\)\u003c/span\u003e\u003c/span\u003e is the input m-value from the positive samples, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({m}_{\\text{c}\\text{o}\\text{n}\\text{t}\\text{r}\\text{o}\\text{l}}\\)\u003c/span\u003e\u003c/span\u003e is the average m-values of up to 19 appropriate control (i.e. normal) tissue samples in the test sets, p is the proportion of normal control tissues contaminated in a tumor sample (ranging from 0 to 1 with 0.05 increment). The control samples were selected based on their control methylation class corresponding to the methylation class tumor as described in Table \u003cspan refid=\"MOESM10\" class=\"InternalRef\"\u003eS10\u003c/span\u003e. The final measurement of the mixed sample was then converted back to beta values for classifier inputs.\u003c/p\u003e \u003cp\u003e \u003cb\u003eModel performance metrics\u003c/b\u003e. All models were evaluated based on accuracy, precision, specificity, recall, and F1 score. Classification accuracy is the number of correct predictions (true positives and true negatives) divided by the total number of predictions. Precision is the ratio of true positives to all the total positives predicted by a classifier. Specificity measures the proportion of true negatives correctly identified by a classification model. Recall or sensitivity is the ratio of true positives to all the ground truth positives. The F1-score is the harmonic mean of precision and recall and a good metric to measure the results in imbalanced classification problems. The higher the F1 score, the better the performance of a model. All measurements were computed using the \u003cem\u003ecaret\u003c/em\u003e R package (v.6.0\u0026ndash;90).\u003c/p\u003e "},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eCNS\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003ecentral nervous system\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003ekNN\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003ek\u0026ndash;nearest neighbors\u003c/p\u003e\u003c/div\u003e\u003cdiv class=\"Term\"\u003eRF\u0026ndash;random forest\u003c/p\u003e\u003c/div\u003e\u003cdiv class=\"Term\"\u003eNN\u0026ndash;multi\u0026ndash;layer perceptron neural net\u003c/p\u003e \u003c/div\u003e "},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eSOFTWARE AVAILABILITY:\u0026nbsp;\u003c/strong\u003eThe generated code is available from the corresponding authors upon reasonable request for non-commercial use.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAUTHOR CONTRIBUTIONS: \u0026nbsp;\u003c/strong\u003eAB developed the MLPNet framework and implemented the RF model. QTT modified and maintained the models, analyzed the results, produced figures and tables, and drafted the manuscript. BAO conceptualized the project, interpreted the results and drafted the manuscript. TL and SP implemented the KNN model. RT, SJA extracted the DNA and produced methylation data.\u0026nbsp;MC, LVF, GR, PN, AG, EA, SS, and DWE provided samples and interpreted the results. MC, DH, and TM modified and maintained the MLPNet. All authors reviewed and edited the manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCOMPETING INTERESTS:\u0026nbsp;\u003c/strong\u003eAll authors declare no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMATERIALS and CORRESPONDENCE:\u0026nbsp;\u003c/strong\u003eAll correspondence and material requests should be addressed to Dr. Brent Orr at [email protected] .\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003e\u003cspan\u003eFerguson, S. \u0026amp; Lesniak, M. S. Percival Bailey and the classification of brain tumors. \u003cem\u003eNeurosurg Focus\u003c/em\u003e \u003cstrong\u003e18\u003c/strong\u003e, e7 (2005). https://doi.org:10.3171/foc.2005.18.4.8\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eKumar, R., Liu, A. P. Y., Orr, B. A., Northcott, P. A. \u0026amp; Robinson, G. W. Advances in the classification of pediatric brain tumors through DNA methylation profiling: From research tool to frontline diagnostic. \u003cem\u003eCancer\u003c/em\u003e \u003cstrong\u003e124\u003c/strong\u003e, 4168\u0026ndash;4180 (2018). https://doi.org:10.1002/cncr.31583\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eMoran, S. \u003cem\u003eet al.\u003c/em\u003e Epigenetic profiling to classify cancer of unknown primary: a multicentre, retrospective analysis. \u003cem\u003eLancet Oncol\u003c/em\u003e \u003cstrong\u003e17\u003c/strong\u003e, 1386\u0026ndash;1395 (2016). https://doi.org:10.1016/S1470-2045(16)30297-2\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003ePajtler, K. W. \u003cem\u003eet al.\u003c/em\u003e Molecular Classification of Ependymal Tumors across All CNS Compartments, Histopathological Grades, and Age Groups. \u003cem\u003eCancer Cell\u003c/em\u003e \u003cstrong\u003e27\u003c/strong\u003e, 728\u0026ndash;743 (2015). https://doi.org:10.1016/j.ccell.2015.04.002\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eSturm, D. \u003cem\u003eet al.\u003c/em\u003e New Brain Tumor Entities Emerge from Molecular Classification of CNS-PNETs. \u003cem\u003eCell\u003c/em\u003e \u003cstrong\u003e164\u003c/strong\u003e, 1060\u0026ndash;1072 (2016). https://doi.org:10.1016/j.cell.2016.01.015\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eCapper, D. \u003cem\u003eet al.\u003c/em\u003e DNA methylation-based classification of central nervous system tumours. \u003cem\u003eNature\u003c/em\u003e \u003cstrong\u003e555\u003c/strong\u003e, 469\u0026ndash;474 (2018). https://doi.org:10.1038/nature26000\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eSmith, K. S. \u003cem\u003eet al.\u003c/em\u003e Patient-derived orthotopic xenografts of pediatric brain tumors: a St. Jude resource. \u003cem\u003eActa Neuropathol\u003c/em\u003e \u003cstrong\u003e140\u003c/strong\u003e, 209\u0026ndash;225 (2020). https://doi.org:10.1007/s00401-020-02171-5\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eHe, C. \u003cem\u003eet al.\u003c/em\u003e Patient-derived models recapitulate heterogeneity of molecular signatures and drug response in pediatric high-grade glioma. \u003cem\u003eNat Commun\u003c/em\u003e \u003cstrong\u003e12\u003c/strong\u003e, 4089 (2021). https://doi.org:10.1038/s41467-021-24168-8\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eKumar, R. \u003cem\u003eet al.\u003c/em\u003e Clinical Outcomes and Patient-Matched Molecular Composition of Relapsed Medulloblastoma. \u003cem\u003eJ Clin Oncol\u003c/em\u003e \u003cstrong\u003e39\u003c/strong\u003e, 807\u0026ndash;821 (2021). https://doi.org:10.1200/JCO.20.01359\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003ePugh, T. J. \u003cem\u003eet al.\u003c/em\u003e Medulloblastoma exome sequencing uncovers subtype-specific somatic mutations. \u003cem\u003eNature\u003c/em\u003e \u003cstrong\u003e488\u003c/strong\u003e, 106\u0026ndash;110 (2012). https://doi.org:10.1038/nature11329\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eRobinson, G. \u003cem\u003eet al.\u003c/em\u003e Novel mutations target distinct subgroups of medulloblastoma. \u003cem\u003eNature\u003c/em\u003e \u003cstrong\u003e488\u003c/strong\u003e, 43\u0026ndash;48 (2012). https://doi.org:10.1038/nature11213\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eNorthcott, P. A. \u003cem\u003eet al.\u003c/em\u003e Medulloblastoma comprises four distinct molecular variants. \u003cem\u003eJ Clin Oncol\u003c/em\u003e \u003cstrong\u003e29\u003c/strong\u003e, 1408\u0026ndash;1414 (2011). https://doi.org:10.1200/JCO.2009.27.4324\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eHovestadt, V. \u003cem\u003eet al.\u003c/em\u003e Robust molecular subgrouping and copy-number profiling of medulloblastoma from small amounts of archival tumour material using high-density DNA methylation arrays. \u003cem\u003eActa Neuropathol\u003c/em\u003e \u003cstrong\u003e125\u003c/strong\u003e, 913\u0026ndash;916 (2013). https://doi.org:10.1007/s00401-013-1126-5\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eNorthcott, P. A. \u003cem\u003eet al.\u003c/em\u003e The whole-genome landscape of medulloblastoma subtypes. \u003cem\u003eNature\u003c/em\u003e \u003cstrong\u003e547\u003c/strong\u003e, 311\u0026ndash;317 (2017). https://doi.org:10.1038/nature22973\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eBroniscer, A. \u003cem\u003eet al.\u003c/em\u003e Gliomatosis cerebri in children shares molecular characteristics with other pediatric gliomas. \u003cem\u003eActa Neuropathol\u003c/em\u003e \u003cstrong\u003e131\u003c/strong\u003e, 299\u0026ndash;307 (2016). https://doi.org:10.1007/s00401-015-1532-y\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eDanielsson, A. \u003cem\u003eet al.\u003c/em\u003e MethPed: a DNA methylation classifier tool for the identification of pediatric brain tumor subtypes. \u003cem\u003eClin Epigenetics\u003c/em\u003e \u003cstrong\u003e7\u003c/strong\u003e, 62 (2015). https://doi.org:10.1186/s13148-015-0103-3\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eFukuoka, K. \u003cem\u003eet al.\u003c/em\u003e Clinical impact of combined epigenetic and molecular analysis of pediatric low-grade gliomas. \u003cem\u003eNeuro Oncol\u003c/em\u003e \u003cstrong\u003e22\u003c/strong\u003e, 1474\u0026ndash;1483 (2020). https://doi.org:10.1093/neuonc/noaa077\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eWeinberger, K. Q. \u0026amp; Saul, L. K. Distance Metric Learning for Large Margin Nearest Neighbor Classification. \u003cem\u003eJ. Mach. Learn. Res.\u003c/em\u003e \u003cstrong\u003e10\u003c/strong\u003e, 207\u0026ndash;244 (2009).\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eZeng, Y., Yang, Y. \u0026amp; Zhao, L. Nonparametric classification based on local mean and class statistics. \u003cem\u003eExpert Syst. Appl.\u003c/em\u003e \u003cstrong\u003e36\u003c/strong\u003e, 8443\u0026ndash;8448 (2009). https://doi.org:10.1016/j.eswa.2008.10.041\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eMylonas, P., Wallace, M. \u0026amp; Kollias, S. in \u003cem\u003eMethods and Applications of Artificial Intelligence\u003c/em\u003e Vol. 3025 \u003cem\u003eLecture Notes in Computer Science\u003c/em\u003e (eds G.A. Vouros \u0026amp; T. Panayiotopoulos) (Springer, Berlin, Heidelberg, 2004).\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eSakr, S. \u003cem\u003eet al.\u003c/em\u003e Comparison of machine learning techniques to predict all-cause mortality using fitness data: the Henry ford exercIse testing (FIT) project. \u003cem\u003eBMC Med Inform Decis Mak\u003c/em\u003e \u003cstrong\u003e17\u003c/strong\u003e, 174 (2017). https://doi.org:10.1186/s12911-017-0566-6\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eAci, M., İnan, C. \u0026amp; Avci, M. A hybrid classification method of k nearest neighbor, Bayesian methods and genetic algorithm. \u003cem\u003eExpert Syst. Appl.\u003c/em\u003e \u003cstrong\u003e37\u003c/strong\u003e, 5061\u0026ndash;5067 (2010). https://doi.org:10.1016/j.eswa.2009.12.004\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eHosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H. \u0026amp; Aerts, H. Artificial intelligence in radiology. \u003cem\u003eNat Rev Cancer\u003c/em\u003e \u003cstrong\u003e18\u003c/strong\u003e, 500\u0026ndash;510 (2018). https://doi.org:10.1038/s41568-018-0016-5\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eKotsiantis, S. B., Zaharakis, I. D. \u0026amp; Pintelas, P. E. Machine learning: a review of classification and combining techniques. \u003cem\u003eArtificial Intelligence Review\u003c/em\u003e \u003cstrong\u003e26\u003c/strong\u003e, 159\u0026ndash;190 (2007). https://doi.org:10.1007/s10462-007-9052-3\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eLibbrecht, M. W. \u0026amp; Noble, W. S. Machine learning applications in genetics and genomics. \u003cem\u003eNat Rev Genet\u003c/em\u003e \u003cstrong\u003e16\u003c/strong\u003e, 321\u0026ndash;332 (2015). https://doi.org:10.1038/nrg3920\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eBalyen, L. \u0026amp; Peto, T. Promising Artificial Intelligence-Machine Learning-Deep Learning Algorithms in Ophthalmology. \u003cem\u003eAsia Pac J Ophthalmol (Phila)\u003c/em\u003e \u003cstrong\u003e8\u003c/strong\u003e, 264\u0026ndash;272 (2019). https://doi.org:10.22608/APO.2018479\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eCurrie, G., Hawk, K. E., Rohren, E., Vial, A. \u0026amp; Klein, R. Machine Learning and Deep Learning in Medical Imaging: Intelligent Imaging. \u003cem\u003eJ Med Imaging Radiat Sci\u003c/em\u003e \u003cstrong\u003e50\u003c/strong\u003e, 477\u0026ndash;487 (2019). https://doi.org:10.1016/j.jmir.2019.09.005\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003ePeiffer-Smadja, N. \u003cem\u003eet al.\u003c/em\u003e Machine learning for clinical decision support in infectious diseases: a narrative review of current applications. \u003cem\u003eClin Microbiol Infect\u003c/em\u003e \u003cstrong\u003e26\u003c/strong\u003e, 584\u0026ndash;595 (2020). https://doi.org:10.1016/j.cmi.2019.09.009\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eTran, Q. T., Alom, M. Z. \u0026amp; Orr, B. A. Comprehensive study of semi-supervised learning for DNA methylation-based supervised classification of central nervous system tumors. \u003cem\u003eBMC Bioinformatics\u003c/em\u003e \u003cstrong\u003e23\u003c/strong\u003e, 223 (2022). https://doi.org:10.1186/s12859-022-04764-1\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003ePidsley, R. \u003cem\u003eet al.\u003c/em\u003e Critical evaluation of the Illumina MethylationEPIC BeadChip microarray for whole-genome DNA methylation profiling. \u003cem\u003eGenome Biol\u003c/em\u003e \u003cstrong\u003e17\u003c/strong\u003e, 208 (2016). https://doi.org:10.1186/s13059-016-1066-1\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eDiaz, L. A., Jr. \u0026amp; Bardelli, A. Liquid biopsies: genotyping circulating tumor DNA. \u003cem\u003eJ Clin Oncol\u003c/em\u003e \u003cstrong\u003e32\u003c/strong\u003e, 579\u0026ndash;586 (2014). https://doi.org:10.1200/JCO.2012.45.2011\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eShen, S. Y. \u003cem\u003eet al.\u003c/em\u003e Sensitive tumour detection and classification using plasma cell-free DNA methylomes. \u003cem\u003eNature\u003c/em\u003e \u003cstrong\u003e563\u003c/strong\u003e, 579\u0026ndash;583 (2018). https://doi.org:10.1038/s41586-018-0703-0\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eStackpole, M. L. \u003cem\u003eet al.\u003c/em\u003e Cost-effective methylome sequencing of cell-free DNA for accurately detecting and locating cancer. \u003cem\u003eNature Communications\u003c/em\u003e \u003cstrong\u003e13\u003c/strong\u003e, 5566 (2022). https://doi.org:10.1038/s41467-022-32995-6\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eAryee, M. J. \u003cem\u003eet al.\u003c/em\u003e Minfi: a flexible and comprehensive Bioconductor package for the analysis of Infinium DNA methylation microarrays. \u003cem\u003eBioinformatics\u003c/em\u003e \u003cstrong\u003e30\u003c/strong\u003e, 1363\u0026ndash;1369 (2014). https://doi.org:10.1093/bioinformatics/btu049\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eTriche, T. J., Jr., Weisenberger, D. J., Van Den Berg, D., Laird, P. W. \u0026amp; Siegmund, K. D. Low-level processing of Illumina Infinium DNA Methylation BeadArrays. \u003cem\u003eNucleic Acids Res\u003c/em\u003e \u003cstrong\u003e41\u003c/strong\u003e, e90 (2013). https://doi.org:10.1093/nar/gkt090\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eSmyth, G. K. \u0026amp; Speed, T. Normalization of cDNA microarray data. \u003cem\u003eMethods\u003c/em\u003e \u003cstrong\u003e31\u003c/strong\u003e, 265\u0026ndash;273 (2003). https://doi.org:10.1016/s1046-2023(03)00155-5\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eHo, T. K. in \u003cem\u003eIEEE.\u003c/em\u003e 278\u0026ndash;282.\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eLiaw, A. \u0026amp; Wiener, M. Classification and Regression by randomForest. \u003cem\u003eR news\u003c/em\u003e \u003cstrong\u003e2\u003c/strong\u003e, 18\u0026ndash;22 (2002).\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eSteele, B. M. Exact bootstrap k-nearest neighbor learners. \u003cem\u003eMachine Learning\u003c/em\u003e \u003cstrong\u003e74\u003c/strong\u003e, 235\u0026ndash;255 (2009). https://doi.org:10.1007/s10994-008-5096-0\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003ein \u003cem\u003eEncyclopedia of Machine Learning\u003c/em\u003e (eds Claude Sammut \u0026amp; Geoffrey I. Webb) 262\u0026ndash;263 (Springer US, 2010).\u003c/span\u003e\u003c/li\u003e\n \u003cli\u003e\u003cspan\u003eFriedman, J., Hastie, T. \u0026amp; Tibshirani, R. Regularization Paths for Generalized Linear Models via Coordinate Descent. \u003cem\u003eJ Stat Softw\u003c/em\u003e \u003cstrong\u003e33\u003c/strong\u003e, 1\u0026ndash;22 (2010).\u003c/span\u003e\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"npj-precision-oncology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjprecisiononcology","sideBox":"Learn more about [npj Precision Oncology](http://www.nature.com/npjprecisiononcology/)","snPcode":"41698","submissionUrl":"https://submission.springernature.com/new-submission/41698/3","title":"npj Precision Oncology","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"deep learning, machine learning, neural network, central nervous system tumor, DNA methylation","lastPublishedDoi":"10.21203/rs.3.rs-3897766/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3897766/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eAs part of the advancement in therapeutic decision-making for brain tumor patients at St. Jude Children\u0026rsquo;s Research Hospital (SJCRH), we develop and compare the performance of three classification models: a deep learning neural network (NN), an exact bootstrap k-nearest neighbor (kNN), and a random forest classifier (RF) model to predict the 82 molecularly distinct central nervous system (CNS) tumor classes based on DNA-methylation profiles of 2,801 patients. We validate their classification accuracy, precision, and recall with 2,054 samples from two independent cohorts. Although all models perform robustly to missing data, the NN model achieves the highest classification accuracy and maintains better balance between precision and recall than kNN and RF. Average precision and recall of NN reduce to that of RF and kNN only when tumor purity was less than 50%. In conclusion, DNA-methylation based deep learning approach provides the most potential advancement toward precision medicine for brain tumors.\u003c/p\u003e","manuscriptTitle":"DNA-Methyaltion-Based Deep Learning for Precision Classification of Central Nervous System Tumors: A Comparative Study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-02-09 17:17:04","doi":"10.21203/rs.3.rs-3897766/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"revise","date":"2024-03-19T16:34:10+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"This content is not available.","date":"2024-02-25T02:46:32+00:00","index":1,"fulltext":"This content is not available."},{"type":"editorInvitedReview","content":"This content is not available.","date":"2024-02-20T21:20:03+00:00","index":3,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2024-02-20T01:21:27+00:00","index":3,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2024-02-19T18:16:04+00:00","index":2,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2024-02-11T17:26:39+00:00","index":1,"fulltext":"This content is not available."},{"type":"reviewersInvited","content":"","date":"2024-02-07T20:21:05+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-01-26T12:07:09+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-01-26T12:06:58+00:00","index":"","fulltext":""},{"type":"submitted","content":"npj Precision Oncology","date":"2024-01-25T16:14:35+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"npj-precision-oncology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjprecisiononcology","sideBox":"Learn more about [npj Precision Oncology](http://www.nature.com/npjprecisiononcology/)","snPcode":"41698","submissionUrl":"https://submission.springernature.com/new-submission/41698/3","title":"npj Precision Oncology","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"bd64d534-de58-4ef5-8f30-564e69ae96f3","owner":[],"postedDate":"February 9th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":28640715,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":28640716,"name":"Health sciences/Oncology/Cancer/CNS cancer"}],"tags":[],"updatedAt":"2024-10-15T05:47:12+00:00","versionOfRecord":{"articleIdentity":"rs-3897766","link":"https://doi.org/10.1038/s41698-024-00718-3","journal":{"identity":"npj-precision-oncology","isVorOnly":false,"title":"npj Precision Oncology"},"publishedOn":"2024-10-02 04:00:00","publishedOnDateReadable":"October 2nd, 2024"},"versionCreatedAt":"2024-02-09 17:17:04","video":"","vorDoi":"10.1038/s41698-024-00718-3","vorDoiUrl":"https://doi.org/10.1038/s41698-024-00718-3","workflowStages":[]},"version":"v1","identity":"rs-3897766","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3897766","identity":"rs-3897766","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}