Comparison of radiomic pre-processing steps in the reproducible prediction of disease free survival across multi-scanners/centers

Comparison of radiomic pre-processing steps in the reproducible prediction of disease free survival across multi-scanners/centers | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Methodology Comparison of radiomic pre-processing steps in the reproducible prediction of disease free survival across multi-scanners/centers Marta Ferreira, Pierre Lovinfosse, Johanne Hermesse, Marjolein Decuypere, and 13 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-875843/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background Features reproducibility and the generalizability of the models are currently among the most important limitations when integrating radiomics into the clinics. Radiomic features are sensitive to imaging acquisition protocols, reconstruction algorithms and parameters, as well as by the different steps of the usual radiomics workflow. We propose a framework for comparing the reproducibility of different pre-processing steps in PET/CT radiomic analysis in the prediction of disease free survival (DFS) across multi-scanners/centers. Results We evaluated and compared the prediction performance of several models that differ in i) the type of intensity discretization, ii) feature selection method, iii) features type i.e, original or tumour to liver ratio radiomic features (OR or TLR). We trained our models using data from one scanner/center and tested on two external scanner/centers. Our results show that there is a low reproducibility in predictions across scanners and discretization methods. Despite of this, TLR based models were generally more robust than OR. Maximum relevance minimum redundancy (MRMR) forward feature selection with Pearson correlation was the feature selection method that had the best mean area under the precision recall curve when using it combining the features from all discretization’s bin’s number (D_All_FBN) with TLR features for two of the four classifiers. Conclusion We evaluated and compared the prediction performance of several models in a data set containing hundred fifty-eight patients with locally advanced cervical cancer (LACC) from three distinct scanners. In our cohort of LAAC patients pre-processing of radiomic features in [ 18 F]FDG PET affects DFS predictions performances across scanners and combining the D_All_FBN TLR approach with the MRMR forward Pearson feature selection method might help increasing robustness of radiomic studies. Bioinformatics Radiomics Intensity discretization Pre-processing radiomics steps Figures Figure 1 Figure 2 Figure 3 Background Radiomics consists of characterizing tumour phenotypes via the extraction of high-dimensional quantitative features from medical images, with the aim of supporting clinical decision-making [ 1 – 3 ]. Radiomic features have been used to characterize cancer subtypes and aggressiveness or to predict the response to treatment [ 4 – 8 ] and have increasingly been combined with machine learning (ML) techniques in order to predict a specific clinical outcome [ 5 – 6 ][ 9 – 13 ]. The challenges facing the integration of radiomics into the clinics are many. Features reproducibility and the generalizability of the models are currently among the most important limitations. Even though standardization guidelines have helped to mitigate some of these reproducibility challenges [ 14 ], radiomic features are sensitive to imaging acquisition protocols and reconstruction algorithms and parameters, but can also be affected by the different steps of the usual radiomics workflow [ 15 – 19 ]. 2-[ 18 F]fluoro-2-deoxy-D-glucose ([ 18 F]FDG) positron emission tomography combined with computed tomography (PET/CT) imaging is especially prone to reproducibility issues due to frequent variations in pre-acquisition settings and scanner properties [ 20 ], despite standardization efforts of acquisition and reconstruction protocols within the context of multicenter trials [ 21 ]. The intensity discretization scheme is one of the steps in radiomics workflow known to affect models reproducibility. Most of radiomic studies assume a specific discretization method based on the results of previous studies [ 5 ][ 20 ]. Additionally, some studies used phantoms or internal data sets and evaluated the impact of some of the pre-processing steps using test-retest analysis [ 18 ][ 22 ]. In this study, we compared the effect of 6 [ 18 F]FDG PET intensity discretization methods on the prediction of disease free survival (DFS) in locally advanced cervical cancer (LACC) patients from 3 different scanners/centers. Moreover, we evaluated the effect of combining these different discretization approaches with 7 distinct feature selection (FS) methods as well as with the effect of feature transformation using tumour to liver ratio (TLR) features as done in our previous work [ 23 ]. In contrast to what was done in our previous work, we trained 84 distinct models using the data of one of the scanners and evaluated these models on the remaining two. We further evaluated these different steps using 4 different classifiers in order to better enhance the study robustness. Materials And Methods Data One hundred and fifty-eight patients with LACC imaged between 2010 and 2016, in three different scanners were included in this retrospective study. PET/CT studies were performed in the CHU of Liège, where 89 studies were acquired using a Philips Gemini TF or BB (scanner A), in the CHU of Brest and ICO St Herblain, where 17 and 34, respectively, were acquired using a Siemens Biograph mCT (scanner B) and at the McGill University Health Center, where 18 studies were performed with a General Electric Discovery ST (scanner C). The patient’s clinical characteristics, treatment, acquisition and reconstruction protocols are described in our previous study [ 23 ]. Experimental design DFS was dichotomized into a binary endpoint, i.e. recurrence or no recurrence, independently of the time-to-event. Next we compared the prediction performance of 84 different models, which differed in i) the type of intensity discretization used before feature calculation, ii) the feature selection method iii) the features type, i.e., original radiomics (OR) or TLR radiomics. Models were trained on the data from scanner A and then evaluated on data from two remaining external scanners independently. The metric used to evaluate model performance was the area under the curve of the precision recall curve (AUCpr) (Fig. 1 ). Statistical and ML analyses were performed using R software, version 4.0.1. Segmentation and interpolation PET images from scanners B and C were interpolated using the research toolbox (Oncoradiomics SA, Liège, Belgium), up-sampling or down-sampling the images using a linear method, so that all datasets had isotropic voxels of 4×4×4 mm 3 (i.e., the voxel size in images of scanner A). The 3D primary tumour volumes were segmented in the [ 18 F]FDG PET images using the 2 classes semi-automatic Fuzzy Local Adaptive Bayesian algorithm [ 24 ]. Volumes of 20 cm 3 in the liver were manually drawn in order to investigate the predictive value of TLR radiomic features, as explained below. All segmentations were reviewed and edited if needed by one nuclear medicine physician with 9 years of experience in clinical PET/CT. Images Radiomic features We extracted two hundred and fifteen features from the segmented volumes, which included first order grey level statistics, geometry, fractals, texture matrix based features and others. Features were extracted using the Oncoradiomics research toolbox and their detailed description can be found in supplementary data of our previous study [ 23 ]. All features were calculated according to the Imaging biomarkers standardization initiative (IBSI) [ 25 ]. We also studied the ratio of the features values calculated in the tumour and in the liver the (TLR versions of features), except for the shape features as done in our previous study [ 23 ]. We hypothesized that TLR features may reduce the variability of radioactive dose uptake within the different patients and across centers by normalizing the radiomic features using the liver which is an organ with an homogenous and reproducible uptake. There were no missing data for any patient. Radiomic features intensity discretization Image intensities were discretized using the two schemes currently standardized by the IBSI: fixed bin number (FBN, with 32 and 64 bins) and fixed bin width (FBW, with 4 different widths of 0.05, 0.1, 0.2 and 0.5 Standardized Uptake Values) [ 25 ]. These two sets of features were considered either alone or by: 1) joining the features from all discretization’s widths/bin’s number (D_All_FBW, D_All_FBN), 2) combining the four discretization’s widths from the FBW discretization method or combining the two number of bins from FBN through the calculation of their median value (D_Med_FBN, D_Med_FBW). Features selection, classifiers and model selection We applied 7 different FS methods to identify the 5 most relevant features: 1-Accuracy decrease obtained from the embedded FS of the random forest (RF) classifier; 2- Gini impurity decrease obtained from the embedded FS of the RF classifier; 3- forward FS using maximum relevance minimum redundancy (MRMR) method with Pearson correlation; 4- backward FS using MRMR with Pearson correlation; 5- forward FS using MRMR with Spearman correlation; 6- backward FS using MRMR with Spearman correlation; 7- forward MRMR based on the mutual information (MI). We also considered 4 ML classifiers: RF, support vector machine (SVM) with radial kernel, Naïve Bayes (NB) and a logistic regression (LR) [ 26 – 28 ]. We used for each classifier the default hyperparameters values in their respective R packages. We used 5-fold cross-validation in the training data to internally validate and select the models with better predictions for each classifier independently. Additionally, models were trained using all the training data then tested in the two external data sets. A paired Wilcoxon Rank Sum test was used to test whether the predictions for each discretization scheme were statistically significantly different from each other in the two external validation schemes. Wilcoxon Rank Sum tests were considered significant if p < 0.05. Holm-Bonferroni correction method was used to correct for multiple hypothesis testing. Results Table 1 depicts the mean AUCpr between the three validation schemes, i.e, i) Internal validation using 5-fold cross validation using scanner A ii) external validation using scanner B iii) external validation using scanner C. The table shows the mean AUCpr of the models using RF, SVM, LR and NB classifier using the different FS methods applied to the OR and TLR features. Additionally, the results shown for the standard FBW and FBN discretization schemes correspond to the model with discretization width/bin number that had a better AUCpr in the internal validation scheme. The AUCpr of the three validation schemes individually are shown in the supplementary material. Our results showed a low reproducibility between scanners. The discretization scheme that showed the higher AUCpr in the validation scheme was D_Med_FBN combined with TLR features. This was not the case for the two external scanners. (Supplementary material). Table 1 MEAN AUCPR BETWEEN VALIDATION SCHEMES FBW OR FBN OR D_Med_FBW OR D_Med_FBN OR D_All_FBW OR D_All_FBN OR FBW TLR FBN TLR D_Med_FBW TLR D_Med_FBN TLR D_All_FBW TLR D_All_FBN TLR RF + MRMR_Forward_pearson 0.51 0.46 0.44 0.48 0.52 0.46 0.44 0.55 0.44 0.52 0.51 0.51 RF + MRMR_Backward_pearson 0.45 0.38 0.45 0.42 0.43 0.39 0.49 0.48 0.44 0.44 0.37 0.49 RF + MRMR_Forward_ spearman 0.46 0.50 0.42 0.57 0.43 0.47 0.50 0.47 0.45 0.57 0.56 0.44 RF + MRMR_Backward_ spearman 0.39 0.47 0.39 0.42 0.46 0.43 0.51 0.55 0.49 0.51 0.48 0.40 RF + MRMR_MI 0.46 0.44 0.47 0.45 0.46 0.40 0.46 0.51 0.45 0.52 0.48 0.51 RF + RF_Accuracy 0.52 0.47 0.54 0.55 0.53 0.49 0.53 0.56 0.54 0.58 0.52 0.49 RF + RF_Gini 0.47 0.49 0.42 0.50 0.51 0.48 0.52 0.56 0.53 0.52 0.51 0.51 SVM + MRMR_ Forward_pearson 0.48 0.49 0.44 0.47 0.52 0.49 0.60 0.51 0.44 0.57 0.57 0.51 SVM + MRMR_ Backward_pearson 0.37 0.41 0.37 0.45 0.41 0.43 0.58 0.48 0.42 0.42 0.43 0.41 SVM + MRMR_Forward_ spearman 0.44 0.40 0.42 0.52 0.52 0.40 0.48 0.43 0.42 0.43 0.45 0.41 SVM + MRMR_Backward_ spearman 0.38 0.46 0.35 0.46 0.40 0.38 0.42 0.47 0.41 0.36 0.38 0.36 SVM + MRMR_MI 0.38 0.39 0.47 0.33 0.43 0.37 0.46 0.48 0.45 0.52 0.46 0.48 SVM + RF_Accuracy 0.52 0.47 0.39 0.44 0.49 0.42 0.51 0.52 0.50 0.51 0.46 0.56 SVM + RF_Gini 0.56 0.51 0.35 0.45 0.56 0.45 0.51 0.50 0.55 0.44 0.50 0.45 LR + MRMR_Forward_pearson 0.49 0.51 0.48 0.48 0.48 0.51 0.55 0.57 0.51 0.51 0.56 0.57 LR + MRMR_ Backward_pearson 0.47 0.46 0.47 0.47 0.50 0.47 0.50 0.50 0.48 0.41 0.40 0.48 LR + MRMR_Forward_ spearman 0.34 0.48 0.34 0.52 0.38 0.43 0.43 0.51 0.38 0.50 0.39 0.46 LR + MRMR_Backward_ spearman 0.43 0.48 0.39 0.48 0.39 0.40 0.48 0.47 0.45 0.56 0.43 0.46 LR + MRMR_MI 0.39 0.44 0.45 0.51 0.41 0.46 0.42 0.48 0.43 0.53 0.42 0.48 LR + RF_Accuracy 0.53 0.48 0.53 0.50 0.50 0.46 0.51 0.57 0.50 0.51 0.48 0.51 LR + RF_Gini 0.50 0.50 0.51 0.49 0.52 0.46 0.48 0.55 0.50 0.52 0.49 0.53 NB + MRMR_Forward_pearson 0.45 0.50 0.44 0.57 0.47 0.50 0.52 0.50 0.47 0.50 0.55 0.58 NB + MRMR_ Backward_pearson 0.44 0.49 0.43 0.43 0.50 0.46 0.48 0.49 0.42 0.38 0.42 0.46 NB + MRMR_Forward_ spearman 0.40 0.49 0.35 0.52 0.42 0.43 0.44 0.49 0.46 0.53 0.44 0.49 NB + MRMR_Backward_spearman 0.43 0.51 0.42 0.46 0.37 0.46 0.45 0.49 0.41 0.52 0.42 0.50 NB + MRMR_MI 0.45 0.51 0.41 0.52 0.41 0.53 0.43 0.46 0.42 0.48 0.43 0.46 NB + RF_Accuracy 0.52 0.56 0.49 0.55 0.49 0.51 0.48 0.56 0.52 0.47 0.46 0.45 NB + RF_Gini 0.54 0.51 0.52 0.54 0.55 0.46 0.47 0.46 0.42 0.46 0.43 0.49 Mean AUCpr between the three validation schemes using the four classifiers (RF, SVM, LR and NB) and the seven FS methods represented in the columns and with the different discretization schemes in the rows. The features discretization were applied to the OR and TLR features. Despite the models low reproducibility, D_All_FBN with TLR features was the model with the better mean AUCpr for the LR and NB classifier (0.57 and 0.58 respectively). It was also the second model with overall higher AUCpr in the two independent scanners with AUCpr of 0.45 and 0.7 in scanner B and C respectively (Fig. 2 ). For the RF classifier the model with higher AUCpr was D_Med_FBN TLR whereas for SVM it was FBW TLR. Regarding the FS method, when combined with the D_All_FBN TLR, MRMR Forward with Pearson correlation was the optimal FS method for at least one of the four classifiers in all validation schemes. MRMR Forward with Pearson correlation is also the FS method that showed better mean AUCpr for 4 out of the 6 discretization schemes when using TLR features. The discretization schemes that showed higher mean AUCpr across classifiers were D_Med_FBN, D_All_FBW and D_All_FBN. All of them were used with TLR features (Fig. 3 ). Despite of the good performance of D_All_FBN with TLR, the only discretization methods that showed to be statistically significant from D_All_FBN TLR were FBW with OR features, and FBW, FBN, D_Med_FBW, D_Med_FBN, D_All_FBW with TLR features. FBW combined with OR features was the only discretization scheme statistically different from all the others (Table 2). TLR based models had higher mean AUCpr values than OR based models for all the classifiers and discretization schemes, except when using D_med_FBW with LR or FBW and D_Med_FBN with NB (Fig. 3 ). TLR based models predictions in the two external validation schemes were statistically significant from all the OR based models (p-value < < 0.05). Table 2 Wilcoxon rank sum test corrected p-values showing the statistical significance between discretization schemes FBW_OR FBN_ OR D_Med_ FBW_OR D_Med_ FBN_OR D_All_ FBW_OR D_All_ FBN_OR FBW_TLR FBN_TLR D_Med_ FBW_TLR D_Med_ FBN_TLR D_All_FBW_TLR D_All_FBN_TLR FBW_OR FBN_OR 2.11E-11 D_Med_ FBW_OR 5.22E-07 1.48E + 01 D_Med_ FBN_OR 5.43E-13 4.30E + 00 2.95E-03 D_All_ FBW_OR 5.63E-09 4.59E + 01 7.51E + 00 3.30E + 01 D_All_ FBN_OR 2.07E-04 2.87E + 02 1.86E + 01 6.37E-05 1.26E-01 FBW_TLR 8.77E-50 3.24E-14 2.01E-37 1.32E-11 1.93E-18 1.60E-25 FBN_TLR 4.41E-25 7.47E-03 3.71E-15 9.63E-03 2.87E-08 2.26E-15 9.22E-07 D_Med_ FBW_TLR 3.33E-44 5.81E-10 1.61E-28 1.62E-06 1.58E-15 2.15E-21 5.18E-02 2.05E-01 D_Med_ FBN_TLR 2.70E-40 7.49E-24 8.82E-29 4.52E-18 1.37E-16 4.89E-36 6.43E + 01 1.41E-09 6.97E + 00 D_All_ FBW_TLR 1.83E-25 2.94E-13 4.28E-16 1.18E-12 1.16E-12 1.25E-13 4.97E + 00 5.07E-03 5.33E + 01 1.71E + 00 D_All_ FBN_TLR 1.51E-04 6.42E + 01 7.73E + 00 3.18E + 01 3.18E + 01 2.18E + 01 2.60E-23 2.26E-09 8.29E-16 4.13E-24 4.20E-10 Discussion Radiomics aims at converting data from medical images into quantitative features providing valuable information regarding the clinical management of patients. These features can be combined with statistical/ML methods in order to derive predictive models of clinically relevant endpoints. The radiomics workflow consists however of multiple pre-processing steps that can affect the radiomic features values and therefore their clinical relevance. In this study we compared the prediction performance of numerous models that differed in their discretization and FS methods. This comparison was carried out within the context of predicting DFS from [ 18 F]FDG PET images in a multi-scanner/center [ 18 F]FDG PET cohort of LACC patients. Additionally, we investigated the effect of features transformation using features ratios with an organ of reference and we combined it with the previous pre-processing steps. We also compared the models performances using four classifiers. Multi-classifier radiomics predictive models, ensemble classifiers or the combination of different classifiers performances to measure feature importance, consistently tend to outperform traditional single classifier approaches [ 29 – 31 ]. Moreover, the choice of classification method is one the most dominant sources of performance variation in radiomics studies [ 12 ]. Due to this, we believe that comparing the results of multiple classifiers is needed when evaluating the robustness of the workflow. The discretization scheme is one of the factors that affect radiomic features reproducibility. FBW discretization in PET has been recommended [ 18 ][ 25 ], although some studies have also reported more favourable properties using FBN [ 32 ]. This is related to the fact that FBN and FBW have different drawbacks and advantages. FBW preserves the relationship between PET units and the corresponding physical substrate, contrary to arbitrary units (such as in some non-quantitative magnetic resonance imaging sequences). FBN on the other hand does not preserve such relationship but introduces a normalization effect that can be favourable when contrast is considered important or when the actual original image intensity value does not have a ‘meaning’. In our study D_All_FBN and FBW both combined with TLR features were the discretization scheme that showed the best AUCpr in the two external scanners. Combining features discretized with different widths/bin numbers can as shown in our study introduce complementary information and be a more reproducible and simple strategy as it also avoids the uncertain assumption or the extensive search of the optimal feature discretization width/bin number. Combining feature discretization schemes has also been done in previous studies [ 33 – 34 ]. Furthermore, our results show that when using D_All_FBN with TLR the FS scheme with higher mean AUCpr in the three validation schemes is MRMR Forward with Pearson correlation for 2 of the 4 classifiers. FS is an effective strategy to improve radiomics-based predictive studies. Different FS strategies are used in radiomics studies, each with his pros and cons, and some known to work better with certain type of features or classifiers [ 35 ]. Finally, we also evaluated the feature type, i.e. OR and TLR radiomics. We have shown in our previous study that using the ratio of the tumour features with a reference organ (TLR radiomics) improves the predictive performance of radiomics model in LACC. In contrast to our previous study, we trained our models using only data from one clinical center/scanner and evaluated our models in two external scanners. We emphasize the conclusions of our previous study, by observing that all of the most robust models used TLR features instead of OR. This can be caused by a normalizing effect of the SUVs on each patient. The importance of data normalization/transformation has also been accessed for other radiomic studies and shown to improve models performances [ 36 – 38 ]. Moreover, feature transformation using an organ of reference has also been investigated by other authors, leading to normalized images and increased reproducibility of radiomic features [ 39 – 40 ]. The results of our study are encouraging and can potentially be used as a first recommendation approach to improve reproducibility of radiomics studies across multi-scanners/centers in LACC. However, it corresponds to a preliminary study and we still observed performance differences between the 3 scanners: for some scanners some models work quite well while for other scanners other models work better. This could be caused by the variation in scanner properties, the different number of patients in each scanner, or the variation in tumour recurrence rate for each population. In future work, the tumour segmentations should be done fully automatically using a recently validated approach [ 41 ], instead of being done by a single observer in a semi-automated way, since segmentation variations within patients can affect radiomics reproducibility [ 20 ]. Within this work, we also did not try combining all the features from the two discretization schemes, mainly due to the resulting need for longer computation times. Combining discretization methods, exploring more classifiers, combining the information from PET with other image modalities and explore image or matrix fusion strategies as done successfully before by other authors [ 27 ] will be investigated in future work. Our findings should also be validated using a larger data set and within the context of other pathologies. Conclusion In the present paper, we proposed a framework for comparing different pre-processing PET strategies using a multi-center series with the intent of predicting DFS in LACC patients. Our results show that there is a low reproducibility in predictions across scanners and discretization methods. Combining features calculated with different numbers of bins, relying on the normalizing effect of tumour to liver ratio, and using the maximum relevance minimum redundancy feature selection method was found to increase the robustness of the developed models and could be recommended for future radiomic studies in a similar context. These recommendations should now be evaluated in larger cohorts and in different cancer types. Abbreviations AUCpr: area under the curve of the precision recall curve CT: computed tomography DFS: disease free survival D_All_FBW: discretization using all widths of the fixed bin width discretization D_All_FBN: discretization using all bin’s number of the fixed bin number discretization D_Med_FBW: discretization using the median value of features with the different widths used in the fixed bin width discretization D_Med_FBN: discretization using the median value of features with the different bins used in the fixed bin number discretization [ 18 F]FDG: 2-[18F]fluoro-2-deoxy-D-glucose FBW: fixed bin width FBN: fixed bin number FS: feature selection IBSI: Imaging biomarkers standardization initiative LACC: locally advanced cervical cancer LR: logistic regression MI: mutual information ML: machine learning NB: Naïve Bayes MRMR: maximum relevance minimum redundancy OR: original radiomics PET: positron emission tomography RF: random forest SVM: support vector machine TLR: tumour to liver ratio Declarations Funding This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 766276. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. Competing interests Dr Philippe Lambin reports, within and outside the submitted work, grants/sponsored research agreements from Varian medical, Oncoradiomics, ptTheragnostic/DNAmito, Health Innovation Ventures. He received an advisor/presenter fee and/or reimbursement of travel costs/external grant writing fee and/or in kind manpower contribution from Oncoradiomics, BHV, Merck, Varian, Elekta, ptTheragnostic and Convert pharmaceuticals. Dr Lambin has shares in the company Oncoradiomics, Convert pharmaceuticals, MedC2 and LivingMed Biotech, he is co-inventor of two issued patents with royalties on radiomics (PCT/NL2014/050248, PCT/NL2014/050728) licensed to Oncoradiomics and one issue patent on mtDNA (PCT/EP2014/059089) licensed to ptTheragnostic/DNAmito, three non-patented invention (softwares) licensed to ptTheragnostic/DNAmito, Oncoradiomics and Health Innovation Ventures and three non-issues, non licensed patents on Deep Learning-Radiomics and LSRT (N2024482, N2024889, N2024889). He confirms that none of the above entities or funding was involved in the preparation of this paper. Ethical approval and consent to participate All procedures were performed in accordance with the principles of the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. The study design and exemption from informed consent were approved by the Institutional Review Board of Liege University Hospital. Consent for publication For this type of retrospective study formal consent is not required. Availability of data and materials The datasets generated and/or analysed during the current study are available in https://github.com/msilvaferreira/Phd/tree/master/FDG%20PET%20radiomics%20to%20predict%20disease%20free%20survival%20in%20Cervical%20Cancer Authors' contributions Study concepts/study design: Marta Ferreira, Patrick E.Meyer and Roland Hustinx Literature research: Marta Ferreira and Patrick E.Meyer Clinical studies: Johanne Hermesse, Marjolein Decuypere, Philippe Robin and Frédéric Kridelka Data analysis: Marta Ferreira, Patrick E.Meyer, Pierre Lovinfosse and Roland Hustinx Statistical analysis: Marta Ferreira and Patrick E.Meyer Manuscript drafting, editing and revision: All authors read and approved the final manuscript Acknowledgements Not applicable References Lambin P, et al. Radiomics: Extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48:441–6. Aerts HJWL, et al., “Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach,” Nat. Commun., vol. 5, 2014. Lambin P, et al. Radiomics: The bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14:749–62. Tsujikawa T, et al., “18F-FDG PET radiomics approaches: comparing and clustering features in cervical cancer,” Ann. Nucl. Med., vol. 31, 2017. Altazi BA, et al. Investigating multi-radiomic models for enhancing prediction power of cervical cancer treatment outcomes. Phys Medica. 2018;46:180–8. Hao H, et al., “Shell feature: A new radiomics descriptor for predicting distant failure after radiotherapy in non-small cell lung cancer and cervix cancer,” Phys. Med. Biol., vol. 63, 2018. Lucia F, et al. Prediction of outcome using pretreatment 18 F-FDG PET/CT and MRI radiomics in locally advanced cervical cancer treated with chemoradiotherapy. Eur J Nucl Med Mol Imaging. 2018;45:768–86. Bowen SR, et al. Tumor radiomic heterogeneity: Multiparametric functional imaging to characterize variability and predict response following cervical cancer radiation therapy. J Magn Reson Imaging. 2017;47:1388–96. Shen WC, et al. Prediction of local relapse and distant metastasis in patients with definitive chemoradiotherapy-treated cervical cancer by deep learning from [18F]-fluorodeoxyglucose positron emission tomography/computed tomography. Eur Radiol. 2019;29:6741–9. Sun W, Jiang M, Dang J, Chang P, Yin FF. “Effect of machine learning methods on predicting NSCLC overall survival time based on Radiomics analysis,” Radiat. Oncol., 2018. Leger S, Zwanenburg A, Pilz K, Lohaus F. “A comparative study of machine learning methods for time-to-event survival data for radiomics risk modelling”, pp. 1–11, 2017. Parmar C, Grossmann P, Bussink J, Lambin P, Aerts HJWL. “Machine Learning methods for Quantitative Radiomic Biomarkers,” Sci. Rep., vol. 5, 2015. Deist TM, et al., “Machine learning algorithms for outcome prediction in (chemo)radiotherapy: An empirical comparison of classifiers,” Med. Phys., 2018. Lei M, et al., “Benchmarking features from different radiomics toolkits / toolboxes using Image Biomarkers Standardization Initiative,” arXiv, 2020. Lovinfosse P, Visvikis D, Hustinx R, Hatt M. “FDG PET radiomics: a review of the methodological aspects,” pp. 379–391, 2018. Shiri I, Rahmim A, Ghaffarian P, Geramifar P, Abdollahi H, Bitarafan-rajabi A. The impact of image reconstruction settings on 18F-FDG PET radiomic features: multi-scanner phantom and patient studies Lesions to Background Ratio Time of Flight Full Width at Half Maximum Response Evaluation Criteria in Solid Tumours. Eur Radiol. 2017;27:4498–509. Shafiq-ul-hassan M, et al. Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels. ” Med Phys. 2017;44:1050–62. Leijenaar RTH, et al., “The effect of SUV discretization in quantitative FDG-PET Radiomics: the need for standardized methodology in tumor texture analysis,” Nat. Publ. Gr., pp. 1–10, 2015. Altazi BA, et al. Reproducibility of F18-FDG PET radiomic features for different cervical tumor segmentation methods, gray-level discretization, and reconstruction algorithms. J Appl Clin Med Phys. 2017;18:32–48. Liberini V, et al., “Impact of segmentation and discretization on radiomic features in 68Ga-DOTA-TOC PET/CT images of neuroendocrine tumor,” EJNMMI Phys., vol. 8, 2021. Aide N, Lasnon C, Veit-Haibach P, Sera T, Sattler B, Boellaard R. EANM/EARL harmonization strategies in PET quantification: from daily practice to multicentre oncological studies. Eur J Nucl Med Mol Imaging. 2017;44:17–31. Schwier M, et al., “Repeatability of multiparametric prostate MRI radiomics features,” Sci. Rep., 2019. Ferreira M, et al., “[18F]FDG PET radiomics to predict disease-free survival in cervical cancer: a multi-scanner/center study with external validation,” Eur. J. Nucl. Med. Mol. Imaging, 2021. Hatt M, Cheze C, Turzo A, Roux C. A fuzzy locally adaptive Bayesian segmentation approach for volume determination in PET. ” IEEE Trans Med Imaging. 2009;28:881–93. Zwanenburg A, et al. The image biomarker standardization initiative: Standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. 2020;295:328–38. Hanchuan Peng F. Long, “Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy. IEEE Trans Pattern Anal Mach Intell. 2005;27:1226–38. Breiman L, Friedman J, Olshen R, Classification and Regression Trees. 1984. Géron A. Hands-On Machine Learning with Scikit-Learn. 1st ed.: O’Reilly; 2017. Zhou Z, et al., “Constructing multi-modality and multi-classifier radiomics predictive models through reliable classifier fusion,” IEEE Comput. Soc., 2017. Zhou Z, et al. “Multifactorial cancer treatment outcome prediction through multifaceted radiomics. ”, arXiv: Medical Physics; 2018. Osman AFI. A Multi-parametric MRI-Based Radiomics Signature and a Practical ML Model for Stratifying Glioblastoma Patients Based on Survival Toward Precision Oncology. Front Comput Neurosci. 2019;13:1–15. Presotto L, et al. Physica Medica Original paper PET textural features stability and pattern discrimination power for radiomics analysis: An ‘ ad-hoc ’ phantoms study. Phys Medica. 2018;50:66–74. Vallières M, et al. Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. Sci Rep. 2017;7:1–14. Lv W, Ashrafinia S, Ma J, Lu L, Rahmim A. Multi-level multi-modality fusion radiomics: Application to PET and CT imaging for prognostication of head and neck cancer. IEEE J Biomed Heal Informatics. 2020;24:2268–77. N. S.-M. & A. A.-B. Verónica Bolón-Canedo, Feature Selection for High-Dimensional Data. Springer, 2016. Scalco E, et al. T2w-MRI signal normalization affects radiomics features reproducibility. Med Phys. 2020;47:1680–91. Haga A, et al. Standardization of imaging features for radiomics analysis. J Med Investig. 2019;66:35–7. Li XT, Huang RY. Standardization of imaging methods for machine learning in neuro-oncology. Neuro-Oncology Adv. 2020;2:iv49–55. Isaksson LJ, et al. Effects of MRI image normalization techniques in prostate cancer radiomics. Phys Medica. 2020;71:7–13. Traverso A, et al. Sensitivity of radiomic features to inter-observer variability and image pre-processing in Apparent Diffusion Coefficient (ADC) maps of cervix cancer patients. Radiother Oncol. 2020;143:88–94. Iantsen A, et al., “Convolutional neural networks for PET functional volume fully automatic segmentation: development and validation in a multi-center setting,” Eur. J. Nucl. Med. Mol. Imaging, 2021. Supplementary Files Supplementarymaterial.pdf Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-875843","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Methodology","associatedPublications":[],"authors":[{"id":50325105,"identity":"668f915b-18d7-4454-84c4-ca82e7fd180f","order_by":0,"name":"Marta Ferreira","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABNElEQVRIie2PMUvDQBiG7zi4LNe6Xmkhv0BIECrBqH/lQuFcWhxcHFMOkiU6WxT8DS6dUwq6RLoWblEKdVFItiDBemlxSoO6idwzvLwH78PxAaDR/FncMqAPwLkLmqrSVAU2avdqyr+UhAOsauuqVNC3SlmD6Vppk/JVo+yGjy9POXNP90MRpDlGJjYiW7rvY7OJAEyzfkXpJif2MGLcuUkmYhQRbAck2TsYXEo7QAC1RuOqEnPoN/KpRaknAKGEYdrvtgeRhErBqLFFmS3hsGAri5rPAhYW3ShOJI9rlTmHgrBY/QIFIszaKCCXXr2yhNcd1rMo8QTqxEzdcn/mXPiyFyAott4y4yB7ZUcWNR4W8O1jZe4Y4m6eF/LwNhSTNKsqNcBgnf5P9yXFb8YajUbzz/kEXfJmKf2NFaUAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0003-4532-6744","institution":"University of Liege: Universite de Liege","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Marta","middleName":"","lastName":"Ferreira","suffix":""},{"id":50325106,"identity":"f29fd711-e52d-4125-9185-9c3fe4c12ca2","order_by":1,"name":"Pierre Lovinfosse","email":"","orcid":"","institution":"CHU de Liège: Centre hospitalier universitaire de Liege","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Pierre","middleName":"","lastName":"Lovinfosse","suffix":""},{"id":50325107,"identity":"3d5c2bdc-50e0-4f72-bdb8-93d1f64f56c0","order_by":2,"name":"Johanne Hermesse","email":"","orcid":"","institution":"CHU de Liège: Centre hospitalier universitaire de Liege","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Johanne","middleName":"","lastName":"Hermesse","suffix":""},{"id":50325108,"identity":"8df317ce-7bce-4fd7-910e-3d3b1c903ea1","order_by":3,"name":"Marjolein Decuypere","email":"","orcid":"","institution":"CHU de Liège: Centre hospitalier universitaire de Liege","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Marjolein","middleName":"","lastName":"Decuypere","suffix":""},{"id":50325109,"identity":"656220eb-7cb4-41b9-99b4-f1ac4a26985c","order_by":4,"name":"Caroline Rousseau","email":"","orcid":"","institution":"Université de Nantes: Universite de Nantes","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Caroline","middleName":"","lastName":"Rousseau","suffix":""},{"id":50325110,"identity":"bdf6a162-3a40-4c3c-afca-22fc02ddffe9","order_by":5,"name":"François Lucia","email":"","orcid":"","institution":"Centre Hospitalier Universitaire de Brest: CHRU de Brest","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"François","middleName":"","lastName":"Lucia","suffix":""},{"id":50325111,"identity":"043a0816-a111-42cb-a6b8-70f2e40c98f3","order_by":6,"name":"Ulrike Schick","email":"","orcid":"","institution":"Centre Hospitalier Universitaire de Brest: CHRU de Brest","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Ulrike","middleName":"","lastName":"Schick","suffix":""},{"id":50325112,"identity":"9f01bbce-da37-43dd-817c-4e104100e034","order_by":7,"name":"Caroline Reinhold","email":"","orcid":"","institution":"McGill University Health Centre","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Caroline","middleName":"","lastName":"Reinhold","suffix":""},{"id":50325113,"identity":"3b95b3d5-b3cf-4b79-b74e-2558be7894e0","order_by":8,"name":"Philippe Robin","email":"","orcid":"","institution":"Centre Hospitalier Universitaire de Brest: CHRU de Brest","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Philippe","middleName":"","lastName":"Robin","suffix":""},{"id":50325114,"identity":"d6b83d29-91be-48b5-9911-5ddc1aa87ff4","order_by":9,"name":"Mathieu Hatt","email":"","orcid":"","institution":"Universite de Brest: Universite de Bretagne Occidentale","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Mathieu","middleName":"","lastName":"Hatt","suffix":""},{"id":50325115,"identity":"237ac825-80c2-46d8-b93c-7785094a021e","order_by":10,"name":"Dimitris Visvikis","email":"","orcid":"","institution":"Universite de Brest: Universite de Bretagne Occidentale","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Dimitris","middleName":"","lastName":"Visvikis","suffix":""},{"id":50325116,"identity":"dc380afe-2048-4ae3-bf3f-9e6b20992f82","order_by":11,"name":"Claire Bernard","email":"","orcid":"","institution":"Central University Hospital of Liege: Centre hospitalier universitaire de Liege","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Claire","middleName":"","lastName":"Bernard","suffix":""},{"id":50325117,"identity":"56902d5d-25ee-49bb-8086-e968ede8da36","order_by":12,"name":"Ralph T.H. Leijenaar","email":"","orcid":"","institution":"Maastricht University Hospital: Maastricht Universitair Medisch Centrum+","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Ralph","middleName":"T.H.","lastName":"Leijenaar","suffix":""},{"id":50325118,"identity":"d601b369-8e28-494e-8426-00ea6305188b","order_by":13,"name":"Frédéric Kridelka","email":"","orcid":"","institution":"Central University Hospital of Liege: Centre hospitalier universitaire de Liege","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Frédéric","middleName":"","lastName":"Kridelka","suffix":""},{"id":50325119,"identity":"bb83b3e2-7bd9-43cf-b0ea-993203c08172","order_by":14,"name":"Philippe Lambin","email":"","orcid":"","institution":"Maastricht University Hospital: Maastricht Universitair Medisch Centrum+","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Philippe","middleName":"","lastName":"Lambin","suffix":""},{"id":50325120,"identity":"18cdaaaa-dfc2-481e-a711-e511d3cf5b6d","order_by":15,"name":"Roland Hustinx","email":"","orcid":"","institution":"University of Liege: Universite de Liege","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Roland","middleName":"","lastName":"Hustinx","suffix":""},{"id":50325121,"identity":"7fca2ba2-fe68-4a32-958d-d325e2b7ef18","order_by":16,"name":"Patrick E. Meyer","email":"","orcid":"","institution":"Liege University: Universite de Liege","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Patrick","middleName":"E.","lastName":"Meyer","suffix":""}],"badges":[],"createdAt":"2021-09-04 04:05:12","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-875843/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-875843/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":13188117,"identity":"0bf05275-dfa6-4071-ac52-709d5e86045c","added_by":"auto","created_at":"2021-09-08 17:06:30","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":14281,"visible":true,"origin":"","legend":"Radiomics workflow.","description":"","filename":"OnlineFig1.png","url":"https://assets-eu.researchsquare.com/files/rs-875843/v1/2cb8942a9267f83ab409d8ec.png"},{"id":13188308,"identity":"a2287397-2acd-4b66-91e1-9791701c8352","added_by":"auto","created_at":"2021-09-08 17:09:30","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":31809,"visible":true,"origin":"","legend":"Bar plot showing the AUCpr in the two external scanners (B and C) for all discretization strategies. The models depicted in the plot use the FS method and classifier which showed better performance in the internal validation scheme (scanner A).","description":"","filename":"OnlineFig2.png","url":"https://assets-eu.researchsquare.com/files/rs-875843/v1/9d7355defba49823499e77c8.png"},{"id":13188307,"identity":"c69a2f7e-c58e-4520-b275-595418294e48","added_by":"auto","created_at":"2021-09-08 17:09:30","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":38116,"visible":true,"origin":"","legend":"Bar plot showing the models with FS that showed the better performances for all intensity discretization schemes and for the four classifiers. The AUCpr correspond to the mean value of the 3 validation schemes.","description":"","filename":"OnlineFig3.png","url":"https://assets-eu.researchsquare.com/files/rs-875843/v1/b3308d12f5d93762a38f2dcd.png"},{"id":17142676,"identity":"40e83d63-4ac7-404b-9532-49b044873cdb","added_by":"auto","created_at":"2022-01-09 20:21:44","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":621131,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-875843/v1/ca09a42e-2603-459c-be21-55c2da1d55ca.pdf"},{"id":13188120,"identity":"1e9deaf3-de86-45f9-b0dd-f0ef84e39b21","added_by":"auto","created_at":"2021-09-08 17:06:30","extension":"pdf","order_by":7,"title":"","display":"","copyAsset":false,"role":"supplement","size":510250,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementarymaterial.pdf","url":"https://assets-eu.researchsquare.com/files/rs-875843/v1/7e99eb0cc2053f0e6e75db32.pdf"}],"financialInterests":"","formattedTitle":"Comparison of radiomic pre-processing steps in the reproducible prediction of disease free survival across multi-scanners/centers","fulltext":[{"header":"Background","content":"\u003cp\u003eRadiomics consists of characterizing tumour phenotypes via the extraction of high-dimensional quantitative features from medical images, with the aim of supporting clinical decision-making [\u003cspan additionalcitationids=\"CR2\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. Radiomic features have been used to characterize cancer subtypes and aggressiveness or to predict the response to treatment [\u003cspan additionalcitationids=\"CR5 CR6 CR7\" citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e] and have increasingly been combined with machine learning (ML) techniques in order to predict a specific clinical outcome [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e][\u003cspan additionalcitationids=\"CR10 CR11 CR12\" citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe challenges facing the integration of radiomics into the clinics are many. Features reproducibility and the generalizability of the models are currently among the most important limitations. Even though standardization guidelines have helped to mitigate some of these reproducibility challenges [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e], radiomic features are sensitive to imaging acquisition protocols and reconstruction algorithms and parameters, but can also be affected by the different steps of the usual radiomics workflow [\u003cspan additionalcitationids=\"CR16 CR17 CR18\" citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. 2-[\u003csup\u003e18\u003c/sup\u003eF]fluoro-2-deoxy-D-glucose ([\u003csup\u003e18\u003c/sup\u003eF]FDG) positron emission tomography combined with computed tomography (PET/CT) imaging is especially prone to reproducibility issues due to frequent variations in pre-acquisition settings and scanner properties [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e], despite standardization efforts of acquisition and reconstruction protocols within the context of multicenter trials [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe intensity discretization scheme is one of the steps in radiomics workflow known to affect models reproducibility. Most of radiomic studies assume a specific discretization method based on the results of previous studies [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e][\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. Additionally, some studies used phantoms or internal data sets and evaluated the impact of some of the pre-processing steps using test-retest analysis [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e][\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eIn this study, we compared the effect of 6 [\u003csup\u003e18\u003c/sup\u003eF]FDG PET intensity discretization methods on the prediction of disease free survival (DFS) in locally advanced cervical cancer (LACC) patients from 3 different scanners/centers. Moreover, we evaluated the effect of combining these different discretization approaches with 7 distinct feature selection (FS) methods as well as with the effect of feature transformation using tumour to liver ratio (TLR) features as done in our previous work [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. In contrast to what was done in our previous work, we trained 84 distinct models using the data of one of the scanners and evaluated these models on the remaining two. We further evaluated these different steps using 4 different classifiers in order to better enhance the study robustness.\u003c/p\u003e"},{"header":"Materials And Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eData\u003c/h2\u003e \u003cp\u003eOne hundred and fifty-eight patients with LACC imaged between 2010 and 2016, in three different scanners were included in this retrospective study. PET/CT studies were performed in the CHU of Li\u0026egrave;ge, where 89 studies were acquired using a Philips Gemini TF or BB (scanner A), in the CHU of Brest and ICO St Herblain, where 17 and 34, respectively, were acquired using a Siemens Biograph mCT (scanner B) and at the McGill University Health Center, where 18 studies were performed with a General Electric Discovery ST (scanner C). The patient\u0026rsquo;s clinical characteristics, treatment, acquisition and reconstruction protocols are described in our previous study [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eExperimental design\u003c/h2\u003e \u003cp\u003eDFS was dichotomized into a binary endpoint, i.e. recurrence or no recurrence, independently of the time-to-event. Next we compared the prediction performance of 84 different models, which differed in i) the type of intensity discretization used before feature calculation, ii) the feature selection method iii) the features type, i.e., original radiomics (OR) or TLR radiomics.\u003c/p\u003e \u003cp\u003eModels were trained on the data from scanner A and then evaluated on data from two remaining external scanners independently. The metric used to evaluate model performance was the area under the curve of the precision recall curve (AUCpr) (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eStatistical and ML analyses were performed using R software, version 4.0.1.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eSegmentation and interpolation\u003c/h2\u003e \u003cp\u003ePET images from scanners B and C were interpolated using the research toolbox (Oncoradiomics SA, Li\u0026egrave;ge, Belgium), up-sampling or down-sampling the images using a linear method, so that all datasets had isotropic voxels of 4\u0026times;4\u0026times;4 mm\u003csup\u003e3\u003c/sup\u003e (i.e., the voxel size in images of scanner A). The 3D primary tumour volumes were segmented in the [\u003csup\u003e18\u003c/sup\u003eF]FDG PET images using the 2 classes semi-automatic Fuzzy Local Adaptive Bayesian algorithm [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]. Volumes of 20 cm\u003csup\u003e3\u003c/sup\u003e in the liver were manually drawn in order to investigate the predictive value of TLR radiomic features, as explained below. All segmentations were reviewed and edited if needed by one nuclear medicine physician with 9 years of experience in clinical PET/CT.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eImages Radiomic features\u003c/h2\u003e \u003cp\u003eWe extracted two hundred and fifteen features from the segmented volumes, which included first order grey level statistics, geometry, fractals, texture matrix based features and others. Features were extracted using the Oncoradiomics research toolbox and their detailed description can be found in supplementary data of our previous study [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. All features were calculated according to the Imaging biomarkers standardization initiative (IBSI) [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]. We also studied the ratio of the features values calculated in the tumour and in the liver the (TLR versions of features), except for the shape features as done in our previous study [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. We hypothesized that TLR features may reduce the variability of radioactive dose uptake within the different patients and across centers by normalizing the radiomic features using the liver which is an organ with an homogenous and reproducible uptake. There were no missing data for any patient.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eRadiomic features intensity discretization\u003c/h2\u003e \u003cp\u003eImage intensities were discretized using the two schemes currently standardized by the IBSI: fixed bin number (FBN, with 32 and 64 bins) and fixed bin width (FBW, with 4 different widths of 0.05, 0.1, 0.2 and 0.5 Standardized Uptake Values) [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]. These two sets of features were considered either alone or by: 1) joining the features from all discretization\u0026rsquo;s widths/bin\u0026rsquo;s number (D_All_FBW, D_All_FBN), 2) combining the four discretization\u0026rsquo;s widths from the FBW discretization method or combining the two number of bins from FBN through the calculation of their median value (D_Med_FBN, D_Med_FBW).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eFeatures selection, classifiers and model selection\u003c/h2\u003e \u003cp\u003eWe applied 7 different FS methods to identify the 5 most relevant features: 1-Accuracy decrease obtained from the embedded FS of the random forest (RF) classifier; 2- Gini impurity decrease obtained from the embedded FS of the RF classifier; 3- forward FS using maximum relevance minimum redundancy (MRMR) method with Pearson correlation; 4- backward FS using MRMR with Pearson correlation; 5- forward FS using MRMR with Spearman correlation; 6- backward FS using MRMR with Spearman correlation; 7- forward MRMR based on the mutual information (MI). We also considered 4 ML classifiers: RF, support vector machine (SVM) with radial kernel, Na\u0026iuml;ve Bayes (NB) and a logistic regression (LR) [\u003cspan additionalcitationids=\"CR27\" citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. We used for each classifier the default hyperparameters values in their respective R packages. We used 5-fold cross-validation in the training data to internally validate and select the models with better predictions for each classifier independently. Additionally, models were trained using all the training data then tested in the two external data sets.\u003c/p\u003e \u003cp\u003eA paired Wilcoxon Rank Sum test was used to test whether the predictions for each discretization scheme were statistically significantly different from each other in the two external validation schemes. Wilcoxon Rank Sum tests were considered significant if p\u0026thinsp;\u0026lt;\u0026thinsp;0.05. Holm-Bonferroni correction method was used to correct for multiple hypothesis testing.\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003eTable 1 depicts the mean AUCpr between the three validation schemes, i.e, i) Internal validation using 5-fold cross validation using scanner A ii) external validation using scanner B iii) external validation using scanner C. The table shows the mean AUCpr of the models using RF, SVM, LR and NB classifier using the different FS methods applied to the OR and TLR features. Additionally, the results shown for the standard FBW and FBN discretization schemes correspond to the model with discretization width/bin number that had a better AUCpr in the internal validation scheme. The AUCpr of the three validation schemes individually are shown in the supplementary material.\u003c/p\u003e\n\u003cp\u003eOur results showed a low reproducibility between scanners. The discretization scheme that showed the higher AUCpr in the validation scheme was D_Med_FBN combined with TLR features. This was not the case for the two external scanners. (Supplementary material).\u003c/p\u003e\u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eMEAN AUCPR BETWEEN VALIDATION SCHEMES\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"13\"\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFBW OR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFBN OR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eD_Med_FBW OR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eD_Med_FBN OR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eD_All_FBW OR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eD_All_FBN OR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eFBW TLR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c9\"\u003e \u003cp\u003eFBN TLR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c10\"\u003e \u003cp\u003eD_Med_FBW TLR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c11\"\u003e \u003cp\u003eD_Med_FBN TLR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c12\"\u003e \u003cp\u003eD_All_FBW TLR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c13\"\u003e \u003cp\u003eD_All_FBN TLR\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRF\u0026thinsp;+\u0026thinsp;MRMR_Forward_pearson\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.55\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRF\u0026thinsp;+\u0026thinsp;MRMR_Backward_pearson\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.39\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.37\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRF\u0026thinsp;+\u0026thinsp;MRMR_Forward_\u003c/p\u003e \u003cp\u003espearman\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRF\u0026thinsp;+\u0026thinsp;MRMR_Backward_\u003c/p\u003e \u003cp\u003espearman\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.39\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.39\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.55\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.40\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRF\u0026thinsp;+\u0026thinsp;MRMR_MI\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.40\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRF\u0026thinsp;+\u0026thinsp;RF_Accuracy\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.54\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.55\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.53\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.53\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.54\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.58\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRF\u0026thinsp;+\u0026thinsp;RF_Gini\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.53\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSVM\u0026thinsp;+\u0026thinsp;MRMR_\u003c/p\u003e \u003cp\u003eForward_pearson\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.60\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSVM\u0026thinsp;+\u0026thinsp;MRMR_\u003c/p\u003e \u003cp\u003eBackward_pearson\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.37\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.37\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.58\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.41\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSVM\u0026thinsp;+\u0026thinsp;MRMR_Forward_\u003c/p\u003e \u003cp\u003espearman\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.40\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.40\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.41\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSVM\u0026thinsp;+\u0026thinsp;MRMR_Backward_\u003c/p\u003e \u003cp\u003espearman\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.35\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.40\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.36\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.36\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSVM\u0026thinsp;+\u0026thinsp;MRMR_MI\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.39\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.33\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.37\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSVM\u0026thinsp;+\u0026thinsp;RF_Accuracy\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.39\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.56\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSVM\u0026thinsp;+\u0026thinsp;RF_Gini\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.35\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.55\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLR\u0026thinsp;+\u0026thinsp;MRMR_Forward_pearson\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.55\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.57\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLR\u0026thinsp;+\u0026thinsp;MRMR_\u003c/p\u003e \u003cp\u003eBackward_pearson\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.40\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLR\u0026thinsp;+\u0026thinsp;MRMR_Forward_\u003c/p\u003e \u003cp\u003espearman\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.34\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.34\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.39\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLR\u0026thinsp;+\u0026thinsp;MRMR_Backward_\u003c/p\u003e \u003cp\u003espearman\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.39\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.39\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.40\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLR\u0026thinsp;+\u0026thinsp;MRMR_MI\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.39\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.53\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLR\u0026thinsp;+\u0026thinsp;RF_Accuracy\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.53\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.53\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLR\u0026thinsp;+\u0026thinsp;RF_Gini\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.55\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.53\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNB\u0026thinsp;+\u0026thinsp;MRMR_Forward_pearson\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.55\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.58\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNB\u0026thinsp;+\u0026thinsp;MRMR_\u003c/p\u003e \u003cp\u003eBackward_pearson\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNB\u0026thinsp;+\u0026thinsp;MRMR_Forward_\u003c/p\u003e \u003cp\u003espearman\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.40\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.35\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.53\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNB\u0026thinsp;+\u0026thinsp;MRMR_Backward_spearman\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.37\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.50\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNB\u0026thinsp;+\u0026thinsp;MRMR_MI\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.53\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNB\u0026thinsp;+\u0026thinsp;RF_Accuracy\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.55\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.45\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNB\u0026thinsp;+\u0026thinsp;RF_Gini\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.54\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.54\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.55\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c11\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c12\"\u003e \u003cp\u003e0.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c13\"\u003e \u003cp\u003e0.49\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"13\"\u003eMean AUCpr between the three validation schemes using the four classifiers (RF, SVM, LR and NB) and the seven FS methods represented in the columns and with the different discretization schemes in the rows. The features discretization were applied to the OR and TLR features.\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eDespite the models low reproducibility, D_All_FBN with TLR features was the model with the better mean AUCpr for the LR and NB classifier (0.57 and 0.58 respectively). It was also the second model with overall higher AUCpr in the two independent scanners with AUCpr of 0.45 and 0.7 in scanner B and C respectively (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). For the RF classifier the model with higher AUCpr was D_Med_FBN TLR whereas for SVM it was FBW TLR. Regarding the FS method, when combined with the D_All_FBN TLR, MRMR Forward with Pearson correlation was the optimal FS method for at least one of the four classifiers in all validation schemes. MRMR Forward with Pearson correlation is also the FS method that showed better mean AUCpr for 4 out of the 6 discretization schemes when using TLR features. The discretization schemes that showed higher mean AUCpr across classifiers were D_Med_FBN, D_All_FBW and D_All_FBN. All of them were used with TLR features (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). Despite of the good performance of D_All_FBN with TLR, the only discretization methods that showed to be statistically significant from D_All_FBN TLR were FBW with OR features, and FBW, FBN, D_Med_FBW, D_Med_FBN, D_All_FBW with TLR features. FBW combined with OR features was the only discretization scheme statistically different from all the others (Table\u0026nbsp;2). TLR based models had higher mean AUCpr values than OR based models for all the classifiers and discretization schemes, except when using D_med_FBW with LR or FBW and D_Med_FBN with NB (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). TLR based models predictions in the two external validation schemes were statistically significant from all the OR based models (p-value\u0026thinsp;\u0026lt;\u0026thinsp;\u0026lt;\u0026thinsp;0.05).\u003c/p\u003e\u003cp\u003e \u003cspan type=\"SmallCaps\" class=\"SmallCaps\" name=\"Emphasis\"\u003eTable\u0026nbsp;2\u003c/span\u003e \u003c/p\u003e \u003cp\u003e \u003cspan type=\"SmallCaps\" class=\"SmallCaps\" name=\"Emphasis\"\u003eWilcoxon rank sum test corrected p-values showing the statistical significance between discretization schemes\u003c/span\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"No\" id=\"Taba\" border=\"1\"\u003e \u003ccolgroup cols=\"14\"\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFBW_OR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFBN_\u003c/p\u003e \u003cp\u003eOR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eD_Med_\u003c/p\u003e \u003cp\u003eFBW_OR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003eD_Med_\u003c/p\u003e \u003cp\u003eFBN_OR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eD_All_\u003c/p\u003e \u003cp\u003eFBW_OR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eD_All_\u003c/p\u003e \u003cp\u003eFBN_OR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c9\"\u003e \u003cp\u003eFBW_TLR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c10\"\u003e \u003cp\u003eFBN_TLR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c11\"\u003e \u003cp\u003eD_Med_\u003c/p\u003e \u003cp\u003eFBW_TLR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c12\"\u003e \u003cp\u003eD_Med_\u003c/p\u003e \u003cp\u003eFBN_TLR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c13\"\u003e \u003cp\u003eD_All_FBW_TLR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c14\"\u003e \u003cp\u003eD_All_FBN_TLR\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFBW_OR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFBN_OR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2.11E-11\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eD_Med_\u003c/p\u003e \u003cp\u003eFBW_OR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5.22E-07\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.48E\u0026thinsp;+\u0026thinsp;01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eD_Med_\u003c/p\u003e \u003cp\u003eFBN_OR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5.43E-13\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4.30E\u0026thinsp;+\u0026thinsp;00\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003e2.95E-03\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eD_All_\u003c/p\u003e \u003cp\u003eFBW_OR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5.63E-09\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4.59E\u0026thinsp;+\u0026thinsp;01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003e7.51E\u0026thinsp;+\u0026thinsp;00\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3.30E\u0026thinsp;+\u0026thinsp;01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eD_All_\u003c/p\u003e \u003cp\u003eFBN_OR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2.07E-04\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.87E\u0026thinsp;+\u0026thinsp;02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003e1.86E\u0026thinsp;+\u0026thinsp;01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e6.37E-05\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1.26E-01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFBW_TLR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e8.77E-50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.24E-14\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003e2.01E-37\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e1.32E-11\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1.93E-18\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1.60E-25\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFBN_TLR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4.41E-25\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7.47E-03\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003e3.71E-15\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e9.63E-03\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e2.87E-08\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e2.26E-15\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e9.22E-07\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eD_Med_\u003c/p\u003e \u003cp\u003eFBW_TLR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3.33E-44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5.81E-10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003e1.61E-28\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e1.62E-06\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1.58E-15\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e2.15E-21\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e5.18E-02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e2.05E-01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eD_Med_\u003c/p\u003e \u003cp\u003eFBN_TLR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2.70E-40\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7.49E-24\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003e8.82E-29\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e4.52E-18\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1.37E-16\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e4.89E-36\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e6.43E\u0026thinsp;+\u0026thinsp;01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e1.41E-09\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e \u003cp\u003e6.97E\u0026thinsp;+\u0026thinsp;00\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eD_All_\u003c/p\u003e \u003cp\u003eFBW_TLR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.83E-25\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.94E-13\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003e4.28E-16\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e1.18E-12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1.16E-12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1.25E-13\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e4.97E\u0026thinsp;+\u0026thinsp;00\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e5.07E-03\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e \u003cp\u003e5.33E\u0026thinsp;+\u0026thinsp;01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e \u003cp\u003e1.71E\u0026thinsp;+\u0026thinsp;00\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eD_All_\u003c/p\u003e \u003cp\u003eFBN_TLR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.51E-04\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e6.42E\u0026thinsp;+\u0026thinsp;01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003e7.73E\u0026thinsp;+\u0026thinsp;00\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3.18E\u0026thinsp;+\u0026thinsp;01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e3.18E\u0026thinsp;+\u0026thinsp;01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e2.18E\u0026thinsp;+\u0026thinsp;01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e2.60E-23\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e2.26E-09\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e \u003cp\u003e8.29E-16\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e \u003cp\u003e4.13E-24\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e \u003cp\u003e4.20E-10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eRadiomics aims at converting data from medical images into quantitative features providing valuable information regarding the clinical management of patients. These features can be combined with statistical/ML methods in order to derive predictive models of clinically relevant endpoints. The radiomics workflow consists however of multiple pre-processing steps that can affect the radiomic features values and therefore their clinical relevance. In this study we compared the prediction performance of numerous models that differed in their discretization and FS methods. This comparison was carried out within the context of predicting DFS from [\u003csup\u003e18\u003c/sup\u003eF]FDG PET images in a multi-scanner/center [\u003csup\u003e18\u003c/sup\u003eF]FDG PET cohort of LACC patients. Additionally, we investigated the effect of features transformation using features ratios with an organ of reference and we combined it with the previous pre-processing steps. We also compared the models performances using four classifiers. Multi-classifier radiomics predictive models, ensemble classifiers or the combination of different classifiers performances to measure feature importance, consistently tend to outperform traditional single classifier approaches [\u003cspan additionalcitationids=\"CR30\" citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e]. Moreover, the choice of classification method is one the most dominant sources of performance variation in radiomics studies [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. Due to this, we believe that comparing the results of multiple classifiers is needed when evaluating the robustness of the workflow. The discretization scheme is one of the factors that affect radiomic features reproducibility. FBW discretization in PET has been recommended [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e][\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e], although some studies have also reported more favourable properties using FBN [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e]. This is related to the fact that FBN and FBW have different drawbacks and advantages. FBW preserves the relationship between PET units and the corresponding physical substrate, contrary to arbitrary units (such as in some non-quantitative magnetic resonance imaging sequences). FBN on the other hand does not preserve such relationship but introduces a normalization effect that can be favourable when contrast is considered important or when the actual original image intensity value does not have a \u0026lsquo;meaning\u0026rsquo;. In our study D_All_FBN and FBW both combined with TLR features were the discretization scheme that showed the best AUCpr in the two external scanners. Combining features discretized with different widths/bin numbers can as shown in our study introduce complementary information and be a more reproducible and simple strategy as it also avoids the uncertain assumption or the extensive search of the optimal feature discretization width/bin number. Combining feature discretization schemes has also been done in previous studies [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e]. Furthermore, our results show that when using D_All_FBN with TLR the FS scheme with higher mean AUCpr in the three validation schemes is MRMR Forward with Pearson correlation for 2 of the 4 classifiers. FS is an effective strategy to improve radiomics-based predictive studies. Different FS strategies are used in radiomics studies, each with his pros and cons, and some known to work better with certain type of features or classifiers [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eFinally, we also evaluated the feature type, i.e. OR and TLR radiomics. We have shown in our previous study that using the ratio of the tumour features with a reference organ (TLR radiomics) improves the predictive performance of radiomics model in LACC. In contrast to our previous study, we trained our models using only data from one clinical center/scanner and evaluated our models in two external scanners. We emphasize the conclusions of our previous study, by observing that all of the most robust models used TLR features instead of OR. This can be caused by a normalizing effect of the SUVs on each patient. The importance of data normalization/transformation has also been accessed for other radiomic studies and shown to improve models performances [\u003cspan additionalcitationids=\"CR37\" citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e]. Moreover, feature transformation using an organ of reference has also been investigated by other authors, leading to normalized images and increased reproducibility of radiomic features [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe results of our study are encouraging and can potentially be used as a first recommendation approach to improve reproducibility of radiomics studies across multi-scanners/centers in LACC. However, it corresponds to a preliminary study and we still observed performance differences between the 3 scanners: for some scanners some models work quite well while for other scanners other models work better. This could be caused by the variation in scanner properties, the different number of patients in each scanner, or the variation in tumour recurrence rate for each population.\u003c/p\u003e \u003cp\u003eIn future work, the tumour segmentations should be done fully automatically using a recently validated approach [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e], instead of being done by a single observer in a semi-automated way, since segmentation variations within patients can affect radiomics reproducibility [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. Within this work, we also did not try combining all the features from the two discretization schemes, mainly due to the resulting need for longer computation times. Combining discretization methods, exploring more classifiers, combining the information from PET with other image modalities and explore image or matrix fusion strategies as done successfully before by other authors [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e] will be investigated in future work. Our findings should also be validated using a larger data set and within the context of other pathologies.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eIn the present paper, we proposed a framework for comparing different pre-processing PET strategies using a multi-center series with the intent of predicting DFS in LACC patients. Our results show that there is a low reproducibility in predictions across scanners and discretization methods. Combining features calculated with different numbers of bins, relying on the normalizing effect of tumour to liver ratio, and using the maximum relevance minimum redundancy feature selection method was found to increase the robustness of the developed models and could be recommended for future radiomic studies in a similar context. These recommendations should now be evaluated in larger cohorts and in different cancer types.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eAUCpr: area under the curve of the precision recall curve\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eCT: computed tomography\u003c/p\u003e\n\u003cp\u003eDFS: disease free survival\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eD_All_FBW: discretization using all widths of the fixed bin width discretization\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eD_All_FBN: discretization using all bin\u0026rsquo;s number of the fixed bin number discretization\u003c/p\u003e\n\u003cp\u003eD_Med_FBW:\u0026nbsp;discretization using the median value of features with the different widths used in the fixed bin width discretization\u003c/p\u003e\n\u003cp\u003eD_Med_FBN: discretization using the median value of features with the different bins used in the fixed bin number discretization\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;[\u003csup\u003e18\u003c/sup\u003eF]FDG: 2-[18F]fluoro-2-deoxy-D-glucose\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFBW: fixed bin width\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFBN: fixed bin number\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFS: feature selection\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIBSI: Imaging biomarkers standardization initiative\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eLACC: locally advanced cervical cancer\u003c/p\u003e\n\u003cp\u003eLR: logistic regression\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eMI: mutual information\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eML: machine learning\u003c/p\u003e\n\u003cp\u003eNB: Na\u0026iuml;ve Bayes\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eMRMR: maximum relevance minimum redundancy\u003c/p\u003e\n\u003cp\u003eOR: original radiomics\u0026nbsp;\u003c/p\u003e\n\u003cp\u003ePET: positron emission tomography\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eRF: random forest\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eSVM: support vector machine\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTLR: tumour to liver ratio\u0026nbsp;\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis project has received funding from the European Union\u0026rsquo;s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 766276. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eDr Philippe Lambin reports, within and outside the submitted work, grants/sponsored research agreements from Varian medical, Oncoradiomics, ptTheragnostic/DNAmito, Health Innovation Ventures. He received an advisor/presenter fee and/or reimbursement of travel costs/external grant writing fee and/or in kind manpower contribution from Oncoradiomics, BHV, Merck, Varian, Elekta, ptTheragnostic and Convert pharmaceuticals. Dr Lambin has shares in the company Oncoradiomics, Convert pharmaceuticals, MedC2 and LivingMed Biotech, he is co-inventor of two issued patents with royalties on radiomics (PCT/NL2014/050248, PCT/NL2014/050728) licensed to Oncoradiomics and one issue patent on mtDNA (PCT/EP2014/059089) licensed to ptTheragnostic/DNAmito, three non-patented invention (softwares) licensed to ptTheragnostic/DNAmito, Oncoradiomics and Health Innovation Ventures and three non-issues, non licensed patents on Deep Learning-Radiomics and LSRT (N2024482, N2024889, N2024889). He confirms that none of the above entities or funding was involved in the preparation of this paper.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthical approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll procedures were performed in accordance with the principles of the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. The study design and exemption from informed consent were approved by the Institutional Review Board of Liege University Hospital.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFor this type of retrospective study formal consent is not required.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe datasets generated and/or analysed during the current study are available in\u003ca href=\"https://github.com/msilvaferreira/Phd/tree/master/FDG%20PET%20radiomics%20to%20predict%20disease%20free%20survival%20in%20Cervical%20Cancer\"\u003ehttps://github.com/msilvaferreira/Phd/tree/master/FDG%20PET%20radiomics%20to%20predict%20disease%20free%20survival%20in%20Cervical%20Cancer\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos; contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eStudy concepts/study design: Marta Ferreira, Patrick E.Meyer and Roland Hustinx\u003c/p\u003e\n\u003cp\u003eLiterature research: Marta Ferreira and Patrick E.Meyer\u003c/p\u003e\n\u003cp\u003eClinical studies: Johanne Hermesse, Marjolein Decuypere, Philippe Robin and Fr\u0026eacute;d\u0026eacute;ric Kridelka\u003c/p\u003e\n\u003cp\u003eData analysis: Marta Ferreira, Patrick E.Meyer, Pierre Lovinfosse and Roland Hustinx\u003c/p\u003e\n\u003cp\u003eStatistical analysis: Marta Ferreira and Patrick E.Meyer\u003c/p\u003e\n\u003cp\u003eManuscript drafting, editing and revision: All authors read and approved the final manuscript\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eLambin P, et al. Radiomics: Extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48:441\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAerts HJWL, et al., \u0026ldquo;Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach,\u0026rdquo; Nat. Commun., vol.\u0026nbsp;5, 2014.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLambin P, et al. Radiomics: The bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14:749\u0026ndash;62.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTsujikawa T, et al., \u0026ldquo;18F-FDG PET radiomics approaches: comparing and clustering features in cervical cancer,\u0026rdquo; Ann. Nucl. Med., vol.\u0026nbsp;31, 2017.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAltazi BA, et al. Investigating multi-radiomic models for enhancing prediction power of cervical cancer treatment outcomes. Phys Medica. 2018;46:180\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHao H, et al., \u0026ldquo;Shell feature: A new radiomics descriptor for predicting distant failure after radiotherapy in non-small cell lung cancer and cervix cancer,\u0026rdquo; Phys. Med. Biol., vol.\u0026nbsp;63, 2018.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLucia F, et al. Prediction of outcome using pretreatment 18 F-FDG PET/CT and MRI radiomics in locally advanced cervical cancer treated with chemoradiotherapy. Eur J Nucl Med Mol Imaging. 2018;45:768\u0026ndash;86.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBowen SR, et al. Tumor radiomic heterogeneity: Multiparametric functional imaging to characterize variability and predict response following cervical cancer radiation therapy. J Magn Reson Imaging. 2017;47:1388\u0026ndash;96.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShen WC, et al. Prediction of local relapse and distant metastasis in patients with definitive chemoradiotherapy-treated cervical cancer by deep learning from [18F]-fluorodeoxyglucose positron emission tomography/computed tomography. Eur Radiol. 2019;29:6741\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSun W, Jiang M, Dang J, Chang P, Yin FF. \u0026ldquo;Effect of machine learning methods on predicting NSCLC overall survival time based on Radiomics analysis,\u0026rdquo; Radiat. Oncol., 2018.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLeger S, Zwanenburg A, Pilz K, Lohaus F. \u0026ldquo;A comparative study of machine learning methods for time-to-event survival data for radiomics risk modelling\u0026rdquo;, pp.\u0026nbsp;1\u0026ndash;11, 2017.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eParmar C, Grossmann P, Bussink J, Lambin P, Aerts HJWL. \u0026ldquo;Machine Learning methods for Quantitative Radiomic Biomarkers,\u0026rdquo; Sci. Rep., vol.\u0026nbsp;5, 2015.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDeist TM, et al., \u0026ldquo;Machine learning algorithms for outcome prediction in (chemo)radiotherapy: An empirical comparison of classifiers,\u0026rdquo; Med. Phys., 2018.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLei M, et al., \u0026ldquo;Benchmarking features from different radiomics toolkits / toolboxes using Image Biomarkers Standardization Initiative,\u0026rdquo; arXiv, 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLovinfosse P, Visvikis D, Hustinx R, Hatt M. \u0026ldquo;FDG PET radiomics: a review of the methodological aspects,\u0026rdquo; pp.\u0026nbsp;379\u0026ndash;391, 2018.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShiri I, Rahmim A, Ghaffarian P, Geramifar P, Abdollahi H, Bitarafan-rajabi A. The impact of image reconstruction settings on 18F-FDG PET radiomic features: multi-scanner phantom and patient studies Lesions to Background Ratio Time of Flight Full Width at Half Maximum Response Evaluation Criteria in Solid Tumours. Eur Radiol. 2017;27:4498\u0026ndash;509.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShafiq-ul-hassan M, et al. Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels. \u0026rdquo; Med Phys. 2017;44:1050\u0026ndash;62.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLeijenaar RTH, et al., \u0026ldquo;The effect of SUV discretization in quantitative FDG-PET Radiomics: the need for standardized methodology in tumor texture analysis,\u0026rdquo; Nat. Publ. Gr., pp.\u0026nbsp;1\u0026ndash;10, 2015.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAltazi BA, et al. Reproducibility of F18-FDG PET radiomic features for different cervical tumor segmentation methods, gray-level discretization, and reconstruction algorithms. J Appl Clin Med Phys. 2017;18:32\u0026ndash;48.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiberini V, et al., \u0026ldquo;Impact of segmentation and discretization on radiomic features in 68Ga-DOTA-TOC PET/CT images of neuroendocrine tumor,\u0026rdquo; EJNMMI Phys., vol.\u0026nbsp;8, 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAide N, Lasnon C, Veit-Haibach P, Sera T, Sattler B, Boellaard R. EANM/EARL harmonization strategies in PET quantification: from daily practice to multicentre oncological studies. Eur J Nucl Med Mol Imaging. 2017;44:17\u0026ndash;31.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSchwier M, et al., \u0026ldquo;Repeatability of multiparametric prostate MRI radiomics features,\u0026rdquo; Sci. Rep., 2019.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFerreira M, et al., \u0026ldquo;[18F]FDG PET radiomics to predict disease-free survival in cervical cancer: a multi-scanner/center study with external validation,\u0026rdquo; Eur. J. Nucl. Med. Mol. Imaging, 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHatt M, Cheze C, Turzo A, Roux C. A fuzzy locally adaptive Bayesian segmentation approach for volume determination in PET. \u0026rdquo; IEEE Trans Med Imaging. 2009;28:881\u0026ndash;93.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZwanenburg A, et al. The image biomarker standardization initiative: Standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. 2020;295:328\u0026ndash;38.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHanchuan Peng F. Long, \u0026ldquo;Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy. IEEE Trans Pattern Anal Mach Intell. 2005;27:1226\u0026ndash;38.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBreiman L, Friedman J, Olshen R, Classification and Regression Trees. 1984.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eG\u0026eacute;ron A. Hands-On Machine Learning with Scikit-Learn. 1st ed.: O\u0026rsquo;Reilly; 2017.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhou Z, et al., \u0026ldquo;Constructing multi-modality and multi-classifier radiomics predictive models through reliable classifier fusion,\u0026rdquo; IEEE Comput. Soc., 2017.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhou Z, et al. \u0026ldquo;Multifactorial cancer treatment outcome prediction through multifaceted radiomics. \u0026rdquo;, arXiv: Medical Physics; 2018.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOsman AFI. A Multi-parametric MRI-Based Radiomics Signature and a Practical ML Model for Stratifying Glioblastoma Patients Based on Survival Toward Precision Oncology. Front Comput Neurosci. 2019;13:1\u0026ndash;15.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePresotto L, et al. Physica Medica Original paper PET textural features stability and pattern discrimination power for radiomics analysis: An \u0026lsquo; ad-hoc \u0026rsquo; phantoms study. Phys Medica. 2018;50:66\u0026ndash;74.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eValli\u0026egrave;res M, et al. Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. Sci Rep. 2017;7:1\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLv W, Ashrafinia S, Ma J, Lu L, Rahmim A. Multi-level multi-modality fusion radiomics: Application to PET and CT imaging for prognostication of head and neck cancer. IEEE J Biomed Heal Informatics. 2020;24:2268\u0026ndash;77.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eN. S.-M. \u0026amp; A. A.-B. Ver\u0026oacute;nica Bol\u0026oacute;n-Canedo, Feature Selection for High-Dimensional Data. Springer, 2016.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eScalco E, et al. T2w-MRI signal normalization affects radiomics features reproducibility. Med Phys. 2020;47:1680\u0026ndash;91.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHaga A, et al. Standardization of imaging features for radiomics analysis. J Med Investig. 2019;66:35\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi XT, Huang RY. Standardization of imaging methods for machine learning in neuro-oncology. Neuro-Oncology Adv. 2020;2:iv49\u0026ndash;55.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIsaksson LJ, et al. Effects of MRI image normalization techniques in prostate cancer radiomics. Phys Medica. 2020;71:7\u0026ndash;13.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTraverso A, et al. Sensitivity of radiomic features to inter-observer variability and image pre-processing in Apparent Diffusion Coefficient (ADC) maps of cervix cancer patients. Radiother Oncol. 2020;143:88\u0026ndash;94.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIantsen A, et al., \u0026ldquo;Convolutional neural networks for PET functional volume fully automatic segmentation: development and validation in a multi-center setting,\u0026rdquo; Eur. J. Nucl. Med. Mol. Imaging, 2021.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Radiomics, Intensity discretization, Pre-processing radiomics steps","lastPublishedDoi":"10.21203/rs.3.rs-875843/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-875843/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eFeatures reproducibility and the generalizability of the models are currently among the most important limitations when integrating radiomics into the clinics. Radiomic features are sensitive to imaging acquisition protocols, reconstruction algorithms and parameters, as well as by the different steps of the usual radiomics workflow. We propose a framework for comparing the reproducibility of different pre-processing steps in PET/CT radiomic analysis in the prediction of disease free survival (DFS) across multi-scanners/centers.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eWe evaluated and compared the prediction performance of several models that differ in i) the type of intensity discretization, ii) feature selection method, iii) features type i.e, original or tumour to liver ratio radiomic features (OR or TLR). We trained our models using data from one scanner/center and tested on two external scanner/centers. Our results show that there is a low reproducibility in predictions across scanners and discretization methods. Despite of this, TLR based models were generally more robust than OR. Maximum relevance minimum redundancy (MRMR) forward feature selection with Pearson correlation was the feature selection method that had the best mean area under the precision recall curve when using it combining the features from all discretization\u0026rsquo;s bin\u0026rsquo;s number (D_All_FBN) with TLR features for two of the four classifiers.\u003c/p\u003e\u003ch2\u003eConclusion\u003c/h2\u003e \u003cp\u003e We evaluated and compared the prediction performance of several models in a data set containing hundred fifty-eight patients with locally advanced cervical cancer (LACC) from three distinct scanners. In our cohort of LAAC patients pre-processing of radiomic features in [\u003csup\u003e18\u003c/sup\u003eF]FDG PET affects DFS predictions performances across scanners and combining the D_All_FBN TLR approach with the MRMR forward Pearson feature selection method might help increasing robustness of radiomic studies.\u003c/p\u003e","manuscriptTitle":"Comparison of radiomic pre-processing steps in the reproducible prediction of disease free survival across multi-scanners/centers","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2021-09-08 17:06:28","doi":"10.21203/rs.3.rs-875843/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e1cac04b-7a5f-4007-a071-ef5ccee44cdc","owner":[],"postedDate":"September 8th, 2021","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":7033981,"name":"Bioinformatics"}],"tags":[],"updatedAt":"2022-01-09T20:21:39+00:00","versionOfRecord":[],"versionCreatedAt":"2021-09-08 17:06:28","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-875843","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-875843","identity":"rs-875843","version":["v1"]},"buildId":"7rjqhiLT3MXkJMwkYKINL","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}