Explainable Machine Learning for Knee Osteoarthritis Diagnosis Based on a Novel Fuzzy Feature Selection Methodology

Explainable Machine Learning for Knee Osteoarthritis Diagnosis Based on a Novel Fuzzy Feature Selection Methodology | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Explainable Machine Learning for Knee Osteoarthritis Diagnosis Based on a Novel Fuzzy Feature Selection Methodology Christos Kokkotis, Charis Ntakolia, Serafeim Moustakidis, Giannis Giakas, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-777000/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 31 Jan, 2022 Read the published version in Physical and Engineering Sciences in Medicine → Version 1 posted 5 You are reading this latest preprint version Abstract Knee Osteoarthritis (ΚΟΑ) is a degenerative joint disease of the knee that results from the progressive loss of cartilage. Due to KOA’s multifactorial nature and the poor understanding of its pathophysiology, there is a need for reliable tools that will reduce diagnostic errors made by clinicians. The existence of public databases has facilitated the advent of advanced analytics in KOA research however the heterogeneity of the available data along with the observed high feature dimensionality make this diagnosis task difficult. The objective of the present study is to provide a robust Feature Selection (FS) methodology that could: (i) handle the multidimensional nature of the available datasets and (ii) alleviate the defectiveness of existing feature selection techniques towards the identification of important risk factors which contribute to KOA diagnosis. For this aim, we used multidisciplinary data obtained from the Osteoarthritis Initiative database for individuals without or with KOA. The proposed fuzzy ensemble feature selection methodology aggregates the results of several FS algorithms (filter, wrapper and embedded ones) based on fuzzy logic. The effectiveness of the proposed methodology was evaluated using an extensive experimental setup that involved multiple competing FS algorithms and several well-known ML models. A 73.55 % classification accuracy was achieved by the best performing model (Random Forest classifier) on a group of twenty-one selected risk factors. Explainability analysis was finally performed to quantify the impact of the selected features on the model’s output thus enhancing our understanding of the rationale behind the decision-making mechanism of the best model. Biomedical Engineering Biotechnology and Bioengineering Nuclear Medicine & Medical Imaging KOA Diagnosis Machine Learning Clinical data Explainability Feature selection Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Introduction Knee Osteoarthritis (KOA) is one of the most common types of osteoarthritis and musculoskeletal disorder. Being the 11th highest global disability, KOA is a multifactorial disease that results from mechanical and constitutional factors [ 1 ]. Obesity, age, gender, knee injuries and lifestyle are likely risk factors of KOA as they have been highlighted in the relevant recent literature [ 2 ]. In addition, swelling, pain and stiffness have been characterized as typical symptoms of the disease with irreversible cartilage damage being KOA’s main consequence [ 3 – 5 ]. KOA is closely associated with a huge economic burden for the healthcare system and an unbearable health burden of the patients and their families. Significant consequences of KOA are the social isolation of the individual and the low quality of life [ 6 , 7 ]. Furthermore, the quantification of KOA is performed with the Kellgren–Lawrence (KL) severity grading scale, which is the most commonly grading system (current gold standard) and consists of five severity grades, from 0 to 4 [ 8 ]. Despite the fact that the scientific community has put a lot of effort into KOA research, a major challenge remains with respect to early diagnosis, long-term diagnosis and treatment of ΚΟΑ. The parallel increase in computing power along with the collection of big datasets combined with the need to address the above challenges has led many research teams to use artificial intelligence (AI) techniques in the field of KOA [ 9 ]. In light of the above, several AI enabled studies have been proposed in the recent literature with the objective to diagnose or predict KOA. Yoo et al. used data from the Fifth Korea National Health and Nutrition Examination Surveys (KNHANES V-1) and the Osteoarthritis Initiative (OAI) to build an artificial neural network (ANN)-based a scoring system for the identification of KOA severity [ 10 ]. The proposed ANN model achieved an area under the curve (AUC) of 0.76 for the symptomatic KOA in an external validation with OAI data. In another study, Lim et al. proposed a method for early diagnosis of KOA based on clinical data from Korean National Health and Nutrition Examination Survey (KNHANES) [ 11 ]. They achieved an 76.8% AUC by using a deep neural network with scaled principal component analysis. In 2019, Christodoulou et al. investigated the deep learning capabilities in KOA diagnosis [ 12 ]. They used clinical data from OAI database and they achieved an 86.95% accuracy working on an aged subgroup (70 +). In another study, Moustakidis et al. worked on self-reported clinical data (OAI) and proposed a deep learning methodology for the recognition of participants being at high risk of developing KOA in at least one knee and participants with symptomatic KOA [ 13 ]. They achieved accuracies up to 86.95%. Furthermore, Kwon et al. proposed an automatic classification of KOA severity that made use of gait analysis data and radiographic imaging (from Seoul National University Hospital) [ 14 ]). They employed Inception-ResNet-v2 for feature extraction from X-rays and a support vector machine for KOA diagnosis achieving accuracies of 0.93, 0.82, 0.83, 0.88, and 0.97 for the KL grades 0–4, respectively. In addition, Moustakidis et al. proposed a KOA classification approach with a focus on both accuracy and fairness [ 15 ]. They worked on different subgroups of participants from self-reported clinical data (OAI) and the dense neural networks methodology improved the accuracy up to 79.6% with fairness measured by balanced equalized odds (~ 92%) and demographic parity (98.5%) in the KOA case study. Given that medical data and features can be subjective or difficult to interpret, medical decision making has a great potential to benefit by the use of fuzzy logic (FL). FL has been used to diagnose or facilitate decision making systems tackling many diseases, including OA. Hardi et al. proposed an expert system based on the fuzzy Tsukamoto method for OA diagnosis [ 16 ]. They treated symptoms of OA as fuzzy values that were further converted into firm value by using a weighted average demonstrating a 90% accuracy in the task of diagnosis of osteoarthritis disease. In general, various feature selection methods have integrated fuzzy logic in their internal mechanisms in order to handle the observed fuzziness and therefore improve the way that features are treated and combined. For instance, with emphasis to medical applications, mutual information method combined with FL was used: (i) to select miRNAs in cancer [ 17 ]; (ii) to classify tumors [ 18 ]; and to select features for multilabel learning [ 19 ]. Similar studies include fuzzy entropy by using thresholds [ 20 ] for feature selection in various medical datasets and fuzzy rough sets [ 21 , 22 ] for dimensionality reduction of feature space to prevent samples from misclassification. It is well known that each one of the existing FS algorithms comes with its own advantages and disadvantages introducing a certain level of bias. To handle the multidimensional nature of the OAI dataset and to avoid bias and alleviate the defectiveness of single feature selection results, a fuzzy ensemble FS methodology is proposed in this paper that aggregates the results of several FS algorithms (filter, wrapper and embedded ones). Fuzzy logic is employed to combine multiple feature importance scores thus leading to a more robust selection of informative features. The proposed method contributes to the significant reduction of the initial OAI feature dimensionality and to a decrease of the computational complexity of the classification models employed. To prove the effectiveness of the proposed methodology, an extensive experimental setup was designed involving multiple competing FS algorithms and several well-known ML models. As a post-hoc explainability, SHAP model was finally employed to identify the contribution of the selected features and the rationale behind the decision-making mechanism of best performing model. Methods Dataset Description For the purpose of this study, data were employed from the osteoarthritis initiative (OAI) database (available on https://nda.nih.gov/oai/ ). OAI is a prospective observational, multi-center and longitudinal study of KOA. OAI has enrolled 4796 women and men, aged 45-79 years. The present study used clinical evaluation data (643 features in total) from the baseline visit from all participants with or without KOA. The features of clinical dataset were divided into seven categories as shown in Table 1. Furthermore, in the present study, Kellgren and Lawrence (KL) grades were used as the outcome for the classification task. Table 1 . Main categories of the clinical evaluation data considered in this study. Category Description Medical history Medications and health histories based on questionnaire results (not included medical imaging outcomes) Symptoms Arthritis symptoms or health-related disability and function based on questionnaire data Subject characteristics Includes variables which describe anthropometric parameters and personal information Nutrition Questionnaire based on Block Food Frequency Physical exam Includes performance measures and knee and hand exams Physical activity Questionnaire results regarding living and leisure activities Behavioral Consists of variables which quantify the social behavior and the quality level of daily routine Methodology The proposed AI methodology for KOA diagnosis consists of five processing steps: i) data pre-processing, ii) application of FS techniques, iii) learning process, iv) evaluation of the classification results and v) explainability analysis, as illustrated in Figure 1. An extensive presentation of the steps of the proposed methodology is given in the following subsections. Problem Definition In this study, we defined the KL-grade prediction task as a binary class classification problem. Specifically, the subjects of the study (3872 subjects in total) were divided into two equal groups: i) KOA - participants who have KL >=2 at baseline. Participant in the group had KL grades equal (early diagnosis) or higher than 2 in at least one of the two knees or in both at baseline; ii) non-KOA - participants who had KL0 or KL1 grade at baseline. Especially, this group of participants do not have ΚΟA in any of their knees. Data Pre-processing Mode imputation was employed to handle categorical and continuous missing values [23]. In our study, data were normalised to [0, 1] to build a common basis for the FS algorithms and learning techniques that follow [24]. Furthermore, to cope with the imbalance data problem a stratified strategy for data resampling was applied. In particular, the number of the subjects in the majority class was reduced in order to become equal with the number of samples on the minority class [25]. Proposed FS methodology The proposed Fuzzy logic enhanced Feature Selection method (FSFL) combines the outputs of six well-known feature selection methods from three feature selection categories (Filter, Wrapper and Embedded). Specifically, from the filter category, the mutual information [26] and the f-ANOVA [27] techniques were applied. From the wrapper category, we employed a recursive feature elimination (RFE) based on logistic regression [28] and an RFE based on support vector machine [29] techniques, respectively. Furthermore, from the embedded category, a LightGBM [30] and a random forest technique [31] were applied. To calculate the importance of a feature for each category, the scores of the associated FS techniques were used as input to the Fuzzy Inference System (FIS) 1 that was implemented with Mamdani inference methodology [32]. The output of the FIS 1 was the defuzzification value that represents the feature importance score for the specific feature selection category. Then, the defuzzification score of each category was used as input to the FIS 2 where the output defuzzification value represents the overall feature importance. Figure 2 illustrates the FSFL flowchart with the defined fuzzy rules for each FIS and the selected feature selection methods for this study. Figure 3 shows the fuzzy sets used in the presented methodology for the input variables for FIS 1 and FIS 2, while Figure 4 shows the fuzzy sets of output variable for FIS 1 and 2. Learning In order to handle the demanding task of KOA classification, we investigated various ML models for their suitability and behavior in this problem. Specifically, random forest (RF) [33], multilayer perceptron (MLP) [34], logistic regression (LR) [35], support-vector machines (SVMs) [36], and k-nearest neighbors (KNN) [37] classifiers were tested. Furthermore, to avoid overfitting, and to optimize the performance of our models hyperparameter selection was applied individually per model. Validation For the experimental evaluation, a repeated stratified 5-fold cross validation was used [38]. Furthermore, the performance of the classifiers was also evaluated in terms of the recall, f1-score and precision as additional evaluation criteria [39]. A brief description of these metrics is given below. Initially, the accuracy is the ratio of correctly predicted observations to the total observations and can be characterized as the most intuitive performance measure. Recall (or Sensitivity) is the ratio of correctly predicted positive observations to all observations in the actual class. Moreover, the ratio of correctly predicted positive observations to the total predicted positive observations is called precision or positive predictive value. F1-score is the weighted average of Precision and Recall. Explainability In the present work, we also examine how the risk factors have contributed to the final decision of KOA diagnosis. In order to achieve this, we used SHapley Additive exPlanations (SHAP), which is an approach to explain individual predictions based on Shapley Values of game theory and local explanations [40,41]. In particular, we employed SHAP to rank features in terms of their impact on the final ML (Random Forest) outputs and to build a mini explainer model, which contributes to understanding the behavioral and the contribution of the risk factors in KOA diagnosis. Results In this section, we demonstrate the overall diagnosis performance of the models in relation to the first 100 selected features, and the highest metrics of the best models are also presented. Then, reference is made in the most important risk factors as they have been selected by the proposed Fuzzy FS methodology. Moreover, a comparative analysis is presented to prove the superiority of the proposed FS methodology compared to a number of well-known FS techniques. For the interpretation of the best model, an explainability analysis is employed to enhance our understanding of the reasoning behind its decision-making mechanism. Diagnosis Performance This subsection presents the results of a comparative analysis over a number of well-known ML models on the diagnosis classification task by using the first 100 selected risk factors. Figure 5 shows the testing accuracy performance (%) of the competing ML models with respect to the number of selected features. Specifically, KNN failed in diagnosis task, recording low testing accuracy performances. The rest of the ML models had an upward trend in the range of the first 15 risk factors. Overall, the best overall performance was achieved by RF with a maximum of 73.55% at 21 features. Furthermore, the classification performance of the best performing ML models was further evaluated with respect to various validation metrics including class precision, recall, and f1-score. Table 2 demonstrates the best performance metrics of RF, MLP, LR, SVMs, and KNN models on the diagnosis task. In particular, RF achieved the best overall performance (73.55% accuracy) on the group of the twenty-one (21) risk factors. SVMs achieved the second-highest accuracy (73.36%). The rest of the ML models achieved lower accuracies. Table 2 . Summary of best metrics per model and number of selected features. Models Accuracy Precision Recall F1-Score Num. of Features RF 73.55 73.82 73.64 73.59 21 MLP 73.20 73.48 73.20 73.13 17 LR 73.27 73.38 73.27 73.24 17 SVMs 73.36 73.68 73.36 73.27 18 KNN 71.55 71.74 71.55 71.49 12 Features Selected Figure 6 reveals more information about the origin of the 21 risk factors as selected by the chosen Fuzzy FS approach. As observed in Figure 6, six features describing subject characteristics were among the selected risk factors e.g., the age of the participants, the body mass index (BMI), and the diastolic blood pressure. Moreover, five out of the 21 selected risk factors come from the symptom’s category, representing clinical parameters related to stiffness, knee difficulty, swelling, and pain, demonstrating the indication of the existence of KOA. Four of the risk factors are related to physical exams, whereas another two medical history and two physical activity parameters were selected as relevant to KOA occurrence. A behavioural risk factor and a nutrition risk factor were also selected by the proposed Fuzzy FS approach. Comparative Analysis The performance of the proposed FSFL methodology was compared with each one of the six FS techniques that were also implemented independently. Finally, another recently published FS technique was also selected as comparative in which the final feature ranking, is decided on the basis of a majority vote scheme [42, 43]. Table 3 shows the maximum achieved accuracy in the first selected 100 features of OAI dataset and the number of features where the maximum accuracy was reached for each feature selection method used in the experimental evaluation with the best performed model (RF). The last row in table 3 shows the dimensionality reduction achieved with the proposed FS method compared to other competitive methods. Specifically, the metric DR was defined to quantify the difference (%) in dimensionality reduction compared to FSFL: The proposed FSFL method achieved the best trade-off between performance and dimensionality reduction being capable of reducing significantly the feature set dimensionality while achieving slightly higher or comparable prediction performance with the rest competing algorithms. Specifically, the proposed FSFL technique reaches the highest accuracy (73.55%) at 21 selected features while the second-best accuracy (73.51%) was achieved by LBGM Emb at 87 features. This shows that the proposed FSFL technique results to a 76% smaller set of selected features compared to the second-best performing technique. On the other hand, the second-best performer with respect to dimensionality reduction was RF Emb with 73.36% accuracy achieved on a considerably larger feature subset with more than double features (43) compared to FSFL (21). Table 3 . Comparative analysis of FS methods. FSFL Vote FS RF Emb FS LGBM Emb FS SVM RFE FS LR RFE FS Filter MI FS Filter f-ANOVA FS Max imum Acc uracy (%) 73.55 72.99 73.36 73.51 70.53 73.50 72.75 73.44 Num ber o f Selected Features 21 76 43 87 96 60 91 53 DR (%) - +72% +51% +76% +78% +65% +77% +60% Explainability Results Figure 7a depicts how the features’ impact shapes the output of the final model (RF) on the testing dataset. The features are sorted by the sum of SHAP value magnitudes over all testing subjects. Furthermore, the SHAP values are used to demonstrate the contribution of each risk factor (negative or positive) on the model’s output. Specifically, blue color represents low feature values, whereas red color represents high values, respectively. In particular, a high value of PO2ELGRISK (knee symptoms, risk factors, or both status) increases the probability of the subjects to be assigned to class KOA. Similarly to PO2ELGRISK, the higher the values of risk factors V00AGE, P02KSRG, P01BM1, V00RKFHDEG, P01WEIGHT, V00LKFHDEG, V00WTMACKG, V00BRDIAS, V00KPLKN1, and P02PA1, the more probable for subjects to belong to class KOA. The rest of the selected risk factors in Figure 7 have the opposite effect pushing the prediction output of the model to the class of healthy subjects. Figure 7b presents the SHAP global feature importance. The risk factors are sorted by the mean [|SHAP value|], which is the average impact on model output magnitude. Figure 8 interprets locally the behavior of the model for the prediction output in a subject that suffers by KOA. P02ELGRISK (with a value of 2) and P01BMI (with a value of 29.8) push the predictions towards the class of KOA patients. Therefore, a high value of the aforementioned risk factors results to the increase of the output probability of the subject to be classified as KOA patient. On the contrary, increase of the risk factors P02KSURG, V00RKFHDEG, V00KOOSQOL, and V00KOOSKPR lowers the probability of a subject to be classified as KOA. Since, our prediction score = 0.51 > base value = 0.49, this subject has been positively classified, i.e., class KOA status. Discussion Handling the multidimensional nature of the OAI dataset, a novel fuzzy ensemble FS methodology was designed, implemented and tested in this paper. Its main novelty lies on the combination of several well-known FS algorithms based on a properly designed fuzzy inference mechanism that effectively aggregates their outputs. The superiority of the proposed FS technique was demonstrated through a thorough comparative investigation that included several state-of-the-art algorithms coming from different FS families (filter, wrapper, embedded and hybrid). The proposed fuzzy FS methodology outperformed the aforementioned FS techniques achieving the best trade-off between dimensionality reduction and prediction accuracy. Working on a high-dimensional dataset of 643 features, twenty-one risk factors were selected for the objective of KOA diagnosis. Observing the nature of the selected risk factors, it was found that subject characteristics, symptoms, and physical exams are the most important risk factors contributing considerably to the KOA diagnosis. Overall, it was concluded that a combination of heterogeneous risk factors coming from different feature categories is needed for the effective diagnosis of KOA. To sanity check the AI models beyond mere performance and further quantify the relevance of the selected risk factors, a post hoc explainability analysis was also conducted using SHAP. As observed by SHAP, P02ELGRISK, P02KSURG, V00AGE, P01BMI and V00KOOSQOL are five risk factors that have a major impact to the prediction output, which are in line with the existing literature. Specifically, P02ELGRISK, that represents knee symptoms, is an important risk factor in the diagnosis of KOA, as it has been identified by Lespasio et al. [ 44 ]. The history of knee surgery (P02KSURG) has been recognised as an important risk factor of KOA by Katz et al. [ 45 ], whereas the age of the subjects was also characterized as crucial in the occurrence of KOA and therefore was considered in the development of a predictive model for KOA diagnosis [ 13 ]. The knee injury and osteoarthritis outcome (KOOS) is a well-known knee-specific instrument that has been widely employed to evaluate quality of life in patients with knee injuries and identify patients who are at risk of developing OA [ 46 ]. Moreover, high BMI is suggested to be a high-risk factor in the development of KOA. High BMI values lead to the increment of knee joint mechanical loading [ 47 ]. Although the proposed FSFL technique selects a subset of risk factors with a significant dimensionality reduction compared to popular FS techniques, the application of a post-hoc explainability is still important in order to identify the contribution of the selected features to prediction output of the model. The use of explainability analysis algorithms for the interpretation of the ML models increases the understanding of the principle of operation of each ML model and reveal the interactions that shape the diagnosis outcome. The proposed methodology can be considered as computationally intensive; however, FS is considered here as an offline process and therefore the execution time does not play a crucial role. Future work will focus on the identification of easily measurable biomarkers and biomechanical parameters derived from musculoskeletal models, in combination with the already selected risk factors for the early diagnosis of KOA in the general population. Hence, to achieve this goal more advanced AI analytics tools in combination with the FSFL algorithm will be employed. Conclusion To enforce the development of more reliable, powerful, and non-invasive diagnostic tools, this study focuses on the identification and interpretation of the risk factors that contribute on the diagnosis of KOA. The proposed methodology is based on a novel fuzzy logic-based feature selection followed by learning algorithms and subsequently a post-hoc explainability analysis. The proposed technique aggregates the results of several FS algorithms (filter, wrapper and embedded ones), whereas fuzzy logic was employed to combine multiple feature importance scores thus leading to a more robust selection of informative features. The results showed that the presented methodology was capable to select a subset of risk factors that increase the performance accuracy of various ML models, compared to popular FS techniques. This was achieved with a significant decrease on the feature dimensionality (up to 78%). SHAP was finally applied to enhance our understanding of the rationale behind the decision-making mechanism of the selected ML model and the impact of the used risk factors on the prediction output. Declarations Funding This research was funded by the European Community’s H2020 Programme, under grant agreement No. 777159 (OACTIVE). Institutional Review Board Statement Not applicable. Informed Consent Statement Not applicable. Data Availability Statement Data from the osteoarthritis initiative (OAI) database (available upon request at https://nda.nih.gov/oai/). Conflicts of Interest The authors declare no conflict of interest. References [1] Malanga G, Niazi F, Kidd VD, Lau E, Kurtz SM, Ong KL, Concoff AL (2020) Knee Osteoarthritis Treatment Costs in the Medicare Patient Population. American health & drug benefits 13 (4) 144. [2] Johnson VL, Hunter DJ (2014) The epidemiology of osteoarthritis. Best practice & research Clinical rheumatology 28 (1) 5-15. 10.1016/j.berh.2014.01.004 [3] Silverwood V, Blagojevic-Bucknall M, Jinks C, Jordan J, Protheroe J, Jordan K (2015) Current evidence on risk factors for knee osteoarthritis in older adults: a systematic review and meta-analysis. Osteoarthritis Cartilage 23 (4) 507-515. 10.1016/j.joca.2014.11.019 [4] Ackerman IN, Kemp JL, Crossley KM, Culvenor AG, Hinman, RS (2017) Hip and Knee Osteoarthritis Affects Younger People, Too. J Orthop Sports Phys Ther 47 (2) 67-79. 10.2519/jospt.2017.7286 [5] Toivanen AT, Heliövaara M, Impivaara O, Arokoski JP, Knekt P, Lauren H, Kröger H (2010) Obesity, physically demanding work and traumatic knee injury are major risk factors for knee osteoarthritis—a population-based study with a follow-up of 22 years. Rheumatology 49 (2) 308-314. 10.1093/rheumatology/kep388 [6] Mahir L, Belhaj K, Zahi S, Azanmasso H, Lmidmani F, El Fatimi A (2016) Impact of knee osteoarthritis on the quality of life. Annals of physical and rehabilitation medicine 59 e159. https://doi.org/10.1016/j.rehab.2016.07.355 [7] Jack Farr I, Miller LE, Block JE (2013) Quality of life in patients with knee osteoarthritis: a commentary on nonsurgical and surgical treatments. The open orthopaedics journal 7 619. 10.2174/1874325001307010619 [8] Kohn MD, Sassoon AA, Fernando ND (2016) Classifications in brief: Kellgren-Lawrence classification of osteoarthritis. Clinical Orthopaedics and Related Research® 474 (8) 1886-1893. 10.1007/s11999-016-4732-4 [9] Kokkotis C, Moustakidis S, Papageorgiou E, Giakas G, Tsaopoulos D (2020) Machine Learning in Knee Osteoarthritis: A Review. Osteoarthritis and Cartilage Open 100069. https://doi.org/10.1016/j.ocarto.2020.100069 [10] Yoo TK, Kim DW, Choi SB, Oh E, Park JS (2016) Simple Scoring System and Artificial Neural Network for Knee Osteoarthritis Risk Prediction: A Cross-Sectional Study. PLoS One 11 (2) e0148724. 10.1371/journal.pone.0148724 [11] Lim J, Kim J, Cheon S (2019) A Deep Neural Network-Based Method for Early Detection of Osteoarthritis Using Statistical Data. International journal of environmental research and public health 16 (7) 1281. 10.3390/ijerph16071281 [12] Christodoulou E, Moustakidis S, Papandrianos N, Tsaopoulos D, Papageorgiou E (2019) Exploring deep learning capabilities in knee osteoarthritis case study for classification. Paper presented at the 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA). 10.1109/IISA.2019.8900714 [13] Moustakidis S, Christodoulou E, Papageorgiou E, Kokkotis C, Papandrianos N, Tsaopoulos D (2019) Application of machine intelligence for osteoarthritis classification: a classical implementation and a quantum perspective. Quantum Machine Intelligence . 10.1007/s42484-019-00008-3 [14] Kwon SB, Han HS, Lee MC, Kim H C, Ku Y (2020) Machine learning-based automatic classification of knee osteoarthritis severity using gait data and radiographic images. IEEE Access 8 120597-120603. 10.1109/ACCESS.2020.3006335 [15] Moustakidis S, Papandrianos N, Christodolou E, Papageorgiou E, Tsaopoulos D (2020) Dense neural networks in knee osteoarthritis classification: a study on accuracy and fairness. Neural Computing and Applications 1-13. https://doi.org/10.1007/s00521-020-05459-5 [16] Hardi S, Triwiyono A (2020) Expert System for Diagnosing Osteoarthritis with Fuzzy Tsukamoto Method. Paper presented at the Journal of Physics: Conference Series. 10.1088/1742-6596/1641/1/012107 [17] Pal JK, Ray SS, Pal SK (2017) Fuzzy mutual information based grouping and new fitness function for PSO in selection of miRNAs in cancer. Computers in biology and medicine 89 540-548. 10.1016/j.compbiomed.2017.08.013 [18] Dai J, Chen J (2020) Feature selection via normative fuzzy information weight with application into tumor classification. Applied Soft Computing 92 106299. https://doi.org/10.1016/j.asoc.2020.106299 [19] Lin Y, Hu Q, Liu J, Li J, Wu X (2017) Streaming feature selection for multilabel learning based on fuzzy mutual information. IEEE Transactions on Fuzzy Systems 25 (6) 1491-1507. 10.1109/TFUZZ.2017.2735947 [20] Jaganathan P, Kuppuchamy R (2013) A threshold fuzzy entropy based feature selection for medical database classification. Computers in biology and medicine 43 (12) 2222-2229. https://doi.org/10.1016/j.compbiomed.2013.10.016 [21] Wang C, Qi Y, Shao M, Hu Q, Chen D, Qian Y, Lin Y (2016) A fitting model for feature selection with fuzzy rough sets. IEEE Transactions on Fuzzy Systems 25 (4) 741-753. 10.1109/TFUZZ.2016.2574918 [22] Qian Y, Wang Q, Cheng H, Liang J, Dang C (2015) Fuzzy-rough feature selection accelerator. Fuzzy Sets and Systems 258 61-78. https://doi.org/10.1016/j.fss.2014.04.029 [23] Silva-Ramírez EL, Pino-Mejías R, López-Coello M, Cubiles-de-la-Vega MD (2011) Missing value imputation on missing completely at random data using multilayer perceptrons. Neural Networks 24 (1) 121-129. https://doi.org/10.1016/j.neunet.2010.09.008 [24] Pihera J, Musliu N (2014) Application of machine learning to algorithm selection for TSP. Paper presented at the 2014 IEEE 26th International Conference on Tools with Artificial Intelligence. 10.1109/ICTAI.2014.18 [25] Japkowicz N (2000) Learning from imbalanced data sets: a comparison of various strategies. Paper presented at the AAAI workshop on learning from imbalanced data sets. [26] Vergara JR, Estévez PA (2014) A review of feature selection methods based on mutual information. Neural Computing and Applications 24 (1) 175-186. https://doi.org/10.1007/s00521-013-1368-0 [27] Suchwalko A, Buzalewicz I, Podbielska H (2013). Identification of bacteria species by using morphological and textural properties of bacterial colonies diffraction patterns. Paper presented at the Videometrics, Range Imaging, and Applications XII; and Automated Visual Inspection. https://doi.org/10.1117/12.2020337 [28] Zhao J, Karimzadeh M, Masjedi A, Wang T, Zhang X, Crawford MM, Ebert DS (2019) FeatureExplorer: Interactive feature selection and exploration of regression models for hyperspectral images. Paper presented at the 2019 IEEE Visualization Conference (VIS). 10.1109/VISUAL.2019.8933619 [29] Ding J, Shi J, Wu FX (2011) SVM-RFE based feature selection for tandem mass spectrum quality assessment. International journal of data mining and bioinformatics 5 (1) 73-88. 10.1504/ijdmb.2011.038578 [30] Ye Y, Liu C, Zemiti N, Yang C (2019) Optimal feature selection for EMG-based finger force estimation using lightGBM model. Paper presented at the 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). 10.1109/RO-MAN46459.2019.8956453 [31] Mate Y, Somai N (2021) Hybrid Feature Selection and Bayesian Optimization with Machine Learning for Breast Cancer Prediction. Paper presented at the 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS). 10.1109/ICACCS51430.2021.9441914 [32] Gayathri B, Sumathi C (2015). Mamdani fuzzy inference system for breast cancer risk detection. Paper presented at the 2015 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC). 10.1109/ICCIC.2015.7435670 [33] Ram M, Najafi A, Shakeri MT (2017) Classification and biomarker genes selection for cancer gene expression data using random forest. Iranian journal of pathology 12 (4) 339. PMID: 29563929 PMCID: PMC5844678 [34] Parisi L, Biggs PR, Whatling GM, Holt CA (2015) A novel comparison of artificial intelligence methods for diagnosing knee osteoarthritis. Paper presented at the XXV congress of the international society of biomechanics. 10.13140/RG.2.1.4197.6163 [35] Ntakolia C, Kokkotis C, Moustakidis S, Tsaopoulos D (2020) A machine learning pipeline for predicting joint space narrowing in knee osteoarthritis patients. Paper presented at the 2020 IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE). 10.1109/BIBE50027.2020.00158 [36] Kubkaddi S, Ravikumar K (2017) Early detection of Knee Osteoarthritis using SVM Classifier. IJSEAT 5 (3) 259-262. [37] Long NP, Park S, Anh NH, Min JE, Yoon SJ, Kim HM, Lim J (2019) Efficacy of integrating a novel 16-gene biomarker panel and intelligence classifiers for differential diagnosis of rheumatoid arthritis and osteoarthritis. Journal of clinical medicine 8 (1) 50. 10.3390/jcm8010050 [38] Wong TT, Yeh PY (2019) Reliable accuracy estimates from k-fold cross validation. IEEE Transactions on Knowledge and Data Engineering 32 (8) 1586-1594. 10.1109/TKDE.2019.2912815 [39] Ghosh M, Sanyal G (2018) An ensemble approach to stabilize the features for multi-domain sentiment analysis using supervised machine learning. Journal of Big Data 5 (1) 1-25. https://doi.org/10.1186/s40537-018-0152-5 [40] Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. Paper presented at the Advances in neural information processing systems. https://dl.acm.org/doi/10.5555/3295222.3295230 [41] Nohara Y, Matsumoto K, Soejima H, Nakashima N (2019) Explanation of machine learning models using improved Shapley Additive Explanation. Paper presented at the Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. 10.1145/3307339.3343255 [42] Kokkotis C, Moustakidis S, Giakas G, Tsaopoulos D (2020) Identification of Risk Factors and Machine Learning-Based Prediction Models for Knee Osteoarthritis Patients. Applied Sciences 10 (19) 6797. https://doi.org/10.3390/app10196797 [43] Ntakolia C, Kokkotis C, Moustakidis S, Tsaopoulos D (2021) Prediction of Joint Space Narrowing Progression in Knee Osteoarthritis Patients. Diagnostics 11 (2) 285. https://doi.org/10.3390/diagnostics11020285 [44] Lespasio MJ, Piuzzi NS, Husni ME, Muschler GF, Guarino A, Mont MA (2017) Knee osteoarthritis: a primer. The Permanente Journal 21 . 10.7812/TPP/16-183 [45] Katz JN, Arant KR, Loeser RF (2021) Diagnosis and treatment of hip and knee osteoarthritis: a review. Jama 325 (6) 568-578. 10.1001/jama.2020.22171 [46] Roos EM, Lohmander, LS (2003) The Knee injury and Osteoarthritis Outcome Score (KOOS): from joint injury to osteoarthritis. Health and quality of life outcomes 1 (1) 1-8. 10.1186/1477-7525-1-64 [47] Cooper C, Snow S, McAlindon TE, Kellingray S, Stuart B, Coggon D, Dieppe PA (2000) Risk factors for the incidence and progression of radiographic knee osteoarthritis. Arthritis & Rheumatism: Official Journal of the American College of Rheumatology 43 (5) 995-1000. 10.1002/1529-0131(200005)43:53.0.CO;2-1 Supplementary Files AppendixA.docx Cite Share Download PDF Status: Published Journal Publication published 31 Jan, 2022 Read the published version in Physical and Engineering Sciences in Medicine → Version 1 posted Reviews received at journal 08 Aug, 2021 Reviewers invited by journal 08 Aug, 2021 Editor invited by journal 03 Aug, 2021 Editor assigned by journal 03 Aug, 2021 First submitted to journal 02 Aug, 2021 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-777000","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":44529554,"identity":"c45a340c-052f-4717-b8e9-fa112683273e","order_by":0,"name":"Christos Kokkotis","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA2klEQVRIiWNgGAWjYDACCTYGxoYDDAxsDMwHQFwZorUAKbYEEJeHeC0MDDwGID5hLfyz2xIfzjhjU8fHf+bzqxs1FjwM7IePbsBryZ1jhw033EiTYJPI3WadcwzoMJ60tBv4tBhIpLdJPvhwGKiFd5txDhtQiwSPGZFa+M88M875R5SWtGOSG24AtTDkMD/ObSNCi8SNtGTDGWfSJNsk0syYc/skeNgI+YV/Rprhw55jNvzy/Ycff875VifHz374GF4tyIBNAkwSqxwEmD+QonoUjIJRMApGDgAAv2tGDTb+ZQ8AAAAASUVORK5CYII=","orcid":"","institution":"CERTH: Ethniko Kentro Ereunas \u0026 Technologikes Anaptyxes","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Christos","middleName":"","lastName":"Kokkotis","suffix":""},{"id":44529555,"identity":"8b004e99-b1ae-4e9a-8012-958b075a4310","order_by":1,"name":"Charis Ntakolia","email":"","orcid":"","institution":"National Technical University Of Athens School of Naval Architecture and Marine Engineering: Ethniko Metsobio Polytechneio Schole Naupegon Mechanologon Mechanikon","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Charis","middleName":"","lastName":"Ntakolia","suffix":""},{"id":44529556,"identity":"b02c8d4b-0f18-476b-8758-572ee68dd5a2","order_by":2,"name":"Serafeim Moustakidis","email":"","orcid":"","institution":"AIDEAS OU","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Serafeim","middleName":"","lastName":"Moustakidis","suffix":""},{"id":44529557,"identity":"37c4ec1f-d49e-4b26-8f73-9b1fac971ecb","order_by":3,"name":"Giannis Giakas","email":"","orcid":"","institution":"Department of Physical Education and Sport Science, University of Thessaly","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Giannis","middleName":"","lastName":"Giakas","suffix":""},{"id":44529558,"identity":"e2428344-e5e5-458b-9b03-2ff08a1024d7","order_by":4,"name":"Dimitrios Tsaopoulos","email":"","orcid":"","institution":"CERTH: Ethniko Kentro Ereunas \u0026 Technologikes Anaptyxes","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Dimitrios","middleName":"","lastName":"Tsaopoulos","suffix":""}],"badges":[],"createdAt":"2021-08-02 16:35:20","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-777000/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-777000/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s13246-022-01106-6","type":"published","date":"2022-01-31T10:59:06+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":12304850,"identity":"78e7c525-ecf9-407c-8fd7-5220ea8d43f3","added_by":"auto","created_at":"2021-08-10 21:19:13","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":48010,"visible":true,"origin":"","legend":"The proposed AI methodology for KOA diagnosis.","description":"","filename":"fig1.png","url":"https://assets-eu.researchsquare.com/files/rs-777000/v1/d553791c822cf5df1da47e98.png"},{"id":12304923,"identity":"fb7aa55b-f813-41b5-a1d1-8d4d1393a920","added_by":"auto","created_at":"2021-08-10 21:22:13","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":180113,"visible":true,"origin":"","legend":"Feature Selection method based on Fuzzy Logic flowchart.","description":"","filename":"fig2.png","url":"https://assets-eu.researchsquare.com/files/rs-777000/v1/c142513426636795e158b045.png"},{"id":12304852,"identity":"3f5c03e4-5497-4ffb-81ce-1d4ae3d9677c","added_by":"auto","created_at":"2021-08-10 21:19:13","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":33592,"visible":true,"origin":"","legend":"Fuzzy set of input variables for FIS 1 and 2.","description":"","filename":"fig3.png","url":"https://assets-eu.researchsquare.com/files/rs-777000/v1/6c71dcb3647298073b962949.png"},{"id":12305248,"identity":"4ef789f2-41dc-49ba-a3df-0aeecef89b5d","added_by":"auto","created_at":"2021-08-10 21:28:13","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":50022,"visible":true,"origin":"","legend":"Fuzzy set of output variable for FIS 1 and 2.","description":"","filename":"fig4.png","url":"https://assets-eu.researchsquare.com/files/rs-777000/v1/ceacdc0771b7b33f2345eec1.png"},{"id":12305029,"identity":"0a873a52-d7d0-487c-8cf0-523567d036b8","added_by":"auto","created_at":"2021-08-10 21:25:13","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":84401,"visible":true,"origin":"","legend":"Curves with testing accuracy scores with respect to the number of selected features for different ML models.","description":"","filename":"fig5.png","url":"https://assets-eu.researchsquare.com/files/rs-777000/v1/a2e5d4e9b90d81a9d90a30ab.png"},{"id":12304855,"identity":"61047edb-a6bf-47de-9eaf-54229d06b97e","added_by":"auto","created_at":"2021-08-10 21:19:13","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":58802,"visible":true,"origin":"","legend":"The 21 most informative selected risk factors per category.","description":"","filename":"fig6.png","url":"https://assets-eu.researchsquare.com/files/rs-777000/v1/df0d3287345318afb62c1a9e.png"},{"id":12304857,"identity":"efdc3aed-2da5-4dff-a3e2-f3cf229dd312","added_by":"auto","created_at":"2021-08-10 21:19:13","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":436015,"visible":true,"origin":"","legend":"a) Features’ impact on Random Forest (21F) model output for the testing set of OAI dataset. b) Features’ average impact magnitude for testing instances.","description":"","filename":"fig7.png","url":"https://assets-eu.researchsquare.com/files/rs-777000/v1/79dcbffc943288695269909e.png"},{"id":12304926,"identity":"0b172fb1-b02c-45ac-8392-0b3f48d05899","added_by":"auto","created_at":"2021-08-10 21:22:13","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":96695,"visible":true,"origin":"","legend":"Risk factors contributions to ML model output for a KOA status subject","description":"","filename":"fig8.png","url":"https://assets-eu.researchsquare.com/files/rs-777000/v1/28755e3f5a1095c4041766c4.png"},{"id":17800261,"identity":"84f6645a-165d-4b93-a20b-7553bfdc9a75","added_by":"auto","created_at":"2022-01-31 10:59:09","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1257916,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-777000/v1/9697fc97-b1a5-4c4a-bc99-441f64a253c8.pdf"},{"id":12304922,"identity":"9453650d-10f0-4a31-8892-ebd54b89db8f","added_by":"auto","created_at":"2021-08-10 21:22:13","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":13415,"visible":true,"origin":"","legend":"","description":"","filename":"AppendixA.docx","url":"https://assets-eu.researchsquare.com/files/rs-777000/v1/13e204f95894d9d5a3d47aeb.docx"}],"financialInterests":"","formattedTitle":"Explainable Machine Learning for Knee Osteoarthritis Diagnosis Based on a Novel Fuzzy Feature Selection Methodology","fulltext":[{"header":"Introduction","content":"\u003cp\u003eKnee Osteoarthritis (KOA) is one of the most common types of osteoarthritis and musculoskeletal disorder. Being the 11th highest global disability, KOA is a multifactorial disease that results from mechanical and constitutional factors [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Obesity, age, gender, knee injuries and lifestyle are likely risk factors of KOA as they have been highlighted in the relevant recent literature [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. In addition, swelling, pain and stiffness have been characterized as typical symptoms of the disease with irreversible cartilage damage being KOA\u0026rsquo;s main consequence [\u003cspan additionalcitationids=\"CR4\" citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. KOA is closely associated with a huge economic burden for the healthcare system and an unbearable health burden of the patients and their families. Significant consequences of KOA are the social isolation of the individual and the low quality of life [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. Furthermore, the quantification of KOA is performed with the Kellgren\u0026ndash;Lawrence (KL) severity grading scale, which is the most commonly grading system (current gold standard) and consists of five severity grades, from 0 to 4 [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eDespite the fact that the scientific community has put a lot of effort into KOA research, a major challenge remains with respect to early diagnosis, long-term diagnosis and treatment of ΚΟΑ. The parallel increase in computing power along with the collection of big datasets combined with the need to address the above challenges has led many research teams to use artificial intelligence (AI) techniques in the field of KOA [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. In light of the above, several AI enabled studies have been proposed in the recent literature with the objective to diagnose or predict KOA. Yoo et al. used data from the Fifth Korea National Health and Nutrition Examination Surveys (KNHANES V-1) and the Osteoarthritis Initiative (OAI) to build an artificial neural network (ANN)-based a scoring system for the identification of KOA severity [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. The proposed ANN model achieved an area under the curve (AUC) of 0.76 for the symptomatic KOA in an external validation with OAI data. In another study, Lim et al. proposed a method for early diagnosis of KOA based on clinical data from Korean National Health and Nutrition Examination Survey (KNHANES) [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. They achieved an 76.8% AUC by using a deep neural network with scaled principal component analysis. In 2019, Christodoulou et al. investigated the deep learning capabilities in KOA diagnosis [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. They used clinical data from OAI database and they achieved an 86.95% accuracy working on an aged subgroup (70 +).\u003c/p\u003e \u003cp\u003eIn another study, Moustakidis et al. worked on self-reported clinical data (OAI) and proposed a deep learning methodology for the recognition of participants being at high risk of developing KOA in at least one knee and participants with symptomatic KOA [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. They achieved accuracies up to 86.95%. Furthermore, Kwon et al. proposed an automatic classification of KOA severity that made use of gait analysis data and radiographic imaging (from Seoul National University Hospital) [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]). They employed Inception-ResNet-v2 for feature extraction from X-rays and a support vector machine for KOA diagnosis achieving accuracies of 0.93, 0.82, 0.83, 0.88, and 0.97 for the KL grades 0\u0026ndash;4, respectively. In addition, Moustakidis et al. proposed a KOA classification approach with a focus on both accuracy and fairness [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. They worked on different subgroups of participants from self-reported clinical data (OAI) and the dense neural networks methodology improved the accuracy up to 79.6% with fairness measured by balanced equalized odds (~\u0026thinsp;92%) and demographic parity (98.5%) in the KOA case study.\u003c/p\u003e \u003cp\u003eGiven that medical data and features can be subjective or difficult to interpret, medical decision making has a great potential to benefit by the use of fuzzy logic (FL). FL has been used to diagnose or facilitate decision making systems tackling many diseases, including OA. Hardi et al. proposed an expert system based on the fuzzy Tsukamoto method for OA diagnosis [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. They treated symptoms of OA as fuzzy values that were further converted into firm value by using a weighted average demonstrating a 90% accuracy in the task of diagnosis of osteoarthritis disease. In general, various feature selection methods have integrated fuzzy logic in their internal mechanisms in order to handle the observed fuzziness and therefore improve the way that features are treated and combined. For instance, with emphasis to medical applications, mutual information method combined with FL was used: (i) to select miRNAs in cancer [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]; (ii) to classify tumors [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]; and to select features for multilabel learning [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. Similar studies include fuzzy entropy by using thresholds [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e] for feature selection in various medical datasets and fuzzy rough sets [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e] for dimensionality reduction of feature space to prevent samples from misclassification.\u003c/p\u003e \u003cp\u003eIt is well known that each one of the existing FS algorithms comes with its own advantages and disadvantages introducing a certain level of bias. To handle the multidimensional nature of the OAI dataset and to avoid bias and alleviate the defectiveness of single feature selection results, a fuzzy ensemble FS methodology is proposed in this paper that aggregates the results of several FS algorithms (filter, wrapper and embedded ones). Fuzzy logic is employed to combine multiple feature importance scores thus leading to a more robust selection of informative features. The proposed method contributes to the significant reduction of the initial OAI feature dimensionality and to a decrease of the computational complexity of the classification models employed. To prove the effectiveness of the proposed methodology, an extensive experimental setup was designed involving multiple competing FS algorithms and several well-known ML models. As a post-hoc explainability, SHAP model was finally employed to identify the contribution of the selected features and the rationale behind the decision-making mechanism of best performing model.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003e\u003cstrong\u003eDataset Description\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFor the purpose of this study, data were employed from the osteoarthritis initiative (OAI) database (available on \u003ca href=\"https://nda.nih.gov/oai/\"\u003ehttps://nda.nih.gov/oai/\u003c/a\u003e). OAI is a prospective observational, multi-center and longitudinal study of KOA. OAI has enrolled 4796 women and men, aged 45-79 years. The present study used clinical evaluation data (643 features in total) from the baseline visit from all participants with or without KOA. The features of clinical dataset were divided into seven categories as shown in Table 1. Furthermore, in the present study, Kellgren and Lawrence (KL) grades were used as the outcome for the classification task.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 1\u003c/strong\u003e. Main categories of the clinical evaluation data considered in this study.\u003c/p\u003e\n\u003cdiv align=\"center\"\u003e\n \u003ctable border=\"1\" cellpadding=\"0\" cellspacing=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003e\u003cstrong\u003eCategory\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003e\u003cstrong\u003eDescription\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003eMedical history\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003eMedications and health histories based on questionnaire results (not included medical imaging outcomes)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003eSymptoms\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003eArthritis symptoms or health-related disability and function based on questionnaire data\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003eSubject characteristics\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003eIncludes variables which describe anthropometric parameters and personal information\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003eNutrition\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003eQuestionnaire based on Block Food\u003c/p\u003e\n \u003cp\u003eFrequency\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003ePhysical exam\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003eIncludes performance measures and knee and hand exams\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003ePhysical activity\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003eQuestionnaire results regarding living and leisure activities\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eBehavioral\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"50%\"\u003e\n \u003cp\u003eConsists of variables which quantify the social behavior and the quality level of daily routine\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethodology\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe proposed AI methodology for KOA diagnosis consists of five processing steps: i) data pre-processing, ii) application of FS techniques, iii) learning process, iv) evaluation of the classification results and v) explainability analysis, as illustrated in Figure 1. An extensive presentation of the steps of the proposed methodology is given in the following subsections.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eProblem Definition\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn this study, we defined the KL-grade prediction task as a binary class classification problem. Specifically, the subjects of the study (3872 subjects in total) were divided into two equal groups:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003ei) KOA - participants who have KL \u0026gt;=2 at baseline. Participant in the group had KL grades equal (early diagnosis) or higher than 2 in at least one of the two knees or in both at baseline;\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eii) non-KOA - participants who had KL0 or KL1 grade at baseline. \u0026nbsp;Especially, this group of participants do not have \u0026Kappa;\u0026Omicron;A in any of their knees.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Pre-processing\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eMode imputation was employed to handle categorical and continuous missing values [23]. In our study, data were normalised to [0, 1] to build a common basis for the FS algorithms and learning techniques that follow [24]. Furthermore, to cope with the imbalance data problem a stratified strategy for data resampling was applied. In particular, the number of the subjects in the majority class was reduced in order to become equal with the number of samples on the minority class [25].\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eProposed FS methodology\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe proposed Fuzzy logic enhanced Feature Selection method (FSFL) combines the outputs of six well-known feature selection methods from three feature selection categories (Filter, Wrapper and Embedded). \u0026nbsp;Specifically, from the filter category, the mutual information [26] and the f-ANOVA [27] techniques were applied. From the wrapper category, we employed a recursive feature elimination (RFE) based on logistic regression [28] and an RFE based on support vector machine [29] techniques, respectively. Furthermore, from the embedded category, a LightGBM [30] and a random forest technique [31] were applied. To calculate the importance of a feature for each category, the scores of the associated FS techniques were used as input to the Fuzzy Inference System (FIS) 1 that was implemented with Mamdani inference methodology [32]. The output of the FIS 1 was the defuzzification value that represents the feature importance score for the specific feature selection category. Then, the defuzzification score of each category was used as input to the FIS 2 where the output defuzzification value represents the overall feature importance. Figure 2 illustrates the FSFL flowchart with the defined fuzzy rules for each FIS and the selected feature selection methods for this study. Figure 3 shows the fuzzy sets used in the presented methodology for the input variables for FIS 1 and FIS 2, while Figure 4 shows the fuzzy sets of output variable for FIS 1 and 2.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eLearning\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn order to handle the demanding task of KOA classification, we investigated various ML models for their suitability and behavior in this problem. Specifically, random forest (RF) [33], multilayer perceptron (MLP) [34], logistic regression (LR) [35], support-vector machines (SVMs) [36], and k-nearest neighbors (KNN) [37] classifiers were tested. Furthermore, to avoid overfitting, and to optimize the performance of our models hyperparameter selection was applied individually per model.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eValidation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFor the experimental evaluation, a repeated stratified 5-fold cross validation was used [38]. Furthermore, the performance of the classifiers was also evaluated in terms of the recall, f1-score and precision as additional evaluation criteria [39]. A brief description of these metrics is given below. Initially, the accuracy is the ratio of correctly predicted observations to the total observations and can be characterized as the most intuitive performance measure. Recall (or Sensitivity) is the ratio of correctly predicted positive observations to all observations in the actual class. Moreover, the ratio of correctly predicted positive observations to the total predicted positive observations is called precision or positive predictive value. F1-score is the weighted average of Precision and Recall.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eExplainability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn the present work, we also examine how the risk factors have contributed to the final decision of KOA diagnosis. In order to achieve this, we used SHapley Additive exPlanations (SHAP), which is an approach to explain individual predictions based on Shapley Values of game theory and local explanations [40,41]. In particular, we employed SHAP to rank features in terms of their impact on the final ML (Random Forest) outputs and to build a mini explainer model, which contributes to understanding the behavioral and the contribution of the risk factors in KOA diagnosis.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eIn this section, we demonstrate the overall diagnosis performance of the models in relation to the first 100 selected features, and the highest metrics of the best models are also presented. Then, reference is made in the most important risk factors as they have been selected by the proposed Fuzzy FS methodology. Moreover, a comparative analysis is presented to prove the superiority of the proposed FS methodology compared to a number of well-known FS techniques. For the interpretation of the best model, an explainability analysis is employed to enhance our understanding of the reasoning behind its decision-making mechanism.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDiagnosis Performance\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis subsection presents the results of a comparative analysis over a number of well-known ML models on the diagnosis classification task by using the first 100 selected risk factors. Figure 5 shows the testing accuracy performance (%) of the competing ML models with respect to the number of selected features. Specifically, KNN failed in diagnosis task, recording low testing accuracy performances. The rest of the ML models had an upward trend in the range of the first 15 risk factors. Overall, the best overall performance was achieved by RF with a maximum of 73.55% at 21 features.\u003c/p\u003e\n\u003cp\u003eFurthermore, the classification performance of the best performing ML models was further evaluated with respect to various validation metrics including class precision, recall, and f1-score. Table 2 demonstrates the best performance metrics of RF, MLP, LR, SVMs, and KNN models on the diagnosis task. In particular, RF achieved the best overall performance (73.55% accuracy) on the group of the twenty-one (21) risk factors. SVMs achieved the second-highest accuracy (73.36%). The rest of the ML models achieved lower accuracies.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 2\u003c/strong\u003e. Summary of best metrics per model and number of selected features.\u0026nbsp;\u003c/p\u003e\n\u003cdiv align=\"center\"\u003e\n \u003ctable border=\"1\" cellpadding=\"0\" cellspacing=\"0\" width=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"19.012797074954296%\"\u003e\n \u003cp\u003e\u003cstrong\u003eModels\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"15.539305301645339%\"\u003e\n \u003cp\u003e\u003cstrong\u003eAccuracy\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e\u003cstrong\u003ePrecision\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e\u003cstrong\u003eRecall\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e\u003cstrong\u003eF1-Score\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"22.120658135283364%\"\u003e\n \u003cp\u003e\u003cstrong\u003eNum. of Features\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"19.012797074954296%\"\u003e\n \u003cp\u003e\u003cstrong\u003eRF\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"15.539305301645339%\"\u003e\n \u003cp\u003e73.55\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.82\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.64\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"22.120658135283364%\"\u003e\n \u003cp\u003e21\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"19.012797074954296%\"\u003e\n \u003cp\u003e\u003cstrong\u003eMLP\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"15.539305301645339%\"\u003e\n \u003cp\u003e73.20\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.48\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.20\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"22.120658135283364%\"\u003e\n \u003cp\u003e17\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"19.012797074954296%\"\u003e\n \u003cp\u003e\u003cstrong\u003eLR\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"15.539305301645339%\"\u003e\n \u003cp\u003e73.27\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.38\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.27\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.24\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"22.120658135283364%\"\u003e\n \u003cp\u003e17\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"19.012797074954296%\"\u003e\n \u003cp\u003e\u003cstrong\u003eSVMs\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"15.539305301645339%\"\u003e\n \u003cp\u003e73.36\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.68\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.36\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e73.27\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"22.120658135283364%\"\u003e\n \u003cp\u003e18\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"19.012797074954296%\"\u003e\n \u003cp\u003e\u003cstrong\u003eKNN\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"15.539305301645339%\"\u003e\n \u003cp\u003e71.55\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e71.74\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e71.55\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"14.442413162705668%\"\u003e\n \u003cp\u003e71.49\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"22.120658135283364%\"\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFeatures Selected\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFigure 6 reveals more information about the origin of the 21 risk factors as selected by the chosen Fuzzy FS approach. As observed in Figure 6, six features describing subject characteristics were among the selected risk factors e.g., the age of the participants, the body mass index (BMI), and the diastolic blood pressure. Moreover, five out of the 21 selected risk factors come from the symptom\u0026rsquo;s category, representing clinical parameters related to stiffness, knee difficulty, swelling, and pain, demonstrating the indication of the existence of KOA. Four of the risk factors are related to physical exams, whereas another two medical history and two physical activity parameters were selected as relevant to KOA occurrence. A behavioural risk factor and a nutrition risk factor were also selected by the proposed Fuzzy FS approach.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003cstrong\u003eComparative Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe performance of the proposed FSFL methodology was compared with each one of the six FS techniques that were also implemented independently. Finally, another recently published FS technique was also selected as comparative in which the final feature ranking, is decided on the basis of a majority vote scheme [42, 43].\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 3 shows the maximum achieved accuracy in the first selected 100 features of OAI dataset and the number of features where the maximum accuracy was reached for each feature selection method used in the experimental evaluation with the best performed model (RF). The last row in table 3 shows the dimensionality reduction achieved with the proposed FS method compared to other competitive methods. Specifically, the metric DR was defined to quantify the difference (%) in dimensionality reduction compared to FSFL:\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAWQAAAArCAYAAABPYvItAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAr5SURBVHhe7dvxkRs3D8Zht5Aa3EJ6SAmpIS2kg3SQDlxBKkgD/jsz6SA93DdP7PebNxzuSrbvZOkOvxmOdkkQAEECu9LZ756GYRiGu2AK8jAMw50wBXkYhuFOmII8DMNwJ0xBHoZhuBOmIA/DMNwJU5CHYRjuhCnIwzAMd8IU5GEY/sPff//9+eo23NrePTMF+ZXyzz//PP35559PHz9+/NzzCYdfv/FHIWv5Hol7rW1xJvfo/Pbbb/+2nI+sq1ufnZynjs/ZnIy1vPtff/31P3rfKlOQXyl//PHH0w8//PD0448/fu75VFz0aWuhvmeylg8fPnzuuQ3svn///umnn346tW1cnH1+C/bnezx0gjX+/vvvn+8+YU1i71N79+7d/31UuMVHMe39OZuT+zWexn/++efPd2+XKcivmF9++eXfwx/cSwgJ1CgEeYvpQi1J+p7MSs9Z5dFzVv2u9SH2c50EDgpe9K9j5q1zXOd+Nyfs5kK/YqPoHM2FAkau1xnM63WF7m/d9kfrOa1X3zXxw5Ftfbt+WMeKwskn0Nlnx9mKHnLx7WyOsaPCm7W/ZaYgv1Ikh2KiAEsKb3t5o+lDL0HyNuMzySPRFMEkXcYbNiJDT65TKDIn9tiPfsmXtytyrn1q9HRxoIseiWys/WA3c9m21u6j3zXbK+vaOy7R55PcjtiJrSZ217Vk3dFvX0CeD+SN/fXXX/9+tl9H8ROXljuy7T59q7/2bO0D3ezycy2k9Kxv1Dib4/4snrt9ektMQX6lSBQJKgEkhmRTrBSB4F4i582GTJJaEdQkvn4ya/KZDzqTZBI+c7XYRevXJ/kiz8f4QoZsMCb52x65+G88b6r6NDqiT1uLgD7j0amIdTEwn2+X4Cf7DVvm0kFv9KTfetF7oY8/4UvjR3fWubMN+vVHvtHfMQ/m6KdnLZZ8yBid4WxOxzyfgQ+r/FtjCvIrJcnlgKeAScJ+YzEmaSGhuiAkWcgrykdIoiS9OV0A6Mw9+12AwMcUM36k4Pvs4m8sBYSN+Mn/vO2RYSP02nas43R0kTJ2Nh/WZ01djJBi5NM6Este71oA9a9x/pL4hSPbPumwn2037AqyPvGFue5D9EKcxBNnc3oMfRZhLdHzVpmC/AqRCHlblAQSU3JLuE56hz9v0D4lcpLYNVljCqDPNfnRhS0F3z17bNNjXgpFF4kuMMb4jRQTDcbYoSfXiP/stB+go4vBCt2Za530dmF1v1tvE39W2OYP3Ylv94uTeeyua+FXHgxfEr9wZtu1tptn7V0sYT6bHRewT1Z/zkrsHM2BMY1v1kGu4Xf2/BHhf/YziPX6oN3FP5wWZIEzuZuNWI1iJyuRL71lPCcS+ugwvCXsjwS0J0Fs9GlBMkkQB0bMXCchksCwh0f7qD+FNDZyT7/71p9iQyY6jbVf/M8cuDaXTL/FGjfWvobWt6Pn8iO2wG/nt/t2mL+Li7XxV0ssYD/Ys77EJnvUY+Ha+DVHtsnSpa2xCuxkDnuRbz3g107f2ZweS1vriL5LMb9X+C5+wT6lDnY/cr46P8NpQYbA58kpWAwptDZ8pWXBIMO3eOrZ3ARgGL4WySKBJNgZipA82CXVo5IHwPcoimpEP2wfibxA7FAPd2MpyisXq9da+ZHiR2mzkz16o35OPAgkSB4Aw/C1JLnOilIK19Gb5iNj3WJwy6KsED/qgy2Fda2F4aggY1fIL1YvbwHrkyuFbw2ivpa1qfrO3pCj61K7ZsOiaxiG4RYoqmffps4K8q5enVYvvwOZsFb/KOoi6W2hZc31e+4tf9OdgjwMwy3Z/SrQnBVkrHX0tHp5s6VwxVOBoi60fjbQl+bN+tZffa4tyPH/UrvEbs60adNeZ9tx6SfZawpy/6pwWnX84W6njJH1j3rehNOXN+trfmNLEb3U+ilyxLwhD8NwS25akHe/H1NOSf+zlvyw3bJe5Xf/EuMlmYI8DMMtudlPFnnLzU8O7hXYXZHOTwCtWB/ZW7L+jj0Mw/CSqHMv/kc9RZgRwmlRfPQHvrSQgq6Iv3SB9IBofz0I3Pdb/DAMw3OTXwcu1UVtRTFfi/XhG/IwDC+HBJaM89JwvyiY1/wdbFdYr2FXyKcgfyN5Qq5PwPyrE5v1KPimwedb/M/KxjcyB5rts69/ksM3NXL989ijkW+g/S+QXDszvt1Zn3U+MnlDtKZbIWbfEredz9cWZef37Ow2qRm7MzwF+RnIZiTAPv2rk0dMqu/xU49Df+kPwAqWYvUcvtFxbfK8BLsk548YWGcK9qNjPdcUs69ht4fy7tI5usTOZ7l8DV5ornkBU4yPmIL8DNhET1UbmWRy309a/eTy9tObTE6/Q2ZTFZ5+ekZnCpL5rvN1p+fDG26+QkWfvsjFN9froSbrUPlsH/mQt1hj/Isv1hU7uwNJji5zffbXtPix8yXETuQ6NmKesU7G1eYaj+hy/6Xxi33yfNbHBzFCryl6mpZF1tB9DX3xOd9eEpOOvXXmWqNPOzs7MEc//dGBXm/WGXJG2ifE17R1TUf+9D72/u5inDXq0+Ivn8Q7+96FlZ4+D+sZjL/a6rMYt08vyRTkb8TG2lCHQXNt82x6HwiHKskZuUBOn42Prj4w9Ll3mIw7MDlUWhLGoQM9se2w0ucwk9OfQxw/g/skBxvkopN+icderslp5tDnmq2GfHSCb6uMcXJnsLcWt/hvbpIUfI9P4G/so+PzJfEznv1lIzHVz17mixu9kV8x3ljXTg76+cYefZnLP41dscl+JUau9cU/9zk7ZLIHxlffyXRcYj9nxVw6yNApXuknE7t9toJ+89ofcpnXe+X+KMa9R8Hc1pP9Z8/cyPcZvMZnMsZuwRTkbySFyYY5PDlkOdhwiIwFh3k9TO5bZqV15KA2OUjGHL7YhgOYw+lw8RH6cg1jOahZA9h2rbEbGewKZbOOu9YX1tgc0WsI+uKX6yTrapP+jof4rFwbv8BGbJvb+0l2jVNjTtNFaIUvCgpSWEKvs4sJuS4s632vp69XObT9oM8afLKfWLUsnUfrX/ecrujovTqLccchnOk5OoPX+KxvCvKDIJlsqJZDYGP1h04WByGHQH8KicKY/l1y9qEy7pod9uhhzzV9bNGZg0ZvDie5HC6yrmMvhYEs/bHX/tNpXnC98zc4zP0A6ASA8dg5Q7L1PMRfiA/9iE3r8NnxyFqMJSm/JH6h12V+rq2FrMbnHV0s0LFmly8hRUbjV+ygY88fDdblOmPiknnG2n5iaK2uyWUeX7LvTfvE78Qp/eLBhs/oavjQsct6s/bE/ijGro/2EOb2Hrae9Qxe4zO9+m/BFORvwOba0LWg6NOyiTbcZjsI5thgB8ahcK2RcWjIOEgrnXw+6U8BIu8+B4qOTvAkFRvkAp3mxF58JGM+efjko36fOcwgH7kdZOk1N0nS6Ms6jrBeOlaSXFoSE4lBx8NawX7G4ovPa+LX6M+67FtiQFZf+ncoEL3m2NjNI6uPT6s+fW03500sjMWGtbinx/5l3SCb/pyrxNLnbg36yCVeQYz1G+c3m20r9FmOLMhmfu53MdZPd+8hHa3HePbw7Axe4/Nu/1+KKcjDd8PBlwxd4HdImiTXayAPuLMH2XOiyAxfhyKdh8ctmII8fBfydnvp7dibzS2L163wEPJWtr6NPTd56OXtcbieWxdjTEEehmG4E6YgD8Mw3AlTkIdhGO6EKcjDMAx3whTkYRiGO2EK8jAMw50wBXkYhuEueHr6HzTlteN5NxGAAAAAAElFTkSuQmCC\"\u003e\u003c/p\u003e\n\u003cp\u003eThe proposed FSFL method achieved the best trade-off between performance and dimensionality reduction being capable of reducing significantly the feature set dimensionality while achieving slightly higher or comparable prediction performance with the rest competing algorithms. Specifically, the proposed FSFL technique reaches the highest accuracy (73.55%) at 21 selected features while the second-best accuracy (73.51%) was achieved by LBGM Emb at 87 features. This shows that the proposed FSFL technique results to a 76% smaller set of selected features compared to the second-best performing technique. On the other hand, the second-best performer with respect to dimensionality reduction was RF Emb with 73.36% accuracy achieved on a considerably larger feature subset with more than double features (43) compared to FSFL (21).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 3\u003c/strong\u003e. Comparative analysis of FS methods.\u003c/p\u003e\n\u003ctable border=\"1\" cellpadding=\"0\" cellspacing=\"0\" width=\"104%\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" width=\"17.02127659574468%\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"7.446808510638298%\"\u003e\n \u003cp\u003e\u003cstrong\u003eFSFL\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"7.446808510638298%\"\u003e\n \u003cp\u003e\u003cstrong\u003eVote FS\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e\u003cstrong\u003eRF Emb FS\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"12.76595744680851%\"\u003e\n \u003cp\u003e\u003cstrong\u003eLGBM Emb FS\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"11.702127659574469%\"\u003e\n \u003cp\u003e\u003cstrong\u003eSVM RFE FS\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e\u003cstrong\u003eLR RFE FS\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e\u003cstrong\u003eFilter MI FS\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"11.702127659574469%\"\u003e\n \u003cp\u003e\u003cstrong\u003eFilter \u003cbr\u003e\u0026nbsp;f-ANOVA \u003cbr\u003e\u0026nbsp;FS\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" width=\"17.02127659574468%\"\u003e\n \u003cp\u003e\u003cstrong\u003eMax\u003c/strong\u003e\u003cstrong\u003eimum \u003cbr\u003e\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eAcc\u003c/strong\u003e\u003cstrong\u003euracy (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"7.446808510638298%\"\u003e\n \u003cp\u003e73.55\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"7.446808510638298%\"\u003e\n \u003cp\u003e72.99\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e73.36\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"12.76595744680851%\"\u003e\n \u003cp\u003e73.51\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"11.702127659574469%\"\u003e\n \u003cp\u003e70.53\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e73.50\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e72.75\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"11.702127659574469%\"\u003e\n \u003cp\u003e73.44\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" width=\"17.02127659574468%\"\u003e\n \u003cp\u003e\u003cstrong\u003eNum\u003c/strong\u003e\u003cstrong\u003eber \u003cbr\u003e\u0026nbsp;o\u003c/strong\u003e\u003cstrong\u003ef\u003c/strong\u003e\u003cstrong\u003e\u0026nbsp;Selected\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eFeatures\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"7.446808510638298%\"\u003e\n \u003cp\u003e21\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"7.446808510638298%\"\u003e\n \u003cp\u003e76\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e43\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"12.76595744680851%\"\u003e\n \u003cp\u003e87\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"11.702127659574469%\"\u003e\n \u003cp\u003e96\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e60\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e91\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"11.702127659574469%\"\u003e\n \u003cp\u003e53\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" width=\"17.02127659574468%\"\u003e\n \u003cp\u003e\u003cstrong\u003eDR (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"7.446808510638298%\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"7.446808510638298%\"\u003e\n \u003cp\u003e+72%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e+51%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"12.76595744680851%\"\u003e\n \u003cp\u003e+76%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"11.702127659574469%\"\u003e\n \u003cp\u003e+78%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e+65%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"10.638297872340425%\"\u003e\n \u003cp\u003e+77%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" width=\"11.702127659574469%\"\u003e\n \u003cp\u003e+60%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eExplainability Results\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFigure 7a depicts how the features\u0026rsquo; impact shapes the output of the final model (RF) on the testing dataset. The features are sorted by the sum of SHAP value magnitudes over all testing subjects. Furthermore, the SHAP values are used to demonstrate the contribution of each risk factor (negative or positive) on the model\u0026rsquo;s output. Specifically, blue color represents low feature values, whereas red color represents high values, respectively. In particular, a high value of PO2ELGRISK (knee symptoms, risk factors, or both status) increases the probability of the subjects to be assigned to class KOA. Similarly to PO2ELGRISK, the higher the values of risk factors V00AGE, P02KSRG, P01BM1, V00RKFHDEG, P01WEIGHT, V00LKFHDEG, V00WTMACKG, V00BRDIAS, V00KPLKN1, and P02PA1, the more probable for subjects to belong to class KOA. The rest of the selected risk factors in Figure 7 have the opposite effect pushing the prediction output of the model to the class of healthy subjects. Figure 7b presents the SHAP global feature importance. The risk factors are sorted by the mean [|SHAP value|], which is the average impact on model output magnitude.\u003c/p\u003e\n\u003cp\u003eFigure 8 interprets locally the behavior of the model for the prediction output in a subject that suffers by KOA. P02ELGRISK (with a value of 2) and P01BMI (with a value of 29.8) push the predictions towards the class of KOA patients. Therefore, a high value of the aforementioned risk factors results to the increase of the output probability of the subject to be classified as KOA patient. \u0026nbsp; On the contrary, increase of the risk factors P02KSURG, V00RKFHDEG, V00KOOSQOL, and V00KOOSKPR lowers the probability of a subject to be classified as KOA. Since, our prediction score = 0.51 \u0026gt; base value = 0.49, this subject has been positively classified, i.e., class KOA status.\u0026nbsp;\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eHandling the multidimensional nature of the OAI dataset, a novel fuzzy ensemble FS methodology was designed, implemented and tested in this paper. Its main novelty lies on the combination of several well-known FS algorithms based on a properly designed fuzzy inference mechanism that effectively aggregates their outputs. The superiority of the proposed FS technique was demonstrated through a thorough comparative investigation that included several state-of-the-art algorithms coming from different FS families (filter, wrapper, embedded and hybrid).\u003c/p\u003e \u003cp\u003eThe proposed fuzzy FS methodology outperformed the aforementioned FS techniques achieving the best trade-off between dimensionality reduction and prediction accuracy. Working on a high-dimensional dataset of 643 features, twenty-one risk factors were selected for the objective of KOA diagnosis. Observing the nature of the selected risk factors, it was found that subject characteristics, symptoms, and physical exams are the most important risk factors contributing considerably to the KOA diagnosis. Overall, it was concluded that a combination of heterogeneous risk factors coming from different feature categories is needed for the effective diagnosis of KOA.\u003c/p\u003e \u003cp\u003eTo sanity check the AI models beyond mere performance and further quantify the relevance of the selected risk factors, a post hoc explainability analysis was also conducted using SHAP. As observed by SHAP, P02ELGRISK, P02KSURG, V00AGE, P01BMI and V00KOOSQOL are five risk factors that have a major impact to the prediction output, which are in line with the existing literature. Specifically, P02ELGRISK, that represents knee symptoms, is an important risk factor in the diagnosis of KOA, as it has been identified by Lespasio et al. [\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e]. The history of knee surgery (P02KSURG) has been recognised as an important risk factor of KOA by Katz et al. [\u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e], whereas the age of the subjects was also characterized as crucial in the occurrence of KOA and therefore was considered in the development of a predictive model for KOA diagnosis [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. The knee injury and osteoarthritis outcome (KOOS) is a well-known knee-specific instrument that has been widely employed to evaluate quality of life in patients with knee injuries and identify patients who are at risk of developing OA [\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e]. Moreover, high BMI is suggested to be a high-risk factor in the development of KOA. High BMI values lead to the increment of knee joint mechanical loading [\u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eAlthough the proposed FSFL technique selects a subset of risk factors with a significant dimensionality reduction compared to popular FS techniques, the application of a post-hoc explainability is still important in order to identify the contribution of the selected features to prediction output of the model. The use of explainability analysis algorithms for the interpretation of the ML models increases the understanding of the principle of operation of each ML model and reveal the interactions that shape the diagnosis outcome.\u003c/p\u003e \u003cp\u003eThe proposed methodology can be considered as computationally intensive; however, FS is considered here as an offline process and therefore the execution time does not play a crucial role. Future work will focus on the identification of easily measurable biomarkers and biomechanical parameters derived from musculoskeletal models, in combination with the already selected risk factors for the early diagnosis of KOA in the general population. Hence, to achieve this goal more advanced AI analytics tools in combination with the FSFL algorithm will be employed.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eTo enforce the development of more reliable, powerful, and non-invasive diagnostic tools, this study focuses on the identification and interpretation of the risk factors that contribute on the diagnosis of KOA. The proposed methodology is based on a novel fuzzy logic-based feature selection followed by learning algorithms and subsequently a post-hoc explainability analysis. The proposed technique aggregates the results of several FS algorithms (filter, wrapper and embedded ones), whereas fuzzy logic was employed to combine multiple feature importance scores thus leading to a more robust selection of informative features. The results showed that the presented methodology was capable to select a subset of risk factors that increase the performance accuracy of various ML models, compared to popular FS techniques. This was achieved with a significant decrease on the feature dimensionality (up to 78%). SHAP was finally applied to enhance our understanding of the rationale behind the decision-making mechanism of the selected ML model and the impact of the used risk factors on the prediction output.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis research was funded by the European Community\u0026rsquo;s H2020 Programme, under grant agreement No. 777159 (OACTIVE).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eInstitutional Review Board Statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eInformed Consent Statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Availability Statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eData from the osteoarthritis initiative (OAI) database (available upon request at https://nda.nih.gov/oai/).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflicts of Interest\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare no conflict of interest.\u003c/p\u003e"},{"header":"References","content":"\u003cp\u003e[1] Malanga G, Niazi F, Kidd VD, Lau E, Kurtz SM, Ong KL, Concoff AL (2020) Knee Osteoarthritis Treatment Costs in the Medicare Patient Population. \u003cem\u003eAmerican health \u0026amp; drug benefits 13\u003c/em\u003e(4) 144.\u003c/p\u003e\n\u003cp\u003e[2] Johnson VL, Hunter DJ (2014) The epidemiology of osteoarthritis. \u003cem\u003eBest practice \u0026amp; research Clinical rheumatology 28\u003c/em\u003e(1) 5-15. 10.1016/j.berh.2014.01.004\u003c/p\u003e\n\u003cp\u003e[3] Silverwood V, Blagojevic-Bucknall M, Jinks C, Jordan J, Protheroe J, Jordan K (2015) Current evidence on risk factors for knee osteoarthritis in older adults: a systematic review and meta-analysis. \u003cem\u003eOsteoarthritis Cartilage 23\u003c/em\u003e(4) 507-515. 10.1016/j.joca.2014.11.019\u003c/p\u003e\n\u003cp\u003e[4] Ackerman IN, Kemp JL, Crossley KM, Culvenor AG, Hinman, RS (2017) Hip and Knee Osteoarthritis Affects Younger People, Too. \u003cem\u003eJ Orthop Sports Phys Ther 47\u003c/em\u003e(2) 67-79. 10.2519/jospt.2017.7286\u003c/p\u003e\n\u003cp\u003e[5] Toivanen AT, Heli\u0026ouml;vaara M, Impivaara O, Arokoski JP, Knekt P, Lauren H, Kr\u0026ouml;ger H (2010) Obesity, physically demanding work and traumatic knee injury are major risk factors for knee osteoarthritis\u0026mdash;a population-based study with a follow-up of 22 years. \u003cem\u003eRheumatology 49\u003c/em\u003e(2) 308-314. 10.1093/rheumatology/kep388\u003c/p\u003e\n\u003cp\u003e[6] Mahir L, Belhaj K, Zahi S, Azanmasso H, Lmidmani F, El Fatimi A (2016) Impact of knee osteoarthritis on the quality of life. \u003cem\u003eAnnals of physical and rehabilitation medicine 59\u003c/em\u003e e159. https://doi.org/10.1016/j.rehab.2016.07.355\u003c/p\u003e\n\u003cp\u003e[7] Jack Farr I, Miller LE, Block JE (2013) Quality of life in patients with knee osteoarthritis: a commentary on nonsurgical and surgical treatments. \u003cem\u003eThe open orthopaedics journal 7\u003c/em\u003e 619. 10.2174/1874325001307010619\u003c/p\u003e\n\u003cp\u003e[8] Kohn MD, Sassoon AA, Fernando ND (2016) Classifications in brief: Kellgren-Lawrence classification of osteoarthritis. \u003cem\u003eClinical Orthopaedics and Related Research\u0026reg; 474\u003c/em\u003e(8) 1886-1893. 10.1007/s11999-016-4732-4\u003c/p\u003e\n\u003cp\u003e[9] Kokkotis C, Moustakidis S, Papageorgiou E, Giakas G, Tsaopoulos D (2020) Machine Learning in Knee Osteoarthritis: A Review. \u003cem\u003eOsteoarthritis and Cartilage Open\u003c/em\u003e 100069. https://doi.org/10.1016/j.ocarto.2020.100069\u003c/p\u003e\n\u003cp\u003e[10] Yoo TK, Kim DW, Choi SB, Oh E, Park JS (2016) Simple Scoring System and Artificial Neural Network for Knee Osteoarthritis Risk Prediction: A Cross-Sectional Study. \u003cem\u003ePLoS One 11\u003c/em\u003e(2) e0148724. 10.1371/journal.pone.0148724\u003c/p\u003e\n\u003cp\u003e[11] Lim J, Kim J, Cheon S (2019) A Deep Neural Network-Based Method for Early Detection of Osteoarthritis Using Statistical Data. \u003cem\u003eInternational journal of environmental research and public health 16\u003c/em\u003e(7) 1281. 10.3390/ijerph16071281\u003c/p\u003e\n\u003cp\u003e[12] Christodoulou E, Moustakidis S, Papandrianos N, Tsaopoulos D, Papageorgiou E (2019) \u003cem\u003eExploring deep learning capabilities in knee osteoarthritis case study for classification.\u003c/em\u003e Paper presented at the 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA). 10.1109/IISA.2019.8900714\u003c/p\u003e\n\u003cp\u003e[13] Moustakidis S, Christodoulou E, Papageorgiou E, Kokkotis C, Papandrianos N, Tsaopoulos D (2019) Application of machine intelligence for osteoarthritis classification: a classical implementation and a quantum perspective. \u003cem\u003eQuantum Machine Intelligence\u003c/em\u003e. 10.1007/s42484-019-00008-3\u003c/p\u003e\n\u003cp\u003e[14] Kwon SB, Han HS, Lee MC, Kim H C, Ku Y (2020) Machine learning-based automatic classification of knee osteoarthritis severity using gait data and radiographic images. \u003cem\u003eIEEE Access 8\u003c/em\u003e 120597-120603. 10.1109/ACCESS.2020.3006335\u003c/p\u003e\n\u003cp\u003e[15] Moustakidis S, Papandrianos N, Christodolou E, Papageorgiou E, Tsaopoulos D (2020) Dense neural networks in knee osteoarthritis classification: a study on accuracy and fairness. \u003cem\u003eNeural Computing and Applications\u003c/em\u003e 1-13. https://doi.org/10.1007/s00521-020-05459-5\u003c/p\u003e\n\u003cp\u003e[16] Hardi S, Triwiyono A (2020) \u003cem\u003eExpert System for Diagnosing Osteoarthritis with Fuzzy Tsukamoto Method.\u003c/em\u003e Paper presented at the Journal of Physics: Conference Series. 10.1088/1742-6596/1641/1/012107\u003c/p\u003e\n\u003cp\u003e[17] Pal JK, Ray SS, Pal SK (2017) Fuzzy mutual information based grouping and new fitness function for PSO in selection of miRNAs in cancer. \u003cem\u003eComputers in biology and medicine 89\u003c/em\u003e 540-548. 10.1016/j.compbiomed.2017.08.013\u003c/p\u003e\n\u003cp\u003e[18] Dai J, Chen J (2020) Feature selection via normative fuzzy information weight with application into tumor classification. \u003cem\u003eApplied Soft Computing 92\u003c/em\u003e 106299. https://doi.org/10.1016/j.asoc.2020.106299\u003c/p\u003e\n\u003cp\u003e[19] Lin Y, Hu Q, Liu J, Li J, Wu X (2017) Streaming feature selection for multilabel learning based on fuzzy mutual information. \u003cem\u003eIEEE Transactions on Fuzzy Systems 25\u003c/em\u003e(6) 1491-1507. 10.1109/TFUZZ.2017.2735947\u003c/p\u003e\n\u003cp\u003e[20] Jaganathan P, Kuppuchamy R (2013) A threshold fuzzy entropy based feature selection for medical database classification. \u003cem\u003eComputers in biology and medicine 43\u003c/em\u003e(12) 2222-2229. https://doi.org/10.1016/j.compbiomed.2013.10.016\u003c/p\u003e\n\u003cp\u003e[21] Wang C, Qi Y, Shao M, Hu Q, Chen D, Qian Y, Lin Y (2016) A fitting model for feature selection with fuzzy rough sets. \u003cem\u003eIEEE Transactions on Fuzzy Systems 25\u003c/em\u003e(4) 741-753. 10.1109/TFUZZ.2016.2574918\u003c/p\u003e\n\u003cp\u003e[22] Qian Y, Wang Q, Cheng H, Liang J, Dang C (2015) Fuzzy-rough feature selection accelerator. \u003cem\u003eFuzzy Sets and Systems 258\u003c/em\u003e 61-78. https://doi.org/10.1016/j.fss.2014.04.029\u003c/p\u003e\n\u003cp\u003e[23] Silva-Ram\u0026iacute;rez EL, Pino-Mej\u0026iacute;as R, L\u0026oacute;pez-Coello M, Cubiles-de-la-Vega MD (2011) Missing value imputation on missing completely at random data using multilayer perceptrons. \u003cem\u003eNeural Networks 24\u003c/em\u003e(1) 121-129. https://doi.org/10.1016/j.neunet.2010.09.008\u003c/p\u003e\n\u003cp\u003e[24] Pihera J, Musliu N (2014) \u003cem\u003eApplication of machine learning to algorithm selection for TSP.\u003c/em\u003e Paper presented at the 2014 IEEE 26th International Conference on Tools with Artificial Intelligence. 10.1109/ICTAI.2014.18\u003c/p\u003e\n\u003cp\u003e[25] Japkowicz N (2000) \u003cem\u003eLearning from imbalanced data sets: a comparison of various strategies.\u003c/em\u003e Paper presented at the AAAI workshop on learning from imbalanced data sets.\u003c/p\u003e\n\u003cp\u003e[26] Vergara JR, Est\u0026eacute;vez PA (2014) A review of feature selection methods based on mutual information. \u003cem\u003eNeural Computing and Applications 24\u003c/em\u003e(1) 175-186. https://doi.org/10.1007/s00521-013-1368-0\u003c/p\u003e\n\u003cp\u003e[27] Suchwalko A, Buzalewicz I, Podbielska H (2013). \u003cem\u003eIdentification of bacteria species by using morphological and textural properties of bacterial colonies diffraction patterns.\u003c/em\u003e Paper presented at the Videometrics, Range Imaging, and Applications XII; and Automated Visual Inspection. https://doi.org/10.1117/12.2020337\u003c/p\u003e\n\u003cp\u003e[28] Zhao J, Karimzadeh M, Masjedi A, Wang T, Zhang X, Crawford MM, Ebert DS (2019) \u003cem\u003eFeatureExplorer: Interactive feature selection and exploration of regression models for hyperspectral images.\u003c/em\u003e Paper presented at the 2019 IEEE Visualization Conference (VIS). 10.1109/VISUAL.2019.8933619\u003c/p\u003e\n\u003cp\u003e[29] Ding J, Shi J, Wu FX (2011) SVM-RFE based feature selection for tandem mass spectrum quality assessment. \u003cem\u003eInternational journal of data mining and bioinformatics 5\u003c/em\u003e(1) 73-88. 10.1504/ijdmb.2011.038578\u003c/p\u003e\n\u003cp\u003e[30] Ye Y, Liu C, Zemiti N, Yang C (2019) \u003cem\u003eOptimal feature selection for EMG-based finger force estimation using lightGBM model.\u003c/em\u003e Paper presented at the 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). 10.1109/RO-MAN46459.2019.8956453\u003c/p\u003e\n\u003cp\u003e[31] Mate Y, Somai N (2021) \u003cem\u003eHybrid Feature Selection and Bayesian Optimization with Machine Learning for Breast Cancer Prediction.\u003c/em\u003e Paper presented at the 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS). 10.1109/ICACCS51430.2021.9441914\u003c/p\u003e\n\u003cp\u003e[32] Gayathri B, Sumathi C (2015). \u003cem\u003eMamdani fuzzy inference system for breast cancer risk detection.\u003c/em\u003e Paper presented at the 2015 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC). 10.1109/ICCIC.2015.7435670\u003c/p\u003e\n\u003cp\u003e[33] Ram M, Najafi A, Shakeri MT (2017) Classification and biomarker genes selection for cancer gene expression data using random forest. \u003cem\u003eIranian journal of pathology 12\u003c/em\u003e(4) 339. PMID: 29563929 PMCID: PMC5844678\u003c/p\u003e\n\u003cp\u003e[34] Parisi L, Biggs PR, Whatling GM, Holt CA (2015) \u003cem\u003eA novel comparison of artificial intelligence methods for diagnosing knee osteoarthritis.\u003c/em\u003e Paper presented at the XXV congress of the international society of biomechanics. 10.13140/RG.2.1.4197.6163\u003c/p\u003e\n\u003cp\u003e[35] Ntakolia C, Kokkotis C, Moustakidis S, Tsaopoulos D (2020) \u003cem\u003eA machine learning pipeline for predicting joint space narrowing in knee osteoarthritis patients.\u003c/em\u003e Paper presented at the 2020 IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE). 10.1109/BIBE50027.2020.00158\u003c/p\u003e\n\u003cp\u003e[36] Kubkaddi S, Ravikumar K (2017) Early detection of Knee Osteoarthritis using SVM Classifier. \u003cem\u003eIJSEAT 5\u003c/em\u003e(3) 259-262.\u003c/p\u003e\n\u003cp\u003e[37] Long NP, Park S, Anh NH, Min JE, Yoon SJ, Kim HM, Lim J (2019) Efficacy of integrating a novel 16-gene biomarker panel and intelligence classifiers for differential diagnosis of rheumatoid arthritis and osteoarthritis. \u003cem\u003eJournal of clinical medicine 8\u003c/em\u003e(1) 50. 10.3390/jcm8010050\u003c/p\u003e\n\u003cp\u003e[38] Wong TT, Yeh PY (2019) Reliable accuracy estimates from k-fold cross validation. \u003cem\u003eIEEE Transactions on Knowledge and Data Engineering 32\u003c/em\u003e(8) 1586-1594. 10.1109/TKDE.2019.2912815\u003c/p\u003e\n\u003cp\u003e[39] Ghosh M, Sanyal G (2018) An ensemble approach to stabilize the features for multi-domain sentiment analysis using supervised machine learning. \u003cem\u003eJournal of Big Data 5\u003c/em\u003e(1) 1-25. https://doi.org/10.1186/s40537-018-0152-5\u003c/p\u003e\n\u003cp\u003e[40] Lundberg SM, Lee SI (2017) \u003cem\u003eA unified approach to interpreting model predictions.\u003c/em\u003e Paper presented at the Advances in neural information processing systems. https://dl.acm.org/doi/10.5555/3295222.3295230\u003c/p\u003e\n\u003cp\u003e[41] Nohara Y, Matsumoto K, Soejima H, Nakashima N (2019) \u003cem\u003eExplanation of machine learning models using improved Shapley Additive Explanation.\u003c/em\u003e Paper presented at the Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. 10.1145/3307339.3343255\u003c/p\u003e\n\u003cp\u003e[42] Kokkotis C, Moustakidis S, Giakas G, Tsaopoulos D (2020) Identification of Risk Factors and Machine Learning-Based Prediction Models for Knee Osteoarthritis Patients. \u003cem\u003eApplied Sciences 10\u003c/em\u003e(19) 6797. https://doi.org/10.3390/app10196797\u003c/p\u003e\n\u003cp\u003e[43] Ntakolia C, Kokkotis C, Moustakidis S, Tsaopoulos D (2021) Prediction of Joint Space Narrowing Progression in Knee Osteoarthritis Patients. \u003cem\u003eDiagnostics 11\u003c/em\u003e(2) 285. https://doi.org/10.3390/diagnostics11020285\u003c/p\u003e\n\u003cp\u003e[44] Lespasio MJ, Piuzzi NS, Husni ME, Muschler GF, Guarino A, Mont MA (2017) Knee osteoarthritis: a primer. \u003cem\u003eThe Permanente Journal 21\u003c/em\u003e. 10.7812/TPP/16-183\u003c/p\u003e\n\u003cp\u003e[45] Katz JN, Arant KR, Loeser RF (2021) Diagnosis and treatment of hip and knee osteoarthritis: a review. \u003cem\u003eJama 325\u003c/em\u003e(6) 568-578. 10.1001/jama.2020.22171\u003c/p\u003e\n\u003cp\u003e[46] Roos EM, Lohmander, LS (2003) The Knee injury and Osteoarthritis Outcome Score (KOOS): from joint injury to osteoarthritis. \u003cem\u003eHealth and quality of life outcomes 1\u003c/em\u003e(1) 1-8. 10.1186/1477-7525-1-64\u003c/p\u003e\n\u003cp\u003e[47] Cooper C, Snow S, McAlindon TE, Kellingray S, Stuart B, Coggon D, Dieppe PA (2000) Risk factors for the incidence and progression of radiographic knee osteoarthritis. \u003cem\u003eArthritis \u0026amp; Rheumatism: Official Journal of the American College of Rheumatology 43\u003c/em\u003e(5) 995-1000. 10.1002/1529-0131(200005)43:5\u0026lt;995::AID-ANR6\u0026gt;3.0.CO;2-1\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"physical-and-engineering-sciences-in-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"apes","sideBox":"Learn more about [Physical and Engineering Sciences in Medicine](http://link.springer.com/journal/13246)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/apes/default.aspx","title":"Physical and Engineering Sciences in Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"KOA Diagnosis, Machine Learning, Clinical data, Explainability, Feature selection ","lastPublishedDoi":"10.21203/rs.3.rs-777000/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-777000/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eKnee Osteoarthritis (ΚΟΑ) is a degenerative joint disease of the knee that results from the progressive loss of cartilage. Due to KOA\u0026rsquo;s multifactorial nature and the poor understanding of its pathophysiology, there is a need for reliable tools that will reduce diagnostic errors made by clinicians. The existence of public databases has facilitated the advent of advanced analytics in KOA research however the heterogeneity of the available data along with the observed high feature dimensionality make this diagnosis task difficult. The objective of the present study is to provide a robust Feature Selection (FS) methodology that could: (i) handle the multidimensional nature of the available datasets and (ii) alleviate the defectiveness of existing feature selection techniques towards the identification of important risk factors which contribute to KOA diagnosis. For this aim, we used multidisciplinary data obtained from the Osteoarthritis Initiative database for individuals without or with KOA. The proposed fuzzy ensemble feature selection methodology aggregates the results of several FS algorithms (filter, wrapper and embedded ones) based on fuzzy logic. The effectiveness of the proposed methodology was evaluated using an extensive experimental setup that involved multiple competing FS algorithms and several well-known ML models. A 73.55 % classification accuracy was achieved by the best performing model (Random Forest classifier) on a group of twenty-one selected risk factors. Explainability analysis was finally performed to quantify the impact of the selected features on the model\u0026rsquo;s output thus enhancing our understanding of the rationale behind the decision-making mechanism of the best model.\u003c/p\u003e","manuscriptTitle":"Explainable Machine Learning for Knee Osteoarthritis Diagnosis Based on a Novel Fuzzy Feature Selection Methodology","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2021-08-10 21:19:11","doi":"10.21203/rs.3.rs-777000/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2021-08-08T12:13:04+00:00","index":0,"fulltext":""},{"type":"reviewersInvited","content":"","date":"2021-08-08T06:09:21+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"Physical and Engineering Sciences in Medicine","date":"2021-08-03T21:44:17+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2021-08-03T08:42:08+00:00","index":"","fulltext":""},{"type":"submitted","content":"Physical and Engineering Sciences in Medicine","date":"2021-08-02T12:34:59+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"physical-and-engineering-sciences-in-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"apes","sideBox":"Learn more about [Physical and Engineering Sciences in Medicine](http://link.springer.com/journal/13246)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/apes/default.aspx","title":"Physical and Engineering Sciences in Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"b48e3b3d-a50a-4004-911a-5e3abe379630","owner":[],"postedDate":"August 10th, 2021","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":6364286,"name":"Biomedical Engineering"},{"id":6364287,"name":"Biotechnology and Bioengineering"},{"id":6364288,"name":"Nuclear Medicine \u0026 Medical Imaging"}],"tags":[],"updatedAt":"2022-01-31T10:59:06+00:00","versionOfRecord":{"articleIdentity":"rs-777000","link":"https://doi.org/10.1007/s13246-022-01106-6","journal":{"identity":"physical-and-engineering-sciences-in-medicine","isVorOnly":false,"title":"Physical and Engineering Sciences in Medicine"},"publishedOn":"2022-01-31 10:59:06","publishedOnDateReadable":"January 31st, 2022"},"versionCreatedAt":"2021-08-10 21:19:11","video":"","vorDoi":"10.1007/s13246-022-01106-6","vorDoiUrl":"https://doi.org/10.1007/s13246-022-01106-6","workflowStages":[]},"version":"v1","identity":"rs-777000","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-777000","identity":"rs-777000","version":["v1"]},"buildId":"rHA-KDH7Qsr4HCuvH75dn","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}