Leveraging Feature Sensitivity and Relevance: A Hybrid Feature Selection Approach for Improved Model Performance in Supervised Classification

Many feature selection algorithms primarily give importance to identifying relevant features and eliminating redundant features. This hybrid work determines the significant features, based on the estimated feature sensitivities and the degree of relevance between the feature and target outcome. The majority of works currently in existence employ mutual information (MI) to calculate the degree of information between two variables. By scaling the range of the MI to [0,1], Symmetrical Uncertainty (SU) can be viewed as the normalized MI. In this proposed work, Symmetrical Uncertainty-Relevance (SU-R) is used to measure the relevance between each feature and the target outcome. Less significant features are iteratively eliminated starting from discarding the least ranked feature identified using the combination of SU-R and PFS Analysis.To evaluate how well our proposed method identifies important features, model’s performance using metrics like F1 score and accuracy. This hybrid work identified the best 450 significant features out of 754 in the Parkinson’s disease dataset, and the top 150 features out of 562 in the smart phone dataset. The efficacy of the SVM classifier with the selected number of significant features with the proposed hybrid PF and SU-R technique outperforms the SVM when applied with existing feature selection met