FS-GKM: A clustering-driven feature selection framework for enhanced supervised learning performance

Abstract

Feature selection is a fundamental process in machine learning that involves identifying and selecting the most informative features from a dataset to be used in model training. This step is crucial, as it can significantly enhance model performance, mitigate overfitting, reduce computational complexity, and improve the interpretability of the resulting models. By eliminating irrelevant or redundant features, feature selection facilitates the development of more efficient and accurate predictive models. Therefore, the development of new feature selection algorithms is of great importance. In this study, a novel feature selection algorithm—Feature Selection using Global k-Means (FS-GKM)—is proposed. In this study, a novel feature selection algorithm—Feature Selection using Global k-Means (FS-GKM)—is proposed. The method leverages the global k-means clustering algorithm to group features based on their similarity. Subsequently, the algorithm assesses the discriminative power of features by analyzing class density distributions within each cluster. This clustering-based evaluation enables the identification of features that contribute most significantly to class separability. To validate the effectiveness of the FS-GKM method, a comprehensive experimental analysis was conducted using 13 benchmark datasets. Each dataset was subjected to dimensionality reduction through 13 different feature selection techniques, and the resulting feature subsets were evaluated using four distinct classifiers. The proposed FS-GKM algorithm achieved superior performance in 40 out of 52 comparative cases, demonstrating its robustness and effectiveness across diverse scenarios.