Robust Feature Selection for Imbalanced Tabular Datasets Using Ensemble Gradient Boosting

Abstract

Imbalanced tabular datasets remain a persistent obstacle for supervised learning pipelines deployed in domains such as fraud detection, industrial defect screening, and rare disease diagnosis, where minority class instances carry disproportionate operational significance. This study proposes a feature selection framework that couples recursive elimination with an ensemble of gradient boosting learners trained under class-aware resampling, aiming to identify predictive variables that remain stable across resampled folds rather than variables that merely correlate with majority class noise. We evaluate the approach across several benchmark tabular collections spanning finance, manufacturing, and healthcare, comparing selected feature subsets against filter-based and wrapper-based baselines using precision-recall area, minority class recall, and selection stability indices. Results indicate that ensemble-guided selection consistently yields smaller, more interpretable feature subsets while preserving minority class detection performance relative to using the full feature space. We further examine how resampling ratio and boosting depth interact with selection stability, offering practical guidance for practitioners tuning these hyperparameters jointly rather than independently. The findings suggest that treating feature selection and imbalance correction as a joint optimization problem, rather than sequential preprocessing steps, produces more robust and auditable machine learning pipelines for high-stakes tabular prediction tasks.