Work

A technical framework for HR-driven imbalanced classification: pipeline architecture, oversampling algorithms, ablation analysis, and threshold engineering for data scientist job-change prediction

Year: 2026
Type: article
Abstract: Predictive modelling of employee job-change propensity from HR survey data presents a canonical cluster of machine-learning engineering challenges: structured mixed-type inputs with systematic missingness, moderate class imbalance, and self-reported labels susceptible to noise. This paper presents the technical design and implementation of an end-to-end HR analytics pipeline applied to the HR Analytics: Job Change of Data Scientists dataset ( \(N = 19{,}158\) ; positive rate \(\approx 24.9\%\) ). We formalise the leak-safe imbalanced-learn Pipeline architecture that confines synthetic oversampling strictly to training folds and empirically quantify—via a controlled leaky-versus-clean experiment—that naive pre-split oversampling inflates 5-fold cross-validated AUC by 0.014–0.018 units for SMOTE-family methods, a margin sufficient to reverse configuration rankings. Nine oversampling algorithms are instantiated and evaluated across 90 model \(\times\) sampler configurations. A systematic ablation study of the feature engineering pipeline identifies missingness indicator construction ( \(\Delta \textrm{AUC} = -0.008\) when removed) and out-of-fold smoothed target encoding ( \(\Delta \textrm{AUC} = -0.009\) ) as the two highest-value components. HistGradientBoosting on the unaugmented training distribution achieves the highest holdout ROC-AUC of 0.816 [95% CI 0.800, 0.832]; all pairwise sampler AUC differences fall within this \(\pm 0.016\) estimation band and are not statistically significant. With threshold optimisation applied uniformly across configurations, performance differences between the best unresampled and best oversampled alternatives are within the confidence interval (F1 \(= 0.644\) vs. 0.648; \(\Delta = 0.004\) , overlapping bootstrap CIs), though oversampled configurations achieve meaningfully higher recall (73.6% vs. 78.1%). Generalizable engineering recommendations are distilled for HR classification tasks with high-cardinality categoricals, systematic missingness, and self-reported labels.
Language: English
Open Access status: gold
Estimated APC paid: $2,690
OpenAlex ID: W7233609381
FWCI:
Cites: 1
Cited by:
Related to: