Empirical Analysis of Model Selection and Augmentation Limits on Small Tabular Datasets
Year: 2026
Type: preprint
Abstract: Machine learning pipelines often fail when applied to small tabular datasets (N <= 150) with high feature-to-sample ratios (P/N >= 0.1). While practitioners frequently default to tree ensembles like Random Forests or oversampling techniques like SMOTE, these methods degrade under severe sample sparsity. In this work, we systematically evaluate model stability across three distinct benchmark datasets using Stratified 5-Fold Cross-Validation. Our experiments show that margin-based classifiers (specifically RBF SVMs) outperform Random Forests by 5.7% in weighted F1-score when P/N = 0.600. Additionally, we show that standard SMOTE oversampling introduces label noise in sparse feature spaces, whereas local Gaussian jittering maintains boundary stability. We synthesize these findings into a practical Data Geometry Scanner that inspects basic dataset properties to recommend suitable model families prior to training.
Author: Singh, Anantdeep
Institution: Punjab Technical University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W7204123580
Cites:
Cited by:
Related to: