Benchmarking Classical Machine Learning and Deep Learning Models on the Cleveland Heart Disease Binary Classification Dataset
Year: 2026
Type: article
Abstract: Many high-stakes applications relied on binary classification, yet practitioners lacked equitable, methodical benchmarks that evaluated diverse model families under identical conditions. This study used the Cleveland Heart Disease tabular dataset to conduct a controlled, empirical benchmark of a regularised linear model (Ridge Classifier), tree-based ensembles (XGBoost, Random Forest), and a hybrid deep learning architecture combining Bidirectional Long Short-Term Memory and Convolutional Neural Network layers (BiLSTM-CNN) on a mildly imbalanced binary classification task. A strict protocol was applied, comprising uniform pre-processing, Synthetic Minority Oversampling Technique (SMOTE) applied to training folds only, a hybrid hold-out design with internal five-fold stratified crossvalidation on the training set (70%), and evaluation using accuracy, positive-class precision, recall, F1 score, and area under the curve (AUC) on a fixed test set (15%). Cross-validation results showed that no single model dominated every metric: the Ridge Classifier unexpectedly achieved the best AUC (0.8945), indicating strong ranking ability; Random Forest and XGBoost achieved the highest accuracy (81.67%); and the BiLSTM-CNN hybrid achieved the highest recall and F1 score (0.8478 and 0.8365, respectively). These findings indicated that, for this dataset, classical and ensemble approaches remained competitive with the more complex hybrid deep learning model, suggesting their continued relevance for similar small-scale, tabular binary classification tasks.
Institution: University of South Africa
Language: English
Subfield: Health Information Management
Field: Health Professions
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $32
OpenAlex ID: W7223330211