Leakage-Aware Multi-Class Email Classification Using Duplicate-Aware Evaluation and Automated Machine Learning
Year: 2026
Type: article
Abstract: Reliable evaluation is important in multi-class email classification because aggregated text datasets may contain duplicated messages or overlapping content that can affect separation between model-development and evaluation data. This study presents a leakage-aware framework for classifying emails into Ham, Phish, and Spam. The framework integrates data cleaning, duplicate auditing, removal of conflicting duplicate groups, group-aware partitioning, training-only feature construction, stratified group cross-validation, locked-test evaluation, error analysis, and bootstrap confidence intervals. The downloaded dataset contained 365,448 records; after cleaning and removing conflicting groups, 365,403 remained. A five-fold StratifiedGroupKFold procedure produced a locked test set of 73,038 messages with zero duplicate-group overlap. Word-level TF-IDF with LinearSVC was used as the primary classifier, while a complementary H2O AutoML branch used 128-component SVD. LinearSVC achieved 98.00% accuracy and 97.32% macro F1 on the locked test set. The findings emphasize the importance of controlling duplicate structure and information flow when evaluating multi-class email classifiers
Source: Al-Farooq Journal of Sciences
DOI: 10.65405/9pvbat43
Authors: Mohammed A Almjbry, Issa M Almagrhe
Language: English
Topic: Spam and Phishing Detection
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7214768790