Work

Machine Learning versus Simple Rule-Based Baselines for Phishing Website Detection: Effects of Domain-Grouped Evaluation and Feature Sources

Year: 2026
Type: article
Abstract: Phishing detectors are often judged by random-split accuracy on public benchmarks, against weak baselines or none. Using the Hannousse and Yahiouche dataset (11,430 URLs, 87 features), we examine how much machine learning (ML) improves on simple rule-based baselines, how much of that accuracy survives when related URLs are kept apart, and which feature groups carry it. We compare a six-rule URL heuristic, a domain blacklist, a one-split decision stump and shallow decision trees with Logistic Regression, Random Forest and XGBoost, using repeated 5-fold cross-validation (15 folds) with random folds and with class-balanced folds grouped by registered domain. XGBoost on all features reached 97.08 ± 0.32% accuracy with random folds and 95.17 ± 1.06% with domain-grouped folds, against 64.3% for the rule heuristic and 78.1% for the blacklist (random folds only; zero recall by construction in grouped folds). The strongest simple baseline, a one-split stump on the external feature google_index, reached 86.5%. Domain grouping lowered tree-model accuracy by 1.6 to 1.9 points with all features and by 3.9 to 4.2 points with URL-only features, and reduced XGBoost's true-positive rate at 1% false positives from 92.7% to 83.1%. External-service features were the most informative group, and URL-only XGBoost (87.5% on grouped folds) was not significantly better than the stump. A dataset audit shows sampling artifacts. ML clearly helps, but part of the apparent gain reflects baseline choice, dataset sampling and external lookups
Language: English
Open Access status: green
OpenAlex ID: W7236296044
FWCI:
Cites:
Cited by:
Related to: 10