Performance Analysis of Transformer-based Language Models for Detection of Phishing Emails
Year: 2026
Type: dissertation
Abstract: This thesis evaluates the performance and adversarial robustness of four transformer-based language models; BERT, RoBERTa, DistilBERT, and GPT-2 which were fine-tuned for binary phishing email classification. Two preprocessing configurations (Basic and Advanced) and five training data fractions are systematically compared using a temporally partitioned dataset that separates pre-2015 training data from 2015–2024 deployment data, simulating real-world generalization across evolving phishing tactics. DistilBERT with Advanced preprocessing achieves the best overall performance, reaching 97.26% accuracy and a weighted F1-score of 0.972. The study further conducts adversarial robustness evaluation using four attack strategies from the TextAttack framework namely; DeepWordBug, Pruthi, TextBugger, and TextFooler which found out that character-level attacks achieve success rates of up to 22.44%. Finally, applying Advanced preprocessing to adversarially perturbed inputs recovers post-attack accuracy to near-clean-data levels across all models, establishing preprocessing as a deployable defense mechanism rather than merely a data cleaning step.
Source: Purdue
Author: Arannonu, Chidozie Princewill
Institution: Purdue University System
Language: English
Topic: Spam and Phishing Detection
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7160619207