Comparative Analysis of Text Classification Techniques for Phishing and Smishing Detection Using Term Frequency–Inverse Document Frequency and Gemini Sentence Embeddings
Year: 2026
Type: article
Abstract: This study presents an approach to phishing and smishing detection using text classification algorithms and two feature extraction techniques: term frequency–inverse document frequency (TF-IDF) and Gemini sentence embeddings. The research uses a curated, self-collected dataset of 27,000 messages, including 2,000 synthetic records to increase linguistic diversity. With TF-IDF, the support vector classifier (SVC) achieved a training accuracy of 97.85% and a test accuracy of 89.37%, indicating a generalization gap. Multilayer perceptron (MLP) and stochastic gradient descent classifiers achieved test accuracies of 88.28% and 86.06%, respectively. Sentence embeddings showed stronger performance, with SVC achieving 94.69% test accuracy, followed by MLP at 92.70% and K-nearest neighbors (KNN) at 92.35%. The decision tree classifier showed comparatively lower performance, with test accuracies of 78.04% using TF-IDF and 64.04% using sentence embeddings. Overall, sentence embeddings outperformed TF-IDF in the evaluated experimental setting. Unlike studies evaluating traditional features or semantic embeddings separately, this work provides a controlled comparison across feature representations, classifiers, and communication channels (email and SMS). The study examines generalization behavior, model stability, and feature representation effects beyond peak accuracy, contributing a structured evaluation framework for natural language processing-based cybersecurity research.
Institution: Shivaji University
Language: English
Topic: Spam and Phishing Detection
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7225174231