Auditable Evaluation Protocol for Spam Detection Under Bounded Paraphrase Attacks
Year: 2026
Type: preprint
Abstract: Spam-filter scores are difficult to interpret when split construction, attack denominators, and candidate validity are not retained. We evaluate an auditable RMF-PC protocol using word and character TF-IDF views, structural cues, calibrated fusion, and an optional paraphrase-consistency gate. The protocol freezes data, fitting, attacks, predictions, and review decisions; separates deterministic L1 rewrites from human-adjudicated L2/L3 candidates; and keeps exact-deduplicated Track A distinct from group-near-deduplicated Track B. On Track A (seed 42), gated RMF-PC achieved clean F1 97.937%, L1 F1 98.117%, and L1 spam FNR 1.971% (11/558). Its FNR difference from linear SVM was -0.538 percentage points (exact McNemar p=0.5078; paired-bootstrap 95% interval -1.613 to +0.538), so superiority is unresolved. On Track B, the same comparison had a raw p=0.0215 but a Holm-adjusted p=0.0859, showing that cohort policy can affect an inferential conclusion. Two reviewers and adjudication retained 38 L2 and 55 L3 candidates; observed misses were 1/38 and 0/55. Enron and third-party Zenodo-curated TREC tracks show corpus sensitivity and fixed 5%/10% adaptation results. Findings are limited to the retained local protocols.
Source: Preprints.org
Language: English
Topic: Spam and Phishing Detection
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225348463