Does Graph Structure Improve AI-Generated Text Detection? A Controlled Audit with Tuned Baselines
Year: 2026
Type: preprint
Abstract: Graph neural network (GNN) detectors of AI-generated text report state-of-the-art accuracy, and the architectural claim that graph topology captures signals invisible to sequential classifiers is now common in the literature. We report the complete lifecycle of one such detector, from construction to falsification. AEGIS, a graph attention network over k-nearest-neighbour text graphs with multi-signal edge weights, reaches 98.50% accuracy on scientific abstracts with only 73K trainable parameters, outperforming five established baselines. We then subject our own system to the evaluation its headline number deserves: generation-time length control, format normalization, leakage-free splits, a tuned frozen-embedding probe as the calibrated null, and a 10 × 10 cross-generator transfer study over three seeds. Under this protocol the graph's apparent contribution collapses from +0.240 AUROC over a surface baseline to −0.0004 AUROC against the tuned probe, inside the measured replication noise floor, and the two detectors' transfer geographies are near-identical (ρ = 0.986). A single untuned regularization default manufactured a +0.027 phantom advantage. The structural differences the graph encodes are real, but they are a re-encoding of information already present in the embeddings. We distil the audit into a practical checklist for evaluating architectural claims in text detection, and argue that accuracy numbers and mechanism claims must be validated separately.
Source: Research Square
Institution: University of Hassan II Casablanca
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225918633
