GeoDetect Revisited: A Leakage-Aware Analysis of Predictive-Entropy Text Classification
Year: 2026
Type: preprint
Abstract: Distinguishing human-authored from synthetic text is difficult when dataset construction introduces shortcuts. We revisit GeoDetect as a supervised, entropy-only pipeline and checks 4,758 recorded passages. Normalized deduplication removes 1,002 repeated rows, leaving 2,404 human-labeled and 1,352 AI-labeled unique passages. Punctuation is absent from every human-labeled passage and present in every AI- labeled passage. We therefore distinguish an input artifact condition from a primary condition with punctuation removed consistently from both classes. A frozen DistilGPT2 scorer produces 14 predictive- entropy statistics, followed by an XGBoost classifier. TF-IDF similarity components are kept intact across development and holdout partitions. On 888 group-isolated holdout passages, the primary model achieves 88.06% accuracy, 87.09% balanced accuracy, 0.9877 AUROC, and a 1.86% human false-positive rate. Pooled five-fold grouped development AUROC is 0.9739; exact word-count matching yields 0.9629. Length and surface-style controls attain higher holdout accuracy than the entropy model. These results characterize separation within differently constructed corpora, including templated synthetic material and human text of unverified provenance; they do not establish general AI-authorship detection. The contribution is a reproducible dataset analysis, a dependence-aware evaluation, and an explicit analysis of preprocessing, threshold, and source-confounding failures, rather than a new geometric detector.
Author: Gogulapati, Mitra Varun
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225828754
Cites:
Cited by:
Related to: