Behaviorally Clean, Distributionally Contaminated: A Detection Boundary for Language Models
Year: 2026
Type: preprint
Abstract: A language model can produce no consistent signal under the behavioral audits we evaluate while failing distributional ones. We train small character-level autoregressive models (a tokenizer-free control regime) on a single-language corpus with three controlled training-data implants (foreign-marker, in-vocabulary, and hidden-trigger), each dormant under ordinary sampling, and show that their presence persistently shifts the per-token conditional distribution. The Neyman–Pearson log-likelihood-ratio test against a matched clean reference detects this shift decisively; emission-based and reference-free probes do not. In the evaluated setting, the detector’s effective power is governed by reference fidelity: against a matched twin the null is exactly zero and the implant is separated at tens of sigma, whereas against an independently seeded clean model basin variation alone produces a signal on the same order as the implant, large enough to mask or mimic it. Two further findings sharpen the boundary. Shuffling the token sequence before scoring destroys and reverses the signal, localizing it to conditional structure rather than marginal token frequencies. Clean-data distillation substantially transfers the teacher’s global on-support distributional fingerprint across all three implant families, while multi-token behavioral content does not transfer and single-token content transfers only partially. We name this pattern information-stratified transfer. This paper is not a detection-method proposal. It demonstrates a detection boundary: the conditions under which the tested detectors succeed, and the training-variation regime they collapse into when those conditions are removed. For the methods evaluated here, decisive audit capability for hidden training-time contamination is concentrated at whoever holds the training lineage; without such access, an auditor is pushed toward a regime in which implant signal and basin noise are quantitatively comparable.
Author: Irrgang, Claudio
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7154723325
Cites:
Cited by:
Related to: