Work

the em—dash fails to em–erge: a registered null result for U+2014 in Polish and an exploratory function-word shift in 60,275 Sejm interpellations, 2015–2025

Year: 2026
Type: preprint
Abstract: Since the release of ChatGPT (30 November 2022) a belief has spread that the em-dash "—" reveals text written with the help of a language model. This preregistered study tested that claim on 60,275 single-author written parliamentary questions (interpelacje) submitted to the Polish Sejm between 2015 and 2025. The decision rule was registered on OSF before the data were seen.The cue does not work in Polish. The registered hypothesis was not confirmed: in the decisive cell the share of documents containing an em-dash rose by 1.73 percentage points, but the confidence interval of that difference includes zero (−4.44 to +8.07 pp), so the second condition of the two-part decision rule was not satisfied, and three registered disconfirmation conditions fired. Follow-up tests point to the cause: models writing in Polish, and machine-translation flows, put a shorter dash there – the en-dash "–". The marker is anglocentric and is not suitable for Polish.An exploratory analysis measured something else: the frequency of Polish closed-class words (conjunctions, prepositions, particles) that a writer does not choose consciously. The reference point was an independently published corpus of Polish human-written and model-written texts, frozen together with its reading rule before measurement. From 2024 the distribution of these words moves toward the profile of the models (rank correlation rho = +0.368; 95% CI +0.228 to +0.461), while at a placebo date – the 2019 change of parliamentary term, where no such effect is expected – it does not move at all (−0.070; difference +0.438, +0.221 to +0.611). The shift survives four term-change controls.The change is small but widespread, and that is the correct description of its strength. Inside a single parliamentary office it is barely visible (median rho +0.125), yet it points the same way in 61 of 84 offices that could be measured separately (72.6%, one-sided p = 2·10⁻⁵), against 53.8% at the control date, which is what chance produces.There is no sudden jump at the date of the ChatGPT release. The change becomes visible in 2024 and unambiguous in 2025, and the shape of the transition (ramp versus late step) is not identified. The result describes a shift across a whole population of documents and allows no statement about the authorship of any single document or about any individual member of parliament. It does not establish a cause: changes in office staff, in subject matter and in editing practice remain possible explanations alongside tools. An independent reimplementation, with no access to the author's code, reproduced the corpus, the deduplication, the author panel and the headline analyses from raw HTML down.
Language: English
Open Access status: green
OpenAlex ID: W7204185173
Cites:
Cited by:
Related to: 10