Recursive Synthetic Training Conditions and Language Model Performance Degradation Across Published Text Generation Benchmark Experiments
Year: 2026
Type: article
Abstract: This analysis examines whether assessments of language model resilience depend on decoding configuration and measurement scale, addressing international debates on recursive synthetic training. A descriptive reanalysis uses 24 published WikiText 2 perplexity endpoints from 2025, forming 12 matched configurations across GPT 2, SmolLM2, and six decoding strategies at generations zero and nine. Paired absolute changes, percentage changes, ratios, means, and medians reveal deterioration in every configuration, with percentage increases ranging from 10.24% to 513.83%. SmolLM2 records a smaller mean absolute increase than GPT 2, but a larger mean percentage increase, while its median deterioration remains lower on both scales. Different baseline denominators and concentrated deterioration under greedy decoding and beam search explain this aggregation reversal. Sampling strategy rankings also differ between models, and temperature sampling reverses their comparison across measurement scales. These findings qualify interpretations of resilience as an intrinsic model property by demonstrating configuration and measurement dependence. International benchmarking should report matched baselines, absolute and proportional changes, and distributional summaries to support transparent configuration selection.
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225454848