Pre-registration of a replication of the LTSF-Linear versus Transformer comparison, with a matched tuning budget (arXiv 2205.13504)
Year: 2026
Type: article
Abstract: Frozen pre-registration (3 Nov 2026) of a private replication of the claim of Zeng et al. (arXiv 2205.13504; code cure-lab/LTSF-Linear, commit 0c113668) that LTSF-Linear models outperform the Transformer forecasters FEDformer, Autoformer and Informer in all cases and often by 20-50 percent. Scope: ETTh1 and ETTh2 at horizons 96, 192, 336, 720 (8 cells). Two verdict lines: arm R (the authors' released scripts, compared with the printed Transformer numbers and with an independent Transformer reproduction control) and arm P (the same number of tuning configurations, K = 8, for each of six models, look-back included, selection by validation MSE). Pre-set bounds (median improvement interval lower end above 6.8 percent, half the printed median over these cells; at most 1 of 8 cells wrong-side), controls that can fire (Repeat baseline reproduces the printed numbers within 1 percent, zero-predictor, reproduction and budget-parity controls). Contains the decision code with its planted-effect table, job design, harness, extraction of the paper's Table 2, cost estimate and the run kernel. Deposited before any confirmatory run. Outcome wording: Replicated / Did not replicate / Inconclusive under protocol X.
Author: Goodman, John
Language: English
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225158210