Work

Grow, Don't Restart: Compact Station World Models, Grown from 1M to 24M Parameters, Beat TimesFM and Chronos on WEATHER-5K

Year: 2026
Type: preprint
Abstract: Time-series foundation models with up to 205M parameters are increasingly used zero-shot for weather-station forecasting. We test whether a compact, task-trained world model that is grown rather than restarted can beat them under paired statistical tests. Atlas-5K is a station-graph model: each station reads its last 48 hours together with its 31 nearest neighbours, the model encodes them into a latent state, and one shared transition with geometry-biased attention rolls the state forward. We grow an adopted 0.97M-parameter Atlas twice with exact, function-preserving maps: to 9.64M (width tiled with zero-sum perturbations, new blocks as the identity), then to 24.1M (each block's hidden layer widened 2.5× with zero-output units, one block added). Every rule and numeric prediction was committed to a public repository before the run it governs. On WEATHER-5K at 24 hours we compare both grown models, on identical windows at 709 stations, with nine foundation-model configurations (TimesFM 2.5 and 3.0, Chronos-2 univariate and multivariate, Chronos-Bolt, TiRex, Sundial, Time-MoE and TTM) and persistence. Both models win all 50 model–variable comparisons (station-block bootstrap and Diebold–Mariano, Holm-corrected, α=0.01) on the 2023 test year and on 19 months of NOAA observations from 2024–25 that Atlas never saw. The 24.1M model's standardised error is 17.0% below the best foundation model, TimesFM 3.0, and its per-variable error is 9–48% below every foundation model. On the full 2023 test it has lower error than the benchmark's best published model, PhysicsFormer (19.3M), on four of five variables, but higher error on sea-level pressure, where ECMWF's operational forecast remains far ahead. Size bought little: 10× the parameters gave -1.95% validation error and a further 2.5× gave -1.03%, with no significant gain on pressure. We report every prediction, including the two that failed, and release the code, rules, logs and weights. Paper 5 of 5 in ParadoxAI Lab's series on compact physical world models (Atlas-5K). Code, rules, logs and weights: https://github.com/paradoxailab/paradox-world-model. Contact: [email protected]
Language: English
Open Access status: green
OpenAlex ID: W7236832597
Cites:
Cited by:
Related to: 9