BQEB FoundationBench: A Reproducibility-First Benchmark Framework for Foundation Models in Energy Forecasting
Year: 2026
Type: preprint
Abstract: A new class of pretrained time-series forecasting models — Chronos, TimesFM, Moirai, TimeGPT, Lag-Llama, and Tiny Time Mixers (TTM) among them — now offers zero-shot and few-shot forecasting capability without per-dataset training, a genuinely new operating mode for energy forecasting that existing BQEB benchmark infrastructure was not built to evaluate; strong supervised architectures such as PatchTST remain relevant as a reference baseline rather than as a member of this new class. This paper introduces BQEB FoundationBench, a reproducibility-first benchmark framework for evaluating foundation models on energy-forecasting tasks. Consistent with the architecture established across this research program, FoundationBench introduces no new architectural layer: it registers as the third module of the existing Benchmark Evolution Layer, alongside ForecastBench (Module 1, executed) and SecBench (Module 2, protocol specified), and extends the existing Benchmark Evolution Layer and Scientific Lineage Layer registries rather than defining parallel mechanisms. The framework specifies a model registry that records each candidate model's architecture, deployment mode, and license rather than treating all foundation models as interchangeable — explicitly distinguishing open-weight zero-shot models from Nixtla's closed-source, API-only TimeGPT, and from PatchTST, whose primary operating mode is supervised or self-supervised training rather than deployed zero-shot inference. It reuses BQEB ForecastBench's existing forecasting targets (next-hour electricity load, day-ahead electricity price) rather than defining new tasks, applying a zero-shot and few-shot evaluation paradigm to them instead of the trained-from-scratch paradigm ForecastBench used. The evaluation protocol specifies point and probabilistic metrics, a mandatory naive-baseline comparison, and bootstrap confidence intervals. This manuscript defines benchmark infrastructure; it is not a model paper, proposes no new forecasting architecture, and reports no execution results. Every planned-evaluation table in this manuscript is explicitly marked reserved for future experimental validation, consistent with the evidentiary discipline established in Papers 1 and 2.
Source: Preprints.org
Language: English
Subfield: Electrical and Electronic Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7212440340