Forecasting Partially Observed Daily Air-Quality Curves Without Imputation: A Functional Autoregressive Approach With Kalman Filtering
Year: 2026
Type: article
Abstract: Air-quality monitoring networks deliver half-hourly pollutant series with substantial, irregular missingness caused by sensor failure and calibration. Standard practice imputes the series before a forecasting model is fitted, which ties the forecast to an imputation model that must extrapolate over long gaps and, with single imputation, provides no measure of the uncertainty it introduces. This paper proposes a partially observed functional autoregression, an imputation-free framework in which each day of measurements is one functional observation on the 24-hour domain, observed at an irregular subset of 48 half-hour slots. The mean function and the covariance and lagged cross-covariance operators are estimated from all observed pairs of measurements by a grid version of the principal-analysis-by-conditional-expectation estimator; a functional autoregression on the previous-day curves is estimated through regularized Yule–Walker equations in the principal-component space, so that no complete day is required; and a Kalman filter in the score space turns partially observed or empty days into best linear predictions whose uncertainty grows through a gap and saturates at the stationary level, calibrated by split-conformal quantiles on a window reserved for that purpose. Identification, consistency, best-linear-prediction and Yule–Walker results are established. On 354 days of half-hourly data for five pollutants from Surabaya (2.8–44.9% missing, up to 121 fully missing days), the method matches the best of nine imputation pipelines combined with the same functional model within 7% (no significant difference in per-origin tests) whereas an imputation-free univariate state-space benchmark is 8–70% less accurate; its Kalman-smoother reconstruction is the most accurate method for gaps longer than six hours. When the most recent day is missing at the forecast origin, its error grows by about 5% at most, it forecasts as well as the best pipeline whose imputer is kept away from the gap, and its prediction bands widen through the gap, whereas pipelines whose imputer extrapolates into the gap can lose more than half of their accuracy or fail outright.
Source: IEEE Access
Authors: Novri Suhermi, Rahmatin Nur AMALIA
Institution: Institut Teknologi Sepuluh Nopember
Language: English
Subfield: Environmental Engineering
Field: Environmental Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,075
OpenAlex ID: W7225639986
