Year: 2026
Type: preprint
Abstract: We consider an infinite-horizon discounted dynamic programming (DP) problem for a countable-innovation belief-state model. The model is a surrogate for a linear-Gaussian dual control problem involving an unknown parameter vector and Gaussian observation noise. The surrogate preserves the first two moments of the Gaussian innovation and the resulting posterior second-moment structure. We allow unbounded belief and control spaces and nonnegative stage costs with quadratic growth in the posterior mean and covariance. Using a posterior-moment weight, we show that the Bellman operator is a contraction in a weighted sup-norm space. We characterize the optimal cost as the unique solution of the Bellman equation in this space, establish the existence of optimal stationary policies. We further prove weighted sup-norm convergence of value iteration, policy iteration, and optimistic policy iteration under vanishing approximation errors. The convergence results under vanishing approximation errors rely on general weighted sup-norm contraction properties and are applicable beyond the particular dual control problem considered here.
Source: arXiv (Cornell University)
Authors: Wang, Ying, Li, Yuchao, Hjalmarsson, Håkan
Institutions: KTH Royal Institute of Technology, Arizona State University
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7221813538