On the stability of Lipschitz continuous control problems and its applications to reinforcement learning
Year: 2026
Type: article
Abstract: We address the crucial yet underexplored stability properties of the Hamilton–Jacobi–Bellman (HJB) equation in model-free reinforcement learning contexts with a focus on Lipschitz continuous optimal control problems. By an established connection between Lipschitz continuous optimal control problems and classical optimal control problems, we provide new insights into the stability of the value functions. Specifically, we investigate the generic convergence property as well as the rate of convergence of value functions as the Lipschitz constraint increases. Moreover, we present a generalized Hamilton–Jacobi based reinforcement learning framework that accommodates general \(L^{p}\) geometries in the control regularization. To evaluate the stability properties and performance of our proposed method, we conduct various numerical experiments using benchmark examples and compare its efficacy with existing approaches.
Authors: Namkyeong Cho, Yeoneung Kim
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $1,890
OpenAlex ID: W7165023853