Infinite-horizon optimal control of nonlinear discrete-time systems: HJB pde, Hamiltonian dynamics and invariant manifolds
Year: 2025
Type: article
Abstract: Nonlinear discrete-time optimal control problems are studied over an infinite horizon with the aim of establishing a connection between the solution of the Bellman equation and the trajectories of the Hamiltonian difference dynamics associated with the problem. First, a discrete-time counterpart of the Hamilton–Jacobi–Bellman partial differential equation is introduced and discussed. The latter is then further instrumental for showing that a certain manifold, involving the costate variable and the gradient of the value function, is invariant for the Hamiltonian dynamics, hence recovering a well-known property of continuous-time optimal control problems. This feature is then leveraged to envision an episodic learning strategy based on the notion of invariant manifold.
Source: Automatica
Author: Mario Sassano
Institution: University of Rome Tor Vergata
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,760
OpenAlex ID: W4411285699