Policy Algebraic Equation for the Discrete-Time Linear Quadratic Regulator Problem
Year: 2024
Type: article
Abstract: The discrete-time, infinite-horizon linear quadratic regulator (LQR) is studied with the objective of establishing a unified perspective on the problem by relying simultaneously on Dynamic Programming and the discrete Minimum Principle. While it is well known that the two strategiesindependentlyyield the optimal solution, it is shown here that theircombinationprovides much deeper insights on the nature of the optimal solution and on the strategies by means of which it can be computed. More precisely, the optimal cost, captured by the matrix P, and the feedback gain matrix K are jointly related via theobservability matrixof the underlying state/costate (Hamiltonian) dynamics when the state alone is measured. Such an abstract property is then instrumental for deriving alternative characterizations of the optimal solution. First, an algebraic equation, referred to as thepolicy algebraic equation, is established in the variable K alone and with dimension typically much smaller than the size of the classic ARE arising in discrete-time LQR, although comprising polynomial equations of higher degree. This equation permits the direct construction of the optimal feedback gain (i.e., theactor) without the need for the simultaneous computation of the optimal cost (i.e., thecritic). The structure of the policy algebraic equation naturally lends itself to an iterative approach towards its solution, which is restricted to thespace of policies aloneand which does not require the explicit solution of any intermediate (linear) equation at each step. Furthermore, as a consequence of the above properties, it is possible to derive a Riccati equation in P, although with coefficients defined by polynomial functions of K, with the property that the constant and quadratic terms are symmetric andsign-definite. This aspect is remarkably different from the classic ARE associated to the discrete-time LQR and more akin to the continuous-time counterpart.
Author: Mario Sassano
Institution: University of Rome Tor Vergata
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $2,495
OpenAlex ID: W4402673173