Year: 2020
Type: article
Abstract: For years, there has been interest in approximation methods for solving dynamic programming problems, because of the inherent complexity in computing optimal solutions characterized by Bellman's principle of optimality. A wide range of approximate dynamic programming (ADP) methods now exists. It is of great interest to guarantee that the performance of an ADP scheme be at least some known fraction, say β, of optimal. This letter introduces a general approach to bounding the performance of ADP methods, in this sense, in the stochastic setting. The approach is based on new results for bounding greedy solutions in string optimization problems, where one has to choose a string (ordered set) of actions to maximize an objective function. This bounding technique is inspired by submodularity theory, but submodularity is not required for establishing bounds. Instead, the bounding is based on quantifying certain notions of curvature of string functions; the smaller the curvatures the better the bound. The key insight is that any ADP scheme is a greedy scheme for some surrogate string objective function that coincides in its optimal solution and value with those of the original optimal control problem. The ADP scheme then yields to the bounding technique mentioned above, and the curvatures of the surrogate objective determine the value β of the bound. The surrogate objective and its curvatures depend on the specific ADP.
Source: IEEE Control Systems Letters
Authors: Yajing Liu, Edwin K. P. Chong, Ali Pezeshki, Zhenliang Zhang
Institutions: National Laboratory of the Rockies, Colorado State University
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: National Science Foundation
Awards: CCF-BSF:CIF:Small:Signal Processing and Machine Learning on Manifolds, with Applications to Invariant Detection and Covariant Estimation, CIF: Small: String Submodularity and Near-Optimal Adaptive Control and Sensing, CRISP Type 2/Collaborative Research: Scalable Decision Model to Achieve Local and Regional Resilience of Interdependent Critical Infrastructure Systems and Communities
OpenAlex ID: W2892363377