Year: 2024
Type: article
Abstract: Here and in a follow-on paper, we consider a simple control problem in which the underlying dynamics depend on a parameter a that is unknown and must be learned. In this paper, we assume that a is bounded, i.e., that |a| \le a_{\textrm{MAX}} , and we study two variants of the control problem. In the first variant, Bayesian control, we are given a prior probability distribution for a and we seek a strategy that minimizes the expected value of a given cost function. Assuming that we can solve a certain PDE (the Hamilton–Jacobi–Bellman equation), we produce optimal strategies for Bayesian control. In the second variant, agnostic control, we assume nothing about a and we seek a strategy that minimizes a quantity called the regret. We produce a prior probability distribution d\textup{Prior}(a) supported on a finite subset of [-a_{\textrm{MAX}},a_{\textrm{MAX}}] so that the agnostic control problem reduces to the Bayesian control problem for the prior d\textup{Prior}(a) .
DOI: 10.4171/rmi/1510
Institutions: Princeton University, University of Bonn
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
Awards: G1523888516, G6562578720
OpenAlex ID: W4404599377