Work

Approximate Information States for Worst Case Control and Learning in Uncertain Systems

Year: 2024

Type: article

Abstract: In this article, we investigate discrete-time decision-making problems in uncertain systems with partially observed states. We consider a nonstochastic model, where uncontrolled disturbances acting on the system take values in bounded sets with unknown distributions. We present a general framework for decision-making in such problems by using the notion of the information state and approximate information state and introduce conditions to identify an uncertain variable that can be used to compute an optimal strategy through a dynamic program (DP). Next, we relax these conditions and defineapproximate information statesthat can be learned from output data without knowledge of system dynamics. We use approximate information states to formulate a DP that yields a strategy with a bounded performance loss. Finally, we illustrate the application of our results in control and reinforcement learning using numerical examples.

Source: IEEE Transactions on Automatic Control

DOI: 10.1109/tac.2024.3422889

Authors: Aditya Dave, Nishanth Venkatesh S., Andreas A. Malikopoulos

Institution: Cornell University

Language: English

Topic: Adaptive Dynamic Programming Control

Subfield: Computational Theory and Mathematics

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Science Foundation, Division of Civil, Mechanical and Manufacturing Innovation

Awards: Collaborative Research: CPS: Medium: An Online Learning Framework for Socially Emerging Mixed Mobility, NRI: Addressing Safe Interaction Between Autonomous and Human-Driven Vehicles