Year: 2025
Type: article
Abstract: Model agnostic controller learning, for instance by direct policy optimization, has been the object of renewed attention lately, since it avoids a computationally expensive system identification step. Indeed, direct policy search has been empirically shown to lead to optimal controllers in a number of cases of practical importance. However, to date, these empirical results have not been backed up with a comprehensive theoretical analysis for general problems. In this paper we use a simple example to show that direct policy optimization is not directly generalizable to other seemingly simple problems. In such cases, direct optimization of a performance index can lead to unstable pole/zero cancellations, resulting in the loss of internal stability and unbounded outputs in response to arbitrarily small perturbations. We conclude the paper by analyzing several alternatives to avoid this phenomenon, suggesting some new directions in direct control policy optimization.
Source: IEEE Control Systems Letters
Authors: Mario Sznaier, Mustafa Bozdag
Institution: Northeastern University
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G2855079135, G6562578720
OpenAlex ID: W4411446209