Year: 2023
Type: preprint
Abstract: Policy gradient methods, where one searches for the policy of interest by maximizing the value functions using first-order information, become increasingly popular for sequential decision making in reinforcement learning, games, and control. Guaranteeing the global optimality of policy gradient methods, however, is highly nontrivial due to nonconcavity of the value functions. In this exposition, we highlight recent progresses in understanding and developing policy gradient methods with global convergence guarantees, putting an emphasis on their finite-time convergence rates with regard to salient problem parameters.
Source: arXiv (Cornell University)
Authors: Cen, Shicong, Chi, Yuejie
Institution: Carnegie Mellon University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G1469985387, RINGS: Learning-Enabled Ground and Air Integrated Networks (GAINs), CIF: Medium: Collaborative Research: Theory of Optimization Geometry and Algorithms for Neural Networks, Collaborative Research: CIF: Medium: Statistical and Algorithmic Foundations of Efficient Reinforcement Learning, G8876996369
OpenAlex ID: W4387559814