Work

Optimism in Reinforcement Learning with Generalized Linear Function Approximation

Year: 2019

Type: preprint

Abstract: We design a new provably efficient algorithm for episodic reinforcement learning with generalized linear function approximation. We analyze the algorithm under a new expressivity assumption that we call "optimistic closure," which is strictly weaker than assumptions from prior analyses for the linear setting. With optimistic closure, we prove that our algorithm enjoys a regret bound of $\tilde{O}(\sqrt{d^3 T})$ where $d$ is the dimensionality of the state-action features and $T$ is the number of episodes. This is the first statistically and computationally efficient algorithm for reinforcement learning with generalized linear functions.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.1912.04136

Authors: Wang, Yining, Wang, Ruosong, Du, Simon S., Krishnamurthy, Akshay

Institutions: University of Florida, Carnegie Mellon University, University of Washington, Institute for Advanced Study, University of Massachusetts Amherst, Microsoft Research New York City (United States)

Language: English

Topic: Reinforcement Learning in Robotics

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: National Science Foundation

Award: Research in Mathematics

OpenAlex ID: W2991929641

Cites: 29

Cited by: 52

Related to: 10