Work

Boosting Soft Q-Learning by Bounding

Year: 2024

Type: preprint

Abstract: An agent's ability to leverage past experience is critical for efficiently solving new tasks. Prior work has focused on using value function estimates to obtain zero-shot approximations for solutions to a new task. In soft Q-learning, we show how any value function estimate can also be used to derive double-sided bounds on the optimal value function. The derived bounds lead to new approaches for boosting training performance which we validate experimentally. Notably, we find that the proposed framework suggests an alternative method for updating the Q-function, leading to boosted performance.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2406.18033

Authors: Adamczyk, Jacob, Makarenko, Volodymyr, Tiomkin, Stas, Kulkarni, Rahul V.

Institutions: University of Massachusetts Boston, San Jose State University

Language: English

Topic: Reinforcement Learning in Robotics

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Science Foundation, University of Massachusetts Boston, PAZY Foundation

Awards: Large Deviations and Driven Processes for Stochastic Models of Gene Expression and Its Regulation, G1802981795, CRII: RI: Interpretable Framework and Transformative Applications for Viability in Autonomous Agents

OpenAlex ID: W4400104547

Cites: 3

Cited by: 2

Related to: 10