Year: 2019
Type: article
Abstract: Inverse reinforcement learning (IRL) infers a reward function from demonstrations, allowing for policy improvement and generalization. However, despite much recent interest in IRL, little work has been done to understand the minimum set of demonstrations needed to teach a specific sequential decisionmaking task. We formalize the problem of finding maximally informative demonstrations for IRL as a machine teaching problem where the goal is to find the minimum number of demonstrations needed to specify the reward equivalence class of the demonstrator. We extend previous work on algorithmic teaching for sequential decision-making tasks by showing a reduction to the set cover problem which enables an efficient approximation algorithm for determining the set of maximallyinformative demonstrations. We apply our proposed machine teaching algorithm to two novel applications: providing a lower bound on the number of queries needed to learn a policy using active IRL and developing a novel IRL algorithm that can learn more efficiently from informative demonstrations than a standard IRL approach.
Authors: Brown, Daniel S., Niekum, Scott
Institution: The University of Texas at Austin
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G3609089545, RI: Small: High Confidence, Efficient Learning Under Rich Task Specifications, G3881771821, CAREER: Safe and Efficient Robot Learning from Demonstration in the Real World, NRI: Collaborative Research: Scalable Robot Autonomy through Remote Operator Assistance and Lifelong Learning, S&AS: INT: Socially-Aware Autonomy for Long-Term Deployment of Always-On Heterogeneous Robot Teams, G8876996369
OpenAlex ID: W2963308241