How Generalizable is My Behavior Cloning Policy? A Statistical Approach to Trustworthy Performance Evaluation
Year: 2024
Type: article
Abstract: With the rise of stochastic generative models in robot policy learning, end-to-end visuomotor policies are increasingly successful at solving complex tasks by learning from human demonstrations. Nevertheless, since real-world evaluation costs afford users only a small number of policy rollouts, it remains a challenge to accurately gauge the performance of such policies. This is exacerbated by distribution shifts causing unpredictable changes in performance during deployment. To rigorously evaluate behavior cloning policies, we present a framework that provides a tight lower-bound on robot performance in an arbitrary environment, using a minimal number of experimental policy rollouts. Notably, by applying the standard stochastic ordering to robot performance distributions, we provide a worst-case bound on theentire distributionof performance (via bounds on the cumulative distribution function) for a given task. We build upon established results to ensure that the bounds hold with a user-specified confidence level and tightness, and are constructed from as few policy rollouts as possible. In experiments we evaluate policies for visuomotor manipulation in both simulation and hardware. Specifically, we i) empirically validate the guarantees of the bounds in simulated manipulation settings, ii) find the degree to which a learned policy deployed on hardware generalizes to new real-world environments, and iii) rigorously compare two policies tested out-of-distribution. Our data, code, and implementation of confidence bounds are open-source.
Institutions: Stanford University, Toyota Research Institute
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: Office of Naval Research
Award: G6951003262
OpenAlex ID: W4401691787