Year: 2024
Type: article
Abstract: Learning from imperfect demonstrations is a crucial challenge in imitation learning (IL). Unlike existing works that still rely on the enormous effort of expert demonstrators, we consider a more cost-effective option for obtaining a large number of demonstrations. That is, hire annotators to label actions for existing image records in realistic scenarios. However, action noise can occur when annotators are not domain experts or encounter confusing states. In this work, we introduce two particular forms of action noise, i.e., state-independent and state-dependent action noise. Previous IL methods fail to achieve expert-level performance when the demonstrations contain action noise, especially the state-dependent action noise. To mitigate the harmful effects of action noises, we propose a robust learning paradigm called USN (Uncertainty-aware Sample-selection with Negative learning). The model first estimates the predictive uncertainty for all demonstration data and then selects sampleswith high loss based on the uncertainty measures. Finally, it updates the model parameters with additional negative learning on the selected samples. Empirical results in Box2D tasks and Atari games show that USN consistently improves the final rewards of behavioral cloning, online imitation learning, and offline imitation learning methods under various action noises. The ratio of significant improvements is up to 94.44%. Moreover, our method scales to conditional imitation learning with real-world noisy commands in urban driving
DOI: 10.1613/jair.1.15819
Authors: Xingrui Yu, Bo Han, Ivor W. Tsang
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
Funders: National Research Foundation, Agency for Science, Technology and Research, Maritime and Port Authority of Singapore, National Research Foundation Singapore, Hong Kong Baptist University, University of Technology Sydney, National Natural Science Foundation of China, China Scholarship Council, Singapore Maritime Institute, National Supercomputing Centre Singapore, Basic and Applied Basic Research Foundation of Guangdong Province
Awards: G4392973527, G7089068806, G7957291013
OpenAlex ID: W4395047259
