Work

Beyond Replay: Automatic Corrections as Direct Policy Supervision for Robotic Manipulation

Year: 2026

Type: article

Abstract: Automatic corrections offer direct policy supervision during assisted robotic reinforcement learning. We evaluate whether regressing on these actions improves learning beyond their use in replay. Uniform corrective distillation (UCD) adds actor regression on initial demonstrations and online corrections to an AutoSERL-derived soft actor–critic system. We compare four objectives on simulated spatula hanging and drawer opening with fixed and randomized starts and extend the randomized UCD–baseline comparison to five training seeds per method. Training stops when the shared process-success criterion is met or the budget is exhausted; each returned model receives 100 fresh test episodes with task assistance disabled. Across five seeds, UCD averaged 100.0% success on spatula and 91.2% on drawer, versus baseline means of 71.6% and 79.8%, with all five spatula UCD models achieving 100/100 successes. A separate comparison matched auxiliary batch size and update frequency; correction-inclusive supervision exceeded demonstration-only supervision in all six task–seed pairs. These results support direct actor regression for reusing automatic corrections under the evaluated stopping protocol.

Source: Applied Sciences

DOI: 10.3390/app162211166

Authors: Xiao Yang, Xiangsheng Huang, Haihao Lin, Weibang Zhou, Simin Zeng

Institutions: University of Chinese Academy of Sciences, Chinese Academy of Sciences, Institute of Automation

Language: English

Topic: Reinforcement Learning in Robotics

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,896

OpenAlex ID: W7225818814

FWCI:

Cites:

Cited by:

Related to: