Year: 2025
Type: preprint
Abstract: We study how vision-language-action (VLA) models can improve through real-world deployments via reinforcement learning (RL). We present a general-purpose method, RL with Experience and Corrections via Advantage-conditioned Policies (RECAP), that provides for RL training of VLAs via advantage conditioning. Our method incorporates heterogeneous data into the self-improvement process, including demonstrations, data from on-policy collection, and expert teleoperated interventions provided during autonomous execution. RECAP starts by pre-training a generalist VLA with offline RL, which we call $π^{*}_{0.6}$, that can then be specialized to attain high performance on downstream tasks through on-robot data collection. We show that the $π^{*}_{0.6}$ model trained with the full RECAP method can fold laundry in real homes, reliably assemble boxes, and make espresso drinks using a professional espresso machine. On some of the hardest tasks, RECAP more than doubles task throughput and roughly halves the task failure rate.
Source: arXiv (Cornell University)
Authors: Intelligence, Physical, Amin, Ali, Aniceto, Raichelle, Balakrishna, Ashwin, Black, Kevin, Conley, Ken, Connors, Grace, Darpinian, James, Dhabalia, Karan, DiCarlo, Jared, Driess, Danny, Equi, Michael, Esmail, Adnan, Fang, Yunhao, Finn, Chelsea, Glossop, Catherine, Godden, Thomas, Goryachev, Ivan, Groom, Lachy, Hancock, Hunter, Hausman, Karol, Hussein, Gashon, Ichter, Brian, Jakubczak, Szymon, Jen, Rowan, Jones, Tim, Katz, Ben, Ke, Liyiming, Kuchi, Chandra, Lamb, Marinda, LeBlanc, Devin, Levine, Sergey, Li-Bell, Adrian, Lu, Yao, Mano, Vishnu, Mothukuri, Mohith, Nair, Suraj, Pertsch, Karl, Ren, Allen Z., Sharma, Charvi, Shi, Lucy Xiaoyang, Smith, Laura, Springenberg, Jost Tobias, Stachowicz, Kyle, Stoeckle, Will, Swerdlow, Alex, Tanner, James, Torne, Marcel, Vuong, Quan, Walling, Anna, Wang, Haohuan, Williams, Blake, Yoo, Sukwon, Yu, Lili, Zhilinsky, Ury, Zhou, Zhiyuan
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4417215404
Cites:
Cited by:
Related to: