Year: 2024
Type: conference-paper
Abstract: Autonomous systems that efficiently utilize tools can assist humans in completing many common tasks such as cooking and cleaning. However, current systems fall short of matching human-level of intelligence in terms of adapting to novel tools. Prior works based on affordance often make strong assumptions about the environments and cannot scale to more complex, contact-rich tasks. In this work, we tackle this challenge and explore how agents can learn to use previously unseen tools to manipulate deformable objects. We propose to learn a generative model of the tool-use trajectories as a sequence of tool point clouds, which generalizes to different tool shapes. Given any novel tool, we first generate a tool-use trajectory and then optimize the sequence of tool poses to align with the generated trajectory. We train a single model on four different challenging deformable object manipulation tasks, using demonstration data from only one tool per task. The model generalizes to various novel tools, significantly outperforming baselines. We further test our trained policy in the real world with unseen tools, where it achieves the performance comparable to human. Additional materials can be found on our project website.1
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
Awards: PURPOSE: THE PURPOSE OF THIS GRANT IS TO DEVELOP MACHINE LEARNING METHODS THAT ALLOW MANUFACTURING ROBOTS TO QUICKLY BE ADAPTED TO PERFORM NEW TASKS USING VISION AND TOUCH SENSING. THIS WILL ESTABLISH WAYS FOR THESE LEARNING METHODS TO BE MORE EASILY ADOPTED BY US MANUFACTURERS. ACTIVITIES TO BE PERFORMED: THIS PROPOSAL WILL INCLUDE THREE ACTIVITIES:ACTIVITY 1 - SELF-SUPERVISED VISION-BASED OBJECT MANIPULATION - THE ROBOTICS TRAINING CENTER WILL TEACH ROBOTS HOW TO TRANSFORM ARBITRARY OBJECTS INTO TARGET POSES. THIS IS USEFUL FOR ASSEMBLY TASKS FOR INDUSTRIAL ROBOTS, IN WHICH THE MANUFACTURING ROBOT NEEDS TO PLACE OBJECTS INTO TARGET POSITIONS. ACTIVITY 2 - VIBROTACTILE SENSING FOR MONITORING ASSEMBLY PROCESSES - EXPLORE THE USE OF VIBROTACTILE DATA FOR MONITORING MANUFACTURING ASSEMBLY PROCESSES TO DETERMINE WHEN AN ASSEMBLY STEP IS COMPLETE OR AN ERROR HAS OCCURRED. ACTIVITY 3 - INTEGRATION AND DEVELOPMENT OF A DIGITAL BACKBONE FOR LARGE-SCALE DATA COLLECTION AND PROCESSING IN ROBOTICS TRAINING - WILL FOCUS ON THE INTEGRATION OF THE ROBOTICS TRAINING CENTER WITH THE MILL 19A DIGITAL BACKBONE AND DATA HISTORIAN. EXPECTED OUTCOMES: THE PROPOSED PROJECT WILL HELP MAKE MANUFACTURING INDUSTRIES MORE COMPETITIVE AND ROBUST TO CHANGES IN MARKETS AND SUPPLY CHAINS.INTENDED BENEFICIARIES: ROBOT VENDORS, ROBOT END USERS, ROBOT MANUFACTURERS, U.S. INDUSTRY, ROBOT SYSTEMS INTEGRATORS.SUBRECIPIENT ACTIVITIES: THE RECIPIENT INTENDS TO HAVE A SUBAWARD TO THE ADVANCED ROBOTICS FOR MANUFACTURING (ARM) INSTITUTE, WHO WILL PUBLICIZE THE TECHNOLOGY AND DOCUMENTATION THAT WILL BE DEVELOPED BY THIS ROBOTICS TRAINING CENTER AND MAKE IT AVAILABLE TO THEIR MEMBERS FOR INCORPORATION INTO OR ADDITION TO ANY OF THEIR ROUGHLY 16,000 ROBOTICS-FOR-MANUFACTURING TRAINING PROGRAMS., CAREER: Self-supervised Representation Learning for Deformable Object Manipulation
OpenAlex ID: W4405788096