Year: 2024
Type: preprint
Abstract: Many robot manipulation tasks can be framed as geometric reasoning tasks, where an agent must be able to precisely manipulate an object into a position that satisfies the task from a set of initial conditions. Often, task success is defined based on the relationship between two objects - for instance, hanging a mug on a rack. In such cases, the solution should be equivariant to the initial position of the objects as well as the agent, and invariant to the pose of the camera. This poses a challenge for learning systems which attempt to solve this task by learning directly from high-dimensional demonstrations: the agent must learn to be both equivariant as well as precise, which can be challenging without any inductive biases about the problem. In this work, we propose a method for precise relative pose prediction which is provably SE(3)-equivariant, can be learned from only a few demonstrations, and can generalize across variations in a class of objects. We accomplish this by factoring the problem into learning an SE(3) invariant task-specific representation of the scene and then interpreting this representation with novel geometric reasoning layers which are provably SE(3) equivariant. We demonstrate that our method can yield substantially more precise placement predictions in simulated placement tasks than previous methods trained with the same amount of data, and can accurately represent relative placement relationships data collected from real-world demonstrations. Supplementary information and videos can be found at https://sites.google.com/view/reldist-iclr-2023.
Source: arXiv (Cornell University)
Authors: Eisner, Ben, Yang, Yi, Davchev, Todor, Vecerik, Mel, Scholz, Jonathan, Held, David
Institution: Carnegie Mellon University
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
Awards: Graduate Research Fellowship Program (GRFP), PURPOSE: THE PURPOSE OF THIS GRANT IS TO DEVELOP MACHINE LEARNING METHODS THAT ALLOW MANUFACTURING ROBOTS TO QUICKLY BE ADAPTED TO PERFORM NEW TASKS USING VISION AND TOUCH SENSING. THIS WILL ESTABLISH WAYS FOR THESE LEARNING METHODS TO BE MORE EASILY ADOPTED BY US MANUFACTURERS. ACTIVITIES TO BE PERFORMED: THIS PROPOSAL WILL INCLUDE THREE ACTIVITIES:ACTIVITY 1 - SELF-SUPERVISED VISION-BASED OBJECT MANIPULATION - THE ROBOTICS TRAINING CENTER WILL TEACH ROBOTS HOW TO TRANSFORM ARBITRARY OBJECTS INTO TARGET POSES. THIS IS USEFUL FOR ASSEMBLY TASKS FOR INDUSTRIAL ROBOTS, IN WHICH THE MANUFACTURING ROBOT NEEDS TO PLACE OBJECTS INTO TARGET POSITIONS. ACTIVITY 2 - VIBROTACTILE SENSING FOR MONITORING ASSEMBLY PROCESSES - EXPLORE THE USE OF VIBROTACTILE DATA FOR MONITORING MANUFACTURING ASSEMBLY PROCESSES TO DETERMINE WHEN AN ASSEMBLY STEP IS COMPLETE OR AN ERROR HAS OCCURRED. ACTIVITY 3 - INTEGRATION AND DEVELOPMENT OF A DIGITAL BACKBONE FOR LARGE-SCALE DATA COLLECTION AND PROCESSING IN ROBOTICS TRAINING - WILL FOCUS ON THE INTEGRATION OF THE ROBOTICS TRAINING CENTER WITH THE MILL 19A DIGITAL BACKBONE AND DATA HISTORIAN. EXPECTED OUTCOMES: THE PROPOSED PROJECT WILL HELP MAKE MANUFACTURING INDUSTRIES MORE COMPETITIVE AND ROBUST TO CHANGES IN MARKETS AND SUPPLY CHAINS.INTENDED BENEFICIARIES: ROBOT VENDORS, ROBOT END USERS, ROBOT MANUFACTURERS, U.S. INDUSTRY, ROBOT SYSTEMS INTEGRATORS.SUBRECIPIENT ACTIVITIES: THE RECIPIENT INTENDS TO HAVE A SUBAWARD TO THE ADVANCED ROBOTICS FOR MANUFACTURING (ARM) INSTITUTE, WHO WILL PUBLICIZE THE TECHNOLOGY AND DOCUMENTATION THAT WILL BE DEVELOPED BY THIS ROBOTICS TRAINING CENTER AND MAKE IT AVAILABLE TO THEIR MEMBERS FOR INCORPORATION INTO OR ADDITION TO ANY OF THEIR ROUGHLY 16,000 ROBOTICS-FOR-MANUFACTURING TRAINING PROGRAMS.
OpenAlex ID: W4395065077