Year: 2018
Type: preprint
Abstract: We describe an ongoing project in learning to perform primitive actions from demonstrations using an interactive interface. In our previous work, we have used demonstrations captured from humans performing actions as training samples for a neural network-based trajectory model of actions to be performed by a computational agent in novel setups. We found that our original framework had some limitations that we hope to overcome by incorporating communication between the human and the computational agent, using the interaction between them to fine-tune the model learned by the machine. We propose a framework that uses multimodal human-computer interaction to teach action concepts to machines, making use of both live demonstration and communication through natural language, as two distinct teaching modalities, while requiring few training samples.
Source: arXiv (Cornell University)
Authors: Do, Tuan, Krishnaswamy, Nikhil, Rim, Kyeongmin, Pustejovsky, James
Institution: Brandeis University
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
SDG: Quality education
Open Access status: green
OpenAlex ID: W2893555925