Work

Perceptual symbol grounding through child-like natural language interaction and learning

Year: 2016
Type: dissertation
Abstract: Current methods of natural language understanding and computer vision require large amounts of labeled training data to accomplish tasks that are seemingly simple for children, who must learn models of objects, actions, and how the world works simultaneously with language. We explore the strategies children use to make sense of the vast amount of information they are presented with, and apply these strategies to a situated language learning agent to see how an understanding of child language learning can inform the learning models of intelligent agents in a natural environment. A combination of these strategies of mutual exclusion, shape bias, pragmatic inference, and cross-situational learning allows this system to learn from very few training examples and even learn by combining information from multiple ambiguous demonstrations. We then use this knowledge learned from natural language demonstrations to identify complex objects with multiple parts and constraints, showcasing the power of grounded language understanding. This learning system will form the basis of a more complete system towards grounded models of language, incorporating an understanding of pragmatics with references to objects in the environment, learning of metaphor, and a representation of perceptual and spatial semantics based on conceptual spaces. We outline the necessary steps for building a grounded language system, and explain a possible framework for incorporating perception, semantic composition, and inference in a unified model.
Authors: Perera, Ian, Allen, James F.
Language: English
Field: Psychology
Open Access status: green
OpenAlex ID: W6986381906
Cites: 141
Cited by:
Related to: 10