Year: 2016
Type: dissertation
Abstract: Current methods of natural language understanding and computer vision require large amounts of labeled training data to accomplish tasks that are seemingly simple for children, who must learn models of objects, actions, and how the world works simultaneously with language. We explore the strategies children use to make sense of the vast amount of information they are presented with, and apply these strategies to a situated language learning agent to see how an understanding of child language learning can inform the learning models of intelligent agents in a natural environment. A combination of these strategies of mutual exclusion, shape bias, pragmatic inference, and cross-situational learning allows this system to learn from very few training examples and even learn by combining information from multiple ambiguous demonstrations. We then use this knowledge learned from natural language demonstrations to identify complex objects with multiple parts and constraints, showcasing the power of grounded language understanding. This learning system will form the basis of a more complete system towards grounded models of language, incorporating an understanding of pragmatics with references to objects in the environment, learning of metaphor, and a representation of perceptual and spatial semantics based on conceptual spaces. We outline the necessary steps for building a grounded language system, and explain a possible framework for incorporating perception, semantic composition, and inference in a unified model.
Authors: Perera, Ian, Allen, James F.
Institution: University of Rochester
Language: English
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
SDG: Quality education
Open Access status: green
OpenAlex ID: W6986381906
