Year: 2017
Type: conference-paper
Abstract: Hearing an object falling onto the ground, humans can recover rich information including its rough shape, material, and falling height. In this paper, we build machines to approximate such competency. We first mimic human knowledge of the physical world by building an efficient, physics-based simulation engine. Then, we present an analysis-by-synthesis approach to infer properties of the falling object. We further accelerate the process by learning a mapping from a sound wave to object properties, and using the predicted values to initialize the inference. This mapping can be viewed as an approximation of human commonsense learned from past experience. Our model performs well on both synthetic audio clips and real recordings without requiring any annotated data. We conduct behavior studies to compare human responses with ours on estimating object shape, material, and falling height from sound. Our model achieves near-human performance.
Language: English
Topic: Music and Audio Processing
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G4211722790, G4724522708, A Center for Brains, Minds and Machines: the Science and the Technology of Intelligence, BIGDATA: F: DKA: Collaborative Research: Structured Nearest Neighbor Search in High Dimensions, RI: Large: Collaborative Research: Reconstructive recognition: Uniting statistical scene understanding and physics-based visual reasoning, G8876996369
OpenAlex ID: W2751765471