Year: 2022
Type: preprint
Abstract: Self-driving cars must detect vehicles, pedestrians, and other traffic participants accurately to operate safely. Small, far-away, or highly occluded objects are particularly challenging because there is limited information in the LiDAR point clouds for detecting them. To address this challenge, we leverage valuable information from the past: in particular, data collected in past traversals of the same scene. We posit that these past data, which are typically discarded, provide rich contextual information for disambiguating the above-mentioned challenging cases. To this end, we propose a novel, end-to-end trainable Hindsight framework to extract this contextual information from past traversals and store it in an easy-to-query data structure, which can then be leveraged to aid future 3D object detection of the same scene. We show that this framework is compatible with most modern 3D detection architectures and can substantially improve their average precision on multiple autonomous driving datasets, most notably by more than 300% on the challenging cases.
Source: arXiv (Cornell University)
Institutions: Cornell University, The Ohio State University
Language: English
Subfield: Automotive Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
Funders: National Science Foundation, U.S. Department of Defense, Bill and Melinda Gates Foundation, Defense Advanced Research Projects Agency, Cornell Center for Materials Research, Materials Research Science and Engineering Center, Harvard University, Office of Naval Research, Division of Materials Research
Awards: G1205195086, G1321126357, AI Institute for Intelligent CyberInfrastructure with Computational Learning in the Environment (ICICLE), G201125081, G2415270547, RI: AF: Small: Collaborative Research: Differentially Private Learning: From Theory to Applications, Collaborative Research: RI: Medium: Robust Perception through End-User Adaptation, CAREER: New Directions for Metric Learning, Cornell Center for Materials Research - MRSEC, G4504108201, HDR Institute: Imageomics: A New Frontier of Biological Information Powered by Knowledge-Guided Machine Learning, G5917061246, Collaborative Research: RI: Medium: Robust Perception through End-User Adaptation, III: Small: Collaborative Research: Towards Interpretable Machine Learning, G6351810044, TRIPODS: Data Science for Improved Decision-Making: Learning in the Context of Uncertainty, Causality, Privacy, and Network Structures, G6875099116, G709468757, G7767248047, G8180032753, S&AS: INT: Inference, Reasoning, and Learning for Robust Autonomous Driving, G8876996369, G8929337198
OpenAlex ID: W4221150806