Work

Multi-Layer Self-Assessment with Filtering for 3D Object Detection in Autonomous Vehicles

Year: 2025

Type: article

Abstract: Reliable detection of road users is critical to the safety of automated driving systems. While object detectors based on deep neural networks are widely used for this purpose, they remain susceptible to errors that could compromise safety. A promising strategy to mitigate these risks involves run-time perception monitoring mechanisms, commonly referred to in the literature as self-assessment or introspection. Current research in this area predominantly addresses anomaly detection, or monitoring camera-based 2D object detection, with insufficient focus on in-distribution errors and 3D object detection. Additionally, existing 2D studies often monitor activation patterns from the final layers of the network backbone, overlooking earlier activations that preserve higher spatial resolution. Yet, high-resolution early-layer activations can be valuable for detecting errors with sparse 3D point clouds. We also argue that not all objects in a scene should equally influence frame-level error detection, a factor often neglected in current methods. To address these gaps, we propose a novel self-assessment mechanism for 3D object detection that leverages activation patterns from multiple network layers. This mechanism employs spatial filtering to focus the model within an area of interest in the close vicinity of the ego vehicle. Additionally, it utilises an object filtering mechanism, which specifically targets the missed objects by excluding the points in those already detected. We evaluate our method using widely recognised object detectors and public datasets. Additionally, we demonstrate its robustness under domain shifts with real-world LiDAR data collected on motorways in diverse weather conditions. Results show the proposed mechanism provides 6% AUROC improvement over last-layer activation methods with spatial filtering on the NuScenes dataset. It also demonstrates a superior ability to transfer knowledge under domain shifts. Code is available at https://github.com/yatbazhakan/multi-layer-introspection .

Source: ACM Transactions on Intelligent Systems and Technology

DOI: 10.1145/3776551

Authors: Hakan Yekta Yatbaz, Konstantinos Koufos, Mehrdad Dianati, Roger Woodman

Institutions: Queen's University Belfast, University of Warwick

Language: English

Topic: Autonomous Vehicle Technology and Safety

Subfield: Automotive Engineering

Field: Engineering

Domain: Physical Sciences

Open Access status: green

Funder: European Commission

Award: ReliablE in-Vehicle pErception and decisioN-making in complex environmenTal conditionS