Work

WildDash - Creating Hazard-Aware Benchmarks

Year: 2018
Type: conference-paper
Abstract: Test datasets should contain many different challenging aspects so that the robustness and real-world applicability of algorithms can be assessed. In this work, we present a new test dataset for semantic and instance segmentation for the automotive domain. We have conducted a thorough risk analysis to identify situations and aspects that can reduce the output performance for these tasks. Based on this analysis we have designed our new dataset. Meta-information is supplied to mark which individual visual hazards are present in each test case. Furthermore, a new benchmark evaluation method is presented that uses the meta-information to calculate the robustness of a given algorithm with respect to the individual hazards. We show how this new approach allows for a more expressive characterization of algorithm robustness by comparing three baseline algorithms.
Language: English
Open Access status: hybrid
Estimated APC paid: $5,392
OpenAlex ID: W2894903065
FWCI: 4.395
Cites: 15
Cited by: 197
Related to: 10