Incorporating the Scenario-Based Framework Underpinned by Operational Design Domain (ODD) for Automated Driving Stack (ADS) Evaluation
Year: 2025
Type: conference-paper
Abstract: The rapid development of open-source Automated Driving System (ADS) stacks has created a pressing need for clear guidance on their evaluation and selection for specific use cases. This paper introduces a scenario-based evaluation framework combined with a modular simulation framework, offering a scalable methodology for assessing and benchmarking ADS solutions, including but not limited to off-the-shelf designs. The study highlights the lack of clear Operational Design Domain (ODD) descriptions in such systems. Without a common understanding, users must rely on subjective assumptions, which hinders the process of accurate system selection. To address this gap, the study proposes adopting a standardised ISO 34503 ODD description format within the ADS stacks. The application of the proposed framework is showcased through a case study evaluating two open-source systems, Autoware and Apollo. By first defining the assumed system’s ODD, then selecting a relevant scenario, and establishing pass/fail criteria, the framework provides objective data to highlight performance differences between both systems. The proposed methodology provides an unbiased foundation for informed decision-making, promoting the safe and effective integration of ADS technologies.
DOI: 10.4271/2025-01-8067
Authors: Emil Chodowiec, Xizhe Zhang, Joe Mitchell, Peter Baker, Siddartha Khastgir, Paul Jennings
Institution: University of Warwick
Language: English
Subfield: Automotive Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4409017740