An approach for objective performance assessment of automated vehicles with human behavior as a benchmark
Year: 2025
Type: dissertation
Abstract: The age of automated vehicles has begun and is expected to significantly shape the future of transport inthe coming years. The rapid development of new technologies and methodologies has enabledconsiderable progress in the automotive industry recently. For legislators and consumers to accept newsystems, their safety, reliability and robustness must be demonstrated on public road driving conditions. Dueto the high complexity of covering countless traffic scenarios under a wide range of environmentalconditions, the initial enthusiasm to achieve automated vehicles with full autonomous support has declined.One of the main reasons for the slow development progress are edge cases, which represent criticalscenarios that rarely occur but must be considered for safety purposes.Scenario-based testing of automated vehicles on closed proving grounds is one of the most establishedapproaches in validation today, and it is used as part of the homologation process by certificationorganizations. The number of possible tests with this methodology is currently limited because theconfiguration, execution, and assessment of scenarios are extremely time-consuming and complex.Therefore, there is a risk of not being able to identify all safety-critical issues that may exist in a system. Theliterature lacks scalable testing methods that can increase test coverage while being more realistic toprovide a robust performance assessment of automated vehicles.The aim of this research is to develop a standardized and scalable solution for the objective performanceassessment of an automated vehicle that will serve as additional input for public road transport approvaldecisions.As a method to evaluate the performance of an automated vehicle in a safe and standardized way, thisresearch used the detection of behavioral differences between human drivers and automated systems toidentify safety-critical events and uncomfortable scenarios on public roads. A novel measuring device, whichdid not exist before, was developed for the purpose of detecting behavioral differences at the steeringwheel. With this innovation, the steering movements of the driver can be detected independently withoutaffecting the original steering wheel of the vehicle and, therefore the behavior of the automated system up toa safety-critical threshold limit. The theoretical framework, implemented concept, and performanceassessment of a production vehicle with automated functions were validated by test drives on public roads.According to the results of this research, current automated functions lack safety, reliability and robustnessunder certain circumstances, resulting in dangerous scenarios within seconds. As the automated systemwas deactivated without prior warning, an identified scenario of this research could have led to a seriousaccident without immediate intervention by a human driver. Testing has demonstrated that the concept andtheoretical framework developed are safe and can be used in a standardized manner on public roads. Inaddition, the test results show that a human driver is capable of de-escalating a scenario within seconds inorder to avoid serious consequences.There are currently limitations to the functions of automated vehicles under certain conditions, as not alltraffic scenarios can be recognized and handled correctly. If a human driver does not intervene in the eventof a problem immediately, or if the system cannot find a solution, this poses a potential safety risk for vehiclepassengers and other road users. Critical events and uncomfortable scenarios on public roads can beidentified by continuously comparing behavioral differences between human and automated systemswithout mutual interference up to a defined safety-critical threshold. The developed concept cancomplement scenario-based testing in simulation environments and closed proving grounds since manyscenarios can be validated with lower effort. The implications of this research for the field of automatedvehicle testing show that better methods for validation are needed. If the systems do not have a high level ofsafety, reliability and robustness, it will not be feasible to distribute them, as legislators and consumers willnot accept them.
Author: Schwarz, Di Manuel
Institution: University of the West of Scotland
Language: English
Subfield: Social Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W6911772946
Cites:
Cited by:
Related to: