Cross or Nah? LLMs Get in the Mindset of a Pedestrian in front of Automated Car with an eHMI
Year: 2025
Type: conference-paper
Abstract: This study evaluates the effectiveness of large language modelbased personas for assessing external Human-Machine Interfaces (eHMIs) in automated vehicles.13 different models namely Bak-LLaVA, ChatGPT-4o, DeepSeek-VL2-Tiny, Gemma3:12B, Gemma3:27B, Granite Vision 3.2, LLaMA 3.2 Vision, LLaVA-13B, LLaVA-34B, LLaVA-LLaMA-3, LLaVA-Phi3, MiniCPM-V and Moondream were tasked with simulating pedestrian decision making for 227 vehicle images equipped with eHMI.Confidence scores (0-100) were collected under two conditions: no memory (images independently assessed) and memory-enabled (conversation history preserved), each in 15 independent trials.The model outputs were compared with the ratings of 1,438 human participants.Gemma3:27B achieved the highest correlation with humans without memory (r = 0.85), while ChatGPT-4o performed best with memory (r = 0.81).DeepSeek-VL2-Tiny and BakLLaVA showed little sensitivity to context, and LLaVA-LLaMA-3, LLaVA-Phi3, LLaVA-13B and Moondream consistently produced limited-range output.
Authors: Md Shadab Alam, Pavlo Bazilinskyy
Institution: Eindhoven University of Technology
Language: English
Subfield: Social Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: bronze
OpenAlex ID: W4415000193