Integrating Vision-Language Models with Reinforcement Learning for Human-Aligned Decision-Making of Autonomous Vehicles
Year: 2024
Type: dissertation
Abstract: This thesis focuses on developing a new approach for improving the decision-making process for autonomous vehicles (AVs) in complex urban driving scenarios, particularly at unsignalized intersections, using the reinforcement learning (RL) framework. One of the primary difficulties in RL environments is designing a suitable reward model, which can often be challenging to achieve manually due to the complexity of the interactions and the driving scenarios. To address this challenge, this work utilizes Vision-Language Models (VLMs), particularly CLIP (Contrastive Language-Image Pretraining), to build an additional reward model based on visual and textual cues. CLIP’s ability to align image and text embeddings provides unique features for translating humanlike instructions into reward signals to guide the AV’s decision-making process. We apply two RL algorithms, Proximal Policy Optimization (PPO) and Deep Q-Network (DQN), to train an agent in complex unsignalized intersection environments. The performance of these algorithms is compared with and without the CLIP-based reward model, which highlights the impact of CLIP on the agent’s ability to learn and optimize its behavior in a way that aligns with desired driving actions. This study’s results show VLMs’ capabilities in improving RL-based decision-making in autonomous driving. We utilize the Highway-env simulation package, built on the OpenAI Gym framework, to test and validate the effectiveness of the proposed framework. Our simulation experiments indicate the framework’s effectiveness in optimizing traffic flow, minimizing collisions, and balancing both individual and collective benefits among road users. The results highlight the potential of integrating VLMs to provide human-aligned instructions, which could guide autonomous vehicle actions toward safer and more socially acceptable behaviors and eventually promote AVs’ safe and trustable deployment in future intelligent transportation systems.
Author: Doroudian, Erfan
Language: English
Subfield: Automotive Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7010564517