Work

A Novel Model for Optimizing Roundabout Merging Decisions Based on Markov Decision Process and Force-Based Reward Function

Year: 2025

Type: article

Abstract: Autonomous vehicles (AVs) are increasingly operating in complex traffic environments where safe and efficient decision-making is crucial. Merging into roundabouts is a key interaction scenario. This paper introduces a decision-making approach for roundabout merging that combines human driving behavior with advanced reinforcement learning (RL) techniques to enhance both safety and efficiency. The proposed framework models the decision-making process of AVs at roundabouts as a Markov decision process (MDP), optimizing the state, action, and reward spaces to more accurately reflect real-world driving behaviors. It simplifies the state space using relative distance and speed and defines three action profiles based on real traffic data to replicate human-like driving behavior. A force-based reward function, derived from constitutive relations, simulates vehicle-roundabout interactions, offering detailed, physically consistent feedback that enhances learning results. The results showed that this method effectively replicates human-like driving decisions, supporting the integration of AVs into dynamic traffic environments. Future research should address the challenges related to partial observability and further refine the state, action, and reward spaces. This research lays the groundwork for adaptive and interpretable decision-making frameworks for AVs, contributing to safer and more efficient traffic dynamics at roundabouts.

Source: Mathematics

DOI: 10.3390/math13060912

Authors: Qingyuan Shen, Haobin Jiang, Aoxue Li, Shidian Ma

Institution: Jiangsu University

Language: English

Topic: Autonomous Vehicle Technology and Safety

Subfield: Automotive Engineering

Field: Engineering

Domain: Physical Sciences

SDGs: Sustainable cities and communities, Industry, innovation and infrastructure

Open Access status: gold

Estimated APC paid: $3,137