Work

Adaptive Clipping Strategy for Proximal Policy Optimization in Autonomous Vehicle Decision Making

Year: 2026
Type: article
Abstract: One of the main tasks of autonomous vehicles driving on highways is lane keeping based on intelligent decisions. However, this becomes challenging in constrained and complex traffic environments due to unstable policies, limited reward structures, and the need for real-time decision-making. To address these, this work presents a reinforcement learning-based approach using Proximal Policy Optimization (PPO) for highway lane keeping and obstacle avoidance. An accurate simulation environment is built using Unity and ML-Agents. PPO employs a clipping strategy designed to prevent large policy updates. These large policy updates can cause unstable training; however, the impact of different clipping strategies on training stability is significantly affected. To address this, this work presents a new clipping strategy, which is named Adaptive Clipping. To understand the effects of clipping parameter strategies on policy learning within PPO's clipped surrogate loss, four types of clipping strategies, constant, linearly decaying, exponentially decaying, and stepwise decaying, are compared with the proposed one. Each strategy is evaluated under the same environmental configuration to have a comprehensive comparison. The results demonstrate that the adaptive clipping strategy achieves highly accurate performance with a final average reward of 301.74 and a mean test success rate of 90.66%. The constant, stepwise, and exponentially decaying strategy presents comparable but slightly lower performance, while the linearly decaying strategy represents slightly worse results with the test success rate of 76.66%. Eventually, this work contributes a complete framework from Unity simulation to policy learning using discrete action control, vehicle-mounted sensors, and a robust reward function.
Language: English
Open Access status: hybrid
Estimated APC paid: $2,645
OpenAlex ID: W7202147366
FWCI:
Cites:
Cited by:
Related to: 10