TD3-Based Reinforcement Learning for Adaptive PID-like Control of Uncertain Dynamical Systems
Year: 2026
Type: article
Abstract: This paper presents a TD3-based reinforcement learning framework for adaptive PID-like control of uncertain dynamical systems. Although proportional–integral–derivative (PID) control remains widely used because of its simplicity, interpretability, and practical effectiveness, fixed-gain PID controllers often experience performance degradation in the presence of external disturbances, parameter variations, and changing operating conditions. To address this limitation, the control task is formulated as a continuous-action reinforcement learning problem in which the observation vector is constructed from PID-related error components, namely the tracking error, its integral, and its derivative. Based on these error-derived observations, a Twin Delayed Deep Deterministic Policy Gradient (TD3) agent learns a bounded continuous control policy through interaction with the environment while preserving a PID-like structural interpretation. The proposed framework is evaluated on a representative mass–spring–damper system under three challenging scenarios: external disturbance, parametric uncertainty, and their simultaneous presence. Its performance is further examined for both constant-reference regulation and sinusoidal reference tracking. The simulation results show that the learned controller achieves stable and accurate tracking, fast transient response, and robust behavior across varying operating conditions. These findings demonstrate the potential of TD3-based reinforcement learning as an effective adaptive PID-like control strategy for uncertain dynamical systems.
Source: Mathematics
DOI: 10.3390/math14101744
Institutions: Sivas Bilim ve Teknoloji Üniversitesi, King Faisal University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,137
OpenAlex ID: W7161623550