Double actor-critic with TD error-driven regularization in reinforcement learning
Year: 2025
Type: article
Abstract: To obtain better value estimation in reinforcement learning, we propose a novel algorithm based on the double actor-critic framework with temporal difference error-driven regularization, abbreviated as TDDR. TDDR employs double actors, with each actor paired with a critic, thereby fully leveraging the advantages of double critics. Additionally, TDDR introduces an innovative critic regularization architecture. Compared to classical deterministic policy gradient-based algorithms that lack a double actor-critic structure, TDDR provides superior estimation. Moreover, unlike existing algorithms with double actor-critic frameworks, TDDR does not introduce any additional hyperparameters, significantly simplifying the design and implementation process. The convergence of the proposed TDDR to the optimal value is analyzed under random updating and simultaneous updating patterns. Extensive experiments on various tasks, including MuJoCo and Box2D, demonstrate that TDDR performs competitively against 13 algorithms, including both benchmarks and state-of-the-art methods. It also achieves statistically significant performance gains across several environments.
Source: Neural Networks
Authors: Haohui Chen, Zhiyong Chen, Aoxiang Liu, Wentuo Fang
Institutions: Central South University, University of Newcastle Australia
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,110
OpenAlex ID: W4416423297