Work

Energy-Aware Cooperative Multi-Agent Deep Reinforcement Learning for Sustainable and Precision Trajectory Tracking of an Industrial Robotic Arm: An SDG-Aligned Approach

Year: 2026
Type: article
Abstract: Control software that lowers actuator effort without degrading precision can be deployed on installed industrial robots without new hardware. This paper formulates trajectory tracking of a 5-degree-of-freedom (DOF) Mitsubishi RV-2AJ arm as a fully cooperative multi-agent problem, in which each joint is driven by a twin delayed deep deterministic policy gradient (TD3) agent under centralized training and decentralized execution (CTDE). Unlike MADDPG, MATD3 and MAPPO, the proposed CTDE-TD3 combines (i) a single twin centralized critic, shared by all joint agents and conditioned on all joint states and torques, which removes the non-stationarity faced by independent agents while each actor uses only three local measurements at run time; (ii) a shared reward with an inter-joint torque-covariance term, whose exact decomposition yields the admissible weight condition μ ≤ 2λ; and (iii) a dimensionless weight normalization. In simulation on three 3D reference trajectories over 10 training seeds, the controller reduced the tracking root-mean-square error (RMSE) by 54.5% and 37.5% and the squared-torque effort by 33.1% and 21.0% relative to centralized and independent-agent TD3, respectively. It was not dominated by MADDPG, MATD3, MAPPO or a tuned proportional–integral–derivative (PID) controller on unseen trajectories. Ablations and a capacity-matched centralized baseline show that the gains stem from the cooperative structure rather than from network size. Because the squared-torque index reflects control effort and copper loss rather than total electrical energy, electricity and CO2 savings are reported only as scenario estimates pending hardware validation.
Source: Automation
Language: English
Open Access status: gold
Estimated APC paid: $1,207
OpenAlex ID: W7236210317
FWCI:
Cites: 36
Cited by:
Related to: 10