Work

Robot Learning Optimal Control via an Adaptive Critic Reservoir

Year: 2025

Type: conference-paper

Abstract: In this paper we develop a reservoir-based adaptive critic framework for optimal real-time control of nonlinear robotic systems. Unlike traditional neural network (NN) based critics, deterministic reservoir computing significantly reduces computational complexity by requiring only the training of output weights. Using deterministic reservoir computing, the method efficiently approximates the value function and computes optimal control laws, achieving rapid error convergence and computational efficiency. In addition, a momentum-enhanced adaptation law is proposed to accelerate convergence rates. The framework also contributes to robot learning by enabling adaptive behavior through iterative policy improvement, allowing robots to autonomously refine their control strategies by self-learning. It is theoretically validated under the persistency of excitation condition and demonstrated through simulations of trajectory tracking compared against traditional NN parameterization.

Source: Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes

DOI: 10.1109/cdc57313.2025.11312880

Authors: Anthony Siming Chen, Kyriakos G. Vamvoudakis

Institutions: University of Nottingham, Georgia Institute of Technology

Language: English

Topic: Adaptive Dynamic Programming Control

Subfield: Computational Theory and Mathematics

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Science Foundation, Faculty of Engineering, University of Nottingham

Awards: SaTC: CORE: Medium: Collaborative: Enforcement of Geofencing Policies for Commercial Unmanned Aircraft Systems, Collaborative Research: CPS: Small: An Integrated Reactive and Proactive Adversarial Learning for Cyber-Physical-Human Systems, G6671297155, CAREER: Towards an Intermittent Learning Framework for Smart and Efficient Cyber-Physical Autonomy