Robust control by applying reinforcement learning to adapt explicit model predictive control policies
Year: 2025
Type: article
Abstract: Reinforcement learning (RL) is a data-driven optimal control technique that has seen limited adoption in the process industries owing to high operational data requirements and the need to balance safe control with exploration. In contrast, Model Predictive Control (MPC) has been established as the benchmark method for optimal control in industrial applications but relies on a dynamic model of the controlled process. In this work, MPC is used as a starting point for online, continuing actor-critic RL applied to the simulated quadruple tank benchmark. Optimal control actions were precomputed as a function of the state space using the plant model, and a neural network was fitted to these data to generate an explicit MPC policy. Subsequently, this policy was adapted during closed-loop interaction of the RL agent with the (simulated) true plant, which exhibited different dynamics to the nominal plant model. The RL controller resolved the effects of plant-model mismatch on closed-loop control performance in the minimum-phase operating region by finding the optimal policy incrementally. In the non-minimum phase operating region, inverse response prevented the RL agent from operating effectively. From multivariable zero analysis, it was shown that process zeros very close to the origin or in the right half plane introduce a significant risk of closed-loop instability under RL control. This work’s findings show both the potential and limitations of RL by adapting precomputed optimal control policies and optimal cost functions (value functions) developed using non-linear MPC.
Source: Computers & Chemical Engineering
Institution: Stellenbosch University
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $4,050
Funder: Universiteit Stellenbosch
OpenAlex ID: W4417213459