Finding Approximate Correlated Equilibrium for Decentralized Control
Year: 2025
Type: article
Abstract: We address the problem of finding an “optimal” decentralized control strategy, defined as achieving approximate Correlated Equilibrium, a generalization of Nash equilibrium. Each controller uses a Reinforcement Learning algorithm within a Markov game framework, abstracting continuous states into discrete ones. Controllers observe the system’s discrete state and incurred costs, updating their strategies privately. We prove convergence to approximate Correlated Equilibrium. Our algorithm is applied to a water distribution network with two controllers managing a tank’s water level in a decentralized setting, using real-world consumption data. Previously, this problem was deemed to be computationally challenging owing to the discrete controls (number of pumps switched on), resulting in a Mixed Integer optimization. The designed controllers aim to ensure stability, safety, and energy minimization. By training over multiple episodes, optimal weighting parameters for objectives are determined using Blackwell’s Approachability theorem. Simulation results show the attainment of approximate correlated equilibrium in accordance with our theoretical results.
Source: IEEE Control Systems Letters
Authors: Rahul Misra, Rafał Wiśniewski, Carsten S. Kallesøe
Institutions: Aalborg University, Grundfos (Denmark)
Language: English
Topic: Game Theory and Applications
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
Funder: Poul Due Jensens Fond
OpenAlex ID: W4410737647