Taming deep reinforcement learning agents with pricing mechanism: Validation in power distribution systems
Year: 2025
Type: article
Abstract: Distributed energy resources, connected to power distribution systems, are increasingly operated by intelligent/learning agents. Such agents, looking to optimize their own payoff, can discover harmful ways to exploit the system. Hence, shielding critical systems of harmful agent behavior is of crucial importance. In this paper, the problem of designing an efficient operating mechanism for a power distribution system is taken on, considering the realistic case where the system’s resources do not possess this information and instead learn to improve their policies through experience. To that end, a multi-agent reinforcement learning algorithm is developed to model the participants’ learning-to-act process and consider the agents’ learning under different pricing schemes that shape the agents’ reward functions. Two popular pricing schemes (pay-as-bid and distribution locational marginal pricing) are presented, exposing that learning agents can discover ways to exploit them, resulting in severe dispatch inefficiency. A game-theoretic pricing scheme is presented that theoretically incentivizes truthful agent behavior, and empirically demonstrate that this property improves the efficiency of the resulting dispatch also in the presence of learning agents. In particular, the proposed scheme is able to outperform the popular distribution locational marginal pricing scheme, in terms of efficiency, by a factor of 15–17%.
Source: Energy and AI
Language: English
Topic: Smart Grid Energy Management
Subfield: Electrical and Electronic Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,630
OpenAlex ID: W4415629103