Work

Assessing long-term electricity market design for ambitious decarbonization targets using multi-agent reinforcement learning

Year: 2025

Type: article

Abstract: Electricity systems are key to transforming today’s society into a carbon-free economy. Long-term electricity market mechanisms, including auctions, support schemes, and other policy instruments, are critical in shaping the electricity generation mix. In light of the need for more advanced tools to support policymakers and other stakeholders in designing, testing, and evaluating long-term markets, this work presents a multi-agent reinforcement learning model capable of capturing the key features of decarbonizing energy systems. Profit-maximizing generation companies make investment decisions in the wholesale electricity market, responding to system needs, competitive dynamics, and policy signals. The model employs independent proximal policy optimization, which was selected for suitability to the decentralized and competitive environment. Nevertheless, given the inherent challenges of independent learning in multi-agent settings, an extensive hyperparameter search ensures that decentralized training yields market outcomes consistent with competitive behavior. The model is applied to a stylized version of the Italian electricity system and tested under varying levels of competition, market designs, and policy scenarios. Results highlight the critical role of market design for decarbonizing the electricity sector and avoiding price volatility. The proposed framework allows assessing long-term electricity markets in which multiple policy and market mechanisms interact simultaneously, with market participants responding and adapting to decarbonization pathways. • This work develops a reinforcement learning model for long-term electricity markets. • Multi-agent Independent Proximal Policy Optimization is used. • The model enables assessment of simultaneous policy and market interactions. • An extensive hyperparameter search is applied to the competitive environment. • Tests in a stylized Italian system under deep decarbonization scenarios.

Source: Energy and AI

DOI: 10.1016/j.egyai.2025.100665

Authors: Javier Gonzalez-Ruiz, Carlos Rodriguez-Pardo, Iacopo Savelli, Alice Di Bella, Massimo Tavoni

Institutions: RFF-CMCC European Institute on Economics and the Environment, CMCC Foundation - Euro-Mediterranean Center on Climate Change, Politecnico di Milano, Bocconi University

Language: English

Topic: Electric Power System Optimization

Subfield: Electrical and Electronic Engineering

Field: Engineering

Domain: Physical Sciences

SDGs: Climate action, Affordable and clean energy, Industry, innovation and infrastructure

Open Access status: gold

Estimated APC paid: $3,630