Work

Price-responsive control using deep reinforcement learning for heating systems: Simulation and living lab experiment

Year: 2025

Type: article

Abstract: While buildings are one of the main sources of global carbon emissions, they also present significant opportunities for decarbonization. Building energy systems are typically controlled using pre-defined schedules and rule-based strategies, which usually lack adaptability to dynamic conditions such as changing weather states, occupancy and energy prices. In this study, we develop a price-responsive Deep Reinforcement Learning (DRL) controller to autonomously control the indoor temperature of a building. The designed controller aims to minimize heating costs while keeping the indoor temperature within the acceptable comfort bounds, responding to dynamic energy prices. To improve learning efficiency, this study introduces Temporally-Weighted State Difference (TSD), a method to compress future price data into a single state observation for the controller, enabling future-aware decisions while reducing training complexity. The controller is developed and trained in a simulation environment using a physically consistent neural network model. It is then implemented in a real-world residential unit in the NEST building in Dübendorf, Switzerland, and evaluated during the winter of 2024 to 2025 using the dynamic electricity prices of the Swiss market. Experimental results indicate that incorporating future price information leads to more informed decisions by the controller and improves the performance of DRL by 11.5%. Experimental results show a 79% reduction in heating costs compared to a rule-based controller, at a cost of exceeding the thermal comfort range by 0.3 °C on average. The study demonstrates the potential of price-responsive DRL in reducing energy costs and providing demand-side flexibility to the energy grid.

Source: Energy

DOI: 10.1016/j.energy.2025.138517

Authors: Reza Mokhtari, Mina Montazeri, Hanmin Cai, Philipp Heer, Rongling Li

Institutions: Technical University of Denmark, Swiss Federal Laboratories for Materials Science and Technology

Language: English

Topic: Building Energy and Comfort Optimization

Subfield: Building and Construction

Field: Engineering

Domain: Physical Sciences

SDGs: Affordable and clean energy, Climate action

Open Access status: hybrid

Estimated APC paid: $4,260