Dual-decoupling of deep reinforcement learning reward function for integrated energy system scheduling: A collaborative mechanism of constraint separation and hierarchical decision-making
Year: 2026
Type: article
Abstract: • A dual-decoupling mechanism for IES scheduling is proposed. • CMDP-based algorithm is designed to decouple objectives and constraints. • Hierarchical architecture is designed to decouple the decision-making of electric and thermal units. • The proposed method balances both the economic efficiency and safety of IES. Deep reinforcement learning (DRL) applied to the integrated energy system (IES) scheduling tasks often leads to reward functions composed of multiple coupled components due to the complex structure of IES. However, reward design determines how agents perceive task objectives and action values, and excessive reward terms can cause policy confusion, resulting in suboptimal optimization. To address the multi-objective reward coupling challenge in IES optimal scheduling, this study proposes a DRL method based on a dual decoupling mechanism for reward functions. At the algorithmic level, this method reconstructs the Markov Decision Process (MDP) as a Constrained Markov Decision Process (CMDP) and introduces two improved algorithms, LTD3 and LSAC, combining dual decomposition to separate economic objectives from safety constraints in reward functions. At the architectural level, a hierarchical reinforcement learning framework (LTD3-LSAC) is established to decouple decision-making for electric units and thermal units, thereby separating electrical and thermal components in reward functions. Through a collaborative technical pathway combining horizontal algorithmic improvement and vertical architectural optimization, the proposed DRL method achieves dual decoupling of reward functions in IES scheduling tasks. Two-stage ablation experiment results show that the proposed LTD3-LSAC framework effectively improves the operational economy of IES: the total operational cost is reduced by 13.28% compared with the traditional strong penalty method, while the cumulative constraint violation index is controlled at 4.21, which is much lower than 36.78 of the weak penalty method; compared with the single-layer architecture, the hierarchical strategy further reduces the total operational cost by 3.23%. These results verify the superiority of the proposed dual decoupling mechanism in this study from the two dimensions of constraint handling and multi-energy coordination, respectively.
Institution: Beijing Jiaotong University
Language: English
Subfield: Electrical and Electronic Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,920
OpenAlex ID: W7119128490