Work

Optimizing Electric Bus Charging Scheduling With Uncertainties Using Hierarchical Deep Reinforcement Learning

Year: 2025

Type: article

Abstract: The growing adoption of electric buses (EBs) represents a significant step toward sustainable development. By utilizing Internet of Things (IoT) systems, charging stations can autonomously determine charging schedules based on real-time data. However, optimizing EB charging schedules remains a critical challenge due to uncertainties in travel time, energy consumption, and fluctuating electricity prices. Moreover, to address real-world complexities, charging policies must make decisions efficiently across multiple time scales and remain scalable for large EB fleets. In this article, we propose a hierarchical deep reinforcement learning (HDRL) approach that reformulates the original Markov decision process (MDP) into two augmented MDPs. To solve these MDPs and enable multitimescale decision-making, we introduce a novel HDRL algorithm, namely, double actor-critic multiagent proximal policy optimization enhancement (DAC-MAPPO-E). Scalability challenges of the double actor-critic (DAC) algorithm for large-scale EB fleets are addressed through enhancements at both decision levels. At the high level, we redesign the decentralized actor network and integrate an attention mechanism to extract relevant global state information for each EB, decreasing the size of neural networks. At the low level, the multiagent proximal policy optimization (MAPPO) algorithm is incorporated into the DAC framework, enabling decentralized and coordinated charging power decisions, reducing computational complexity and enhancing convergence speed. Extensive experiments with real-world data demonstrate the superior performance and scalability of DAC-MAPPO-E in optimizing EB fleet charging schedules.

Source: IEEE Internet of Things Journal

DOI: 10.1109/jiot.2025.3552532

Authors: Jiaju Qi, Lei Lei, Thorsteinn Jonsson, Dusit Niyato

Institutions: University of Guelph, Nanyang Technological University

Language: English

Topic: Electric Vehicles and Infrastructure

Subfield: Electrical and Electronic Engineering

Field: Engineering

Domain: Physical Sciences

SDGs: Industry, innovation and infrastructure, Affordable and clean energy, Sustainable cities and communities

Open Access status: green

Funder: Natural Sciences and Engineering Research Council of Canada