Work

TSB-Forecast: A Short-Term Load Forecasting Model in Smart Cities for Integrating Time Series Embeddings and Large Language Models

Year: 2025

Type: article

Abstract: In smart cities, energy management systems are essential for efficient resource utilization, enhanced operational efficiency, and sustainability promotion. This work presents a novel load forecasting model, TSB-Forecast (Time Series BERT), a hybrid machine learning model aiming to improve short-term electrical demand forecasting by integrating structured and unstructured data. The model uses Sentence Bidirectional Encoder Representations from Transformers (SBERT) to extract semantic characteristics from textual news and Time to Vector (Time2Vec) to capture temporal patterns, acquiring cyclical behavior and context-sensitive impacts. TSB-Forecast shows significant improvements reducing MAE by 38.7%, RMSE by 18.2%, and SMAPE by 50.6% compared to several baselines, including the official ENTSO-E forecast, Extra Trees Regressor (ETR), a hybrid model with basic text processing (M6), and a deep learning model combining GRU and CNN (CNN-GRU).TSB-Forecast demonstrates a stronger ability to combine time-related and contextual information, which helps reduce errors. Our contributions in this paper are fourfold and can be summarized as follows: The model initially integrates multi-source inputs, encompassing load demand, BBC News, meteorological data, and holiday information. Secondly, it utilizes Time2Vec and SBERT for improved temporal and semantic feature encoding. Subsequently, Extreme Gradient Boosting (XGBoost) and Extra Trees Regressor (ETR) collectively enhance the robustness of a stacked ensemble. Finally, a thorough empirical analysis showing consistent outperformance of the proposed model over all the forecasting measures against baseline models.

Source: IEEE Access

DOI: 10.1109/access.2025.3597421

Authors: Mohamed Mahmoud Hasan, Neamat El-Tazi, Ramadan Moawad, Amany H. B. Eissa

Institutions: Future University in Egypt, Cairo University

Language: English

Topic: Energy Load and Power Forecasting

Subfield: Electrical and Electronic Engineering

Field: Engineering

Domain: Physical Sciences

SDGs: Industry, innovation and infrastructure, Affordable and clean energy, Sustainable cities and communities, Climate action

Open Access status: gold

Estimated APC paid: $2,075