Work

ElectriQ: LLM dialogue benchmark for electric power marketing in renewable-integrated power systems

Year: 2026

Type: article

Abstract: As power systems decarbonise and digitalise, electric power marketing (EPM) is becoming a key interface between regulation, system operation and sustainable-energy deployment. However, many utilities still rely on human agents and rule-based chatbots with fragmented knowledge bases, which struggle with long, cross-scenario dialogues and often fail to provide compliant and verifiable responses. Although large language models (LLMs) show strong conversational ability, existing benchmarks underrepresent the sector-specific terminology, regulatory reasoning and multi-turn stability required in EPM. To address this gap, we present ElectriQ, a large-scale benchmark for LLM evaluation in EPM. ElectriQ contains over 550k multi-turn dialogues across six service domains and 24 sub-scenarios, and defines a unified evaluation protocol combining human ratings, automatic metrics, and two dedicated stress tests: Statutory Citation Correctness (SCC) and Long-Dialogue Consistency (LDC). For reproducible comparison, we further implement SEEK-RAG as a reference baseline to support evidence-grounded responses on ElectriQ. Experiments on 13 LLMs show that domain-aligned 7B models can match or surpass much larger models at lower computational cost, providing an auditable basis for deploying LLM-based EPM assistants in demand-side management, renewable integration and resilient grid operation.

Source: Energy Reports

DOI: 10.1016/j.egyr.2026.109669