What Limits Lithium-Ion Battery State of Charge Estimation with a Hybrid Large Language Model? Layer Type Aware Adaptation and the Initial Condition Bottleneck
Year: 2026
Type: article
Abstract: State of charge (SoC) is the unobserved quantity that a battery management system must reconstruct from terminal voltage, current and temperature. Recent estimators adapt frozen large language models by prompt-tuning the key–value cache of a dense decoder, an interface that does not exist in the linear-attention majority of current hybrid models. We adapt a frozen four-billion-parameter hybrid decoder by matching the injection to each layer type: learned key–value prefixes in the full-attention minority, and a trainable initial recurrent state (S0) in the gated-delta majority. Training only 22.3 million parameters (0.55%), this layer-type-aware adaptation reduces mean absolute error (MAE) by a factor of 2.3 relative to prefixes alone on a preliminary single-cell diagnostic. On the final four-billion-parameter configuration, the same comparison yields a factor of 1.86: pooled mean absolute error decreases from 2.18% for key–value prefixes alone to 1.17% for the full adapter. The nickel-based drive-cycle pool is 1.17% MAE on 18,712 windows. That pool does not include LFP. The LFP value on shared files is a fit of 1.22%. Held-out files of the same cell reach 2.53% in a separate run, and the nickel-based pool of that run is 1.24%. This represents parity with dedicated dense-LLM estimators without claiming superiority over compact conventional observers. On the same drive-cycle split, a gated recurrent unit with 0.2 million parameters reaches 1.18% pooled MAE, and the adapter reaches 1.17%. Instead, the primary contribution is mechanistic. We demonstrate that 98.4% of the residual error is a constant per-window offset on that diagnostic. On this configuration, a constant per-window offset accounts for 0.98 of the residual energy. The estimate therefore behaves as a near-exact Coulomb integration after an initial-condition inverse problem at t=0. Compact models trained on the same windows show the same offset-dominated residual, so this pattern is a property of the short-window task as well as of the backbone. This factorization is consistent with the observed failure regimes—concentration of error below 0 °C, and a zero-shot LFP error of 12% MAE on the reference range of about 0–47% state of charge, about ten times the nickel-based pool. Held-out files of an LFP cell partly seen in training reach 2.53% in a separate run. Consequently, we argue that the engineering target for LLM-based battery estimation should transition from expanding backbone capacity to explicitly conditioning the initial state.
Source: Batteries
Author: Alexey A. Druzhinin
Language: English
Subfield: Automotive Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,258
OpenAlex ID: W7222334280