Preconditioned Test-Time Adaptation for Out-of-Distribution Debiasing in Narrative Generation
Year: 2026
Type: conference-paper
Abstract: Although debiased large language models (LLMs) excel at handling known or low-bias prompts, they often fail on unfamiliar and high-bias prompts.We demonstrate via out-ofdistribution (OOD) detection that these highbias prompts cause a distribution shift, degrading static model performance.To enable realtime correction, we propose CAP-TTA, a testtime adaptation framework.CAP-TTA triggers context-aware LoRA updates only when a bias-risk score exceeds a set threshold.By utilizing an offline precomputed diagonal preconditioner, it ensures fast and stable optimization.Across multiple benchmarks and human evaluations, CAP-TTA effectively reduces toxicity/bias score with significantly lower latency than standard optimization methods (e.g., AdamW or SGD).Furthermore, it prevents catastrophic forgetting, and substantially improves narrative fluency over state-of-the-art baselines without compromising debiasing performance.
Authors: Hanwen Shen, Ting Ying, Jiajie Lu, Shanshan Wang
Institutions: Stevens Institute of Technology, University of Macau
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166874049