Year: 2026
Type: conference-paper
Abstract: Annotator disagreement on tasks like natural language inference (NLI) reflects genuine linguistic ambiguity, yet most fine-tuning recipes treat every example as equally learnable.We ask whether this external signal of ambiguity predicts per-example learning dynamics under LoRA, the most widely used parameter-efficient fine-tuning method, and find that it does.Correlating annotation entropy (from ChaosNLI's 100 labels per example) with per-example area under the loss curve (AULC) on SNLI and MNLI, the correlation is positive in all 25 conditions tested (Spearman ρ=0.06-0.43),with decoder-only models showing stronger correlations than encoders at matched LoRA rank.More strikingly, under LoRA contested examples exhibit unlearning: their gold-label loss increases during training, a pattern that is largely absent under full fine-tuning and IA 3 in the encoder setting where matched comparisons are available, and that we also observe under LoRA on two decoder-only models.The effect survives partial-correlation controls and replicates across seeds and datasets.A preliminary noiseinjection experiment is consistent with these findings.
Author: Brady Steele
Institution: Georgia Institute of Technology
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166886957