Contextual Embedding Bias Predicts LLM Near-Synonym Confusion Where Isolated Geometry Fails: Evidence from Domain-Specific Medical and Cultural Heritage Corpora
Year: 2026
Type: preprint
Abstract: Near-synonym confusion —- where a semantically similar but functionally distinct term is substituted for another –– constitutes a clinically important failure case of LLMs deployed in medical and high-stakes contexts. In retrieval- augmented generation safety literature, it is commonly assumed that the cosine similarity between isolated term embeddings predicts the probability of such a confusion. We directly test this assumption. We present NEARSYNONYMBENCH, a benchmark of 120 domain- specific near-synonym pairs from medical, Indian cultural heritage and emergency blood resource domains, annotated with confusion probes and stakes levels. We test four sentence transformer architectures and three LLM families under forced choice conditions. Our main result is dissociation: isolated embedding geometry accounts for no variance in LLM confusion behavior at all (r = −0.005, p = 0.958), while contextual embedding bias, or difference in the proximity of the query probe to each candidate term, constitutes a statistically significant predictor (ρ = 0.299, p < 0.001), accounting for 45.3 times more variance. This effect is especially pronounced for high stakes pairs (r = 0.312, p = 0.018).We additionally identify terminological evasion — accurate description without term commitment, as an underreported failure mode accounting for 28.3% of open-ended responses. NEARSYNONYMBENCH and all code are released publicly.
Author: Asthana, Priyanka
Institution: Indian Army
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7212239098
Cites:
Cited by:
Related to: