Calibrated? Not for Everyone: How Sexual Orientation and Religious Markers Distort LLM Accuracy and Confidence in Medical QA
Year: 2026
Type: conference-paper
Abstract: Safe clinical deployment of Large Language Models (LLMs) requires not only high accuracy but also robust uncertainty calibration to ensure models defer to clinicians when appropriate.Our paper investigates how social descriptors of a patient (specifically sexual orientation and religious affiliation) distort these uncertainty signals and model accuracy.Evaluating nine general-purpose and biomedical LLMs on 2,364 medical questions and their counterfactual variants, we demonstrate that identity markers cause a "calibration crisis".Homosexual markers consistently trigger performance drops, and intersectional identities produce idiosyncratic, non-additive harms to calibration.Moreover, a clinician-validated case study in an open-ended generation setting confirms that these failures are not an artifact of the multiplechoice format.Our results demonstrate that the presence of social identity cues does not merely shift predictions; it affects the reliability of confidence signals, posing a significant risk to equitable care and safe deployment in confidence-based clinical workflows.(A) Accuracy (↑ is better) (Base in %, others: ∆ vs. Base; shaded cells p ≤ 0.05) Base +hetero +homo +Cat +Mus +Ath +homo+Cat +homo+Mus +homo+Ath Llama-3.2-3B55.58 +0.72 -0.33 +1.15 +0.09 +0.60 -3.46 -1.31 -2.66 Qwen3-4B 56.77 -1.52 -3.39 +0.51 -1.06 -0.51 -1.10 -1.86 -2.46 Bio-Medical-Llama-3-8B 64.21 -1.60 -2.37 -1.10 -2.41 -0.72 -5.58 -4.44 -4.27 Llama-3.1-8B57.57-1.65 -2.62 -1.52 -1.31 -0.12 -4.19 -2.32 -3.38 Qwen3-30B 73.39 -0.25 -0.97 -0.97 -1.60 -0.21 -1.39 -2.28 -1.73 Llama-3.1-70B84.31 -1.74 -2.92 -0.77 -1.48 -1.10 -3.47 -1.95 -2.84 OpenBioLLM-70B 77.44 -2.65 -7.21 -1.86 -2.06 -2.19 -5.10 -2.65 -3.78 GPT-4.1-mini78.43 -1.40 -3.05 -1.53 -1.15 -0.55 -2.46 -3.56 -2.88 GPT-5.1 89.21 -0.80 -1.35 -0.84 -0.63 -0.67 -0.59 -1.44 -1.52 Llama-3.2-3B64.78 -0.53 -1.31 -2.53 -1.57-1.08 -2.74 -1.01 -1.81 Qwen3-4B 74.56 -0.46 -2.73 -0.11 -0.97 -0.43 -1.82 -1.40 -2.09Bio-Medical-Llama-3-8B 74.47 -1.43 -1.75 -1.04 -0.98 -0.84 -2.64 -2.24 -2.46 Llama-3.1-8B59.98 -1.69 -3.43 -2.38 -2.06 -1.37 -4.02 -2.74 -2.61 Qwen3-30B 83.23 -0.62 -1.17 -1.31 -0.93 -0.36 -0.39 -0.75 -0.90 Llama-3.1-70B85.88 -0.95 -2.56 -1.04 -1.59 -1.06 -2.19 -2.27 -2.07 OpenBioLLM-70B 74.93 -1.99 -5.49-2.41 -4.41 -2.76 -4.96 -4.59 -3.65 GPT-4.1-mini91.49 +0.14 -0.13 +0.20 -0.09 -0.08 -0.39 -0.18 -0.09GPT-5.1 92.67 -0.81 -1.27 -0.70 -0.94 -0.55 -0.84 -1.14 -0.86
Authors: Alberto Testoni, Iacer Calixto
Institutions: Amsterdam Public Health, Amsterdam University Medical Centers
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: hybrid
OpenAlex ID: W7166851702