Work

Demographic prompt cues shift clinical recommendations in multimodal large language models: a multi-model audit of 25,380 dental radiograph assessments

Year: 2026

Type: article

Abstract: BACKGROUND: Multimodal large language models (LLMs) are increasingly being evaluated for clinical image interpretation, but whether patient demographic cues influence their recommendations remains unclear in dental radiology-a leading domain for AI-assisted diagnostics, and one marked by pronounced disparities in oral health. METHODS: We conducted a within-image, between-condition audit study using 705 panoramic dental radiographs from the publicly available DENTEX dataset. Each image was submitted to three commercial multimodal LLMs (Gemini 2.5 Flash, GPT-5.4, Claude Sonnet 4.6) under 12 experimental conditions: five race/ethnicity groups × two sex categories plus two controls, yielding 25,380 independent zero-temperature API calls. Each call returned a structured JSON object containing ten prespecified clinical recommendation variables. Delta variables were computed against the primary control, and linear mixed-effects models with image identity as a random intercept tested for race/ethnicity (H1), sex (H2), and Race × Sex interaction (H3) effects. The five primary ordinal outcomes were evaluated against a Bonferroni-corrected significance threshold of α = 0.01. RESULTS: All three models altered recommendations when demographic cues were introduced. Claude showed the broadest sensitivity, with significant race effects on treatment invasiveness, prognosis, and confidence (all p < .001), and significant sex effects on prognosis, confidence, and invasiveness. GPT-5.4 effects were concentrated in confidence and in invasiveness for Hispanic patients (β = +0.052, p < .001). Gemini produced higher invasiveness for Black patients (β = +0.026, p < .001). No Race × Sex interactions reached significance. All ordinal shifts were below 6% of one category width. Inter-model diagnostic agreement under control prompts was near chance (Cohen's κ = 0.009-0.159). Findings were robust across four prespecified sensitivity analyses. CONCLUSIONS: Demographic prompt cues produced small but systematic shifts in multimodal LLM clinical recommendations for dental radiographs, with model-specific sensitivity profiles. Because clinical accuracy was not benchmarked, these findings should be read as demographic-cue sensitivity rather than clinical bias. Inter-model agreement was near chance, indicating that model choice currently introduces more variability than any demographic label. Demographic-cue audits and model-specific reliability evaluations should both accompany clinical deployment, particularly in oral health, where racial and socioeconomic disparities are already documented.

Source: BMC Oral Health

DOI: 10.1186/s12903-026-08644-5