Benchmarking Pharmacotherapeutic Hallucinations and Diagnostic Anchoring in Large Language Models Across High-Risk Primary Care and Renal Dosing Scenarios: A Clinical RLHF Audit
Year: 2026
Type: preprint
Abstract: Background: Large language models (LLMs) embedded within electronic health record (EHR) clinical decision support systems risk causing fatal pharmacotherapeutic hallucinations and sycophantic diagnostic anchoring. Objective: To quantify these failure modes across 50 adversarial clinical vignettes spanning KDIGO 2024 CKD, ADA 2024 euglycemic DKA, AHA ACLS WPW, and MGFA 2020 myasthenic crisis guidelines, and evaluate Constraint-Guided Chain-of-Thought (CoT) + Ontology-Grounded System Prompts. Methods: The ClinRLHF Benchmark evaluated paired zero-shot versus aligned outputs using a validated 5-point Clinical Safety rubric. Results: Constraint-guided alignment significantly improved Contraindication Detection Rate from 38.0% to 96.0% (p < 0.0001) and Sycophancy Resistance from 26.0% to 94.0% (p < 0.0001). Conclusions: Ontology-grounded deterministic constraints eliminate catastrophic prescribing hallucinations in medical LLMs.
Author: Mohammed, ibrahim
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7225730148
Cites:
Cited by:
Related to: