Do Language Models Know Which Drug Is Unsafe? Three Measurement Pitfalls and a Severity Gradient
Year: 2026
Type: preprint
Abstract: Medication-safety benchmarks report that language models often choose a drug the patient's risk rules out, but not whether the models lack the knowledge or fail to use it. Separating the two for open models (7B–72B), with a decision rule written before each experiment, we find that three common measurement choices can each mislead. The answer key: on 25 random RxSafeBench interaction cases, three auditor models from different providers name the key's unsafe drug in only 20–28%. Drug popularity: on drug pairs drawn from FDA labels, how often a drug is cited separates interacting from non-interacting pairs (AUROC 0.79) better than any open model. Answer format: moving a model between written probabilities and one-token scoring shifts its accuracy by up to fifteen points. Once these are controlled, the knowledge is graded by severity, also within popularity strata. In a pre-registered confirmation on 900 new pairs, every model separates major interactions better than moderate ones; open models reach AUROC 0.59–0.73 on major and 0.45–0.57 on moderate interactions, while a frontier commercial model reaches 0.95 and 0.72, so the gap does not shrink where errors are most dangerous. Use is distractible: shown the risk sentence for the unsafe drug only, 8–72B models gain 14–24 points; when every candidate carries a similar-looking warning, as real labels do, they gain 1–9 points. A simple retriever and a hazard-review prompt make open models more cautious, not more accurate. Code, pre-registrations and raw model outputs are included.
Author: Goin del Rio, Sebastian
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7235971876
