Trigger-first large language model-assisted adverse drug reaction surveillance: A pharmacist-supervised retrospective cohort study
Year: 2026
Type: article
Study design: Observational Study
Abstract: Background Active pharmacovigilance in low-prevalence settings requires review of large electronic health record cohorts. We evaluated a pharmacist-supervised workflow combining electronic triggers with large language model (LLM)-assisted review. Objective To assess candidate enrichment and stage-specific performance for adverse drug reaction (ADR) surveillance. Methods In this retrospective cohort study, two pharmacists established a blinded reference standard for 7950 exposure episodes from 7950 unique inpatients. A prespecified Chinese Hospital Pharmacovigilance System trigger selected candidates for Qwen3-4B-Instruct and an institution-specific low-rank adaptation (LoRA)-fine-tuned model trained on 410 patient-disjoint cases involving other medicines. Paired errors were compared using exact McNemar tests. Results Twenty-eight episodes were ADR-positive. The trigger selected 167 candidates (2.10%), with 100.0% sensitivity (95% confidence interval, 87.66–100.0%) and 98.25% specificity. For the base versus LoRA-fine-tuned model, sensitivity was 82.14% versus 92.86%, specificity 88.49% versus 94.96%, positive predictive value 58.97% versus 78.79%, and F1 score 68.66% versus 85.25%. LoRA reduced false positives from 16 to 7 (9 corrected, 0 new; P = 0.0039). The numerical increase in sensitivity was not statistically significant ( P = 0.250), with only 28 ADR-positive episodes available for evaluation. Conclusions Trigger-first screening reduced review volume, while LoRA fine-tuning improved false-positive control. LLM classifications should support pharmacist review and should not be used as an autonomous exclusion rule.
Language: English
Subfield: Toxicology
Domain: Life Sciences
Open Access status: gold
Estimated APC paid: $2,140
Award: G2258465308
OpenAlex ID: W7223015685