The textual shell: language model evaluations in therapeutic drug monitoring test the numerical end and leave the interface unmeasured
Year: 2026
Type: preprint
Abstract: Therapeutic drug monitoring (TDM) turns a measured concentration into a dose or a monitoring plan. It has a computational core and a textual shell: the record is read into inputs at the front, the number written into a report at the back. Large language models fail by fabrication and pharmacokinetic engines by specification, so the models belong on the shell. Reported performance spans 10% to 94.9% across incompatible scales, and this review asks whether the evaluations have looked where the models would work. Of thirty evaluations on the chain, twenty-eight test the model on work something else settles: a measured concentration once, a study-supplied criterion twenty-seven times. Where a measured concentration adjudicates, the comparator ran once — 37.0% against 15.6% within ±2 mg/L — and on the only shared denominator both methods sat near a third, where the dispersion of the data places them. At the back of the shell sit two studies: ten fictional cases scored by one pharmacologist, and eleven patient questions rated against expert answers. At the front, extraction from a TDM record is unevaluated in what we retrieved. Thirty-three evaluations across nine dimensions show why the shell is unmeasured: accuracy has an independent value in thirty-two, completeness in five, harm grading in ten, repeatability in seven, reference reliability in seven of twenty-five. Little headroom where the answer is measured; none where it is written. We set out what shell-side evaluation requires and propose a ten-item checklist.
Authors: Lei Zhang, Yichao Xu
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7225452042