CLINES: Clinical LLM-based Information Extraction and Structuring Agent
Year: 2026
Type: article
Abstract: Objective Clinical narratives in electronic health records (EHRs) hold diagnostic, therapeutic, and temporal information often missing from structured fields; manual chart review remains the de facto standard but is slow, costly, and variable. We aimed to develop and evaluate CLINES, a pipeline that extracts and normalizes clinical concepts from free-text notes into ontology-grounded, auditable, schema-ready data without task-specific fine-tuning. Methods and Analysis We developed CLINES, a modular multi-step pipeline: semantic chunking of long notes; extraction by reasoning-capable large language models; attribute assignment (assertion, value+unit); UMLS normalization; explicit and relative date resolution; aggregation into an i2b2-style schema. Evaluation was conducted without task-specific supervised training on de-identified MIMIC-III, 4CE, and CORAL (breast, pancreas) notes. Comparators: rule/lexicon systems, transformer encoders, single-prompt LLM baselines, alongside chain-of-thought LLMs and full-size clinical NER encoders (BERT-base, GatorTron-base). Out-comes were F1 for entity, assertion, value+unit, and date processing, reported with 95% bootstrap CIs and paired-bootstrap FDR-corrected tests; inter-annotator agreement, hallucination taxonomy, cost, and component ablations are in Supplementary Materials. Results Across all datasets and tasks, CLINES ranked first on every task-dataset combination. F1 ranges were 0.69–0.87 (entity), 0.84–0.93 (assertion), 0.77–0.90 (value+unit), and 0.73–0.79 (date, where gold annotations were available); per-dataset values are in Supplementary Materials. Gains over the strongest single-prompt LLM were +0.21–0.38; transformer encoders trailed by +0.28–0.68 F1 on entity extraction. Performance remained stable as notes lengthened, while transformer baselines lost recall. Code-field gains and most assertion/value comparisons were significant at p FDR < 0.05 on paired-bootstrap tests; mention-level inter-annotator F1 on a cross-annotated subset was 0.82 (assertion κ = 0.85); a seven-class taxonomy on 700 stratified false positives found 54% of apparent false positives to be annotation gaps rather than true hallucinations. Conclusion CLINES translates narrative EHR text into ontology-grounded, auditable, schema-ready data, offering a practical route to scale chart-review-like extraction for cohort discovery and real-world evidence. CLINES substantially reduces, but does not eliminate, the need for manual review: the residual hallucination quantified here means human verification remains necessary for safety-critical use. CLINES is model agnostic–open and closed models can be substituted to meet cost, performance, and privacy goals. Summary box What is already known on this topic Free-text clinical notes hold diagnostic, temporal, and quantitative detail absent from structured EHR fields; rule-based systems and fine-tuned transformer encoders detect entities but normalize and attribute them inconsistently, and single-prompt large language models hallucinate and miss instructions on long notes. What this study adds CLINES, a training-free multi-step large-language-model pipeline, produces UMLS-normalized, attributed, time-stamped output that substantially outperforms single-prompt and chain-of-thought LLMs (+0.37 to +0.54 code F1) and full-size clinical encoders; the gains hold across open- and closed-weight backbones and are quantified with confidence intervals, inter-annotator agreement, a hallucination taxonomy, per-note cost, and component ablations. How this study might affect research, practice, or policy CLINES offers an auditable, model-agnostic route to scale chart-review-like extraction for cohort discovery and real-world evidence, while the quantified residual hallucination rate makes explicit that human verification remains necessary before safety-critical use.
Source: BMJ Digital Health & AI
Authors: Zongxin Yang, Hongyi Yuan, Raheel Sayeed, Amelia Tan, Enci Cai, Mohammed Moro, Xiudi Li, Huaiyuan Ying, Nicholas Brown, Griffin Weber, Sheng Yu, Isaac Kohane, Tianxi Cai
Language: English
Subfield: Molecular Biology
Domain: Life Sciences
Open Access status: diamond
OpenAlex ID: W4417152296