Rule‐Based Versus Generative Extraction of Psychological Symptoms From Forensic Medical Certificates: A Feasibility Study in the French ORFeAD Network
Year: 2026
Type: article
Abstract: Forensic medical certificates for victims of interpersonal violence describe psychological symptoms in unstructured prose, rarely explicit and requiring inference. To assess the feasibility of two extraction pipelines-rule-based and a locally served generative large language model-in a French multicentric forensic network, and to identify variables usable without human verification. 110 randomly selected 2021 certificates from 12 ORFeAD units were processed by both pipelines: rule-based (segmentation, lexicons, regular expressions, negation detection) and generative (Llama 3 8B, 4-bit, Ollama, near-default settings). 35 binary variables, 23 psychological or subjective, were coded by two forensic physicians with third-party arbitration. Sensitivity, specificity and accuracy were computed per variable; 85% reliability defined usability. No inferential testing was performed. Specificity was high for most variables; sensitivity was heterogeneous and low for low-prevalence symptoms. 24 of 35 variables met the threshold under at least one pipeline, 10 under both, 11 under neither. Processing required 0.1-0.15 CPU-seconds per document versus 20-60 GPU-seconds. Out of the box, the local model did not clearly outperform the rule-based pipeline, at three orders of magnitude greater energy cost and requiring dedicated GPUs. Per-variable qualification is a prerequisite for research use; neither pipeline supports individual-level decisions.
Source: Behavioral Sciences & the Law
DOI: 10.1002/bsl.70097
Institutions: Centre Hospitalier Sud Francilien, Centre National de la Recherche Scientifique, Inserm, Université Sorbonne Paris Nord, Assistance Publique – Hôpitaux de Paris, Hôpital Avicenne, Institut de Recherche Interdisciplinaire sur les Enjeux Sociaux, École des hautes études en sciences sociales
Language: English
Field: Social Sciences
Domain: Social Sciences
Open Access status: bronze
OpenAlex ID: W7218477197