From replicates to casework: a systematic methodological review of data independence and transportability in artificial intelligence models for complex forensic chemical and biological evidence
Year: 2026
Type: article
Study design: Systematic Review
Abstract: Artificial intelligence (AI), particularly machine-learning models, is increasingly applied to complex forensic chemical, material, and biological evidence. High reported performance does not itself demonstrate generalization to new forensic material. We conducted a systematic methodological review of 446 reports and 1,005 eligible evaluation regimes, examining independence of final evaluation, physical and biological parent grouping, information leakage, external or shifted testing, and transportability. Following source-based methodological assessment and human adjudication, source-supported leakage was classified in 284/446 reports (63.7%); 36/446 (8.1%) showed strong potential concern, 85/446 (19.1%) had uncertain boundaries, and 41/446 (9.2%) had no identified leakage concern. External or shifted evaluation was reported in 278/446 reports (62.3%), although only 94/278 (33.8%) had an evaluation satisfying five strict criteria for transportability of an unchanged pipeline. At regime level, 135/521 external or shifted evaluations (25.9%) satisfied these criteria. Source-supported leakage occurred in 195/308 Biological reports (63.3%) and 87/133 Chemical/Material reports (65.4%): prevalence ratio 1.033 (95% CI 0.889–1.200). The primary comparison did not identify a significant difference across these domains. Casework claims should therefore be evaluated against the independence of the relevant forensic unit, the complete development–test boundary, the shift actually examined, any target-data adaptation, and the applicable performance denominator.
Source: Frontiers in Analytical Science
Authors: Paolo Bailo(Writing – original draft, Writing – review & editing), Carlotta Virginia Di Francesco, Giulia Vignali(Writing – original draft, Writing – review & editing), Riccardo Primavera(Data curation, Validation, Writing – review & editing), Andrea Piccinini(Writing – original draft, Writing – review & editing), Lorenzo Franceschetti(Writing – original draft, Writing – review & editing)
Institutions: Università di Camerino, University of Milan
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,353
OpenAlex ID: W7236183945
