Construction of a Large Language Model-Based Literature Evidence Grading System with Dual-Model Adversarial Review and Its Validation in Target Review of Active Components of Lithocarpus litseifolius
Year: 2026
Type: preprint
Abstract: Background High-quality component–target knowledge is essential for elucidating drug mechanisms, advancing translational research, and supporting precision prescribing. However, existing biomedical databases generally lack systematic evidence grading for component–target associations, while manual review is difficult to scale to the rapidly growing literature. Objective We aimed to develop a large language model (LLM)–based literature evidence grading system incorporating dual-model adversarial review and to validate its performance using the active compounds of Lithocarpus litseifolius as a case study. Methods The system comprised a language model layer and a deterministic rule layer. Two commercial LLMs served as the reviewer and error checker, respectively, forming an adversarial review process; deterministic rules that did not involve language models performed consensus merging and enforced calibration constraints, thereby eliminating stochastic variability at the merging stage. The system incorporated evidence grading criteria (Levels Ia–IV), HUGO Gene Nomenclature Committee (HGNC)–based target standardization, and pathogen-target classification rules. Validation used 776 PubMed records on trilobatin, phloridzin, and phloretin, with manual final review by a researcher as the reference standard. A same-source paired design compared single-model review with dual-model adversarial review. Results The error-checking model raised 1,149 challenges, of which 1,120 (97.5%) were adopted; 30 incorrectly listed targets were removed, and 148 and 58 records were downgraded and upgraded, respectively. Compared with single-model review, dual-model review increased verification accuracy from 78.5% to 84.8% (recall, 95.7%; F1 score, 80.4%) and five-level grading accuracy from 69.8% to 81.6% (Cohen’s κ, 0.460 to 0.652; McNemar’s test, both P < 0.001). The system supported the construction of a high-quality target library comprising 257 traceable records, which were consolidated into 74 unique targets (67 human and 7 pathogen targets); 2, 9, 58, and 5 component–target pairs were classified as Levels Ib, II, III, and IV, respectively. Conclusion Dual-model adversarial review combined with deterministic rule calibration significantly improved the accuracy and consistency of literature review and corrected incorrectly listed targets, erroneous mappings, and grading deviations produced by single-model review. This approach provides a reusable framework for scalable, high-quality biomedical knowledge-base construction and evidence-based data governance.
Source: Research Square
Institution: Guangxi University of Chinese Medicine
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7225816646