Year: 2026
Type: preprint
Abstract: TrialCalibre1 is an agentic system that connects clinical evidence, study design, and executable analyses. It supports reference-trial replication, direct target-trial emulation, and BenchExCal, which combines benchmarking, extension, and calibration sensitivity analysis. We evaluated an ASCET-based benchmark and an extension comparing clopidogrel with aspirin among adults aged 81–90 years using synthetic PCORnet-style claims. Two reviewers assessed benchmark extraction, protocol decisions, and implementation. After discussion, 53/62 and 50/62 assessable SQL checks were rated met; both reviewers rated 12/13 assessable R checks met. Protocol and proxy disagreements remained. A separate ten-question routing development set yielded 10/10 route matches and 7/7 applicable referencetrial matches. The agent benchmark and extension included 6,069 and 1,602 patients. The extension risk ratio was 0.848 (95% CI 0.569–1.184), with a calibrated sensitivity estimate of 1.274 (0.439–3.700). The human implementation provided descriptive comparisons, not same-data validation. Dose proxies, unpopulated covariates, residual imbalance, implementation deviations, and ignored cross-cohort covariance limited interpretation. This case demonstrates a connected workflow whose protocols, code, and results can be reviewed.
Source: medRxiv
Authors: Amir Habibdoust, Xing Song
Institution: University of Missouri
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7221814723