What a low-bias filter leaves out: measured performance on the excluded negatives of a drug–target interaction benchmark
Year: 2026
Type: preprint
Abstract: Eligibility rules define which interactions contribute to a benchmark score. The low-bias BindingDB benchmark retains a drug only when it has both positive and negative labels, excluding measured inactive pairs involving drugs without a measured active. We reconstructed this rule in a declared BindingDB population and evaluated retained and excluded negatives against the same positive reference. After cleaning and an additional connectivity-overlap restriction, the excluded region contained 78.4% of the measured negative pairs. DrugBAN and MolTrans were trained in three evaluation designs with five seeds; a released BioT5 checkpoint was evaluated on its published split. Across these evaluations, pooled AUROC was 0.072–0.136 lower on excluded negatives. Lower performance also occurred within proteins with sufficient observations in both regions. DrugBAN remained ahead of MolTrans, but the change in their advantage depended on aggregation: it generally increased under pooled AUROC, whereas equalprotein weighting produced smaller or oppositely directed changes. Cross-protein comparisons accounted for approximately 99.8–99.9% of pooled AUROC's pair-comparison weight. The closer BioT5–DrugBAN comparison additionally depended on scoring convention. These results measure regional performance differences, conditional on the chosen positive reference; they do not estimate a causal filter effect or deployment performance. Benchmark reports should specify eligibility, coverage at the evaluated unit, metric aggregation, and performance on excluded regions where labels are available.
Source: ChemRxiv
Author: Justin Li
Institution: Tulane University
Language: English
Subfield: Drug Discovery
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7223096364