Work

Graph Neural Network and Fingerprint Attributions of Antibacterial Activity: Atom-Level Agreement Varies Across Random Seeds Reference distributions, topology controls and multi-instance replication

Year: 2026
Type: preprint
Abstract: ML models with different inductive biases are routinely used to prioritise compounds for antibacterial screening, and attribution methods are used to explain them. Circular fingerprints consumed by tree ensembles and message-passing graph networks share neither featurisation nor optimisation, so agreement between their explanations would carry information that neither method alone provides. Whether two independently derived attribution schemes identify the same atoms of the same molecule, and whether such agreement survives retraining, has received little quantitative examination. We curated a published Staphylococcus aureus screen (39,312 compounds, 512 actives) to 38,748 compounds by stereochemistry-aware deduplication and partitioned it by Bemis–Murcko scaffold into training (26,345; 123 actives), validation (4651; 31), and held-out (7752; 302; 3.90%) sets. Fingerprint Random Forest reached a held-out AUPRC of 0.1233 fitted on the training portion alone, and 0.1518 when refitted on training plus validation, with a 9.75-fold enrichment in its top 50; eight AttentiveFP networks differing only in random seed reached a median of 0.0834 (range 0.0587–0.0949). TreeSHAP attributions were verified for additivity (error 3.69 × 10⁻⁷) and projected onto atoms through Morgan bit environments with a coverage normalisation that reduced their correlation with atom degree from + 0.268 to + 0.069; readout attention was obtained by a forward-pass replication verified against the model's own output (logit error 1.91 × 10⁻⁶). Across 60 molecules the median per-molecule Spearman correlation between the two maps was + 0.0890, exceeding a size-matched cross-molecule reference in 6 of 8 instances. The agreement is not topological: the attention–degree association varies in sign across instances (− 0.2562 to + 0.1497) while the |SHAP|–degree association is invariant at + 0.0439. Across instances, the agreement spans − 0.1067 to + 0.1547, a range including both signs, and no prespecified antibiotic core survived Benjamini–Hochberg correction under either scheme. Cross-model atom-level attribution agreement in this low-data regime is small, exceeds a demanding reference in most but not all training runs, and is not attributable to atom connectivity. Because its across-instance range includes both signs, a single training run can misstate both the magnitude and the direction of the agreement it reports. Attribution comparisons in comparable data regimes should be reported as distributions over independently trained instances rather than as point estimates.
Institution: ITMO University
Language: English
Subfield: Drug Discovery
Open Access status: green
OpenAlex ID: W7226011229
Cites: 30
Cited by:
Related to: 10