Year: 2026
Type: preprint
Abstract: MoleculeACE is a widely used benchmark that annotates activity-cliff compounds in 30 ChEMBL datasets and reports that machine-learning models predict them poorly. Its “soft consensus” criterion admits a compound pair when any of three similarities reaches 0.9: ECFP4 Tanimoto similarity, Tanimoto similarity of anonymized molecular graphs, or normalized Levenshtein similarity between SMILES strings. The first two are computed after parsing; the third operates on the supplied strings, which the benchmark code does not canonicalize. This study held molecules, potencies, thresholds and code fixed and varied only how the same molecules werewritten. A re-implementation reproduced the distributed labels exactly in 27 of 30 datasets, and canonical round-tripping changed amedian of 0.00% of labels. Re-serializing themolecules with two independent cheminformatics toolkits, with every rewritten string verified to encode the original molecule, changed a median of 8.98% (Indigo) and 9.48% (Open Babel) of compound labels; Kekulé SMILES from RDKit changed 5.23%, and randomized SMILES 17.05%. The graph-derived criteria were invariant across all rewrites. Because MoleculeACE stratifies its train/test split on the cliff label, changing the labels caused split membership to approach that of an independent redraw (31.3% of compounds reassigned, against a 32.21% ceiling). An end-to-end repair, with serialization-invariant labels, re-derived splits and retrained descriptorbased models, moved the pooled cliff penalty by −0.006 to +0.020 log units; every confidence interval included zero against a minimum detectable effect of 0.041–0.057, and for SVM the shift fell within the variability of same-definition split redraws of comparable magnitude. The distributed annotation is therefore reproducible but not invariant to valid serialization. Benchmarks that retain string-based criteria should specify and enforce the canonicalization contract (toolkit, version and output mode); otherwise, labels should be defined on parsed molecular representations. Serialization-invariant labels, splits and per-molecule predictions are released.
Source: ChemRxiv
Author: Phongsakon Sriwicha
Institution: King Mongkut's University of Technology Thonburi
Language: English
Subfield: Drug Discovery
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7220653861