A Framework for Quantifying and Attributing Molecular Standardization Reproducibility in Cheminformatics Pipelines
Year: 2026
Type: preprint
Abstract: Molecular structure standardization, which normalizes salts, charge states, tautomers, isotopes and stereochemistry to a canonical form, is mandatory in essentially every cheminformatics pipeline. The protocols used to perform it nonetheless vary across tools, databases and configurations. Disagreement between standardization pipelines is documented at production scale, but existing analyses compare one pipeline against one reference, so they cannot identify which normalization decision produced a given disagreement. We present a measurement framework that treats standardization reproducibility as a quantifiable property of a dataset. Given an arbitrary set of protocols, the method determines whether they agree on each molecule's standardized identity, then attributes each disagreement to a specific normalization operation by systematic ablation. Agreement is evaluated under two identity conventions, canonical SMILES and InChIKey, which we show are not interchangeable. We applied the framework to a seeded random sample of 4,972 ChEMBL structures using three protocols that bracket the plausible range: a minimal one, one approximating the ChEMBL curation pipeline, and a deliberate upper-bound stress case. They agreed on 54.1% of molecules under canonical-SMILES identity (95% CI 52.8–55.5%) and 57.7% under InChIKey identity (56.3–59.0%). Ablation attributed divergence mainly to stereochemistry handling (66.6% of divergent molecules) and tautomer canonicalization (22.1%). Disabling stereochemistry removal, a step no major public pipeline performs, raises agreement to 75.5% and leaves tautomer canonicalization responsible for 87.4% of what remains. The cause spectrum is thus a property of the protocols compared and the dataset audited, not an intrinsic ranking of operations. A hand-assembled benchmark inverts it entirely, which argues for auditing the dataset actually in use instead of relying on published rates. Extending the comparison to the pipelines databases actually run, namely ChEMBL's own curation code and MolVS in two configurations, yields two further results. Our composed ChEMBL-like protocol reproduces ChEMBL's production pipeline on 99.8% of structures, so conclusions drawn from the composed protocols transfer to the real one. And two configurations of a single standardizer agreed with each other on only 56.0% of structures, while two independently developed standardizers agreed on 77.2%. Configuration choice within one tool spans a wider range of outcomes than the choice between tools, so naming the software used is not sufficient provenance. To test whether such divergence carries a practical cost, we applied the same protocols to 27,152 IC50 measurements across four targets. Grouping by standardized identity is what deduplication does, so protocol choice silently changes a dataset. The most aggressive protocol removed up to 9.5% of compounds by merging, fusing into single training labels compounds whose measured activities differed by up to three log units, a thousandfold difference in potency on a cardiac-safety endpoint. Cross-validated model performance was nonetheless statistically indistinguishable across protocols on all four targets, under both random-split and generalization-stressing scaffold-split cross-validation.
Source: ChemRxiv
Authors: Sree Vasthav Upputoori, S. Madhav Varma
Institution: University College Dublin
Language: English
Subfield: Drug Discovery
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7220651318