Model cards are not primary artifacts: stereochemistry in the MoLFormer and SMI-TED chemical foundation models
Year: 2026
Type: article
Abstract: Pretrained chemical language models are selected, embedded with, and fine tuned on the basis of what their documentation says they are. We report that the released model card for MoLFormer, one of the most widely used chemical foundation models, states that isomeric information was removed from its training data, and that this statement is contradicted by artifacts distributed in the model's own repository. The shipped token frequency table records the tetrahedral stereocentre token [C@H] occurring 669,642,207 times, making it the twelfth most frequent token in a corpus of 47.2 billion tokens, ahead of fluorine, and stereochemical tokens account for 3.17% of all token occurrences. The described stripping is real but occurs at a different pipeline stage: isomeric = False is hardcoded into the fine tuning scripts and applied to downstream benchmark data, not to pretraining. We find a related divergence in SMI-TED, whose released vocabulary contains 2,393 tokens rather than the 2,988 reported, and whose shipped frequency table is identical to MoLFormer's, byte for byte, across all 2,357 shared tokens, despite the paper describing a different corpus. We release the extracted frequency table and an audit tool, and recommend that chemical model cards state stereochemistry handling separately for pretraining, fine tuning, and inference.
Source: Journal of Cheminformatics
Author: Nathan Kaplan
Institution: The University of Texas at Austin
Language: English
Subfield: Drug Discovery
Domain: Life Sciences
Open Access status: gold
Estimated APC paid: $2,390
OpenAlex ID: W7225834799