Vocabulary recurrence, rather than biological grounding, explains tokenization transfer in protein function prediction.
Year: 2026
Type: preprint
Abstract: Sequence-based protein function prediction requires a choice of tokenization. K-mers, the common default, correspond to no biological unit; enzymatic digestion offers a principled alternative, since tryptic peptides are the fragments proteases generate and mass spectrometry measures. We asked whether this grounding improves transfer to unseen proteins. Using six Gene Ontology terms and 23,723 UniProt proteins, we compared tryptic peptides, peptide topics (groups of related peptides), 3-mers, and amino- acid composition, with ESM-2 embeddings and homology transfer as reference points. Peptide performance followed vocabulary coverage: 92.2% of peptide tokens in test proteins were never seen in training, whereas almost every 3-mer had been seen. As a result, 38.9% of test proteins contained no peptide the model had seen, and with nothing to work from, the classifier gave all of them the same answer. The remaining 61.1% of proteins were classified almost perfectly with a random forest; however, because of this uncovered fraction, tryptic peptides scored lower overall than 3-mers under all three classifiers tested (macro-F1 0.640 against 0.690). These results suggest that vocabulary recurrence matters more than biological grounding for tokenization transfer.
Language: English
Subfield: Molecular Biology
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7236057337
