Homology-aware cross-validation strategies for generalization assessment in RNA secondary structure prediction
Year: 2026
Type: preprint
Abstract: RNA secondary structure prediction is a fundamental challenge in bioinformatics, essential for understanding the functional roles of non-coding RNAs. Recently, deep learning models have transformed the field with impressive results, leading to critical discussions regarding the validity of current crossvalidation strategies. On the one hand, traditional random partitioning yields overoptimistic results due to data leakage from uncontrolled homology. On the other hand, removing every training sequence that even slightly resembles the testing set penalizes learning-based methods, requiring generalization to completely out-of-distribution sequences. While removing sequences and retraining a machine-learned model is straightforward, the experimental data used for development and parameter tuning of classical thermodynamic methods cannot be removed, leaving these methods to benefit from knowledge leakage. Here we critically analyze the cross-validation strategies currently used for RNA secondary structure prediction: random splitting, clustering-based splitting, and leaving one RNA family out for testing. We demonstrate that this last strategy can leave structural patterns present only in the test partition, making the learning task impossible. Overall, we find that the apparent superiority of either type of method is an artifact of the splitting strategy. We propose here two homology-aware strategies: human-learned folds, which exclude from testing the structures already captured by decades of human learning, and similarity folds, which measure performance across the full spectrum of test-totrain similarity. Both enable fair comparisons, applying the same rigorous assessment to classical and learning-based methods. Data and source code are available at https://github.com/sinc-lab/xvalRNAfolding
Language: English
Subfield: Molecular Biology
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7166689216