Work

OrthoSimulator: Empirically trained simulations for ground-truth benchmarking of orthology inference

Year: 2026
Type: preprint
Abstract: Orthology inference is foundational for comparative genomics, enabling the reconstruction of phylogenetic relationships and transfer of biological knowledge between species. Methods often infer different orthogroups and orthologous relationships, making robust benchmarking essential for evaluating their accuracy. Existing resources such as the Quest for Orthologs benchmark service and OrthoBench provide valuable benchmarks grounded in real biological data and expert curation. However, the true evolutionary histories in all of these benchmarks are unknown, and therefore the true accuracy of orthology inference methods is unknown. Simulations provide an approach to evaluate accuracy where the complete ground truth is known. However, simulations are only useful for benchmarking if they realistically reflect empirical data. Here, we introduce OrthoSimulator, a Python package for generating realistic, empirically calibrated benchmark datasets with known orthogroups, orthologous relationships, gene trees, and evolutionary events. We demonstrate its utility by benchmarking widely used orthology inference methods across simulations calibrated from multiple methods and empirical datasets. We show that method-to-method variability in ortholog inference accuracy is robust to variation in simulation parameters. We further provide a detailed evaluation of ortholog inference accuracy through independent cross-validation, and show that OrthoFinder is the most accurate of the four methods tested. OrthoSimulator is freely available at https://github.com/lauriebelch/OrthoSimulator . Significance Statement Comparative genomics relies on orthology, the identification of genes that descend from a single gene in a common ancestor. Many methods infer orthologs, and they often disagree, but their true accuracy is hard to measure because the true history of real genes is never known. We present OrthoSimulator, which learns from real genomes to simulate realistic gene families whose full history is recorded. The simulations complement existing benchmarks built from real genes by adding complete ground truth. We use them to compare four widely used methods and to measure errors that cannot be seen in real data, and the simulated genomes can test any method that reconstructs gene family evolution.
Language: English
Open Access status: green
OpenAlex ID: W7233975950
Cites: 47
Cited by:
Related to: 10