Work

Deep learning perturbation models can outperform baselines on calibrated metrics

Year: 2026

Type: article

Abstract: Recent benchmarks report that deep-learning-based genetic perturbation models fail to outperform uninformative baselines. Here we introduce a positive control baseline and metric calibration framework, showing—across 14 datasets and 18 metrics—that common benchmarking metrics (for example, mean squared error and Pearson Δ ) are frequently miscalibrated, with reduced sensitivity to measure positive model performance. Under well-calibrated metrics, we find that deep-learning-based genetic perturbation models can outperform uninformative baselines.

Source: Nature Biotechnology

DOI: 10.1038/s41587-026-03307-w

Authors: Henry E. Miller, Gabriel M. Mejia, Francis J. A. Leblanc, Brendan Swain, Bo Wang, Lucas Paulo de Lima Camillo

Institutions: University Health Network, University of Toronto, Vector Institute, University of Cambridge

Language: English

Topic: Gene Regulatory Network Analysis

Subfield: Molecular Biology

Field: Biochemistry, Genetics and Molecular Biology

Domain: Life Sciences

Open Access status: hybrid

Estimated APC paid: $12,690

OpenAlex ID: W7215048940