HydroTuring: A physics probe benchmark testing mass, energy and momentum conservation in hydrologic models
Year: 2026
Type: preprint
Abstract: Hydrologic models, including deep learning systems now deployed for operational flood forecasting, are evaluated primarily by how closely their simulated hydrographs match observed streamflow. However, high hydrograph skill does not guarantee physical consistency: in a synthetic calibration experiment, a conceptual model that loses 15% of its water input to an unreported sink still reaches a Nash-Sutcliffe efficiency of 0.93, and no shared benchmark tests whether models obey fundamental conservation laws. Here we introduce HydroTuring (v0.1.0), a black-box benchmark of 37 physics probes that test mass (24 probes), energy (7) and momentum (6) conservation in hydrologic models. Each probe runs a controlled experiment on forcing generated at run time from a recorded seed and applies binary pass or fail criteria validated against 18 physically exact and 59 deliberately broken reference models. Across 32 evaluated models (3 exact references, 7 AI-based models and 22 process-based models), only the 3 exact references pass every probe among the 28 models scored on 5 or more probes. Process-based and AI-based models pass similar shares of their scored probes (76% and 74%), and a pre-trained flood-forecasting LSTM fails 6 of the 11 runoff-only probes it can be scored on. Furthermore, 49% of all 1,184 model-probe pairs remain unscored because models omit the required state or flux variables, while the scored failures occur in perturbation responses and state bounds rather than long-term water-budget closure. HydroTuring provides a reproducible smoke test for model development, peer review and operational deployment under an open community protocol.
Institutions: Institute of Arctic and Alpine Research, University of Colorado Boulder, South China University of Technology, Columbia University, Hong Kong University of Science and Technology, Politecnico di Milano, École Polytechnique Fédérale de Lausanne, Michigan State University, Dalian University of Technology, Chinese Academy of Sciences, Institute of Mountain Hazards and Environment, Indian Institute of Technology Mandi, University of Oklahoma, Beijing Normal University, University of Arizona, University of Illinois Urbana-Champaign, Xi'an Jiaotong University, Stanford University, Sun Yat-sen University, University at Buffalo, State University of New York
Language: English
Subfield: Water Science and Technology
Field: Environmental Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225700805
