Which Climate Model Evaluation Methods Can Consistently Select Skillful Models from the CMIP6 Ensemble?
Year: 2025
Type: article
Abstract: When considering the possible use of climate model data, it is necessary to choose which model is most appropriate to use. There are many methods for evaluating and selecting climate models in the literature, but there is no established consensus on which method is the most robust for determining model skill. In this article, we tested seven widely used methods for evaluating climate models in the Arctic using CMIP6 surface air temperature data: a single statistical metric method (root mean square error, spatial trends), a single skill score method (Taylor skill score, probability density function), a combination of several statistical metric methods (Taylor diagram, interannual variability skill score, comprehensive rating metric, etc.), and a multiple statistical criteria method (percentile-based approach). To evaluate their consistency, each method was applied to two periods: 1951-1980 and 1981-2010. For each method, the models were ranked and classified into three quality groups (very good, satisfactory, unsatisfactory). The comparison of methods was performed by comparing the differences in the average values of the normalized statistical measures, the differences in the model ranks, and the definition of the model quality groups. For each method, an optimal set of models corresponding to the top 25% was selected. One of the main objectives of the study was to compare the ability of the methods to identify the best model for the selected ensemble, regardless of the time period (i.e., without sensitivity to natural variability). The results suggest a preference for methods using root mean square error and a percentile-based approach.
Language: English
Subfield: Global and Planetary Change
Field: Environmental Science
Domain: Physical Sciences
SDG: Climate action
Open Access status: diamond
OpenAlex ID: W4412056971