Earth System Model Evaluation Tool (ESMValTool) v2.0 –
diagnostics for extreme events, regional and impact evaluation and
analysis of Earth system models in CMIP

Weigel, Katja; Gier, Bettina K.; Lauer, Axel; Righi, Mattia; Schlund, Manuel; Adeniyi, Kemisola; Andela, Bouwe; Arnone, Enrico; Berg, Peter; Caron, Louis-Philippe; Cionni, Irene; Corti, Susanna; Drost, Niels; Hunter, Alasdair; Lledó, Llorenç; Mohr, Christian Wilhelm; Paçal, Aytaç; Pérez-Zañón, Núria; Predoi, V.; Sandstad, Marit; Sillmann, Jana; Sterl, Andreas; Vegas-Regidor, Javier; Hardenberg, Jost von; Eyring, Veronika

doi:10.5194/gmd-2020-244

Cited by 12 publications

(9 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Journal of Geophysical Research: Atmospheres et al, 2020; Weigel et al, 2020). The ESMValTool is fully integrated into the ESGF infrastructure at the Deutsches Klimarechenzentrum (DKRZ) where all the model output and the observations are stored in a local replica and the tool is run.…”

Section: 1029/2019jd032321mentioning

confidence: 99%

“…For more information on contributing to the ESMValTool, general guidelines, code style, etc., we refer to the ESMValTool user's guide available at https://docs.esmvaltool.org website. A general overview on the ESMValTool is given by Eyring, Righi, et al (2016), technical details of the latest version (v2.0) can be found in Righi et al (2020), diagnostics and metrics newly added to v2.0 are described in three companion papers (Eyring et al, 2020; Lauer et al, 2020; Weigel et al, 2020). The ESMValTool is fully integrated into the ESGF infrastructure at the Deutsches Klimarechenzentrum (DKRZ) where all the model output and the observations are stored in a local replica and the tool is run.…”

Section: Models and Observationsmentioning

confidence: 99%

See 1 more Smart Citation

Quantifying Progress Across Different CMIP Phases With the ESMValTool

et al. 2020

Self Cite

View full text Add to dashboard Cite

More than 40 model groups worldwide are participating in the Coupled Model Intercomparison Project Phase 6 (CMIP6), providing a new and rich source of information to better understand past, present, and future climate change. Here, we use the Earth System Model Evaluation Tool (ESMValTool) to assess the performance of the CMIP6 ensemble compared to the previous generations CMIP3 and CMIP5. While CMIP5 models did not capture the observed pause in the increase in global mean surface temperature between 1998 and 2013, the historical CMIP6 simulations agree well with the observed recent temperature increase, but some models have difficulties in reproducing the observed global mean surface temperature record of the second half of the twentieth century. While systematic biases in annual mean surface temperature and precipitation remain in the CMIP6 multimodel mean, individual models and high-resolution versions of the models show significant reductions in many long-standing biases. Some improvements are also found in the vertical temperature, water vapor, and zonal wind speed distributions, and root-mean-square errors for selected fields are generally smaller with reduced intermodel spread and higher average skill in the correlation patterns relative to observations. An emerging property of the CMIP6 ensemble is a higher effective climate sensitivity with an increased range between 2.3 and 5.6 K. A possible reason for this increase in some models is improvements in cloud representation resulting in stronger shortwave cloud feedbacks than in their predecessor versions.

show abstract

Section: 1029/2019jd032321mentioning

confidence: 99%

Section: Models and Observationsmentioning

confidence: 99%

Quantifying Progress Across Different CMIP Phases With the ESMValTool

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, the Earth System Model Evaluation Tool (ESMValTool; Weigel et al, 2020) is a systematic and efficient tool for model evaluation, which has been widely used in related studies (e.g., Valdes et al, 2017;Righi et al, 2020;Waliser et al, 2020). It has many distinct advantages, such as providing the well-documented analysis and no need for preprocessing of evaluated datasets, compared with our tool.…”

Section: Discussionmentioning

confidence: 99%

An improved multivariable integrated evaluation method and tool (MVIETool) v1.0 for multimodel intercomparison

Zhang

Han

et al. 2021

Geosci. Model Dev.

View full text Add to dashboard Cite

Abstract. An evaluation of a model's overall performance in simulating multiple fields is fundamental to model intercomparison and development. A multivariable integrated evaluation (MVIE) method was proposed previously based on a vector field evaluation (VFE) diagram, which can provide quantitative and comprehensive evaluation on multiple fields. In this study, we make further improvements to this method from the following aspects. (1) We take area weighting into account in the definition of statistics in the VFE diagram and MVIE method, which is particularly important for a global evaluation. (2) We consider the combination of multiple scalar fields and vector fields against multiple scalar fields alone in the previous MVIE method. (3) A multivariable integrated skill score (MISS) is proposed as a flexible index to measure a model's ability to simulate multiple fields. Compared with the multivariable integrated evaluation index (MIEI) proposed in the previous study, MISS is a normalized index that can adjust the relative importance of different aspects of model performance. (4) A simple-to-use and straightforward tool, the Multivariable Integrated Evaluation Tool (MVIETool version 1.0), is developed to facilitate an intercomparison of the performance of various models. Users can use the tool coded either with the open-source NCAR Command Language (NCL) or Python3 to calculate the MVIE statistics and plotting. With the support of this tool, one can easily evaluate model performance in terms of each individual variable and/or multiple variables.

show abstract

“…A wide range of methods and tools to identify biases in the simulation of extreme events exist (Eyring et al, 2019) that can be applied as tests for UNSEEN applications. For example, the ‘ESMValTool’ has been developed for climate model evaluation (Eyring et al, 2016) including extreme events (Weigel et al, 2021). Furthermore, metrics common to the evaluation of numerical weather prediction systems, such as the forecast reliability and rank histograms, can be used for prediction systems across timescales (Bellprat et al, 2019; Palmer & Weisheimer, 2018; Suarez‐Gutierrez et al, 2021; Weisheimer & Palmer, 2014).…”

Section: The Protocolmentioning

confidence: 99%

An open workflow to gain insights about low‐likelihood high‐impact weather events from initialized predictions

Kelder

Marjoribanks²,

Slater

et al. 2022

Meteorological Applications

View full text Add to dashboard Cite

Low-likelihood weather events can cause dramatic impacts, especially when they are unprecedented. In 2020, amongst other high-impact weather events, UK floods caused more than £300 million damage, prolonged heat over Siberia led to infrastructure failure and permafrost thawing, while wildfires ravaged California. Such rare phenomena cannot be studied well from historical records or reanalysis data. One way to improve our awareness is to exploit ensemble prediction systems, which represent large samples of simulated weather events. This 'UNSEEN' method has been successfully applied in several scientific studies, but uptake is hindered by large data and processing requirements, and by uncertainty regarding the credibility of the simulations.Here, we provide a protocol to apply and ensure credibility of UNSEEN for studying low-likelihood high-impact weather events globally, including an open workflow based on Copernicus Climate Change Services (C3S) seasonal predictions. Demonstrating the workflow using European Centre for Medium-Range Weather Forecasts (ECMWF) SEAS5, we find that the 2020 March-May Siberian heatwave was predicted by one of the ensemble members; and that the record-shattering August 2020 California-Mexico temperatures were part of a strong increasing trend. However, each of the case studies exposes challenges with respect to the credibility of UNSEEN and the sensitivity of the outcomes to user decisions. We conclude that UNSEEN can provide new insights about low-likelihood weather events when the decisions are transparent, and the challenges and sensitivities are acknowledged. Anticipating plausible lowlikelihood extreme events and uncovering unforeseen hazards under a changing climate warrants further research at the science-policy interface to manage high impacts.

show abstract

Earth System Model Evaluation Tool (ESMValTool) v2.0 – diagnostics for extreme events, regional and impact evaluation and analysis of Earth system models in CMIP

Cited by 12 publications

References 53 publications

Quantifying Progress Across Different CMIP Phases With the ESMValTool

Quantifying Progress Across Different CMIP Phases With the ESMValTool

An improved multivariable integrated evaluation method and tool (MVIETool) v1.0 for multimodel intercomparison

An open workflow to gain insights about low‐likelihood high‐impact weather events from initialized predictions

Contact Info

Product

Resources

About