Validation of the ENSEMBLES global climate ­models over southwestern Europe using probability density functions, from a downscaling perspective

Abstract

In this study we analyzed the performance of 12 state-of-the-art global climate models (GCMs) from 2 different model generations used in the ENSEMBLES project (a European Commissionfunded climate-change research project) over southwestern Europe. For this purpose, we assessed the similarity of the simulated and quasi-observed (reanalysis) probability density functions for circulation, temperature, and humidity variables at various pressure levels, which we chose from a statisticaldownscaling point of view. Our main goals were to assess which GCM variables can be reliably used as predictors for downscaling, and which GCMs perform especially well over the region under study. Results showed that specific humidity is as reliably reproduced as circulation and temperature variables, and that overall performance is best for the Hadley Centre's HADGEM2 model. Secondary goals were to estimate the skillful scale of the models, and to measure the added value of bias correction, a post-processing step commonly used in practice. We found that all models lack performance at the scale of individual grid boxes, indicating that they are not robustly skillful at their smallest scale. We also found that model performance generally improves after removing monthly bias. However, model errors at higher-order moments, which cannot be removed by simply correcting the bias, were common in some models.

On the Use of Reanalysis Data for Downscaling

“…Consequently, p values for hypothesis testing (H 0 : both the ERA-40 and NCEP-NCAR time series come from the same underlying distribution) can be directly estimated (Wilks 2006). For the PDF score, however, no theoretical distribution is available and computationally costly Monte-Carlo methods cannot be circumvented if a statistical inference is to be made (Brands et al 2011a).…”

“…In addition to these distribution-oriented scores, the correspondence of the day-to-day sequences is estimated with the Pearson correlation coefficient. Note that both types of differences are important from a downscaling point of view, since they affect the distributional and serial characteristics of the regionalized time series (Charles et al 2007;Brands et al 2011b). …”

A height dependent evaluation of wind and temperature over Europe in the CMIP5 Earth System Models

“…The score is suitable to compare temperature, which is considered to be Gaussian, and wind speeds, which are Weibully distributed. Moreover, the PDF score accounts for errors along the whole distribution (Brands et al 2011a). In this way, not only the performance of the model to reproduce the mean state but also the frequency of occurrence of rare values is evaluated.…”

“…The evaluation is based on the simple and robust score presented by Perkins et al (2007), which will hereafter be denoted as the PDF score. The score has been frequently used in recent publications on model evaluation (Maxino et al 2008, Pitman & Perkins 2009, Mao et al 2010, Brands et al 2011a, 2011b, Kjellström et al 2010, Devis et al 2013.…”

“…The domain covers flat and orographic land, open and closed sea and coastal areas within 27 to 72°N, 32°W to 35°E. The climate of the study area is mostly controlled by the unstable nature of the North-Atlantic dynamics (Collins et al 2011): westerlies carry moist air from the Atlantic, while easterlies bring cold (warm) continental air in winter (summer) (Brands et al 2011a). …”