More than ten state-of-the-art regional air quality models have been applied as part of the Air Quality Model Evaluation International Initiative (AQMEII). These models were run by twenty independent groups in Europe and North America. Standardised modelling outputs over a full year (2006) from each group have been shared on the web-distributed ENSEMBLE system, which allows for statistical and ensemble analyses to be performed by each group. The estimated ground-level ozone mixing ratios from the models are collectively examined in an ensemble fashion and evaluated against a large set of observations from both continents. The scale of the exercise is unprecedented and offers a unique opportunity to investigate methodologies for generating skilful ensembles of regional air quality models outputs. Despite the remarkable progress of ensemble air quality modelling over the past decade, there are still outstanding questions regarding this technique. Among them, what is the best and most beneficial way to build an ensemble of members? And how should the optimum size of the ensemble be determined in order to capture data variability as well as keeping the error low? These questions are addressed here by looking at optimal ensemble size and quality of the members. The analysis carried out is based on systematic minimization of the model error and is important for performing diagnostic/probabilistic model evaluation. It is shown that the most commonly used multi-model approach, namely the average over all available members, can be outperformed by subsets of members optimally selected in terms of bias, error, and correlation. More importantly, this result does not strictly depend on the skill of the individual members, but may require the inclusion of low-ranking skill-score members. A clustering methodology is applied to discern among members and to build a skilful ensemble based on model association and data clustering, which makes no use of priori knowledge of model skill. Results show that, while the methodology needs further refinement, by optimally selecting the cluster distance and association criteria, this approach can be useful for model applications beyond those strictly related to model evaluation, such as air quality forecasting

Ten state-of-the-science regional air quality (AQ) modeling systems have been applied to continental-scale domains in North America and Europe for full-year simulations of 2006 in the context of Air Quality Model Evaluation International Initiative (AQMEII), whose main goals are model intercomparison and evaluation. Standardised modeling outputs from each group have been shared on the web-distributed ENSEMBLE system, which allows statistical and ensemble analyses to be performed. In this study, the one-year model simulations are inter-compared and evaluated with a large set of observations for ground-level particulate matter (PK10 and PM2.5) and its chemical components. Modeled concentrations of gaseous PM precursors, SO2 and NO2, have also been evaluated against observational data for both continents. Furthermore, modeled deposition (dry and wet) and emissions of several species relevant to PM are also inter-compared. The unprecedented scale of the exercise (two continents, one full year, fifteen modeling groups) allows for a detailed description of AQ model skill and uncertainty with respect to PM. Analyses of PM10 yearly time series and mean diurnal cycle show a large underestimation throughout the year for the AQ models included in AQMEII. The possible causes of PM bias, including errors in the emissions and meteorological inputs (e.g., wind speed and precipitation), and the calculated deposition are investigated. Further analysis of the coarse PM components, PM2.5 and its major components (SO4, NH4, NO3, elemental carbon), have also been performed, and the model performance for each component evaluated against measurements. Finally, the ability of the models to capture high PM concentrations has been evaluated by examining two separate PM2.5 episodes in Europe and North America. A large variability among models in predicting emissions, deposition, and concentration of PM and its precursors during the episodes has been found. Major challenges still remain with regards to identifying and eliminating the sources of PM bias in the models. Although PM2.5 was found to be much better estimated by the models than PM10, no model was found to consistently match the observations for all locations throughout the entire yea

Copyright 2011 Elsevier B.V., All rights reserved.The CMAQ modeling system has been used to simulate the air quality for North America and Europe for the entire year of 2006 as part of the Air Quality Model Evaluation International Initiative (AQMEII). The operational model performance of tropospheric ozone (O), fine particulate matter (PM) and total particulate matter (PM) for the two continents has been assessed. The model underestimates daytime (8am-8pm LST) O mixing ratios by 13% in the winter for North America, primarily due to an underestimation of daytime O mixing ratios in the middle and lower troposphere from the lateral boundary conditions. The model overestimates winter daytime O mixing ratios in Europe by an average of 8.4%. The model underestimates daytime O by 4-5% in the spring for both continents, while in the summer daytime O is overestimated by 9.8% for North America and slightly underestimated by 1.6% for Europe. The model overestimates daytime O in the fall for both continents, grossly overestimating daytime O by over 30% for Europe. The performance for PM varies both seasonally and geographically for the two continents. For North American, PM is overestimated in the winter and fall, with an average Normalized Mean Bias (NMB) greater than -30%, while performance in the summer is relatively good, with an average NMB of -4.6%. For Europe, PM is underestimated throughout the entire year, with the NMB ranging from -24% in the fall to -55% in the winter. PM is underestimated throughout the year for both North America and Europe, with remarkably similar performance for both continents. The domain average NMB for PM ranges between -45% and -65% for the two continents, with the largest underestimation occurring in the summer for North American and the winter for Europe

Through the comparison of several regional-scale chemistry transport modeling systems that simulate meteorology and air quality over the European and North American continents, this study aims at (i) apportioning error to the responsible processes using timescale analysis, (ii) helping to detect causes of model error, and (iii) identifying the processes and temporal scales most urgently requiring dedicated investigations. The analysis is conducted within the framework of the third phase of the Air Quality Model Evaluation International Initiative (AQMEII) and tackles model performance gauging through measurement-to-model comparison, error decomposition, and time series analysis of the models biases for several fields (ozone, CO, SO, NO, NO, PM, PM, wind speed, and temperature). The operational metrics (magnitude of the error, sign of the bias, associativity) provide an overallsense of model strengths and deficiencies, while apportioning the error to its constituent parts (bias, variance, and covariance) can help assess the nature and quality of the error. Each of the error components is analyzed independently and apportioned to specific processes based on the corresponding timescale (long scale, synoptic, diurnal, and intraday) using the error apportionment technique devised in the former phases of AQMEII. The application of the error apportionment method to the AQMEII Phase 3 simulations provides several key insights. In addition to reaffirming the strong impact of model inputs (emission and boundary conditions) and poor representation of the stable boundary layer on model bias, results also highlighted the high interdependencies among meteorological and chemical variables, as well as among their errors. This indicates that the evaluation of air quality model performance for individual pollutants needs to be supported by complementary analysis of meteorological fields and chemical precursors to provide results that are more insightful from a model development perspective. This will require evaluaion methods that are able to frame the impact on error of processes, conditions, and fluxes at the surface. For example, error due to emission and boundary conditions is dominant for primary species (CO, particulate matter (PM)), while errors due to meteorology and chemistry are most relevant to secondary species, such as ozone. Some further aspects emerged whose interpretation requires additional consideration, such as the uniformity of the synoptic error being region- and model-independent, observed for several pollutants; the source of unexplained variance for the diurnal component; and the type of error caused by deposition and at which scale.

Abstract. This study is conducted in the framework of the Air Quality Modelling Evaluation International Initiative (AQMEII) and aims at the operational evaluation of an ensemble of 12 regional-scale chemical transport models used to predict air quality over the North American (NA) and European (EU) continents for 2006. The modelled concentrations of ozone and CO, along with the meteorological fields of wind speed (WS) and direction (WD), temperature (T ), and relative humidity (RH), are compared against highquality in-flight measurements collected by instrumented commercial aircraft as part of the Measurements of OZone, water vapour, carbon monoxide and nitrogen oxides by Airbus In-service airCraft (MOZAIC) programme. The evaluation is carried out for five model domains positioned around four major airports in NA (Portland, Philadelphia, Atlanta, and Dallas) and one in Europe (Frankfurt), from the surface to 8.5 km. We compare mean vertical profiles of modelled and measured variables for all airports to compute error and variability statistics, perform analysis of altitudinal error correlation, and examine the seasonal error distribution for ozone, including an estimation of the bias introduced by the lateral boundary conditions (BCs). The results indicate that model performance is highly dependent on the variable, location, season, and height (e.g. surface, planetary boundary layer (PBL) or free troposphere) being analysed. While model performance for T is satisfactory at all sites Published by Copernicus Publications on behalf of the European Geosciences Union. E. Solazzo et al.: Evaluating regional air quality models in the vertical(correlation coefficient in excess of 0.90 and fractional bias ≤ 0.01 K), WS is not replicated as well within the PBL (exhibiting a positive bias in the first 100 m and also underestimating observed variability), while above 1000 m, the model performance improves (correlation coefficient often above 0.9). The WD at NA airports is found to be biased in the PBL, primarily due to an overestimation of westerly winds. RH is modelled well within the PBL, but in the free troposphere large discrepancies among models are observed, especially in EU. CO mixing ratios show the largest range of modelled-to-observed standard deviations of all the examined species at all heights and for all airports. Correlation coefficients for CO are typically below 0.6 for all sites and heights, and large errors are present at all heights, particularly in the first 250 m. Model performance for ozone in the PBL is generally good, with both bias and error within 20 %. Profiles of ozone mixing ratios depend strongly on surface processes, revealed by the sharp gradient in the first 2 km (10 to 20 ppb km −1 ). Modelled ozone in winter is biased low at all locations in the NA, primarily due to an underestimation of ozone from the BCs. Most of the model error in the PBL is due to surface processes (emissions, transport, photochemistry), while errors originating aloft appear to have relatively limited impact on model...