No abstract

The relative abilities of 2, dimensioned statistics -the root-mean-square error (RMSE) and the mean absolute error (MAE) -to describe average model-performance error are examined. The RMSE is of special interest because it is widely reported in the climatic and environmental literature; nevertheless, it is an inappropriate and misinterpreted measure of average error. RMSE is inappropriate because it is a function of 3 characteristics of a set of errors, rather than of one (the average error). RMSE varies with the variability within the distribution of error magnitudes and with the square root of the number of errors (n 1/2 ), as well as with the average-error magnitude (MAE). Our findings indicate that MAE is a more natural measure of average error, and (unlike RMSE) is unambiguous. Dimensioned evaluations and inter-comparisons of average model-performance error, therefore, should be based on MAE.KEY WORDS: Model-performance measures · Root-mean-square error · Mean absolute errorResale or republication not permitted without written consent of the publisher

Quantitative approaches to the evaluation of model performance were recently examined by Fox (1981). His recommendations are briefly reviewed and a revised set of performance statistics is proposed. It is suggested that the correlation between model-predicted and observed data, commonly described by Pearson's product-moment correlation coefficient, is an insufficient and often misleading measure of accuracy. A complement of difference and summary univariate indices is presented as the nucleus of a more informative, albeit fundamentally descriptive, approach to model evaluation. Two models that estimate monthly evapotranspiration are comparatively evaluated in order to illustrate how the recommended method(s) can be applied.cussion focuses on the general utility and information content of the correlation and "difference" measures, although other topics, such as hypothesis testing and graphics, are briefly examined. Since there are far too many types of models within the atmospheric sciences to adequately discuss their "scientific" evaluation within a single paper, this discussion emphasizes "operational" evaluation, even though the scientific merit of any model-particularly one constructed for "explanatory" purposes (Mather et al, 1980)can be extremely important.

Procedures that may be used to evaluate the operational performance of a wide spectrum of geophysical models are introduced. Primarily using a complementary set of difference measures, both model accuracy and precision can be meaningfully estimated, regardless of whether the model predictions are manifested as scalars, directions, or vectors. It is additionally suggested that the reliability of the accuracy and precision measures can be determined from bootstrap estimates of confidence and significance. Recommended procedures are illustrated with a comparative evaluation of two models that estimate wind velocity over the South Atlantic Bight.

Using traditional land-based gauge measurements and shipboard estimates, a global climatology of mean monthly precipitation has been developed. Data were obtained from ten existing sources, screened for coding errors, and redundant station records were removed. The edited data base contains 24,635 spatially independent terrestrial station records and 2223 oceanic grid-point records. A procedure for correcting gauge-induced biases is presented and used to remove systematic errors caused by wind, wetting on the interior walls of the gauge, and evaporation from the gauge. These 'corrected' monthly precipitation observations were then interpolated to a 0.5" of latitude by 0.5" of longitude grid using a spherically based interpolation procedure. Bias-corrected spatial distributions of the annual mean and intraannual variance are presented along with a harmonic decomposition of the intra-annual variance.