Skill Scores Based on the Mean Square Error and Their Relationships to the Correlation Coefficient

Murphy, Allan H.

doi:10.1175/1520-0493(1988)116<2417:ssbotm>2.0.co;2

Cited by 733 publications

(610 citation statements)

References 0 publications

Supporting

Mentioning

596

Contrasting

Unclassified

Order By: Relevance

“…Decomposition of the MSESS (as detailed in Murphy 1988) suggests that calibration errors are present in most of the models, but most notably in certain ones. Revealed is a tendency toward overly confident (i.e., too-high amplitude) forecasts in predictions made prior to the northern spring predictability barrier for targets near or shortly following the barrier.…”

Section: Discussionmentioning

confidence: 99%

“…Such a linear rescaling has been applied to the real-time NMME predictions of Niño3.4 anomaly shown in NOAA/Climate Prediction Center's page: http://www.cpc.ncep.noaa.gov/ products/NMME/current/plume.html. The degree to which calibration errors reduce MSESS, resulting in lower skill than that corresponding to the correlation in their absence, can be determined through a decomposition of the MSESS as detailed in Murphy (1988). The relevant equation, showing the three components of MSESS is:…”

Section: Deterministic Verification Of the Nmme Predictionsmentioning

confidence: 99%

See 1 more Smart Citation

Deterministic skill of ENSO predictions from the North American Multimodel Ensemble

et al. 2017

View full text Add to dashboard Cite

the presence of calibration errors in some of the models. In particular, the amplitudes of some model predictions are too high when predictability is limited by the northern spring ENSO predictability barrier and/or when the interannual variability of the SST is near its seasonal minimum. The skill of the NMME system is compared to that of the MME from the IRI/CPC ENSO prediction plume, both for a comparable hindcast period and also for a set of real-time predictions spanning 2002-2011. Comparisons are made both between the MME predictions of each model group, and between the average of the skills of the respective individual models in each group. Acknowledging a hindcast versus real-time inconcsistency in the 2002-2012 skill comparison, the skill of the NMME is slightly higher than that of the prediction plume models in all cases. This result reflects well on the NMME system, with its large total ensemble size and opportunity for possible complementary contributions to skill.Abstract Hindcasts and real-time predictions of the eastcentral tropical Pacific sea surface temperature (SST) from the North American Multimodel Ensemble (NMME) system are verified for 1982-2015. Skill is examined using two deterministic verification measures: mean squared error skill score (MSESS) and anomaly correlation. Verification of eight individual models shows somewhat differing skills among them, with some models consistently producing more successful predictions than others. The skill levels of MME predictions are approximately the same as the two best performing individual models, and sometimes exceed both of them. A decomposition of the MSESS indicates This paper is a contribution to the special collection on the North American Multi-Model Ensemble (NMME) seasonal prediction experiment. The special collection focuses on documenting the use of the NMME system database for research ranging from predictability studies, to multi-model prediction evaluation and diagnostics, to emerging applications of climate predictability for subseasonal to seasonal predictions

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Deterministic Verification Of the Nmme Predictionsmentioning

confidence: 99%

Deterministic skill of ENSO predictions from the North American Multimodel Ensemble

et al. 2017

View full text Add to dashboard Cite

show abstract

“…In this section, a quantitative assessment of the model performance in terms of forecast quality of the seasonal mean ISMR is performed by using the skill score (SS) for the accuracy of the forecasts described in the study by Murphy (1988). The accuracy of the forecasts is represented by the mean square error (MSE) between forecasts and observations.…”

Section: Systematic Biases Interannual Variability and Their Relatimentioning

confidence: 99%

“…So the more negative the climatological SS is, the worse the forecast skill is. Under the condition of the equivalence between the long-term means of reference forecasts and observations (see Section 3 in the work of Murphy (1988)), the SS is decomposed by adding and subtracting both the long-term means of forecasts and observations in the calculation of the MSE in Equation (1):…”

Section: Systematic Biases Interannual Variability and Their Relatimentioning

confidence: 99%

Evaluation of the DEMETER performance for seasonal hindcasts of the Indian summer monsoon rainfall

Rodó

Doblas‐Reyes

2011

Intl Journal of Climatology

View full text Add to dashboard Cite

On the basis of the observed all-India rainfall (AIR), Climate Prediction Center (CPC) merged analysis of precipitation (CMAP), and the multimodel rainfall simulated in the Development of a European multimodel ensemble system for seasonal to interannual prediction (DEMETER) project, the model performance in terms of forecast quality of the seasonal mean Indian summer monsoon rainfall (ISMR) was evaluated. The approach used included the individual model (single-model ensemble) comparison as well as its comparison with the multimodel ensemble (MME). The results obtained reveal that systematic biases are the main source of the low skill in predicting the ISMR, while improved reproduction of interannual variability may increase the overall forecast quality for the ISMR. The MME shows superior performance in reproducing AIR features than the single-model ensembles. This improved forecast quality achieved by the MME with respect to the ISMR variations primarily depends on model performance and secondly on ensemble size. The MME has improved capability of simulating the ISMR variations during 1972-1990 decades, whereas it exhibits a lower performance in the 1990s. The reason of this decadal varying forecast quality clearly deserves more investigation. In summary, these results suggest that the MME's increase in skill absolutely depends on enhancing the individual model quality.

show abstract

“…Model skill, on the other hand, is defined as the model accuracy relative to the accuracy of hindcasts produced by some reference procedure such as climatology or persistence. To measure the model skill, we may compute the reduction of mse over the climatological hindcasts (Murphy 1988),…”

Section: Model Accuracy and Skillmentioning

confidence: 99%

Evaluation of the Princeton Ocean Model Using South China Sea Monsoon Experiment (SCSMEX) Data

Chu

Lü²,

Chen³

2001

J. Atmos. Oceanic Technol.

View full text Add to dashboard Cite

The Princeton Ocean Model (POM) has been implemented in the South China Sea for hindcast of circulation and thermohaline structure. A two-step technique is used to initialize POM with temperature, salinity, and velocity for 1 April 1998 and integrate it from 1 April 1998 with synoptic surface forcing for 3 months with and without data assimilation. Hydrographic and current data acquired from the South China Sea Monsoon Experiment (SCSMEX) from April through June 1998 are used to verify, and to assimilate into, POM. The mean SCSMEX data (Apr-Jun 1998) are about 0.5ЊC warmer than the mean climatological data above the 50-m depth, and slightly cooler than the mean climatological data below the 50-m depth, and are fresher than the climatological data at all depths and with the maximum bias (0.2-0.25 ppt) at 75-m depth.POM without data assimilation has the capability to predict the circulation pattern and the temperature field reasonably well, but has no capability to predict the salinity field. The model errors have Gaussian-type distribution for temperature hindcast, and non-Gaussian distribution for salinity hindcast with six to eight times more frequencies of occurrence on the negative side than on the positive side. Data assimilation enhances the model capability for ocean hindcast, if even only conductivity-temperature-depth (CTD) data are assimilated. When the model is reinitialized using the assimilated data at the end of a month (30 Apr; 31 May 1998) and the model is run for a month without data assimilation (hindcast capability test), the model errors for both temperature and salinity hindcast are greatly reduced, and they have Gaussian-type distributions for both temperature and salinity hindcast. Hence, POM gains capability in salinity hindcast when CTD data are assimilated.

show abstract

Skill Scores Based on the Mean Square Error and Their Relationships to the Correlation Coefficient

Cited by 733 publications

References 0 publications

Deterministic skill of ENSO predictions from the North American Multimodel Ensemble

Deterministic skill of ENSO predictions from the North American Multimodel Ensemble

Evaluation of the DEMETER performance for seasonal hindcasts of the Indian summer monsoon rainfall

Evaluation of the Princeton Ocean Model Using South China Sea Monsoon Experiment (SCSMEX) Data

Contact Info

Product

Resources

About