Contrasting probabilistic scoring rules

Machete, Reason L.

doi:10.1016/j.jspi.2013.05.012

Cited by 23 publications

(14 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Equivalently, the associated divergence or discrepancy function (Dawid, 1998), given by D.P; Q/ WD S.P; Q/ S.P; P /, is always non-negative. There is a very wide variety of proper scoring rules: for general characterizations see, among others, McCarthy (1956) and Savage (1971), and for various special cases, see Dawid (1998Dawid ( , 2007, Dawid & Musio (2014), Gneiting & Raftery (2007) and Machete (2013). We now consider some of these in more detail.…”

Section: Proper Scoring Rulesmentioning

confidence: 99%

Minimum Scoring Rule Inference

Dawid

Monachini

Ventura

2015

Scandinavian J Statistics

View full text Add to dashboard Cite

Proper scoring rules are methods for encouraging honest assessment of probability distributions. Just like likelihood, a proper scoring rule can be applied to supply an unbiased estimating equation for any statistical model, and the theory of such equations can be applied to understand the properties of the associated estimator. In this paper we develop some basic scoring rule estimation theory, and explore robustness and interval estimation preoperties by means of theory and simulations.

show abstract

Section: Proper Scoring Rulesmentioning

confidence: 99%

Minimum Scoring Rule Inference

Dawid

Monachini

Ventura

2015

Scandinavian J Statistics

View full text Add to dashboard Cite

show abstract

“…Quoting the authors: "it is insufficient to use a scoring rule simply because it is strictly proper; instead, it is beneficial to consider the specific way in which the scoring rule rewards and penalizes forecasts" (Merkle and Steyvers 2013, p. 302; emphasis added). Machete (2013) also suggested that the proper scoring rule one chooses should depend on the application at hand, and an issue to consider may be future decisions associated with high impact, low probability events. In particular, given two forecasts whose errors from the ideal distribution differ only by the sign, Machete (2013) found that the logarithmic scoring rule scores higher the forecast with the highest entropy, the spherical scoring rule scores higher the forecast with the lowest entropy, and the quadratic scoring rule does not distinguish between the two forecasts.…”

Section: Choosing a Proper Scoring Rulementioning

confidence: 99%

“…Machete (2013) also suggested that the proper scoring rule one chooses should depend on the application at hand, and an issue to consider may be future decisions associated with high impact, low probability events. In particular, given two forecasts whose errors from the ideal distribution differ only by the sign, Machete (2013) found that the logarithmic scoring rule scores higher the forecast with the highest entropy, the spherical scoring rule scores higher the forecast with the lowest entropy, and the quadratic scoring rule does not distinguish between the two forecasts. In other words, the logarithmic (respectively, spherical) scoring rule should be used when more (respectively, less) uncertainty is desirable.…”

Section: Choosing a Proper Scoring Rulementioning

confidence: 99%

An Overview of Applications of Proper Scoring Rules

Carvalho

2016

Decision Analysis

View full text Add to dashboard Cite

W e present a study on the evolution of publications about applications of proper scoring rules. Specifically, we consider articles reporting the use of proper scoring rules when either measuring the accuracy of forecasts or for inducing honest reporting of private information within a certain context. Our analysis of a data set containing 201 articles published between 1950 and 2015 suggests that there has been a tremendous increase in the number of published articles about proper scoring rules over the years. Moreover, the weather/climate, prediction markets, psychology, and energy domains are the four most popular application areas. After providing some insights on how proper scoring rules are applied in different domains, we analyze the publication outlets where the articles in our data set were published. In this regard, we find that an increasing number of articles are now being published in conference proceedings related to artificial intelligence, as opposed to traditional academic journals. We conclude this review by suggesting that the wisdom-of-crowds phenomenon might be a driving force behind the recent popularity of proper scoring rules.

show abstract

“…To deal with complex models or model misspecifications, useful surrogate likelihoods can be obtained trough proper scoring rules. A scoring rule (see, for instance, the recent overviews by Machete, 2013, andMusio, 2014, and references therein) is a special kind of loss function designed to measure the quality of a probability distribution for a random variable, given its observed value. It is proper if it encourages honesty in the probability evaluation.…”

Section: Introductionmentioning

confidence: 99%

Objective Bayesian inference with proper scoring rules

Giummole

Mameli

Ruli

et al. 2018

TEST

View full text Add to dashboard Cite

Standard Bayesian analyses can be difficult to perform when the full likelihood, and consequently the full posterior distribution, is too complex or even impossible to specify or if robustness with respect to data or to model misspecifications is required. In these situations, we suggest to resort to a posterior distribution for the parameter of interest based on proper scoring rules. Scoring rules are loss functions designed to measure the quality of a probability distribution for a random variable, given its observed value. Important examples are the Tsallis score and the Hyvärinen score, which allow us to deal with model misspecifications or with complex models. Also the full and the composite likelihoods are both special instances of scoring rules.The aim of this paper is twofold. Firstly, we discuss the use of scoring rules in the Bayes formula in order to compute a posterior distribution, named SR-posterior distribution, and we derive its asymptotic normality. Secondly, we propose a procedure for building default priors for the unknown parameter of interest, that can be used to update the information provided by the scoring rule in the SR-posterior distribution. In particular, a reference prior is obtained by maximizing the average α−divergence from the SR-posterior distribution. For 0 ≤ |α| < 1, the result is a Jeffreys-type prior that is proportional to the square root of the determinant of the Godambe information matrix associated to the scoring rule. Some examples are discussed.

show abstract

Contrasting probabilistic scoring rules

Cited by 23 publications

References 23 publications

Minimum Scoring Rule Inference

Minimum Scoring Rule Inference

An Overview of Applications of Proper Scoring Rules

Objective Bayesian inference with proper scoring rules

Contact Info

Product

Resources

About