Comonotonic proper scoring rules to measure ambiguity and subjective beliefs

Abstract

Proper scoring rules serve to measure subjective degrees of belief. Traditional proper scoring rules are based on the assumption of expected value maximization. There are, however, many deviations from expected value, primarily due to risk aversion. Correcting techniques have been proposed in the literature for deviations due to nonlinear utility. These techniques still assumed expected utility maximization. More recently, corrections for deviations from expected utility have been proposed. The latter concerned, however, only the quadratic scoring rule, and could handle only half of the domain of subjective beliefs. Further, beliefs close to 0.5 could not be discriminated. This paper generalizes the correcting techniques to all bounded binary proper scoring rules, covers the whole domain of beliefs and, in particular, can discriminate between all degrees of belief. Thus, we fully extend the properness requirement (in the sense of identifying all degrees of subjective beliefs) to virtually all models that deviate from expected value.

How do risk attitudes affect measured confidence?

“…Nor did we find any relationship between ambiguity attitudes and self-reported confidence. This is, of course, far from conclusive evidence that there is no relationship to discover, and there is certainly scope for further research into this issue and the broader question-previously highlighted by Hoelzl and Rustichini (2005); Offerman et al (2009) and Kothiyal et al (2011)-of how to assess and control the potential impact of ambiguity attitudes in the context of incentivised belief elicitation.…”

A Penny for Your Thoughts: A Survey of Methods for Eliciting Beliefs

“…In an experiment, they find that this scoring rule leads subjects to correctly reproduce induced objective probabilities, obviating the need to elicit a correction function. Kothiyal et al (2011) also investigate the over-reporting of 0.5, and consider the performance of a more general set of scoring rules under probabilistic sophistication allowing for non-additive beliefs. The authors explain the bunching as the result of the reversal of payoff ranks at a belief of 0.5 and propose a comonotonic scoring rule that preserves the rank order of payoffs.…”

Lossed in translation: an off-the-shelf method to recover probabilistic beliefs from loss-averse agents

“…Kothiyal et al. (2011) extend this method to overcome the problem of discriminating between moderate beliefs in a range around the baseline probability of 1/2, for which agents give the same optimal report. By adding a fixed constant to one of the QSR payoffs, they both eliminate the excess of uninformative baseline reports and yield an invertible response mapping that makes possible the recovery of true beliefs, while maintaining the properness of the original scoring rule.…”