Limitations of proposed signatures of Bayesian confidence

Abstract

The Bayesian model of confidence posits that confidence is the observer's posterior probability that the decision is correct. It has been proposed that researchers can gain evidence in favor of the Bayesian model by deriving qualitative signatures of Bayesian confidence, i.e., patterns that one would expect to see if an observer was Bayesian, and looking for those signatures in human or animal data. We examine two proposed qualitative signatures, showing that their derivations contain hidden assumptions that limit their applicability, and that they are neither necessary nor sufficient conditions for Bayesian confidence. One signature is an average confidence of 0.75 for trials with neutral evidence. This signature only holds when class-conditioned stimulus distributions do not overlap and internal noise is very low. Another signature is that, as stimulus magnitude increases, confidence increases on correct trials but decreases on incorrect trials. This signature is also dependent on stimulus distribution type. There is an alternative form of this signature that has been applied in the literature; we find no indication that it is expected under Bayesian confidence, which resolves an ostensible discrepancy. We conclude that, to determine the nature of the computations underlying confidence reports, there may be no shortcut to quantitative model comparison.

Differentiating between Models of Perceptual Decision Making Using Pupil Size Inferred Confidence

“…size with signal strength for higher signal strengths in Animal B. Such a pattern is expected if decision confidence is informed not only by the strength of the sensory evidence, as described above, but also by decision time as observed in human observers (Kiani et al, 2014; see also Adler and Ma, 2017) for how increasing confidence ratings may actually be compatible with Bayesian confidence. Indeed, fits of the model by Kiani et al (2014) correlated well with the data ( p Ͻ 10 Ϫ4 and p Ͻ 0.01 for Animal A, and p Ͻ 10 Ϫ6 and p Ͻ 0.01 for Animal B in Fig.…”

Distinct temporal difference error signals in dopamine axons in three regions of the striatum in a decision-making task

“…A given model and task structure makes specific predictions on the shape of three key signatures relating stimulus discriminability, choice and confidence. The predictions can vary depending on task design (Adler and Ma, 2018;Rausch and Zehetleitner, 2019), but the structure of our task follows the original predictions (Hangya et al, 2016). Additionally, in our task, the mice combined the information about reward size with the strength of sensory evidence to select an action (confidence, or uncertainty) ( Figure 1).…”

Comparing Bayesian and non-Bayesian accounts of human confidence reports

“…Recent work has proposed possible qualitative signatures of Bayesian confidence [21]. However, the observation (or lack thereof) of these signatures provides an uncertain amount of evidence in favor of (or against) the Bayesian model, and the signatures are therefore not useful for determining which computations underlie confidence reports [22]. To objectively and quantitatively determine whether confidence ratings appear to be Bayesian, we use a formal model comparison approach.…”