Confidence reflects a noisy decision reliability estimate

Abstract

Decisions vary in difficulty. Humans know this and typically report more confidence in easy than in difficult decisions. However, confidence reports do not perfectly track decision accuracy, but also reflect response biases and difficulty misjudgments. To isolate the quality of confidence reports, we developed a model of the decision-making process underlying choice-confidence data. In this model, confidence reflects a subject's estimate of the reliability of their decision. The quality of this estimate is limited by the subject's uncertainty about the uncertainty of the variable that informs their decision ("meta-uncertainty"). This model provides an accurate account of choice-confidence data across a broad range of perceptual and cognitive tasks, revealing that meta-uncertainty varies across subjects, is stable over time, generalizes across some domains, and can be manipulated experimentally. The model offers a parsimonious explanation for the computational processes that underlie and constrain the sense of confidence.Humans are aware of the fallibility of perception and cognition. When we experience a high degree of confidence in a perceptual or cognitive decision, that decision is more likely to be correct than when we feel less confident 1 . This "metacognitive" ability helps us to learn from mistakes 2 , to plan future actions 3 , and to optimize group decision-making 4 . There is a long-standing interest in the mental operations underlying our sense of confidence [5][6][7] , and the rapidly expanding field of metacognition seeks to understand how metacognitive ability varies across domains 8 , individuals 9 , clinical states 10 , and development 11 .Quantifying a subject's ability to introspect about the correctness of a decision is a challenging problem [12][13][14] . There exists no generally agreed-upon method 15 . Even in the simplest decision-making tasks, several distinct factors influence a subject's confidence reports. Consider a subject jointly reporting a binary decision about a sensory stimulus (belongs to "Category A"

How do humans give confidence? A comprehensive comparison of process models of perceptual metacognition.

“…Generation of the primary choice in the CASANDRE model follows standard SDT assumptions. For confidence, the model assumes an additional stage of processing based on the observer’s estimate of the reliability of their choices (Boundy-Singer et al, 2023). Therefore, in CASANDRE, the confidence variable represents choice reliability, r conf , as rconf=|rsens−c0|normalσfalse^normalsens.…”

“…A very popular model—the Bayesian confidence hypothesis (BCH)—proposes that confidence is computed as the posterior probability of a correct choice (BCH model; Sanders et al, 2016). Finally, a recent model proposes that confidence represents an observer’s estimate of uncertainty in the decision variable but that the metacognitive system only has access to a noisy estimate of this uncertainty (confidence as a noisy decision reliability estimate [CASANDRE] model; Boundy-Singer et al, 2023).…”

The future of metacognition research: balancing construct breadth with measurement rigor

“…Of particular interest here is whether confidence reflects a heuristic such as distance to a decision criterion or bound (Kepecs et al, 2008;Vickers, 1979), or whether it is Bayesian or quasi-Bayesian in being sensitive to uncertainty Aitchison & Lengyel, 2017;Denison et al, 2018;Li & Ma, 2020). It is beyond the scope of the current paper to review this literature, but we note one promising way forward here is to consider metacognitive capacity (and summary statistics such as meta-d') as resulting from the fidelity of a number of different processing stages, including sensitivity to perceptual or evidential uncertainty (Boundy-Singer et al, 2022;Geurts et al, 2022), frame-of-reference shifts needed to monitor one's own response (Bang & Fleming, 2018;Desender et al, 2021;Fleming & Daw, 2017), and finally the requirement to explicitly represent or use a metacognitive estimate in communication and behavioural control (Bang et al, 2020;Donoso et al, 2014;Shekhar & Rahnev, 2018). Unpacking these processing stages, and providing a more detailed computational account of metacognition, remains a major goal for the field (Rahnev et al, 2022).…”

“…Another issue is that the meta-d' framework is not a process model of how confidence ratings are generated (Shekhar & Rahnev, 2021a), and thus cannot identify distinct sources of metacognitive inefficiency (Shekhar & Rahnev, 2021b). Thus, just as vision scientists may investigate the different component processes that lead to a particular d', metacognition researchers are increasingly turning to richer computational models to decompose the different stages involved in confidence formation (Bang & Fleming, 2018;Boundy-Singer et al, 2022;Guggenmos, 2022;Shekhar & Rahnev, 2018). Of particular interest here is whether confidence reflects a heuristic such as distance to a decision criterion or bound (Kepecs et al, 2008;Vickers, 1979), or whether it is Bayesian or quasi-Bayesian in being sensitive to uncertainty Aitchison & Lengyel, 2017;Denison et al, 2018;Li & Ma, 2020).…”

Abstract deliberation by visuomotor neurons in prefrontal cortex

“…4a, right). For this reason, evidence estimates are thought to not only inform decision outcome, but also determine a subject’s commitment to an evolving decision [8, 9] and factor into their confidence in a decision [21, 22]. If the neural populations we recorded from are involved in the deliberation process, their activity should thus reflect a graded representation of evidence.…”