Common computations in automatic cue combination and metacognitive confidence reports

Abstract

Appropriate perceptual decision making necessitates the accurate estimation and use of sensory uncertainty. Such estimation has been studied in the context of both low-level multisensory cue combination and metacognitive estimation of confidence, but it remains unclear whether the same computations underlie both sets of uncertainty estimation. We created visual stimuli with low vs. high overall motion energy, such that the high-energy stimuli led to higher confidence but lower accuracy in a visual-only task. Importantly, we tested the impact of the low- and high-energy visual stimuli on auditory motion perception in a separate task. Despite being irrelevant to the auditory task, both visual stimuli impacted auditory judgments presumably via automatic low-level mechanisms. Critically, we found that the high-energy visual stimuli influenced the auditory judgments more strongly than the low-energy visual stimuli. This effect was in line with the confidence but contrary to the accuracy differences between the high- and low-energy stimuli in the visual-only task. These effects were captured by a simple computational model that assumes common computational principles underlying both confidence reports and multisensory cue combination. Our results reveal a deep link between automatic sensory processing and metacognitive confidence reports, and suggest that vastly different stages of perceptual decision making rely on common computational principles.

Human-like dissociations between confidence and accuracy in convolutional neural networks

“…In addition, they reproduced another signature of confidence that is popularly regarded as evidence for the positive evidence bias Webb et al, 2023). Further, we show that the confidence increase in the CNNs was due to an increase in separability and variance of evidence distributions, which is essentially the signal-and-variance increase hypothesis that has been proposed for humans too (Fetsch et al, 2014;Gao et al, 2023;Morales et al, 2015;Rahnev et al, 2011Rahnev et al, , 2012Rahnev et al, , 2013Zylberberg et al, 2016). These results demonstrate that CNNs exhibit human-like dissociations between confidence and accuracy, suggesting that higher-order mechanisms are unnecessary to explain the stimulus energyinduced effects on confidence.…”

“…Mechanism behind the confidence-accuracy dissociation in CNNs While findings of confidence-accuracy dissociations in CNNs support the signal-andvariance-increase hypothesis (Fetsch et al, 2014;Gao et al, 2023;Morales et al, 2015;Rahnev et al, 2011Rahnev et al, , 2013Rahnev, Maniscalco, et al, 2012;Zylberberg et al, 2016),a more direct test of this hypothesis can be derived from examining how changes in stimulus energy affect the CNNs' internal distributions. Specifically, the signal-and-variance-increase hypothesis predicts that increasing stimulus energy leads to greater separation of evidence between the two stimulus categories as well as an increase in the variance of evidence.…”

“…A particular type of confidence-accuracy dissociations has been induced by stimulus manipulations referred to as “energy manipulations” (Gao et al, 2023; Zylberberg et al, 2012). Specifically, high energy stimuli can be created by simultaneously increasing stimulus features that aid recognition (e.g., stimulus contrast, motion coherence, etc.)…”

“…Mechanistically, an increase in stimulus energy could lead to greater separation between evidence distributions a well as higher overall variability in the observed evidence – which we call the signal-and-variance-increase hypothesis. Consequently, a larger proportion of this distribution is shifted towards extreme values, thus increasing overall confidence (Gao et al, 2023). Critically, this explanation does not rely on higher-order mechanisms for producing the confidence-accuracy dissociations observed with energy manipulations.…”

How the brain controls decision making in a multisensory world

“…Confidence itself is considered a form of metacognition (thinking about thinking) [5], the study of which (see below) is rapidly advancing in both humans and animal models. Interestingly, the new results [4] support the idea that confidence and cue combination may draw upon a common estimate of sensory uncertainty, despite the former being considered metacognitive and the latter a more automatic perceptual process (see also [6]).…”