Brain dynamics for confidence-weighted learning

Abstract

Learning in a changing, uncertain environment is a difficult problem. A popular solution is to predict future observations and then use surprising outcomes to update those predictions. However, humans also have a sense of confidence that characterizes the precision of their predictions. Bayesian models use a confidence-weighting principle to regulate learning: for a given surprise, the update is smaller when the confidence about the prediction was higher. Prior behavioral evidence indicates that human learning adheres to this confidence-weighting principle. Here, we explored the human brain dynamics sub-tending the confidenceweighting of learning using magneto-encephalography (MEG). During our volatile probability learning task, subjects' confidence reports conformed with Bayesian inference. MEG revealed several stimulus-evoked brain responses whose amplitude reflected surprise, and some of them were further shaped by confidence: surprise amplified the stimulus-evoked response whereas confidence dampened it. Confidence about predictions also modulated several aspects of the brain state: pupil-linked arousal and beta-range (15-30 Hz) oscillations. The brain state in turn modulated specific stimulus-evoked surprise responses following the confidence-weighting principle. Our results thus indicate that there exist, in the human brain, signals reflecting surprise that are dampened by confidence in a way that is appropriate for learning according to Bayesian inference. They also suggest a mechanism for confidence-weighted learning: confidence about predictions would modulate intrinsic properties of the brain state to amplify or dampen surprise responses evoked by discrepant observations.

Neural surprise in somatosensory Bayesian learning

“…Evidence for a sensitivity to confidence of prior knowledge in humans has been reported in a variety of tasks and modalities [96,97,98]. This further speaks to the possibility that CS informs belief updating, as confidence has been suggested to modulate belief updating for other modalities in the literature [99,100] and is explicitly captured in terms of belief precision by other promising Bayesian models [101,102,103]. We suspect that, similarly, confidence concerns the influence of new observations on current beliefs in somatosensation.…”

Confidence-controlled Hebbian learning efficiently extracts category membership from stimuli encoded in view of a categorization task

“…Theoreticians have studied different models of surprise-based learning, in the absence of reward (see [Gerstner et al, 2018] for a review). More recently, making use of fMRI [Meyniel and Dehaene, 2017], EEG [Jepma et al, 2016, Nassar et al, 2019 or MEG [Meyniel, 2019] techniques, it has been shown that this surprise signal is controlled by confidence. Confidence has been shown to grade the reward signal and impact the subsequent learning in a categorization task with mice [Lak et al, 2020].…”

“…The idea that confidence plays a role in learning has been proposed [Summerfield and De Lange, 2014, Meyniel and Dehaene, 2017, Meyniel, 2019. Within the framework of drift-diffusion models and taking a Bayesian viewpoint, [Drugowitsch et al, 2019] have shown that the optimal learning rate for categorization tasks should depend on the confidence in one's decision, where confidence is defined as the probability of having answered correctly.…”

Learning the statistics of pain: computational and neural mechanisms

“…Based on previous evidence in other sensory domains, we hypothesised that subjects use an optimal Bayesian strategy to infer the statistics over time (Meyniel, 2020;Meyniel et al, 2016). We fit subjects' ratings to four variations of a Bayesian model, according to two factors: first, sequence inference through stimulus frequency (by assuming the sequence as generated by a Bernoulli process, where subjects track how often they encountered previous stimuli), versus inference through transition probability (by assuming the sequence follows a Markov transition probability between successive stimuli, where the subject tracks such transition of previous stimuli).…”