Joint representation of working memory and uncertainty in human cortex

Abstract

Neural representations of visual working memory (VWM) are noisy, and thus, decisions based on VWM are inevitably subject to uncertainty. However, the mechanisms by which the brain simultaneously represents the content and uncertainty of memory remain largely unknown. Here, inspired by the theory of probabilistic population codes, we test the hypothesis that the human brain represents an item maintained in VWM as a probability distribution over stimulus feature space, thereby capturing both its content and uncertainty. We used a neural generative model to decode probability distributions over memorized locations from fMRI activation patterns. We found that the mean of the probability distribution decoded from retinotopic cortical areas predicted memory reports on a trial-by-trial basis. Moreover, in several of the same mid-dorsal stream areas the spread of the distribution predicted subjective trial-by-trial uncertainty judgments. These results provide evidence that VWM content and uncertainty are jointly represented by probabilistic neural codes.

Persistent Activity During Working Memory From Front to Back

“…Hypothesis 1: These monkey data predict that lesions to human dlPFC will impair spatial WM performance, including the accuracy of memory-guided saccades. However, human neuroimaging studies typically find persistent activity or multivoxel decoding of information restricted to the PCS, posterior to the likely homolog of the monkey principal sulcus in the dlPFC ( Courtney et al, 1998 ; Srimal and Curtis, 2008 ; Jerde et al, 2012 ; Sprague et al, 2014 ; Hallenbeck et al, in press ; Li et al, in press ). Hypothesis 2: These data predict that lesions to human PCS, not dlPFC, will impair WM performance.…”

“…In the decade since, dozens of studies have applied similar methods to decode visual stimulus features such as orientation, motion direction, color, spatial position, and the identity of a spatial pattern from brain activation patterns measured from striate and extrastriate visual cortex ( Christophel et al, 2017 ). Moreover, modified versions of these decoding methods, including cvMANOVA ( Allefeld and Haynes, 2014 ; Christophel et al, 2018a ), inverted encoding models (IEMs) ( Figure 4A ; Ester et al, 2013 ; Sprague et al, 2014 ), and Bayesian decoding methods ( van Bergen et al, 2015 ; van Bergen and Jehee, 2018 , 2021 ; Brissenden et al, 2021 ; Li et al, in press ) have increasingly improved the resolution and sensitivity of these methods to differences between conditions, and, ultimately, between individual trials. These new methods have revealed feature-selective representations broadly across visual, parietal, and frontal cortex ( Christophel et al, 2012 , 2018a , b ; Jerde et al, 2012 ; Christophel and Haynes, 2014 ; Sprague et al, 2014 ; Ester et al, 2015 ; Yu and Shim, 2017 ; Rahmati et al, 2018 ; Li et al, in press ), along with subcortical regions including the SC ( Rahmati et al, 2020 ) and cerebellum ( Brissenden et al, 2021 ).…”

“…Experiments parametrically manipulating task demands in concert with visual distraction may help further clarify the relative role different brain regions play in WM tasks. An especially promising avenue for future exploration is comparing decoded feature values from single trials of fMRI activation to behavioral errors on those same trials ( Ester et al, 2016 ; Hallenbeck et al, in press ; Li et al, in press ).…”

An enhanced inverted encoding model for neural reconstructions

“…standard procedure. The lack of uncertainty information has been noted in other contexts, with some recent alternatives to IEM proposed to incorporate uncertainty 42,43 .…”

“…Individual trials in a neuroimaging study can vary substantially in signal quality (driven by e.g., attentional fluctuations, alertness, head motion, scanner noise) but the standard IEM procedure does not incorporate uncertainty into decoding performance. The lack of uncertainty information has been noted in other contexts, with some recent alternatives to IEM proposed to incorporate uncertainty 41,42 . eIEM easily and automatically produces a trial-by-trial measure of prediction uncertainty within the IEM framework itself.…”