Decoding visual colour from scalp electroencephalography measurements

Hajonides, Jasper E.; Nobre, Anna C.; Ede, Freek van; Stokes, Mark G.

doi:10.1016/j.neuroimage.2021.118030

“…For each model, we calculated the correlation between the weights from each cue unit and the recurrent units (weights from an example model shown in Fig 4B). Consistent with other recent observations, we found that the models learnt opposing weight patterns for the two retro-cue inputs [11,23]. More specifically, the recurrent units that had positive connections weights with the first retro-cue unit had negative connection weights linking them with the other cue unit, and vice versa.…”

Section: Learning Dynamics and Connectivitysupporting

confidence: 90%

A recurrent neural network model of prefrontal brain activity during a working memory task

Piwek

¹

,

Stokes

²

,

Summerfield

³

2022

Preprint

1

0

View full text Add to dashboard Cite

When multiple items are held in short-term memory, cues that retrospectively prioritise one item over another (retro-cues) can facilitate subsequent recall. However, the neural and computational underpinnings of this effect are poorly understood. One recent study recorded neural signals in the macaque lateral prefrontal cortex (LPFC) during a retro-cueing task, contrasting delay-period activity before (pre-cue) and after (post-cue) retrocue onset. They reported that in the pre-cue delay, the individual stimuli were maintained in independent subspaces of neural population activity, whereas in the post-cue delay, the prioritised items were rotated into a common subspace, potentially allowing a common readout mechanism. To understand how such representational transitions can be learnt through error minimisation, we trained recurrent neural networks (RNNs) with supervision to perform an equivalent cued-recall task. RNNs were presented with two inputs denoting conjunctive colour-location stimuli, followed by a pre-cue memory delay, a location retrocue, and a post-cue delay. We found that the orthogonal-to-parallel geometry transformation observed in the macaque LPFC emerged naturally in RNNs trained to perform the task. Interestingly, the parallel geometry only developed when the cued information was required to be maintained in short-term memory for several cycles before readout, suggesting that it might confer robustness during maintenance. We extend these findings by analysing the learning dynamics and connectivity patterns of the RNNs, as well as the behaviour of models trained with probabilistic cues, allowing us to make predictions for future studies. Overall, our findings are consistent with recent theoretical accounts which propose that retrocues transform the prioritised memory items into a prospective, action-oriented format.

show abstract

“…Data were further preprocessed. Magnitudes of magnetometers were approximately matched to gradiometers by multiplication (factor 20) and subjected to spatiotemporal decoding (code available at https://pypi.org/project/temp-dec/ ; as described previously, Wolff et al, 2017 , 2020 ; Hajonides et al, 2021 ). Data from all 306 MEG sensors across a sliding window of 30 time points (150 ms) were concatenated into a 9180-dimensional vector.…”

Section: Methodsmentioning

confidence: 99%

Multiple and Dissociable Effects of Sensory History on Working-Memory Performance

Hajonides

¹

,

Ede

²

,

Stokes

³

et al. 2023

Self Cite

View full text Add to dashboard Cite

Behavioural reports of sensory information are biased by stimulus history. The nature and direction of such serial-dependence biases can differ between experimental settings – both attractive and repulsive biases towards previous stimuli have been observed. How and when these biases arise in the human brain remains largely unexplored. They could occur either via a change in sensory processing itself and/or during post-perceptual processes such as maintenance or decision-making. To address this, we tested 20 participants (11 female) and analysed behavioural and magnetoencephalographic data from a working-memory task in which participants were sequentially presented with two randomly oriented gratings, one of which was cued for recall at the end of the trial. Behavioural responses showed evidence for two distinct biases: 1) a within-trial repulsive bias away from the previously encoded orientation on the same trial, and 2) a between-trial attractive bias towards the task-relevant orientation on the previous trial. Multivariate classification of stimulus orientation revealed that neural representations during stimulus encoding were biased away from the previous grating orientation, regardless of whether we considered the within- or between-trial prior orientation – despite opposite effects on behaviour. These results suggest that repulsive biases occur at the level of sensory processing and can be overridden at post-perceptual stages to result in attractive biases in behaviour.SIGNIFICANCE STATEMENT:Recent experience biases behavioural reports of sensory information, possibly capitalizing on the temporal regularity in our environment. It is still unclear at what stage of stimulus processing such serial biases arise. Here, we recorded behaviour and neurophysiological (MEG) data to test whether neural activity patterns during early sensory processing show the same biases seen in participants' reports. In a working-memory task that produced multiple biases in behaviour - responses were biased towards previous targets, but away from more recent stimuli - neural activity patterns were uniformly biased away from all previously relevant items. Our results contradict proposals that all serial biases arise at an early sensory processing stage. Instead, neural activity exhibited mostly adaptation-like responses to recent stimuli.

show abstract

“…Data were further pre-processed, where magnitudes of magnetometers were approximately matched to gradiometers by multiplication (factor 20) and subjected to spatio-temporal decoding as described in (Hajonides, Nobre, van Ede, & Stokes, 2021;Wolff, Jochim, Akyürek, Buschman, & Stokes, 2020;Wolff, Jochim, Akyürek, & Stokes, 2017). Data from all 306 MEG sensors across a sliding window of 30 time points (150 ms) were concatenated into a vector.…”

Section: Lda Classificationmentioning

confidence: 99%

Multiple and Dissociable Effects of Sensory History on Working-Memory Performance

Hajonides

¹

,

Ede²,

Stokes

³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Behavioural reports of sensory information are biased by stimulus history. The nature and direction of such serial-dependence biases can differ between experimental settings – both attractive and repulsive biases towards previous stimuli have been observed. How and when these biases arise in the human brain remains largely unexplored. They could occur either via a change in sensory processing itself, post-perceptual maintenance or decision-making processes, or both. Here, we analysed behavioural and magnetoencephalographic data from a working-memory task in which participants were sequentially presented with two randomly oriented gratings, one of which was cued for recall at the end of the trial. Behavioural responses showed evidence for two distinct biases: 1) a within-trial repulsive bias away from the previously encoded orientation on the same trial, and 2) a between-trial attractive bias towards the task-relevant orientation on the previous trial. Multivariate classification of stimulus orientation revealed that neural representations during stimulus encoding were biased away from the previous grating orientation, regardless of whether we considered the within- or between-trial prior orientation – despite opposite effects on behaviour. These results suggest that repulsive biases occur at the level of sensory processing and can be overturned at post-perceptual stages to result in attractive biases in behaviour.

show abstract

Decoding visual colour from scalp electroencephalography measurements

Cited by 39 publications

References 52 publications

A recurrent neural network model of prefrontal brain activity during a working memory task

A recurrent neural network model of prefrontal brain activity during a working memory task

Multiple and Dissociable Effects of Sensory History on Working-Memory Performance

Multiple and Dissociable Effects of Sensory History on Working-Memory Performance

Contact Info

Product

Resources

About