Linear reconstruction of perceived images from human brain activity

Schoenmakers, Sanne; Barth, Markus; Heskes, Tom; Gerven, Marcel van

doi:10.1016/j.neuroimage.2013.07.043

Cited by 130 publications

(121 citation statements)

References 29 publications

Supporting

Mentioning

121

Contrasting

Order By: Relevance

“…One field that has pioneered this approach is visual image reconstruction (Thirion et al, 2006;Miyawaki et al, 2008;Naselaris et al, 2009;Nishimoto et al, 2011;Schoenmakers et al, 2013). Image reconstruction refers to the attempt at decoding arbitrary images (rather than a few known samples) from brain signals.…”

Section: Emerging Directions: Encoding Reconstruction and Computatimentioning

confidence: 99%

A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives

Haynes

2015

Neuron

396

307

View full text Add to dashboard Cite

Human fMRI signals exhibit a spatial patterning that contains detailed information about a person's mental states. Using classifiers it is possible to access this information and study brain processes at the level of individual mental representations. The precise link between fMRI signals and neural population signals still needs to be unraveled. Also, the interpretation of classification studies needs to be handled with care. Nonetheless, pattern-based analyses make it possible to investigate human representational spaces in unprecedented ways, especially when combined with computational modeling.

show abstract

Section: Emerging Directions: Encoding Reconstruction and Computatimentioning

confidence: 99%

A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives

Haynes

2015

Neuron

396

307

View full text Add to dashboard Cite

show abstract

“…We compare our proposed multi-target model with two state-of-the-art voxel-wise models (Ridge and Lasso); these two models were widely employed in fMRI encoding models (Agrawal et al 2014;Kay et al 2008;Schoenmakers et al 2013;Güçlü and van Gerven 2014). Only data from training sessions were used to construct models and select regularization coefficients 1 , 2 ; data from validation sessions were used to validate the model performances.…”

Section: Resultsmentioning

confidence: 99%

“…As done in the previous study (Schoenmakers et al 2013), we sample lambda in the range (10 −5 , 10 5 ) on a log scale. For the convenience of discussion, we refer the voxel-wise model with ridge regularizer as Ridge and the model with Lasso regularizer as Lasso.…”

Section: Voxel-wise Modelsmentioning

confidence: 99%

“…In the regression part of encoding, regularized linear regression models such as lasso (Kay et al 2008), ridge regression (Güçlü and van Gerven 2014) and graph-constrained elastic net (Kay et al 2008;Schoenmakers et al 2013) were most commonly used. Recently, a more advanced sparse nonparametric regression model was proposed (Vu et al 2011).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A spatial-constrained multi-target regression model for human brain activity prediction

Wen

2016

Appl Inform

View full text Add to dashboard Cite

“…However, these hemifield manipulations provide only a very coarse measure of how spatial attention is distributed across the visual field. Multivariate BOLD reconstruction has shown greater promise in accessing high-resolution spatial representations in the brain, successfully reconstructing simple contrast patterns 27,28 , handwritten digits 29 , natural images and movies 30,31 , and faces 32 . More recently, reconstruction techniques have been used to access covert spatial representations in the brain.…”

Section: Accessing and Reconstructing Representations Of Spatial Attementioning

confidence: 99%

Predicting eye movements from deep neural network activity decoded from fMRI responses to natural scenes

Chun

2017

Preprint

View full text Add to dashboard Cite

Computational models of selective spatial attention can reliably predict eye movements to complex images. However, researchers lack a simple way to measure covert representations of spatial attention in the brain and their link to overt eye movement behavior, especially in response to natural scenes. Here, we predict eye movement patterns from spatial priority maps reconstructed from brain activity measured with functional magnetic resonance imaging (fMRI). First, we define a computational spatial attention model using a deep convolutional neural network (CNN) pre-trained for scene categorization. Next, we decode CNN unit activity from fMRI activity and reconstruct spatial priority maps by applying our computational spatial attention model to decoded CNN activity. Finally, we predict eye movements in a subsequent behavioral experiment within and between individuals using reconstructed spatial priority maps. These results demonstrate that features represented in CNN unit activity can guide spatial attention and eye movements, providing a crucial link between CNN models, brain activity, and behavior.

show abstract

Linear reconstruction of perceived images from human brain activity

Cited by 130 publications

References 29 publications

A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives

A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives

A spatial-constrained multi-target regression model for human brain activity prediction

Predicting eye movements from deep neural network activity decoded from fMRI responses to natural scenes

Contact Info

Product

Resources

About