Encoding brain network response to free viewing of videos

Abstract

A challenging goal for cognitive neuroscience researchers is to determine how mental representations are mapped onto the patterns of neural activity. To address this problem, functional magnetic resonance imaging (fMRI) researchers have developed a large number of encoding and decoding methods. However, previous studies typically used rather limited stimuli representation, like semantic labels and Wavelet Gabor filters, and largely focused on voxel-based brain patterns. Here, we present a new fMRI encoding model to predict the human brain's responses to free viewing of video clips which aims to deal with this limitation. In this model, we represent the stimuli using a variety of representative visual features in the computer vision community, which can describe the global color distribution, local shape and spatial information and motion information contained in videos, and apply the functional connectivity to model the brain's activity pattern evoked by these video clips. Our experimental results demonstrate that brain network responses during free viewing of videos can be robustly and accurately predicted across subjects by using visual features. Our study suggests the feasibility of exploring cognitive neuroscience studies by computational image/video analysis and provides a novel concept of using the brain encoding as a test-bed for evaluating visual feature extraction.

Optimal path-finding through mental exploration based on neural energy field gradients

“…Later, Redish et al combined the two cognitive maps in 1998 to make symmetric and asymmetric connections coexist. There have been many promising works about neural coding and the cognitive map in the recent years (Han et al 2014; Duch and Dobosz 2011; Strauss et al 2010; Sato and Yamaguchi 2009; Huyck 2009; Wagatsuma and Yamaguchi 2007). And there are also many wonderful pioneering researches on neural dynamics model which set up a framework for the theory and application of neural system model (Adeli and Park 1998; Park and Adeli 1995, 1997; Adeli and Karim 1997; Senouci and Adeli 2001; Adeli and Kim 2001; Tashakori and Adeli 2002; Ahmadkhanlou and Adeli 2005).…”

The graph theoretical analysis of the SSVEP harmonic response networks

“…Increasing evidence demonstrates the value of brain network topology for understanding cognitive processes and diseases (Han et al 2014;Li et al 2009;Dubbelink 2014;Pei et al 2014;Yener and Basar 2010). The graph theory serves as a powerful tool to investigate the brain network topology (Bullmore and Sporns 2009;Rubinov and Sporns 2010).…”

Exploring Hierarchical Auditory Representation via a Neural Encoding Model

“…These studies offer promising prospects to understand the fundamental mechanisms of brain functions responding to external stimuli. Specifically, brain encoding models ( Naselaris et al, 2011 ; Han et al, 2014 ; Mesgarani et al, 2014 ; Du et al, 2019 ) have been used to establish the relationship between acoustic features represented in different layers of DNNs and brain activities. Brain regions of interest that selectively respond to extracted features in different layers were then inferred according to encoding performance.…”