Structured sparse multiset canonical correlation analysis of simultaneous fNIRS and EEG provides new insights into the human action-observation network

Abstract

The action observation network (AON) is a network of brain regions involved in the execution and observation of a given action. The AON has been investigated in humans using mostly electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), but shared neural correlates of action observation and action execution are still unclear due to lack of ecologically valid neuroimaging measures. In this study, we used concurrent EEG and functional Near Infrared Spectroscopy (fNIRS) to examine the AON during a live-action observation and execution paradigm. We developed structured sparse multiset canonical correlation analysis (ssmCCA) to perform EEG-fNIRS data fusion. MCCA is a generalization of CCA to more than two sets of variables and is commonly used in medical multimodal data fusion. However, mCCA suffers from multi-collinearity, high dimensionality, unimodal feature selection, and loss of spatial information in interpreting the results. A limited number of participants (small sample size) is another problem in mCCA, which leads to overfitted models. Here, we adopted graph-guided (structured) fused least absolute shrinkage and selection operator (LASSO) penalty to mCCA to conduct feature selection, incorporating structural information amongst the variables (i.e., brain regions). Benefitting from concurrent recordings of brain hemodynamic and electrophysiological responses, the proposed ssmCCA finds linear transforms of each modality such that the correlation between their projections is maximized. Our analysis of 21 right-handed participants indicated that the left inferior parietal region was active during both action execution and action observation. Our findings provide new insights into the neural correlates of AON which are more fine-tuned than the results from each individual EEG or fNIRS analysis and validate the use of ssmCCA to fuse EEG and fNIRS datasets.

Simultaneous Multimodal fNIRS-EEG Recordings Reveal New Insights in Neural Activity During Motor Execution, Observation, and Imagery

“…Our unimodal fNIRS ndings are consistent with previous fNIRS ndings. In our previous publication, we extensively discussed the different cortical activation between ME and MO [7]. In the current study, we added the MI condition and compared its differences in cortical activation with the other two conditions.…”

“…optode was digitized in reference to the nasion, inion, and preauricular landmarks using a 3D-magnetic space digitizer (Fastrak, Polhemus). Additional information about these methods can be found in our prior publication [7,38].…”

“…We applied the ssmCCA technique to integrate EEG and fNIRS datasets for multiple subjects [7]. Our EEG matrix for each participant consisted of the changes in the power over the α frequency band in EEG channels.…”

“…Moreover, ndings from different neuroimaging modalities are inconsistent in terms of regions showing differentiated neural activity between conditions [5,6]. Given the strengths of electroencephalography's (EEG) temporal resolution and functional near-infrared spectroscopy's (fNIRS) spatial resolution along with their tolerance of movement artifacts, our research group previously used a multimodal approach that includes simultaneous EEG-fNIRS recording and a fusion analytical method (i.e., structured sparse multiset canonical correlation analysis, ssmCCA) to examine neural activity over AON during ME and MO [7]. In the current study, we further examined the neural activities during MI using this approach and compared both unimodal and multimodal neural activation among the three conditions.…”

Investigation of neuromodulatory effect of anodal cerebellar transcranial direct current stimulation on the primary motor cortex using functional near-infrared spectroscopy

“…fNIRS provides metabolic information non-invasively and with adequate sensitivity to detect even small changes in the cerebral hemodynamic response. It allows estimation of metabolic-based specificity [25], and robustness to various artifacts [26]. Furthermore, fNIRS can be used simultaneously with tDCS to measure brain activity even during the stimulation period without significant electro-optic interference.…”