Multimodal approach to estimate the ocular movements during EEG recordings: A coupled tensor factorization method

Rivet, Bertrand; Duda, Marc; Guérin–Dugué, Anne; Jutten, Christian; Comon, Pierre

doi:10.1109/embc.2015.7319999

Cited by 28 publications

(30 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Either a single electrode has been selected [9] or the threeway EEG data has been unfolded as a matrix [10]. Recently, several studies have incorporated multi-way data from various neuroimaging modalities, e.g., joint factorization of EEG and magnetoencephalography (MEG) [11], EEG and electro-ocular artifacts (EOG) [12], EEG and fMRI [13], [14], [15]. However, in these studies all electrodes are taken into account in joint analysis and it may not necessarily be better to analyze all electrodes simultaneously, in particular, if the electrodes are from different functional areas.…”

Section: Introductionmentioning

confidence: 99%

Tensor-based fusion of EEG and FMRI to understand neurological changes in schizophrenia

Acar

Levin-Schwartz

Calhoun

2017

2017 IEEE International Symposium on Circuits and Systems (ISCAS)

View full text Add to dashboard Cite

Neuroimaging modalities such as functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) provide information about neurological functions in complementary spatiotemporal resolutions; therefore, fusion of these modalities is expected to provide better understanding of brain activity. In this paper, we jointly analyze fMRI and multi-channel EEG signals collected during an auditory oddball task with the goal of capturing brain activity patterns that differ between patients with schizophrenia and healthy controls. Rather than selecting a single electrode or matricizing the third-order tensor that can be naturally used to represent multi-channel EEG signals, we preserve the multi-way structure of EEG data and use a coupled matrix and tensor factorization (CMTF) model to jointly analyze fMRI and EEG signals. Our analysis reveals that (i) joint analysis of EEG and fMRI using a CMTF model can capture meaningful temporal and spatial signatures of patterns that behave differently in patients and controls, and (ii) these differences and the interpretability of the associated components increase by including multiple electrodes from frontal, motor and parietal areas, but not necessarily by including all electrodes in the analysis.

show abstract

Section: Introductionmentioning

confidence: 99%

Tensor-based fusion of EEG and FMRI to understand neurological changes in schizophrenia

Acar

Levin-Schwartz

Calhoun

2017

2017 IEEE International Symposium on Circuits and Systems (ISCAS)

View full text Add to dashboard Cite

show abstract

“…In addition, through the exploration of the multilinear structure inherent to such data greater understanding of brain activity can be achieved [9], [10], [11]. Therefore, recent studies have arranged multi-channel EEG signals as higher-order tensors and used CMTF-type methods to fuse EEG and magnetoencephalography [12], EEG and gaze data [13] as well as EEG and fMRI [14], [3].…”

Section: Introductionmentioning

confidence: 99%

ACMTF for fusion of multi-modal neuroimaging data and identification of biomarkers

Acar

Levin-Schwartz

Calhoun

2017

2017 25th European Signal Processing Conference (EUSIPCO)

View full text Add to dashboard Cite

Abstract-Joint analysis of neuroimaging data from multiple modalities has the potential to improve our understanding of brain function since each modality provides complementary information. In this paper, we address the problem of jointly analyzing functional magnetic resonance imaging (fMRI), structural MRI (sMRI) and electroencephalography (EEG) data collected during an auditory oddball (AOD) task with the goal of capturing neural patterns that differ between patients with schizophrenia and healthy controls. Traditionally, fusion methods such as joint independent component analysis (jICA) have been used to jointly analyze such multi-modal neuroimaging data. However, previous jICA analyses typically analyze the EEG signal from a single electrode or concatenate signals from multiple electrodes, thus ignoring the potential multilinear structure of the EEG data, and models the data using a common mixing matrix for both modalities. In this paper, we arrange the multi-channel EEG signals as a third-order tensor with modes: subjects, time samples and electrodes, and jointly analyze the tensor with the fMRI and sMRI data, both in the form of subjects by voxels matrices, using a structure-revealing coupled matrix and tensor factorization (CMTF) model. Through this modeling approach, we (i) exploit the multilinear structure of multi-channel EEG data and (ii) capture weights for components indicative of the level of contribution from each modality. We compare the results of the structurerevealing CMTF model with those of jICA and demonstrate that, while both models capture significant distinguishing patterns between patients and controls, the structure-revealing CMTF model provides more robust activation.

show abstract

“…Hence, the CMTF is based on the strong assumption that components in the shared dimension are equal. To relax this assumption, several alternatives have been introduced, such as Advanced CMTF (Karahan et al 2015;Acar et al 2014) which allows both shared and nonshared components in the common mode between the matrix and the tensor or Relaxed Advanced CMTF (Rivet et al 2015), soft coupling (Seichepine et al 2014), and approximate coupling (Farias et al 2016) which provides similarity rather than the equivalence between the common components. Only recently, Chatzichristos et al (Chatzichristos et al 2018) for the first time, used the coupled tensor-tensor decomposition (CTTD) of the EEG and fMRI data fusion.…”

Section: Tensor Factorization-based Eeg-fmri Analysismentioning

confidence: 99%

Extraction of common task features in EEG-fMRI data using coupled tensor-tensor decomposition

Jonmohamadi

Muthukumaraswamy²,

Chen³

et al. 2019

Preprint

View full text Add to dashboard Cite

The fusion of simultaneously recorded EEG and fMRI data is of great value to neuroscience research due to the complementary properties of the individual modalities. Traditionally, techniques such as PCA and ICA, which rely on strong strong nonphysiological assumptions such as orthogonality and statistical independence, have been used for this purpose. Recently, tensor decomposition techniques such as parallel factor analysis have gained more popularity in neuroimaging applications as they are able to inherently contain the multidimensionality of neuroimaging data and achieve uniqueness in decomposition without imposing strong assumptions. Previously, the coupled matrix-tensor decomposition (CMTD) has been applied for the fusion of the EEG and fMRI. Only recently the coupled tensor-tensor decomposition (CTTD) has been proposed. Here for the first time, we propose the use of CTTD of a 4th order EEG tensor (space, time, frequency, and participant) and 3rd order fMRI tensor (space, time, participant), coupled partially in time and participant domains, for the extraction of the task related features in both modalities. We used both the sensor-level and source-level EEG for the coupling. The phase shifted paradigm signals were incorporated as the temporal initializers of the CTTD to extract the task related features. The validation of the approach is demonstrated on simultaneous EEG-fMRI recordings from six participants performing an N-Back memory task. The EEG and fMRI tensors were coupled in 9 components out of which 7 components had a high correlation (more than 0.85) with the task. The result of the fusion recapitulates the well-known attention network as being positively, and the default mode network working negatively time-locked to the memory task.

show abstract

Multimodal approach to estimate the ocular movements during EEG recordings: A coupled tensor factorization method

Cited by 28 publications

References 9 publications

Tensor-based fusion of EEG and FMRI to understand neurological changes in schizophrenia

Tensor-based fusion of EEG and FMRI to understand neurological changes in schizophrenia

ACMTF for fusion of multi-modal neuroimaging data and identification of biomarkers

Extraction of common task features in EEG-fMRI data using coupled tensor-tensor decomposition

Contact Info

Product

Resources

About