The Neural Dynamics of Facial Identity Processing: Insights from EEG-Based Pattern Analysis and Image Reconstruction

Abstract

Uncovering the neural dynamics of facial identity processing along with its representational basis outlines a major endeavor in the study of visual processing. To this end, here, we record human electroencephalography (EEG) data associated with viewing face stimuli; then, we exploit spatiotemporal EEG information to determine the neural correlates of facial identity representations and to reconstruct the appearance of the corresponding stimuli. Our findings indicate that multiple temporal intervals support: facial identity classification, face space estimation, visual feature extraction and image reconstruction. In particular, we note that both classification and reconstruction accuracy peak in the proximity of the N170 component. Further, aggregate data from a larger interval (50–650 ms after stimulus onset) support robust reconstruction results, consistent with the availability of distinct visual information over time. Thus, theoretically, our findings shed light on the time course of face processing while, methodologically, they demonstrate the feasibility of EEG-based image reconstruction.

How are visual words represented? Insights from EEG‐based visual word decoding, feature derivation and image reconstruction

“…To further boost accuracy, we considered the possibility that averaging the similarity matrices of the participants may increase the SNR of the data used for reconstruction purposes (Cowen, Chun, & Kuhl, ; Nemrodov et al, ). Specifically, a single average similarity matrix across the 14 participants was used for word space derivation, feature synthesis and word reconstruction.…”

“…For convenience, the figure shows only the first two dimensions for one representative participant (the two dimensions account for 7.6 and 6.5% of the variance, respectively) similarity see Supporting information, Visual similarity and image reconstruction.). To further boost accuracy, we considered the possibility that averaging the similarity matrices of the participants may increase the SNR of the data used for reconstruction purposes (Cowen, Chun, & Kuhl, 2014;Nemrodov et al, 2018). Specifically, a single average similarity matrix across the 14 participants was used for word space derivation, feature synthesis and word reconstruction.…”

Spatiotemporal dynamics in human visual cortex rapidly encode the emotional content of faces

“…Furthermore, recent applications of MVPA to electrophysiological data have resolved face identity processing to early latencies (50-70 ms after stimulus onset; Davidesco et al, 2014;Nemrodov et al, 2016;Vida, Nestor, Plaut, & Behrmann, 2017). In addition to revealing the temporal dynamics of visual processing, multivariate methods have furthered our understanding of the transformations performed by cells in macaque face patches to encode face identity (Chang & Tsao, 2017) and have allowed face reconstruction based on non-invasive neural data in humans (Nemrodov et al, 2018;Nestor, Plaut, & Behrmann, 2016).…”

Temporal Dynamics of the Neural Representation of Social Relationships

“…Other recent work has highlighted the role of surface texture on the neural representation of face identity (Nemrodov et al, 2018(Nemrodov et al, , 2019. Surface features and invariant shape properties associated with gender and age might be expected to contribute to (dis)similarity in HMAX and/or face space (Johnston et al, 1997), although clearly these models do not capture the entirety of early representations of face identity.…”