Neural Signals Evoked by Stimuli of Increasing Social Scene Complexity Are Detectable at the Single-Trial Level and Right Lateralized

Abstract

Classification of neural signals at the single-trial level and the study of their relevance in affective and cognitive neuroscience are still in their infancy. Here we investigated the neurophysiological correlates of conditions of increasing social scene complexity using 3D human models as targets of attention, which may also be important in autism research. Challenging single-trial statistical classification of EEG neural signals was attempted for detection of oddball stimuli with increasing social scene complexity. Stimuli had an oddball structure and were as follows: 1) flashed schematic eyes, 2) simple 3D faces flashed between averted and non-averted gaze (only eye position changing), 3) simple 3D faces flashed between averted and non-averted gaze (head and eye position changing), 4) animated avatar alternated its gaze direction to the left and to the right (head and eye position), 5) environment with 4 animated avatars all of which change gaze and one of which is the target of attention. We found a late (> 300 ms) neurophysiological oddball correlate for all conditions irrespective of their complexity as assessed by repeated measures ANOVA. We attempted single-trial detection of this signal with automatic classifiers and obtained a significant balanced accuracy classification of around 79%, which is noteworthy given the amount of scene complexity. Lateralization analysis showed a specific right lateralization only for more complex realistic social scenes. In sum, complex ecological animations with social content elicit neurophysiological events which can be characterized even at the single-trial level. These signals are right lateralized. These finding paves the way for neuroscientific studies in affective neuroscience based on complex social scenes, and given the detectability at the single trial level this suggests the feasibility of brain computer interfaces that can be applied to social cognition disorders such as autism.

Technology-Facilitated Diagnosis and Treatment of Individuals with Autism Spectrum Disorder: An Engineering Perspective

“…Some studies did use such virtual reality systems/devices [28][29][30][31][32][33][34][35][36]. However, many studies used conventional computer monitors to display virtual reality scenes with one or more avatars [37][38][39][40][41][42][43][44][45][46][47]. Hence, there is certain degree of overlap between the virtual reality based presentation mode and the conventional computer based presentation mode.…”

“…In [28,37,[39][40][41]57], the Vizard Virtual Reality Toolkit, a commercial product from WorldViz (http://www.worldviz.com/vizard-virtual-reality-software/) was used to produce the avatars in the study. The toolkit contains a PeopleMaker program, which users can use to custom design morphable 3D heads for avatars for different emotion expressions.…”

“…In [39], two different experimental paradigms, flashing paradigm and animated paradigm were used. The flashing paradigm involves three scenarios: (1) flashed schematic eyes, where the target event consists of eye balls slightly rotated while the non-target event consists of two eyeballs on the background screen; (2) flashed face with eye position change, where the target event consists of eyes gazing to the face's right and the non-target event consists of a face facing straight front; (3) flashed face with eye and head position change, where the target event consists of the face facing towards the left while the non-target event consists of a face facing direct forward.…”

“…Hence, it may be desirable to obtain such information during a session of treatment to gain better insight to the state of an individual with ASD. We have seen more autism research that employs various sensors to collected physiological signals, including Pulse Plethysmogram (PPG), Skin Temperature (ST), Electrodermal Activity (EDA), Electrocardiographic (ECG), and Electroencephalography (EEG) [39,40,51,139,140]. Typically, the classification of such signals was done offline.…”

“…Hence, the classification accuracy does not impact the flow of the program and cannot cause harm to the participants. Nevertheless, in the long run, obtaining responses through EEG has the potential to develop brain-computer interfaces to improve social skills of individuals with ASD [39]. EEG data were also collected in [141][142][143][144].…”