Coarse behavioral context decoding

Abstract

Objective. Current brain-computer interface (BCI) studies demonstrate the potential to decode neural signals obtained from structured and trial-based tasks to drive actuators with high performance within the context of these tasks. Ideally, to maximize utility, such systems will be applied to a wide range of behavioral settings or contexts. Thus, we explore the potential to augment such systems with the ability to decode abstract behavioral contextual states from neural activity. Approach. To demonstrate the feasibility of such context decoding, we used electrocorticography (ECoG) and stereo-electroencephalography (sEEG) data recorded from the cortical surface and deeper brain structures, respectively, continuously across multiple days from three subjects. During this time, the subjects were engaged in a range of naturalistic behaviors in a hospital environment. Behavioral contexts were labeled manually from video and audio recordings; four states were considered: engaging in dialogue, rest, using electronics, and watching television. We decode these behaviors using a factor analysis and support vector machine (SVM) approach. Main results. We demonstrate that these general behaviors can be decoded with high accuracies of 73% for a four-class classifier for one subject and 71% and 62% for a three-class classifier for two subjects. Significance. To our knowledge, this is the first demonstration of the potential to disambiguate abstract naturalistic behavioral contexts from neural activity recorded throughout the day from implanted electrodes. This work motivates further study of context decoding for BCI applications using continuously recorded naturalistic activity in the clinical setting.

Behavioral and Neural Variability of Naturalistic Arm Movements

“…Similar spectral power changes have also been observed in EEG and local field potential recordings across a wide variety of movement behaviors (Chung et al, 2018;Milekovic et al, 2015;Ofori et al, 2015;Tan et al, 2016). An important attribute of ECoG recordings is that the patients are being continuously monitored over long periods of time, often approximately a week, providing unique opportunities to collect long-term datasets during unconstrained, uninstructed movements (Alasfour et al, 2019;Chao et al, 2010;Gabriel et al, 2019;Vansteensel et al, 2013;Wang et al, 2018Wang et al, , 2016. However, the behavioral and neural variability of such spontaneous, naturalistic movements remains unexplored.…”

Behavioral and Neural Variability of Naturalistic Arm Movements

“…Similar spectral power changes have also been observed in EEG and local field potential recordings across a wide variety of movement behaviors [59][60][61][62][63]. An important attribute of ECoG recordings is that the patients are being continuously monitored over long periods of time, often approximately a week, providing unique opportunities to collect long-term datasets during unconstrained, uninstructed movements [64][65][66][67][68][69]. However, the behavioral and neural variability of such spontaneous, naturalistic movements remains unexplored.…”

The Potential of Stereotactic-EEG for Brain-Computer Interfaces: Current Progress and Future Directions

“…As sEEG targets deeper brain structures including limbic regions such as the amygdala, it is well-suited to detect and decode brain activity associated with such user states. Alasfour et al (2019) demonstrated the classification of abstract naturalistic behavioral contexts from ECoG and sEEG recordings, which could be used to adapt interfaces to the coarse behavioral context of users in the future. Sani et al (2018) showed that mood variations during natural behavior can be decoded from intracranial recordings (including sEEG).…”