Simultaneous real-time monitoring of multiple cortical systems

Abstract

Objective Real-time monitoring of the brain is potentially valuable for performance monitoring, communication, training or rehabilitation. In natural situations, the brain performs a complex mix of various sensory, motor, or cognitive functions. Thus, real-time brain monitoring would be most valuable if (a) it could decode information from multiple brain systems simultaneously, and (b) this decoding of each brain system were robust to variations in the activity of other (unrelated) brain systems. Previous studies showed that it is possible to decode some information from different brain systems in retrospect and/or in isolation. In our study, we set out to determine whether it is possible to simultaneously decode important information about a user from different brain systems in real time, and to evaluate the impact of concurrent activity in different brain systems on decoding performance. Approach We study these questions using electrocorticographic (ECoG) signals recorded in humans. We first document procedures for generating stable decoding models given little training data, and then report their use for offline and for real-time decoding from 12 subjects (6 for offline parameter optimization, 6 for online experimentation). The subjects engage in tasks that involve movement intention, movement execution and auditory functions, separately, and then simultaneously. Main results Our real-time results demonstrate that our system can identify intention and movement periods in single trials with an accuracy of 80.4% and 86.8%, respectively (where 50% would be expected by chance). Simultaneously, the decoding of the power envelope of an auditory stimulus resulted in an average correlation coefficient of 0.37 between the actual and decoded power envelope. These decoders were trained separately and executed simultaneously in real time. Significance This study yielded the first demonstration that it is possible to decode simultaneously the functional activity of multiple independent brain systems. Our comparison of univariate and multivariate decoding strategies, and our analysis of the influence of their decoding parameters, provides benchmarks and guidelines for future research on this topic.

Music can be reconstructed from human auditory cortex activity using nonlinear decoding models

“…We obtained a unique iEEG dataset where 29 neurosurgical patients passively listened to the popular rock song Another Brick in the Wall, Part 1 (by Pink Floyd), while their neural activity was recorded from a total of 2,668 electrodes directly lying on their cortical surface (electrocorticography (ECoG)). This dataset has been used in previous studies asking different research questions without employing decoding or encoding models [39][40][41][42][43]. Passive listening is particularly suited to our stimulus reconstruction approach, as active tasks involving target detection [3,7,8] or perceptual judgments [6,10], while necessary to study key aspects of auditory cognition, can confound the neural processing of music with decision-making and motor activity adding noise to the reconstruction process.…”

Encoding and decoding analysis of music perception using intracranial EEG

“…The dataset we analyzed has been the focus of previous studies, although not employing encoding and decoding models (Potes et al, 2012, 2014; Kubanek et al, 2013; Gupta et al, 2014; Sturm et al, 2014). These studies linked several musical elements, such as sound intensity or timber, to neural activity in the posterior superior temporal gyrus (STG) or sensorimotor areas.…”