A multimodal brain-based feedback and communication system

Abstract

The Thought Translation Device (TTD) is a brain-computer interface based on the self-regulation of slow cortical potentials (SCPs) and enables completely paralyzed patients to communicate using their brain potentials. Here, an extended version of the TTD is presented that has an auditory and a combined visual and auditory feedback modality added to the standard visual feedback. This feature is necessary for locked-in patients who are no longer able to focus their gaze. In order to test performance of physiological regulation with auditory feedback 54 healthy participants were randomly assigned to visual, auditory or combined visual-auditory feedback of slow cortical potentials. The training consisted of three sessions with 500 trials per session with random assignment of required cortical positivity or negativity in half of the trials. The data show that physiological regulation of SCPs can be learned with auditory and combined auditory and visual feedback although the performance of auditory feedback alone was significantly worse than with visual feedback alone.

EEG single-trial classification of visual, auditive and vibratory feedback potentials in Brain-Computer Interfaces

“…Despite the visual system being the sensory input that produces the best improvements in learning processes [8] there are many situations where other types of feedback are required, due to the pathology itself or requirements of the rehabilitation process. This is the reason supporting the first attempts to analyze from a psychological point of view different feedback modalities in complex settings, such as driving a wheelchair [3].…”

Brain-Computer Interfaces in Medicine

“…Steady-state visual evoked potentials have been used for binary selection, [48][49][50][51] both discrete and continuous control of a cursor in 2 dimensions, 52 prosthesis control, 53 FES, 54,55 spelling, 56 and environmental control. 57 For patients with impaired vision, various auditory 39,40,[58][59][60][61] and tactile [62][63][64][65] paradigms have been investigated. A few studies are now focused on the critical need to move BCI systems out of the laboratory and into patients' homes, which raises many complex patient, caregiver, and implementation issues.…”

The Use of Brain-Computer Interfacing in Ambient Intelligence

“…for instance, self modulation by imagination of movements can result in changes in EEG rhythm in central region of the scalp overlying the sensorimotor cortex [3] [8] [33] [45]. These rhythms are the basis of several BCI systems [3] [8] in which imagination of hand movement gives rise to an amplitude suppression in the α-band (8-12 Hz) and β-band (13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28) 4 rhythms over the contralateral primary hand motor cortical area [33]. Wolpaw and co-workers [43] [45] used continuous changes in the amplitudes of these rhythms to move a cursor in a computer screen.…”

“…Attentional modulation seems to constitute the cognitive strategy in the physiological regulation of SCP. The team lead by Birbaumer has widely shown that healthy subjects as well as severely paralyzed patients can learn to self-control their SCPs through operant conditioning to move an object on a computer screen in a BCI referred to as Thought Translation Device (TTD) [17].…”