Using Real-Time fMRI to Control a Dynamical System by Brain Activity Classification

Abstract

Abstract. We present a method for controlling a dynamical system using real-time fMRI. The objective for the subject in the MR scanner is to balance an inverted pendulum by activating the left or right hand or resting. The brain activity is classified each second by a neural network and the classification is sent to a pendulum simulator to change the force applied to the pendulum. The state of the inverted pendulum is shown to the subject in a pair of VR goggles. The subject was able to balance the inverted pendulum during several minutes, both with real activity and imagined activity. In each classification 9000 brain voxels were used and the response time for the system to detect a change of activity was on average 2-4 seconds. The developments here have a potential to aid people with communication disabilities, such as locked in people. Another future potential application can be to serve as a tool for stroke and Parkinson patients to be able to train the damaged brain area and get real-time feedback for more efficient training.

fMRI analysis on the GPU—Possibilities and challenges

“…We have demonstrated that it is possible to control a dynamical system [32] and communicate [33] by classifying the brain activity every second; similar work has been done by LaConte et al [34]. deCharms et al [35] helped subjects to suppress their pain by letting them see their own brain activity in real-time.…”

Decoding fMRI brain states in real-time

“…Although our reported real-time system used an SVM implementation (Joachims, 1999), this is really a modular part of the system that allows future extensions in which multiple classifiers (either individually or combined) can be used (LaConte et al, 2007). Recently Eklund et al (2009) 4. An example AFNI rtfMRI display with stimulus window.…”

“…Using this system, Papageorgiou et al (2009) has recently reported the ability to provide feedback based on slow vs. fast inner, automatic speech. Using a similar approach based on neural networks, Eklund et al (2009) has reported the ability to control the dynamics of a simulated inverted pendulum, using classification of left, right, and resting conditions. Using the relevance vector machine, Hollmann et al (2009) presented a subject's neuroeconomic decisions to the operator before that subject pressed a button to convey his decision.…”

A Brain Computer Interface for Communication Using Real-Time fMRI

“…While previous fMRI based BCI have classified between two classes [1] or three classes [2], we instead classify between five different classes, to give the subject five degrees of freedom. We choose to use our system for communication, since EEG has been used for communication [3] and many have claimed that fMRI is far too slow for similar setups.…”