Slides
International Joint Conference on Neural Networks 2011
Classification of EEG During Imagined Mental Tasks by Forecasting with Elman Recurrent Neural Networks (Selected Student Speaker)
Abstract:
The ability to classify EEG recorded while a subject performs varying imagined mental tasks may lay the foundation for building usable Brain-Computer Interfaces as well as improve the performance of EEG analysis software used in clinical settings. Although a number of research groups have produced EEG classifiers, these methods have not yet reached a level of performance that is acceptable for use in many practical applications. We assert that current approaches are limited by their ability to capture the temporal and spatial patterns contained within EEG. In order to address these problems, we propose a new generative technique for EEG classification that uses Elman Recurrent Neural Networks. EEG recorded while a subject performs one of several imagined mental tasks is first modeled by training a network to forecast the signal a single step ahead in time. We show that these models are able to forecast EEG well with an RMSE as low as 0.11. A separate model is then trained over EEG belonging to each class. Classification of previously unseen data is performed by applying each model and assigning the class label associated with the network that produced the lowest forecasting error. This approach is tested on EEG collected from two able-bodied subjects and one subject with a high-level spinal cord injury. Classification rates as high as 93.3% are achieved for a two-task problem with decisions made every second yielding a bitrate of 38.7 bits per minute.
Front Range Neuroscience Group 2011
Modeling and Classification of EEG by Forecasting with Recurrent Artificial Neural Networks (Selected Student Speaker)
Abstract:
A Brain-Computer Interface (BCI) is a system that allows a user to operate a computerized
device by voluntarily manipulating their mental state. BCI bypass our innate motor-based means
of communication by directly observing changes in neural activity. This may yield an exciting new
form of communication, particularly for those who suffer from disabilities that make interaction with
the outside word difficult, such as amyotrophic lateral sclerosis, high-level spinal cord injury and
some forms of stroke.
Recently, there has been significant interest in building non-invasive BCI systems using elec-
troencephalography (EEG). In these approaches, an array of electrodes is placed on the surface
of a subject’s scalp in order to monitor brain activity. Machine learning and pattern analysis algo-
rithms can then be used identify changes in mental state and issue the appropriate commands to
the device that the user wishes to control. Although a number of research groups have demon-
strated that this approach can deliver working BCI systems, current approaches do not perform
well enough for use in many practical, real-world applications.
We assert that current approaches are often limited by the use of purely frequency-based
feature representations and by linear classification classification algorithms. In order to address
these limitations, we have developed an algorithm for identifying patterns in EEG that utilizes
versatile learning machines known as Recurrent Artificial Recurrent Neural Networks (RNN). RNN
consist of a number of simple computational units with weighted interconnections. An RNN can
be trained to map inputs to outputs by adjusting the strength of the connections between the
computational units. RNN also contain delayed feedback connections which gives them an intrinsic
memory and the ability to learn complex spatiotemporal patterns.
In order to identify a user’s mental state through EEG, we first train a separate RNN to model
sample EEG recorded during each mental state by forecasting the signal a single step ahead in
time. Thus, if we have K imagined mental states we train K different RNN. Each of these RNN
models can then be thought of as an expert at predicting EEG produced during each mental state.
Previously unseen EEG can then be identified by applying each RNN and assigning the label
associated with the model that was able to best predict the signal.
We test this approach on EEG recorded from five subjects, two of which are able-bodied, two
with high-level spinal cord injuries and one with severe multiple sclerosis. A cue was presented
to each subject on an LCD screen instructing them to perform one of four imagined mental tasks:
imagined right hand movement, counting backward from 100 by 3’s, silently sing a favorite song
and visualization of a tumbling cube. Although our preliminary analysis is performed offline, a BCI
can be operated in this fashion by associating a command with each imagined mental task. For
example, imagined right hand movement may turn a wheelchair to the right while silently singing
a song may turn it to the left. Although cumbersome at first, this technique may become second-
nature with extended periods of practice.
The application of our classification algorithm to these datasets indicates that a BCI user may
be able to communicate as many 34.5 bits per minute when selecting between four tasks with
decisions made every second. On average, these subjects are able to communicate 13.2 bits per
minute, suggesting that this technique outperforms a number of other state-of-the-art approaches.
Additionally, we observe that placing a feedback loop between the inputs and outputs of a trained
RNN, forming an autonomous and self-driven system, produces rich and long-term dynamics that
strongly resemble true EEG.