Jeshua Bratman's CSU Homepage

CEBL

CSU EEG Brain-Computer Interface Lab (CEBL):

Project Information

Since 2005 I have been a part of the EEG research group in the Computer Science department of Colorado State University. Here is the website for the group.

My part in this group has been developing a graphical user interface to connect the many parts of EEG BCI research:

EEG cap and amplifier
Classification algorithms
Data filtering and feature extraction algorithms
Visual representation of EEG data
User training interfaces
User input interfaces
Storing and loading of previously experiments

A more detailed overview of the project is contained below.

Abstract

Here's my abstract for my most recent poster:

Use A Computer With Your Mind Alone: A Real-Time Brain-Computer Interface
Jeshua Bratman, and Chuck Anderson
Department of Computer Science, Colorado State University

This research project aims to use electrical signals recorded from the brain from electroencephalograms (EEGs) to create a form of communication between a person and a computer separate from the regular channels of muscle movement and speech. The long-range purpose is to allow victims of paralyzing diseases or accidents a new way to communicate with the outside world. Immediate goals include recognizing mental tasks performed by a subject in real time, designing useful computer input interfaces, and creating a portable and inexpensive EEG brain-computer interface (BCI) system.

Recognizing mental tasks based on EEG signals received from a subject is the first and most important goal of this project. Mental tasks are imagined exercises performed by the user which can be easily distinguished by analyzing the EEG signal. Examples of tasks include imagined hand or foot movement, and mental arithmetic. To accomplish the classification of these tasks, various machine-learning techniques are applied to the EEG input in order to develop methods to accurately determine what mental task the user is performing.

Creating a useful interface to a computer from the recognition of mental tasks is the second goal, and an important piece of any BCI system. The higher the number of different mental tasks, the lower the accuracy of classification; therefore, any input system will rely on a very limited number of separate inputs. This leads to the problem of designing efficient user-computer interaction from this limited input. To illustrate this difficulty, imagine using a computer with a keyboard having just three or four keys. We have designed several interfaces aiming to solve this problem.

Designing a portable piece of software for EEG BCI experimentation is the third goal. This project encapsulates the first two in many ways. The software we have designed connects all the previous aspects of BCI research allowing easy experimentation and data collection. Not only does this system help immensely in accomplishing the previous two goals, it will be an asset to the BCI community as a whole.

Goals and Methods

Colorado State University Department of Computer Science EEG Brain-Computer Interface Project

Goals

computer control with no muscle movement whatsoever
different cognitive tasks for different computer input. e.g.:
- imagined hand/foot movement
- multiplying numbers in your head
- imagining writing a letter
- etc.
to allow victims of paralysis to have a new channel of communication into the world
to develop a real-time BCI including fast and effective classification algorithms
- real-time data analysis and training
- an inexpensive yet sophisticated EEG BCI research system including hardware and free, open source software.

My Contribution

free and open source alternative to expensive BCI research software
implementation of various learning algorithms and data manipulation algorithms
- Linear Discriminant Analysis, Quadratic Discriminant Analysis
- Data Lagging
- MNF Filtering
- Passband filter
- Incremental SVD

easy to get running and to modify using well-known libraries
- graphical user interface libraries
  - GTK – GIMP Toolkit – available on many platforms. (gtk.org)
- c++ matrix manipulation libraries as an alternative to mathematical languages such as MATLAB (mathworks.com) and R (r-project.org)
  - boost uBLAS (boost.org)
  - Automatically Tuned Algebra Software (ATLAS) for fast matrix manipulation (math-atlas.sourceforge.net/)
set of interfaces to allow users to interact via EEG
- alphabet selection
- cursor movement
- cursor movement/graphical keyboard
plug-in classification algorithms for easy experimentation
- this will allow quick analysis of different learning algorithms and comparison between them
black-box model for expandable features of the gui such as learning algorithms and interfaces
- the GUI will behave the same no matter what sort of algorithm/EEG interface is being used
- accomplished with a set of generic c++ class interfaces
Future Work:
- client/server interface to BCI
  - simple games across the Internet
  - control wireless robots with EEG
- Incremental SVD
- interface to R to allow algorithms to be implemented outside C++
- support for multiple EEG amplifiers
  - right now the only driver is for the Mindset 24
- support for multiple platforms
  - EEG amplifier driver only for unix/linux
    - will require some work to write for Microsoft Windows
  - GUI only for unix/linux
    - should be fairly simple to port to Microsoft Windows

Go to the top of this page.

News

[April 16 2009] CEBL 2.0 released. Site hasn't been updated for a year and a half so I thought I would work on it. CEBL has been re-designed from the ground up and is now released as cebl 2.0 to be more modular and separated between view and model. All code is now separated between gui and model pieces so it is easy to write a new interface to the system. I've scrapped automake and moved to waf which is a wonderful and simple python-based build system. Training data now contains lots of information about the data and recording session. Training, data collection, real time classification, extracting features, and training classifiers are all threaded so CEBL no longer locks while performing these tasks. Added ability to inturrupt long actions. Re-designed plotting to make all plots overlap, and easier to view. Full list of features listed in section about cebl 2.0.

[November 30 2007] Pie menu now uses bars instead of the cursor when classifying. Made interfaces detachable by double-clicking them. Added configuration menu for interfaces. Added required function to features which is called each time a class is selected.

[October 17 2007] Lots of things over the past few months: Added controls to EEG device tab to set the block size and sample rate for recording. Fixed feature applying. Added several functions to cppR, and cleaned up this library separating it into several header files. Expanded on cppR tutorial on website. Added panel to enter class labels to pie menu in the train tab. Added several changes to the session files which weren't being saved before. After alot of trying I finally gave up on getting boost::serialization working properly on dynamically loaded shared libraries. Although serialization in features, classifiers, and filters does work, it is somewhat limited and not exactly how it is meant to work. As it is you have to write individual serialization functions for any data type you wish to save in classifiers or features (including STL data types). Primitives should save to archive with no problem. Tryed to get Ratoc u2scx SCSI to USB converter to work in linux with no luck. Although I did manage to get the Adaptec usb2xchange to work using a kernel patch and firmware I found on the internet, we decided that it would be more convenient to have an adapter which didn't require a kernel patch. In any case we plan on sticking to the pcmcia scsi cards to connect to the EEG Amplifier. Spent a little time making a debian package for the project. CEBL now has a darcs repository read-accessible to the world. See the download section.

[August 6 2007] New: MNF filter actually works now, added more functions to cppR.hpp and fixed some old ones, LDA and QDA classifiers added by Dr. Anderson, combined channels and data process tab, removed data source tab, added makefile for all shared libraries, renamed channels to EEG Recording, and Mindset to EEG Device, created a darcs repository, added scroll bar for channel selection. Also, usb interface to mindset works using adaptec usbxchange and a kernel patch.

[July 2 2007] New: filters are now loaded as plugins, cebl no longer requires lapack to compile, saving/loading session finished

[June 29 2007] New: added session saving and loading, added serialization to classifiers and features to make this possible, made the feature extractors load from shared libraries like the classifiers. Current Working on: making the gui compile without the need for lapack, this consists of removing the filters from the core and making them load as shared libraries as well because filters, feature extractors, and classifiers are the only pieces which need to use lapack routines. This will make compiling the gui easier. Also working on this site a bit more.

[June 22 2007] CEBL 0.4 released. New: Added saving of training data, classifiers are loaded as plugins, wrote tutorial on how to make a classifier, cebl searches for and creates a configuration directory as well as searching in a default system-wide directory, loads preferences (though there are no preferences yet, redesigned menu system using gtkuimanager, session saving and loading is mostly done, preferences window (with no preferences).

[May 23 2007] I'm learning GNU autotools. CEBL now has a configure script with a few tests to generate a makefile.

[May 21 2007] CEBL brought completely up to speed with the old R gui. Training and cursor movement on multi-class pie menu using MNF filter, Referencing, Lag feature extractor, and LDA classifier all working.

Go to the top of this page.

Download

Option 1: Get it from the SVN Repository:

You must have a CSU Computer Science login for this.
Check out repository to a target directory of cebl:

$ svn cosvn+ssh://USER@MACHINE.cs.colostate.edu/s/chopin/j/proj/eeg/svn/cebl/trunk cebl

Where USER is your username and MACHINE is a computer in the CS department (list of machines).
OR, if you are on the cs filesystem

$ svn co file:///s/chopin/j/proj/eeg/svn/cebl/trunk cebl

OR from Eclipse create a new project, select check out from svn and when it asks for a repository enter "svn+ssh://MACHINE.cs.colostate.edu/s/chopin/j/proj/eeg/svn/cebl/trunk".

Option 2: Nightly Tar of Repository

cebl_nightly.tar.gz

Go to the top of this page.

Installation

Dependencies

On the CSU Computer Science Department Machines

Two of libraries aren't on the CS machines. These are libfakekey and gtkextra. I have compiled these libraries locally in the CEBL directoy, but you need to set some environmental variables to make the configure and make scripts find these files. I have compiled both the 64 and 32 bit libraries. The script will determine which ones to use.

if you use a csh variant: > source eeg/CEBL/csh_environment

if you use bash: $ source eeg/CEBL/bash_environment

Compile

CEBL 2.x

$ cd cebl/cebl_2.x $ ./waf configure $ ./waf build $ ./waf install --plugins $ build/default/cebl

CEBL 1.x

$ cd cebl/cebl_1.x #or whatever directory you have the source in $ ./configure $ make #now you need to make the plugins. #the make install script will put the plugins in ~/.cebl/plugins $ cd src/plugins $ make && make install $ cd .. #now you can install it or run it from here $ su -c "make install" #or just run it $ ./cebl

If you are running KDE I highly suggest getting gtk-engines-qt in order to make gtk apps (such as cebl) look nice in KDE.

Go to the top of this page.

Documentation

Doxygen Documentation
How to create new classifiers, features extractors, and filters.

Go to the top of this page.

Versions

CEBL 2.0

Main differences from 1.x:

Uses the waf build system.
Separate code for internals of cebl and GTK gui.

CEBL 1.x

Old Projects

Before writing the GUI in C++ I wrote the same application in R (an open-source statistical language similar to MATLAB). I started this project in Fall 2004 as suggested by my mentor Dr. Charles Anderson. He wanted to transition to R from MATLAB (which he had used on this project for years) for many reasons including the lower cost of using open source software (R) instead of proprietary (MATLAB). I successfully wrote the classifiers, features and filtering algorithms in this system. However I ran into several problems in terms of the GUI: poor event handling between GTK bindings in R and the R event loop, slow animation in data plots and training interfaces, and most of all difficulty to install. In order to get a working event loop I used the R Event Loop package which is an excellent package in many respects, but is very outdated and somewhat glitchy. Furthermore, to use this package I had to use gtk+1.x bindings to R. Gtk+1.x is very outdated and most systems do not even include the libraries. These problems made this application difficult to install and generally clunky.

r_bci_gui-1.11.tar.bz2 - R version of the gui - almost a complete application, but a pain to get running. Requires R packages: RGtk, and REventLoop

In early Fall 2006 I realized that GTK1 was too outdated so I modified the R gui to work with R GTK2 bindings. Although the widgets worked I now had no way of running timed events for animation and live plotting.
I tried several methods to get some sort of event processing:

gtk timeout events - these did not work due to the nature of the R event loop. This problem had been handled with the R Event Loop package discussed above, unfortunately this package was designed for GTK1
modifying the R Event Loop for gtk2 - I worked at this for a few weeks, but I decided that although it was possible it would require too much delving into the R source code.
embedding R into a threaded C program - this worked well until it came to plotting. As soon as I tried telling R to plot from a separate C thread I ran into X11 threading problems. Clearly R is not particularly thread safe. This may have been possible to resolve, but in order to embed R in a C program the R source was required - I decided this also was too clunky.
embedding C programs into R - although this method would certainly work I decided that at this point I might as well transition completely to a C or C++ program.

r_bci_gui-2.03.tar.bz2 - this is the R GTK2 GUI at step 3 in the list above. Does not contain plotting or training due to the event loop problems mentioned above. Requires R 2.3.x source, and the RGTK2 package for R (available on r-project.org).

Although these two R GUIs did not turn out completely successfully they did make my task of creating CEBL much simpler.