Wednesday, September 17, 2008



This chapter describes different BCI systems. After the BCIs are introduced they are compared to each other.

3.1 Different approaches to BCI:
This section provides an overview of five BCI systems based on the scalp EEG. Three of them are based on the pattern recognition approach like ABI. The other two use the operant conditioning approach.
3.1.1 BCI research at the University of Alberta:
Alexandar Kostov and Mark Polak started their BCI research at the University of Alberta, Canada. Their BCI system was based on the pattern recognition approach. In the study reported in EEG data was recorded with 28 electrodes arranged according to the international 10-20 electrode system. Signal amplification and the initial filtering were done by Brain Imager, a device manufactured by Neuroscience Inc.
The EEG signals were digitized at a sampling rate of 200 Hz. The EEG patterns were classified with an adaptive neural network called Adaptive Logic Network (ALN) in on-line experiments. Training was done in 30-min sessions. The subject was seated in front of the feedback monitor while the EEG signals were recorded. Feedback was provided in the form of the cursor control.
The first half of the training session was used to train the new ALN classifier as the subject was attempting to move the cursor towards the target. When the subject achieved control of the cursor, the training of the ALN was halted and the second half of the session was used to evaluate the performance
The goal of this project was to develop a BCI capable of accurate two-dimensional (2-D) cursor control. In addition, the intention was to develop range of applications for the BCI. One application reported to date has been an environmental control device.

3.1.2 BCI research at the Oxford University:
William Penny and Stephen Roberts started BCI research at the Oxford University. The EEG was recorded from one bipolar channel with two electrodes located 3 cm behind C3 and C4 of the international 10-20 system. The ground electrode was placed on the right mastoid. The signal was band-pass filtered with 3 dB points set at 0.1 Hz and 100 Hz and digitized at 384 Hz. The EEG data was analyzed using the 8th order autoregressive (AR) model. This model was fitted to 1/3 second blocks of data (128 samples) which slid 32 samples (1/12 second) from one processing time step to next.
Penny and Roberts experimented with two methods in order to improve the performance of their BCI. The methods were a latent-space smoothing and a reject option. The latent-space smoothing means that the low certainty decisions may be rejected or “over-ruled” by the higher certainty decisions from the recent past. A two second window was used. In the reject option a third class called “reject” was added. Using Bayesian decision theory the certainty of the classification was calculated. If the certainty of the classification did not exceed a particular threshold then the EEG was classified to “reject” class. Penny and Roberts reported the classification results for three scenarios: hard rejection, soft rejection and baseline (no rejection). The scenarios are explained as follows
Hard rejection:.The latent-space smoothing and reject option was used. If, however, more than 50 % of an experimental block was rejected then the entire block was removed from the data set as a “corrupted” data epoch.
Soft rejection: The latent-space smoothing and the reject option was once more applied but no removal of experimental blocks was performed.
Baseline: No smoothing or rejection was performed and classification was made on a sample-by-sample (each 1/12 second) basis.
When the hard rejection scenario was used, 21 % of the data blocks were entirely rejected and of the remaining data samples an average of 28% were rejected. Using the soft rejection method, an average of 34 % of the data samples was rejected.
3.1.3 BCI research at the Wadsworth Center:
Jonathan Wolpaw and his colleagues have done BCI research at the Wadsworth Center, the United States Their BCI is based on the self-regulation of the 8-12 Hz _ or the 13-28 Hz _ rhythms.64 EEG channels were recorded from the surface of the scalp from 4 subjects (one with ALS). Each channel was referenced to the electrode in the right ear. 62 of 64 channels were digitized at 128 Hz and stored for later evaluation. Two remaining channels located over each hemisphere of the sensimotor cortex (e.g., C3 and C4) were digitized at 196 Hz. They were converted to either a common average reference (CAR) derivation or a large Laplacian derivation.
The feature extraction and the classification were done as follows. The EEG data was analyzed using the autoregressive (AR) algorithm and amplitude (i.e., the square root of power) was calculated in a 3-Hz wide frequency band. The frequency band corresponded to 8-12 Hz _ rhythm or 18-24 Hz central _ rhythm (2 subjects). The sum of the amplitudes from the two channels was calculated every 100 ms using the preceding 200 ms segment of the EEG data. This sum was the independent variable in a linear equation that determined a cursor movement.
The training was done in 30 min sessions divided into 8 runs lasting 3 minutes each and separated by 1 minute rest periods. Each run consisted of several trials. In each trial the user tried to move the cursor from the center of the screen to the target located at the top or bottom of the screen. The distance to top or bottom was 94 cursor steps. The cursor moved every 100 ms up or down according to the linear equation described above.
3.1.4 The Thought Translation Device (TTD):
During 1990’s Birbaumer and his colleagues developed a BCI called the Thought Translation Device (TTD) at the University of T˝ubingen in Germany. Over the years the TTD has been used by 12 ALS or other patients with severe or total paralysis. Birbaumer and coworkers studied five patients using the TTD. The EEG was recorded from the electrode Cz referred to mastoids at a sampling rate of 256 Hz. The EEG signal was filtered and corrected for the eye movement artifacts. SCPs were then extracted from the EEG signal. The training day usually consisted of 6-12 sessions, each of them consisting of 70-100 trials and lasting 5-10 minutes. The patients were trained several times a week.
When the subject achieved stable performance of 75 % correct trials, he or she can begin to work with a language support program. In the language support program the alphabet was split into two halves (letter-banks). These letter-banks were shown successively at the bottom of the screen. The subject could choose the letter bank shown by producing a SCP shift (either SCP negativity or positivity according to subject’s preference). If the subject produced the required SCP shift the letter bank was split into two new halves. This continued until each of the letter-banks contained only one letter. When the subject selected one of them, the selected letter was displayed in the top text field of the screen and a new selection began from the start.
The program also included a “return function”. If the subjects rejected two successive letter-banks the option to erase the last symbol in the text field appeared. Beside the language support program the TTD has two other applications: The environment control unit and the Internet browser “Descartes”. It will try to combine the slow cortical capacity of the TTD with the _ and _ rhythm capacity of the Wadsworth BCI.
3.1.5 Graz brain-computer interface:
Pfurtscheller and his group in the Graz University of Technology, in Austria, started the “Graz Brain-Computer Interface”. The Graz BCI has moved through various stages of prototypes. However, all this time it has been based on the detection of the ERD and the ERS patterns during the motor imagery. One aim of
the research was to study how the number of mental tasks affected the channel capacity. Classification was done offline.
The EEG was recorded with 29 gold electrodes (see Figure 3.3). The ground electrode was placed on the forehead. The EEG signals were filtered between 0.5 Hz and 30 Hz and digitized at the sampling rate of 256 Hz. EMG and EOG artifacts were excluded from the data sets. The logarithm of the band power for five bands (7-10 Hz, 10-13 Hz, 16-20 Hz, 20-24 40Hz, 24-30 Hz) was calculated for every channel using a fifth-order Butterworth filter in a window from seconds 4 to 8 of each trial. This formed a feature vector consisting of 145 components describing all EEG signals from all electrodes. A subset of features was calculated using step-by-step procedure. A hidden Markov model (HMM) was used as a classification method. Classification accuracy was evaluated using 5-fold cross-validation test.

Figure 3.3: Positions of the 29 electrodes used in the Graz BCI. The 17 electrode
positions with bold circles belong to the international 10-20 system. The rest
twelve electrodes were inserted in between, in order to increase spatial resolution

3.2 The Adaptive Brain Interface (ABI):
The BCI system used in the experimental part of this work is called Adaptive Brain Interface (ABI). The ABI has been developed under the project “Adaptive Brain Interfaces” financed by European Commission.
In this section the older version of the ABI is described, whereas a new ABI version was used in the experiments. The ABI is based on the mutual learning process where the system and the user adapt to each other. The system learns to classify each user’s individual EEG patterns generated during the mental tasks. This is made possible by neural network classifier which learns these user-specific patterns. The other part of the learning process, the user, learns to undertake the mental tasks in a way that the system recognizes them better. The user may choose the mental tasks (see the section 3.2.2) he or she uses and the strategies to undertake those mental tasks (e.g., thinking of moving a finger, the hand or the whole arm). The learning process can be enhanced with feedback.

3.2.1 Overview of the system:
The ABI system can be divided to components according to Figure 3.4. Each component described below in more detail according to.

Figure 3.4: The key components of the ABI system.

Acquisition system: The portable EEG system has 8 scalp electrodes. They are placed on F3, F4, C3, Cz, C4, P3, Pz and P4 according to the international 10-20 system (see Figure 3.5). The sampling rate of the system is 128 Hz. A surface Laplacian (SL) is estimated locally over the six electrodes (C3, Cz, C4, P3, Pz and P4) by using a finite difference method in which the mean activity of the neighboring electrodes is subtracted for each position of interest. Then the signal is bandpass filtered with a second order 4-45 Hz Butterworth filter. In addition, the signals are referred to a baseline, which is the average of initial resting period. This period lasts 1 minute and the users are instructed to remain in resting state (eyes open). This baselining is done, because the brain activity is not stable over time.

Figure 3.5: The electrodes used in the ABI are painted gray in this picture.

Feature extraction: The extracted features are power spectrum density components in the frequency band of 8-30 Hz within 1/2 second EEG segments averaged from 1 second sequences. The overlapping between the segments is 50%. As a result each EEG sample is represented by 72 features (6 channels times 12 components each). Thus, an EEG sample is computed every 1/2 second. Feature classification: Classification is done using a classifier called local neural classifier. In this classifier, every mental task (class) is presented by a prototype in a high dimensional input space. The aim is to find the appropriate position of the prototypes in this space to differentiate the classes. Therefore, during training, the prototypes are pulled toward the EEG samples of the mental task they present and pushed away from the EEG samples of other tasks.
Biofeedback: Biofeedback is provided in the form of colored buttons. If, for example, three mental tasks are used, three colored buttons are displayed each representing one particular mental task. The subject performs the mental tasks spontaneously and a button lights up if the arriving EEG data is classified to a corresponding mental task (see Figure3.6).

3.2.3 Training:
The first training session with the new subject is done offline. The EEG data recorded in this session is used to train the first individual neural classifier. Recording is done as follows.
The subject is instructed to remain in a resting state the first 60 seconds of the recording. In the resting state the subject keeps his or her eyes opened but do not undertake any particular task. The average resting pattern is computed over this initial period and used as a baseline for all the other tasks. The neural classifier trained at the first session is embedded in the BCI and used in the second session. From this session onwards training can be done online and biofeedback can be used to enhance the learning process. Otherwise the training protocol is the same with or without biofeedback. The neural classifier can be tuned with the EEG data recorded in the second session. This classifier can then be used in the third session and so on. After the subject reaches the desirable level of performance, training is halted and the subject can start using the applications

3.2.5 Applications:
Today, there exist two applications: the Virtual keyboard and the Pacman game. ABI has also been used to control a robot. The user controls Pacman with two commands to make it turn left or right. Pacman stops when it reaches the wall. The goal of the game is 47 to collect all the dots (euro coins in this case) from the maze. There are no ghosts in this version of the Pacman game.
The Virtual keyboard works as follows. The keyboard is first split into three areas each containing 9 letters The areas are indicated by colored frames. Each color is configured to one of the three mental tasks used. Classifications are made every half a second, as usual, and the flashing of the colored area gives the feedback for the user. The Graz BCI has been used to control a prosthetic arm. The Alberta BCI has been used as the environmental control device. The Wadsworth BCI has been used to answer simple YES and NO questions. The ABI has the Virtual keyboard and Pacman applications.
3.3.9 Summary and discussion:
1) Accuracy: Accuracy is maybe the most important aspect in any BCI. As described in the section 2.8, the accuracy affects greatly the channel capacity, and thus, the 58 performance of a BCI. If a BCI is to be used in the control applications (environmental control, hand prosthesis, wheel-chair, etc.), the accuracy is crucial.
2) Speed: Beside accuracy, speed is also very important when considering using a BCI for communication the speed of a particular BCI is affected by the trial length, i.e., the time needed for one selection. Typically, one trial lasts many seconds. This time should be shortened in order to make a BCI effective in communication.
3) Usability: The preparation for the use of a BCI takes time, because of the EEG measurement. Ideally, the user, even a disabled one, could use a BCI independently after the EEG cap or electrodes have been put on. However, an operator is normally needed and one cannot use a BCI independently
4) Feedback: The most common type of feedback has been the cursor control.
5) Asynchronous use: At least four of the six BCIs (Alberta, Graz, Wadsworth and TTD) cannot currently be used in asynchronous mode.
Using a BCI for controlling cursor:
Considering that many BCIs have used the cursor control as feedback, those interfaces are still far a way from an ordinary mouse. A mouse enables us to do several things which seem not possible with today’s BCI technology. First, using the ordinary mouse, one can move the cursor to any direction in 2-dimensional space. Today’s BCIs can provide only 2 directions with reasonable accuracy.