Wednesday, September 17, 2008



In the first international meeting devoted to BCI research held in June 1999 at the Rensselaerville Institute near Albany, New York, it was defined as follows:

“A brain-computer interface is a communication system that does not depend on the brains normal output pathways of peripheral nerves and muscles”.

According to this definition, a BCI should be able to detect the user’s wishes and commands while the user remains silent and immobilized. In order to do this, the brain activity must be monitored. Today there exist various techniques to do this. These include, for example, functional Magnetic Resonance Imaging (fMRI), magneto-encephalography (MEG), Positron Emission Tomography (PET), and Single Photon Emission Computer Tomography (SPECT). optical brain imaging, single neuron recording (with microelectrodes) and electroencephalography (EEG).
From these methods, MEG, EEG and single neuron recording give continuous and instantaneous recordings of the brain activity (time resolution about 1 ms), which is required for real-time BCI. However, MEG is not practical to be used with BCI. Almost all of BCIs reported to date have been based on EEG.

Figure 2.1: A BCI based on the classification of two mental tasks. The user is thinking task number 2 and the BCI classifies it correctly and provides feedback in the form of cursor movement.

In the second approach the user has to learn to self-regulate his or her EEG response, for example change the -rhythm amplitude. Unlike in the pattern recognition approach, the BCI itself is not trained but it looks for particular changes (for example higher amplitude of a certain frequency) in the EEG signal. This requires usually a long training period, because all the training load is on the user. This kind of approach can be called an operant conditioning approach.

There are at least five components necessary for effective BCI system:
1) Knowing what to look for;
2) Knowing the relevant physiological signals;
3) Gathering the data from the user;
4) Extracting useful information from the raw signal;
5) Interface design.
Figure 2.1 shows a schematic picture of a BCI, which is based on pattern recognition approach. The BCI can classify two mental tasks and provides feedback in the form of cursor control.

2.1 The Human Brain:

The average human brain weights around 1400 grams. The brain can be divided into four structures: cerebral cortex, cerebellum, brain stem, hypothalamus and thalamus. The most relevant of them concerning BCIs is the cerebral cortex. The cerebral cortex can be divided into two hemispheres. The hemispheres are connected with each other via corpus callosum. Each hemisphere can be divided into four lobes. They are called frontal, parietal, occipital and temporal lobes. Cerebral cortex is responsible for many “higher order” functions like problem solving, language comprehension and processing of complex visual information.

The cerebral cortex can be divided into several areas, which are responsible of different functions. These areas can be seen in Figure 2.2. The functions are described in Table 2.1. These kinds of knowledge have been used with BCIs system based on the pattern recognition approach. The mental tasks are chosen in such a way that they activate different parts of the cerebral cortex.

2.2 Electroencephalography (EEG):
Electroencephalography (EEG) is a method used in measuring the electrical activity of the brain. This activity is generated by billions of nerve cells, called neurons. Each neuron is connected to thousands of other neurons. Some of the connections are excitatory while others are inhibitory. The signals from other neurons sum up in the receiving neuron. When this sum exceeds a certain potential level called a threshold, the neuron fires nerve impulse. The electrical activity of a single neuron cannot be measured with scalp EEG. However, EEG can measure the combined electrical activity of millions of neurons.
The electrical activity goes on continuously in every living human’s brain. We may sleep one third of our life times, but the brain never rests. Even when one is unconscious the brain remains active. Much of the time, the brain waves are irregular and no general pattern can be observed. All these means that an overwhelming majority of neuronal communication is practically invisible in EEG. However, there exist various properties in EEG, which can be used as a basis for a BCI:
1. Rhythmic brain activity
2. Event-Related Potentials (ERPs)
3. Event-Related De-synchronization (ERD) and event-related synchronization (ERS).

2.2.1 Rhythmic brain activity:
The EEG can be divided into several frequency ranges as displayed in Table 2.2. They are named after Greek letters ( , , , ). These ranges set the limits in which the different brain rhythms (named according to same letter as the frequency range) can be observed. The order of the letters is not logical and can be understood only in the historical view. Figure 2.3 illustrates examples of the brain rhythms. These rhythms (alpha, beta, delta and theta) are explained later in this section. One of them is the mu rhythm. It is also included in this section, because it has significance in BCI research.
Alpha rhythm: Amplitude is variable but is mostly below 50 _V in adults. Best seen with eyes closed and under conditions of physical relaxation and relative mental inactivity. Blocked or attenuated by attention, especially visual, and mental effort.
The alpha rhythm is temporarily blocked, i.e., its amplitude decreased, by eye opening, other afferent stimuli or mental activities. The degree of reactivity varies.
Mu rhythm: Mu rhythm frequency is around 10 Hz and amplitude mostly below 50 V. Mu stands for motor and the mu rhythm is strongly related to the functions of the motor cortex, but also to the adjacent somatosensory cortex. The mu rhythm is blocked by movements or light tactile stimuli. The fact that the thoughts about performing movements and readiness to move can also block the mu rhythm, have made it important in BCI research.
Beta rhythms: Any rhythmical activity in the frequency band of 13-30 Hz may be regarded as a beta rhythm. Beta rhythm amplitudes are seldom larger than 30 V.

BCIs based on the rhythmic activity: Many BCI researches have considered about using the imagination of hand or foot movements as the basis of the BCI. Therefore, the mu rhythm plays an essential role in them. According to studied the use of the mu rhythm in BCI and concluded that “mu rhythm is not only modulated by the expression of self-generated movement but also by the observation and imagination of movement.”. However, in EEG biofeedback, self-regulation of, for example, alpha or beta rhythms, has been used extensively.

2.2.2 Event-related De-synchronization (ERD) and Event-related Synchronization (ERS):
Event-related de-synchronization (ERD) and event-related synchronization (ERS) can be defined as follows:
1. Event-related de-synchronization (ERD) is an amplitude attenuation of a certain EEG rhythm.
2. Event-related synchronization (ERS) is an amplitude enhancement of a certain EEG rhythm.
In order to measure an ERD or an ERS, the power of a certain frequency band (for example, 8-12 Hz) is calculated before and after certain “event” over a number of EEG trials. The event can be externally-paced (such as light stimulus) or internally paced (such as voluntary finger movement). The power (averaged over a number of trials) is then measured in percentage relative to the power of the reference interval. The reference interval is defined, for example, as 1 second interval between 4.5 and 3.5 seconds before the event. The ERS is the power increase (in percents) and the ERD is the power decrease relative to the reference interval (which is defined as 100 %). To keep the power at the reference interval at the resting level, the interval between two consecutive events should be random and not shorter than a few seconds.
BCI based on ERD and ERS:
The Graz BCI is based on detecting ERD and ERS of the different and rhythm bands during the imagined left and right hand movements.

2.3 Two different BCI approaches:
What are the thoughts the user thinks in order to control a BCI? An ideal BCI could detect the user’s wishes and commands directly. However, this is not possible with today’s technology. Therefore, BCI researches have used the knowledge they have had of the human brain and the EEG in order to design a BCI. There are basically two different approaches that have been used. The first one called a pattern recognition approach is based on cognitive mental tasks. The second one called an operant conditioning approach is based on the self-regulation of the EEG response.

2.3.1 Pattern recognition approach based on mental tasks:
BCIs including the ABI are based on different mental tasks. These tasks should activate different cortical areas and produce different EEG rhythms. This approach can be called the pattern recognition approach. The BCIs based on the pattern recognition approach include the ABI. The mental tasks used in BCIs have included motor imagery, visual, arithmetic and baseline task. In order to produce different EEG patterns, mental tasks should activate different parts of the brain. Therefore, the knowledge of cortical areas and their function has been used when choosing the mental tasks.
2.3.2 Operant conditioning approach based on self-regulation of EEG:
A couple of BCI research groups have based their BCIs on the self-regulation of one of these rhythms or potentials. This approach can be called the operant conditioning approach have based their BCI called a Thought Translation Device (TTD) on the self-regulation of the SCPs There are three elements important for successfully learning to self-regulate the EEG response:
1. Real-time feedback of the specific EEG activity
2. Positive reinforcement of correct behavior
3. Individual shaping schedule in which progressively more demanding tasks are rewarded.

2.4 Measuring EEG:
In the scalp EEG the electrical activity of the brain is recorded non-invasively, i.e. from the surface of the scalp using normally small metal plate electrodes. While the number of the electrodes varies from study to study, they are usually arranged according to an international 10-20 system. Recordings can be made either using reference electrode(s) or bipolar linkages. The EEG signal can be affected by many artifacts coming from the equipment or the subject.
2.4.1 Electrodes:
The EEG is recorded with electrodes, which are placed on the scalp. Electrodes are small plates, which conduct electricity. They provide the electrical contact between the skin and the EEG recording apparatus by transforming the ionic current on the skin to the electrical current in the wires. Electrolyte gel is applied between the electrode and the skin in order to provide good electrical contact. Usually small metal-plate electrodes are used in the EEG recording.
2.4.2 Electrode placements:
In order to make patient’s records comparable over time and to other patient’s records, a specific system of electrode placement called International 10-20 system is used. The system is for 21 electrodes. Each electrode position has a letter (to identify the underlying brain lobe) and a number or another letter to identify the hemisphere location. Odd numbers are on the left side and even on the right side. Z (for zero) refers to electrode placements at midline.
2.4.3 Reference and bipolar recordings:
The EEG recordings can be divided into two major categories: Reference recordings and scalp-to-scalp bipolar linkages. In the reference recording each electrode is referred to either distant reference electrode. The reference electrode(s) must be placed on the parts of the body where potential remains fairly constant. In bipolar recordings differential measurements are made between successive pairs of electrodes.
2.4.4 Artifacts:
When measuring the EEG, all of the signals do not come from the electrical activity of the brain. Many potential changes seen in the EEG may be from other sources. These changes are called artifacts and their sources may be the equipment or the subject. These artifacts include:
Technical artifacts:
– Mains interruption. The surrounding electrical equipment may induce 50-Hz or 60-Hz component in the signal.
– Electrode artifacts. If electrodes are improperly attached or in poor condition, their impedances may vary.
Physiological artifacts:
– Motion artifacts. Subject’s movements cause electrodes or electrode cables to
– EMG artifacts. The tension of muscles (especially masticatory, neck and forehead muscles) causes EMG artifacts.
– Cardiac artifacts. The heart causes many different artifacts: ECG, pulsation
artifact, ballistocardiographic artifact, pacer artifact, respiration artifact
– Oculographic artifacts. These include the eye blink artifact and the eye movement artifact.
– Sweating. This can affect, for example, the impedances of the electrodes.

2.5 BCI components:
A typical BCI device consists of several components. These include electrode cap, EEG amplifiers, computer and subject’s screen. A critical issue is how the user’s commands, i.e., the changes in the EEG, are converted to actions on the feedback screen or the application.
This process can be divided into five stages:
1) Measurement of EEG: This is done by using the electrodes. Many BCIs use a special electrode cap, in which the electrodes are already in the right places, typically according to the international 10-20 system.
2) Preprocessing: This includes amplification, initial filtering of EEG signal and possible artifact removal. Also A/D conversion is made, i.e. the analog EEG signal is digitized.
3) Feature extraction: In this stage, certain features are extracted from the preprocessed and digitized EEG signal. In the simplest form a certain frequency range is selected and the amplitude relative to some reference level measured.
4) Classification: Different BCIs can classify different number of classes, typically 2 to 5 classes. The classifier can be anything from a simple linear model to a complex nonlinear neural network that can be trained to recognize different mental tasks.
5) Device control: The classifier’s output is the input for the device control. The device control simply transforms the classification to a particular action. The action can be, e.g., an up or down movement of a cursor on the feedback screen or a selection of a letter in a writing application. However, if the classification was “nothing” or “reject”, no action is performed, although the user may be informed about the rejection.

2.6 Feedback:
Feedback is an important factor in BCIs. In the BCIs based on the operant conditioning approach, feedback training is essential for the user to acquire the control of his or her EEG response. However, feedback can speed up the learning process and improve performance. Cursor control has been the most popular type of feedback in BCIs.
2.6.1 Biofeedback in general:
Biofeedback can be defined as follows: “Biofeedback is the process in which a subject receives information about his biological state. Usually a subject is not aware of his physiological functions, especially those controlled by the autonomic nervous system.
The most popular types of biofeedback machines or techniques include: Electroencephalography (EEG), electromyography (EMG), skin temperature and galvanic skin response (GSR).

2.6.2 EEG biofeedback:
The basic idea in the EEG biofeedback is the operant conditioning of certain EEG parameters. Typically, the goal of the training is to increase the activity on a certain frequency band and decrease it in another. This is possible by providing feedback for the subject. The feedback can be, for example, a car in the computer game. The speed of the car can be coupled with the desired condition. The car moves faster, if the patient’s EEG gets closer to the desired condition and slower, if it gets farther.
Biofeedback in the form of games is especially important with children. EEG biofeedback requires attention and the session typically lasts around 30 min. Usually EEG biofeedback treatment requires tens of sessions. For this reason it is necessary to provide children with interesting feedback in order to keep them engaged in the treatment.
2.6.3 Feedback in BCIs:
In most BCIs some kind of feedback is provided to the user. The most popular form of feedback has been the cursor control We chose cursor movement because it is objective, easily implemented, simple for the user to learn, and can serve as a prototype for control of a wide variety of applications.
The cursor control is an example of continuous feedback. However, the classifications in BCIs are made in discrete manner. In order to make cursor movement to look continuous. Ideally continuous feedback would be instantaneous, i.e., real-time. Beside continuous, feedback can also be discrete. ABI uses this kind of feedback by presenting each mental task as a colored ball. The ball lights up when the EEG sample is classified as belonging to a corresponding task with Graz BCI.

2.6.4 Effect of feedback:
In BCIs using the operant conditioning approach (see section 2.3.2), the feedback about the performance is essential in skill development, i.e., in acquiring control over the EEG response.
Beneficial effects:
1. Furnishes continual motivation
2. Ensures attention to the task by maintaining the subject’s interest
3. Improves performance by allowing rapid reaction to wrong classifications
Harmful effects:
1. The feedback stimulus might prevent concentration on internal states
2. The false classifications can elicit frustration and thus affect the EEG response (for example, cause EEG de-synchronization)
3. The correct classifications might lead to anticipation and thus affect the EEG response (for example, cause EEG synchronization)
4. The visual feedback stimulus might affect the alpha rhythm
The kinds of short-term effects the removal of cursor movement

2.7 Human training issues:
To date, most of the BCI research has concentrated mainly on technical issues; how to measure, process and classify the EEG signal better and better. However, the producer of this EEG signal, the human being, may be as important or even a more important factor in a successful BCI than the technical developments. Therefore, the issues concerning the human training are worth considering.
2.7.1 Training protocol:
The protocols vary from one BCI to another. Typically, however, training is divided into series of sessions and each session is divided into a certain number of trials. One session typically consists of tens of trials and lasts 5-30 minutes. Using a BCI requires so much concentration that usually half an hour to one hour of training is enough for one day.

2.8 BCI performance:
To this day, none of the BCIs have achieved the communication speed or the accuracy of the other interfaces. In addition, there is a problem of evaluating BCI performance. The results are reported differently from one BCI paper to another. This makes it difficult to compare different BCIs.
2.8.1 Measurements of accuracy in BCIs:
There have been many different measurements. One method of reporting accuracy is to give a correct classification rate. More comprehensive way to report accuracy is to present a confusion matrix. The confusion matrix tells not only the correct classification rates of each class, but also in which classes the false classifications were classified

2.8.2 Bit rate and channel capacity:
An information transfer rate, a bit rate, can be used in order to take into account both accuracy and speed of a BCI. The bit rate is a standard measure of any communication system (which a BCI basically is). It tells the amount of information communicated per time unit. The highest bit rate a noisy communication system can theoretically have is called a channel capacity The channel capacity can be calculated in a closed form if the following conditions are:
1) Classes have equal classification rates;
2) Errors are distributed symmetrically;
3) The rejection is not used.

2.9 Categorizing BCIs:
Invasive and non-invasive BCIs: Non-invasive BCIs are based on EEG measured with the scalp electrodes. In invasive BCIs, the of the brain is recorded from inside the head (e.g., from the cerebral cortex). The recordings are made, for example, with one or more microelectrodes.
Synchronous and asynchronous BCIs: In an asynchronous mode, the brain activity is analyzed continuously.
Universal and individual BCIs: Universal BCI relies on assumption that by gathering EEG data from few users it is possible to find a classification function that should be valid for everybody. In individual BCI the fact that no two people are the same, both physiologically and psychologically is taken into account. Therefore, the BCI is different with different users.
Online BCIs are the actual working BCIs. The signal processing, features extraction, classification, and device control (see 2.5) are done in real-time. The performance of the offline BCI can be evaluated using, for example, cross-validation test. The results are comparable with the same kind of online BCI (without biofeedback), if all the recorded EEG data are used.
Imagery and mental tasks: From the user’s point of view, BCIs can be categorized according to what kind of imagery they require. Motor imagery has been used in many BCIs.