Asymmetric EEG-based coding and decoding method for brain-computer interfaces

Abstract

The present invention provides an asymmetric EEG-based coding and decoding methods for BCIs, the BCI system includes an evoked stimulus module, an acquisition module and an EEG signal data set including a training set X.sub.k and a testing sample Y, and an EEG signal decoding module; the evoked stimulus module sends a hybrid coding visual stimulus to subjects to evoke a specific EEG signal as required; the acquisition module obtains data information by amplifying and filtering the EEG signal so as to constitute EEG signal module; the decoding module coverts the data information into an instruction set for outputting the coding method of the present invention uses asymmetric characteristics of brain electrophysiological activity response to stimulus, combines with coding strategies such as SDMA, CDMA, FDMA and phase division multiple access coding.

Claims

1. An asymmetric EEG-based coding method for BCIs, comprising the following steps of: Step 1: performing hybrid coding comprising the SDMA coding, CDMA coding, FDMA, and phase division multiple access coding, and constructing an evoked stimulus module in a BCI system; Step 2: sending, by the evoked stimulus module, a hybrid coding visual stimulus to subjects to evoke a specific EEG signal as required; Step 3: amplifying and filtering the EEG signal by an acquisition module so as to obtain data information; and Step 4: converting, by a decoding module, the data information into an instruction set for outputting.

2. The coding method of claim 1, wherein the hybrid coding generated by the evoked stimulus module comprises at least any two combinations of SDMA coding, CDMA coding, FDMA, and phase division multiple access coding.

3. The coding method of claim 2, wherein the hybrid coding generated by the evoked stimulus module comprises SDMA coding, CDMA coding, FDMA, and phase division multiple access coding.

4. An asymmetric EEG-based decoding method for BCIs, comprising the following steps of: Step 1: constructing an EEG signal data set including a training set X.sub.k and a testing sample Y based on the BCI system; Step 2: performing frequency domain filtering and downsampling data processing to the testing sample Y; Step 3: based on Fisher's linear discriminant criterion, calculating the training set X.sub.k to obtain a projection matrix W; Step 4: performing spatial filtering by using DSP algorithm to the training set X.sub.k and the testing sample Y to obtain eigenvector W.sup.T{circumflex over (X)}.sub.k and W.sup.TY according to the equations (5), (6); $\begin{array}{cc}{S}_w^{-1}{S}_B*W=\left[\begin{array}{ccc}{\lambda }_1& & \\ & \ddots & \\ & & {\lambda }_{N_c}\end{array}\right]*W& \left(5\right)\\ S_D={\Sigma }_{11}+{\Sigma }_{22}-{\Sigma }_{12}-{\Sigma }_{21}{S}_w={\sigma }_1^2+{\sigma }_2^2& \left(6\right)\end{array}$ Step 5: based on the eigenvector W.sup.T{circumflex over (X)}.sub.k and W.sup.TY, performing spatial filtering by using CCA algorithm to construct two projection matrixes U.sub.k and V.sub.k by equation (8); $\begin{array}{cc}\mathrm{CCA}\left(W^T{\hat{X}}_k,W^{T}Y\right)-\underset{U_k,V_k}{\max }\frac{\mathrm{.Math.}\left[U_k^T{W}^T{\hat{X}}_k{Y}^T{\mathrm{WV}}_k\right]}{\sqrt{\mathrm{.Math.}\left[U_k^T{W}^T{\hat{X}}_k{\hat{X}}_k^T{\mathrm{WU}}_k\right].Math.\delta \left[V_k^T{W}^T{\mathrm{YY}}^T{\mathrm{WV}}_k\right]}}& \left(8\right)\end{array}$ Step 6: based on the eigenvector W.sup.T{circumflex over (X)}.sub.k, W.sup.TY and, the projection matrixes U.sub.k and V.sub.k, performing pattern matching to obtain an eigenvector ρ.sub.k by equation (9); $\begin{array}{cc}{\rho }_k=\left[\begin{array}{c}{\rho }_{k1}\\ {\rho }_{k2}\\ {\rho }_{k3}\\ {\rho }_{k4}\\ {\rho }_{k5}\end{array}\right]=\left[\begin{array}{c}\mathrm{corr}\left(W^T{\hat{X}}_k,W^{T}Y\right)\\ -\mathrm{dist}\left(W^T{\hat{X}}_k,W^{T}Y\right)\\ \mathrm{CCA}\left(W^T{\hat{X}}_k,W^{T}Y\right)\\ \mathrm{corr}\left(U_k^T{W}^T{\hat{X}}_k,U_k^T{W}^{T}Y\right)\\ \mathrm{corr}\left(V_k^T{W}^T{\hat{X}}_k,V_k^T{W}^{T}Y\right)\end{array}\right],k=1,2& \left(9\right)\end{array}$ and Step 7: recognizing the eigenvector ρ.sub.k by different classifier modules and then outputting them.

5. The decoding method of claim 4, wherein X.sub.k ∈ R.sup.N.sup.c.sup.×N.sup.t.sup.×N.sup.s are the training sets of pattern k=1, 2, Y ∈ R.sup.N.sup.c.sup.×N.sup.t is the testing sample, where N.sub.c is the number of channels in which EEG is collected, N.sub.t is the length of the intercepted signal, and N.sub.s is the number of samples in the training set.

6. An asymmetric EEG-based coding method for BCIs, comprising the following steps of: Step 1: constructing an evoked stimulus module in a BCI system; Step 2: sending, by the evoked stimulus module, a hybrid coding visual stimulus to subjects to evoke a specific EEG signal as required; Step 3: amplifying and filtering the EEG signal by an acquisition module so as to obtain data information; and Step 4: converting, by a decoding module, the data information into an instruction set for outputting, wherein the hybrid coding generated by the evoked stimulus module comprises SDMA coding, CDMA coding, FDMA, and phase division multiple access coding.

7. The coding method of claim 6, wherein the coding method is applied to an auditory evoked BCI.

8. The coding method of claim 6, wherein the coding method is applied to a somatosensory evoked BCI.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The accompanying drawings illustrate one or more embodiments of the present invention and, together with the written description, serve to explain the principles of the invention. Wherever possible, the same reference numbers are used throughout the drawings to refer to the same or like elements of an embodiment.

(2) FIG. 1 is a schematic diagram of a BCI system of the present invention;

(3) FIG. 2 is a schematic diagram of a visual division and stimuli presentation of the present invention; and

(4) FIG. 3 is a flow chart of a method for recognizing the asymmetric EEG feature of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

(5) The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. The embodiments of the present invention are intended to understand the present invention and are not intended to limit the invention.

(6) Brain lateralization is an important field in cognitive neuroscience. Due to the lateral effects of brain structure and function, asymmetry exists in different stimuli evoked EEG features, such as visual stimuli evoked asymmetric visual evoked potential, which reflects the asymmetry of neuronal activity between the two hemispheres of the brain.

(7) The present invention provides an asymmetric EEG-based evoking method for BCIs, and use the spatial contralateral domination to hybrid code the SDMA coding, CDMA coding, FDMA coding and phase division multiple access coding, effectively expanding the number of system instruction sets, which is expected to obtain considerable social and economic benefits. The present invention can be used in the fields of rehabilitation of disabled persons, electronic entertainment, industrial control, etc., which is expected to obtain a perfect BCI system in the future and is expected to obtain considerable social and economic benefits. The present invention is to achieved by the following steps:

(8) An asymmetric EEG-based coding method for BCIs, including the following steps of:

(9) Step 1: constructing an evoked stimulus module in the BCI system;

(10) Step 2: sending, by the evoked stimulus module, a hybrid coding visual stimulus to subjects to evoke a specific EEG signal as required;

(11) Step 3: amplifying and filtering the EEG signal by an acquisition module so as to obtain data information;

(12) Step 4: converting, by a decoding module, the data information into an instruction set for outputting.

(13) The hybrid coding generated by the evoked stimulus module includes at least any two combinations of SDMA coding, CDMA coding, FDMA coding, and phase division multiple access coding.

(14) The hybrid coding generated by the evoked stimulus module includes SDMA coding, CDMA coding, FDMA coding, and phase division multiple access coding.

(15) The SDMA coding (as shown in FIG. 1) uses spatial information for coding, wherein up/down/left/right, which are visual divisions, corresponds to four spatial information, respectively (the left and right visions are shown in FIG. 2); and the coding information increases as the spatial information increases.

(16) Based on SDMA scheme, the CDMA coding performs digital coding to the up/down/left/right information as code 0, 1, 2 and 3; and the coding information increases as the spatial information increases (Table 1 only shows the coding of left/right lateral spatial location).

(17) For the FDMA coding, different stimulation cycles will evoke different frequencies, such as a successive stimulation with 100 ms will evoke a background EEG frequency of 10 Hz; for the phase division multiple access: the different stimulation starting time will change the stimulation phase.

(18) As shown in Table 2, the hybrid coding generated by the evoked stimulus module includes SDMA coding, CDMA coding, FDMA coding, and phase division multiple access coding. The 2×2×5×2=40 hybrid coding scheme can be achieved as below: the SDMA is left/right coding; the CDMA is 0/1 two-digit CDMA coding, the FDMA includes five frequencies of 12 Hz (a 83.33 ms stimulus), 13 Hz (a 76.92 ms stimulus), 14 Hz (a 71.43 ms stimulus), 15 Hz (a 66.67 ms stimulus) and 16 Hz (a 62.5 ms stimulus); five frequencies and the phase division multiple access is 0°/90° coding.

(19) Taking the case of evoked visual asymmetrical EEG feature as an example, FIG. 1 is a schematic diagram of a BCI system of the present invention. The BCI system includes a LCD stimulation interface, an EEG electrode, an EEG acquisition system such as an EEG amplifier and a computer processing platform. The BCI system uses the method according to the present invention for stimulation, adopts an EEG digital acquisition system of Neuroscan Ltd. to collect the EEG signals. The signals are amplified and filtered by the EEG amplifier, and then input into a computer for calculation, and finally the EEG signals are decoded and converted into the BCI instruction for outputting. Stimulus presentation and data processing analysis are run in the Matlab platform environment.

(20) When using the system, subjects were asked to sit in front of the stimulation interface within a certain distance and focus on the center of the stimulation interface, as shown in FIG. 2, a sign of “+” is located in the center indicating the visual focus point, while the visual stimuli appeared at different positions (up/down/left/right position) will evoke different EEG signals. By using the spatial position characteristics of the EEG signal for coding, a SDMA coding scheme can be obtained. Taking a visual stimulus evoked asymmetric VEP (aVEP) as an example, visual stimuli appear in the subjects left and right vision, as shown in FIG. 2, and the aVEP EEG signal is evoked in the corresponding spatial position of the subject's brain, that is, when the stimulus appears on the left side, the right occipital area of the brain will induce more significant VEP characteristics, Meanwhile when the stimulus appears on the right side, the left occipital area of the brain will induce more significant VEP characteristics.

(21) The parameters such as stimulus shape and area can be adjusted according to different requirements. Taking FIG. 2 as an example, a white dot with diameter of 2 mm appears in FIG. 2, the stimulus generated in the left visual side is regarded as code “0”, the stimulus generated in the right visual side is regarded as code “1”. By using the “0/1” binary coding strategy and adding CDMA scheme according to visual stimuli sequence at different times, the space and code hybrid coding scheme of the present invention can be achieved. Taking the 4 characters “ABCD” as an example, the hybrid coding can be achieved by encoding “left/right” by SDMA scheme and encoding code “0/1” by CDMA, as shown in Table 1.

(22) TABLE-US-00001 TABLE 1 Space and code hybrid coding scheme Code sequence Character Spatial position of stimuli (two-digit) A First left stimulus, and 00 then left stimulus B First left stimulus, and 01 then right stimulus C First right stimulus, and 10 then left stimulus D First right stimulus, and 11 then right stimulus

(23) Different stimulation time will change the frequency evoked by the successive stimuli (e.g. FDMA coding), such as a successive stimulation with 100 ms will evoke a background EEG frequency of 10 Hz, a successive stimulation with 50 ms will evoke a background EEG frequency of 20 Hz, and different stimulation starting time will change the stimulation phase (e.g. phase division multiple access coding). Taking two frequencies (10/20 Hz) and two phase coding (0°/90°) as an example, the coding scheme of characters A to H, shown in table 2, can be achieved by encoding one-digit code “0/1” by CDMA.

(24) TABLE-US-00002 TABLE 2 Hybrid coding scheme based on SDMA, CDMA, FDMA and phase division multiple access coding Spatial position Code sequence Frequency Phase Character of stimulus (one-digit) (Hz) (°) A Left 0 10 0 B Left 0 10 90 C Left 0 20 0 D Left 0 20 90 E right 1 10 0 F right 1 10 90 G right 1 20 0 H right 1 20 90

(25) Comparing Table 1 and Table 2, it can be seen that the frequency and phase coding schemes can effectively expand the instruction set, and the number of instruction sets can be expanded by increasing the digit number of the CDMA. Furthermore, parameters such as the duration of stimulation and its duty ratio, repetition times of the stimuli sequence, interval time between two sequences can be adjusted according to actual requirements, and then the collected EEG signal is decoded so as to position the target character which is gazed by the subject. Take an example of achieving a stimulus paradigm of 40 instruction set, the hybrid coding scheme can be achieved by adopting the left/right SDMA coding; 0/1 two-digit CDMA coding, FDMA coding of five frequencies including 12 Hz (a 83.33 ms stimulus), 13 Hz (a 76.92 ms stimulus), 14 Hz (a 71.43 ms stimulus), 15 Hz (a 66.67 ms stimulus) and 16 Hz (a 62.5 ms stimulus). Table 2 shows a coding scheme, having two spatial positions, two-digit, two frequencies and two phase coding, can encode 40 characters. And if any one of the parameters is increased, such as spatial position, frequency and phase, the number of characters can be increased.

(26) An asymmetric EEG-based decoding method for BCIs, of the present invention including the following steps of:

(27) FIG. 1 can also represent a schematic diagram of a BCI system with a 32-instruction set of the present invention. The BCI system includes a LCD stimulation interface, an EEG electrode, an EEG acquisition system such as an EEG amplifier and a computer processing platform. The BCI system applies visual stimulus paradigm coding to evoke two patterns of asymmetric EEG features, adopts an EEG digital acquisition system of Neuroscan Ltd. to collect the EEG signals. The signals are amplified and filtered by the EEG amplifier, and then input into a computer for classifying the two EEG features, and finally the EEG signals are decoded and converted into the BCI instruction for outputting. Stimulus presentation and data processing analysis are run in the Matlab platform environment.

(28) The SNR of two asymmetric EEG signals are −17.98 dB and −14.90 dB, wherein SNR represents ratio of signal energy to noise energy, which could be estimated as:

(29) $\begin{array}{cc}\mathrm{SNR}=10\times {\log }_{10}\frac{{\left(\frac{1}{N}\underset{i=1}{\overset{N}{.Math.}}{\mathrm{AMP}}_i\right)}^2}{\frac{1}{N}\underset{i=1}{\overset{N}{.Math.}}{\mathrm{AMP}}_i^2-{\left(\frac{1}{N}\underset{i=1}{\overset{N}{.Math.}}{\mathrm{AMP}}_i\right)}^2}& \left(11\right)\end{array}$

(30) Where AMP.sub.i is the amplitude of the target potential in the i.sup.th trial, N is the number of trials.

(31) Twelve subjects are tested the BCI system of the present invention, and the experiment results demonstrate that after applying the present invention, the average classification accuracy rate of the 12 subjects is increased by 17.88%, the accuracy improvement was significant (the paired T-test results are: t.sub.11=−8.91, p<0.01), and the SNR of two patterns after the spatial filter by using the DSP algorithm was improved to −9.71 dB and −8.68 dB, respectively.

(32) As shown in FIG. 3, the present invention provides an asymmetric EEG-based decoding method for BCIs, including the following steps of:

(33) Step 101, constructing an EEG signal data set including a training set X.sub.k and a testing sample Y based on the BCI system;

(34) Suppose X.sub.k ∈ R.sup.N.sup.c.sup.×N.sup.t.sup.×N.sup.s are the training sets of pattern k=1, 2, Y ∈ R.sup.N.sup.c.sup.×N.sup.t is the testing sample, where N.sub.c is the number of channels in which EEG is collected, N.sub.t is the length of the intercepted signal, N.sub.s is the number of samples in the training set. They are both zero mean across time, that is, value s.sub.t at each time point minus the time average value s in the time window [t.sub.1, t.sub.2], which could be estimated by equation (1):
ŝ.sub.t=s.sub.t−s,t∈[t.sub.1,t.sub.2]  (1)

(35) The template of pattern k, written as {circumflex over (X)}.sub.k ∈ R.sup.N.sup.c.sup.×N.sup.t, is the average across training samples. The covariance matrix of

(36) $\left[\begin{array}{c}{\hat{X}}_1\\ {\hat{X}}_2\end{array}\right]\hspace{1em}$
is written as:

(37) $\begin{array}{cc}\Sigma =\left[\begin{array}{cc}{\Sigma }_{11}& {\Sigma }_{12}\\ {\Sigma }_{21}& {\Sigma }_{22}\end{array}\right]=\left[\begin{array}{cc}{\hat{X}}_1{\hat{X}}_1^T& {\hat{X}}_1{\hat{X}}_2^T\\ {\hat{X}}_2{\hat{X}}_1^T& {\hat{X}}_2{\hat{X}}_2^T\end{array}\right]& \left(2\right)\end{array}$
The variances of X.sub.1 and X.sub.2 are

(38) $\begin{array}{cc}{\sigma }_1^2=\frac{1}{N}\underset{i=1}{\overset{N_s}{.Math.}}\left(X_{1,i}-{\hat{X}}_1\right){\left(X_{1,i}-{\hat{X}}_1\right)}^T& \left(3\right)\\ {\sigma }_2^2=\frac{1}{N}\underset{i=1}{\overset{N_s}{.Math.}}\left(X_{2,i}-{\hat{X}}_2\right){\left(X_{2,i}-{\hat{X}}_2\right)}^T& \left(4\right)\end{array}$

(39) Step 102: performing frequency domain filtering and downsampling data processing to the testing sample Y ∈ R.sup.N.sup.c.sup.×N.sup.t selected from the EEG signal date set;

(40) Step 103: based on Fisher's linear discriminant criterion, calculating the training set X.sub.k to obtain a projection matrix W;

(41) Step 104: performing spatial filtering by using DSP algorithm to the training set X.sub.k and the testing sample Y to obtain eigenvector W.sup.T{circumflex over (X)}.sub.k and W.sup.TY according to the equations (5), (6);

(42) Based on Fisher's linear discriminant criterion, DSP finds a projection matrix W which could be regarded as a set of spatial filters to make the two patterns more discriminative after transformation. The matrix W can be used as a spatial filter and the solution algorithms are:

(43) $\begin{array}{cc}{S}_w^{-1}{S}_B*W=\left[\begin{array}{ccc}{\lambda }_1& & \\ & \ddots & \\ & & {\lambda }_{N_c}\end{array}\right]*W& \left(5\right)\\ S_D={\Sigma }_{11}+{\Sigma }_{22}-{\Sigma }_{12}-{\Sigma }_{21}& \left(6\right)\\ S_w={\sigma }_1^2+{\sigma }_2^2& \left(7\right)\end{array}$
Where λ.sub.i is the eigenvector of i.sup.th column of W, N.sub.w is the number of the selected spatial filters. After removing the common mode noise by W, the CCA algorithm is used to reveal the underlying correlation between W.sup.T{circumflex over (X)}.sub.k and W.sup.TY by finding two projection matrixes, U.sub.k, V.sub.k, which equals to solve CCA by equation (8).

(44) Step 105: based on the eigenvector W.sup.T{circumflex over (X)}.sub.k and W.sup.TY, performing spatial filtering by using CCA algorithm to construct two projection matrixes U.sub.k and V.sub.k by equation (8);

(45) $\begin{array}{cc}\mathrm{CCA}\left(W^T{\hat{X}}_k,W^{T}Y\right)-\underset{U_k,V_k}{\max }\frac{\mathrm{.Math.}\left[U_k^T{W}^T{\hat{X}}_k{Y}^T{\mathrm{WV}}_k\right]}{\sqrt{\mathrm{.Math.}\left[U_k^T{W}^T{\hat{X}}_k{\hat{X}}_k^T{\mathrm{WU}}_k\right].Math.\delta \left[V_k^T{W}^T{\mathrm{YY}}^T{\mathrm{WV}}_k\right]}}& \left(8\right)\end{array}$
where ε[.Math.] is the mathematical expectation. Canonical correlation analysis is a statistical analysis method that measures the linear correlation between two multidimensional variables. Different from linear regression, using straight lines to fit sample points, CCA treats multidimensional feature vectors as a whole, and uses mathematical methods to find a set of optimal solutions, so that the two entities have the greatest correlation weight, that is, have the largest value calculated by formula (8). This is the purpose of a typical correlation analysis.

(46) Step 106: based on the obtained eigenvector W.sup.T{circumflex over (X)}.sub.k and W.sup.TY and the projection matrixes U.sub.k and V.sub.k, performing pattern matching to obtain an eigenvector ρ.sub.k by equation (9);

(47) In pattern matching, the training template is constructed by the data of the training set, and the templates can be adjusted according to the different simulation types. Taking the classification of asymmetric EEG signals as an example, the similarity between the training template and the testing sample Y is represented as a vector ρ.sub.k shown in equation (9);

(48) 0$\begin{array}{cc}{\rho }_k=\left[\begin{array}{c}{\rho }_{k1}\\ {\rho }_{k2}\\ {\rho }_{k3}\\ {\rho }_{k4}\\ {\rho }_{k5}\end{array}\right]=\left[\begin{array}{c}\mathrm{corr}\left(W^T{\hat{X}}_k,W^{T}Y\right)\\ -\mathrm{dist}\left(W^T{\hat{X}}_k,W^{T}Y\right)\\ \mathrm{CCA}\left(W^T{\hat{X}}_k,W^{T}Y\right)\\ \mathrm{corr}\left(U_k^T{W}^T{\hat{X}}_k,U_k^T{W}^{T}Y\right)\\ \mathrm{corr}\left(V_k^T{W}^T{\hat{X}}_k,V_k^T{W}^{T}Y\right)\end{array}\right],k=1,2& \left(9\right)\end{array}$
Where corr(*) refers to the Pearson's correlation, dict(*) refers to the Euclidean distance. More similar it is between Y and {circumflex over (X)}.sub.k, more larger the ρ.sub.k1, ρ.sub.k2, ρ.sub.k3, ρ.sub.k4 and ρ.sub.k5 will be, and ρ.sub.k6 are connected to obtain the feature vector ρ.sub.k.

(49) Step 107: recognizing the eigenvector ρ.sub.k by different classifier modules and then outputting them.

(50) Linear Discriminant Analysis (LDA), Common Spatial Pattern (CSP), Support Vector Machine (SVM)

(51) According to the feature vector ρ.sub.k, different classifier models of different recognition algorithms such as Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM) are established. The testing sample Y is sent to the classifier for recognition after preprocessing and feature extraction, thereby predicting the type of the sample and outputting the result.

(52) Although the functions and working processes of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited thereto. The foregoing specific implementations are merely illustrative but not limiting. A person of ordinary skill in the art may make various forms under the teaching of the present invention without departing from the purpose of the present invention and the protection scope of the appended claims, and all the forms shall fall into the protection scope of the present invention.

(53) The foregoing description of the exemplary embodiments of the present invention has been presented only for the purposes of illustration and description and is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teaching.

(54) The embodiments were chosen and described in order to explain the principles of the invention and their practical application so as to activate others skilled in the art to utilize the invention and various embodiments and with various modifications as are suited to the particular use contemplated. Alternative embodiments will become apparent to those skilled in the art to which the present invention pertains without departing from its spirit and scope. Accordingly, the scope of the present invention is defined by the appended claims rather than the foregoing description and the exemplary embodiments described therein.