Real time audification of neonatal electroencephalogram (EEG) signals

Abstract

The present invention discloses a method and system of providing a real time audification of neonatal EEG signals. The method comprises the steps of: receiving preprocessed neonatal EEG signals; changing a characteristic of the preprocessed signals in a phase vocoder; resampling the output signals from the vocoder to a predetermined audio frequency range; converting the resampled signals into stereo signals; and selecting a plurality of channels from the stereo signals as the output audio signals.

Claims

1. A method of providing a real time audification of neonatal EEG signals, comprising the steps of: receiving preprocessed neonatal EEG signals; changing a characteristic of the preprocessed neonatal signals in a phase vocoder; resampling output signals from the phase vocoder to a predetermined audio frequency range; converting the resampled signals into stereo signals; selecting a plurality of channels from the stereo signals to form an output stereo audio signal, and wherein the preprocessed neonatal signals include temporal characteristics and the method includes changing the temporal characteristics of the preprocessed neonatal signals by: segmenting the preprocessed neonatal signals into overlapping frames; applying a short-time Fourier transform to each frame; separating a magnitude and phase of each frame; estimating an instantaneous frequency of each frame; determining a phase function associated with the frames; and performing an inverse short-time Fourier transform on each frame.

2. The method of claim 1, wherein the step of converting the resampled signals into stereo signals comprises the step of converting the resampled signals into left and right stereo channels corresponding to left and right brain hemispheres.

3. The method of claim 1, wherein the step of selecting a plurality of channels from the stereo signals to form the output audio signal comprises the step of selecting a channel from a left stereo channel and selecting a channel from a right stereo channel to form the output audio signal.

4. The method of claim 3, further comprising the step of: determining segments of the output audio signal which satisfy a predetermined criterion; and amplifying the gain of the output audio signal for the determined segments.

5. The method of claim 3, wherein the step of selecting a channel from the left stereo channel and selecting a channel from the right stereo channel as the output audio signal further comprises the steps of: determining a channel from the left brain hemisphere and a channel from the right brain hemisphere which has the highest cumulative probability of exhibiting a seizure; and selecting the determined channel from the left brain hemisphere and the determined channel from the right brain hemisphere to form the output audio signal.

6. The method of claim 3, comprising the step of determining segments of the output audio signal which satisfy a predetermined criterion corresponds to determining those segments which have the highest probability of exhibiting a seizure.

7. The method of claim 1, wherein a temporal characteristic of the preprocessed neonatal EEG signals is changed by slowing down the preprocessed neonatal EEG signals by a factor of 100.

8. The method of claim 1 wherein the resampling corresponds to mapping those frequencies known to correspond to the dominant frequencies of a seizure to the audible frequency range.

9. The method of claim 1, wherein the predetermined audio frequency range corresponds to 3-4 KHz.

10. The method of claim 1, further comprising the steps of preprocessing the neonatal EEG signals to produce preprocessed neonatal signals by: filtering of the neonatal EEG signals by low pass and high pass filters; and downsampling the frequency of the filtered neonatal EEG signals to 32 Hz.

11. The method of claim 1, wherein the output signals from the vocoder are resampled at a 32 kHz sampling rate.

12. A system for providing a real time audification of neonatal EEG signals, comprising: means for receiving preprocessed neonatal EEG signals; means for changing a characteristic of the preprocessed signals in a phase vocoder; means for resampling output signals from the vocoder to a predetermined audio frequency range; means for converting the resampled signals into stereo signals; and means for selecting a plurality of channels from the stereo signals as the output audio signals, and wherein the preprocessed neonatal signals include temporal characteristics and the system further includes means for changing the temporal characteristics of the preprocessed neonatal signal comprising: means for segmenting the preprocessed neonatal signals into overlapping frames; means for applying a short-time Fourier transform to each frame; means for separating a magnitude and phase of each frame; means for estimating an instantaneous frequency of each frame; means for determining a phase function associated with the frames; and means for performing an inverse short-time Fourier transform on each frame.

13. A method of providing a real time audification of neonatal EEG signals, comprising the steps of: receiving preprocessed neonatal EEG signals; changing a characteristic of the preprocessed signals in a phase vocoder with application specific parameters in order to increase audio intelligibility of the neonatal EEG signals; re-sampling output signals from the phase vocoder to a predetermined audio frequency range to match a human hearing system; converting the re-sampled signals into stereo audio signals and playing the created stereo audio; and selecting a plurality of channels from the signals to form an output stereo audio signal, and wherein the preprocessed neonatal signals include temporal characteristics and the method includes changing the temporal characteristics of the preprocessed neonatal signal by: segmenting the preprocessed neonatal signals into overlapping frames; applying a short-time Fourier transform to each frame; separating a magnitude and phase of each frame; estimating an instantaneous frequency of each frame; determining a phase function associated with the frames; and performing an inverse short-time Fourier transform on each frame.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The invention will be more clearly understood from the following description of an embodiment thereof, given by way of example only, with reference to the accompanying drawings, in which:

(2) FIG. 1 details the main steps in the process of the invention which provides a real time audification of neonatal EEG signals.

(3) FIG. 2 shows a flowchart for the audification of neonatal EEGs;

(4) FIG. 3 shows a probabilistic output for each stereo channel representing the EEG signal; and

(5) FIG. 4 shows a flowchart of the main processing steps of the phase vocoder of the present invention.

DETAILED DESCRIPTION OF THE DRAWINGS

(6) The present invention provides a method and system for listening to the EEG of newborn babies or neonates. By listening to the EEG of a neonate, various patterns of the EEG signals can be detected by the listener. In the described embodiment of the invention, such patterns are used to indicate to the listener the presence of a seizure in the brain of a neonate.

(7) FIG. 1 details the main steps in the process of the invention which provides a real time audification of neonatal EEG signals. In step 100 preprocessed neonatal EEG signals are received. In step 105, a characteristic of the preprocessed signals is changed in a phase vocoder. The output signals from the vocoder are then resampled to a predetermined audio frequency range (step 110). The resampled signals are then converted into stereo signals (step 115). Finally, a plurality of channels from the stereo signals are selected as the output audio signals (step 120).

(8) FIG. 2 details the main steps in one embodiment of the audification process of the present invention, with an example of 1000s of EEG as an input. In the first step, the output signals of the EEG are pre-processed prior to being input to a vocoder through the filtering of the EEG signals by low pass and high pass filters, to attenuate any unwanted information. This is followed by the downsampling of the frequency of the signals to 32 Hz.

(9) In the vocoder, the temporal characteristics of the signals are changed. This intuitively corresponds to stretching the time-base of the signal spectrogram while retaining their short-time spectral characteristics. Therefore, the signals are slowed down by a factor of N, which in the preferred embodiment has a value of 100. These signals are then resampled to an audible range.

(10) In the frequency mapper, the output signals from the vocoder are resampled at a 32 kHz sampling rate. This corresponds to pitch shifting the original frequency range of 0.5-16 Hz to a new range of 0.5-16 kHz. Through frequency mapping, the most dominant frequencies of seizure (typically 0.5-6 kHz) can be mapped to signals within the audible range, in particular to the range of human scream, namely 3-4 kHz.

(11) As a result, the EEG real-time playback is sped up by a factor which is inversely proportional to N. It will be appreciated that this provides for a significant time saving for the listener, for example where in the preferred embodiment 1 hour of EEG can be played back in around 6 minutes.

(12) In a further step, the audio signals are converted into stereo signals, such that the left stereo signal corresponds to the left brain hemisphere and the right stereo signal corresponds to the right brain hemisphere. These stereo signals are then input to a seizure detection module. The output of the seizure detection module corresponds to the final audified EEG signal.

(13) In the preferred embodiment of the seizure detection module, the channel from each hemisphere which is determined to have the highest cumulative probability of a seizure event is selected to form the output audio signal. This is determined by computing the cumulative seizure probability from the per-channel probabilistic outputs which are shown in FIG. 3, which shows a plot of probability output for each channel versus time (in minutes). The channels with the maximum cumulative probability is taken separately from the left (F3-C3, C3-O1, Cz-C3, C3-T3) and right hemisphere (F4-C4, C4-O2, T4-C4, C4-Cz). In this figure, the maximum probabilistic output across channel is shown in bold, while seizure onset and offset are annotated as 16m37s-30m05s and superimposed on top in red. The signal gain is also controlled by the probabilistic output of the system, such that the signal is multiplied by the maximum probabilistic output of the system. As a result, EEG segments which are likely to be seizures are more audible than the background.

(14) FIG. 4 shows a flowchart of the main processing steps of the phase vocoder of the present invention. It comprises three main processing stages, namely analysis, processing, and synthesis. During the analysis stage, the signal is segmented into overlapping frames with analysis time-instants t.sub.a=uR.sub.a, for an integer u, and the short-time Fourier transform (STFT) is then applied to each frame:

(15) $X\left(t_a^u,{}_k\right)=\underset{n=-}{\overset{}{.Math.}}h\left(n\right)x\left(t_a^u+n\right)e^{-j{}_{k}n}$

(16) Where x is the original signal, h(n) is the analysis window, .sub.k=2k/N is the centre frequency of kth STFT channel, and N is the size of the STFT. X(t,) is a function of both time and frequency.

(17) In the synthesis stage, the inverse short-time Fourier transform is performed on each frame spectrum. The overlapping spectrum frames result in the output segments overlapping each other. The overlapping output segments are all summed together, yielding the following output signal (known as the overlap-add method) in the time domain:

(18) $y\left(n\right)=\frac{1}{N}\underset{k=0}{\overset{N-1}{.Math.}}Y\left(t_s^u,{}_k\right)e^{j{}_{k}n}$

(19) In the absence of modifications, t.sub.s=t.sub.a, =R.sub.a/R.sub.s=1, and the output signal is identical to the original signal, where R.sub.a and R.sub.s are the hop size of the analysis and synthesis, respectively. It should be noted that all modifications that are done in the spectrum representation need to preserve the appropriate correlation between adjacent frequency bins and time frames.

(20) In the processing stage, the magnitude and phase are separated. The time evolution of sine-wave amplitude is modified simply by setting:
|Y(t.sub.s.sup.u,.sub.k)|=|X(t.sub.a.sup.u,.sub.k)|

(21) In the synthesis stage, the process requires phase unwrapping. The phase, which is measured modulo 2, is unwraped by keeping track of cumulative phase variation and taking its principal determination between :
.sub.k.sup.u=<X(t.sub.a.sup.u,.sub.k)<X(t.sub.a.sup.u1,.sub.k)R.sub.a.sub.k

(22) Where <X(t.sup.u) and <X(t.sup.u1) are the phases at time instances t.sup.u and t.sup.u1, respectively. Phase unwrapping is a process whereby the phase increment between two consecutive frames is used to estimate the instantaneous frequency of the closest sinusoid:

(23) $w_k\left(t_a^u\right)={}_k+\frac{1}{R_a}{}_k^u$

(24) where w.sub.k is the instantaneous frequency.

(25) The phase increment is simply the small phase shift resulting from w.sub.k being close but not equal to .sub.k. Once the instantaneous frequency is estimated, the phase of the time-scaled output signal Y is set according to the phase-propagation formula
<Y(t.sub.s.sup.u,.sub.k)=<Y(t.sub.s.sup.u1,.sub.k)+R.sub.sw.sub.k(t.sub.a.sup.u)

(26) This phase function can be decimated or interpolated to the new time scale, so that the phase of the output short-time Fourier Transform, STFT can be computed at any given synthesis time-instant:

(27) $Y\left(t_s^u,{}_k\right)=Y\left(t_s^0,{}_k\right)+\underset{i=1}{\overset{u}{.Math.}}{R}_s{w}_k\left(t_a^i\right)$

(28) It will be appreciated that the method of audification of neonatal EEG signals of the present invention provides a number of advantages when compared to existing adult EEG signal audification methods. Firstly, the phase vocoder of the present invention allows for time stretching without affecting the pitch, and pitch scaling without affecting the signal duration. In addition, the method of the present invention enables direct EEG playback without any artificially created synthetic sounds.

(29) While the present invention has been described in an application where the audification of a neonate's EEG signals are used as a unique audio alarm for seizures, the audification of present invention could equally be used in a number of other applications. For example, the audio signals could be used to help classify the different background EEG patterns occurring in term and preterm babies, and in the sleep state analysis of newborns. In addition, the audio signals could allow parents to listen to their babies brain waves, and could also be played to sick newborns to help reduce stress.

(30) The embodiments in the invention described with reference to the drawings comprise a computer apparatus and/or processes performed in a computer apparatus. However, the invention also extends to computer programs, particularly computer programs stored on or in a carrier adapted to bring the invention into practice. In particular computer programs for controlling the system and method as hereinbefore described. The program may be in the form of source code, object code, or a code intermediate source and object code, such as in partially compiled form or in any other form suitable for use in the implementation of the method according to the invention. The carrier may comprise a storage medium such as ROM, e.g. CD ROM, or magnetic recording medium, e.g. a floppy disk or hard disk. The carrier may be an electrical or optical signal which may be transmitted via an electrical or an optical cable or by radio or other means.

(31) Additionally, at least a portion of the systems, methodologies and techniques described with respect to the exemplary embodiments of present disclosure can incorporate a machine, such as, but not limited to, computer system, or any other computing device within which a set of instructions, when executed, may cause the machine to perform any one or more of the methodologies or functions discussed above. The machine may be configured to facilitate various operations conducted by the systems disclosed herein. For example, the machine may be configured to, but is not limited to, assist the systems by providing processing power to assist with processing loads experienced in the systems, by providing storage capacity for storing instructions or data traversing the systems, or by assisting with any other operations conducted by or within the systems.

(32) Dedicated hardware implementations including, but not limited to, application specific integrated circuits, programmable logic arrays and other hardware devices can likewise be constructed to implement the methods described herein. Applications that may include the apparatus and systems of various embodiments broadly include a variety of electronic and computer systems. Some embodiments implement functions in two or more specific interconnected hardware modules or devices with related control and data signals communicated between and through the modules, or as portions of an application-specific integrated circuit. Thus, the example system is applicable to software, firmware, and hardware implementations.

(33) In accordance with various embodiments of the present disclosure, the methods described herein are intended for operation as software programs running on a computer processor. Furthermore, software implementations can include, but not limited to, distributed processing or component/object distributed processing, parallel processing, or virtual machine processing can also be constructed to implement the methods described herein.

(34) In the specification the terms comprise, comprises, comprised and comprising or any variation thereof and the terms include, includes, included and including or any variation thereof are considered to be totally interchangeable and they should all be afforded the widest possible interpretation and vice versa.

(35) The invention is not limited to the embodiments hereinbefore described but may be varied in both construction and detail.