METHOD AND APPARATUS FOR DETECTING A SLEEP STATE

Abstract

A method and device for detecting a sleep state by heart-rate and movement data. By neural network software, a sleep state is detected using as input data movement data (move_count) and data derived from heart-rate data and/or data derived from inter-beat interval data such as (MAD) and respiration data (RESP). At least a portion of the variables (HRD, MHR, MAD, Resp, GRD) derived from the heart-rate data/inter-beat interval data are further modified by one, most preferably 2-4, artificial average functions and the cumulative sleep time is one input datum.

Claims

1. A method for detecting a sleep state by heart-rate and movement data, in which method neural network software is used for detecting a sleep state and in which the input data are movement data (move_count) and data derived from heart-rate data and data derived from inter-beat interval data and respiration data (RESP), comprising: inputting time changing variables used as input data for the neural network software including: move count heart rate deviation (HRD) heart rate difference (from minimum HR) (MHR) mean absolute difference between successive RR-intervals (MAD) respiration (Resp) GRD, which is defined as GRD(t)=f_ma(MAX(0,hr(t)-min_hr)/MHRc(t)), where MHRc(t) is MHR(t) calculated with a chosen time window c, cumulative sleep time, and at least a portion of the variables (HRD, MHR, MAD, Resp, GRD) derived from the heart-rate and/or inter-beat interval data are further modified by 2-4 artificial average functions, which have the form f_ma(t)=y(t)=(c*y(t1)+F_input(t))/(c+1), where c is 20-80 with a sample window of 5 s and F_input is the input function to be averaged.

2. The method according to claim 1, wherein the heart-rate data and night-time inter-beat interval data are generated with a PPG device.

3. The method according to claim 1, wherein in addition to the real-time detection of the method, a correction is calculated during a dynamic delay so that a time window of a chosen length, e.g. a five-minute time window, is applied to the incoming signal, whereupon an auxiliary logic corrects the incoming signal in accordance with previously defined rules, wherein the episodes constituting the bulk of one sleep state are first sought in the incoming signals, that part is selected in accordance with a main rule, e.g. 80% of the episodes, and, apart from previously defined exceptions, the whole episode is modified in accordance with the bulk of the sleep state.

4. The method in accordance with claim 3, wherein, as a special rule, AWAKE states are not modified in accordance with the bulk of the state.

5. The method according to claim 1, wherein a calculation of night-time averages of the heart-rate variability variables occurs in the method in such a way that two neural models are implemented as follows: during the measurement a first night, a less accurate neural model is used for the detection of a sleep state and an average of the heart-rate variability variables is stored in real time, e.g. the first 4 h of the night are a sufficient sample for the variables to stabilize, after which it is possible to switch to using a neural model that utilizes these averages.

6. The method according to claim 5, wherein values of the heart-rate variability variables are distributed in the calculation by the average night-time heart-rate variability.

7. The method according to claim 5, wherein the method includes two different neural models for the detection, of which a first, less accurate, model is used when the night-time background parameters have not yet been calculated and the subsequent model is used when the background parameters have been calculated.

8. A device for detecting a sleep state by heart-rate and movement data, the device comprising: a software-operating system, including a CPU, RAM and ROM memory, and input unit for receiving heart-rate data and movement data, an output device and neural network software, wherein the device includes software configured to detect a sleep state by the method by: inputting time changing variables used as input data for the neural network software including: move count heart rate deviation (HRD) heart rate difference (from minimum HR) (MHR) mean absolute difference between successive RR-intervals (MAD) respiration (Resp) GRD, which is defined as GRD(t)=f_ma(MAX(0,hr(t)-min_hr)/MHRc(t)), where MHRc(t) is MHR(t) calculated with a chosen time window c, cumulative sleep time, and at least a portion of the variables (HRD, MHR, MAD, Resp, GRD) derived from the heart-rate and/or inter-beat interval data are further modified by 2-4 artificial average functions, which have the form f_ma(t)=y(t)=(c*y(t1)+F_input(t))/(c+1), where c is 20-80 with a sample window of 5 s and F_input is the input function to be averaged.

9. The device according to claim 8, wherein the device includes a PPG wrist device as its only source of heart-rate data.

10. The device according to claim 8, wherein the device includes an ECG heart-rate monitor as its only source of heart-rate data.

Description

BRIEF DESCRIPTION OF THE FIGURES

[0027] The invention is described in the following with reference to the attached drawings in which certain applications of the invention and details of the calculation are shown.

[0028] FIG. 1 shows an advantageous neural network model for defining a sleep state

[0029] FIG. 2 shows a typical hardware configuration of a wrist-worn device with its interfaces

[0030] FIGS. 3a-3g show input values of the neural network during measurement, of which

[0031] FIG. 3a shows a move_count variable expressing a state of movement

[0032] FIG. 3b shows an HRD variable and a variable modified from the former with three delays

[0033] FIG. 3c shows an MHR variable and a variable modified from the former with three delays

[0034] FIG. 3d shows an MAD variable and a variable modified from the former with three delays

[0035] FIG. 3e shows a RESP variable and a variable modified from the former with three delays

[0036] FIG. 3f shows an GRD variable and a variable modified from the former with three delays

[0037] FIG. 3g shows a sleep_time variable representing the cumulated sleep state

[0038] FIG. 4 shows the progression of a calculation from measurement signals

[0039] FIG. 5a shows plots of the delay-calculated signals from FIGS. 3c (section a)

[0040] FIG. 5b shows plots of the delay-calculated signals from FIGS. 3e (section b)

[0041] FIG. 6 shows the processing of the output of the neural network

[0042] FIG. 7 shows a block diagram describing the sleep state detection in thee subfunctions

[0043] The above abbreviations are HRD=Heart Rate Deviation, MAD=Mean Absolute Difference between successive RR-intervals, MHR=Heart rate difference (from Minimum HR), RESP=Respiration (respiratory rate), PPG=Photoplethysmogram, GRD=Gradient of the heart rate difference.

DETAILED DESCRIPTION

[0044] FIG. 4 shows the progression of a calculation from measurement signals (top bar) to a real-time calculation (middle bar) and a final result obtained by means of a dynamic-delay calculation (bottom bar). FIGS. 5a and 5b illustratively show plots of the delay-calculated signals from FIGS. 3c (section a) and 3e (section b). These are a pivotal feature of the basic invention. Together with the chosen group of variables, the neural network calculation produces a markedly accurate detection of a sleep state with a relatively small quantity of input data.

[0045] The fundamental idea of neural networks is based on natural neural networks. However, generally speaking, the objective these days is not a precise simulation of natural neural networks, but rather the development of neural network techniques is based more on, for instance, statistical science and signal-processing theory.

[0046] There are numerous alternatives with respect to the software that can be chosen for neural network applications. The above-mentioned Matlab version is well suited to this purpose.

[0047] The neural network 40 application of FIG. 1 is quite simple, as there is a very limited number of input data, namely 21. There is only one output datum, i.e. the sleep state, which is stored to a result register. Different wrist-worn device manufacturers implement neural network 40 software configured for these devices.

[0048] In accordance with the invention, the method can be realized in particular with a very conventional wrist-worn device, the hardware of which is depicted in FIG. 2. The apparatus, here a wrist-worn device, includes the required measurement sensors 12, 70 (heart-rate and acceleration sensors), a bus channel 36, keys 18 and a unit 31 for inputting data, a central processing unit 32, ROM memory 31.2 and RAM memory 31.2, including a buffer (loop buffer memory), an output unit 34, a display 15 as well as potential voice synthesizers and speakers 35. The device generally has an interface 37 for interfacing with a PC 38 or for connecting directly to the internet. In the following, the memory sizes required by the invention are explained. The buffer memory required by the auxiliary logic constitutes a relatively small resource.

[0049] From the sleep-state classification, it is typically desirable to store and share with the user inter alia the average ratios and/or times of the sleep stages, the time of falling asleep, as well as the total time slept. The dynamic delay described above can change said variables generated in real time. Whenever the sleep state is changed after the fact, it is possible to update the internal counters in real time (e.g. the cumulative totals of the sleep stages). Moreover, the final results are typically not called up during sleep, but after sleep has ended so that the final results are generally ready or will no longer be subject to significant changes. Potential inaccuracies resulting from the dynamic delay are small compared to typical errors caused by machine learning when classifying sleep states. The empirically (n=110 expert-classified sleep data sets) described method is in real time 93% of the time, while the average correction delay is 8.6 min the remaining 7% of the time.

[0050] Reference is made to FIG. 4 in the following. In this illustrative figure, the expert-classified sleep states are in the top graph, the real-time sleep states provided by the neural model are in the middle and the sleep states provided with a delay are in the bottom graph. When comparing the bottom and middle graph, one observes: [0051] The momentary incorrect detections occurring at the points 24 min, 278 min, 1188 min have been erased. [0052] At the point 638 min, the fragmentation of REM sleep has been smoothed over, thus improving accuracy. [0053] The sleep interruption at the point 767 min has been corrected to AWAKE. [0054] By means of the dynamic delay, the average accuracy of sleep detection improved from 64.40%=>68.75% (F.sub.1 score).

[0055] In addition to the improved accuracy, a significant benefit of the method resides in the subsequent graphic display of the sleep states to the user. It is possible to implement the method efficiently in a wrist-worn device in which the calculated detection of a sleep state in real time also renders possible the displaying of the results of the sleep analysis (time slept, sleep feedback, time of falling asleep, sleep state distributions) immediately after waking up. By means of an efficient coding, the transmission and display of the results e.g. on a mobile device with a larger monitor is simple and the results are identical with the results of the wrist-worn device.

[0056] To our knowledge, a method of the type described that enables an operation in real time in a wrist-worn device does not exist on the market, but rather the analyses occur after the fact e.g. with a mobile device. Moreover, some of the methods (such as Polar) generate results about 1 h after sleep has ended. The described method enables the generation of feedback regarding sleep times on the wrist-worn device immediately after sleep has ended. The feedback information at this stage can also be a verbal text produced by an expert system and a higher-level classification further including guidance for ameliorating potential observed sleep irregularities (short sleep, long time falling asleep, numerous sleep interruptions).

[0057] Empirical Testing

[0058] For the production of the generated error values and figures in this illustration, a sleep data population (different test persons) of n=110 was used in which actual sleep states were detected in a sleep laboratory using more extensive sensor data, such as an electroencephalogram. In addition, the modelling database includes n=150 expert classified sleep start/end-times, n=781 user estimated sleep start/end-times. The intention is to detect sleep states with a heart-rate device as well as possible solely on the basis of sensor data pertaining to heart rate and heart-rate acceleration.

[0059] Expanding the input variable time window into the past and future in a real-time system for sleep state detection

[0060] Based on empirical and physiological research, sleep state detection benefits from entering temporal data relating to the input properties into the neural model. The so-called F.sub.1 score, also known as the Srensen-Dice coefficient or Dice similarity coefficient (DSC), accuracy significantly improves if, in addition to samples calculated in real time from input vectors, samples relating to the past (e.g. 40 min 20 min from the current point in time) as well as the future (+20 min +40 min from the current point in time) are entered into the neural model. Typically, the F.sub.1 score improves for both operations by an average of approx. 10%. The detection of REM sleep particularly requires temporal information in order for the neural model to be able to relate the current heart rate variability and heart rate to the average changing rate over time.

[0061] Although the described sample window improves the accuracy of the system, it produces undesirable attributes in a system that works in real time in a small device: an input variable sample window over time requires a significant use of memory (altogether as much as an 80 min buffer for each input variable) while the future samples create a delay (40 min in the examples) for the real-time system.

[0062] The aforementioned weakness can be bypassed by outputting input variables that yield corresponding information for the neural model. The samples relating to the past can be corrected by calculating the input variables with different moving average (MA) windows (increasing the constant c). In the illustrative figure, FIG. 3c, four different averages (c1=20, c2=70, c3=140, c4=210) have been calculated from the input variable MHR. The input variables have been scaled between 0-1. It is evident in the figure that the addition of the moving average smooths out the variable in addition to the temporal displacement. The operation thus does not naturally correspond to the collection of the samples from the original variables at different points in time. Via empiricism, however, it can be observed that the solution is able to correct the sample window and provide the neural model with sufficient temporal information regarding the past. It is also possible for the neural network to utilize input variables generated in this manner inter alia in order to render the temporal resolution more precise and to distinguish between input variable gradients.

[0063] FIG. 5a shows the inset a from FIG. 3c. In this figure, the lag of the artificial signal produced by varying the coefficient ci.e. c0, c1, c2, c3 in the figureis clearly visible. A corresponding representation is shown in FIG. 5b, which shows the inset b from the FIG. 3e. By means of the coefficients, the neural network is supplied with the temporal behaviour of the functions.

[0064] It is significant that the depicted moving average only requires two 32-bit RAM variables and an 8-bit ROM variable in order to generate the input variable in real time.

[0065] Neural Network

[0066] In the input data of the neural network 40 in FIG. 1, four main input variables are varied, i.e. four different moving averages (Matlab) are calculated for the same:

[0067] HRD(t)=f_ma(ABS(hr(t)-hr(t1))))=(c*HRD(t1)+ABS(hr(t)-hr(t1))))/(c+1)abs. fluctuation of heart ratemeasures the absolute difference between the heart rate at time instant t, i.e. hr(t), and the average heart rate at time instant t1, i.e. hr(t1).

[0068] MHR(t)=f_ma(MAX(0, hr(t)-min_hr)))differentiable function of heart ratemeasures the difference between the heart rate at time instant t, i.e. hr(t), and the minimum heart rate in the background parameters of the person, i.e. min_hr. [0069] MAD(t)=f_ma(mad(t))inter-beat interval functionmeasures the mean absolute difference between successive RR-intervals.

[0070] RESP(t)=f_ma(resp(t))respiratory rate function.

[0071] Moving average is calculated as f_ma(t)=(c*f ma(t1)+inpunt))/(c+1), where input is the averaged data, which can be HRD, move_count, MHR, MAD, Resp or GRD, and c is a coefficient referring to a window length. In practice, c is an integer that defines how much the formerly calculated moving average f_ma(t-1) is weighted. If c is large, the new value input(t) has little effect on f_ma and the curve becomes very smooth. If c is small (e.g. 1 equals the average of two values) the curve is temporally more sensitive for changes, i.e. new values input(t) move the average quickly towards themselves.

[0072] In addition, three different moving averages are calculated for the GRD(t) input variable,

[0073] GRD(t)=f_ma(MAX(0,hr(t)-min_hr)/MHRc(t)), where MHRc(t) is MHR(t) calculated with a chosen coefficient c, i.e. three MHR(t) variants are used here, thus producing three different GRD(t) values for a point in time t.

[0074] In the equation for GRD, the difference between the heart rate at time t, i.e. hr(t), and the minimum heart rate in the background parameters of the person, i.e. min_hr, is divided by MHRc(t). MHRc(t) is MHR(t) calculated with a selected coefficient c, i.e.

[0075] x(t)=MAX(0, hr(t)-min_hr)))

[0076] MHR(t)=f_ma(x(t))=(x(t)+c1*f_ma(x(t1))/(1+c1),

[0077] MHR2(t)=f_ma(x(t))=(x(t)+c2*f ma(x(t1))/(1+c2),

[0078] MHR3(t)=f_ma(x(t))=(x(t)+c3*f ma(x(t1))/(1+c3),

[0079] MHR4(t)=f_ma(x(t))=(x(t)+c4*f ma(x(t1))/(1+c4),

where the coefficients c1, c2, c3, c4 refer to different time constants. When the MHRc(t) values are first calculated, the three different GRD(t) values can then be calculated as

[0080] GRD2(t)=f_ma(MAX(0,hr(t)-min_hr)/MHR2(t)),

[0081] GRD3(t)=f_ma(MAX(0,hr(t)-min_hr)/MHR3(t)),

[0082] GRD4(t)=f_ma(MAX(0,hr(t)-min_hr)/MHR4(t)),

where f_ma is calculated with coefficient c that is different to the coefficient c used in the calculation for MHRc. Coefficients c in the moving average for GRD can be, for example, 4, 8 and 12.

[0083] The foregoing describes input variables when heart-rate variability averages during sleep have not been calculated. If these are known, then the averaged input variables for HRD(t) and MAD(t), i.e. HRD2(t) and MAD2(t), respectively, are calculated as

[0084] HRD2(t)=f_ma(abs(hr(t)-hr(t1))/ave_hrd), where the difference between the heart rate at time instant t, i.e. hr(t), and the heart rate at time instant t1, i.e. hr(t1) is divided by the known average HRD value, i.e. ave_hrd, before the moving average is calculated.

[0085] MAD2(t)=f_ma(mad(t)/ave_mad), where the mean absolute difference between successive RR-intervals is divided by the known average MAD value, i.e. ave_mad, before the moving average is calculated.

[0086] Moreover, two important main input variables for which separate temporal averages are not generated are move_count and detected time slept. Of these, the move_count is calculated as follows: the total absolute sum (act) for the three channels of the acceleration sensor is stored in 5 s intervals and, on this basis, the moving average move_count(t)=(emove_count(t1)+(act>threshold))/(c+1) is calculated, where threshold is a value indicating significant movement, such as walking or moving hand rapidly. The detected time slept is stored to a sum register 52 which increases the cumulative sleep time when sleep is detected in block 51 (FIG. 4).

[0087] In total, the neural network 40 thus has 21 input variables. The input data of the neural network 40 are scaled between 0-1.

[0088] The input data is fed into feed-forward neural network 40 divided into three subfunctions 41, 42, 43 as illustrated in the block diagram of FIG. 7. The first neural network subfunction 41 (ffnet1) studies if the person is sleeping or not (sleep state 0). If sleep is detected in block 55 the calculation continues to the second neural network subfunction 42 (ffnet2), which detects the possible AWAKE state (sleep state 5). Furthermore, if awake state is not detected in block 56, the calculation continues to the third neural network subfunction 43 (ffnet3), which studies the DEEP/LIGHT/REM sleep (sleep state 2-4). If DEEP/LIGHT/REM sleep is detected the cumulative sleep time counter is updated and the updated value is forwarded to the next input. The feed-forward neural network analysis uses the following code in all three subfunctions:

TABLE-US-00002 fxint ffnet(uint8 n0, uint8 n1, fxint input[ ], const int32 w[ ]) { fxint finalbias = w[n0 * n1+ 2 * n1]; for (uint8 i = 0; i < n1; i++) { fxint sum = w[i + n0 * n1 + n1]; for (uint8 j = 0; j < n0; j++) { sum += fx_mul(input[j], w[i * n0 + j]); } finalbias += fx_mul(fx_log_sig(sum), w[i + n0 * n1]); } return finalbias; // linear output }

[0089] In the code above n0 is the number of inputs, n1 is the number of hidden neurons, vector input[ ] includes the input variables, and vector w[ ] includes the feed-forward neural network 40 parameters, which are different for all three subfunctions 41, 42, 43. The number of the neural network 40 weighting parameters, i.e. the size of vector w[ ], for the thee subfunctions 41, 42, 43 (ffnet1, ffnet2, ffnet3) is 22, 70, and 185, respectively. The number of hidden neurons in the thee subfunctions 41, 42, 43 (ffnet1, ffnet2, ffnet3) is 3, 3, and 8, respectively. The first subfunction 41 (ffnet1) utilizes only 5 selected input variables, while the other subfunctions (ffnet2 and ffnet3) utilize all 21 input variables. The feed-forward neural network parameters (number of inputs, number of hidden neurons and weightings) have been carefully selected, using empirical and physiological data described above and the Matlab software Neural Network Toolbox, in order to minimize the memory usage and calculation time while reaching an accurate result for the sleep detection. The output of the neural network analysis is a numerical value that is compared to a carefully selected threshold value in order to determine the sleep state.

[0090] FIG. 6 shows the processing of the output of the neural network 40. The output of the neural network 40, i.e. the real-time values depicted in the middle bar of FIG. 4, is simplified in block 66 and saved to host 60 in a simplified 8-bit format comprising the sleep state and the corresponding sleep state length of time. The raw output state values are also momentarily saved to 30 min loop buffer 62. The buffer 62 is read by auxiliary logic 64, which corrects the incoming signal in accordance with predetermined rules. The auxiliary logic 64 retrospectively corrects the state values to the host 60, and, thus, the final result obtained by means of a dynamic-delay calculation, depicted in the bottom bar of FIG. 4, is obtained.

[0091] ROM/RAM Requirements,

[0092] The input data (32 bit) according to the foregoing are 1: movecount

[0093] 2-5 HRD

[0094] 6-9 MHR

[0095] 10-13 MAD

[0096] 14-17: RESP

[0097] 18-20: GRD

[0098] 21: Sleep-time

[0099] Output (8 bit): Sleep state, the values of which are Blue: deep sleep, Yellow: light sleep;

[0100] Green: REM, Red: awake.

[0101] The method according to invention described here using 21 input variables for the feedforward-neural network model requires the storage of a total of 466 16-bit units in the ROM memory (0.932 kb). The parameters of the model are optimized empirically e.g. by utilizing the Matlab neural network toolbox (or optimization toolbox).

[0102] The dynamic delay is realized with two average windows, of which the larger window (25 min) defines the number of sleep states to be stored (the smaller window is 5 min). As it is not worth calculating sleep states with a resolution greater than 1 min, a storage (51 bytes) of 51 8-bit values in a loop buffer is sufficient for this purpose. Other dynamic-delay logics (inter alia feedback synthesis) require the maintenance of three uint16 time indexes in real time.

[0103] In addition to the foregoing, cumulative values such as times slept in different sleep states are calculated and the values of the night-time heart-rate variability variables ave_hrd, ave_mad are updated in the moving average windows for the provision of feedback statements.

TABLE-US-00003 #define SLEEP_STAGE_INPUT_SIZE 21 #define SLEEP_STAGE_DIV 12 // calculation update 5s, DIV=12 => resolution for sleep calculation 1min #define SLEEP_STAGE_N (612/SLEEP_STAGE_DIV) typedef struct { int8 lag_state[2]; uint8 acw, i, ffnet_index, state, n_act; uint16 ave_n, state_sum[2][5], sleep_time[2][4], t1, t2, lag_len, lag_t, an,fr1,fr2,fr3; uint32 ave_sleep[3], 1st, 1st4h, act; fxint ave_sleep_hrd[2], ave_sleep_mad[2]; fxint input[2][SLEEP_STAGE_INPUT_SIZE]; // Sleep Stage Detection input - NN Input layer uint8 sleep[SLEEP_STAGE_N]; // loop buffer for sleep states, resolution SLEEP_STAGE_DIV } iete_sleep_stage_variables;

[0104] In all, the part required for sleep detection uses a dynamic memory in accordance with the structure described above (2*8+5*8+27*16+6*32+4*32+42*32)/8=269 bytes=0.269 kb.