Remote monitoring of vital signs
09615749 ยท 2017-04-11
Assignee
Inventors
- David Andrew Clifton (Oxford, GB)
- Mauricio Christian Villarroel Montoya (Oxford, GB)
- Lionel Tarassenko (Oxford, GB)
Cpc classification
A61B5/0077
HUMAN NECESSITIES
G06V40/15
PHYSICS
G06T5/94
PHYSICS
A61B5/02416
HUMAN NECESSITIES
A61B5/0816
HUMAN NECESSITIES
A61B5/0205
HUMAN NECESSITIES
A61B5/725
HUMAN NECESSITIES
International classification
A61B5/1455
HUMAN NECESSITIES
A61B5/1171
HUMAN NECESSITIES
A61B5/08
HUMAN NECESSITIES
A61B5/02
HUMAN NECESSITIES
A61B5/00
HUMAN NECESSITIES
A61B5/0205
HUMAN NECESSITIES
Abstract
A method of remote monitoring of vital signs by detecting the PPG signal in an image of a subject taken by a video camera such as a webcam. The PPG signal is identified by auto-regressive analysis of ambient light reflected from a region of interest on the subject's skin. Frequency components of the ambient light and aliasing artefacts resulting from the frame rate of the video camera are cancelled by auto-regressive analysis of ambient light reflected from a region of interest not on the subject's skin, e.g. in the background. This reveals the spectral content of the ambient light allowing identification of the subject's PPG signal. Heart rate, oxygen saturation and breathing rate are obtained from the PPG signal. The values can be combined into a wellness index based on a statistical analysis of the values.
Claims
1. A method of suppressing ambient light interference in a photoplethysmographic (PPG) image signal, the method comprising: imaging a first region of interest on a skin of a subject using a video camera to obtain a PPG image signal, wherein the PPG image signal comprises periodic intensity variations corresponding to ambient light reflected from the region of interest; imaging a first reference region of interest not on the skin using the video camera to obtain a reference signal; spectrally analyzing, via a computer, the reference signal using a first auto-regressive (AR) all pole model to identify poles corresponding to spectral components in the first AR all pole model for the reference signal; and spectrally analyzing, via a computer, the PPG image signal using a second auto-regressive (AR) all pole model to identify poles corresponding to spectral components in the second AR all pole model for the PPG image signal and removing poles in the second AR all pole model corresponding to the spectral components of the reference signal to suppress the ambient light interference; estimating, via the computer, a first vital-sign of the subject based on a remaining portion of the second AT all pole model and after removal of the selected ones of the poles from the second AR all pole model corresponding to the spectral components of the reference signal; and storing, displaying or transmitting, via the computer, the estimated first vital sign.
2. The method according to claim 1, wherein the reference signal and PPG image signal are output signals from (i) at least one of red, green and blue channels of the video camera, or (ii) an infrared channel of another camera.
3. The method according to claim 1, further comprising: imaging a first plurality of regions of interest on the skin, wherein the first plurality of regions of interest include the first region of interest; and imaging a second plurality of regions of interest, wherein the second plurality of regions of interest comprise the reference region of interest and other reference regions of interest.
4. The method according to claim 3, wherein each of the first plurality of regions of interest and each of the second plurality of regions of interest is centered on a single camera pixel.
5. The method according to claim 1, further comprising obtaining vital sign data from remaining components of the PPG image signal.
6. The method according to claim 1, wherein: the reference signal and the PPG image signal are each analyzed using a plurality of models having respective orders; and the plurality of models comprise the first AR all pole model and the second AR all pole model.
7. The method according to claim 6, wherein the plurality of models have respectively orders 8 to 20.
8. The method according to claim 6, further comprising averaging the spectral components of the reference signal and the PPG image signal over different order models, wherein the different order models include the first AR all pole model and the second AR all pole model.
9. The method according to claim 1, further comprising obtaining a measurement of a heart rate of the subject by identifying a pole in the PPG image signal as representing the heart rate, which is not present in the reference signal.
10. The method according to claim 9, wherein the measurement of the heart rate is obtained by identifying a pole corresponding to a spectral component and having a frequency between 0.67 Hz and 4 Hz.
11. A method according to claim 9, further comprising obtaining a measurement of a blood oxygen saturation level of the subject by obtaining a ratio of intensity of light at two different wavelengths reflected from the region of interest on the skin, wherein the intensity of light is obtained from a magnitude of the pole identified as representing the heart rate.
12. A method according to claim 11, wherein the two different wavelengths are red and green wavelengths detected by the video camera.
13. A method according to claim 11, wherein the two different wavelengths comprise (i) a red wavelength detected by the video camera, and (ii) an infrared wavelength detected by a second video camera.
14. A non-transistory computer readable storage medium storing software executable by at least one of a processor and a controller, wherein the software includes code adapted to: execute the method of claim 1; and obtain a measurement of one or more vital signs of the subject by PPG imaging the subject, wherein the one or more vital signs include the estimated first vital sign, and wherein ambient light interference associated with the PPG imaging is suppressed by execution of the method of claim 1.
15. A method of measuring one or more vital signs including the first vital sign, the method comprising: PPG imaging the subject using the video camera including suppressing the ambient light interference by performing the method of claim 1; and performing a facial recognition process on an image of the subject obtained by the video camera to link the identity of the subject to the one or more vital signs.
16. A method according to claim 1, further comprising obtaining a measurement of a breathing rate of the subject by low-pass filtering and downsampling the PPG image signal before spectrally analyzing the PPG image signal using a third AR all pole model.
Description
(1) The invention will be further described by way of example with reference to the accompanying drawings in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
(26)
(27) Once the representative intensity for each frame has been obtained, a time series of these intensities is assembled for a series of frames in a time window of, for example, 30 seconds. The length of the time window can be varied, for example from 10 seconds to one minute.
(28) In step 33 a plurality of auto-regressive (AR) models are fitted to each time series (that is to say to each of the red, green and blue time series from ROIr and to each of the red, green and blue time series from ROIs). Assuming a 24 frame per second video camera frame rate, for a 30-second window there will be 720 samples on each of the three channels for the reference background and for the subject.
(29) It may be useful here to give a brief explanation of the general principles of autoregressive (AR) modelling, though AR modelling is well-known, for example in the field of speech analysis.
(30) AR modelling can be formulated as a linear prediction problem where the current value x(n) of the signal can be modelled as a linearly weighted sum of the preceding p values. Parameter p, which is the number of samples over which the sum is taken, is the model order, which is usually much smaller than the length N of the sequence of values forming the signal. Thus:
(31)
(32) The value of the output x(n) is therefore a linear regression on itself, with an error e(n), which is assumed to be normally distributed with zero mean and a variance of .sup.2. More usefully for this application the model can alternatively be visualised in terms of a system with input e(n), and output x(n), in which case the transfer function H can be formulated as shown below:
(33)
(34) As shown in Equation 2, the denominator of H(z) can be factorised into p terms. Each of these terms defines a root z.sub.i of the denominator of H(z), corresponding to a pole of H(z). Since H(z) has no finite zeros, the AR model is an all-pole model. The poles occur in complex-conjugate pairs and define spectral peaks in the power spectrum of the signal. They can be visualised in the complex plane as having a magnitude (distance from the origin) and phase angle (angle with the positive real axis). Higher magnitude poles correspond to higher magnitude spectral peaks and the frequency of each spectral peak is given by the phase angle of the corresponding pole. The phase angle corresponding to a given frequency f, is defined by Equation 3 which shows that it is also dependent on the sampling interval t (reciprocal of the sampling frequency):
=2ft(3)
(35) Thus fitting a suitable order AR model to a signal, and obtaining the poles, reveals the spectral composition of the signal.
(36) To find the poles, the model parameters a.sub.k are first obtained, for example using the Burg or Yule-Walker equations to fit the model to the signal, and from the values of a.sub.k the values of the p poles z.sub.1 to z.sub.p can be calculated (see, for example, Pardey J, Roberts S, Tarassenko L, A review of parametric modelling techniques for EEG analysis, Medical Engineering & Physics, 1996, 18(1), 2-11). The p poles of H(z), which correspond to the p roots z.sub.i (i=1 to p) of the denominator of H(z) are found using standard mathematical procedures (for example, the MATLAB routine roots). As each pole z.sub.k can be written as a complex number x.sub.k+jy.sub.k, the frequency represented by that pole can be calculated from the phase angle of that pole in the upper half of the complex plane:
=tan.sup.1 y/x=2f.sub.k.Math.1/f.sub.s(4) where f.sub.s is the sampling frequency and the magnitude r is (x.sup.2+y.sup.2).sup.1/2.
(37) Thus the AR model fitting of step 33 reveals the dominant spectral components in both the signal from the reference region of interest and the PPG image signal from the subject region of interest. Because the two regions of interest are both imaged by the same camera, any ambient light interference or aliasing artefacts will be present in both the reference and the PPG signal. However the signal from the subject region of interest will additionally have poles corresponding to spectral components representing the PPG signal.
(38) In step 34 any poles in the AR model fitted to the subject data of
(39) Then in step 36 the remaining pole which is closest to the horizontal axis, i.e. has the minimum angle and thus the lowest frequency in the allowed range is identified and the frequency it represents calculated. Alternatively, as represented by step 36a it is possible to obtain the frequency response of the filter characterised by the a.sub.k coefficients of Eq. (1) and to select the frequency which has the largest magnitude in the frequency response. This is the frequency which corresponds to the subject's heart rate. In the data of
(40) These steps are conducted on all of the different order AR models fitted to the same 30-second window of data and in step 37 a robust estimate of the resulting heart rate estimates is obtained, for example the median value. This value is stored and displayed in step 38 and then in step 39 the 30-second window is moved forward by 1 second and steps 33 to 38 repeated. The heart rate estimates are sent in step 40 to the remote server 6.
(41) In
(42) The AR fitting method above also allows for the oxygen saturation to be measured. In
(43)
(44)
(45)
(46)
(47) Previous work on acutely ill patients has shown how the distributions of vital signs in a population of such patients can be used to compute an Early Warning Score, the value of which increases with the severity of illness.
(48) The Early Warning Score was obtained by constructing an alerting system using the hypothesis that an Early Warning Score (EWS) of 3 should be generated when a vital sign is below the 1st centile or above the 99th centile for that variable (for a double-sided distribution), a score of 2 should correspond to the vital sign being between the 1st and 5th centiles or between the 95th and 99th centiles and that a score of 1 should correspond to the vital sign being between the 5th and 10th centiles or between the 90th and 95th centiles. (For SpO.sub.2, with a one-sided distribution starting at 100%, values above the 98th centile will give a score of 3, values between the 90th and 98th centiles a score of 2, and values between the 80th and 90th centiles a score of 1). The vertical lines on the cdf plots of
(49) In the EWS systems currently used in hospitals, the scores for each individual vital sign are quantised with integer precision (i.e. they can only take on a value of 0, 1, 2 or 3). There is no reason why this should be the case as the cdf curves are smooth, however, and in this embodiment of the invention a wellness index with a much smaller quantisation (steps of 0.1 for each vital sign) is used. A set of curves for an EWS system with 0.1 quantisation in the range from 1 to 3 is shown in
(50) A sick in-hospital patient will have a high EWS score (a score of 3 for three vital signs, for example, will give an EWS of 9). In this embodiment a wellness score is calculated instead which decreases with vital sign abnormality. For example, a patient with normal heart rate, normal breathing rate and normal SpO.sub.2, will have a cardio-respiratory wellness index of 10. The further away from the centre of the distributions any vital sign is, the lower the value of the cardio-respiratory wellness index will be. For example, if the wellness index is derived from the heart rate (HR), respiratory rate/breathing rate [RR/BR] and SpO.sub.2, estimated as described above, the wellness index could be obtained from the simple formula:
Index=10.0{score[HR]+score[RR/BR]+score[SpO.sub.2]}
where the score is for each parameter is read off from the y-axis on the plot for that parameter on
(51) In the case of having a measurement of blood pressure also, then the four distributions will be used to derive a cardiovascular index of wellness, also on a scale from 0 to 10.
(52) Over time, it is possible to design a patient-specific set of wellness indices. This requires sufficient vital sign data to be collected, over the full range of daytime hours, so that histograms and cdfs for that individual can be constructed. Once this has been achieved, a centile-based wellness index which is patient-specific can be created.
(53) Another important aspect of this invention is that the vital signs can be uniquely linked to the individual whose physiology they represent, through face recognition software. With the usual methods for the remote monitoring of vital signs, there is no guarantee that the vital signs are those of the individual presumed to have generated them, as the probes or electrodes could be attached to anyone in the vicinity of the individual (with or without their knowledge). With this invention, any uncertainty as to the origin of the vital signs is removed as the face of the subject is captured by the camera during the estimation of the values of the vital signs.
(54) While the embodiments of the invention above have concentrated on use by subjects at home, they are equally applicable to use in a hospital setting. For good signals to be obtained the subject needs to be relatively still in front of the camera, but in a hospital this can be the case in a critical care or neo-natal unit and thus the invention is useful in these cases too. The invention is applicable in any PPG imaging situation. For example PPG imaging could be used for screening for those suffering from infections which often elevates heart rate and breathing rate, such screening being useful at for example points of entry such as ports, airports and building entrances. It can also be useful as part of the parameter detection used in lie detection.