SYSTEM AND METHOD FOR DETERMINING AT LEAST ONE VITAL SIGN OF A SUBJECT

Abstract

The present invention relates to a system and method for determining at least one vital sign of a subject. To improve the ambient light robustness, said system comprises an illumination unit (2) comprising a plurality of radiation sources (21, 22), which are configured to emit electromagnetic radiation and which are controllable individually or in groups, a control unit (3) configured to control the radiation sources (21, 22) of the illumination unit (2), wherein the control unit (3) is configured to control one or more of the direction, the beam width, and the strength of a radiation beam emitted by a radiation source based on skin region information indicating the location of one or more skin regions of the subject and/or based on vital sign information indicating the quality of the extracted vital sign to illuminate the one or more skin regions of the subject, a detection unit (4) configured to detect high-frequency variations in electromagnetic radiation reflected from the skin region of the subject and to generate a detection signal from the detected high-frequency variations in the electromagnetic radiation, and a vital signs determination unit (5) configured to extract a vital sign from the detection signal.

Claims

1. A system for determining at least one vital sign of a subject, said system comprising: an illuminator comprising a plurality of radiation sources, which are configured to emit electromagnetic radiation and which are controllable individually and/or in groups, a controller for controlling the radiation sources of the illuminator, wherein the controller is configured to control one or more of the direction, the beam width, and the strength of a radiation beam emitted by a radiation source based on skin region information indicating the location of one or more skin regions of the subject and/or based on vital sign information indicating the quality of the extracted vital sign to illuminate the one or more skin regions of the subject, a detector for detecting high-frequency variations in electromagnetic radiation reflected from the skin region of the subject and to generate a detection signal from the detected high-frequency variations in the electromagnetic radiation, and a vital signs determinator for extracting a vital sign from the detection signal.

2. The system according to in claim 1, wherein the detector detects high-frequency variations in the electromagnetic radiation reflected from the skin region of the subject in a frequency range higher than 1 kHz.

3. The system according to claim 1, wherein the detector comprises a single photo-diode or an array of photo-diodes.

4. The system according to claim 1, wherein the illuminator comprises a pixelated light source.

5. The system according to claim 1, wherein the illuminator emits modulated electromagnetic radiation.

6. The system according to claim 5, wherein the illuminator modulates electromagnetic radiation at a modulation frequency in the range higher than 1 kHz.

7. The system according to claim 1, wherein the illuminator emits electromagnetic radiation at several different wavelengths or in several different wavelength ranges.

8. The system according to claim 7, wherein the illuminator differently modulates the electromagnetic radiation of the different wavelengths or the different wavelength ranges.

9. The system according to claim 8, wherein the illuminator applies a first modulation frequency for modulating electromagnetic radiation of a first wavelength or wavelength range, which is at least 8 Hz apart from a second modulation frequency applied for modulating electromagnetic radiation of a second wavelength or wavelength range.

10. The system according to claim 7, wherein the illuminator differently polarizes the electromagnetic radiation of the different wavelengths or the different wavelength ranges.

11. A method for determining at least one viral sign of a subject, said method comprising: emitting electromagnetic radiation by use of a plurality of radiation sources, which are controllable individually or in groups, controlling the radiation sources, wherein one or more of the direction, the beam width, and the strength of a radiation beam emitted by a radiation source are controlled leased on skin region information indicating the location of one or more skin regions of the subject and/or based on vital sign information indicating the quality of the extracted vital sign to illuminate the one or more skin regions of the subject, detecting high-frequency variations in electromagnetic radiation reflected from the skin region of the subject, generating a detection signal from the detected high-frequency variations in the electromagnetic radiation, and extracting a vital sign from the detection signal.

12. The system according to claim 2, wherein the frequency range is from 10 kHz to 100 MHz.

13. The system according to claim 6, wherein the frequency range is from 10 kHz to 100 MHz.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0040] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. In the following drawings

[0041] FIG. 1 shows a schematic diagram of a first embodiment of a system according to the present invention;

[0042] FIG. 2 shows a schematic diagram of a second embodiment of a system according to the present invention;

[0043] FIG. 3 shows a schematic diagram of a third embodiment of a system according to the present invention; and

[0044] FIG. 4 shows a schematic diagram of a fourth embodiment of a system according to the present invention.

DETAILED DESCRIPTION OF THE INVENTION

[0045] FIG. 1 shows a schematic diagram of a first embodiment of a system 1 for determining at least one vital sign of a subject 50 according to the present invention. The subject 50 may be a patient, e.g. lying in a bed in a hospital or other healthcare facility, but may also be a neonate or premature infant, e.g. lying in an incubator, or person at home or in a different environment.

[0046] The system 1 comprises an illumination unit 2 configured to illuminate a skin region of the subject 50. The illumination unit 2 comprises a plurality of radiation sources 21, 22, e.g. a plurality of LEDs, which are configured to emit electromagnetic radiation and which are controllable individually and/or in groups. In an embodiment the illumination unit 2 comprises a pixelated light source.

[0047] The system 1 further comprises a control unit 3 configured to control the radiation sources 21, 22 of the illumination unit 2. The control unit 3 may e.g. be a controller or processor.

[0048] The system 1 further comprises a detection unit 4 configured to detect high-frequency variations in electromagnetic radiation reflected from the skin region of the subject and to generate a detection signal from the detected high-frequency variations in the electromagnetic radiation. The detection unit 4 may e.g. be a single photo-diode or an array of photo-diodes.

[0049] Finally, the system 1 comprises a vital signs determination unit 5 configured to extract a vital sign from the detection signal. The vital signs determination unit 5 may e.g. be a processor or computer or dedicated hardware.

[0050] In an embodiment, at least two (generally N) different carriers (HF variations) are used. For each carrier a demodulation is performed (mixing each carrier to DC followed by low-pass filtering (the typical bandwidth equals 4 Hz, i.e. the width of the pulse rate frequencies). Next the low-pass filtered, demodulated signals are normalized (e.g. divided by temporal mean over an analysis interval (or logarithm)). The resulting normalized N signals are input to a pulse extraction unit (e.g. a PBV-method, CHROM, BSS, etc., as will be explained below in more detail) in case the N signals correspond to N different wavelengths (in other embodiments, different polarization directions may be used instead of wavelength signals, requiring some adaptations). Instead of deriving a pulse signal, it is also possible to derive a respiration signal, or an estimate of the oxygenation level of the arterial blood.

[0051] The system 1 may optionally further comprise an interface 6 for displaying the determined information and/or for providing medical personnel with an interface to change settings of the system's components. Such an interface may comprise different displays, buttons, touchscreens, keyboards or other human machine interface means.

[0052] A system 1 as illustrated in FIG. 1 may, e.g., be located in a hospital, healthcare facility, elderly care facility or the like. Apart from the monitoring of patients, the present invention may also be applied in other fields such as neonate monitoring, general surveillance applications, security monitoring or so-called live style environments, such as fitness equipment, a wearable, a handheld device like a smartphone, or the like. The uni- or bidirectional communication between two or more components of the system 1 may work via a wireless or wired communication interface.

[0053] The embodiment shown in FIG. 1 is a low-cost embodiment, particularly suitable for automotive applications. The illumination unit 2 and the detection unit 4 are viewing the scene with the subject 50 through a common optical element 7, e.g. a common lens. Since the face of the subject 50 is relatively close to the illumination unit 2 it reflects far more light than the surfaces behind the subject. A semi-transparent mirror 8 may be used in such an exemplary embodiment, but may in practice not sufficiently suppress direct light to the detection unit 4, i.e. separate lenses for the illumination unit 2 and the detection unit 4 may instead be used.

[0054] Typically, the electromagnetic radiation is in the range of 400 nm to 1000 nm for pulse, respiration and blood oxygen saturation measurement, particularly in the range of 620 nm to 920 nm. This particular range is most suitable for SpO2 measurement and is attractive for unobtrusive monitoring during sleep (for near darkness further limitation to a range of 760 nm to 920 nm may be preferred), but the visible part of the spectrum may allow a higher quality in case visibility of the light is not obtrusive (i.e. NIR is not necessarily the preferred option in all cases). The detection signals may be acquired by a photo-sensor (or a photo-sensor array) remotely sensing the subject's skin.

[0055] By use of the illumination unit 2, illumination of non-skin surfaces can in principle be prevented. Various methods can be used to realize this, but an example is that the pixelated light source is embedded in a feedback loop, where the quality (e.g. SNR) of the extracted PPG signal is optimized by varying the intensity of “pixels” of the pixelated light source. In this case, a photo-diode suffices to sense the area including some skin segments of the subject to extract the PPG signal.

[0056] In practical embodiments, the illumination unit 2 may emit radiation in at least two or three different wavelength intervals (RGB, or invisible wavelengths using NIR), preferably in a modulated (frequency or phase) fashion to enable ambient light robustness. The (pseudo)-color signals can then be demodulated robustly from these modulated wavelength channels (or alternatively be obtained from different photo-diodes equipped with different optical filters), and the PPG signal can be extracted from the wavelength channels motion-robustly using the same basic algorithms that have been proposed for camera-based extraction (such as ICA, PCA, CHROM, POS, (A)PBV-method, etc.), as will be explained in more detail below.

[0057] To achieve a good robustness for ambient illumination variations, the illumination unit 2 can be modulated at a relatively high frequency, e.g. between 1 kHz and several MHz, where the ambient light spectrum typically can be expected to be fairly clean, and thus cause no interference with the demodulated wavelength channels. Because the detector is a high-speed detector, e.g. a photo-diode, rather than the imaging sensor typically used in remote PPG applications, these high frequencies are easily feasible. Different wavelengths can be modulated at different frequencies, although the channels need not be separated very far. Around 8 Hz separation could already suffice, as the sidebands due to the pulse signal is limited to about +/−4 Hz for the maximum human pulse rate.

[0058] In another embodiment ambient light robustness is improved by using a detection unit, e.g. a photo-detector, equipped with an optical filter that selectively transmits the wavelengths used in the modulated illumination unit, but blocks all (or most) other ambient light, e.g. by means of a visible light-blocking filter.

[0059] A further embodiment may increase ambient light robustness by using separate photo-detectors for each wavelength used, where each photo-detector may be equipped with an illumination unit matching optical filter.

[0060] To further improve ambient light robustness or decrease sensitivity for specular reflections, polarizers may be used in front of the detection unit and/or the illumination unit. Also having two signals obtained with different polarization filters may in itself allow for a combination of the (partly independent) signals that is cleaner than the individual signals. This enables options with a single wavelength, or provides two times more detection signals to be combined than without polarizers.

[0061] The control unit 3 for controlling the illumination unit 2 may be configured to control the direction and/or width and/or intensity of the (generally relatively narrow) beam 30 from the illumination unit 2 so that e.g. as much skin as possible and as little non-skin surfaces as possible are illuminated, using the same optimization criterion. Preferably, the radiation elements 21, 22 can be controlled individually or in groups to achieve this effect.

[0062] In the embodiment of the system 1 shown in FIG. 1 different wavelengths can be multiplexed or multiple LEDs (different wavelengths) modulated with different frequencies. Very high modulation frequencies (kHz-MHz) are possible, and a good separation is possible using e.g. a product detector (or heterodyne demodulator). For instance, demodulation may be effected by computing the inner-product over a sliding analysis window with a sine- and a cosine-wave, and computing the square root of the squared inner-products, and finally low-pass filtering (e.g. 4 Hz cut-off) the resulting values from the sliding window processing. The high modulation frequencies prevent problems with motion, and can make the system very robust for ambient illumination, i.e. the frequency selectivity makes the system “blind” for ambient illumination variations, which are typically lower than 1 kHz (50/100 Hz for classical light sources, several hundreds of Hz for LED light), and lower than 100 kHz. In this case, the relative proximity of the face (compared to background) can prevent the necessity for pixelated light, or alternatively a highly absorbing background immediately behind the subject can prevent damaging contributions from non-skin surfaces.

[0063] FIGS. 2 to 4 show schematic diagrams of a further embodiments of a system according to the present invention. In these figures, mainly the illumination unit and the detection unit are illustrated, while other components of the system like the control unit and the vital signs detection unit are omitted.

[0064] FIG. 2 shows a schematic diagram of a second embodiment of a system 100 according to the present invention. In this embodiment, it is possible to have the illumination unit 2 cast a narrow beam 30 towards the subject 50, e.g. a beam 30 that is wide enough to only allow the necessary freedom to move as a car driver, and have the detection unit 4 view the subject 50 with a similarly narrow viewing angle 40. Hence, different from the system 1, the light path of source and sensor are separate in this embodiment. The face is illuminated most because it is closest to the illumination unit 2, hence the background has little effect on the signal quality (or highly absorbing background (e.g. headrest)).

[0065] Both light paths 30 and 40 are narrow and, provided their intersection mainly contains skin, the PPG signal will be of good quality even if the background is not black. Using a dark background may be a viable option in some applications too. If the background plays a significant part in the reflected light from the scene, it is still possible to get a high quality PPG signal using the variance of the light in a few sensors (rather than the mean, i.e. a single photo diode).

[0066] Using a photo-diode as detection unit, its detection signal can be synchronously demodulated in the kHz-range. For instance, a 3-wavelength system (e.g. using 760 nm, 800 nm and 900 nm) modulated in frequency multiplex, can have it central frequencies around 10 kHz with the wavelength channels being at least 8 Hz apart, e.g. 10, 11, or 12 kHz). This separation is preferred due to the range of possible pulse frequencies (0.5-4 Hz, which gives +/−4 Hz side-bands to the modulation frequencies. Therefore, they should be 8 Hz apart to prevent interference.

[0067] There are other interference considerations though. Non-linearities in the radiation source (e.g. LEDs) cause harmonics of the modulation frequencies, which also should not cause interference in the pulse band of other wavelengths (e.g. a 10 kHz modulation frequency causes harmonics at 20, 30, 40, etc. kHz). If any of the demodulators mixes down any of these harmonics into the pulse rate band interference results (e.g. when using 10 kHz for wavelength 1, other modulation frequencies should also stay clear of 40 kHz+/−32 Hz to prevent interference).

[0068] Hence, there may be various constraints to choosing the modulation frequencies. There is not only a lower bound, but there are many frequency bands which should not be used, as becomes clear from the above example.

[0069] It is also possible to modulate the received light with different polarization directions (cross and parallel to the polarization of the light of the illumination beam 20). For this purpose polarizers 9 and 10 may be used. The use of polarized light for illumination and a cross-polarizer 10 in front of or as part of the detection unit 4 (in particular in front of the sensor that senses the received radiation) may help suppress specularly reflected light. Also, the use of polarizers may (partially) replace the use of extra wavelengths to improve motion robustness and the SNR of the PPG signal.

[0070] In order to suppress specular reflection it suffices to use a single polarizer for all radiation sources and an orthogonal (cross) polarizer in front of the detection unit. To improve motion robustness, multiple options exists:

[0071] a) use a single polarizer in front of all radiation sources and a cross polarizer and a parallel polarizer in front of two detection elements (forming the detection unit);

[0072] b) alternate (switch) the polarization in front of a single detection element (forming the detection unit);

[0073] c) use a single polarizer in front of the detection unit and switch the polarization in front of the radiation sources; or

[0074] d) use multiple polarizers on parallel radiation sources.

[0075] Generally, as polarizer a polarization filter may be used. However, for this purpose, not only transmission filters can be employed, but reflectors and/or mirrors (e.g. polarization mirrors) may be used to achieve the same effect.

[0076] FIG. 3 shows a schematic diagram of a third embodiment of a system 200 according to the present invention. In this embodiment multiple illumination units 2a, 2b are used each with a relatively narrow beam 30a, 30b and optionally each with a polarizer 9a, 9b. Only objects in the intersection of the narrow beams 30a, 30b with the narrow sensor viewing angle 40 will contribute to the detection signal.

[0077] FIG. 4 shows a schematic diagram of a fourth embodiment of a system 300 according to the present invention. This embodiment is particularly suitable for automotive application since background illumination can be prevented altogether. The subject's face is illuminated from the sides, preferably with polarized light. The detection unit 4 is equipped with a cross-polarizer 10 and consequently only sees reflections that have been depolarized by translucent objects in its view. In the car it should be relatively easy to prevent any other (significantly reflecting) objects in the view, so that only the scattered light reflected from the skin is “seen” by the detection unit 4.

[0078] A further alternative embodiment without a pixelated light source as detection unit, employs a very narrow beam (narrow enough to cause an illuminated area that is smaller than the aimed for body-part (e.g. a face)) of light that can be directed by a controller. In this embodiment the detection unit may “see” a larger environment, but it can nonetheless give a good quality PPG signal if the spot is reaching the skin of a subject only. Possibly, the controller changes the orientation of the light beam, in this way optimizing the PPG signal quality.

[0079] Generally, a PPG signal results from variations of the blood volume in the skin. Hence, the variations give a characteristic pulsatility “signature” when viewed in different spectral components of the reflected/transmitted light. This “signature” is basically resulting as the contrast (difference) of the absorption spectra of the blood and that of the blood-less skin tissue. If the detector, e.g. a sensor, has a discrete number of color channels, each sensing a particular part of the light spectrum, then the relative pulsatilities in these channels can be arranged in a “signature vector”, also referred to as the “normalized blood-volume vector”, PBV. It has been shown in G. de Haan and A. van Leest, “Improved motion robustness of remote-PPG by using the blood volume pulse signature”, Physiol. Meas. 35 1913, 2014, which is herein incorporated by reference, that, if this signature vector is known, then a motion-robust pulse signal extraction on the basis of the color channels and the signature vector is possible. For the quality of the pulse signal, it is essential though that the signature vector is accurate, as otherwise the known methods mixes noise into the output pulse signal in order to achieve the prescribed correlation of the pulse vector with the normalized color channels as indicated by the signature vector.

[0080] Details of the PBV method and the use of the normalized blood volume vector (called “predetermined index element having a set orientation indicative of a reference physiological information”) have also been described in US 2013/271591 A1, whose details are also herein incorporated by reference.

[0081] The characteristic wavelength-dependency of the PPG signal varies when the composition of the blood changes. Particularly, the oxygen saturation of the arterial blood has a strong effect on the light absorption in the wavelength range between 620 nm and 780 nm. This changing signature for different SpO2 values leads to relative PPG pulsatility that depends on the arterial blood oxygen saturation. This dependency can be used to realize a motion-robust remote SpO2 monitoring system that has been named adaptive PBV method (APBV) and is described in detail in M. van Gastel, S. Stuijk and G. de Haan, “New principle for measuring arterial blood oxygenation, enabling motion-robust remote monitoring”, Nature Scientific Reports, November 2016. The description of the details of the APBV method in this document is also herein incorporated by reference.

[0082] The PBV method gives the cleanest pulse signal when the PBV vector, reflecting the relative pulsatilities in the different wavelength channels is accurate. Since this vector depends on the actual SpO2 value, testing the PBV method with different PBV vectors, corresponding to a range of SpO2 values, the SpO2 value results as the one corresponding to the PBV vector giving the pulse-signal with the highest SNR.

[0083] In the following some basic considerations with respect to the PBV method shall be briefly explained, using RGB-wavelength channels as an example.

[0084] The beating of the heart causes pressure variations in the arteries as the heart pumps blood against the resistance of the vascular bed. Since the arteries are elastic, their diameter changes in synchrony with the pressure variations. These diameter changes occur not only in the big arteries, but even in the smaller vessels of the skin at the arterial side of the capillary bed, the arterioles, where the blood volume variations cause a changing absorption of the light.

[0085] The unit length normalized blood volume pulse vector (also called signature vector) is defined as PBV, providing the relative PPG-strength in the red, green and blue camera signal, i.e.

[00001]${\stackrel{.fwdarw.}{P}}_{\mathrm{bv}}=\frac{\left[\sigma \left({\stackrel{.fwdarw.}{R}}_n\right),\sigma \left({\stackrel{->}{G}}_n\right),\sigma \left({\stackrel{.fwdarw.}{B}}_n\right)\right]}{\sqrt{{\sigma }^2\left({\stackrel{.fwdarw.}{R}}_n\right)+{\sigma }^2\left({\stackrel{.fwdarw.}{G}}_n\right)+{\sigma }^2\left({\tilde{B}}_n\right)}}$

with σ indicating the standard deviation.

[0086] To quantify the expectations, the responses H.sub.red(w), H.sub.green(w) and H.sub.blue(w) of the red, green and blue channel, respectively, were measured as a function of the wavelength w, of a global-shutter color CCD camera, the skin reflectance of a subject, ρ.sub.s(w), and used an absolute PPG-amplitude curve PPG(w). From these curves, shown e.g. in FIG. 2 of the above cited paper of de Haan and van Leest, the blood volume pulse vector PBV is computed as:

[00002]${\stackrel{\stackrel{->}{^}}{P}}_{\mathrm{bv}}^T=\left[\begin{array}{c}\frac{{\int }_{w=400}^{700}{H}_{\mathrm{red}}\left(w\right)I\left(w\right)\mathrm{PPG}\left(w\right)\mathrm{dw}}{{\int }_{w=400}^{700}{H}_{\mathrm{red}}\left(w\right)I\left(w\right){\rho }_s\left(w\right)\mathrm{dw}}\\ \frac{{\int }_{w=400}^{700}{H}_{\mathrm{green}}\left(w\right)I\left(w\right)\mathrm{PPG}\left(w\right)\mathrm{dw}}{{\int }_{w=400}^{700}{H}_{\mathrm{green}}\left(w\right)I\left(w\right){\rho }_s\left(w\right)\mathrm{dw}}\\ \frac{{\int }_{w=400}^{700}{H}_{\mathrm{blue}}\left(w\right)I\left(w\right)\mathrm{PPG}\left(w\right)\mathrm{dw}}{{\int }_{w=400}^{700}{H}_{\mathrm{blue}}\left(w\right)I\left(w\right){\rho }_s\left(w\right)\mathrm{dw}}\end{array}\right]$

which, using a white halogen illumination spectrum I(w), leads to a normalized PBV=[0.27, 0.80, 0.54].

[0087] The blood volume pulse predicted by the used model corresponds reasonably well to an experimentally measured normalized blood volume pulse vector, PBV=[0.33, 0.78, 0.53] found after averaging measurements on a number of subjects under white illumination conditions. Given this result, it was concluded that the observed PPG-amplitude, particularly in the red, and to a smaller extent in the blue camera channel, can be largely explained by the crosstalk from wavelengths in the interval between 500 and 600 nm. The precise blood volume pulse vector depends on the color filters of the camera, the spectrum of the light and the skin-reflectance, as the model shows. In practice, the vector turns out to be remarkably stable though given a set of wavelength channels (the vector will be different in the infrared compared to RGB-based vector).

[0088] It has further been found that the relative reflectance of the skin, in the red, green and blue channel under white illumination does not depend much on the skin-type. This is likely because the absorption spectra of the blood-free skin is dominated by the melanin absorption. Although a higher melanin concentration can increase the absolute absorption considerably, the relative absorption in the different wavelengths remains the same. This implies an increase of melanin darkens the skin but hardly changes the normalized color of the skin. Consequently, also the normalized blood volume pulse PBV is quite stable under white illumination. In the infrared wavelengths, the influence of melanin is further reduced as its maximum absorption occurs for short wavelengths (UV-light) and decreases for longer wavelengths.

[0089] The main reason the PBV vector is not affected much by the melanin is that melanin is in the epidermis and therefore acts as an optical filter on both the AC and the DC. Hence, it may reduce the pulsatility, but at the same time also the DC value of the reflection. Hence the AC/DC (relative pulsatility) does not change at all.

[0090] The stable character of PBV can be used to distinguish color variations caused by blood volume change from variations due to alternative causes, i.e. the stable PBV can be used as a “signature” of blood volume change to distinguish their color variations. The known relative pulsatilities of the color channels PBV can thus be used to discriminate between the pulse-signal and distortions. The resulting pulse signal S using known methods can be written as a linear combination (representing one of several possible ways of “mixing”) of the individual DC-free normalized color channels:

S=WC.sub.n

with WW.sup.T=1 and where each of the three rows of the 3×N matrix C.sub.n contains N samples of the DC-free normalized red, green and blue channel signals R.sub.n, G.sub.n and B.sub.n, respectively, i.e.:

[00003]${\stackrel{.fwdarw.}{R}}_n=\frac{1}{\mu \left(\stackrel{.fwdarw.}{R}\right)}\stackrel{.fwdarw.}{R}-1,{\stackrel{.fwdarw.}{G}}_n=\frac{1}{\mu \left(\stackrel{.fwdarw.}{G}\right)}\stackrel{.fwdarw.}{G}-1,{\stackrel{.fwdarw.}{B}}_n=\frac{1}{\mu \left(\stackrel{.fwdarw.}{B}\right)}\stackrel{.fwdarw.}{B}-1.$

Here the operator μ corresponds to the mean. Key difference between the different methods is in the calculation of the weighting vector W. In one method, the noise and the PPG signal may be separated into two independent signals built as a linear combination of two color channels. One combination approximated a clean PPG signal, the other contained noise due to motion. As an optimization criterion the energy in the pulse signal may be minimized. In another method a linear combination of the three color channels may be used to obtain the pulse signal.

[0091] The PBV method generally obtains the mixing coefficients using the blood volume pulse vector as basically described in US 2013/271591 A1 and the above cited paper of de Haan and van Leest. The best results are obtained if the band-passed filtered versions of R.sub.n, G.sub.n and B.sub.n are used. According to this method the known direction of PBV is used to discriminate between the pulse signal and distortions. This not only removes the assumption (of earlier methods) that the pulse is the only periodic component in the video, but also eliminates assumptions on the orientation of the distortion signals. To this end, it is assumed as before that the pulse signal is built as a linear combination of normalized color signals. Since it is known that the relative amplitude of the pulse signal in the red, green and blue channel is given by PBV, the weights, W.sub.PBV, are searched that give a pulse signal S, for which the correlation with the color channels R.sub.n, G.sub.n, and B.sub.n equals PBV

{right arrow over (S)}C.sub.n.sup.T=k{right arrow over (P)}.sub.bv⇔{right arrow over (W)}.sub.PBVC.sub.nC.sub.n.sup.T=k{right arrow over (P)}.sub.bv,  (1)

and consequently the weights determining the mixing are determined by

{right arrow over (W)}.sub.PBV=k{right arrow over (P)}.sub.bvQ.sup.−1 with Q=C.sub.nC.sub.n.sup.T,  (2)

and the scalar k is determined such that W.sub.PBV has unit length. It is concluded that the characteristic wavelength dependency of the PPG signal, as reflected in the normalized blood volume pulse, PBV, can be used to estimate the pulse signal from the time-sequential RGB pixel data averaged over the skin area. This algorithm is referred to as the PBV method.

[0092] In other words, the weights indicate how the detection signals should be (linearly) combined in order to extract a pulse signal from the detection signals. The weights are unknown and need to be computed/selected.

[0093] The signature vector (PBV vector) represent the given (known or expected) relative pulsatilities in different wavelength channels (i.e. the detection signals), caused by the absorption spectrum of the blood and the penetration of light into the skin (if photons are more absorbed by blood, a volume change of blood leads to a larger signal than when the blood is nearly transparent). With this knowledge, and the observed data (i.e. the detection signals) the weights (e.g. a weight vector) can be determined. The resulting weights are data dependent, i.e. depend on the detection signals.

[0094] Since the pulse signal has a different ratio AC/DC (this is also called the relative signal strength/pulsatility) in each wavelength channel, it can be seen that the spectrum shows the pulse peak in the spectrum with different peak values for the different colors. This spectrum is the result of a Fourier analysis, but it basically means that if a sinusoid having the pulse frequency is correlated (multiplied) with the detection signals (RGB in the example, NIR-wavelengths for SpO2), exactly the peak values in the spectrum are obtained, which by definition are called the signature vector (PBV vector): these peak values are the relative strength of the normalized amplitudes of the pulse signal in the different detection signals.

[0095] The consequence of this is that a clean pulse signal S can be obtained (assuming the pulse signal is the result of a weighted sum of the detection signals), using this prior knowledge (i.e. the signature vector). One option to do this is to compute an inversion of a covariance matrix Q of the normalized detection signals C.sub.n. Hence, the weights W to linearly mix the detection signals in order to extract the pulse signal S can be computed from the covariance matrix of the detection signals in the current analysis window (Q, which is data dependent, i.e. changes continuously over time), using the constant signature vector PBV.

[0096] It is recognized that e.g. in the NIR-light spectrum, particularly between 620 and 770 nm, the blood absorption spectrum changes depending on the SpO2 level. For this reason it is proposed to extract the pulse signal with different signature vectors (different PBV vectors), where each PBV vector is chosen to correspond with the relative pulsatilities in the detection signals for a particular vital sign value, e.g. an SpO2 value. Since the extracted pulse signal quality depends on the correctness of the PBV vector, the different extracted pulse signals will have a different quality. By selecting the best quality pulse signal, the vital sign value (e.g. SpO2 value) from the signature vector that caused this favorable pulse signal.

[0097] In another embodiment the APBV method is used to extract an SpO2 value from two or more different combinations of three wavelength channels, e.g. from [λ1, λ2], from [λ1, λ3], and/or from [λ1, λ2, λ3]. In the following some basic considerations with respect to the APBV method shall be briefly explained.

[0098] Instead of extracting features from the PPG waveforms, APBV determines SpO2 indirectly based on the signal quality of the pulse signals extracted with SpO2 ‘signatures’. This procedure can mathematically be described as:

[00004]$\begin{array}{cc}{\mathrm{SpO}}_2=\underset{{\mathrm{SpO}}_2\in {\mathrm{SpO}}_2}{\mathrm{argmax}}\mathrm{SNR}\left(\stackrel{\stackrel{{\stackrel{->}{W}}_{\mathrm{PBV}}}{︷}}{{\mathrm{kPbv}\left({\mathrm{SpO}}_2\right)\left[C_n{C}_n^T\right]}^{-1}}{C}_n\right),& \left(3\right)\end{array}$

where C.sub.n contains the DC-normalized color variations and scalar k is chosen such that {right arrow over (W)}.sub.PBV has unit length. The SpO2 signatures compiled in {right arrow over (P)}.sub.bv can be derived from physiology and optics. Assuming identical cameras the PPG amplitudes of N cameras can be determined by:

[00005]$\begin{array}{cc}\stackrel{->}{P_{\mathrm{bv}}}=.Math.\left[\begin{array}{c}{\left(\frac{\mathrm{AC}}{\mathrm{DC}}\right)}^1\\ {\left(\frac{\mathrm{AC}}{\mathrm{DC}}\right)}^2\\ \mathrm{.Math.}\\ {\left(\frac{\mathrm{AC}}{\mathrm{DC}}\right)}^N\end{array}\right].Math.=.Math.\left[\begin{array}{c}\frac{{\int }_{\lambda }I\left(\lambda \right)F^1\left(\lambda \right)C\left(\lambda \right)\mathrm{PPG}\left(\lambda \right)d\lambda }{{\int }_{\lambda }I\left(\lambda \right)F^1\left(\lambda \right)C\left(\lambda \right){\rho }_s\left(\lambda \right)d\lambda }\\ \frac{{\int }_{\lambda }I\left(\lambda \right)F^2\left(\lambda \right)C\left(\lambda \right)\mathrm{PPG}\left(\lambda \right)d\lambda }{{\int }_{\lambda }I\left(\lambda \right)F^2\left(\lambda \right)C\left(\lambda \right){\rho }_s\left(\lambda \right)d\lambda }\\ \mathrm{.Math.}\\ \frac{{\int }_{\lambda }I\left(\lambda \right)F^N\left(\lambda \right)C\left(\lambda \right)\mathrm{PPG}\left(\lambda \right)d\lambda }{{\int }_{\lambda }I\left(\lambda \right)F^N\left(\lambda \right)C\left(\lambda \right){\rho }_s\left(\lambda \right)d\lambda }\end{array}\right].Math..& \left(4\right)\end{array}$

Here the PPG amplitude spectrum, PPG(λ), can be approximated by a linear mixture of the light absorption spectra from the two most common variants of the main chromophore in arterial blood, hemoglobin; oxygenated (HbO2) and reduced (Hb):

[00006]$\begin{array}{cc}\begin{array}{c}\mathrm{PPG}\left(\lambda \right)\approx {ϵ}_{\mathrm{Hb}}\left(\lambda \right)c_{\mathrm{Hb}}+{ϵ}_{{\mathrm{HbO}}_2}\left(\lambda \right)c_{{\mathrm{HbO}}_2}=\left(1-{\mathrm{SaO}}_2\right){ϵ}_{\mathrm{Hb}}\left(\lambda \right)+{\mathrm{SaO}}_2{ϵ}_{{\mathrm{HbO}}_2}\left(\lambda \right)\\ ={ϵ}_{\mathrm{Hb}}\left(\lambda \right)+{\mathrm{SaO}}_2\left[{ϵ}_{{\mathrm{HbO}}_2}\left(\lambda \right)-{ϵ}_{H_b}\left(\lambda \right)\right],\end{array}& \left(5\right)\end{array}$

where it is assumed that the optical path length differences are negligible for 600<λ<1000 nm and SaO2 ∈[0, 1]. It is recognized that the wavelength-dependent effect of scattering could render this assumption invalid. When using two wavelengths, the ratio-of-ratios parameter R and the ratio of APBV parameter {right arrow over (P)}.sub.bv coincide. The wavelength selection may be based on three criteria: 1) the desire to measure oxygen saturation in darkness (λ>700 nm) for clinical applications, 2) have a reasonable SpO2 contrast, and 3) wavelengths within the spectral sensitivity of the camera. The idea to use three instead of the common two wavelengths used in pulse-oximetry was motivated by the improved robustness of the SpO2 measurement by a factor of two. This can be explained by how motion affects the PPG waveforms when measured with a camera. Since motion-induced intensity variations are equal for all wavelengths, suppression of these artifacts is possible for the APBV method if the pulse signature {right arrow over (P)}.sub.bv is not equal to this motion signature, which can be described as a vector with equal weights.

[0099] It shall be noted that even if the pulse quality is very good, it does not always mean that the estimated SpO2 value is sufficiently reliable and can be trusted. This may particularly happen when unexpected blood-species (e.g. COHb) are available causing the SpO2 calibration curve to shift, i.e. causing a different signature vector to lead to the optimal pulse quality when using the PBV method or APBV method for pulse extraction.

[0100] It is critical in the above, that the PBV method assumes the relative pulsatilities in the wavelength channels are known, which is true if the desired vital sign information, e.g. the SpO2, were known. This however, in SpO2 monitoring, essentially is not the case since this is the parameter that is searched for. If the weights are chosen correctly, the correlation of the resulting pulse with the individual detection signals C.sub.n are exactly these relative strengths of the pulse in detection signals C.sub.n. Now, if the vital sign information (e.g. the SpO2) is wrong or unknown, the result will be a pulse signal with a relatively poor SNR (i.e. a poor quality indicator).

[0101] The essence of the APBV method is to run a number of PBV methods in parallel with different PBV vector. The PBV method gives the cleanest pulse signal when the PBV vector, reflecting the relative pulsatilities in the different wavelength channels is accurate. Since this vector depends on the actual SpO2 value, testing the PBV method with different PBV vectors, corresponding to a range of SpO2 values, the SpO2 value results as the one corresponding to the PBV vector giving the pulse signal with the highest SNR.

[0102] Although the ABPV method is revolutionary in that it allows for the first time a motion robust remote SpO2 measurement, there is a weakness which limits the robustness. As can be seen from the diagram shown in FIG. 1, the relative pulsatilities in the three wavelength-′channels become more similar when the oxygenation level of the blood drops. The motion robustness of the PBV method is indicated by the angle between the PBV vector and the vector [1 1 1] that corresponds to the dominant relative strength of the motion-induced signal component in the three wavelength channels. The smaller this angle, the less the motion robustness.

[0103] Typically, the electromagnetic radiation used according to the present invention is in the range of 400 nm to 1000 nm for pulse, respiration and blood oxygen saturation measurement, particularly in the range of 620 nm to 920 nm. This particular range is most suitable for SpO2 measurement and is attractive for unobtrusive monitoring during sleep (darkness), but if pulse or respiratory signals are required, the visible part of the spectrum may allow a higher quality (i.e. NIR is not necessarily the preferred option in all cases).

[0104] The above described methods can be applied on detection signals that have been acquired using contactless sensors. By way of example, the present invention can be applied in the field of healthcare, e.g. unobtrusive remote patient monitoring, general surveillances, security monitoring and so-called lifestyle environments, such as fitness equipment or the like. Applications may include monitoring of oxygen saturation (pulse oximetry), pulse rate, blood pressure, cardiac output, respiration, blood perfusion variations, assessment of autonomic functions, and detection of peripheral vascular diseases. The present invention can e.g. be used for rapid and reliable pulse detection of a critical patient, for instance during automated CPR (cardiopulmonary resuscitation). The system can be used for monitoring of vital signs of neonates with very sensitive skin e.g. in NICUs and for patients with damaged (e.g. burnt) skin, but may also be more convenient than contact sensors as used in the general ward, and offer better solutions for motion robustness. Further application is in the automotive field. The present invention particularly solves issues with ambient-light robust multiple wavelength systems, offers a lower cost solution and eliminates privacy concerns.

[0105] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

[0106] In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0107] Any reference signs in the claims should not be construed as limiting the scope.