System and method for determining vital sign information of a subject

Abstract

Systems and methods for remote photoplethysmography and in particular for determining vital sign information of a subject are provided. The system includes a marker that is applied to a skin of the subject. The marker further includes a first marker area configured to transmit light at a first wavelength and a second marker area configured to transmit light at a second wavelength. The system further includes a detection unit that detects radiation received from the first marker area and from the second marker area of the marker. The system further includes an analysis unit that determines the vital sign information of the subject from the detected radiation from the first marker area and from the second marker area.

Claims

1. A system for determining vital sign information of a subject comprising: a marker that is configured to be applied to a skin of the subject, said marker further comprising: a carrier layer; a first marker area carried by a portion of the carrier layer, the first marker area including a first optical filter plate attached to the carrier layer and configured to transmit light at a first wavelength reflected by tissue of the subject from underneath the carrier layer and not transmit light of a second wavelength, wherein an intensity of the light at the first wavelength passing through the first marker area is modulated by an amount of light at the first wavelength is reflected by the tissue of the subject; and a second marker area carried by a portion of the carrier layer, the second marker area including a second optical filter plate attached to the carrier layer and configured to transmit light at the second wavelength reflected by tissue of the subject from underneath the carrier layer and not transmit light of the first wavelength, wherein an intensity of the light at the second wavelength passing through the second marker area is modulated by an amount of light at the second wavelength is reflected by the tissue of the subject; wherein a portion of the carrier layer is configured to define a reference area with a predefined light reflection characteristic, the reference area being configured to prevent passage of reflected light from the skin of the patient, wherein light is transmitted from the marker from underneath the carrier and through either the first marker area or the second marker area, and wherein the reference area does not comprise the entire carrier layer; a camera configured to detect light of the first wavelength received from the first marker area and light of the second wavelength received from the second marker area of the marker, and generate a video image stream of the marker; one or more computer processors programmed to: identify, the reference area, the first marker area and the second marker area irk the image; and analyze the modulated intensity of the detected first and second wavelength light from the first marker area and from the second marker area in the video image stream to determine the vital sign information; wherein each of the camera and each of the one or more processors are physically displaced from the marker.

2. The system according to claim 1, wherein the marker further comprises a graphical pattern carried b the carrier layer.

3. The system according to claim 1, wherein an attachment layer is configured to attach the carrier layer to the subject, wherein the light of the first wavelength and the light of the second wavelength is directed toward and reflected from the skin of the subject.

4. The system according to claim 1, wherein one of the first and second wavelengths is red and the other is infrared.

5. The system according to claim 1, wherein the light transmitted through the first and second optical filter plates has been reflected by blood in the skin and is encoded with information about the blood in the skin.

6. The system according to claim 1, wherein the carrier layer further comprises an adhesive that is configured to attach the carrier layer to the skin of the subject.

7. The system according to claim 6, wherein the adhesive includes a stimulant configured to increase the blood perfusion in a tissue of the subject in contact with the adhesive.

8. An apparatus for determining vital sign information of a subject comprising: a patch layer configured to be applied to a skin of the subject, a portion of the patch layer being configured to define a reference area with a predefined light reflection characteristic, the reference area being configured to prevent passage of reflected light from the skin of the patient, the reference area not comprising the entire carrier layer; a first marker area including a first optical filter plate applied to the skin of the patient adjacent the patch layer, the first filter plate being configured to transmit light at a first wavelength reflected by tissue of the patient from an area underneath the reference area; a second marker area including a second optical plate applied to the skin of the patient adjacent the patch layer, the second optical plate being configured to transmit light reflected by the tissue of the patient from the area underneath the reference area at a second wavelength that is different from the first wavelength; a camera disposed displaced from the subject and configured to detect light of the first and second wavelengths received from the first and second optical filter plates and generate a video image stream; and at least one image processor disposed displaced from the subject and configured to identify the first marker area and the second marker area from the video image stream from the camera, and configured to determine the vital sign information of the subject from detected time-variant intensities of the first and second wavelength light from the identified first and second marker areas of the video image stream; wherein light emitted from the marker is transmitted from either the first marker area or the second marker area from the area underneath the reference area and not from the reference area.

9. The apparatus according to claim 8, wherein the intensity of the detected light from the first and second marker areas varies in accordance with heart rate and wherein the determined vital sign includes heart rate.

10. The apparatus according to claim 8, wherein the camera is configured to detect light at the first and second wavelengths reflected in the skin and transmitted through the first and second optical filter plates and wherein the processor is programmed to determine a blood oxygen concentration by comparing relative intensities of the detected light of the first and second wavelengths.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. In the following drawings

(2) FIG. 1 shows an exemplary embodiment of the system for determining vital sign information of a subject according to the present invention;

(3) FIG. 2 shows the determination of vital sign information with the system according to the present invention;

(4) FIG. 3 shows a first example of a marker;

(5) FIG. 4 shows a second example of a marker;

(6) FIG. 5 shows a third example of a marker;

(7) FIG. 6 shows an example of a marker with sub-areas;

(8) FIG. 7 shows an alternative example of a marker with sub-areas.

DETAILED DESCRIPTION OF THE INVENTION

(9) FIG. 1 shows an exemplary embodiment of a system 1 for determining vital sign information 7 of a subject 100 according to the present invention. The system 1 comprises a marker 10 that is applied to a skin of the subject 100, a detection unit 2 and an analysis unit 6 as the basic components. In this example, the system for determining vital sign information of the subject is employed in a clinical setting, where the subject 100 lies in bed 103.

(10) The marker 10 further comprises a first marker area 11 configured to transmit light at a first wavelength and a second marker area 12 configured to transmit light at a second wavelength. The detection unit 2 is adapted to detect radiation received from the first marker area 11 and from the second marker area 12 of the marker 10. In this example, the detection unit 2 is connected to an optional image processing unit 4. The detection unit 2 provides radiation data 3 that represents the detected radiation, to the image processing unit 4 in form of a video stream. The image processing unit 4 identifies the first marker area 11 and the second marker area 12 in the radiation data 3. The image processing unit 4 in turn is connected to the analysis unit 6. The image processing unit 4 provides preprocessed radiation data 5 to the analysis unit 6. The preprocessed radiation data 5 in this example comprises information about which region of the images of the video stream of the radiation data 3 depict the first marker area 11 and the second marker area 12. The analysis unit 6 in turn determines the vital sign information 7 of the subject from time-variant intensity in the first marker area 11 and in the second marker area 12. In this example, the vital sign information comprises a heart rate and a blood oxygen saturation.

(11) The image processing unit 4 that identifies the first marker area 11 and the second marker area can also be incorporated into the analysis unit 6. Alternatively, the radiation data 3 is directly provided to the analysis unit 6. In this case, the first marker area 11 and the second marker area 12 can be either determined by manually selecting the marker areas in the images of the video stream. Alternatively, the subject 100 with the marker 10 has to be located in a predetermined position within the field of view of the detection unit 2 such that the first marker area 11 and the second marker area 12 are located at a predetermined position. However, an automated identification of the marker 10 in the radiation data 3 by the image processing unit 4 is preferred.

(12) In the shown example, the marker 10 is directly applied to the bare skin of the forehead 101 of the subject 100. An alternative marker 10 with first marker area 11 and second marker area 12 is located at the left forearm 102 of the subject 100. The size and shape of the marker 10, 10 can be adapted depending on the anatomic location.

(13) The scenery is illuminated by a source of radiation, such as sunlight 7a or an artificial light source 7b. The radiation source 7a, 7b directly or indirectly emits radiation 8a, 8b towards the subject 100. In addition, or in the alternative, the system 1 can also comprise an optional system light source 7c that emits light 8c towards the subject 100. The use of a system light source 7c is particularly beneficial if the ambient light sources 7a, 7b do not provide sufficient light or if the spectrum of the ambient light sources 7a, 7b does not provide sufficient power at the first wavelength and at the second wavelength.

(14) An optional control unit 9 is adapted to control the sensitivity of the detection unit 2 and/or to control the power of the system light source 7c. Because the dynamic range of a detector or image sensor that is used as the detection unit 2 is limited, shutters and electronic offsets may have to be adjusted according to the lighting situation in the observed scene. The system light source 7c can be part of a control loop which sets an optimal operating point of the image sensor of the detection unit 2. Optimal in this context refers to an output signal without signal clipping, no saturation of individual detectors of the image sensors and a good signal-to-noise ratio at least for the detector area corresponding to first and/or second marker area.

(15) FIG. 2 illustrates the determination of vital sign information of a subject with the system 1 according to the present invention. FIG. 2 shows a light source 21, a detection unit 22 and a marker having a first marker area 23 and a second marker area 24. The marker is applied to a skin tissue 104 of a subject. The tissue comprises blood vessels 105.

(16) In this embodiment, the light source 21 emits light at at least a first wavelength (indicated with dotted lines) and a second wavelength (indicated with dashed lines). The first marker area 23 is configured to transmit light at a first wavelength, wherein said first wavelength corresponds to the first wavelength of the light source 21. The second marker area 24 is configured to transmit light at a second wavelength, wherein said second wavelength corresponds to the second wavelength of the light source 21. FIG. 2 sketches two light rays A, B at the first wavelength and two light rays C, D at the second wavelength. Since the first marker area 23 is configured to transmit light at the first wavelength, the ray A passes through the marker and penetrates into the skin 104 of the subject 100. Some of the light is absorbed within the skin 104, while some of the light is reflected or scattered in the tissue and reaches the detection unit 22. The absorption and/or reflection characteristic is time-variant and represents the time-variant perfusion of the tissue 104 with its blood vessels 105.

(17) The detection unit 22 comprises receiving optics, for example a receiver lens, and an array 25 of photo detectors or pixels that form an image sensor. Light that is received from the first marker area is imaged onto a first group or array of pixels 26. Correspondingly, light received from the second marker area 24 is imaged onto a second group of pixels 27.

(18) Since the absorption of light in the tissue 104 is time-variant, the light intensity incident on the image sensor of the detection unit 22 is also time-variant. The time-variant intensity on the area of pixels 26 is depicted by curve 28. The time-variant intensity incident on the group of pixels 27 is depicted by curve 29.

(19) Since the first marker area is configured to transmit light at the first wavelength only, light at the second wavelength does not pass the marker area and penetrate into the tissue as indicated by light ray C. Nonetheless, some of the light can be scattered back at the marker surface and reach the detection unit 22. This light is not modulated by the pulsatile variation of the blood volume and gives an offset. The intensity modulation depicted by curve 28 is thus due to the time-variant reflection in the tissue 104 at the first wavelength.

(20) Correspondingly, a ray of light D at the second wavelength can pass through the second marker area 24 whereas a ray of light B at the first wavelength is blocked. The intensity modulation depicted by curve 29 is thus due to the time-variant reflection in the tissue 104 at the second wavelength.

(21) The pulse rate of the subject can be directly determined from the time-variant intensity in one of the curves 28 or 29. However for determining the blood oxygen saturation by photoplethysmography at least two wavelengths are required, as exemplarily explained below.

(22) Contact pulse oximeters typically transmit red (R) and infrared (IR) (or, more precisely, in some cases near infrared) light through a vascular tissue of the subject of interest. The respective light portions (R/IR) can be transmitted and detected in an alternating (fast-switching) manner. Given that the respective spectral portions are differently absorbed by oxygenated hemoglobin (HbO.sub.2) and reduced hemoglobin (Hb), blood oxygen saturation eventually can be processed. An oxygen saturation (SO.sub.2) estimation algorithm can make use of a ratio of the signals related to the red and the infrared portion. Furthermore, the algorithm can consider a non-pulsatile signal component. Typically, the PPG signal comprises a DC component and a relatively small pulsatile AC component. Furthermore, SO.sub.2 estimation generally involves an empirically derived calibration factor applied to the processed values. Typically, the calibration factor (or, calibration curve) is determined upon reference measurements involving invasive blood oxygen saturation measurements. A calibration factor is required since a PPG device basically detects a ratio of (spectral) signal portions which has to be transferred into a blood oxygen saturation value which typically involves a ratio of HbO.sub.2 and Hb. For instance, but not intended to limit the present disclosure, blood oxygen saturation estimation can be based on the following general equation:

(23) $\begin{array}{cc}{\mathrm{SO}}_2=\frac{{\mathrm{HbO}}_2}{{\mathrm{HbO}}_2+\mathrm{Hb}},& \left(1\right)\end{array}$
whereas PPG devices merely immediately detect HbO.sub.2 and Hb from the spectral response at at least two wavelengths.

(24) Generally, the measured intensity curve 28, 29 as a characteristic signal is considered to contain a considerably constant (DC) portion and an alternating (AC) portion superimposing the DC portion. Applying signal processing measures, the AC portion can be extracted and, furthermore, compensated for disturbances. For instance, the AC portion of the characteristic signal can comprise a dominant frequency which can be highly indicative of the subject's 100 vascular activity, in particular the heart beat. Still, the characteristic signal, in particular the AC portion, can be indicative of further vital parameters. In this connection, the detection of arterial blood oxygen saturation is an important field of application. As indicated above, basically, arterial blood oxygen saturation-representative values can be computed taking into account the behavior of the AC portion of the characteristic signal at distinct spectral portions thereof. In other words, a degree of arterial blood oxygen saturation can be reflected in different radiation absorbance at blood vessels. Furthermore, one can make use of the fact that the difference in absorbance due to the grade of oxygenation also varies significantly across different spectral portions. Moreover, also the DC portion of the signal can be utilized for blood oxygen saturation detection. Typically, the DC component represents the overall light absorption of the tissue, venous blood, and non-pulsatile arterial blood. By contrast, the AC component may represent the pulsatile arterial blood's absorption. Consequently, the determination of arterial blood oxygen saturation (SaO.sub.2) can be expressed as:

(25) $\begin{array}{cc}{\mathrm{SaO}}_2=C.Math.\frac{{\left(\mathrm{AC}/\mathrm{DC}\right)}_{\mathrm{red}}}{{\left(\mathrm{AC}/\mathrm{DC}\right)}_{\mathrm{infrared}}},& \left(2\right)\end{array}$
where C is a calibration parameter. C may stand for a large variety of calibration parameters applicable to the AC/DC relationship and should therefore not be interpreted in the strict algebraic sense of equation (2). C may, for example, represent a fixed constant value, a set of fixed constants or an adjustable calibration parameter. By way of example, another exemplary SaO.sub.2 derivation model can be expressed as:

(26) $\begin{array}{cc}{\mathrm{SaO}}_2=C_1+C_2.Math.\frac{{\left(\mathrm{AC}/\mathrm{DC}\right)}_{\mathrm{red}}}{{\left(\mathrm{AC}/\mathrm{DC}\right)}_{\mathrm{infared}}},& \left(3\right)\end{array}$
where C.sub.1 and C.sub.2 can be considered calibration parameters of a linear approximation. In an exemplary embodiment, the signal calibration parameter determination can be directed to adjust or adapt the parameter C.sub.1. Still, in the alternative, SaO.sub.2 derivation may also be based on value tables deposited in (or accessible by) the device 1. The value tables (or: data bases) may provide for a discrete representation of the relationship between detected PPG signals and the desired calibration parameter. Also in that case an adaptable calibration parameter may be applied to improve the accuracy of the vital parameter determination.

(27) It should be understood that the equations (2) and (3) are primarily presented for illustrative purposes. They should not be construed as limiting the scope of the present disclosure. In practice, the skilled person may determine and establish further appropriate SaO.sub.2 derivation models. Alternative wavelength combinations, for example green and red, can be used depending on the substance to be detected. While the measurement of SaO.sub.2 has been described in detail, this is to be understood as an example for the general concept of measuring the concentration of a substance in blood and/or tissue.

(28) FIG. 3 shows a more detailed embodiment of a marker for use in a system 1 for determining vital sign information of a subject according to the present invention. The marker 30 features a first graphical pattern 32, a second graphical pattern 33, reference areas 34a, 35a, 36a, 37A, marker areas 34b, 35b, 36b configured to transmit light at three different wavelength, and a transparent marker area 37b.

(29) The carrier element 31 carries the marker 30 with its elements and provides mechanical stability. In this embodiment, the carrier element 31 is a patch or band-aid that can be directly attached to the skin of the subject 100 with an adhesive. The carrier element is made from an opaque rubber-like material that does not transmit light. Therefore, the carrier element features openings or windows at the positions of the marker areas 34b, 35b, 36b, 37b.

(30) In each of the windows 34b, 35b, 36b, an optical filter plate is placed wherein the filter plate 34b is configured to transmit light at a first wavelength, the filter plate 35b is configured to transmit light at a second wavelength, the filter plate 36b is configured to transmit light at a third wavelength. Marker area 37b does not have a frequency-selective filter but is transparent so that the skin underneath the area 37b can be seen by the detection unit as an unfiltered reference of the skin.

(31) The reference areas 34a, 35a, 36a are reference areas for specific wavelengths. Preferentially, the color of the reference area corresponds to the transmission wavelength of the neighboring filter. For example, marker area 34b is configured to transmit red light and the reference area 34a has a red color so that it can serve as a reference in particular as to how much red light is available in the radiation that is incident on the marker.

(32) The graphical pattern 32 serves as an alignment indicator. The geometrical structure enables the image processing unit 4 to determine the position and/or orientation of the marker 30. The marker 30 furthermore features a machine-readable graphical pattern 33 in form of a QR-code. This code comprises information about the subject such that the determined vital sign information can be assigned to the subject, for example to an electronic health record (EHR). The code can also comprise information about the site where the marker is attached to the subject's body, so that different measurements can be taken from the same subject at the same time that are specific for the measurement location (e.g. forehead and arm).

(33) Even though FIG. 3 shows a plurality of features, not all of them are mandatory. FIG. 4 shows the most basic configuration having two marker areas. In this example, the marker comprises a first dye directly applied to the skin of the subject in the first marker area and a second dye directly applied to the skin of the subject in a second marker area. The dye can be, for example, rubber stamped, drawn, sprayed or printed on the skin. The dye of the first marker area is configured to transmit light at a first wavelength, whereas the dye at the second marker area is configured to transmit light at the second wavelength. An additional reference area can be applied accordingly.

(34) FIG. 5 shows a side view of a marker similar to the one presented in FIG. 3. It is attached to the skin 104 of a subject. A carrier element 51 provides mechanical stability to the marker 50. On the side of the carrier element 51 facing the skin 104 of the subject, the carrier element 51 comprises an adhesive 52 that attaches the carrier element to the skin 104 of the subject. The adhesive 52 further comprises a stimulant for increasing the blood perfusion in the tissue 104 of the subject that is in contact with the carrier element 51 with the marker 50.

(35) On the opposite side of the carrier element 51, the marker 50 features reference areas 54a, 55a as well as a first marker area 54b configured to transmit light at a first wavelength and a second marker area 55b configured to transmit light at a second wavelength. This is achieved by optical filter plates 56, 57 that are placed in position of the first and second marker area and laminated into the carrier element 51. Optionally, the adhesive is not applied in the marker areas 54b, 55b in order not to influence the transmission characteristic of the marker areas.

(36) FIGS. 6 and 7 show different patterns of optical windows with first and second marker areas as well as reference areas. In this example, the first and second marker areas are divided into sub-areas. The patterns of the reference areas and the optical windows comprising the first and second marker areas can be chosen according to the needs of a specific measurement.

(37) For some measurements, it is important to get the signals for each wavelength from closely spaced areas of the tissue. In this case it is advantageous to have several smaller optical windows, for example in a check board design. FIG. 6 shows a marker 60 with a check board design for two wavelengths with alternating reference areas 61a and first marker areas 61b for a first wavelength and reference areas 62a and second marker areas 62b for a second wavelength. For example, for blood oxygen saturation measurement red is used as the first wavelength and infrared is used as the second wavelength.

(38) FIG. 7 shows an alternative embodiment of a marker 70 wherein the distribution of the reference areas 71a, 72a and optical windows for the first and second marker areas 71b, 72b are optimized depending on the expected signal strength. For example, the signal at a first required wavelength, for example green, is stronger than signal at the second required wavelength, for example red. Thus, the overall second marker area 72b is increased with respect to the overall first marker area 71b to achieve a similar signal strength at both wavelengths.

(39) By way of example, the present invention can be applied in the field of health care, e.g. unobtrusive remote patient monitoring, general surveillances, security monitoring and so-called lifestyle environments, such as fitness equipment, or the like. Applications may include monitoring of oxygen saturation (pulse oximetry), heart rate, blood pressure, cardiac output, changes of blood perfusion, assessment of autonomous functions, and detection of peripheral vascular diseases.

(40) While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

(41) In the claims, the word comprising does not exclude other elements or steps, and the indefinite article a or an does not exclude a plurality. A single element or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

(42) A computer program may be stored/distributed on a suitable non-transitory medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

(43) Furthermore, the different embodiments can take the form of a computer program product accessible from a computer usable or computer readable medium providing program code for use by or in connection with a computer or any device or system that executes instructions. For the purposes of this disclosure, a computer usable or computer readable medium can generally be any tangible device or apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution device.

(44) In so far as embodiments of the disclosure have been described as being implemented, at least in part, by software-controlled data processing devices, it will be appreciated that the non-transitory machine-readable medium carrying such software, such as an optical disk, a magnetic disk, semiconductor memory or the like, is also considered to represent an embodiment of the present disclosure.

(45) The computer usable or computer readable medium can be, for example, without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or a propagation medium. Non-limiting examples of a computer readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk, and an optical disk. Optical disks may include compact diskread only memory (CD-ROM), compact diskread/write (CD-R/W), and DVD.

(46) Further, a computer usable or computer readable medium may contain or store a computer readable or usable program code such that when the computer readable or usable program code is executed on a computer, the execution of this computer readable or usable program code causes the computer to transmit another computer readable or usable program code over a communications link. This communications link may use a medium that is, for example, without limitation, physical or wireless.

(47) A data processing system or device suitable for storing and/or executing computer readable or computer usable program code will include one or more processors coupled directly or indirectly to memory elements through a communications fabric, such as a system bus. The memory elements may include local memory employed during actual execution of the program code, bulk storage, and cache memories, which provide temporary storage of at least some computer readable or computer usable program code to reduce the number of times code may be retrieved from bulk storage during execution of the code.

(48) Input/output, or I/O devices, can be coupled to the system either directly or through intervening I/O controllers. These devices may include, for example, without limitation, keyboards, touch screen displays, and pointing devices. Different communications adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems, remote printers, or storage devices through intervening private or public networks. Non-limiting examples are modems and network adapters and are just a few of the currently available types of communications adapters.

(49) The description of the different illustrative embodiments has been presented for purposes of illustration and description and is not intended to be exhaustive or limited to the embodiments in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. Further, different illustrative embodiments may provide different advantages as compared to other illustrative embodiments. The embodiment or embodiments selected are chosen and described in order to best explain the principles of the embodiments, the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.