Abstract
The present invention relates to a device (100) for determining a vital sign of a subject by use of PPG. The device comprises a processing unit (110) configured to obtain multiple first detection signals derived from electromagnetic radiation (90) from multiple measurement spots (30) on a skin region (12) of the subject illuminated by one or more illumination spots (20), wherein the electromagnetic radiation (90) enters the skin region (12) at the one or more illumination spots (20) and exits the skin region (12) at the multiple measurement spots (30). The processing unit (110) further obtains 3D image information (130) of the skin region (12) which allows determining the actual spot distance (15) between the measurements spots (30) and an illumination spot (20), respectively. This information can be used to interpret the PPG signal more precisely as without having the 3D image information only a projected distance can be determined if the skin surface is tilted. Additionally, this allows averaging PPG signals derived from a homogenous skin area to improve PPG signal strength and reduce noise. A system (500) comprising such a device and a corresponding method are further described.
Claims
1. A device for determining a vital sign of a subject, comprising a processor configured to: obtain multiple first detection signals derived from electromagnetic radiation from multiple measurement spots on a skin region of the subject illuminated by one or more illumination spots, wherein the electromagnetic radiation enters the skin region at the one or more illumination spots and exits the skin region at the multiple measurement spots (30), obtain a 3D image information of the skin region, determine for the multiple first detection signals the respective spot distance between the respective measurement spot and an illumination spot based on the obtained 3D image information of the skin region, assign the multiple first detection signals to their respective spot distance and to the illumination spot from which the respective spot distance is determined, and average the first detection signals assigned to the same illumination spot and to a respective spot distance smaller than a threshold distance value to obtain an averaged first detection signal, and determine an averaged vital sign from said averaged first detection signal, or determine vital signs from the first detection signals assigned to the same illumination spot and to a respective spot distance smaller than a threshold distance value, and average said vital signs to obtain an averaged vital sign.
2. The device according to claim 1, wherein the processor is further configured to derive a first detection function from the first detection signals assigned to the same illumination spot, wherein the first detection function is a function of the spot distance.
3. The device according to claim 1, wherein the processor is configured to determine the 3D image information of the skin region from the multiple first detection signals in case the skin region is illuminated by multiple illumination spots.
4. The device according to claim 1, wherein, in the case of multiple illumination spots, the processor is configured to determine for the multiple first detection signals the respective spot distance between the respective measurement spot and the illumination spot which is the closest illumination spot to the respective measurement spot in relation to the other illumination spots.
5. The device according to claim 1, wherein the processor is configured to: obtain a second detection signal derived from a spatial integral of all the electromagnetic radiation reflected from the skin region of the subject illuminated by one or more illumination spots or derived from electromagnetic radiation reflected from or transmitted through skin region of the subject illuminated by a homogenous illumination, and determine one or more normalized signals, wherein a normalized signal is calculated by calculating a ratio of a first detection signal of the multiple first detection signals and the second detection signal.
6. A device according to claim 1, wherein the multiple first detection signals and the second detection signal are derived from electromagnetic radiation with the same wavelength.
7. A system for determining a vital sign of a subject, said system comprising: an illuminator configured to emit electromagnetic radiation to illuminate a skin region of the subject by one or more illumination spots, a camera configured to record an image of the detected electromagnetic radiation reflected from the skin region illuminated by one or more illumination spots and to derive multiple first detection signals from the recorded images, and a device for determining an averaged vital sign of a subject according to claim 1.
8. A system according to claim 7, wherein the one or more illumination spots have the shape of any one of a circle, ellipse, dot or stripe.
9. A system according to claim 7, further comprising a time-of-flight, TOF, sensor configured to determine 3D image information of the skin region from the detected electromagnetic radiation reflected from the skin region illuminated by one or more illumination spots and/or a stereo depth sensor configured to determine 3D image information of the skin region by stereo vision.
10. A system according to claim 7, further comprising: a controller, wherein, in the case where the illuminator is configured to emit electromagnetic radiation by one or more illumination spots having the shape of a circle or an ellipse, the controller is configured to receive 3D image information of the skin region and a color image of the illuminated skin region recorded by the camera, to control the shape of the one or more illumination spots emitted by the illuminator based on said received 3D image information and said received color image to emit one or more illumination spots having the shape of such an ellipse that the one or more illumination spots have the shape of a circle on the skin region in the color image recorded by the camera.
11. The system according to claim 9, wherein the controller is configured to receive the 3D image information from the processor of the device or from the TOF sensor or from the stereo depth sensor.
12. The system according to claim 7, wherein the camera comprises a plurality of detection elements, in particular an array of photo diodes, a CCD array or a CMOS array.
13. A method for determining a vital sign of a subject, said method comprising the steps of: obtaining multiple first detection signals derived from electromagnetic radiation from multiple measurement spots on a skin region of the subject illuminated by one or more illumination spots, wherein the electromagnetic radiation enters the skin region at the one or more illumination spots and exits the skin region at the multiple measurement spots, obtaining a 3D image information of the skin region, determining for the multiple first detection signals the respective spot distance between the respective measurement spot and an illumination spot based on the obtained 3D image information of the skin region, assigning the multiple first detection signals to their respective spot distance and to the illumination spot from which the respective spot distance is determined, and averaging the first detection signals assigned to the same illumination spot and to a respective spot distance smaller than a threshold distance value to obtain an averaged first detection signal, and determine an averaged vital sign from said averaged first detection signal, or determining vital signs from the first detection signals assigned to the same illumination spot and to a respective spot distance smaller than a threshold distance value, and average said vital signs to obtain an averaged vital sign.
14. A non-transitory computer readable medium comprising code, that when executed, causes a computer to carry out the steps of the method as claimed in claim 13.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0040] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. In the following drawings
[0041] FIGS. 1A and 1B show schematic diagrams illustrating the paths of electromagnetic radiation through skin region of a subject for widefield PPG and radial PPG;
[0042] FIGS. 2A and 2B show schematic diagrams illustrating the detected reflected light of widefield PPG and radial PPG;
[0043] FIG. 3 shows a schematic diagram of Monte Carlo simulations of the paths of light through a skin region of a subject for radial PPG;
[0044] FIGS. 4A and 4B show schematic diagrams illustrating the path of electromagnetic radiation through skin region of a subject for radial PPG;
[0045] FIGS. 5A-5C show schematic diagrams illustrating ambiguity problems arising for conventional PPG;
[0046] FIG. 6 shows a system for determining a vital sign of a subject according to the present invention;
[0047] FIG. 7 shows a modified embodiment of the system shown in FIG. 6 in order to illuminate a skin region homogenously;
[0048] FIG. 8 shows a flowchart illustrating a method to be executed by the device according to the present invention;
[0049] FIG. 9 shows a schematic diagram illustrating problems arising from radial PPG without having knowledge about the 3D structure of a measured skin region;
[0050] FIG. 10 shows an infrared image and a 3D depth image of a mannequin;
[0051] FIG. 11 shows a flowchart illustrating an optional method to be executed by the device according to the present invention;
[0052] FIGS. 12A and 12B show schematic diagrams illustrating problems arising from conventional radial PPG without having knowledge about the 3D structure of a measured skin region;
[0053] FIG. 13 shows a schematic diagram illustrating an area illuminated by a spot pattern;
[0054] FIGS. 14A and 14B show a schematic diagrams illustrating a further possibility of reducing PPG noise;
[0055] FIGS. 15A-15C show schematic diagrams illustrating the projection of a pattern of circles on a skin region of a subject;
[0056] FIG. 16 shows a schematic diagram illustrating a comparison of an illumination with a dot pattern and an illumination with a circular pattern;
[0057] FIG. 17 shows a schematic diagram illustrating locally measured PPG amplitudes;
[0058] FIG. 18 shows a processing of radial PPG signals and widefield PPG signals to overcome problems arising from conventional PPG; and
[0059] FIG. 19 shows a schematic diagram illustrating a process of obtaining improved PPG color maps.
DETAILED DESCRIPTION OF EMBODIMENTS
[0060] FIGS. 1A and 1B show schematic diagrams illustrating the path of electromagnetic radiation 90 through skin region 12 of a subject for widefield PPG (left) and radial PPG (right). Widefield PPG is the mode that is commonly used in camera mode for various years, where the illumination by electromagnetic radiation 90 is homogenously distributed across the skin region 12 and the PPG signal is measured across that same skin area 12. This mode is shown in FIG. 1A. Exemplarily, the electromagnetic radiation 90 is emitted by multiple illumination sources 200, travels through the epidermis 13 of the skin region 12 into deeper skin layers with various venules 14 and arterioles 16 before exiting the skin region 12 and getting detected by a detection unit 300, such as a camera. The detected electromagnetic radiation and the derived PPG signal is an average of all the electromagnetic radiation 90 detected by the camera 300.
[0061] FIG. 1B shows a radial PPG mode. The simplest form of contactless radial PPG is the use of just one spot of light injection as shown in FIG. 1B. This may be achieved by use of a laser configured to emit electromagnetic radiation 90 located, e.g., in the red or infrared spectral range. Radial PPG is in principle quite similar to conventional contact probe PPG measurements. The skin region 12 is illuminated by a spot (such as a circle, a stripe, a dot, an ellipse, etc.) and the PPG signal is measured several millimeters or centimeters away from that illumination spot 20 on the skin region 12. The spot distance 15 between the illumination spot 20 on the skin region 12 and the measurement spot 30 from which the PPG signal is measured is exemplarily illustrated for one reflected beam in FIG. 1B. Hence, the radial PPG signal is in general a signal which depends on the spot distance 15. To be more concrete, the electromagnetic radiation reflected from the skin region 12 of the subject is reflected from said skin region 12 by entering into the skin region 12 through the epidermis 13 at the illumination spot 20, getting reflected from constituents of the skin region 12 and exiting the skin region 12 through the epidermis 13 at a measurement spot 30, which is located at a spot distance 15 away from the illumination spot 20. In other words, the electromagnetic radiation detected by the detection unit 300 is scattered back from the skin region 12 and collects pulsatile information from the different venules 14 and arterioles 16 located in the skin region 12 below the epidermis 13.
[0062] It will be shown below that the present invention combines according to some embodiments PPG signals obtained from these two measurement modes, i.e., from radial PPG and widefield PPG.
[0063] FIGS. 2A and 2B show schematic diagrams illustrating the detected reflected light of widefield PPG and radial PPG. The schematic diagrams on the top of FIGS. 2A and 2B are the same that have already been discussed with reference to previous FIGS. 1A and 1B.
[0064] FIG. 2A shows (on the bottom) a diagram illustrating the detected reflected light of a widefield PPG setup. For this purpose, the axis of ordinate illustrates the reflected detected light (indicated by “R”) and the axis of abscissae illustrates the measurement time. The curve 253 illustrates the reflected detected light, which comprises a DC component 255 and an AC component 254. The AC component 254 represents the pulsatile components of optical absorption originating from the pulsatile arterial blood and the DC component 255 represents the non-pulsatile components containing contribution from non-pulsatile arterial blood, venous blood, and other tissues.
[0065] FIG. 2B shows in the middle row three different diagrams illustrating the detected reflected light of a radial PPG setup. For this purpose, the axes of ordinate illustrate the reflected detected light and the axes of abscissae illustrate the measurement time. It becomes apparent from the three curves 263, 264, 265 that the larger the spot distance 15 between the illumination spot 20 and the respective measurement spot 30 (cf. description with reference to FIGS. 1B), the smaller the DC component of the reflected detected light. This is schematically illustrated by the offsets of the curves 263, 264, 265, respectively. The reason for this is that more light or electromagnetic radiation 90 is absorbed by the skin region 12, if the light travels through skin by a longer path.
[0066] These dependencies are also illustrated in the lowermost diagram at the bottom of FIG. 2B. This diagram illustrates the AC or DC component of the reflected detected light on the axis of ordinate and the spot distance 15 on the axis of abscissae. The first curve 273 illustrates the DC component of the reflected detected light and the second curve 274 illustrates the AC/DC component (relative PPG amplitude) of the reflected detected light. The already discussed trend becomes apparent. It becomes clear that the larger the spot distance 15, the larger the ratio AC/DC (the larger the relative pulsatile component).
[0067] FIG. 3 shows a schematic diagram of Monte Carlo simulations of the path of light through the skin region 12 of a subject for radial PPG. The light distribution inside the skin region 12 is visualized to illustrate the different penetration depths 20, which depicts how depth the incoming electromagnetic radiation 90 penetrates into the skin region 12 before being scattered back from venules and arterioles.
[0068] FIGS. 4A and 4B show further schematic diagrams illustrating even more clearly the path of electromagnetic radiation 90 through skin region 12 of a subject for radial PPG. As shown in FIG. 4A, the electromagnetic radiation 90 enters the skin region 12 at an illumination spot 20, is reflected from constituents of the skin region 12 (mainly venules 14 and arterioles 16) and exits the skin region 12 at measurement spots 30 distanced away from the illumination spot 20. Exemplarily, three different measurement spots are shown in FIG. 4A. These three different measurement spots 30 exemplarily have a distance of 0.1 mm, 0.2 mm and 0.3 mm to the illumination spot 20 at which a pencil beam (e.g., a laser beam) hits the skin surface.
[0069] The light intensity of the detected electromagnetic radiation 90 derived from the three different measurement spots 30 is illustrated in dependence on the measurement time in FIG. 4B. The AC and DC components of the three different curves 413, 414 and 415 clearly depend on how long the electromagnetic radiation 90 has travelled through tissue. It becomes apparent that the longer the light has travelled through skin region 12 with pulsatile volumetric absorbers, the larger the modulation depth of its intensity.
[0070] FIG. 5 shows a schematic diagram illustrating ambiguity problems arising for conventional PPG. While PPG images do give an indication of blood perfusion as illustrated in FIGS. 4A and 4B, the actual amplitude (i.e., the ratio of the AC and DC component, also referred to as normalized pulsatility) is difficult to interpret, because it is the integral of the PPG modulation resulting from reflected light from different skin depths 20. Hence, superficial vessels with a very small pulsatility as shown in the diagram on the left may give the same PPG signal strength as deeper vessels with stronger pulsatility as shown on the right of FIG. 5. Thus, different locations of strongly or less strongly pulsatile components in the skin region 12 as shown in FIG. 5 may result in exact the same PPG signal strength as illustrated by the curves I(t) in the upper diagrams.
[0071] FIG. 6 shows a system 500 for determining a vital sign of a subject according to the present invention. The system 500 comprises at least an illumination unit 200, a detection unit 300 and a device 100 for determining a vital sign of a subject. A more detailed explanation of said device 100 will be given below with reference to FIG. 8.
[0072] The illumination unit 200 is configured to emit electromagnetic radiation 90 to illuminate a skin region 12 of a subject. Preferably, said illumination unit 200 is configured to emit a controllable narrow beam of electromagnetic radiation 90, which is preferably located in the visible or infrared spectral range. Infrared radiation is not visible and thus preferred in some clinical settings in order to not disturb the patient.
[0073] According to the embodiment shown in FIG. 6, the system 500 may optionally further comprise a support 250 for limiting the skin region 12 of the subject to a limited skin area to be measured. The support 250 may be placed on the skin of the subject as shown in FIG. 6 and is preferably made of a material that is non-transparent for the incoming electromagnetic radiation 90. Still preferably, the skin region 12 used for measurement is thus limited to an area which comprises a skin region 12 with a homogenous surface along the area to be measured.
[0074] The inlet in FIG. 6 shows a picture of the illumination profile of an illumination spot 20 on the skin region 12. It shall be understood that one illumination spot 20 is only exemplarily. The illumination unit 20 may also be configured to project an illumination pattern, i.e., multiple illumination spots on the skin region 12 of the subject.
[0075] FIG. 7 shows a modified embodiment of the system shown in FIG. 6 in order to illuminate a skin region homogenously. For said purpose, the system 500 may comprise a diffuser 220. Said diffuser 220 is configured to diffuse the controllable narrow beam of electromagnetic radiation 90 emitted by the illumination unit 200 to generate a homogenous illumination profile and/or structured light on the skin region 12 of the subject. Hence, the configuration shown in FIG. 7 allows to measure widefield PPG.
[0076] It shall be understood that it may also be an option to use separate illumination units for measuring widefield PPG with a homogenous illumination and radial PPG with a spot illumination. The embodiment shown in FIG. 7, however, provides advantages as a first measurement may be performed for measuring radial PPG by not placing the diffuser into the path of the emitted electromagnetic radiation 90 and a second measurement may be performed for measuring widefield PPG by placing the diffuser into the path of the emitted electromagnetic radiation 90. It will be explained below that the PPG signals obtained for radial PPG and widefield PPG are combined according to some embodiments of the present invention.
[0077] FIG. 8 shows a flowchart illustrating a method to be executed by the device 100 for determining a vital sign of a subject according to the present invention.
[0078] The device 100 comprises a processing unit 110, which, in a step S10, obtains multiple first detection signals 120 derived from electromagnetic radiation 90 from multiple measurement spots on a skin region 12 of the subject illuminated by one or more illumination spots 20, wherein the electromagnetic radiation 90 enters the skin region 12 at the one or more illumination spots 20 and exits the skin region 12 at the multiple measurement spots 30. This has already been discussed with reference to FIG. 4A, where exemplarily one illumination spot 20 is shown, at which the electromagnetic radiation 90 enters the skin region 12.
[0079] In a step S20, the processing unit 110 obtains 3D image information 130 of the skin region 12. The processing unit may either obtain the 3D image information from an external sensor or may determine the 3D image information of the skin region 12 from the multiple first detection signals, in particular in case the skin region 12 is illuminated by multiple illumination spots 20.
[0080] In a step S30, the processing unit 110 determines for the multiple first detection signals 120 the respective spot distance 15 between the respective measurement spot 30 and an illumination spot 20 based on the obtained 3D image information 130 of the skin region 12. In a step S40, the processing unit 110 assigns the multiple first detection signals 120 to the respective spot distance 15 and to the illumination spot 20 from which the respective spot distance 15 is determined. Subsequently, the processing unit 110 either executes i) steps S50 and S60, or ii) steps S70 and S80. Thus, in a step S50, the processing unit 110 may average the first detection signals 120 assigned to the same illumination spot 20 and to a respective spot distance 15 smaller than a threshold distance value to obtain an averaged first detection signal 140, and in a step S60, determine an averaged vital sign 160 from said averaged first detection signal. Alternatively, the processing unit 110 may, in a step S70, determine vital signs 150 from the first detection signals 120 assigned to the same illumination spot 20 and to a respective spot distance 15 smaller than a threshold distance value, and in a step S80, average said vital signs 150 to obtain an averaged vital sign 160. It shall be understood that these two options are equivalent. Hence, the averaging may either take place before or after determining a vital sign.
[0081] FIG. 9 shows a schematic diagram illustrating problems arising from radial PPG without having knowledge about the 3D structure of the skin region. Exemplarily, a module 210 is used which comprises an illumination unit 200 for emitting electromagnetic radiation to illuminate a skin region 12 (e.g., the head) of a subject by multiple illumination spots 20. Further, the module 210 comprises three cameras 300 configured to record images of the detected electromagnetic radiation 90 reflected from the skin region 12 illuminated by the multiple illumination spots 20. It shall be understood that this is only exemplarily. In general, at least one camera 300 and at least one illumination unit 200 is needed, which, however, do also not have to be integrally formed in one module as exemplarily illustrated in FIG. 9.
[0082] The distance between the illumination spots 20 on the forehead of the subject (indicated by arrow 914) may be easily determined by the distance to the module 210 to the subject, i.e., by use of time-of-flight, TOF, for example. For said purpose, the system as shown in FIGS. 6 and 7 may further comprise, e.g., a TOF sensor configured to determine 3D image information of the skin region 12. Another option may, for example, to use stereo depth sensors for said purpose, where distance information is obtained by stereo vision.
[0083] In difference to the spots 20 on the forehead of the subject, a problem arises from the distance between the illumination spots 20 on the cheeks (as indicated by arrows 913 and 915). Said distances are larger due to the angle of skin with respect to the illumination direction. Thus, without having knowledge about the 3D structure of the skin area 12 it is not possible to quantitatively interpret the PPG signals since only a projected distance may be determined from a color image recorded by the camera 300. This leads to the drawback that the radial PPG signals are wrongly assigned to the projected distance instead of the actual distance.
[0084] The present invention is able to overcome said problems by using 3D depth image information. This allows determining the actual distances of the illumination spots 20 on the surface of the skin area 20, and not only a projected distance. Another advantage arising from the present invention is that by having knowledge about the 3D structure of the skin region, multiple measurement spots from a homogenous skin area can be averaged to increase the PPG signal strength.
[0085] FIG. 10 shows an infrared image (left) and a 3D depth image (right) of a mannequin. It is a well-known technique to illuminate objects by a spot pattern to extract depth information. For said purpose, an infrared pattern is projected onto an object (or a mannequin as shown in FIG. 10) and, for example, one or more standard CMOS sensors of a camera image the reflected light. One can then estimate depth for each pixel using knowledge of the projected light pattern or by using disparity estimation (when using the images of at least two CMOS sensors). This may, e.g., be achieved by using the module 210 shown in FIG. 9, where an infrared pattern is projected onto a subject and used to infer depth using three cameras.
[0086] FIG. 11 shows a flowchart illustrating an optional method to be executed by the processor 110 of the claimed device 100. As already explained with reference to FIG. 8, in a step S10, the processor obtains first detection signals derived from electromagnetic radiation 90 from multiple measurement spots on the skin region 20 of the subject. In case the skin region 12 is illuminated by multiple illumination spots 20, the processor may determine, in a step S22, the 3D image information from the multiple first detection signals 120. Hence, the processor may determine this information by its own, without obtaining this 3D information from an external sensor (such as a TOF sensor). This provides the advantage that an external sensor configured to measure the depth information is not needed. The device 100 comprising the processor 110 may thus be a compact device, which may further be portable to be used in various different settings, such as for non-stationary settings and ambulant patients.
[0087] FIGS. 12A and 12B show a schematic diagrams illustrating, as FIG. 9, problems arising from conventional radial PPG without having knowledge about the 3D structure of the measured skin region 12. As already mentioned above, radial PPG aims at measuring light intensities as a function of lateral (radial) distance away from the illumination spot 20. Exemplarily, one illumination unit 200 is shown which illuminates the skin region 12 by one illumination spot 20. A camera 300 detects the electromagnetic radiation reflected from the skin region 12, wherein only the reflected light within a field of view, FOV, of the camera 300 is detected.
[0088] FIG. 12A illustrates a scenario where the surface of the skin region 12 is perpendicular to the illumination direction of the illumination unit 200. In other words, a normal to a plane defined by the skin surface is parallel to the illumination direction. In this case, distances on the skin region may be easily determined by the distance of the skin region 12 to the camera 300, i.e., by measuring said distance using, for example, TOF technique. Exemplarily, a unit distance and a distance r are shown, while the distance r may be the spot distance 15 between the illumination spot 20 and a measurement spot 30 (as explained above).
[0089] FIG. 12B illustrates a scenario where the surface of the skin region is not perpendicular to the illumination direction of the illumination unit 200. In this case, distances on the skin region 12 cannot easily be determined without having knowledge about the 3D structure of the skin region 12. This is illustrated again by the same unit distance and the distance r. The lengths of the arrows indicating said unit distance and the distance r (or the spot distance 15) are smaller than the lengths of the arrows in FIG. 12A. This illustrates that there is a need for correcting for projection if the skin is not perpendicular to the illumination direction of the illumination unit 200. Otherwise the measured radial PPG signal is assigned to a projected (smaller) distance, rather than to the actual (correct) distance.
[0090] FIG. 13 shows a schematic diagram illustrating an area illuminated by a spot pattern. The area may be the skin region illuminated for PPG measurements as described above.
[0091] The same module 210 as already explained with reference to FIG. 9 may be used here, which comprises an illumination unit 200 and three cameras 300. The illumination unit 200 illuminates the skin area 12 by multiple illumination spots 20. Exemplarily, the illumination spots 20 have the shape of dots. However, it shall be understood that the illumination spots 20 may also have the shape of a line, an ellipse, circles, etc.
[0092] Each dashed circle around an illumination spot 20 represents possible measurement spots 30, which all have the same spot distance 15 to the respective illumination spot 20. Thus, the measurement spots on the inner dashed circles may exemplarily have a spot distance of 1 mm to the respective illumination spot 20. The measurement spots on the outer dashed circles may exemplarily have a spot distance of 2 mm to the respective illumination spot. The first detection signals derived from electromagnetic radiation from multiple measurement spots on a skin region of the subject illuminated by multiple illumination spots may thus be abbreviated by I(r=1 mm) and I(r=2 mm), respectively, as shown in FIG. 13. If the area is not flat, but curved, i.e., if the area is not perpendicular to the illumination direction of the illumination unit 200, the dashed circles transform into dashed ellipses, which represents the problem discussed already with reference to FIGS. 9 and 12. Thus, the dashed circles transform into ellipses for a curved area as shown in the lower part of the area shown in FIG. 13.
[0093] FIGS. 14A and 14B show a schematic diagrams illustrating the use of multiple modules 210 each comprising an illumination unit 200 and three cameras 300. A powerful technique is to reduce noise of the measured first detection signals by combining multiple modules 210 and use the spot pattern to register the camera images of these. As shown in FIG. 14A, one of the modules 210 may be used to project the illumination pattern (as indicated by the arrows), while the other modules are used for imaging only, i.e., to record the electromagnetic radiation reflected from the area (such as the measured skin region). FIG. 14B shows a schematic diagram illustrating the use of 10 modules as a module array 211. A vertical placement allows for placing 10 (or even more) modules 210 at a small distance with limited parallax. Thus, noise may be further reduced by averaging the signals obtained from a plurality of modules 210.
[0094] FIG. 15 shows schematic diagrams illustrating the projection of a pattern of circles on a skin region of a subject. The skin region 12 may either be on the head of the subject as shown in FIG. 12A or on the hand as shown in FIG. 12B. However, it shall be understood that these two skin regions are only exemplarily. It shall be noted that measuring PPG signals from the forehead of the cheek of a subject as illustrated in FIG. 15A is well-known to give decent PPG signal strengths, while measuring PPG signals on other anatomical locations, such as on the arms, is known to give much lower PPG signal strengths.
[0095] As shown in FIG. 15A, an illumination unit 200 is used to project the pattern of circles onto the skin region 12 of the subject, which is located at a distance D away from the illumination unit 200. An image of such a pattern projected onto a hand of the subject is shown in FIG. 15B. This image may be recorded by the camera 300 shown in FIG. 15A, which may be located directly adjacent to the illumination unit 200. A second camera may be used as well to correct for parallax.
[0096] It becomes apparent that the shape of the projected pattern depends on the angle α of the skin region 12 with respect to the illumination direction. If the angle is zero, i.e., if the skin region 12 is perpendicular to the illumination direction, the projected pattern on the skin region 12 comprises circles, while if the angle is different from zero, i.e., if the skin region 12 is not perpendicular to the illumination direction, the projected pattern comprises ellipses. In other words, the projected pattern of circles appears as ellipses on the skin region 12 due to the non-zero angle α.
[0097] FIG. 15C shows a schematic diagram illustrating the shape of the projected circles on different locations on the skin region 12 indicated by “A” and “B” in FIG. 15A, where location “B” is located further away from the nose of the subject than location “A”. The angle α is much larger for location “B”, while said angle is almost zero for location “A”. Hence, the diameter d.sub.A of the projected circle onto location “A” is strongly modified as shown in FIG. 15C. The projected circle appears as an ellipse with a modified diameter d.sub.A′=d.sub.A/cos(α). The diameter d.sub.B of the projected circle onto location “B” is hardly modified (d.sub.B≈d.sub.B′). Hence, the circle projected onto location “B” appears as a circle and does not transform into an ellipse. This can also be seen in FIG. 15 B, where the projected circles on locations with a large angle α, such as the insides of the fingers, appear as ellipses. It shall be understood that the modifications of the projected circles into ellipses does not only depend on the diameters of the projected circles, and angle α, but also on the distance D between the illumination unit 200 and the akin region 12 of the subject.
[0098] It will be shown later that the PPG signal strength drop off increases for larger spot distances (i.e., large distances between an illumination of the spot pattern and the respective measurement spot). Even with a low noise/high sensitivity camera this is a very challenging dynamic range. By using circular illumination patterns, much more camera pixels are within a certain dynamic range without losing the ability to measure a radial PPG signal.
[0099] Hence, to only use a pattern of circles there are various possibilities. One option is to discard all illumination spots with a strong ellipsoidal illumination (e.g., for angles α>10°) for further processing of PPG signals. It may thus be an option to discard the signals derived from an image area in the recorded image of the camera 200 which comprises many ellipsoidal illuminations (such as location “A” in FIG. 15A).
[0100] Another option is to modify the illumination of the illumination spots on location “A” such that the actual illumination is a circle again. This may be done as follows: A control unit 400 may be connected to the illumination 200 and the camera 300 as shown in FIG. 15A. Said control unit 400 may receive 3D image information of the skin region 12 and a color image of the illuminated skin region 12 recorded by the camera 300 being a color camera. Based on this, the control unit 400 may control the shape of the illumination spots (the pattern of circles) emitted by the illumination unit 200 to emit illumination spots having the shape of such an ellipse that the illumination spots appear as circular spots again (in the image recorded by the camera 300).
[0101] FIG. 16 shows a schematic diagram illustrating a comparison of an illumination by a dot pattern and an illumination by a circular pattern. The upper three diagrams illustrate respectively the DC component of the reflected intensity in dependence on the spot distance 15. In case of a circular spot pattern, the spot distance is again determined by the (smallest) distance of the respective measurement 30 spot to an illumination spot 20. The upper curves in the respective diagrams illustrate the reflected intensity derived for a circular spot pattern, while the lower curves illustrate the reflected intensity derived for a dot pattern. The diagram on the left illustrates the reflected intensity derived for a circular spot pattern, where the projected spots have a diameter of 1 mm. The middle diagram and the right diagram refer to illuminations with circular diameters of 3 mm and 5 mm, respectively (cf. sketches of the respective circles below). As in all three diagrams the circular spot pattern curves comprise larger intensity values than the dot pattern curves, the DC intensity drop-off inside an illumination circle is much less than for a dot. This illustrates that by using circular illumination patterns, much more camera pixels are within a certain dynamic range without losing the ability to measure a radial PPG signal.
[0102] FIG. 17 shows a schematic diagram illustrating locally measured PPG amplitudes. The color map image on the right displays the amplitude of locally measured PPG amplitudes and the diagrams on the left show time signals of the respective PPG amplitudes measured at four different locations on the skin area 12 indicated by numbers. The color image is difficult to interpret for clinical use, because the signal displayed is a convolution of PPG modulations stemming from different skin depths as already explained with reference to FIG. 5. Superficial vessels with a very small pulsatility may, e.g., give the same PPG signal strength as deeper vessels with a stronger pulsatility. Hence, it is not known from the color map shown in FIG. 17 whether the large signal in face region indicated by number “1” is due to relatively shallow arterioles, or whether there is a higher density of arterioles since the overall signal is a weighted sum of pulsatile components at different depths. This ambiguity may also be one of the reasons why PPG imaging experiences slow acceptance in clinical practice. When a clinician interprets the color map, it is difficult to explain what they actually refer to since many parameters are involved, such as the optical depth of the pulsatile vessels, strength of the volumetric pulsatility, density of pulsatile vessels and absorption coefficient of the blood at the wavelength used. With reference to the subsequent FIG. 18 it is explained how to define a signal which indicates the optical depth of the pulsatile vessels and may thus be used for less ambiguous PPG imaging.
[0103] FIG. 18 shows a processing of radial PPG signals and widefield PPG signals to overcome problems arising from conventional PPG. As explained above, the first detection signals refer to radial PPG signals and the processing unit of the claimed device may be configured to derive a first detection function 170 from the first detection signals assigned to the same illumination spot 20. The first detection function 170 is a function of the spot distance as shown in FIG. 18 and may depict the relative pulsatile component (AC/DC) of the first detection signals.
[0104] The second detection signal, as explained above, refers to a widefield PPG signal. The processing unit of the claimed device may further be configured to derive a second detection function 174 from the second detection signal. As the second detection function refers to widefield PPG and a homogenous illumination, the second detection signal, which may also depict the relative pulsatile component (AC/DC) of the second detection signal, is independent from the spot distance.
[0105] In a next step, the processing unit may be configured to determine a normalized signal 190 by calculating a ratio of the first detection function 170 derived from the first detection signals and the second detection function 174 derived from the second detection signal. This normalized signal 190 which refers to a ratio of radial PPG signal amplitudes and widefield PPG signal amplitudes, can be used to give an index, which depends on the spot distance 15. This index may be referred to as “Optical Depth Index” or “ODI” and represents an estimate of the relative optical depth of the source of the PPG. In other words, said normalized signal 190 or said ODI gives an estimate for the dominant depth of the pulsatile vessels Thus, this ODI allows obtaining information about the dominant depth of pulsatile vessels at which the electromagnetic radiation is scattered back.
[0106] FIG. 19 shows a schematic diagram illustrating a process of obtaining improved PPG color maps. The uppermost color map represents a typical color map derived from conventional PPG imaging, where the color represents the strength of the PPG amplitude. Said color map is, as already mentioned above, difficult to interpret as it is not known from said color map whether the large signal in the face region indicated by number “1” is due to relatively shallow arterioles or whether there is a higher density of arterioles since the overall signal is a weighted sum of pulsatile components at different depths. However, as explained with reference to FIG. 18, the ratio of radial PPG signals and a widefield PPG signal gives an index, ODI, which allows estimating the penetration depth of the electromagnetic radiation in the skin of the subject. This ODI is, as shown in the right diagram of FIG. 18, dependent on the spot distance 15. For simplifications, a constant ODI value may thus be defined by setting, e.g., the spot distance to 1 cm. This allows determining a color map for each skin pixel with a discrete ODI value. This color map is shown in the bottom left of FIG. 19. Said color map illustrates that the ODI on the forehead is typically low, while the ODI on the cheek is typically large. The color map on the bottom right of FIG. 19 is a sum of the ODI color map and the color map obtained from conventional PPG imaging without having information about the optical depth of the pulsatile layers. This color map is thus supposed to give more accurate results and improves PPG imaging as the PPG signal strengths is weighted with the ODI.
[0107] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.
[0108] In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
[0109] A computer program may be stored/distributed on a suitable non-transitory medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.
[0110] Any reference signs in the claims should not be construed as limiting the scope.
