DEVICE, SYSTEM AND METHOD FOR DETECTION OF PULSE OF A SUBJECT

Abstract

The present invention relates to a device, system and method for improved non-invasive and objective detection of pulse of a subject. The device comprises an input unit (2a) configured to obtain a series of images of a skin region of the subject and a processing unit (2b) for processing said series of images by detecting pulse-related motion of the skin within the skin region from the series of images, generating a motion map of the skin region from the detected pulse-related motion, comparing the generated motion map with an expected motion map of the skin region, and determining the presence of pulse within the skin region based on the comparison.

Claims

1. Device for detection of pulse of a subject comprising: an input unit configured to obtain a series of images of a skin region of the subject, and a processing unit for processing said series of images by detecting pulse-related motion of the skin within the skin region from the series of images, generating a motion map of the skin region from the detected pulse-related motion, comparing the generated motion map with an expected motion map of the skin region for a healthy subject, and determining the presence of pulse within the skin region based on the comparison.

2. Device as claimed in claim 1, wherein the processing unit is further configured to derive a dilatation signal reflecting arterial dilatation from the generated motion map and to compare the derived dilatation signal with an expected dilatation signal related to the expected motion map for comparing the generated motion map with the expected motion map of the skin region for a healthy subject.

3. Device as claimed in claim 2, wherein the processing unit is further configured to detect frequency and/or amplitude of the derived dilatation signal and to compare the detected frequency and/or amplitude with an expected frequency and/or amplitude and/or with a frequency and/or amplitude threshold or range.

4. Device as claimed claim 1, wherein the processing unit is further configured to detect a pulse movement region within the series of images for use as skin region.

5. Device as claimed in claim 4, wherein the processing unit is further configured to detect landmarks within the series of images and to detect the pulse movement region based on the detected landmarks.

6. Device as claimed in claim 1, wherein the processing unit is further configured to detect landmarks within the series of images and to perform motion compensation of the series of images for compensating motion of the subject's body not related to pulse of the subject.

7. Device as claimed in claim 5, wherein the processing unit is further configured to detect the landmarks from vascular network information of the subject.

8. Device as claimed in claim 7, wherein the processing unit is further configured to generate the vascular network information from said series of images or separate image information of the subject, in particular near-infrared images of the skin region of the subject.

9. Device as claimed claim 1, wherein the processing unit is further configured to generate a photo-plethysmographic, PPG, signal from the series of images, derive a pulse signal from the PPG signal, and determine the presence of pulse within the skin region based on the comparison and the pulse signal.

10. Device as claimed in claim 2, wherein the processing unit is further configured to check the pulse signal and the dilatation signal for coincidence of pulse presence, in particular for a time difference between pulses in the pulse signal and the dilatation signal.

11. Device as claimed in claim 10, wherein the processing unit is further configured to determine the presence of pulse if the pulse signal and the dilatation signal independently indicate a pulse having an amplitude above a first amplitude threshold and a frequency above a first frequency threshold, in particular an amplitude above a first amplitude threshold and below a second amplitude threshold and a frequency above a first frequency threshold and below a second frequency threshold.

12. Device as claimed in claim 4, wherein the processing unit is further configured to generate guiding information from a series of images for guiding an imaging unit for acquiring the series of images or a user operating the imaging unit to the pulse movement region and generate a control signal controlling the imaging unit to switch from a remote mode, in which first images of the skin region are acquired from a distance, to a contact mode, in which second images of the skin region are acquired with the imaging unit being in contact with or at a distance below 1 cm from the skin for the skin region, wherein the second images are used for determining the presence of pulse.

13. System for detection of pulse of a subject comprising: an imaging unit configured to acquire a series of images of a skin region of the subject, and a device or detection of pulse of a subject as claimed in claim 1 based on the acquired series of images.

14. Method for detection of pulse of a subject comprising: obtaining a series of images of a skin region of the subject, and processing said series of images by detecting pulse-related motion of the skin within the skin region from the series of images, generating a motion map of the skin region from the detected pulse-related motion, comparing the generated motion map with an expected motion map of the skin region for a healthy subject, and determining the presence of pulse within the skin region based on the comparison.

15. Computer program comprising program code means for causing a computer to carry out the steps of the method as claimed in claim 14 when said computer program is carried out on the computer.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0039] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. In the following drawings

[0040] FIG. 1 shows a schematic diagram of a first embodiment of a system and a device according to the present invention,

[0041] FIG. 2 shows a flow chart of a first embodiment of a method according to the present invention,

[0042] FIG. 3 shows a flow chart of a second embodiment of a method according to the present invention,

[0043] FIG. 4 shows a flow chart of a third embodiment of a method according to the present invention,

[0044] FIG. 5 shows an NIR image of the venous network of a hand,

[0045] FIG. 6A shows a schematic diagram of a second embodiment of a system according to the present invention in remote mode,

[0046] FIG. 6B shows a schematic diagram of the second embodiment of the system according to the present invention in contact mode,

[0047] FIG. 7 shows a flow chart of a fourth embodiment of a method according to the present invention,

[0048] FIG. 8 illustrates video stabilization based on feature point tracking in NIR images of skin,

[0049] FIG. 9A shows exemplary measurement signals including an accelerometer signal, a pulse signal obtained from skin motion and a respiration signal,

[0050] FIG. 9B shows a comparison of a heart rate inference from reference sensor and a camera signal, and

[0051] FIG. 10 shows an exemplary motion map of the carotid location.

DETAILED DESCRIPTION OF THE INVENTION

[0052] FIG. 1 shows a schematic diagram of a first embodiment of a system 1 and a device 2 according to the present invention. The system 1 for detection of pulse of a subject 100 comprises an imaging unit 3, e.g. a camera, configured to acquire a series of images 10 of a skin region 101 of the subject. The subject 100 may e.g. be an emergency patient who is treated by CPR or a patient in a hospital or an elderly care home. The skin region 101 may be region of the skin or neck, preferably the carotid location where pulse can be well observed. The images may depict a larger scene not only including skin, which may require further image processing to identify a desired skin region.

[0053] The system 1 further comprises a device 2 for detection of pulse of the subject 100 based on the acquired series of images. The device 2 generally comprises an input unit 2a (e.g. a data interface) configured to obtain (i.e. receive or retrieve) a series of images 10 of a skin region of the subject from the camera 3, either via a wired or wireless connection, and a processing unit 2b (e.g. a processor) for processing said series of images 10. The device 2 may e.g. be implemented in software and/or hardware, e.g. as a correspondingly programmed processor or computer. It may be a separate device or part of a conventional user device, such as a smartphone, smart watch, tablet, picture camera, glasses, etc.

[0054] FIG. 2 shows a flow chart of a first embodiment of a method 20 according to the present invention. In a first step 21 pulse-related motion 11 of the skin within the skin region is detected from the series of images 10. In a second step 22 a motion map 12 of the skin region 101 is generated from the detected pulse-related motion 11. In a third step 23 the generated motion map 12 is compared with an expected motion map 13 of the skin region, which may e.g. be stored in a storage unit. In a fourth step 24 the presence of pulse is determined within the skin region based on the comparison, i.e. the result 14 of the comparison is used to generate an indication 15 if pulse is present within the subject 100 or not.

[0055] The expected motion map 13 preferably represents pulse-related skin motion of a healthy person. Hereby, different expected motion maps may be available for different kinds of patients, e.g. for different ages, different genders, different health status, etc.

[0056] In an embodiment, step 23 of comparing the generated motion map 12 with an expected motion map 13 of the skin region comprises a step of deriving a dilatation signal reflecting arterial dilatation from the generated motion map and a step of comparing the derived dilatation signal with an expected dilatation signal related to the expected motion derived compared with an expected frequency and/or amplitude. Alternatively or in addition, the frequency and/or amplitude of the derived dilatation signal may be compared with a frequency and/or amplitude threshold or range. For instance, in the case of a male adult patient, the typical range of pulse rate and amplitude of pulse of a healthy male adult person may be used for this comparison.

[0057] FIG. 3 shows a flow chart of a second embodiment of a method 30 according to the present invention. In this embodiment, a second signal is derived from the images and used for independently deriving a pulse signal used in the determination of pulse. In particular, in step 31 a PPG signal 16 is derived from the series of images 10, which is then used to derive a pulse signal 17 from the PPG signal 16 in step 32. This pulse signal 17 is then used in addition to the result 14 of the comparison in step 23 to determine the presence of pulse within the skin region in step 24.

[0058] In a practical realization of this embodiment, remote monitoring of pulse from a distance with a static or handheld camera is exploited. Video processing of data acquired by the camera generates robust motion maps from the area of interest (e.g. head and carotid location) and deduces pulse presence from i) color changes of the skin (using remote PPG technique) and ii) presence of a dilatation signal from the carotid. As result, an indication (signal 15) of a life sustaining pulse is provided if both signal sources indicate pulse presence and the detected pulse has a sufficiently high and stable rate. Hereby, various modifications and implementations, e.g. as needed for a particular application or as appropriate for a particular type of patient, may be made. Further, in an implementation the existence of pulse in only one of the two signal sources may be considered sufficient to determine that pulse is present, e.g. if the other signal source is unreliable or not existent at all. Further, in an implementation the device may be configured to take appropriate action, e.g. to issue an alarm or guide the user to change a setting of the camera (e.g. the viewing direction, the distance from the patient, the sensitivity, etc.) if no signal or only the dilatation signal is present.

[0059] FIG. 4 shows a flow chart of a third embodiment of a method 40 according to the present invention. In this embodiment, in an initial step 41 (or repeatedly or continuously during the operation) optimal image acquisition settings, such as optimal magnification and viewing direction, are determined and used to control the camera (via a control signal 18) for subsequent acquisition or during current acquisition of images 10 in step 42. For instance, a vascular (arterial and/or venous) network, represented by vascular network information 19, information may be acquired in advance or during the image acquisition by a near-infrared (NIR) camera or the same camera as used for image acquisition and may e.g. be stored in a storage unit. The venous network of a hand is shown as an example in FIG. 5.

[0060] Further, in step 41 a pulse movement region for use as skin region, such as the carotid location, may be detected from the vascular network information 19. For instance, landmarks may be used, e.g. derived from the vascular network, such as characteristic branches and/or junctions.

[0061] As an alternative, the pulse movement region may be detected initially in step 43 (substantially corresponding to step 21) based on the series of images 10. Within the images 10 landmarks may be detected and used to detect the pulse movement region.

[0062] Body landmarks and, if available, vascular network (e.g. made visible as reference) may further be used for compensation of relative motion artefacts in step 22 or 23 in order to stabilize the motion map and remove motion that is not caused by the beating heart as much as possible, such a motion caused by movements of the body initiated by the subject or resulting from CPR chest compression. Further, other image artefacts as e.g. caused by changing lighting conditions may be observed and compensated as well.

[0063] In step 44 (substantially corresponding to step 22) a real-time stabilized motion (i.e., skin pulsation) in the form of a motion map 12 is computed and in step 45 (substantially corresponding to step 31) a PPG map 16′ is computed based on body landmarks and, optionally, venous/arterial trees by appropriate imaging techniques (e.g. NIR).

[0064] Subsequently a combined analysis of the stabilized motion map 12 and the PPG map 16′ is performed. The presence of artefacts may be checked so that they can be corrected, before the stabilized motion map 12 and the PPG map 16′ are used to derived pulse-related signals.

[0065] In step 46 (substantially corresponding to step 23) pulse is checked in the motion (i.e., skin pulsation) map 12. For instance, it is checked for pulse presences at expected skin area for an artery dilation signal including plausibility checks in expected signal decrease with increasing distance from carotid location. If the contrast is appropriate, pulse presence is deduced from the artery dilatation.

[0066] In step 47 (substantially corresponding to step 32) pulse is checked in video color information (substantially corresponding to step 32) (i.e. the PPG map 16′).

[0067] In step 48 it is checked for coincidence of pulse presence in both signals 14 the pulses in both independent pulse signals should be within a certain acceptable range. For instance, in an exemplary embodiment presence of pulse is determined if the pulse signal and the dilatation signal independently indicate a pulse having an amplitude above a first amplitude threshold and a frequency above a first frequency threshold, in particular an amplitude above a first amplitude threshold and below a second amplitude threshold and a frequency above a first frequency threshold and below a second frequency threshold. If pulse presence is only detected in the dilatation signal (i.e. the motion map) a low perfusion pressure may be indicated and it may be recommended to have manual palpation check or to start CPR.

[0068] FIG. 6 shows a schematic diagram of a second embodiment of a system 1′ and a device 3′ according to the present invention, wherein FIG. 6A shows it in remote mode and FIG. 6B shows it is contact mode. FIG. 7 shows a flow chart of a corresponding embodiment of a method 50 according to the present invention, which may be used by this system 1′. In this embodiment an on-body (contact) vital sign check for assessment in acute care situation is used.

[0069] In a first step 51 the images 61 (in particular a video) are continuously acquired in remote mode (shown in FIG. 6A) from a distance using a movable or handheld camera 2. These images 61 are used in step 52 to guide the user (or directly the camera 2) to the desired skin region, e.g. the recommended carotid location. Location information is maintained when the camera 3′ is approaching the body and the sensor mode switches to contact mode (shown in FIG. 6B) in step 53, when the camera 3′ is close to the skin (e.g. at a distance of 1 cm or less) or touches the skin.

[0070] In contact mode, inferred from motion robust maps from the optimal area of interest (e.g. head and carotid location) pulse presence is deduced in step 54 from i) color changes of the skin and ii) presence of a dilatation signal from the carotid. An indication of a life-sustaining pulse is provided in step 55 if both signal sources indicate pulse presence and the detected pulse has a sufficiently high and stable rate. If no signal or only the dilatation signal is present, it is indicated in step 55 to take appropriate action.

[0071] In contact mode a dedicated light source 4, e.g. an LED mounted at the camera 3′, preferably illuminates the skin region. The light received from the skin in response to said illumination is sensed by the camera and evaluated to obtain the dilatation signal and the PPG signal.

[0072] Further, an orientation sensor 5, preferably mounted at the camera, may be Thus, the camera or user can be guided to the optimal placement, e.g. by use of a light spot to mark the carotid location. Further, orientation changes may thus be corrected via signals from the orientation sensor and, optionally, body landmarks.

[0073] With this embodiment of the system 1′, when approaching the body, the user can be guided to the best recommended location for pulse detection. The video recording is started off-body. The body location and the orientation of the camera may be taken as references and corrected for the changes in the camera position relative to the body when approaching the body, in order to keep physiological constraints for optimal body location.

[0074] As in other embodiments, a motion map (i.e. skin pulsation) and a PPG map may be used for pulse presence inference. Further, an automatic adjustment may be provided for optimal magnification, e.g. for the size of the detected arteries/veins. A check is made for coincidence of pulse presence in both signals. If the PPG signal is missing, it may be checked for motion presence including a plausibility check of geometries/physiological constraints. Motion contrast in the motion map may be used to deduce motion presence in plausible body area. Further, a check for pulsatility versus artifact may be made. If only motion is present, a low perfusion pressure may be indicated and/or it may be recommended to have manual palpation check or start CPR.

[0075] The camera may also be used to perform an assessment of environmental conditions, hazards and terrain challenges. Challenging terrain (e.g., bush, soft snow, sloping, rocky or uneven ground, etc.), harsh environmental conditions (e.g., snow, mist, fog, rain, low lighting) and safety hazards (e.g., downed power lines, fire, smoke, vehicle traffic, etc.) present significant barriers to the delivery of effective CPR, and may partly contribute to the reported low post-CPR survival rate (for out of hospital cardiac arrest) of 9.5-11.4%. For example, performing chest compressions during CPR on an inclined or soft back support surface will produce shallower, less effective compressions, which will lead to poorer survival outcomes. Moreover, current CPR guidelines recommend that prior to initiation of CPR lay rescuers should always check for any potential hazards and should only approach the victim after determining that the scene is safe. This is especially important since many people may act impulsively and place themselves in harm's way, due to difficulties in thinking clearly as a result of the highly stressful emergency situation.

[0076] In practical applications there is a high possibility of relative motion between the camera and the subject. In such cases, the output video from the camera should be stabilized so that the only motion now left is due to heart beat, e.g. the pulsating carotid. This contrast like hair etc., but in low contrast areas such as the region around the carotid stabilization can be challenging with conventional RGB cameras. One way to improve stabilization or registration of frames can be using the underlying venous network to provide the necessary distinctive features. FIG. 5 shows an example of the increase of contrast and distinctive features when the skin is viewed in near-infrared.

[0077] With such a distinctive set of features the video stabilization can be made very robust. An example of this approach is illustrated in FIG. 8. Here the infrared image of the hand is shown along with the detection of the distinct feature points (high contrast regions such as corners). In each frame these features are detected and then the image is transformed such that the features lie on top of each other in subsequent frames. This leads to a stabilized video where the relative motion between the camera and the subject is minimized greatly if not completely removed. Once stabilized the infrared video stream can also be used to measure the carotid motion.

[0078] FIG. 9A shows exemplary measurement signals including an accelerometer signal 60, a pulse signal 61 obtained from skin motion and a respiration signal 62 and FIG. 9B shows a comparison of a heart rate inference 70 from reference sensor and a camera signal 71. Obviously there is a good overlap. It shall be noted that absolute heart rate accuracy is of less importance for CPR applications.

[0079] FIG. 10 shows an exemplary motion map 80 of the carotid location. The grey zones mark areas with high motion signal due to carotid dilation. Obviously, observed motion is largest in an area, where the carotid is located underneath. The motion signal strength decreases when moving outwards away from the carotid artery, which is expected as well. In combination with a detected arterial/venous network, the obtained dilatation signal can be more accurately and reliably evaluated even when there is no signal present.

[0080] The present invention can be applied in advanced cardiopulmonary resuscitation (CPR), e.g. for in-hospital CPR (e.g. to monitor defibrillators) or out-of-hospital CPR by medical professionals (e.g. advanced AEDs having PPG and a cuff-based blood pressure measurement on board). Further applications are in basic life support and lay responders (e.g. AEDs for public use, with PPG and cuff-based blood pressure measurements on board), mass casualty triaging, combat casualty assessment in military settings, and home monitoring solutions where people can sit in front of a (web-)camera to measure their pulse and breathing rate. The disclosed device may be implemented in smartphones, smartwatches tablets or as standalone device.

[0081] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

[0082] In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0083] A computer program may be stored/distributed on a suitable non-transitory medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0084] Any reference signs in the claims should not be construed as limiting the scope.