CONTACTLESS PHYSIOLOGICAL MEASUREMENT DEVICE AND METHOD

Abstract

A contactless physiological measurement device is disclosed. The contactless physiological measurement device is an electronic device comprising a processor and a memory storing an application program. When the application program is executed, the processor controls a front camera and a rear camera of the electronic device to photograph a user, so as to obtain a face image and a hand image. Subsequently, after extracting a first rPPG signal and a second rPPG signal from the face image and the hand image respectively, physiological parameters are calculated by apply a signal process to the first/second rPPG signal. Moreover, a signal parameter difference is also acquired after applying a signal difference calculation to the two rPPG signals. Consequently, an estimation value of blood pressure is outputted by inputting user anthropometric parameter, said signal parameter difference and at least one physiological parameter into a pre-trained blood pressure estimating model.

Claims

1. A contactless physiological measurement device, being provided in a form of an electronic device having a front camera and a rear camera, and comprising: a memory storing an application program; and a processor, being coupled to the memory, and being also coupled to the front camera and the rear camera; wherein the application program includes instructions, such that in case the application program is executed, the processor being configured for: acquiring, by controlling the front camera and the rear camera to photograph a user, at least one first image frame containing a face image and at least one second image frame containing a hand image; extracting, from the face image of said first image frame and the hand image of said second image frame, respectively, a first rPPG signal and a second rPPG signal; calculating, by applying a signal difference calculation to the first rPPG signal and the second rPPG signal, at least one signal parameter difference; calculating, by applying a signal process to the first rPPG signal and the second rPPG signal, a plurality of physiological parameters; and outputting, by inputting at least one user anthropometric parameter, at least one said physiological parameter and said signal parameter difference into a pre-trained blood pressure estimating model, an estimation value of blood pressure.

2. The contactless physiological measurement device of claim 1, wherein the electronic device is selected from a group consisting of smart phone, tablet computer, laptop computer, and all-in-one computer.

3. The contactless physiological measurement device of claim 1, wherein the plurality of physiological parameters comprises heart rate (HR), heart rate variability (HRV), respiratory rate, and blood oxygen saturation (SpO2).

4. The contactless physiological measurement device of claim 1, wherein said user anthropometric parameter is selected from a group consisting of weight, height, body mass index (BMI) and age.

5. The contactless physiological measurement device of claim 1, wherein the memory comprises a database storing at least one user anthropometric data and the plurality of physiological parameters.

6. The contactless physiological measurement device of claim 1, wherein the application program consists of a plurality of subprograms, and the plurality of subprograms comprising: a first subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to detect the face image from the first face image frame; a second subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to extract the first rPPG signal and the second rPPG signal from the face image of said first image frame and the hand image of said second image frame, respectively; a third subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to apply said signal difference calculation to the first rPPG signal and the second rPPG signal, thereby calculating the at least one signal parameter difference; and a fourth subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to input said user anthropometric parameter, said signal parameter difference and at least one said physiological parameter into the pre-trained blood pressure estimating model, such that the pre-trained blood pressure estimating model outputs said estimation value of blood pressure.

7. A contactless physiological measurement method, being compiled to be an application program so as to be stored in a memory of an electronic device, and being conducted by a processor of the electronic device; the contactless physiological measurement method comprising the steps of: (1) acquiring, by controlling a front camera and a rear camera of the electronic device to photograph a user, at least one first image frame containing a face image and at least one second image frame containing a hand image; (2) extracting, from the face image of said first image frame and the hand image of said second image frame, respectively, a first rPPG signal and a second rPPG signal; (3) calculating, by applying a signal difference calculation to the first rPPG signal and the second rPPG signal, at least one signal parameter difference; (4) calculating, by applying a signal process to the first rPPG signal and the second rPPG signal, a plurality of physiological parameters; and (5) outputting, by inputting at least one user anthropometric parameter, at least one said physiological parameter and said signal parameter difference into a pre-trained blood pressure estimating model, an estimation value of blood pressure.

8. The contactless physiological measurement method of claim 7, wherein the electronic device is selected from a group consisting of smart phone, tablet computer, laptop computer, and all-in-one computer.

9. The contactless physiological measurement method of claim 7, wherein said user anthropometric parameter is selected from a group consisting of weight, height, body mass index (BMI) and age.

10. The contactless physiological measurement method of claim 7, wherein the plurality of physiological parameters comprises heart rate (HR), heart rate variability (HRV), respiratory rate, and blood oxygen saturation (SpO2).

11. The contactless physiological measurement method of claim 7, wherein the memory comprises a database storing at least one user anthropometric data and the plurality of physiological parameters.

12. The contactless physiological measurement method of claim 7, wherein the application program consists of a plurality of subprograms, and the plurality of subprograms comprising: a first subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to detect the face image from the first image frame; a second subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to extract the first rPPG signal and the second rPPG signal from the face image of said first image frame and the hand image of said second image frame, respectively; a third subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to apply said signal difference calculation to the first rPPG signal and the second rPPG signal, thereby calculating the at least one signal parameter difference; and a fourth subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor 1P to input said user anthropometric parameter, said signal parameter difference and at least one said physiological parameter into the pre-trained blood pressure estimating model, such that the pre-trained blood pressure estimating model outputs said estimation value of blood pressure.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The invention as well as a preferred mode of use and advantages thereof will be best understood by referring to the following detailed description of an illustrative embodiment in conjunction with the accompanying drawings, wherein:

[0030] FIG. 1 shows a first stereo diagram of a contactless physiological measurement device according to the present invention;

[0031] FIG. 2 shows a second stereo diagram of the contactless physiological measurement device;

[0032] FIG. 3 shows a block diagram of a memory and a processor;

[0033] FIG. 4 shows a diagram for describing how to use the contactless physiological measurement device;

[0034] FIG. 5 shows a waveform diagram of a first rPPG signal and a second rPPG signal; and

[0035] FIG. 6 shows a flowchart of a contactless physiological measurement method according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0036] To more clearly describe a contactless physiological measurement device and method according to the present invention, embodiments of the present invention will be described in detail with reference to the attached drawings hereinafter.

[0037] With reference to FIG. 1, there is shown a first stereo diagram of a contactless physiological measurement device according to the present invention. Moreover, FIG. 2 illustrates a second stereo diagram of the contactless physiological measurement device. As FIG. 1 and FIG. 2 shows, the present invention discloses a contactless physiological measurement device 1, which is provided in a form of an electronic device, e.g., smart phone, tablet computer, laptop computer, and all-in-one computer, and comprises a front camera 11, a rear camera 12, a memory 1M, and a processor 1P. As described in more detail below, the processor 1P is coupled to the memory 1M, and is also coupled to the front camera 11 and the rear camera 12.

[0038] FIG. 3 shows a block diagram of the memory 1M and the processor 1P, and FIG. 4 shows a diagram for describing how to use the contactless physiological measurement device 1. Particularly, there is an application program 1AP stored in the memory 1M, and the application program 1AP comprises: a first subprogram 1FD, a second subprogram 1rP, a third subprogram 1SD, and a fourth subprogram 1BP. By such arrangements, in case the application program 1AP is executed, the processor 1P firstly controls the front camera 11 and the rear camera 12 to photograph a user, so as to acquire at least one first image frame containing a face image and at least one second image frame containing a hand image. For example, as FIG. 4 shows, the contactless physiological measurement device 1 is provided in a form of smart phone, and is held in left hand of a user. Moreover, the front camera 11 of the smart phone is faced to the front of the user, and the rear camera 12 of the smart phone is faced to an index finger of the user's left hand. In such case, the front camera 11 is controlled (by the processor 1P) to acquire at least one first image frame from the front of the user, and the rear camera 12 is controlled to acquire at least one second image frame from the left hand of the user, such that said first image frame contains a (user) face image, and said second image frame contains a hand image.

[0039] As explained in more detail below, the first subprogram 1FD is compiled to be integrated in the application program 1AP by one type of programming language, and includes instructions for configuring the processor 1P to perform a face detection function, thereby detecting the face image from the first image frame.

[0040] In the execution of the application program 1AP, the processor 1P subsequently extract a first rPPG signal from the face image of said first image frame and a second rPPG signal from the hand image of said second image frame. In one embodiment, the second subprogram 1rP is also compiled to be integrated in the application program 1AP by one type of programming language, and includes instructions for configuring the processor 1P to extract the first rPPG signal and the second rPPG signal from the face image of said first image frame and the hand image of said second image frame, respectively. FIG. 5 shows a waveform diagram of the first rPPG signal and the second rPPG signal.

[0041] According FIG. 3 and FIG. 4, the processor 1P subsequently applies a signal difference calculation to the first rPPG signal and the second rPPG signal, so as to calculate at least one signal parameter difference between the two rPPG signals. In FIG. 5, said signal parameter difference is labeled to ΔT and/or ΔA, where ΔT means a pulse interval time (PT), and AA represents a pulse amplitude difference. As explained in more detail below, in a practicable embodiment, the third subprogram 1SD is compiled to be integrated in the application program 1AP by one type of programming language, and including instructions for configuring the processor 1P to perform the function of calculating the at least one signal parameter difference between the first rPPG signal and the second rPPG signal.

[0042] In the execution of the application program 1AP, the processor 1P subsequently calculates a plurality of physiological parameters by applying a signal process to the first rPPG signal or the second rPPG signal, wherein the plurality of physiological parameters comprise, but are not limit to, heart rate (HR), heart rate variability (HRV), respiratory rate, and blood oxygen saturation (SpO2).

[0043] In one practicable embodiment, a time-domain signal is generated after applying said signal process to the first rPPG signal or the second rPPG signal, such that at least one time-domain parameter is allowed extracted from the time-domain signal. Said time-domain parameter is adopted in the calculation of heart rate variability (HRV), and can be any one of, but is not limit to, standard deviation of all normal to normal intervals (SDNN), root mean square successive differences (RMSSD), number of pairs of adjacent normal to normal intervals differing by more than 50 ms (NN50), and proportion of NN50 divided by a total number of all normal to normal intervals (pNN50).

[0044] In another one practicable embodiment, a frequency-domain signal is generated after applying said signal process to the first rPPG signal or the second rPPG signal, such that at least one frequency-domain parameter is allowed extracted from the frequency-domain signal. Said frequency-domain parameter can be, but is not limit to, total power (TP), high frequency power (HF), low frequency power (LF), very low frequency power (VLF), ultra low frequency power (ULF), low frequency proportion (LF %), or LF/HF ratio. Herein, it is worth mentioning that, literature 1 has fully introduced the relationship between TP, HF, LF, VLF, ULF, and/or LF % and the physiological parameters. Literature 1, written by F. Shaffer et al, is entitled with “An Overview of Heart Rate Variability Metrics and Norms” and has DOI:10.3389/fpubh.2017.00258.

[0045] Eventually, the instructions included in the application program 1AP configure the processor 1P to input at least one user anthropometric parameter, at least one said physiological parameter and said signal parameter difference into a pre-trained blood pressure estimating model, such that the pre-trained blood pressure estimating model outputs an estimation value of blood pressure. Herein, said user anthropometric parameter can be, but is not limit to, weight, height, body mass index (BMI) and age. As explained in more detail below, the fourth subprogram 1BP is compiled to be integrated in the application program by one type of programming language, and includes instructions for configuring the processor 1P to perform the function of inputting said user anthropometric parameter, said signal parameter difference and at least one said physiological parameter into the pre-trained blood pressure estimating model, thereby outputting said estimation value of blood pressure.

[0046] In addition, according to FIG. 3, the memory 1M can be arranged to comprises a database storing at least one user anthropometric data and the plurality of physiological parameters.

[0047] Therefore, through the above descriptions, all embodiments of the contactless physiological measurement device 1 according to the present invention have been introduced completely and clearly. Moreover, the present invention simultaneously discloses a contactless physiological measurement method. As FIG. 1, FIG. 2 and FIG. 3 show, the method is compiled to be an application program 1AP so as to be stored in a memory 1M of an electronic device, and is conducted by a processor 1P of the electronic device.

[0048] FIG. 6 illustrates a flowchart of a contactless physiological measurement method according to the present invention. According to FIG. 6, the contactless physiological measurement method comprising the steps of: [0049] S1: acquiring, by controlling a front camera 11 and a rear camera 12 of the electronic device 1 to photograph a user, at least one first image frame containing a face image and at least one second image frame containing a hand image; [0050] S2: extracting, from the face image of said first image frame and the hand image of said second image frame, respectively, a first rPPG signal and a second rPPG signal; [0051] S3: calculating, by applying a signal difference calculation to the first rPPG signal and the second rPPG signal, at least one signal parameter difference; [0052] S4: calculating, by applying a signal process to the first rPPG signal and the second rPPG signal, a plurality of physiological parameters; and [0053] S5 outputting, by inputting at least one user anthropometric parameter, at least one said physiological parameter and said signal parameter difference into a pre-trained blood pressure estimating model, an estimation value of blood pressure.

[0054] Therefore, above descriptions have introduced the contactless physiological measurement device and method according to the present invention completely and clearly. Moreover, the above description is made on embodiments of the present invention. However, the embodiments are not intended to limit the scope of the present invention, and all equivalent implementations or alterations within the spirit of the present invention still fall within the scope of the present invention.