PULSE RATE ESTIMATION METHOD, DEVICE, AND SYSTEM

Abstract

Provided is a method for estimating a pulse rate with high accuracy in the case that short-term burst noise is mixed in an estimation interval. Included are a pulse wave signal generating step of generating a pulse wave signal from an image of a pulse rate estimation target; a peak component suppression step of limiting an amplitude value of the pulse wave signal that is larger than a first threshold value to the first threshold value, and outputting a pulse wave analysis signal; and a frequency analysis step of outputting a frequency spectrum of the pulse wave analysis signal.

Claims

1. A pulse rate estimation method comprising: a pulse wave signal generating step of generating a pulse wave signal from an image of a pulse rate estimation target; a peak component suppression step of limiting an amplitude value of the pulse wave signal that is larger than a first threshold value to the first threshold value, and outputting a pulse wave analysis signal; a frequency analysis step of outputting a frequency spectrum of the pulse wave analysis signal; and an output determination step of outputting, as a pulse rate, a frequency at which a maximum peak value is obtained when a ratio of a second largest peak value to a maximum peak value of the frequency spectrum is less than a second threshold value.

2. (canceled)

3. A pulse rate estimation method comprising: a pulse wave signal generating step of generating a pulse wave signal from an image of a pulse rate estimation target; a peak component suppression step of limiting an amplitude value of the pulse wave signal that is larger than a first threshold value to the first threshold value, wherein a feature of a waveform of the pulse wave signal is extracted, and a sample value of the pulse wave signal that has a large amplitude outside a range of the first threshold value calculated based on the feature of the waveform is replaced with the first threshold value, and outputting a pulse wave analysis signal; a frequency analysis step of outputting a frequency spectrum of the pulse wave analysis signal; and an output determination step of outputting, as a pulse rate, a frequency at which a maximum peak value is obtained when a ratio of a second largest peak value to a maximum peak value of the frequency spectrum is less than a second threshold value.

4. The pulse rate estimation method of claim 3, wherein the feature of the waveform is a standard deviation.

5. A pulse rate estimation device comprising: a pulse wave signal generation unit for generating a pulse wave signal from an image of a pulse rate estimation target; a noise suppression unit for limiting an amplitude value of the pulse wave signal that is larger than a first threshold value to the first threshold value, and outputting a pulse wave analysis signal; a pulse wave signal analysis unit for outputting a frequency spectrum of the pulse wave analysis signal; and an output determination unit for outputting, as a pulse rate, a frequency at which a maximum peak value is obtained when a ratio of a second largest peak value to a maximum peak value of the frequency spectrum is less than a second threshold value.

6. (canceled)

7. The pulse rate estimation device according to claim 5, wherein the noise suppression unit extracts a feature of a waveform of the pulse wave signal, and replaces a sample value of the pulse wave signal that has a large amplitude outside a range of the first threshold value calculated based on the feature of the waveform with the first threshold value.

8. The pulse rate estimation device according to claim 7, wherein the feature of the waveform is a standard deviation.

9. A pulse rate estimation system comprising: an imaging unit for supplying the image; the pulse rate estimation device according to claim 5; and a display unit for displaying an output pulse rate.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0020] FIG. 1 is a functional block diagram of a pulse rate estimation system according to the embodiments of the present invention.

[0021] FIG. 2 is a flow chart of a pulse rate estimation process of the pulse rate estimation device illustrated in FIG. 1.

[0022] FIG. 3 is a flow diagram of the pulse signal generation step illustrated in FIG. 2.

[0023] FIG. 4 is a flowchart of the output determination step illustrated in FIG. 2.

[0024] FIG. 5 is a conceptual diagram of a signal s1+s2 in a case that burst noise 52 is mixed in a pulse wave signal Si and a signal (a pulse wave analysis signal) whose amplitude is limited to a threshold value a.

[0025] FIG. 6 is a frequency spectrum of s1+s2 and the pulse wave analysis signal illustrated in FIG. 5.

DESCRIPTION OF EMBODIMENT(S)

[0026] Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the present invention is not limited to these embodiments. Further, in the description of the drawings, the same portions are denoted by the same reference numerals.

[0027] Embodiments of the present invention will be described below with reference to the drawings.

[0028] FIG. 1 is a functional block diagram of a pulse rate estimation system according to the embodiments of the present invention, and FIG. 2 is a flow chart of the pulse rate estimation process of the pulse rate estimation device 200 illustrated in FIG. 1.

[0029] The pulse rate estimation system includes an imaging unit 100, a pulse rate estimation device 200, and a display unit 300. The imaging unit 100 is an image capture device realized by an image sensor such as a CMOS or CCD, has a function of acquiring an image of a pulse rate estimation target, and may be, for example, a camera mounted on a smartphone or a notebook computer, a surveillance camera, a home video camera or the like. The pulse rate estimation device 200 is a device for estimating a pulse rate from the acquired image, and is realized by a computing processor such as a CPU, GPU, FPGA, or the like mounted in a personal computer, a smartphone, or the like. The display unit 300 has a function for displaying an estimation result of the pulse rate, an alert based on the estimation result, or the like, and is realized by a liquid crystal display or the like. In the present embodiment, a display example using a liquid crystal display is illustrated, but voice notification may be performed by a speaker. In addition, the estimation result of the pulse rate may be saved on a hard disk in the personal computer or in a cloud, and may be used as data for statistical analysis.

[0030] The pulse rate estimation device 200 includes a pulse wave signal generation unit 210, a noise suppression unit 220, a pulse wave signal analysis unit 230, an output determination unit 240 and a history storage unit 250. The pulse wave signal generation unit 210 generates a pulse wave signal from an image of a pulse rate estimation target, and includes an image acquisition unit 211, a region extraction unit 212, a waveform generation unit 213, a waveform storage unit 214 and a pulse wave signal extraction unit 215.

[0031] Next, a pulse rate estimation process in the pulse rate estimation device 200 will be described with reference to FIG. 2 to FIG. 4.

[0032] The pulse rate estimation process includes a pulse wave signal generation step S100, a peak component suppression step S200, a frequency analysis step S300, and an output determination step S400. The pulse wave signal generation step S100 generates a pulse wave signal from an image of the pulse rate estimation target, and includes an image input step S101, a face detection and tracking step S102, a skin extraction step S103, a pixel value averaging step S104, a time waveform generation step S105 and a pulse wave signal extraction step S106. The output determination step S400 outputs, as the pulse rate, a frequency at which a maximum peak value is obtained when the ratio of the second largest peak value to the maximum peak value of the frequency spectrum (in the case of decibels, the difference between the maximum peak value and the second largest peak value) is less than a second threshold value, and includes a peak detection step S401, a previous pulse rate display step S402, a pulse rate display step S403, and a display pulse rate storage step S404.

[0033] (S101: Image Input Step) The pulse rate estimation device 200 receives a video signal obtained from the imaging unit 100 by the image acquisition unit 211. In the present embodiment, an example of an image sensor capable of sensing an RGB spectrum is shown, but video signals may be generated by sensors that can sense spectra such as C (cyan), O (orange), and near infrared rays as described in Non-Patent Document 2 (an example of a five-band RGBCO camera), and Non-Patent Document 3 (an example of a near-infrared camera).

[0034] (S102: Face Detection/Tracking Step, S103 Skin Extraction Step) The region extraction unit 212 analyzes the received video signal, detects a face region, and then extracts a region (pixels) including the skin to be detected. Since video signals are sequentially sent from the imaging unit 100, the tracking processing of the face area is also performed so as to correspond to the movement of the face. In the present embodiment, a pulse rate estimation process using a face image is described, but the pulse rate estimation process may be performed on, for example, a palm, a neck, or the like, and may be performed on any region where skin is exposed.

[0035] (S104: Pixel Value Averaging Step, S105: Temporal Waveform Generation Step) The waveform generation unit 213 calculates the average value of the pixel values of the skin regions detected for each of the R (red), G (green), and B (blue) components, and stores the average value in the waveform storage unit 214. The waveform storage unit 214 stores the pixel values in chronological order, and can generate a time-series waveform by sequentially reading them out.

[0036] (S106: Pulse Wave Signal Extraction Step) The pulse wave signal extraction unit 215 extracts a component corresponding to the pulse wave signal from the time series waveform. A variety of extraction methods have been proposed, including those that use only the G components, those that perform independent component analysis (ICA), those that perform principal component analysis (PCA), and synthesis methods based on SNR (signal-to-noise power ratio) for RGB. It should be noted that, in the processing before and after the extraction of the pulse wave signal, the accuracy of the signal to be extracted may be improved by correction of the sampling interval, a bandpass filter (BPF), or the like.

[0037] (S200: Peak Component Suppression Step) In order to eliminate burst noise caused by body movements and changes in the light source, the noise suppression unit 220 limits the amplitude values of pulse wave signals that are larger than a first threshold value a to the first threshold value a and outputs the pulse wave analysis signal. FIG. 5 is a conceptual diagram of a signal s.sub.1+s.sub.2 in a case that burst noise s.sub.2 is mixed in a pulse wave signal s.sub.1 and a signal whose amplitude is limited to a threshold value a. In this example, the results are shown for a case in which a standard deviation a of the amplitude value of the signal between time t=0 seconds and t=5 seconds is used and where α=aσ and a=0.5. The standard deviation coefficient a may be set to an arbitrary positive value. The larger the value, the smaller the suppression effect of the noise s2, and the smaller the va1 lie, the greater the suppression effect of the noise s2, and at the same time, an effect of suppressing the pulse wave signal s.sub.1 is also achieved. Accordingly, it is desirable to make adjustments before starting operation of the equipment.

[0038] (S300: Frequency Analysis Step) The pulse wave signal analysis unit 230 performs a fast Fourier transform (FFT) with respect to the pulse wave analysis signal and outputs a signal energy (frequency spectrum) for each frequency. The frequency spectrum may be derived by using a plurality of bandpass filters having different pass bands, or by using a periodic signal extraction process corresponding to the bandpass filter as described in Non-Patent Document 1. FIG. 6 illustrates the frequency spectrum of s.sub.1+s.sub.2 of FIG. 5 and the pulse wave analysis signal. As illustrated in the figure, the pulse wave analysis signal has a spectrum in which the pulse wave signal has the maximum energy due to the effect of the noise suppression unit 220, and the pulse rate can be accurately estimated. Further, when the noise is small, the amplitude of the pulse wave signal in the sections where noise is not present is also limited, but since the energy of the pulse wave analysis signal is mainly due to the pulse wave signal, this is not problematic.

[0039] (S401: Peak Detection Step, S402: Previous Pulse Rate Display Step, S403: Pulse Rate Display Step, S404: Display Pulse Rate Storage Step) The output determination unit 240 detects the peak of the frequency spectrum. Further, the relationship between the maximum peak p.sub.1 and the second peak p.sub.2 (the second largest peak) is compared. In the case that p.sub.2/p.sub.1<β, it is assumed that the frequency f.sub.p1 having the maximum peak corresponds to the pulse wave, and 60f.sub.p1 is displayed on the display unit 300 as the pulse rate. In the case that noise occurs, multiple large peaks appear, such that measurement cannot be performed accurately. Accordingly, when p.sub.2/p.sub.1≥β, the pulse rate is not estimated, and the previously displayed pulse rate is displayed on the display unit 300. The second threshold value β may be set to an arbitrary value in the range of 0≤β≤1, and the displayed pulse rate is stored in the history storage unit 250.

[0040] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the essence of the present invention. For example, although the above-described embodiment relates to non-contact type pulse rate estimation using a camera image, the present invention can be applied to pulse rate estimation using radar as well as other periodic signal estimation.

[0041] When the pulse rate estimation device 200 is realized by a computer processor such as a CPU, a GPU, or a FPGA mounted in a personal computer or a smart phone as in the above-described embodiments, programs for causing the computer to execute the above-described pulse rate estimation processing are also within the scope of the present disclosure.

REFERENCE SIGNS LIST

[0042] 100: Imaging unit, 200: Pulse rate estimation device, 210: Pulse wave signal generation unit, 211: Image acquisition unit, 212: Region extraction unit, 213: Waveform generation unit, 214: Waveform storage unit, 215: Pulse wave signal extraction unit, 220 : Noise suppression unit, 230: Pulse wave signal analysis unit, 240: Output determination unit, 250: History storage unit, 300: Display unit, S100: Pulse wave signal generation step, S101: Image input step, S102: Face detection and tracking step, S103: Skin extraction step, S104: Pixel value averaging step, S105: Time waveform generation step, S106: Pulse wave signal extraction step, S200: Peak component suppression step, S300: Frequency analysis step, S400: Output determination step, S401: Peak detection step, S402: Previous pulse rate display step, S403: Pulse rate display step, 5S04: Display pulse rate storage step, s.sub.1: Pulse wave signal, s.sub.2: Noise signal