Living Subject Identification Using Image/Video Discriminator For RADAR Systems

Abstract

A method and system for presence and vitals detection of a living subject is disclosed herein. The system comprises a device that monitors and is an interface. The device comprises an RGB/IR imaging sensor, a radar, a processor, and a first communication module. The processor is configured to run an algorithm to perform digital signal processing on data provided by the radar and the RGB/IR imaging sensor to generate presence and vitals information for the living subject for communication to the device.

Claims

1. A system for presence and vitals detection of a living subject, the system comprising: an IR imaging sensor; a radar; a processor; and a user interface; wherein the IR imaging sensor is utilized to detect light reflected by a living subject from ambient or controlled light sources; wherein the radar emits a radiofrequency at a specific frequency, and detects the frequency change of reflections of a plurality of targets which have subtle movements from the living subject; wherein the processor is configured to run an algorithm to perform digital signal processing on data provided by the radar and the IR sensor to generate presence and vitals information for the living subject for communication to the user interface.

2. The system according to claim 1 wherein the subtle movements are caused by the respiration and/or ballistocardiography from the living subject.

3. The system according to claim 1 wherein the radar is a pulsed Doppler radar or a continuous wave Doppler radar.

4. The system according to claim 1 wherein the user interface comprises a second communication module for receiving data from a first communication module in communication with the processor.

5. The system according to claim 1 wherein the IR imaging sensor is one of an RGB IR imaging sensor or SW IR imaging sensor.

6. The system according to claim 1 wherein the user interface presents data using a LED, a display or a speaker.

7. A method for presence and vitals detection of a living subject, the method comprising: detecting at an IR imaging sensor of a monitoring device, light reflected by a living subject from ambient or controlled light sources; emitting from a radar of the monitoring device a radiofrequency at a specific frequency, and detecting the frequency change of reflections of a plurality of targets which have subtle movements from the living subject; receiving at a processor of the monitoring device the data from the radar and the IR imaging sensor; running on the processor an algorithm to perform digital signal processing on the data provided by the radar and the IR imaging sensor to generate presence and vitals information for the living subject; communicating from a first communication module of the monitoring device the presence and vitals information for the living subject to a second communication module of an interface device; and presenting on a user interface module of the interface device the presence and vitals information for the living subject.

8. The method according to claim 7 wherein the subtle movements are caused by the respiration and/or ballistocardiography from the living subject.

9. The method according to claim 7 wherein the radar is a pulsed Doppler radar or a continuous wave Doppler radar.

10. The method according to claim 7 wherein the IR imaging sensor is an RGB IR imaging sensor or a SW IR imaging sensor.

11. The method according to claim 7 wherein the first communication module and the second communication module operate on a WiFi communication protocol, a BLUETOOTH communication protocol, a FM communication protocol, or a FHSS communication protocol.

12. A system for presence and vitals detection of a living subject, the system comprising: a monitoring device comprising an imaging sensor, a radar, a processor, and a first communication module; and an interface device comprising a second communication module and a user interface module; wherein the RGB imaging sensor is utilized to detect light reflected by a living subject from ambient or controlled light sources; wherein the radar emits a radiofrequency at a specific frequency, and detects the frequency change of reflections of a plurality of targets which have subtle movements from the living subject; wherein the processor is configured to run an algorithm to perform digital signal processing on data provided by the radar and the RGB imaging sensor to generate presence and vitals information for the living subject for communication to the interface device.

13. The system according to claim 12 further comprising a memory configured to store sensor output from the sensor.

14. The system according to claim 12 wherein the subtle movements are caused by the respiration and/or ballistocardiography from the living subject.

15. The system according to claim 12 wherein the user interface presents data using a LED, a display or a speaker.

16. The system according to claim 12 wherein the user interface comprises a second communication module for receiving data from a first communication module in communication with the processor.

17. The system according to claim 16 wherein the first communication module and the second communication module operate on a WiFi communication protocol, a BLUETOOTH communication protocol, a FM communication protocol, or a FHSS communication protocol.

18. The system according to claim 12 wherein the IR imaging sensor is an RGB IR imaging sensor or a SW IR imaging sensor.

19. The system according to claim 12 wherein the radar is a pulsed Doppler radar or a continuous wave Doppler radar.

Description

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0014] FIG. 1 is an illustration of an antenna pattern.

[0015] FIG. 2 is a block diagram of one embodiment of a system for presence and vitals detection of a living subject.

[0016] FIG. 3 is a block diagram of a process for presence and vitals detection of a living subject.

[0017] FIG. 4 is a flow chart for a method for presence and vitals detection of a living subject.

[0018] FIG. 5 is a flow chart for a method for presence and vitals detection of a living subject.

DETAILED DESCRIPTION OF THE INVENTION

[0019] Radar has the sensitivity to detect vitals but for presence detection has low confidence. A camera is able to detect presence with high confidence but does not have the resolution for reliable vitals detection. However, using both radar and a camera, presence and vitals detection with high confidence can be achieved.

[0020] An RGB/SW IR imaging sensor is used to detect light reflected by a living subject from ambient (sunlight, room lighting, etc.) or controlled (light bulb, SW IR LED, et.) light sources. A Doppler radar, either pulsed or continuous wave, emits RF at a specific frequency, and detects the frequency change of reflections of targets which have subtle movements caused by the respiration and/or ballistocardiography from a living subject.

[0021] The field of view of the imaging sensor and the transmit and receive antenna lobe sensitivity are roughly collocated with similar attitude (azimuth and zenith angle) or their offsets calibrated.

[0022] As shown in FIG. 2, digital sensor data from these sensor modules are provided for ingestion by a processor unit 36, which runs an algorithm internally to perform digital signal processing, feature extraction, decision logic, and preparation for communication with the user's module 40. Data is then transmitted from the sensor system's 30 communication module 38 to the corresponding user interface system's 40 communication module 42.

[0023] A user interface system 40 consists of a communication module 42 which receives data from its corresponding sensor system 30. This data is presented to the user via a user interface 44 such as an LED, a display, a speaker, or any other manner of interface.

[0024] An RGB/SW IR imaging sensor determines a presence of a living subject and vitals under certain circumstances. An example commercial product is a Wyze camera, or a Ring camera. A Doppler radar detects the vitals of a living subject by detecting reflected RF. An example commercial product is a Xethru X4.

[0025] A compute module 36 with memory 35 and a communication module 38 performs the entirety of presence and vitals detection within the sensing system 30.

[0026] A user interface system 40 with communication module 42 receives data and presents information to the user.

[0027] The radar vitals detection includes the following: monitor a sensor output with a sample rate S Hz; X=x[n] for CW doppler radar; X=x.sub.1[i], x.sub.2[i], . . . x.sub.n[i] for pulsed doppler radar with range bins i in Lmin≤i≤Lmax for minimum observed range Lmin and maximum observed range Lmax); S could be 25 Hz, for example; Band pass filter x around vitals frequencies f.sub.c; For X, compute discrete Fourier transform Z for a window of time k; k could be 25 seconds, for example; Perform peak-search algorithm on Z and estimate peak level; Example peak search includes argmax; Using peak, estimate noise floor level; Example noise floor estimation includes argmin; Compute SNR[n] by subtracting noise floor from peak; Compare SNR[n] to threshold value b; If SNR[n]>b; Presence verified; Return frequency of peak; Else-Presence disqualified.

[0028] The video presence detection includes the following: Collect image I[t] output with sample rate S Hz or when otherwise requested by system; S could be 0.2 Hz for example; Perform classification or object detection model inference using trained model; Example trained model includes AWS Sagemaker Image Classifier ResNet model; Use response from model to update status of baby presence.

[0029] A video vitals detection includes the following: Collect image I[t] output with sample rate S Hz or when otherwise requested by system; S could be 25 Hz for example; For each image, compute points of interest using sobel operator; Estimate motion vector, X=x[n], by tracking change in k-means centroids per frame; Band pass filter x around vitals frequencies f.sub.c; For X, compute discrete fourier transform Z for a window of time k; k could be 25 seconds, for example; Perform peak-search algorithm on Z and estimate peak level; Example peak search includes argmax; Using peak, estimate noise floor level; Example noise floor estimation includes argmin; Compute SNR[n] by subtracting noise floor from peak; Compare SNR[n] to threshold value b; If SNR[n]>b; Presence verified; Return frequency of peak; Else-Presence disqualified.

[0030] FIG. 1 is an illustration of an antenna pattern 100 showing the target of interest 10 (the living subject), the camera's field of view 15, and the radar antenna sensitivity pattern 20.

[0031] of one embodiment of a system for presence and vitals detection of a living subject

[0032] FIG. 2 is a block diagram of one embodiment of a system 200 for presence and vitals detection of a living subject. The system 200 comprises an RGB/IR imaging sensor 32, a radar 34, a processor 36, and a user interface 40. The RGB/IR imaging sensor 32 is utilized to detect light reflected by a living subject 10 from ambient or controlled light sources. The radar 34 emits a radiofrequency at a specific frequency, and detects the frequency change of reflections of a plurality of targets, which have subtle movements from the living subject 10. The processor 36 is configured to run an algorithm to perform digital signal processing on data provided by the radar 34 and the IR sensor 32 to generate presence and vitals information for the living subject 10 for communication to the user interface 40.

[0033] FIG. 3 is a block diagram of a process 300 for presence and vitals detection of a living subject. The process includes detecting at an RGB/IR imaging sensor 332 of a monitoring device 330, light reflected 12 by a living subject 10 from ambient or controlled light sources. The process includes filtering 333 the signal and using a feature extraction 335 and an object detection model 337 at a processor, then applying a presence detection logic 339. The process also includes emitting from a radar 334 of the monitoring device 330 a radiofrequency at a specific frequency, and detecting the frequency change of reflections 14 of a plurality of targets which have subtle movements from the living subject 10. The method also includes receiving at a processor of the monitoring device 330 the data from the radar 334, filtering the data 341, applying feature extraction 343 to the data and applying vitals detection logic 345 to the extracted data. The method also includes packaging 347 the data. The method also includes communicating from a first communication module 338 of the monitoring device 330 the presence and vitals data for the living subject 10 to a second communication module 342 of an interface device 340. The method also includes presenting 349 on a user interface module 340 of the interface device 340 the presence and vitals information for the living subject 10.

[0034] FIG. 4 is a flow chart for a method 400 for presence and vitals detection of a living subject. The living subject is an infant, a pet, or an elderly adult. At step 401, light reflected by a living subject from ambient or controlled light sources is detected at an IR imaging sensor of a monitoring device. The IR imaging sensor is preferably an RGB IR imaging sensor. Alternatively, the IR imaging sensor is an SW IR imaging sensor. The radar is a pulsed Doppler radar or a continuous wave Doppler radar.

[0035] In step 402, the radar emits a radiofrequency at a specific frequency, and then detects the frequency change of reflections of a plurality of targets which have subtle movements, caused by the respiration and/or ballistocardiography from the living subject.

[0036] At step 403, the processor of the monitoring device receives data from the radar and from the IR imaging sensor. Preferably, the monitoring device also comprises a memory configured to store sensor output from the IR imaging sensor. At step 404, the processor runs an algorithm to perform digital signal processing on the data, provided by the radar and by the IR imaging sensor, to generate presence and vitals information for the living subject.

[0037] At step 405, a first communication module of the monitoring device communicates the presence and vitals information for the living subject to a second communication module of an interface device. Preferably, both communication modules operate on a WiFi communication protocol, a BLUETOOTH communication protocol, a FM communication protocol, or a FHSS communication protocol.

[0038] Finally, at step 406, the presence and vitals information for the living subject is presented on a user interface module of the interface device, preferably using a LED, a display or a speaker.

[0039] FIG. 5 is a flow chart for a method 500 for presence and vitals detection of a living subject. In step 501, light reflected by a living subject from ambient or controlled light sources is detected at a RGB imaging sensor of a monitoring device. In step 502, the radar emits a radiofrequency at a specific frequency, and then detects the frequency change of reflections of a plurality of targets which have subtle movements from the living subject. At step 503, the processor of the monitoring device receives data from the radar and from the RGB imaging sensor. At step 504, the processor runs an algorithm to perform digital signal processing on the data, provided by the radar and by the RGB imaging sensor, to generate presence and vitals information for the living subject. At step 505, a first communication module of the monitoring device communicates the presence and vitals information for the living subject to a second communication module of an interface device. At step 506, the presence and vitals information for the living subject is presented on a user interface module of the interface device.

[0040] White et al., U.S. patent Ser. No. 10/825,314 for a Baby Monitor, is hereby incorporated by reference in its entirety.

[0041] From the foregoing it is believed that those skilled in the pertinent art will recognize the meritorious advancement of this invention and will readily understand that while the present invention has been described in association with a preferred embodiment thereof, and other embodiments illustrated in the accompanying drawings, numerous changes modification and substitutions of equivalents may be made therein without departing from the spirit and scope of this invention which is intended to be unlimited by the foregoing except as may appear in the following appended claim. Therefore, the embodiments of the invention in which an exclusive property or privilege is claimed are defined in the following appended claims.