Method for robust vehicle occupancy detection with vital sign monitoring

Abstract

A radar sensor system and a method for detecting an occupancy in an interior of a vehicle and with vital sign monitoring. The radar sensor system includes a radar transmitting unit, a radar receiving unit and a signal processing and control unit. The method includes: transmitting a radar wave towards a scene within the vehicle interior, receiving at least one radar wave that has been generated by reflection of the transmitted radar wave, decomposing the received radar wave into range, Doppler and angular information, quantifying and tracking a movement in each region of interest by angular gating and range gating, detecting and monitoring vital signs of occupants in each region of interest, and determining whether quantified and tracked movements in the scene are related to an occupant or to external or internal disturbances, based on a fulfillment of at least one predefined condition concerning the and/or the detected vital signs.

Claims

1. A method of operating a radar sensor system for detecting an occupancy in an interior of a vehicle, with vital sign monitoring; the radar sensor system comprising: a radar transmitting unit being configured for transmitting radar waves towards a scene within the vehicle interior, a radar receiving unit being configured for receiving radar waves that have been transmitted by the radar transmitting unit and have been reflected by an object within the scene, and a signal processing and control unit that is at least configured to derive range information, Doppler information and angle-of-arrival information from the received radar waves; and the method including at least the following steps, which are to be carried out in an iterative manner: operating the radar transmitting unit for transmitting a radar wave towards a scene within the vehicle interior, operating the radar receiving unit for receiving at least one radar wave that has been generated by reflection of the transmitted radar wave, operating the signal processing and control unit for: decomposing the received at least one radar wave into range, Doppler and angular information, and detecting and monitoring vital signs of occupants in each region of interest of a plurality of predefined different regions of interest, wherein the step of operating the signal processing and control unit includes: quantifying and tracking a movement in each region of interest of a plurality of predefined different regions of interest by angular gating and range gating, and for each region of interest, determining whether quantified and tracked movements in the scene are related to an occupant or to external or internal disturbances, based on a fulfillment of at least one predefined condition concerning the quantified and tracked movements and/or the detected and monitored vital signs, wherein the at least one predefined condition includes that if movement is present in all or several range regions of interest, a disturbance is likely to be present, and if movement is detected only in a region with a seating position, and vital signs are found, it is determined that the seat is occupied.

2. The method as claimed in claim 1, wherein the step of determining whether quantified and tracked movements in the scene are related to an occupant or to external or internal disturbances is based on a temporal development of fulfillments of the at least one predefined condition concerning the quantified and tracked movements and the detected and monitored vital signs.

3. The method as claimed in claim 1, wherein the at least one predefined condition concerning the quantified and tracked movements and the detected and monitored vital signs includes consideration of a constraint regarding a number of passengers in the vehicle interior to be constant while the vehicle is driving.

4. The method as claimed in claim 1, wherein the step of decomposing the received radar waves into range, Doppler and angular information includes applying a fast Fourier transform to an intermediate frequency signal of the received radar waves for converting the intermediate frequency signal of the received radar waves from the time domain into the range domain.

5. The method as claimed in claim 4, wherein the step of decomposing the received radar waves further includes removing a static scene from the fast Fourier transform of the intermediate frequency signal of the received radar waves by making use of a fast Fourier transform of an intermediate frequency signal of at least one previously received radar waves.

6. The method as claimed in claim 1, wherein the step of quantifying and tracking a movement in each region of interest includes comparing an amount representing the quantified movement with at least one predefined detection threshold value for the quantity.

7. The method as claimed in claim 6, wherein the at least one predefined detection threshold value for the quantity is a function of at least one out of vehicle speed and vehicle acceleration.

8. The method as claimed in claim 1, wherein the step of detecting and monitoring vital signs of occupants comprises calculating a phase signal from the fast Fourier transform of IF signals of the received radar waves at determined movements in each region of interest, and processing each phase signal by reconstructing the frequency domain signal into a phase-coherent time domain signal.

9. The method as claimed in claim 1, wherein the step of quantifying and tracking a movement in each region of interest includes calculating power integrals in predefined regions for each region of interest.

10. The method as claimed in claim 1, wherein the plurality of predefined different regions of interest includes at least one out of a range-azimuth, range-elevation or azimuth-elevation region of interest.

11. The method as claimed in claim 1, comprising a step of operating the signal processing and control unit for generating an output signal that is indicative of the determined occupancy of the vehicle interior.

12. A radar sensor system for detecting an occupancy in an interior of a vehicle and for vital sign monitoring, the radar sensor system comprising: a radar transmitting unit being configured for transmitting radar waves towards a scene within the vehicle interior, a radar receiving unit being configured for receiving radar waves that have been transmitted by the radar transmitting unit and have been reflected by an object within the scene, and a signal processing and control unit that is at least configured to derive range information, Doppler information and angle-of-arrival information from the received radar waves, to control operation of the radar transmitting unit and the radar receiving unit and to automatically execute steps of the method of claim 1.

13. The radar sensor system as claimed in claim 12, wherein the radar transmitting unit includes a plurality of radar transmitting antennas and the radar receiving unit includes a plurality of radar receiving antennas, and the radar transmitting antennas and the radar receiving antennas are configured to operate in a multiple-input and multiple-output configuration.

14. The radar sensor system as claimed in claim 12, wherein the radar transmitting unit is configured to transmit frequency-modulated continuous radar waves.

15. A non-transitory digital data memory unit on which is stored a software module for controlling automatic execution of the method as claimed in claim 1, wherein method steps to be conducted are converted into a program code of the software module, wherein the program code is executable by a processor unit of the radar sensor system or a separate control unit.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Further details and advantages of the present invention will be apparent from the following detailed description of non-limiting embodiments with reference to the attached drawing, wherein:

(2) FIG. 1 schematically illustrates, in a side view, a configuration of an embodiment of a radar sensor system in accordance with the invention for detecting an occupancy in an interior of a vehicle and vital sign monitoring by executing an operating method in accordance with the invention,

(3) FIGS. 2a and 2b show a flowchart of an embodiment of the method in accordance with an embodiment of the invention of operating the radar sensor system pursuant to FIG. 1,

(4) FIG. 3 shows a plot of range information after a step of decomposing received radar waves into range, Doppler and angular information,

(5) FIG. 4 is a detailed flowchart illustrating a step of quantifying and tracking a movement in regions of interest,

(6) FIG. 5 is a detailed flowchart illustrating an alternative step of quantifying and tracking a movement in regions of interest,

(7) FIG. 6 schematically illustrates predefined different regions of interest covering a rear seat bench located in the vehicle passenger department pursuant to FIG. 1,

(8) FIG. 7 illustrates a step of determining seat occupancy based on a temporal development of fulfillments of a predefined condition including consideration of a constraint regarding the number of passengers, and

(9) FIG. 8 schematically illustrates fast and slow transitions between determined occupancies.

DESCRIPTION OF PREFERRED EMBODIMENTS

(10) FIG. 1 schematically illustrates, in a side view, a configuration of an embodiment of a radar sensor system 10 in accordance with the invention for detecting an occupancy in an interior 34 of a vehicle 32 with vital sign monitoring by executing an operating method in accordance with an embodiment of the invention. The interior 34 of the vehicle 32 is formed by a passenger car compartment of a sedan-type passenger car. FIG. 1 shows a side view of a passenger 28 occupying a driver's seat and another passenger 30 occupying one seat of three-seat rear bench 46 of the passenger car. More passengers (not shown) may be present, occupying a passenger front seat or other seats of the three-seat rear bench 46.

(11) The radar sensor system 10 comprises a radar transmitting unit 12 that in this specific embodiment is configured to transmit frequency-modulated continuous radar waves (FMCW), and in this specific embodiment includes a plurality of three radar transmitting antennas 14. In other embodiments, the radar transmitting unit may be configured to transmit radar waves modulated by frequency-shift keying (FSK), or the radar transmitting unit may be configured to transmit radar pulses. Also, in other embodiments, the number of transmitting antennas may be different. In this specific embodiment, the radar transmitting unit 12 is configured for supplying the three radar transmitting antennas 14 with radar signals having a radar carrier frequency that is frequency-modulated with a saw-tooth pattern, as is known in the art, but other frequency modulation patterns such as triangular, sinusoidal modulation or any other pattern that appears suitable to those skilled in the art are also contemplated. In this specific embodiment, the radar carrier frequency of the radar sensor system 10 is selectable within a radar frequency range between 77 GHz and 81 GHz, but other frequency ranges, for instance a range between 57 GHz and 64 GHz, may be chosen, depending on the application. The radar transmitting antennas 14 are installed in a front region of the headliner and are directed rearwards. The radar transmitting unit 12 is configured for transmitting radar waves 16 via the radar transmitting antennas 14 towards a scene within the vehicle interior 34 given by the vehicle passenger compartment, and particularly towards a chest and an abdominal region of the driver 28 and other potentially present passengers 30.

(12) The radar sensor system 10 further includes a radar receiving unit 18 having three radar receiving antennas 20 and being configured for receiving radar waves 22 that have been transmitted by the radar transmitting unit 12 and have been reflected by an object within the scene, among them the passengers 28, 30 that are present in the vehicle interior 34.

(13) In this specific embodiment, each of the three radar transmitting antennas 14 is paired with one of the three radar receiving antennas 20 to be co-located in a monostatic arrangement, which is indicated in FIG. 1 by use of a combined symbol. In other embodiments, a bi-static arrangement or any other arrangement of an equal or unequal number of radar transmitting antennas and radar receiving antennas may also be possible. In this specific embodiment, the radar transmitting unit 12 and the radar receiving unit 18 each form an integral part of a transceiver unit 24, sharing common electronic circuitry and a common housing. In other embodiments, the radar transmitting unit and the radar receiving unit may be designed as separate units.

(14) In this specific embodiment, the three transceiver antennas formed by the three radar transmitting antennas 14 and the three radar receiving antennas 20 are configured to operate in a multiple-input and multiple-output (MIMO) configuration.

(15) In the MIMO configuration, each transceiver antenna is understood to be able to transmit radar waves in an independent manner that represent mutually orthogonal codes. Each transceiver antenna is further understood to be able to receive radar waves that have been transmitted by itself and any of the other transceiver antenna units and that have been reflected by an object, without any cross talk disturbance. The MIMO radar configuration provide benefits regarding enlarged size of virtual aperture, improved spatial resolution and less sensitivity to interference signals, as is well known in the art.

(16) Furthermore, the radar sensor system 10 comprises a signal processing and control unit 26 that is connected to the radar transmitting unit 12 for controlling operation of the radar transmitting unit 12. The signal processing and control unit 26 is also connected to the radar receiving unit 18 for receiving radar signals generated by the radar receiving unit 18 from the received radar waves 22.

(17) The signal processing and control unit 26 is configured to derive range information, Doppler information and angle-of-arrival information from the IF signals of the received radar signals. The signal processing and control unit 26 comprises a processor unit and a digital data memory unit (not shown) to which the processor unit has data access.

(18) In the following, an embodiment of a method of operating the radar sensor system 10 for detecting an occupancy in the interior of the vehicle with vital sign monitoring will be described with reference to FIG. 1 and FIGS. 2a and 2b in general, which provides a flowchart of the method. In preparation of operating the radar sensor system 10, it shall be understood that all involved units and devices are in an operational state and configured as illustrated in FIG. 1.

(19) In order to be able to carry out the method automatically and in a controlled way, the signal processing and control unit 26 comprises a software module 36. The method steps to be conducted are converted into a program code of the software module 36. The program code is implemented in the digital data memory unit of the signal processing and control unit 26 and is executable by the processor unit of the signal processing and control unit 26.

(20) Execution of the method may be initiated for instance, but not limited to, by turning on the passenger car ignition. Other initiation methods are also possible. As shown in FIGS. 2a and 2b, the steps of the method are carried out in an iterative manner.

(21) In a first step 60 of the method, the radar transmitting unit 12 is operated by the signal processing and control unit 26 for transmitting a radar wave 16 towards the vehicle passenger compartment. In another step 62, the radar receiving unit 18 is operated by the signal processing and control unit 26 for receiving radar waves 22 that have been transmitted by the radar transmitting unit 12 and that have been generated by reflection at any object in the scene, and in particular at the passengers 28, 30 that are present in the vehicle passenger compartment, and, more specifically, by reflection at the chest and the abdominal region of the passengers 28, 30.

(22) In a next step 64, the signal processing and control unit 26 is operated to decompose an IF signal of the received radar signal into range information, Doppler information and angular information. To this end, the signal processing and control unit 26 applies a fast Fourier transform (FFT) to the IF signal of the received radar signal, which is a time domain signal, for extracting the range information. By that, the IF signal of the received radar signal is converted to the frequency domain. In another step 66, a static scene is removed by subtracting an FFT of an IF signal of a previously received radar signal, for example an IF signal of a radar signal that has been received in the previous iteration.

(23) FIG. 3 shows a plot of the range information after the step 64 of decomposing the IF signal of the received radar signal into range, Doppler and angular information and after the step 66 of removing the static scene. The range represented in FIG. 3 by range bins on the horizontal axis comprises a plurality of predefined regions of interest: a plurality of short-range regions of interest 38, a seat region of interest 40, which corresponds to a distance between the radar receiving unit 18 and the three-seat rear bench 46, and a plurality of long-range regions of interest 42. The range information of the FFT of the IF signal of the received radar signal shows a peak that is spread over a number of range bins in the seat region of interest 40.

(24) In a next step 68 of the method, movement in each predefined region of interest 38, 40, 42 is quantified and tracked by angular gating and range gating.

(25) To this end, a movement indicator ind.sub.seat is calculated that represents a quantity of the movement. In this specific embodiment, the movement indicator ind.sub.seat is given by a power integral to be calculated in predefined regions for each region of interest 38, 40, 42. Along the range information of the FFT of the IF signal of the received radar signal, maximum values are searched for by comparing the power integrals as amounts that represent the movement with a predefined detection threshold value for the power integral. If the amount exceeds the predefined detection threshold value, the range bin index of each one of the found maxima is placed into a buffer of predefined size. Then, the median of each buffer contents is calculated in every new iteration.

(26) In this specific embodiment, the predefined detection threshold value for the power integral is a number that is constant for each predefined region of interest 38, 40, 42. In alternative embodiments, a predefined detection threshold value for the power integral may be a function of at least one out of vehicle speed and vehicle acceleration.

(27) An example for detection of a disturbance is illustrated by FIG. 4, showing a detailed flowchart of the step 68 of quantifying and tracking a movement in three different regions of interest: a short-range region 38, the seat region 40 and a long-range region 42. For each region of interest 38, 40, 42 a detection threshold value thr.sub.seat, thr.sub.short, thr.sub.long for the power integral has been predefined. The various detection threshold values thr.sub.seat, thr.sub.short, thr.sub.long may be equal but do not have to be. In this specific embodiment, all three detection threshold values thr.sub.seat, thr.sub.short, thr.sub.long are mutually different.

(28) Whenever a movement indicator ind.sub.seat, ind.sub.s, ind.sub.L exceeds the detection threshold value thr.sub.seat, thr.sub.short, thr.sub.long of the respective region of interest, a corresponding detection flag 44 is set to value “1”, with the meaning that significant movement has been detected in the respective region of interest 38, 40, 42. If significant movement has been detected in all three regions of interest 38, 40, 42, it is quite certain to assume that a strong disturbance is present, which may be caused for instance by shaking the vehicle 32 by strong wind gusts, or the like.

(29) An alternative step of quantifying and tracking a movement in regions of interest is illustrated in the detailed flowchart of FIG. 5. Herein, the received radar signal FFT is divided in N short-range regions of interest, M long-range regions of interest and a seat region of interest.

(30) The movement indicator ind.sub.Si, i=1, . . . , N of each short-range region is compared against a corresponding detection threshold Thr, as shown in FIG. 5. Whenever the indicator of the i.sup.th short-range region exceeds the corresponding detection threshold, a detection flag p.sub.si is set to value “1”, otherwise the variable detection flag p.sub.si remains at value “0”. The values of p.sub.si, i=1, . . . , N are then used to compute a probability, which is compared to a probability threshold p.sub.thr to determine if a short-range disturbance is present.

(31) In a similar manner, movement indicators ind.sub.Lj, j=1, . . . , M of each long-range region are compared against the corresponding detection threshold Thr, and long-range detection flags p.sub.Lj, j=1, . . . , M are obtained. A probability is calculated from values of the long-range detection flags p.sub.Lj, which is compared to a probability threshold p.sub.thr to determine if a long-range disturbance is present.

(32) In a variation of this embodiment, variables p.sub.si, i=1, . . . , N and p.sub.Li, j=1, M can take weighting values instead of “1” and “0”.

(33) As described before, the three-seat rear bench 46 of the vehicle 32 lies in the range seat region 40 of interest pursuant to FIG. 3. A volume encompassing the three-seat rear bench 46 is further divided into three predefined regions of interest corresponding to a right seating position 48, a middle seating position 50 and a left seating position 52 (FIG. 6). The predefined regions of interest are defined in terms of range and azimuth (angular quantities), which can be converted to Cartesian coordinates x, y and z in the usual manner.

(34) A movement indicator is calculated for each seating position 48, 50, 52 and compared to a detection threshold that corresponds to the respective seating position 48, 50, 52. To this end, a power integral is calculated in each predefined region, and a median of each buffer contents, which is the range bin index of each one of the found maxima, is calculated in every new iteration, as described before. Every new iteration of the method steps contributes to a temporal development of the buffers.

(35) A buffer strategy regarding the temporal development of the buffer may be implemented similar to the one described in international application WO 2015/022358 A1.

(36) As another step 78 of the method, vital signs of occupants, formed by a breathing movement, are detected and monitored in each region of interest 38, 40, 42. This is accomplished in a step 80 by calculating a phase signal from the FFT of the IF signal of the received radar waves 22 at determined movements in each region of interest 38, 40, 42, i.e. at every median of the buffer contents, and processing each phase signal in a step 82 of reconstructing the frequency domain signal into a phase-coherent time domain signal.

(37) For each region of interest 38, 40, 42, it is determined in another step 84 whether quantified and tracked movements in the scene are related to an occupant or to external or internal disturbances, based on a fulfillment of predefined conditions concerning the quantified and tracked movements and/or detected and monitored vital signs.

(38) In this specific embodiment, a first predefined condition is that if movement is detected in a seat region of interest 40 only, and vital signs are detected, it is determined that the movements in the scene are related to a seat occupant occupying the specific seat region of interest 40. A second predefined condition is that if movement is detected in a short-range region of interest 38, a seat region of interest 40 and a long-range region of interest 42, it is determined that the movements in the scene are not related to a seat occupant but are related to the presence of a strong disturbance such as the vehicle 32 shaking by wind gusts or by rough road driving.

(39) In another step 86 of the method, the signal processing and control unit 26 is operated for generating an output signal that is indicative of the determined occupancy of the vehicle interior 34.

(40) In a variation of the method, the step 84 of determining whether quantified and tracked movements in the scene are related to an occupant or to external or internal disturbances is based on a temporal development of fulfillments of the predefined conditions concerning the quantified and tracked movements and the detected and monitored vital signs. In this variation of the method, the fulfillment or non-fulfillment of the predefined conditions has to be persistent for a predefined number of iterations, which relates to a predefined qualification time.

(41) The qualification time can either be decreased to improve reactivity at an expense of a higher susceptibility to false alarms, or can be increased to improve robustness at an expense of a reduced reactivity, i.e. a detection of an occupant, who is changing seat positions, might take a longer time.

(42) In order to overcome the long qualification time issue, the predefined conditions concerning the quantified and tracked movements and the detected and monitored vital signs includes consideration of a constraint regarding a number of passengers in the vehicle interior 34 to be constant while the vehicle 32 is operated with closed vehicle doors (FIG. 1).

(43) FIG. 7 illustrates a step of determining seat occupancy based on a temporal development of fulfillments of a predefined condition including consideration of the above-mentioned constraint.

(44) The occupancy states of the three-seat rear bench 46 can be represented by a 3-bit word, for instance (0,1,0), with the meaning: “Empty” (left seating position), “Occupied” (middle seating position), “Empty” (right seating position).

(45) An occupant changing from one seating position to another is detected after a buffer value 54 that is related to a new seating position has been consecutively increased over a predefined time period 58 that is shorter than the qualification time, while another history buffer value 56 that is related to an old seating position has been consecutively decreased over the same time period 58. If this is detected, the occupancy state of the new seating position is set to “Occupied”, while the occupancy state of the old seating position is immediately set to “Empty”.

(46) This occupancy state transition is carried out in any one of the cases of occupancy transitions with a constant number of occupants as shown in FIG. 8. These potential occupancy transitions, which can be called “fast transitions”, as they do not require any opening and closing of doors of the vehicle 32, are shown for the three-seat rear bench 46 of the vehicle 32 and one seat occupant on the left-hand side of FIG. 8. On the right-hand side of FIG. 8, potential “fast occupancy transitions” for the three-seat rear bench 46 of the vehicle 32 and two seat occupants are shown. Potential occupancy transitions with a non-constant number of occupants, called “slow transitions”, require opening and closing of doors of the vehicle 32, and are therefore determined after the regular qualification time.

(47) While an embodiment of the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.

(48) Other variations to be disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality, which is meant to express a quantity of at least two. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting scope.