Method for the recognition of an object

Abstract

In a method for the recognition of an object by means of a radar sensor system, a primary radar signal is transmitted into an observation space, a secondary radar signal reflected by the object is received, a Micro-Doppler spectrogram of the secondary radar signal is generated, and at least one periodicity quantity relating to an at least essentially periodic motion of a part of the object is determined based on the Micro-Doppler spectrogram. The determining of the at least one periodicity quantity includes the following steps: (i) determining the course of at least one periodic signal component corresponding to an at least essentially periodic pattern of the Micro-Doppler spectrogram, (ii) fitting a smoothed curve to the periodic signal component, (iii) determining the positions of a plurality of peaks and/or valleys of the smoothed curve, and (iv) determining the periodicity quantity based on the determined positions of peaks and/or valleys.

Claims

1. A method for a recognition of an object by a radar sensor system, the method comprising: transmitting a primary radar signal into an observation space; receiving a secondary radar signal reflected by the object; generating a Micro-Doppler spectrogram of the secondary radar signal; and determining at least one periodicity quantity relating to an at least essentially periodic motion of a part of the object is determined based on the Micro-Doppler spectrogram, wherein the determining of the at least one periodicity quantity includes the steps: (i) determining a course of at least one periodic signal component corresponding to an at least essentially micro-Doppler pattern of the Micro-Doppler spectrogram, (ii) fitting a smoothed curve to the periodic signal component, (iii) determining positions of a plurality of peaks and valleys of the smoothed curve, and (iv) determining the periodicity quantity based on the determined positions of peaks and valleys.

2. The method in accordance with claim 1, wherein in the step (ii), the smoothed curve is fitted to the periodic signal component by an adaptive curve-fitting process.

3. The method in accordance with claim 2, wherein at least one process parameter of the adaptive curve-fitting process is continuously adapted during the curve-fitting process.

4. The method in accordance with claim 3, wherein the process parameter is adapted based on a determined speed variation of the object.

5. The method in accordance with claim 1, wherein in step (i), the course of an upper envelope, a lower envelope and a difference between an upper envelope and a lower envelope of the Micro-Doppler spectrogram is determined as the at least one periodic signal component.

6. The method in accordance with claim 1, wherein in step (ii), the fitting of the smoothed curve to the periodic signal component is performed using a window function, in particular using a triangle kernel, a polynomial kernel or a Savitzky-Golay kernel.

7. The method in accordance with claim 6, wherein a window size of the window function is continuously adapted during the fitting of the smoothed curve to the periodic signal component.

8. The method in accordance with claim 1, wherein the Micro-Doppler spectrogram is generated by a time-frequency analysis, in particular by a Short-Time-Fourier-Transform or a Wigner-Ville-Distribution technique.

9. The method in accordance with claim 1, wherein the step (iii) includes a segmentation process, wherein a first peak of a sequence of peaks is defined as a starting position of a segment and a last peak of a sequence of peaks is defined as an ending position of a segment.

10. The method in accordance with claim 1, wherein in step (iv), the periodicity quantity is estimated by a recursive state estimator.

11. The method in accordance with claim 1, wherein the step (i) includes determining a spread measure of the Micro-Doppler spectrogram.

12. The method in accordance with claim 1, wherein the periodic signal component is determined by a percentile-based method or a curve-fitting method.

13. A system comprising: an electronic processing device configured to: receive, from a radar sensor, a secondary radar signal reflected by an object; generate a Micro-Doppler spectrogram based on the secondary radar signal; determine at least one periodicity quantity relating to an at least essentially periodic motion of a part of the object based on the Micro-Doppler spectrogram; determine the at least one periodicity quantity by determining a course of at least one periodic signal component corresponding to an at least essentially micro-Doppler pattern of the Micro-Doppler spectrogram; fit a smoothed curve to the periodic signal component by determining positions of a plurality of one of the peaks and valleys of the smoothed curve; and determine the periodicity quantity based on the determined positions of one of the peaks and valleys.

14. The system in accordance with claim 13, wherein the smoothed curve is fitted to the periodic signal component by an adaptive curve-fitting process.

15. The system in accordance with claim 14, wherein at least one process parameter of the adaptive curve-fitting process is continuously adapted during the curve-fitting process.

16. The system in accordance with claim 13, wherein the fitting of the smoothed curve to the periodic signal component is performed using a window function, in particular using a triangle kernel, a polynomial kernel or a Savitzky-Golay kernel.

17. The system in accordance with claim 13, wherein the course of an upper envelope, a lower envelope, and a difference between an upper envelope and a lower envelope of the Micro-Doppler spectrogram is determined as the at least one periodic signal component.

Description

BRIEF DESCRIPTION OF DRAWINGS

(1) Subsequently, the present invention is explained in more detail based on an exemplary embodiment with reference to the accompanying figures, wherein:

(2) FIG. 1 shows, in a top view, a motor vehicle, a radar sensor system mounted to the motor vehicle and a pedestrian to be detected by the radar sensor system;

(3) FIG. 2 shows a Micro-Doppler spectrogram generated by the radar sensor system according to FIG. 1;

(4) FIG. 3 is a flowchart showing different steps of a method according to an embodiment of the invention;

(5) FIG. 4 shows an enlarged section of a Micro-Doppler spectrogram; and

(6) FIG. 5 shows the course of a periodic signal component corresponding to an at least essentially periodic pattern of a Micro-Doppler spectrogram as well as a smoothed curve fitted to the course.

DETAILED DESCRIPTION

(7) Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

(8) ‘One or more’ includes a function being performed by one element, a function being performed by more than one element, e.g., in a distributed fashion, several functions being performed by one element, several functions being performed by several elements, or any combination of the above.

(9) It will also be understood that, although the terms first, second, etc. are, in some instances, used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact.

(10) The terminology used in the description of the various described embodiments herein is for describing embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses all possible combinations of one or more of the associated listed items. It will be further understood that the terms “includes,” “including,” “comprises,” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

(11) As used herein, the term “if” is, optionally, construed to mean “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context. Similarly, the phrase “if it is determined” or “if [a stated condition or event] is detected” is, optionally, construed to mean “upon determining” or “in response to determining” or “upon detecting [the stated condition or event]” or “in response to detecting [the stated condition or event],” depending on the context.

(12) In FIG. 1, there is shown a motor vehicle 11 and a radar sensor system 13 mounted to a front section of the motor vehicle 11. The radar sensor system 13 is preferably based on a millimeter wave radar sensor. A single channel radar sensor may be provided to minimize the production costs, while a multiple channel radar sensor may be provided to enhance the detection performance. The radar sensor system 13 can be connected to an electronic processing device (not shown), for example an advanced emergency braking system, a pedestrian collision avoidance system or an autonomous driving system. While a central mounting of the radar sensor system 13 is shown, a mounting to a corner section, a side section or a rear section of the motor vehicle 11 could equally be provided.

(13) In operation, the motor vehicle 11 is moving in a driving direction 15 on a lane 17. An object 19 in the form of a pedestrian crossing the lane 17 is present in the observation space 23 in front of the motor vehicle 11. The object 19 is moving in a moving direction 21. Other examples of objects to be observed by the radar sensor system 13 are cyclists and vehicles.

(14) The radar sensor system 13 is configured for transmitting a primary radar signal into the observation space 23 and for detecting objects 19 present in the observation space 23 on the basis of a secondary radar signal reflected by the objects 19. The line of sight 25 which extends from the object 19 to the active region of the radar sensor system 13 is called “line of sight”. The observed bulk speed v.sup.ob of the object 19, i. e. the speed component related to the main body 27 of the object 19 and oriented along the line of sight 25, can be determined in a known manner using the Doppler effect. Specifically, it is known that the relationship between the observed bulk speed v.sup.ob and the speed v of the object 19 in the moving direction 21 is given as:
v=v.sup.ob/cos(θ)
wherein θ is the illumination angle, i.e. the angle between the moving direction 21 and the line of sight 25.

(15) An exemplary Micro-Doppler spectrogram 30 of a moving cyclist is shown in FIG. 2. The horizontal axis is a time axis, whereas the vertical axis is a Doppler shift axis or a radial velocity axis. The generation of the Micro-Doppler spectrogram 30 can be carried out by means of a Short-Time-Fourier-Transform (STFT) or a Wigner-Ville-Distribution technique (WVD technique). In the left portion of FIG. 2, there is indicated a first Micro-Doppler pattern 31 that corresponds to a wheel rotation cycle. To the right of the first Micro-Doppler pattern 31, there are several further Micro-Doppler patterns 32 that correspond to pedal rotation cycles. A Micro-Doppler spectrogram of a pedestrian, a motor vehicle or a helicopter includes similar Micro-Doppler patterns that correspond to a gait cycle of the pedestrian, a wheel rotation of the motor vehicle or a rotor blade movement of the helicopter, respectively.

(16) Since the motions of individual components of an observed object 19 like arms, legs, wheels, pedals or rotor blades usually are of an essentially periodic nature, the Micro-Doppler patterns 31, 32 caused by these motions are at least essentially periodic. By determining a periodicity quantity relating to a Micro-Doppler pattern 31, 32, it is possible do discriminate between different types of objects 19, for example between pedestrians and cyclists or between adults and children. The periodicity quantity can be an average cycle duration, an average repetition frequency or a number of time bins related to one period or cycle of the corresponding pattern. In case of a pedestrian, the periodicity quantity is characteristic for the pedestrian's gait cycle.

(17) According to the invention, a periodicity quantity relating to an at least essentially periodic motion of a part of the object 19 is determined on the basis of a Micro-Doppler spectrogram analysis by means of the electronic processing device, as is explained in greater detail below.

(18) As shown in FIG. 3, a Micro-Doppler spectrogram 30 is generated as an input for the subsequent steps. In a step 41, the course of a periodic signal component corresponding to a Micro-Doppler pattern 31, 32 is determined. In a step 42, a spread measure of the signal component is estimated. In a step 43, a smoothed curve is fitted to the periodic signal component by means of an adaptive curve-fitting process. In a step 44, the positions of the peaks and/or valleys of the smoothed curve are determined. In a step 45, a periodicity quantity is determined based on the determined positions of the peaks and valleys.

(19) FIG. 4 shows an exemplary Micro-Doppler spectrogram 30′ of a pedestrian in an enlarged view. Three curves corresponding to the courses of different motion components are shown in the Micro-Doppler spectrogram 30′. Specifically, the periodic signal component 50 of the observed bulk speed v.sup.ob is shown as a solid black line, whereas the upper envelope 51 of the Micro-Doppler spectrogram 30′ and the lower envelope 52 of the Micro-Doppler spectrogram 30′ are shown as dashed lines.

(20) To determine the periodic signal component 50 of the observed bulk speed v.sup.ob, the upper envelope 51 and the lower envelope 52, the cumulative amplitude distribution function is determined for each time slice. The periodic signal component 50 of the observed bulk speed v.sup.ob is assigned to a percentile of about 50% of the cumulative amplitude distribution function. The upper envelope 51 is assigned to a percentile of about 95% of the cumulative amplitude distribution function, whereas the lower envelope 52 is assigned to a percentile of about 5% of the cumulative amplitude distribution function.

(21) The spread of the Micro-Doppler spectrogram 30′ is estimated by determining the envelope difference E.sub.diff; i.e. the absolute value of the difference between the upper envelope 51 and the lower envelope 52:
E.sub.diff=|E.sub.upper−E.sub.lower|

(22) An exemplary periodic signal component 54 of E.sub.diff over time is shown in FIG. 5. A smoothed curve 55 fitted to this periodic signal component 54 in step 43 (FIG. 3) is equally shown. The window size used in the adaptive curve-fitting process is adapted to the variation of the observed bulk speed v.sup.ob. According to an embodiment of the invention, the window size N.sub.n.sup.win at the time t.sub.n is given as:

(23) $\begin{array}{cc}{N}_n^{\mathrm{win}}=\mathrm{round}\left(N_{n-1}^{\mathrm{win}}\left[\frac{V_{n-1}^{\mathrm{ob}}}{V_n^{\mathrm{ob}}}\right]\right)& \left(1\right)\end{array}$
wherein the subscripts n, n−1 indicate the current state and the previous state, respectively, and the function round(⋅) returns the nearest integer value of the input.

(24) The initial value of the window size can be set as:

(25) $\begin{array}{cc}{N}_0^{\mathrm{win}}=\mathrm{round}\left(\frac{L_0^{\mathrm{cycle}}}{\Delta t*V_0}\right)& \left(2\right)\end{array}$
wherein Δt is the time-bin resolution of the Micro-Doppler spectrogram 30, 30′, L.sub.0.sup.cycle is the average length of the related periodic motion cycle and V.sub.0 is the average speed of the object 19. In the context of the recognition of pedestrians, a preferred value for L.sub.0.sup.cycle is 1.4 m, since the typical gait-cycle of a pedestrian is around 1.2 m-1.6 m, whereas a preferred value for V.sub.0 is 1.5 m/s.

(26) The adaptive curve fitting process in step 43 (FIG. 3) is carried out by means of a kernel function. Preferably, a kernel function is selected that smooths the raw signal so as to remove high frequency components while simultaneously maintaining the peak magnitude levels. Preferred kernel functions are high-order least-square polynomial kernels and high-order Savitzky-Golay kernels. According to a specific embodiment, a 2.sup.nd order or a 3.sup.rd order kernel is selected.

(27) In step 44 (FIG. 3), a peak finding process is performed to find the peaks 57 of the smoothed curve 55. In particular, peaks 57 are determined that satisfy the following conditions: It is a local peak, and the minimum distance between each two consecutive peaks is larger than k.Math.N.sub.n.sup.win, where k is a scale factor. Typical values for k range from 0.5 to 1, preferably from 0.7 to 0.9.

(28) The last mentioned condition increases the robustness of the peak finding process. Instead of the peaks 57 or in additions to the peaks 57, the valleys 58 of the smoothed curve 55 can be found in an analogous manner.

(29) Supposing that N.sub.n.sup.cycle is the number of samples contained in one cycle of a Micro-Doppler pattern 31, 32 near time slice t.sub.n, the cycle duration or period T.sub.n.sup.cycle can be expressed as:
T.sub.n.sup.cycle=N.sub.n.sup.cycle*Δt  (3)

(30) The repetition frequency f.sub.n.sup.cycle of a Micro Doppler pattern 31, 32 is the reciprocal of the period T.sub.n.sup.cycle:

(31) $\begin{array}{cc}{f}_n^{\mathrm{cycle}}=\frac{1}{T_n^{\mathrm{cycle}}}& \left(4\right)\end{array}$

(32) Once N.sub.n.sup.cycle is determined, the period T.sub.n.sup.cycle and the repetition frequency f.sub.n.sup.cycle of the corresponding Micro-Doppler pattern 31, 32 can be determined.

(33) Supposing that D.sub.n is the number of samples that are present between two consecutive peaks 57 near time slice t.sub.n, the relationship between D.sub.n and N.sub.n.sup.cycle is N.sub.n.sup.cycle=2.Math.D.sub.n for pedestrians and N.sub.n.sup.cycle=D.sub.n for objects having rotating components such as bikes, motor vehicles or helicopters.

(34) For pedestrians, the duration between two consecutive peaks 57 or valleys 58 is the duration of one footstep cycle. A gait-cycle includes two footsteps.

(35) To further improve the robustness of the method according to the invention, a recursive state estimator, in particular a Kalman filter, can be used. In an algorithm based on a Kalman Filter, the state space model and the measurement model can be given as:

(36) $\begin{array}{c}\left[\begin{array}{c}{V}_n\\ {\stackrel{.}{V}}_n\\ L_n^{\mathrm{cycle}}\end{array}\right]=\left[\begin{array}{ccc}1& \Delta t& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{V}_{n-1}\\ V_{n-1}^{\prime }\\ L_{n-1}^{\mathrm{cycle}}\end{array}\right]+w_{n-1}\\ T_n^{\mathrm{cycle}}=\frac{L_n^{\mathrm{cycle}}}{V_n}+q_n\end{array}$
wherein the random variables w.sub.n-1 and q.sub.n represent the process noise and the measurement noise. The subscripts n, n−1 indicate the current state and the previous state, respectively. The dot refers to the derivative of V.

(37) The peaks 57 found in the peak finding process can be used for a segmentation process, wherein the starting point and the ending point of the corresponding Micro-Doppler pattern 31, 32 are determined. For example, the first peak 57 of a sequence of peaks 57 can be defined as a starting position of a segment, whereas the last peak 57 of a sequence of peaks 57 is defined as an ending position of a segment.

(38) The invention enables a reliable recognition of moving objects 19 by means of a radar sensor system 13 even in case the available observation time is rather short. While this invention has been described in terms of the preferred embodiments thereof, it is not intended to be so limited, but rather only to the extent set forth in the claims that follow.