RECONSTRUCTION OF ELEVATION INFORMATION FROM RADAR DATA

Abstract

A method for reconstructing elevation information from measured data that were recorded with the aid of at least one radar device and include a two-dimensional spatial distribution of at least one physical measured variable. The measured data are fed as input variables to at least one generator module that is designed as a neural network. At least one output variable is retrieved from the generator module that represents a measure of the elevation angles from which radar radiation was reflected to the radar device from at least one object. A method for training a generator module, and a method including a complete active chain up to activating a vehicle, are also described.

Claims

1. A method for reconstructing elevation information from measured data that were recorded using at least one radar device and include a two-dimensional spatial distribution of at least one physical measured variable, the method comprising the following steps: feeding the measured data as input variables to at least one generator module that is configured as a neural network; and retrieving, from the generator module, at least one output variable that represents a measure of elevation angles from which radar radiation was reflected to the radar device from at least one object.

2. The method as recited in claim 1, wherein the at least one output variable of the generator module indicate, with regard to locations in the two-dimensional spatial distribution, from which elevation angles radar radiation that contributed to a value of a physical measured variable at each location was reflected to the radar device.

3. The method as recited in claim 1, wherein the at least one output variable of the generator module include a three-dimensional spatial distribution of the at least one physical measured variable.

4. The method as recited in claim 3, wherein: the two-dimensional spatial distribution is divided into multiple segments that are contiguous in each case, for the input variables that belong to each of the segments, corresponding output variables are ascertained, and all ascertained output variables are combined to obtain the three-dimensional spatial distribution of the at least one physical measured variable.

5. The method as recited in claim 1, wherein the generator module includes: an encoder configured to translate the input variables into latent variables in a space, whose dimensionality is smaller than a dimensionality of a space of the input variables and smaller than a dimensionality of a space of the at least one output variable; and a decoder configured to translate the latent variables into the at least one output variables.

6. The method as recited in claim 5, wherein the at least one output variable of the generator module include multiple two-dimensional spatial distributions of the at least one physical measured variable, each distribution of the distributions corresponding to an area, from which that portion of radar radiation that generated the distribution was reflected to the radar device.

7. The method as recited in claim 1, wherein the measured data include measured data recorded using a radar device whose antenna array does not enable a direct elevation measurement.

8. The method as recited in 1, wherein the measured data include measured data recorded using a moved synthetic aperture radar device.

9. The method as recited in claim 1, wherein the at least one output variable of the generator module are used to evaluate: at least one category of a predefined classification of traffic signs, or other road users, or roadway markings, or signaling systems, or other traffic-relevant objects, in surroundings of a vehicle, and/or at least one position, and/or spatial measurement and/or speed of an object including a traffic sign, or other road user, or roadway marking, or signaling system, or another traffic-relevant object, in the surroundings of a vehicle.

10. A method for training a generator module, comprising the following steps: providing training measured data recorded using a radar device whose antenna array enables a direct elevation measurement; ascertaining a two-dimensional spatial distribution of at least one physical measured variable from the training measured data as input variables for the generator module; ascertaining setpoint output variables from the training measured data that represent a measure of elevation angles from which radar radiation was reflected back to the radar device from at least one object; processing the input variables to form the output variables using the generator module; and optimizing generator parameters that characterize a behavior of the generator module with an objective that the output variables reproduce sufficiently well the setpoint output variables in accordance with a predefined generator cost function.

11. The method as recited in claim 10, wherein: the output variables and the setpoint output variables are additionally fed to a discriminator module that is configured as a further neural network; and discriminator parameters that characterize a behavior of the discriminator module are optimized alternatingly with the generator parameters with an objective that the discriminator module well differentiates the output variables, generated by the generator module, from the setpoint output variables in accordance with a discriminator cost function.

12. The method as recited in claim 10, wherein the radar device includes multiple channels outside of a plane, in which its azimuth angle is defined.

13. The method as recited in claim 12, wherein a distance relative speed spectrum of the training measured data is ascertained for each channel and the input variables for the generator module and the setpoint output variables are ascertained from the distance relative speed spectra.

14. A method, comprising the following steps: training a generator module by: providing training measured data recorded using a radar device whose antenna array enables a direct elevation measurement, ascertaining a two-dimensional spatial distribution of at least one physical measured variable from the training measured data as input variables for the generator module, ascertaining setpoint output variables from the training measured data that represent a measure of elevation angles from which radar radiation was reflected back to the radar device from at least one object, processing the input variables to form the output variables using the generator module, and optimizing generator parameters that characterize a behavior of the generator module with an objective that the output variables reproduce sufficiently well the setpoint output variables in accordance with a predefined generator cost function; recording measured data from surroundings of the vehicle using at least one radar device that is mounted at or in a vehicle; reconstructing elevation information from the measured data by: feeding the measured data as input variables to the generator module, the generator module being configured as a neural network; and retrieving, from the generator module, at least one output variable that represents a measure of elevation angles from which radar radiation was reflected to the radar device from at least one object; forming an activation signal by using the elevation information; and activating the vehicle using the activation signal.

15. A non-transitory machine-readable data carrier on which is stored a parameter set including generator parameters, the generator parameters being obtained by training a generator module by: providing training measured data recorded using a radar device whose antenna array enables a direct elevation measurement; ascertaining a two-dimensional spatial distribution of at least one physical measured variable from the training measured data as input variables for the generator module; ascertaining setpoint output variables from the training measured data that represent a measure of elevation angles from which radar radiation was reflected back to the radar device from at least one object; processing the input variables to form the output variables using the generator module; and optimizing the generator parameters that characterize a behavior of the generator module with an objective that the output variables reproduce sufficiently well the setpoint output variables in accordance with a predefined generator cost function.

16. A machine-readable data carrier on which are stored machine-readable instructions for reconstructing elevation information from measured data that were recorded using at least one radar device and include a two-dimensional spatial distribution of at least one physical measured variable, the instructions, when executed by at least one computer, causing the at least one computer to perform the following steps: feeding the measured data as input variables to at least one generator module that is configured as a neural network; and retrieving, from the generator module, at least one output variable that represents a measure of elevation angles from which radar radiation was reflected to the radar device from at least one object.

17. A computer configured to reconstruct elevation information from measured data that were recorded using at least one radar device and include a two-dimensional spatial distribution of at least one physical measured variable, the computer configured to: feed the measured data as input variables to at least one generator module that is configured as a neural network; and retrieve, from the generator module, at least one output variable that represents a measure of elevation angles from which radar radiation was reflected to the radar device from at least one object.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0056] FIG. 1 shows one exemplary embodiment of method 100 for reconstructing elevation information 2a from measured data 2, in accordance with the present invention.

[0057] FIG. 2 shows an illustration of combining multiple two-dimensional distributions 6g through 6j of a reconstructed variable to a three-dimensional distribution 6b, in accordance with an example embodiment of the present invention.

[0058] FIG. 3 shows an illustration of the segmented processing of a two-dimensional distribution 3 of a measured variable, in accordance with an example embodiment of the present invention.

[0059] FIG. 4 shows one exemplary embodiment of method 200 for training a generator 4, in accordance with the present invention.

[0060] FIG. 5 shows one exemplary embodiment of method 300 including the complete active chain, in accordance with the present invention.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0061] FIG. 1 is a schematic flowchart of method 100 for reconstructing elevation information 2a from measured data 2 recorded with the aid of at least one radar device 1. In step 110, measured data 2, which include a spatial distribution 3 of at least one physical measured variable, are fed to at least one generator module 4 as input variables 5. In step 120, at least one output variable 6 is retrieved from generator module 4 that represents a measure of the elevation angles 6a from which radar radiation was reflected to radar device 1 from at least one object. In other words, at least one output variable 6 includes elevation information 2a. According to block 121, output variables 6 of generator module 4 indicate in particular with regard to locations 3a in spatial distribution 3, from which concrete elevation angles 6a radar radiation that contributed to the value of the physical measured variable at particular location 3a was reflected to radar device 1.

[0062] Elevation information 2a is further evaluated in step 130, to ascertain [0063] at least one category 7a of a predefined classification of traffic signs, other road users, roadway markings, signaling systems or other traffic-relevant objects in the surroundings of a vehicle and/or [0064] at least one position 7b, spatial position, dimension 7c and/or speed 7d of an object of this type.

[0065] According to step 105, measured data 2 recorded with the aid of a radar device 1, whose antenna array does not enable a direct elevation measurement, may be selected. According to step 106, measured data 2 may be selected that were recorded with the aid of a moved synthetic aperture radar device.

[0066] According to block 111, generator module 4 may include an encoder 41, which translates input variables 5 into low-dimensional latent variables 5*, as well as a decoder 42, which translates these latent variables 5* into output variables 6 including an again increased dimensionality.

[0067] According to block 112, two-dimensional spatial distribution 3 of the physical measured variable, which is used as input variable 5, may be divided into multiple segments 3c through 3f that are contiguous in each case. According to block 123, input variables 5 may then be processed in segmented portions 5c through 5f to particular output variables 6c through 6f that may then be combined according to block 124. In this way, a three-dimensional spatial distribution 6b of at least one sought physical measured variable is obtained.

[0068] In general, the output variables of generator module 4 may include a three-dimensional spatial distribution of at least one sought physical measured variable according to block 122. According to block 125, generator module 4 may output multiple two-dimensional spatial distributions 6g through 6j in this case. Each of these distributions 6g through 6j then corresponds to an area of elevation angles, from which that portion of the radar radiation that generated particular distribution 6g through 6j, was reflected to radar device 1.

[0069] It is illustrated in FIG. 2 how such distributions 6g through 6j may be combined to form a three-dimensional distribution 6b of the sought measured variable. Input variables 5 are initially fed to generator module 4 that is made up of an encoder 41 reducing the dimensionality and a decoder 42 again increasing the dimensionality. At the interface between encoder 41 and decoder 42, the data are present as latent variables 5*.

[0070] Decoder 42 outputs in the coordinates distance r and azimuth angle ν for different elevation angles φ two-dimensional distributions 6g through 6j of the sought physical measured variable. These two-dimensional distributions 6g through 6j may be organically combined to form a tensor that then indicates a three-dimensional distribution 6b of the sought physical measured variable.

[0071] FIG. 3 illustrates the segmented processing of input variable 5. Two-dimensional spatial distribution 3 of a physical measured variable embodied by measured data 2 and that is to be used as input variable 5 is divided into segments, of which four segments 3c through 3f are illustrated by way of example in FIG. 3. Input variables 5c through 5f that belong to each segment 3c through 3f are translated into output variables 6c through 6f according to block 123. Generator module 4 used for this purpose may have smaller dimensions than one that translates present input variables 5 into output variables 6 all at once.

[0072] According to block 124, output variables 6c through 6f are combined to form three-dimensional distribution 6b of the sought physical measured variable. As illustrated in FIG. 3, output variables 6c through 6f provide information on completely different locations in this three-dimensional distribution 6b. Not even all locations, regarding which an individual portion 6c through 6f of output variables 6 makes a statement, must be in a contiguous segment.

[0073] FIG. 4 is a schematic flowchart of one exemplary embodiment of method 200 for training generator module 4. In step 210, training measured data 2* are provided that were recorded with the aid of a radar device 1*. The antenna array of this radar device 1* enables a direct elevation measurement. In step 205, in particular, a radar device 1* may be selected that includes multiple channels outside of the plane, in which its azimuth angle is defined.

[0074] In step 220, a two-dimensional spatial distribution 3 of at least one physical measured variable is ascertained from training measured data 2* as input variables 5 for generator module 4. In step 230, from training measured data 2* setpoint output variables 6* are furthermore ascertained that represent a measure of the elevation angles 6a from which radar radiation was reflected to radar device 1* from at least one object.

[0075] Input variables 5 are processed to form output variables 6 with the aid of generator module 4. Generator parameters 4a that characterize the behavior of generator module 4 are optimized in step 250 with the objective that output variables 6 reproduce setpoint output variables 6* sufficiently well in accordance with a predefined generator cost function 4b. The result of this training is fully trained state 4a* of generator parameters 4a.

[0076] In the example shown in FIG. 4, the neural network of generator module 4 is expanded to include an additional discriminator module 8 to form a conditional GAN, cGAN. Output variables 6, setpoint output variables 6* as well as input variables 5 are additionally fed to discriminator module 8. Discriminator parameters 8a that characterize the behavior of discriminator module 8 are optimized in step 270 alternatingly with generator parameters 4a with the objective that discriminator module 8 well differentiates output variables 6 generated by generator module 4 from setpoint output variables 6* in accordance with a discriminator cost function 8b. The training provides fully trained state 8a* of discriminator parameters 8a and fully trained state 4a* of generator parameters 4a as a result.

[0077] After completing the training, however, only generator module 4, whose behavior is characterized by its fully trained parameters 4a*, is needed to evaluate real measured data 2.

[0078] According to block 211, a distance relative speed spectrum 211a of particular training measured data 2* may be ascertained for each channel. According to block 221, input variables 5 for generator module 4 as well as setpoint output variables 6* may then also be ascertained from the distance relative speed spectra according to block 231.

[0079] FIG. 5 is a schematic flowchart of one exemplary embodiment of method 300 including the complete active chain. In step 310, a generator module 4 is trained with the aid of method 200 shown in FIG. 4, resulting in that its parameters 4a assume their fully trained state 4a*. In step 320, measured data 2 from the surroundings of vehicle 50 are recorded with the aid of at least one radar device 1 that is mounted at or in a vehicle 50.

[0080] In step 330, elevation information 2a is reconstructed from measured data 2 with the aid of method 100 shown in FIG. 1 in the form of output variables 6 of used generator module 4, for example in the form of elevation angles 6a and/or a three-dimensional distribution 6b of a sought physical measured variable. The latter is used to form an activation signal 9 in step 340. In step 350, vehicle 50 is activated using this activation signal 9.