Method and Device for Training a Machine Learning Algorithm

Abstract

A method is provided for training a machine-learning algorithm which relies on primary data captured by at least one primary sensor. Labels are identified based on auxiliary data provided by at least one auxiliary sensor. A care attribute or a no-care attribute is assigned to each label by determining a perception capability of the primary sensor for the label based on the primary data and based on the auxiliary data. Model predictions for the labels are generated via the machine-learning algorithm. A loss function is defined for the model predictions. Negative contributions to the loss function are permitted for all labels. Positive contributions to the loss function are permitted for labels having a care attribute, while positive contributions to the loss function for labels having a no-care attribute are permitted only if a confidence of the model prediction for the respective label is greater than a threshold.

Claims

1. A method for training a machine-learning algorithm configured to process primary data captured by at least one primary sensor in order to determine at least one property of entities in an environment of the at least one primary sensor, the method comprising: receiving auxiliary data from at least one auxiliary sensor; identifying labels based on the auxiliary data, the identifying labels comprising determining a respective spatial area to which each label is related; assigning at least one of a care attribute or a no-care attribute to each identified label by determining a perception capability of the at least one primary sensor for the respective label based on the primary data captured by the at least one primary sensor and based on the auxiliary data captured by the at least one auxiliary sensor, the primary data usable to determine a reference value for a respective spatial area and, for each label, the care attribute is assigned to the respective label if the reference value is greater than a reference threshold and the no-care attribute is assigned to the respective label if the reference value is smaller than or equal to the reference threshold; generating model predictions for the labels via a machine-learning algorithm; defining a loss function for the model predictions, wherein the loss function receives a positive loss contribution for which weights of a model on which the machine-learning algorithm relies are increased if the weights contribute constructively to a prediction corresponding to the respective label and a negative loss contribution for which weights of the model are decreased if the weights contribute constructively to a prediction not corresponding to the respective label; permitting negative contributions to the loss function for all labels; permitting positive contributions to the loss function for labels having a care attribute; and permitting positive contributions to the loss function for labels having a no-care attribute only if a confidence value of the model prediction for the respective label is greater than a predetermined threshold.

2. The method according to claim 1, wherein the predetermined threshold for the confidence value is zero.

3. The method according to claim 2, wherein: the at least one primary sensor includes at least one radar sensor; and the reference value is determined based on radar energy detected by the radar sensor within the spatial area to which the respective label is related.

4. The method according to claim 3, wherein: ranges and angles at which radar energy is perceived are determined based on the primary data captured by the radar sensor; and the ranges and angles are assigned to the spatial areas to which the respective labels are related in order to determine the at least one of the care attribute or the no-care attribute for each label.

5. The method according to claim 4, wherein: an expected range, an expected range rate and an expected angle are estimated for each label based on the auxiliary data; and the expected range, the expected range rate and the expected angle of the respective label are assigned to a range, a range rate and an angle derived from the primary data of the radar sensor in order to determine the radar energy associated with the respective label.

6. The method according to claim 5, wherein the expected range rate is estimated for each label based on a speed vector which is estimated for a respective label by using differences of label positions determined based on the auxiliary data at different points in time.

7. The method according to claim 2, wherein: a subset of auxiliary data points is selected which are located within the spatial area related to the respective label; for each auxiliary data point of the subset, it is determined whether a direct line of sight exists between the at least one primary sensor and the auxiliary data point; and for each label, a care attribute is assigned to the respective label if a ratio of a number of auxiliary data points for which the direct line of sight exists to a total number of auxiliary data points of the subset is greater than a further predetermined threshold.

8. The method according to claim 7, wherein: the at least one primary sensor includes a plurality of radar sensors; and the auxiliary data point is regarded as having a direct line of sight to the at least one primary sensor if the auxiliary data point is located within an instrumental field of view of at least one of the radar sensors and has a direct line of sight to at least one of the radar sensors.

9. The method according to claim 8, wherein: for each of the radar sensors, a specific subset of the auxiliary data points is selected for which the auxiliary data points are related to a respective spatial area within an instrumental field of view of the respective radar sensor; the auxiliary data points of the specific subset are projected to a cylinder or sphere surrounding the respective radar sensor; a surface of the cylinder or sphere is divided into pixel areas; for each pixel area, the auxiliary data point having a projection within the respective pixel area and having the closest distance to the respective radar sensor is marked as visible; for each label, a number of visible auxiliary data points is determined which are located within the spatial area related to the respective label and which are marked as visible for at least one of the radar sensors; and the care attribute is assigned to the respective label if the number of visible auxiliary data points is greater than a visibility threshold.

10. The method according to claim 1, wherein: identifying labels based on the auxiliary data includes determining a respective spatial area to which each label is related; a reference value for the respective spatial area is determined based on the primary data; a subset of auxiliary data points is selected which are located within the spatial area related to the respective label; for each auxiliary data point of the subset, it is determined whether a direct line of sight exists between the at least one primary sensor and the auxiliary data point; and for each label, a care attribute is assigned to the respective label if the reference value is greater than a reference threshold and if a ratio of a number of auxiliary data points for which the direct line of sight exists to a total number of auxiliary data points of the subset is greater than a further predetermined threshold.

11. A system for training a machine-learning algorithm, the system comprising: at least one primary sensor configured to capture primary data; at least one auxiliary sensor configured to capture auxiliary data; and a processing unit configured to be used by the machine-learning algorithm to process the primary data in order to determine at least one property of entities in an environment of the at least one primary sensor, the processing unit further configured to: receive labels identified based on the auxiliary data and a respective spatial area to which each label is related; assign at least one of a care attribute or a no-care attribute to each identified label by determining a perception capability of the at least one primary sensor for the respective label based on the primary data captured by the at least one primary sensor and based on the auxiliary data captured by the at least one auxiliary sensor, the primary data usable to determine a reference value for a respective spatial area and, for each label, the care attribute is assigned to the respective label if the reference value is greater than a reference threshold, and the no-care attribute is assigned to the respective label if the reference value is smaller than or equal to the reference threshold; generate model predictions for the labels via the machine learning algorithm; define a loss function for the model predictions, the loss function receives a positive loss contribution for which weights of a model on which the machine learning algorithm relies are increased if the weights contribute constructively to a prediction corresponding to the respective label, and a negative loss contribution for which weights of the model are decreased if the weights contribute constructively to a prediction not corresponding to the respective label, permit negative contributions to the loss function for all labels; permit positive contributions to the loss function for labels having a care attribute; and permit positive contributions to the loss function for labels having a no-care attribute only if a confidence value of the model prediction for the respective label is greater than a predetermined threshold.

12. The system according to claim 11, wherein: the at least one primary sensor includes at least one radar sensor, and the at least one auxiliary sensor includes at least one of a light ranging and detection (LIDAR) sensor or at least one a camera.

13. The system according to claim 11, wherein the predetermined threshold for the confidence value is zero.

14. The system according to claim 13, wherein: the at least one primary sensor includes at least one radar sensor; and the reference value is determined based on radar energy detected by the radar sensor within the spatial area to which the respective label is related.

15. The system according to claim 14, wherein: ranges and angles at which radar energy is perceived are determined based on the primary data captured by the radar sensor; and the ranges and angles are assigned to the spatial areas to which the respective labels are related in order to determine the at least one of the care attribute or the no-care attribute for each label.

16. The system according to claim 15, wherein: an expected range, an expected range rate and an expected angle are estimated for each label based on the auxiliary data; and the expected range, the expected range rate and the expected angle of the respective label are assigned to a range, a range rate and an angle derived from the primary data of the radar sensor in order to determine the radar energy associated with the respective label.

17. The system according to claim 16, wherein the expected range rate is estimated for each label based on a speed vector which is estimated for a respective label by using differences of label positions determined based on the auxiliary data at different points in time.

18. The system according to claim 17, wherein: the at least one primary sensor includes a plurality of radar sensors; and the auxiliary data point is regarded as having a direct line of sight to the at least one primary sensor if the auxiliary data point is located within an instrumental field of view of at least one of the radar sensors and has a direct line of sight to at least one of the radar sensors.

19. The system according to claim 18, wherein: for each of the radar sensors, a specific subset of the auxiliary data points is selected for which the auxiliary data points are related to a respective spatial area within an instrumental field of view of the respective radar sensor; the auxiliary data points of the specific subset are projected to a cylinder or sphere surrounding the respective radar sensor; a surface of the cylinder or sphere is divided into pixel areas; for each pixel area, the auxiliary data point having a projection within the respective pixel area and having the closest distance to the respective radar sensor is marked as visible; for each label, a number of visible auxiliary data points is determined which are located within the spatial area related to the respective label and which are marked as visible for at least one of the radar sensors; and the care attribute is assigned to the respective label if the number of visible auxiliary data points is greater than a visibility threshold.

20. A non-transitory computer-readable storage medium storing one or more programs comprising instructions, which when executed by a processor, cause the processor to perform operations including: receiving auxiliary data from at least one auxiliary sensor; identifying labels based on the auxiliary data, the identifying labels comprising determining a respective spatial area to which each label is related; assigning at least one of a care attribute or a no-care attribute to each identified label by determining a perception capability of the at least one primary sensor for the respective label based on the primary data captured by at least one primary sensor and based on the auxiliary data captured by the at least one auxiliary sensor, the primary data usable to determine a reference value for a respective spatial area and, for each label, the care attribute is assigned to the respective label if the reference value is greater than a reference threshold and the no-care attribute is assigned to the respective label if the reference value is smaller than or equal to the reference threshold; generating model predictions for the labels via a machine-learning algorithm; defining a loss function for the model predictions, wherein the loss function receives a positive loss contribution for which weights of a model on which the machine-learning algorithm relies are increased if the weights contribute constructively to a prediction corresponding to the respective label, and a negative loss contribution for which weights of the model are decreased if the weights contribute constructively to a prediction not corresponding to the respective label; permitting negative contributions to the loss function for all labels; permitting positive contributions to the loss function for labels having a care attribute; and permitting positive contributions to the loss function for labels having a no-care attribute only if a confidence value of the model prediction for the respective label is greater than a predetermined threshold.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0043] Example implementations and functions of the present disclosure are described herein in conjunction with the following drawings, showing schematically:

[0044] FIG. 1 depicts a so-called activation tagging for labels representing vehicles surrounding a host vehicle;

[0045] FIG. 2 depicts a scheme for determining whether an object or label is occluded for a radar sensor;

[0046] FIG. 3 depicts an example for a so-called geometric tagging for labels which represent vehicles surrounding the host vehicle; and

[0047] FIG. 4 depicts the improvement in terms of precision and recall which is achieved by the method and the device according to the disclosure.

DETAILED DESCRIPTION

[0048] FIGS. 1 and 2 depict a host vehicle 11 which includes radar sensors 13 (see FIG. 2) and a LIDAR system 15 which are in communication with a processing unit 17. As shown in FIG. 1, other vehicles are located in the environment of the host vehicle 11. The other vehicles are represented by bounding boxes 19 which are also referred to as labels 19 since these bounding boxes are provided based on data from the LIDAR system 15 for training a machine-learning algorithm. The training of the machine-learning algorithm is performed via the processing unit 17 (which also executes the algorithm itself) and uses primary data provided by the radar sensors 13.

[0049] The primary data or input for the training is received from the radar sensors 13 and is represented as normalized radar energy 21 which is depicted in the form of shadows (as indicated by the arrows) in FIG. 1. The normalized radar energy 21 refers to a vehicle coordinate system 16 having an x-axis 18 along the longitudinal axis of the host vehicle 11 and a y-axis 20 along the lateral direction with respect to the host vehicle 11. In detail, the maximum radar energy is shown in FIG. 1 only for all doppler or range rate values derived from raw radar data. For the method and the device according to the disclosure, however, the full range rate or doppler information is used.

[0050] FIG. 1 depicts a scene in which the host vehicle 11 is driving in a lane on a highway and in which the vehicle represented by the labels or bounding boxes 19 on the right side are moving on a further lane which leads to the highway as a downward pointing ramp. That is, the lane in which the four vehicles on the right side are moving joins the highway in the upper part of FIG. 1.

[0051] The LIDAR system 15 (see FIG. 2) is mounted on the roof of the host vehicle 11, whereas the radar sensors 13 are mounted at the height of a bumper of the host vehicle 11. Therefore, the LIDAR system 15 has direct lines of sight to all labels 19 representing other vehicles, whereas some of the labels 19 are blocked for the lower mounted radar sensors 13. Therefore, if the labels 19 provided by the LIDAR system 15 were directly used for training the machine-learning algorithm which relies on the primary data captured by the radar sensors 13, the labels 19 would force the machine-learning algorithm to predict objects for which no reliable primary data from the radar sensors 13 are available.

[0052] The labels 19 which are derived from the data provided by the LIDAR system 15 are used as ground truth for a cross-domain training of the machine-learning algorithm since reliable labels cannot be derived from the radar data directly, i.e., neither by humans nor by another automated algorithm, as can be recognized by the representation of the normalized radar energy 21 in FIG. 1. The machine-learning algorithm is implemented as a radar neural network which requires a cross-domain training via the labels 19 provided by the LIDAR system 15. The LIDAR system 15 is therefore regarded as auxiliary sensor which provides auxiliary data from which the labels 19 are derived.

[0053] In order to avoid the above problem, i.e., forcing the radar neural network to predict objects which are not recognizable for the radar sensors 13, the labels 19 are additionally provided with an attribute 22 which indicates how the respective label 19 is to be considered for the training of the machine-learning algorithm. In detail, each label 19 is provided with a care attribute or a no-care attribute, wherein the care attribute indicates that the respective label is to be fully considered for the training of the machine-learning algorithm or radar neural network, whereas specific labels 19 provided with the no-care attribute are partly considered only for the training of the radar neural network. This will be explained in detail below. Since the labels 19 are adapted by the attribute 22 in order to provide a ground truth for cross-domain training of a radar neural network, the entire procedure is referred to as ground truth adaptation for a cross-domain training of radar neural networks (GARNN).

[0054] For assigning the attribute 22, i.e., a care attribute or a no-care attribute, to the ground truth labels 19 derived from auxiliary data which are captured by the LIDAR system 15, two procedures are concurrently performed which are referred to as activation tagging and geometric tagging. For the activation tagging, it is decided for each label 19 whether the respective label 19 can be perceived in the input or primary data captured by the radar sensors 13 or whether the label 19 cannot be perceived in the primary data and would therefore force the machine-learning algorithm to predict a label or object where no signal exists, which would lead to an increase of false detection rates.

[0055] The raw data received by the radar sensors 13 are processed in order to generate a so-called compressed data cube (CDC) as a reference for assigning the suitable attribute to the labels 19. For each radar scan or time step, the compressed data cube includes a range dimension, a range rate or doppler dimension and an antenna response dimension.

[0056] As a first step of activation tagging, angles are estimated at which the radar sensors 13 are able to perceive energy. The angles are estimated by using a classical angle finding procedure, e.g., a fast Fourier transform (FFT) or an iterative adaptive approach (IAA). As a result, a three-dimensional compressed data cube is generated including range, range rate and angle dimensions. Thereafter, the perceived radar energy is normalized, e.g., using a corner reflector response or a noise floor estimation.

[0057] As a next step, a speed vector is assigned to each label 19 (see FIG. 1). That is, the movement of the bounding boxes representing the labels 19 is monitored over time via the auxiliary data from the LIDAR system 15 (see FIG. 2). At two different points in time, the labels 19 (see FIG. 1) will have different spatial positions if the velocity of the objects they enclose is greater than zero. Via the position differences, the absolute value and the direction of the speed vector can be estimated. The speed vector is projected to the radial direction of the radar sensors 13 (see FIG. 2) in order to estimate a correspondence to the range rate or doppler dimension of the compressed data cube which is based on the data from the radar sensors 13.

[0058] In addition to the range rate which is estimated based on the respective speed vector of the label 19, an expected distance and an expected angle with respect to the position of the radar sensors 13 are determined for each label 19. Based on the expected distance, range rate and angle for the respective label 19 which are derived from LIDAR data 23 (see FIG. 2), a reverse lookup is performed in the compressed data cube in order to extract the radar energy which is related to the respective label.

[0059] In FIG. 1, the normalized radar energy 21 is represented by the shadows which are indicated by respective arrows, and the normalized energy which is related to the respective label 19 is represented by the respective part of the shadows (representing the radar energy 21) which falls into the respective bounding box representing the label 19. As is shown in FIG. 2, each bounding box or label 19 typically includes more than one LIDAR data point 23 which is detected by the LIDAR system 15. Therefore, the steps of determining the speed vector and estimating the expected distance, the range rate and the angle are repeated for each LIDAR data point 23 belonging to the respective label 19 or bounding box in order to extract a respective normalized radar energy for each LIDAR data point 23. For the respective label 19, the normalized radar energy is determined as the mean value over the energy values determined for the LIDAR data points 23 which are related to the respective bounding box or label 19. Alternatively, the maximum value or the sum over all normalized radar energy values of the LIDAR data point 23 belonging to the respective label 19 may be estimated.

[0060] If the normalized radar energy is greater than or equal to a predefined threshold for the respective label 19, this label can be perceived by the radar sensors 13. Therefore, the care attribute is assigned to this label 19. Conversely, if the normalized radar energy is smaller than the predefined threshold for a certain label 19, this label is regarded is not perceivable for the radar sensors 13. Hence, the no-care attribute is assigned to this label 19. As shown in FIG. 1, the attribute 22 is the care attribute for those labels 19 for which sufficient normalized radar energy 21 has been determined, whereas the attribute 22 is the no-care attribute for the other labels 19 for which the normalized radar energy 21 is too low.

[0061] The reliability of the activation tagging described above, i.e., associating the normalized radar energy with the respective labels 19, can be limited by a high angular uncertainty of the radar detection. The high angular uncertainty can be recognized in FIG. 1 in which the shadows representing the normalized radar energy 21 extend over quite a far range in the azimuth angle. While this drawback may be reduced for moving labels 19 by considering the range rate or doppler dimension of the compressed data cube, the angular uncertainty can be a problem for activation tagging of stationary or almost stationary labels 19 for which the range rate or doppler dimension is close to zero. Due to the angular uncertainty, labels 19 which are actually occluded for the radar sensors 13 may erroneously be assigned the care attribute although the respective label 19 may be hidden e.g., behind another object or label 19.

[0062] Therefore, a second procedure which is called geometric tagging is additionally considered which determines whether a direct line of sight 25 (see FIG. 2) exists between the radar sensors 13 and the respective label 19, i.e., between at least one of the radar sensors 13 and the LIDAR detection points 23 which belong to the respective label 19. Since a LIDAR point cloud is dense (see e.g., FIG. 3), i.e., much denser than a radar point cloud, the geometric tagging can reliably determine whether an object or label 19 is occluded for the radar sensors 13 or not.

[0063] For the geometric tagging, the LIDAR data points 23 are selected first which belong to the respective bounding box or label 19. The selected LIDAR data points 23 are transformed into a coordinate system of the radar sensors 13, i.e., into the “perspective” of the radar sensors 13. While for the activation tagging a “map” for the normalized radar energy has been considered (see FIG. 1) for all radar sensors 13, for the geometric tagging each antenna of the radar sensors 13 is considered separately. That is, the radar sensors 13 include an array of radar antennas having slightly different locations at the host vehicle 11. In FIG. 2, the situation for a single antenna is shown with respect to the visibility of the LIDAR data points 23.

[0064] Each antenna of the radar sensors 13 has a certain aperture angle or instrumental field of view. For the geometric tagging, all LIDAR data points 23 which are located outside the aperture angle or instrumental field of view of the respective antenna are therefore marked as “occluded” for the respective antenna. For the remaining LIDAR data points 23, a cylinder 27 (see FIG. 2) is wrapped around the origin of the radar coordinate system, i.e., around the respective antenna. The cylinder axis is in parallel to the upright axis (z-axis) of the vehicle coordinate system 16 (see FIG. 1).

[0065] The surface of the cylinder 27 is divided into pixel areas, and the LIDAR data points 23 which fall into the aperture angle or instrumental field of view of the respective antenna of the radar sensors 13 are projected to the surface of the cylinder 27. For each pixel area of the cylinder 27, the projections of LIDAR data points 23 are considered which fall into this area, and these LIDAR data points 23 are sorted with respect to their distance to the origin of the radar coordinate system. The LIDAR data point 23 having the closest distance to the respective radar sensor 13 is regarded as visible for the respective pixel area, while all further LIDAR data points 23 are marked as “occluded” for this pixel area and for the respective antenna.

[0066] In the example shown in FIG. 2, all LIDAR data points 23 of the left label 19 are considered as visible for the radar sensors 13, while for the right label 19 the upper three LIDAR data points 23 are regarded as visible only since they have a direct line of sight to at least one of the radar sensors 13, and the further LIDAR data points denoted by 29 are marked as “occluded” since there is no direct line of sight to one of the radar sensors 13. In detail, for the occluded LIDAR data points 29 there is another LIDAR data point 23 which has a projection within the same pixel area on the cylinder 27 and which has a closer distance to the origin of the radar coordinate system.

[0067] In order to determine whether the entire bounding box or label 19 is regarded as occluded for the radar sensors 13, the number of LIDAR data points 23 belonging to the respective label 19 and being visible (i.e., not marked as “occluded”) for at least one single radar antenna is counted. If this number of visible LIDAR data points 23 is lower than a visibility threshold, the no-care attribute is assigned to the respective label 19. The visibility threshold may be set to two LIDAR data points, for example. In this case, the right object or label 19 as shown in FIG. 2 would be assigned the care attribute although the LIDAR data points 29 are marked as occluded.

[0068] FIG. 3 depicts a practical example for a cloud of LIDAR data points 23 detected by the LIDAR system 15 of the vehicle 11. Since the LIDAR system 15 is mounted on the roof of the host vehicle 11, there is a circle or cylinder around the vehicle 11 denoting a region 31 for which no LIDAR data points 23 are available. The darker LIDAR data points denoted by 33 are marked as visible by the geometric tagging procedure as described above, while the lighter LIDAR data point denoted by 35 are marked as occluded for the radar sensors 13.

[0069] For most of the bounding boxes or labels 19 which are shown in FIG. 3, there is a plurality of visible LIDAR data points 33, i.e., visible for the radar sensors 13. Therefore, the care attribute is assigned to these labels by the geometric tagging. For the bounding box or label in the lower part of FIG. 3 which is denoted by 37, there are LIDAR data points 23 which belong to this bounding box, but these LIDAR data points 23 are marked as occluded LIDAR data points 35 by the geometric tagging. Simply speaking, the “view” from the host vehicle 11 to the label 37 is blocked by the other bounding boxes or labels 19 in between. Hence, the number of visible LIDAR data points 33 falls below the visibility threshold for the label 37, and the no-care attribute is assigned to this label 37.

[0070] It is noted that the above procedure of geometric tagging is also referred to as z-buffering in computer graphics. As an alternative to the cylinder 27 (see FIG. 2), a sphere could also be used for the projection of the LIDAR data points 23 to a respective pixel area in order to determine the LIDAR data point 23 having the closest distance via z-buffering.

[0071] For providing a reliable ground truth for the training of the machine-learning algorithm, the attributes 22 determined by activation tagging and by geometric tagging are combined. That is, a label 19 obtains the care attribute only if both the activation tagging and the geometric tagging have provided the care attribute to the respective label, i.e., if the label can be perceived by the radar sensors 13 due to sufficient radar energy and is geometrically visible (not occluded) for at least one of the radar sensors 13.

[0072] For the training of the machine-learning algorithm, i.e., of the radar neural network, labelled data are provided which include inputs in the form of primary data from the radar sensors 13 and the labels 19 which are also referred to as ground truth and which are provided in the form of bounding boxes 19 (see FIGS. 1 to 3). As mentioned above, each label 19 additionally has a care attribute or a no-care attribute due to the combination of the activation tagging and the geometric tagging. During the training, parameters of the machine-learning algorithm or radar neural network are adjusted for classification and regression. This adjustment is controlled via a loss function which is based on error signals due to a comparison between predictions of the model or machine-learning algorithm and the desired output being represented by the labels 19. The goal of the training procedure is to minimize the loss function.

[0073] During the training, two types of contributions may be received by the loss function. For positive loss contributions, weights of a model on which the machine-learning algorithm relies are increased if these weights contribute constructively to a prediction corresponding to the ground truth or label 19. Conversely, for negative loss contributions the weights of the model are decreased if these weights contribute constructively to a prediction which does not correspond to the ground truth, i.e., one of the labels 19.

[0074] For the training procedure according to the present disclosure, the labels 19 having the care attribute are generally permitted to provide positive and negative loss contributions to the loss function. For the labels 19 having the no-care attribute, neither positive nor negative contributions could be permitted, i.e., labels having the no-care attribute could simply be ignored. Hence, the machine-learning algorithm could not be forced to predict any label or object which is not perceivable by the radar sensors 13. However, any wrong prediction would also be ignored and not analyzed by a negative loss contribution in this case. Therefore, the negative loss contribution is at least to be permitted for labels having the non-care attribute.

[0075] To improve the training procedure, a dynamic positive loss contribution is also permitted for the labels 19 having the no-care attribute. In detail, a positive loss contribution is generated for a label 19 having a no-care attribute only if a confidence value P for predicting a ground truth label 19 is greater than a predefined threshold τ, i.e., P>τ, wherein τ is greater than or equal to 0 and smaller than 1.

[0076] Allowing dynamic positive loss contributions for labels 19 having a no-care attribute allows the machine-learning algorithm or model to use complex queues, i.e., multi-reflections, temporal queues, to predict e.g., the presence of objects. Hence, permitting positive loss contributions for labels having the no-care attribute in a dynamic manner (i.e., by controlling the predefined threshold τ for the confidence value P) will strengthen complex decisions and improve the performance of the model predictions via the machine-learning algorithm.

[0077] In FIG. 4, the impact of training a machine-learning algorithm by using the care and no-care attributes of the labels 19 is depicted. Precision and recall are typical figures of merit for machine-learning algorithms, i.e., neural networks. Precision describes the portion of positive predictions or identifications which are actually correct, whereas recall describes the portion of actual positive predictions or identifications which are identified correctly. In addition, recall is a measure for the stability of the machine-learning algorithm or neural network.

[0078] In FIG. 4, the precision is represented on the y-axis over the recall on the x-axis. An ideal neural network would have a precision and recall in the range of 1, i.e., a data point in the upper right corner of FIG. 4. The curves 41 to 46 as shown in FIG. 4 are generated by calculating precision and recall for different predetermined thresholds τ for the confidence value P of a respective prediction. The prediction is not considered if its confidence value is below the threshold. If the threshold is lowered, recall is usually increased while precision is decreased.

[0079] The solid lines 41, 43 and 45 represent the results for a machine-learning algorithm which does not use the attributes 22 for the labels 19, i.e., the care and no-care attributes have not used for the training of the machine-learning algorithm. In contrast, the dashed lines 42, 44 and 46 depict the results for a machine-learning algorithm which has been trained via labels 19 having the care or no-care attribute according to the present disclosure.

[0080] Pairs of lines shown in FIG. 4 represent the results for predicting a respective object class, i.e., the lines 41 and 42 are related to the object class “pedestrian”, the lines 43 and 44 are related to the object class “moving vehicle”, and the lines 45 and 46 are related to the object class “stationary vehicle”. As shown by the dashed lines 42, 44, 46 in comparison to the respective solid lines 41, 43, 45 for the same object class, the training of the machine-learning algorithm which includes the care and no-care attributes generates a better precision and a better recall in comparison to the training of the machine-learning algorithm which does not consider any attributes 22.

REFERENCE NUMERAL LIST

[0081] 11 host vehicle [0082] 13 radar sensor [0083] 15 LIDAR system [0084] 16 vehicle coordinate system [0085] 17 processing unit [0086] 18 x-axis [0087] 19 bounding box, label [0088] 20 y-axis [0089] 21 normalized radar energy [0090] 23 LIDAR data point [0091] 25 line of sight [0092] 27 cylinder [0093] 29 occluded LIDAR data point [0094] 31 region without LIDAR data points [0095] 33 LIDAR data point visible for the radar sensor [0096] 35 LIDAR data point occluded for the radar sensor [0097] 37 occluded label [0098] 41 line for class “pedestrian”, labels without attributes [0099] 42 line for class “pedestrian”, labels with attributes [0100] 43 line for class “moving vehicle”, labels without attributes [0101] 44 line for class “moving vehicle”, labels with attributes [0102] 45 line for class “stationary vehicle”, labels without attributes [0103] 46 line for class “stationary vehicle”, labels with attributes