METHOD FOR DETERMINING THE RELIABILITY OF OBJECTS

Abstract

A method for determining the reliability of detected and/or tracked objects for use in the driver assistance or the at least semiautomated driving of a vehicle. The method includes: a) receiving sensor data for the detected objects from a plurality of sensors, b) associating pieces of object existence information with each object, c) checking, considering, or representing redundancy of pieces of object existence information.

Claims

1. A method for determining reliability of detected and/or tracked objects for use in driver assistance or the at least semiautomated driving of a vehicle, the method comprising the following steps: a) receiving sensor data for the detected objects from a plurality of sensors; b) associating pieces of object existence information with each of the objects; and c) checking or considering or representing redundancy of the pieces of object existence information.

2. The method as recited in claim 1, wherein the pieces of object existence information are provided in the form of object existence indicators or existence indicator flags.

3. The method as recited in claim 1, wherein the pieces of object existence information include one or multiple of the following existence indicator flags: object within a field of view of a sensor, object detectable, measurement of a sensor in conjunction with the object in a present measuring frame, measurement of a sensor increases an existence probability of the object in a present measuring frame, signal flags.

4. The method as recited in claim 1, wherein the pieces of object existence information are provided in binary form or by outputting a certain number of bits.

5. The method as recited in claim 1, wherein the redundancy of the pieces of object existence information is considered in a determination of at least one value that describes the reliability of the object recognition.

6. The method as recited in claim 1, wherein the plurality of sensors include one or multiple sensor modalities.

7. The method as recited in claim 1, wherein the redundancy of the pieces of object existence information is checked or considered or represented in conjunction with objects which are detected by various sensor modalities that detect the same detection area.

8. A non-transitory machine-readable memory medium on which is stored a computer program for determining reliability of detected and/or tracked objects for use in driver assistance or the at least semiautomated driving of a vehicle, the computer program, when executed by a computer, causing the computer to perform the following steps: a) receiving sensor data for the detected objects from a plurality of sensors; b) associating pieces of object existence information with each of the objects; and c) checking or considering or representing redundancy of the pieces of object existence information.

9. A system for determining reliability of objects which are detected for use in driver assistance or during at least semiautomated driving of a vehicle, the system comprising: a plurality of sensors configured to provide sensor data for the detected objects, the plurality of sensors including one or multiple sensor modalities; an electronic tracking unit configured to receive the sensor data, the electronic tracking unit configured to process the sensor data to: connect pieces of object existence information to each of the objects, and check or consider or represent redundancy of the pieces of object existence information.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG. 1 schematically shows an exemplary flowchart of the method presented here, according to the present invention.

[0036] FIG. 2 schematically shows a vehicle including an exemplary structure of a system described here, according to the present invention.

[0037] FIG. 3 schematically shows an exemplary possible application of an embodiment variant of the method described here, according to the present invention.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0038] FIG. 1 schematically shows an exemplary flowchart of the method presented here. The method is used to determine the reliability of detected and/or tracked objects for use in the driver assistance or the at least semiautomated driving of a vehicle 1 (cf. FIG. 2). The order of steps a), b), and c) shown by blocks 110, 120, and 130 is by way of example and may be run through at least once in the order shown, for example, to carry out the method.

[0039] In block 110, according to step a), sensor data are received for the detected objects from a plurality of sensors 2, 3, 4. In block 120, according to step b), pieces of object existence information are associated with each object. In block 130, according to step c), redundancy of pieces of object existence information is checked, considered, or represented.

[0040] FIG. 2 schematically shows a vehicle 1 including an exemplary structure of a system described here. The system is used to determine the reliability of objects, which are detected for use in the driver assistance or during at least semiautomated driving of a vehicle 1. The system includes a plurality of sensors 2, 3, 4 for providing sensor data for the detected objects, the plurality of sensors 2, 3, 4 including one or multiple sensor modalities.

[0041] Furthermore, the system includes an electronic tracking unit 5 for receiving the sensor data, electronic tracking unit 5 being configured to process the sensor data in order to connect pieces of object existence information to each object and check, consider, or represent a redundancy of pieces of object existence information.

[0042] FIG. 1 schematically shows by way of example in this context a system for determining the reliability of detected objects for use in the driver assistance or the at least semiautomated driving of a vehicle 1. The system includes a plurality of sensors 2, 3, 4 for providing sensor data for the detected objects, the plurality of sensors including one or multiple sensor modalities. The system furthermore includes an electronic tracking and/or fusion unit 5 for receiving the sensor data, the electronic tracking and/or fusion unit being configured in such a way that it processes the sensor data in order to associate pieces of information about the presence of objects with each object and check, consider, or represent redundancy of pieces of information about the presence of objects.

[0043] In addition, the system or vehicle 1 may include a planner or an electronic planning unit 6. Planning unit 6 may receive and process data or results from tracking unit 5. For example, planning unit 6 may use data or results from tracking unit 5 for route or trajectory planning. This may advantageously contribute to automating the driving operation of vehicle 1.

[0044] In particular, in one specific embodiment, planning unit 6 may enable the driver assistance and in another specific embodiment, it may enable the at least semiautonomous control of vehicle 1. Planner or planning unit 6 may receive the data or pieces of information from tracking or fusion unit 5, for example, in the form of one or multiple interfaces.

[0045] Various functions in the planner or in planning unit 6 may require different threshold values for the reaction to objects. In some cases, a reaction may also take place to objects, the presence of which is not certain, for example, it may be reasonable not to carry out a lane change if there is an indication of an object on the adjacent lane. In other cases, it may be reasonable for a reaction to take place only when an object is present with a high level of certainty, for example, in the event of a hard evasion maneuver. In particular if the existence indicators for each object are made available to planner 6, it is advantageously possible to decide on the basis of the present reaction and possibly additional pieces of information from other sources (for example, an HD map), whether the object has to be considered or not.

[0046] The interpretation of the objects may take place outside fusion module 5 (for example, within planner component 6). In this case, the flags may be visible, for example, directly at the interfaces. If the flags are not directly visible at the interface, the fusion output may supply the signals computed on the basis of the existence indicator flags.

[0047] In one specific embodiment, an architecture is provided for representing the redundancy of pieces of object existence information. The architecture supports itself in particular on existence indicators which are connected to each object. The basic concept is that instead of computing multiple variables, for example, sensor-specific existence and discovery probabilities within a fusion system and then providing them with a threshold value in order to make a decision, a high or higher number of existence indicator flags is used. The discretization may thus advantageously already take place at the input, and computing and memory space may thus be saved.

[0048] The pieces of object existence information may include, for example, one or multiple of the following existence indicator flags, in particular for each object and/or each sensor 2, 3, 4: [0049] object within the field of view of sensor 2, 3, 4, [0050] object detectable (detection probability above a threshold value), [0051] measurement of sensor 2, 3, 4 in conjunction with the object in the present measuring frame, [0052] measurement of sensor 2, 3, 4 increases the existence probability of the object in the present measuring frame, [0053] signal flags (for example: was in the field of view once; was recognized at least once).

[0054] Multiple sensors 2, 3, 4 may advantageously include one or multiple sensor modalities, for example, radar, LIDAR, camera, and/or video image sensors. Advantageously, multiple sensors 2, 3, 4 may include different ones of the sensor modalities (types of sensors different from one another), such as a video sensor and a LIDAR sensor.

[0055] This may also particularly advantageously contribute to the fact that the redundancy of the pieces of object existence information may be checked, considered, or represented in conjunction with objects which are detected by various sensor modalities which detect the same detection area.

[0056] FIG. 3 schematically shows an exemplary possible application of an embodiment variant of the method described here. FIG. 3 schematically shows in this context by way of example a visual representation of an exemplary set of binary existence indicators, particularly advantageously here in the form of binary existence probability flags 7.

[0057] This represents an example that and possibly how the pieces of object existence information may be provided in the form of object existence indicators or existence indicator flags 7.

[0058] For example, flags 7 may be output in each case for detections of the three sensors 2, 3, 4 observed here by way of example, which flags describe in preferably binary form whether an object is located in the detection area of particular sensor 2, 3, 4.

[0059] In particular, in one specific embodiment of the method, an object existence redundancy representation may be provided using existence indicator flags 7.

[0060] For each of these flags, it is possible to represent them either in a binary format or by output of N bits. An N-bit flag for the discovery probability enables, for example, the division of the range of possible values [0, 1] into 2{circumflex over ( )}N intervals. Due to the use of a significantly higher number of flags in comparison to the desired variables such as presence and detection probability, the information loss as a result of the discretization may advantageously be compensated for by the information gain, which results in particular in that the data of sensors having different fields of view and capabilities are not merged. In addition, indicator flags 7 advantageously enable the combination of flags 7 for various subsets of sensors 2, 3, 4.

[0061] This represents an example that and possibly how the pieces of object existence information may be provided in binary form or by outputting a certain number of bits.

[0062] Multiple approaches are possible for the interpretation of existence indicator flags 7. On the one hand, it may simply be counted how many of sensors 2, 3, 4 have confirmed an object (of sensors 2, 3, 4 which have the object in the field of view). On the other hand, flags 7 could be used to train a classifier or a neural network in order to compute several values that are simple to interpret on the basis of all flags 7 available for the object. Possible values are, inter alia, existence probability, existence uncertainty, object redundancy, and conflict between sensor modalities.

[0063] This represents an example that and possibly how the redundancy of the pieces of object existence information may be considered in the determination of at least one value which describes the reliability of the object recognition. A particularly advantageous example of such a value is the existence probability.

[0064] The method and system described here may represent an expansion of the disclosure in US 2020/0377121 A1, which is hereby incorporated by reference. The method and/or system described here may therefore be combined with a method and/or system which computes sensor-type-specific or sensor-specific existence probabilities. Both approaches may also be combined to increase the redundancy (for example: use of existence indicator flags 7 in a preferably AI-based approach and classic existence probability computations according to the approach presented in US 2020/0377121 A1).

[0065] The described method and the described system may contribute to an improvement for difficult scenarios in particular having contradictory pieces of sensor information in particular about the existence of objects. The pieces of detailed information per sensor are advantageously useful for the interpretation and reaction to complex cases, for example, temporary/intermittent sensor failures, contradictory pieces of information, and/or earlier estimation errors.