METHOD, APPARATUS AND COMPUTER PROGRAM FOR ENABLING A SENSOR SYSTEM FOR DETECTING OBJECTS IN AN ENVIRONMENT OF A VEHICLE

Abstract

A method for obtaining approval of a sensor system for detecting objects in a vehicle's environment includes providing a combined probability distribution for deviations between output data from a sensor system and reference data at the programming level for detecting objects by the sensor system, at the sensor level and/or at the fusion level, sampling deviation combinations and calculating occurrence probabilities for the sampled deviation combinations using the combined probability distribution, subjecting the reference data to the sampled deviation combinations, processing these reference data with a fusion unit, and obtaining fusion results, removing occurrence probabilities from the combined probability distribution from which those fusion results are obtained that satisfy a predefined condition, and obtaining a residual probability distribution, taking the integral of the residual probability distribution and obtaining an absolute error probability, and obtaining approval of the sensor system based on the absolute error probability.

Claims

1-13. (canceled)

14. A method for obtaining approval of a sensor system for detecting objects in a vehicle's environment, the method comprising: providing a combined probability distribution for deviations between output data from a sensor system and reference data at at least one of a programming level for detecting objects by the sensor system, a sensor level in the sensor system, or a fusion level in the sensor system; sampling deviation combinations and calculating occurrence probabilities for the sampled deviation combinations by means of the combined probability distribution; subjecting the reference data to the sampled deviation combinations, processing these reference data with a fusion unit in the sensor system, and obtaining fusion results; removing those occurrence probabilities from the combined probability distribution from which the fusion results are obtained that satisfy a predefined condition, and obtaining a residual probability distribution; taking an integral of the residual probability distribution and obtaining an absolute error probability; and obtaining approval of the sensor system on the basis of the absolute error probability.

15. The method according to claim 14, wherein the deviations between the output data from the sensor system and the reference data comprise at least one of deviations in distance measurements or deviations dependent on the weather or the state of the vehicle.

16. The method according to claim 14, wherein a probability distribution is provided for the deviations of each sensor in the sensor system and the deviation combinations are sampled by means of the individual probability distributions of the sensors.

17. The method according to claim 14, wherein the combined probability distribution is provided as a multidimensional standard deviation for the individual sensors in the sensor system.

18. The method according to claim 14, wherein the output data from the sensor system and the reference data are obtained while driving the vehicle, wherein the reference data comprise at least one of reference data recorded by a reference sensor system or reference data generated by means of sensor models.

19. The method according to claim 14, wherein the reference data are processed in at least one of Model in the Loop, Software in the Loop, or Hardware in the Loop simulations or with surrogate models.

20. The method according to claim 14, wherein the fusion results are obtained using a redundancy in the sensor system.

21. The method according to claim 14, wherein deviations between the fusion results and the reference data are evaluated using a limit state function in order to remove the occurrence probabilities from the probability distribution.

22. The method according to claim 14, wherein amounts of output data and amounts of reference data are increased incrementally.

23. An apparatus for obtaining approval of a sensor system for detecting objects in a vehicle's environment, comprising at least one processor configured to execute the steps of the method according to claim 14.

24. A sensor system for detecting objects in a vehicle's environment, comprising a plurality of sensors using at least one specific sensor technology, and the apparatus according to claim 23.

25. A non-transitory computer readable medium having stored thereon commands for obtaining approval of a sensor system for detecting objects in a vehicle's environment, wherein the commands, when executed a at least one processor, execute the method according to claim 14.

26. A vehicle comprising: the sensor system according to claim 24; at least one control unit; actuators, wherein the sensor system, the control unit, and the actuators are connected for signal transmission, such that the vehicle is configured to drive at least partially autonomously; and an interface for at least one reference system; wherein the absolute error probability is obtained while driving the vehicle, and the error probability is supplied as feedback to a driving strategy for the vehicle.

Description

[0071] In the drawings of exemplary embodiments illustrating the invention:

[0072] FIG. 1: shows a camera data clustering over distance;

[0073] FIG. 2: shows a histogram of the distribution of the deviations of the camera data from FIG. 1;

[0074] FIG. 3: shows a bootstrapping distribution of the mean value for the distribution of the camera data from FIG. 2;

[0075] FIG. 4: shows the bootstrapping distribution of the standard deviation of the distribution of the camera data from FIG. 2;

[0076] FIG. 5: shows fitted probability distribution deviations of the camera data from FIG. 2 with confidence limits from the mean value from FIG. 3 and standard deviation from FIG. 4;

[0077] FIG. 6: shows a histogram of the distribution of deviations of further camera data and the fitted probability distribution of these deviations;

[0078] FIG. 7: shows a histogram of the distribution of deviations of radar data and a fitted probability distribution of these deviations;

[0079] FIG. 8: shows a histogram of the distribution of synthetic deviations of lidar data and a fitted probability distribution of these deviations;

[0080] FIG. 9: shows a combined probability distribution of deviations of camera data and radar data for a predefined deviation of lidar data;

[0081] FIG. 10: shows the residual probability distribution of the combined probability distribution from FIG. 9;

[0082] FIG. 11: shows, by means of trapezoid numerical integration, the integrated residual probability distribution with boxes indicating the width of the confidence interval;

[0083] FIG. 12: shows the fitted absolute error probability with a lower and upper confidence;

[0084] FIG. 13: shows an offline exemplary embodiment of a method according to the invention;

[0085] FIG. 14: shows an online exemplary embodiment of a method according to the invention;

[0086] FIG. 15: shows the technical structure of an exemplary embodiment of an apparatus according to the invention; and

[0087] FIG. 16: shows an exemplary embodiment of a vehicle according to the invention.

[0088] Identical reference symbols in the drawings indicate the same or functionally similar parts. For purposes of clarity, relevant reference parts are indicated in the respective figures. Distance measurements and deviations are given in meters in all of the figures.

[0089] The diagram in FIG. 1 shows the deviations a of a camera sensor over a distance d. The camera is a sensor 11 in a sensor system 10, see FIG. 16. The deviations a are derived from a comparison of camera data from recorded measurements with reference data from a reference system 20. The reference system comprises high resolution sensors 21, e.g. a high resolution camera. The reference data comprise measurement data from the high resolution camera. The camera data and reference data are stored in a data base, e.g. a cloud memory or data farm. An evaluation logics unit 13, e.g. a computer, processor core, or a chip with a logic module, determines the difference between the camera data and the reference data and outputs this as the deviation a, see FIG. 13. This is how the measurement errors of the camera are recorded.

[0090] Deviations for other sensors 11 in the sensor system 10, e.g. radar and/or lidar, are obtained analogously according to the invention. The deviations a for lidar data are generated synthetically, for example. Data sheets of lidar sensors are used for the reference data in this case.

[0091] Deviations a between the sensor system 10 and the reference system 20 are also examined for different intensities of precipitation or different vehicle speeds according to the invention.

[0092] The data from the sensor system 10 comprise 50 recorded scenarios, for example, such as “stopping and going,” with ca. 3 million data points.

[0093] The histogram in FIG. 2 is created from the deviations shown in FIG. 1. The histogram illustrates the occurrence probabilities p for the individual deviations a of the camera. Similar histograms are obtained analogously for other sensors 11 in the sensor system.

[0094] The mean value and standard deviation for the deviations a are obtained from the bootstrapping of the deviations a of the camera data shown in FIGS. 3 and 4. Bootstrapping is carried out analogously for other sensors 11 in the sensor system 10. A number n of random samples are selected for the bootstrapping, and the mean value and standard deviation are calculated for each random sample. This results in the distribution of mean values shown in FIG. 3 and the standard deviation shown in FIG. 4.

[0095] The histogram in FIG. 2 is approximated with a probability distribution by means of the confidence interval limits obtained for the mean value, see FIG. 3, and the standard deviation, see FIG. 4. The resulting probability distribution for the deviations a of the camera data is shown in FIG. 5.

[0096] FIG. 6 shows another probability distribution for the deviations a of further camera data, FIG. 7 shows a probability distribution for the deviations a of radar data, and FIG. 8 shows a probability distribution for the deviations a of lidar data. FIG. 8 is based on synthetic deviations a of the lidar data, which are based on a data sheet that was used as the reference data for the lidar sensor.

[0097] The combined probability distribution shown in FIG. 9 is a multidimensional probability distribution obtained from individual probability distributions of the deviations a of the camera, radar, and lidar data. The deviations a of the radar data are shown in FIG. 9 on the x-axis, the deviations a of the camera data are shown on the y-axis, and the combined probability distribution is shown on the z-axis. The combined probability distribution is shown for a fixed deviation a of the lidar data. A corresponding FIG. 9 is obtained for every deviation a of the lidar data. The combined probability distribution is obtained according to the invention in step V1 of the method.

[0098] Deviation combinations are sampled in step V2, and occurrence probabilities p are calculated for the sampled deviation combinations by means of the combined probability distribution. This generates critical error combinations. By way of example, error combinations are simulated in a simulation and/or reprocessing environment. The error effect is then evaluated by means of the combined probability distribution.

[0099] The reference data are then subjected to the sampled deviation combinations in a third step. These reference data are processed by a fusion unit 12 in the sensor system 10, and fusion results are obtained.

[0100] The fusion results are evaluated with a limit state function in a fourth step V4 in order to remove the combined probability distribution of those occurrence probabilities p, from the fundamental deviation combinations of which such fusion results are obtained that satisfy a predefined condition. The result is the residual probability distribution shown in FIG. 10.

[0101] The residual probability distribution is integrated in a fifth step V5, and an absolute error probability P is obtained, see FIGS. 11 and 12. The absolute error probability P is approximated, for example, by means of trapezoid numerical integration.

[0102] The sensor system 10 is approved in a sixth step V6 in a testing and approval process, depending on the absolute error probability P for the sensor system 10. The absolute error probability P forms a legitimization basis for the test base, i.e. the output data from the sensor system 10, the reference data, and the scenarios that are used. Because the 2 billion kilometers, or the equivalent thereof in driving time, currently required in the reference sources do not need to actually be driven, e.g. by the vehicle 1 and the sensor system 10 and reference system 20 mounted thereon, as a result of the examination of the individual error images, the legitimization of a small testing basis using the absolute error probability P enables approval of an autonomous vehicle technology.

[0103] FIG. 13 shows an overview of the subject matter of the invention in an offline variation. The autonomous vehicle 1 is the people mover shown by way of example in FIG. 16. The mathematically random model comprises the probability distribution of the individual sensors 11 in the sensor system 10 and the resulting multidimensional combined probability distribution. The method according to the invention is carried out by the processor 30. The method steps V2 and V3 are carried out in a simulation and reprocessing environment, which can be part of the processor. The simulation environment is a software, by way of example, e.g. CarMaker. A prediction regarding residual risk is obtained in steps V4 and V6.

[0104] FIG. 14 shows an overview of the subject matter of the invention in the online variation. Unlike in FIG. 13, the absolute error probability P is evaluated on a continuous basis and supplied as feedback to the driving strategy for the vehicle 1.

[0105] FIG. 15 shows the technical structure of the processor 30 in the apparatus according to the invention. The core of the method, which is carried out by the processor 30, is obtaining the probability distributions for the deviations of the individual sensors 11, see FIGS. 6, 7, and 8, the resulting combined probability distribution, the sampling of deviation combinations, and obtaining the absolute error probability P. The data input for the processor 30 comprises the output data from the sensor system 10 and the reference data from the reference sensor system 20. The data output of the processor comprises the absolute error probability P.

[0106] The people mover in FIG. 16 is an example of a vehicle 1 according to the invention. The vehicle 1 comprises a sensor system 10 that contains numerous sensors 11. Only one of these sensors 11 is shown in FIG. 16, for purposes of clarity. The output data from the individual sensors 11 are combined in the fusion unit 12 in the sensor system 10. The vehicle 1 also comprises a reference sensor system 20 that contains numerous reference sensors 21. Only one reference sensor 21 is shown in FIG. 16, for purposes of clarity. The output data from the individual sensors 21 are combined in a fusion unit 22 in the reference sensor system 20.

[0107] The people mover in the form of an electric bus can comfortably accommodate 15 people. It does not take up any more surface area than a large limousine. As a result, it can be easily maneuvered in urban environments. The people mover has no emissions, offers various connectivity services, and can be driven in a highly automated manner. According to one aspect of the invention, the vehicle 1, e.g. the people mover, comprises a connectivity interface. A user can request the people mover as a shuttle in a flexible manner via an app, and is therefore not limited to rigid scheduling.

REFERENCE SYMBOLS

[0108] 1 vehicle [0109] 10 sensor system [0110] 11 sensor [0111] 12 fusion unit [0112] 13 evaluation logics unit [0113] 14 data base [0114] 20 reference system [0115] 21 sensor [0116] 22 fusion unit [0117] 30 processor [0118] V1-V6 method steps [0119] p occurrence probability [0120] P absolute error probability [0121] d distance [0122] a deviation