METHOD AND DEVICE FOR GENERATING COMBINED SCENARIOS

Abstract

A method for generating combined scenarios for testing an object detection unit, wherein the method comprises provision of first sensor data of a first scenario and of second sensor data of a second scenario, wherein the first sensor data and the second sensor data in each case are a point cloud comprising a plurality of points, wherein the method further comprises a classification of the respective points of the first sensor data and of the respective points of the second sensor data into relevant or not relevant and merging of the first sensor data and of the second sensor data for obtaining third sensor data of a combined scenario, wherein only relevant points of the first sensor data and relevant points of the second sensor data are merged to form third sensor data of the combined scenario.

Claims

1. A method for generating combined scenarios for testing an object detection unit, wherein the method comprises provision of first sensor data of a first scenario and of second sensor data of a second scenario, wherein the first sensor data and the second sensor data in each case have at least one point cloud comprising a plurality of points, wherein the method comprises classification of the respective points of the first sensor data and of the respective points of the second sensor data into relevant or not relevant, wherein the method comprises merging of the first sensor data and of the second sensor data for obtaining third sensor data of a combined scenario, and wherein only relevant points of the first sensor data and relevant points of the second sensor data are merged to form third sensor data of the combined scenario.

2. The method according to claim 1, wherein the first sensor data and the second sensor data are represented in a common voxel map, wherein free voxels are identified as free spaces.

3. The method according to claim 1, wherein the method comprises respective identification of objects on a basis of the first sensor data and of the second sensor data, wherein positions of detected objects of the first sensor data and of the second sensor data are compared in order to detect possible overlaps and/or masking.

4. The method according to claim 1, wherein the method comprises the identification of noise points.

5. The method according to claim 2, wherein classification takes place on the basis of identified objects, identified free spaces and/or identified noise points.

6. The method according to claim 2, wherein masked points of a point cloud are classified as not being relevant as part of the classification, wherein the masked points within the common voxel map are masked by at least one point of the same and/or of the other point cloud.

7. The method according to claim 1, wherein points of a point cloud are classified as not being relevant as part of the classification when the points are masked by an object of other point cloud and/or when the points lie within an object of the other point cloud and/or when the points originate from a first object, which overlaps with a second object of the other point cloud and/or when the points appear in the first sensor data and the second sensor data.

8. The method according to claim 1, wherein the merging of the first sensor data and of the second sensor data comprises an entry of the points, which are classified as being relevant, in a common coordinate system.

9. The method according to claim 1, wherein the method comprises provision of a first ground truth for the first scenario and of a second ground truth for the second scenario.

10. The method according to claim 9, wherein the method comprises a merging of the first ground truth and of the second ground truth to form a ground truth of the combined scenario.

11. The method according to claim 9, wherein the method comprises testing of an object detection unit, wherein the testing comprises provision of a third sensor data of a combined scenario and a comparison of objects detected by the object detection unit to the ground truth of the combined scenario.

12. The method according to claim 1, wherein the method comprises generation of a library for generating sensor data of combined scenarios, wherein sensor data of different scenarios and optionally the corresponding ground truth of the scenarios are stored in the library.

13. A device for generating combined scenarios for testing an object detection unit, wherein the device comprises a unit for merging first sensor data and second sensor data for obtaining third sensor data of the combined scenarios, wherein the device comprises a classification unit for classifying respective points of the first sensor data and of respective points of the second sensor data into relevant and not relevant.

14. A computer program product comprising a computer-readable storage medium on which a program is stored, which, after it has been loaded into a memory of the computer, makes it possible for the computer to carry out a method for generating combined scenarios for testing an object detection unit, wherein the method comprises provision of first sensor data of a first scenario and of second sensor data of a second scenario, wherein the first sensor data and the second sensor data in each case have at least one point cloud comprising a plurality of points, wherein the method comprises a classification of the respective points of the first sensor data and of the respective points of the second sensor data into relevant or not relevant, wherein the method comprises merging of the first sensor data and of the second sensor data for obtaining third sensor data of a combined scenario, and wherein only relevant points of the first sensor data and relevant points of the second sensor data are merged to form the third sensor data of the combined scenario.

15. A computer-readable storage medium on which a program is stored, which, after it has been loaded into a memory of the computer, makes it possible for the computer to carry out a method for generating combined scenarios for testing an object detection unit, wherein the method comprises provision of first sensor data of a first scenario and of second sensor data of a second scenario, wherein the first sensor data and the second sensor data in each case have at least one point cloud comprising a plurality of points, wherein the method comprises a classification of the respective points of the first sensor data and of the respective points of the second sensor data into relevant or not relevant, wherein the method comprises merging of the first sensor data and of the second sensor data for obtaining third sensor data of a combined scenario, and wherein only relevant points of the first sensor data and relevant points of the second sensor data are merged to form third sensor data of the combined scenario.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Schematically:

[0061] FIG. 1 shows a schematic diagram of a method according to the disclosure;

[0062] FIG. 2 shows first sensor data;

[0063] FIG. 3 shows second sensor data;

[0064] FIG. 4 shows a common voxel map with the first sensor data of FIG. 2 and the second sensor data of FIG. 3;

[0065] FIG. 5 shows the common voxel map of FIG. 4 after identification of noise points;

[0066] FIG. 6 shows the common voxel map of FIG. 5 after classification of masked points as not being relevant;

[0067] FIG. 7 shows a voxel map with the third sensor data of the combined scenario;

[0068] FIG. 8 shows a device according to the disclosure; and

[0069] FIG. 9 shows a schematic sequence of a method according to the disclosure, integrated into the device according to FIG. 8.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0070] FIG. 1 shows a schematic diagram of a method 100 according to the disclosure, which comprises the following steps:

[0071] The method 100 serves the purpose of generating 101 combined scenarios for testing 117 an object detection unit 17. The method 100 can in particular comprise the generation 101 of combined scenarios.

[0072] The method 100 comprises the provision 103 of first sensor data 11 of a first scenario and of second sensor data 12 of a second scenario. The method 100 can comprise the prior acquisition 102 of the first sensor data 11 and of the second sensor data 12.

[0073] The method 100 can further comprise the provision 105 of a first ground truth for the first scenario and of a second ground truth for the second scenario, wherein the method 100 can further preferably comprise the acquisition 104 of the two ground truths.

[0074] Based on the provided ground truth, the first sensor data 11 of the first scenario and the second sensor data 12 of the second scenario can be validated 106.

[0075] The method 100 comprises a classification 107 of the respective points 11a of the first sensor data 11 and of the respective points 12a of the second sensor data 12 into relevant or not relevant. For the classification, the method 100 can comprise the representation 108 of the first sensor data 11 and of the second sensor data 12 in a common voxel map 23, wherein free voxels can be identified 109 as free spaces.

[0076] The method 100 can further comprise the identification 110 of objects on the basis of the first sensor data 11 and of the second sensor data 12. The positions of the detected objects can be compared 111 thereby and possible overlaps and/or masking of the objects can thus be detected 112. The method 100 can further comprise the identification 113 of noise points 24.

[0077] The method 100 further comprises the merging 114 of the first sensor data 11 and of the second sensor data 12 for obtaining third sensor data 28 of the combined scenario. A combined scenario is generated 101 in this way.

[0078] The merging 114 can comprise the entry 115 of the points classified as being relevant in a common coordinate system.

[0079] The method 100 can further comprise the merging 116 of the first ground truth and of the second ground truth to form a ground truth of the combined scenario.

[0080] The method can additionally comprise the testing 117 of an object detection unit 17, wherein the testing 117 can comprise the provision 118 of the third sensor data 28 of the combined scenario as well as a comparison 119 of the objects detected by the object detection unit 17 to the ground truth of the combined scenario.

[0081] FIG. 2 shows first sensor data 11. The first sensor data thereby represent a point cloud, wherein the individual points 11a of the first sensor data are clearly visible.

[0082] FIG. 3 illustrates second sensor data 12, which, as point cloud, likewise consists of individual points 12a.

[0083] A common voxel map 23, in which the first sensor data 11 of FIG. 2 and the second sensor data 12 of FIG. 3 are illustrated, is shown in FIG. 4. The common voxel map 23 with voxels 23a comprises 16 columns and 16 rows. A voxel 23a was defined unambiguously by specifying the line/column.

[0084] The first sensor data 11 and the second sensor data 12, as they are shown in FIGS. 2 and 3, have no spatial relation to one another. A spatial relationship to one another is established only by the assignment of the points to voxels and then the merging in a common voxel map 23.

[0085] For the sake of a simplified illustration, the first sensor data 11, the second sensor data 12, and the common voxel map 23 as well as the third sensor data 28, which are then created therefrom (see FIG. 7), are illustrated in a two-dimensional manner, even though it goes without saying that they are typically present in a three-dimensional manner.

[0086] The points in the common voxel map 23 in the voxels 23a at the positions 5/8, 6/4, 10/7, 10/8, 10/9, 11/6, 11/10, 12/11 are points 11a of the first point cloud, while the points 12a of the second sensor data 12 are entered in the voxels 23a at the positions 6/11, 6/12, 6/13, 7/10, 8/9, 9/2, 10/15.

[0087] In FIG. 5, the voxel map of FIG. 4 is shown, after some points have already been identified as noise points 24. This relates to the points 9/2, 6/4, 5/8, and 10/15. These noise points 24 are classified as irrelevant points 27.

[0088] It can furthermore be seen in FIG. 5, how points 11a of the first sensor data and points 12a of the second sensor data create masked regions 29. This is so because all voxels 23a behind the points 11a at the positions 11/6, 10/7, and 10/8 are masked. The term “behind” refers to being arranged radially behind it, which, in the voxel map 23, means that the corresponding voxels 23a have a higher line number.

[0089] Analogously, the points 12a of the second sensor data 12 in the voxels 23a at the positions 8/9, 7/10, 6/11, 6/12, and 6/13 create a corresponding, masked region 29. Three points 11a of the first sensor data 11 lie in the masked region 29 and are classified as masked points 25. These are the points 11a in the voxels 23a at the positions 10/9, 11/10, 12/11.

[0090] FIG. 6 shows the voxel map of FIG. 5, after the masked points 25 have been classified as not being relevant. The remaining points 11a of the first sensor data 11 and the remaining points 12a of the second sensor data 12 are further classified as being relevant points 26.

[0091] A voxel map with the third sensor data 28, which results by merging the points 26, which are classified as being relevant, of the first sensor data 11 and the points 26, which are classified as being relevant, of the second sensor data 12, is shown in FIG. 7. The third sensor data 28 likewise represent a point cloud with corresponding points 28a, namely with the points 28a in the voxels 23a at the positions 6/11, 6/12, 6/13, 7/10, 8/9, 10/7, 10/8, and 11/6.

[0092] FIG. 8 shows a device 10 according to the disclosure, which can comprise a unit 13 for detecting the first sensor data 11 and the second sensor data 12. The device 10 further comprises a classification unit 15 and a unit 14 for merging the first sensor data 11 and the second sensor data 12.

[0093] The classification unit 15 can comprise a voxelizer 16, a free space identification unit 19, an object detection unit 17, an object merging unit 18, and a noise point identification unit 20, as well as a point cloud marker 21. The voxelizer 16 serves the purpose of representing the first sensor data 11 and the second sensor data 12 in a common voxel map 23, wherein the free space identification unit 19 classifies free voxels as free spaces. The object detection unit 17 serves the purpose of detecting objects and the object merging unit 18 serves the purpose of putting them in spatial relation to one another in order to detect possible masking and/or overlaps. The noise point identification unit 20 serves the purpose of identifying noise points 24. The point cloud marker 21 serves the purpose of classifying all points of the first sensor data 11 and of the second sensor data 12 as being relevant or not relevant. Only relevant points 26 are subsequently merged to form third sensor data 28 by means of the unit 14 for merging.

[0094] A schematic diagram of the method 100, integrated into the device 10 according to FIG. 8, is shown in FIG. 9. The device 10 thereby does not have several illustrated units, such as, for example, voxelizers or object detection units, but it is only illustrated schematically in FIG. 9, which paths the sensor data cover within a device 10.

[0095] As input, first sensor data 11 and second sensor data 12 are provided, which are fed to the classification unit 15, namely first to the voxelizer 16 and subsequently to the free space identification unit 19, in order to represent the first sensor data 11 and the second sensor data 12 in a common voxel map 23 and to identified free voxels as free spaces.

[0096] The first sensor data 11 and the second sensor data 12 are further fed to the object detection unit 17 and subsequently to the object merging unit 18, in order to compare the positions of the detected objects to one another and in order to detect possible overlaps and/or masking.

[0097] The first sensor data and the second sensor data are additionally fed to the noise point identification unit 20, which can identify noise points 24.

[0098] The point cloud marker 21 can subsequently mark the noise points and identified points, which are masked or which are otherwise classified as not being relevant, as not being relevant, while the remaining points are marked as being relevant. The relevant classified points 26 are fed to the unit 14 for merging the first sensor data 11 and the second sensor data 12, which merges them to form third sensor data 28.