GENERATION OF TEST DATA HAVING CONSISTENT INITIAL CONDITIONS FOR STIMULATING A CONTROL UNIT TO BE TESTED

Abstract

A method for processing raw data recorded from the real world with the aid of a sensor into test data for stimulating a control unit, including: providing recorded raw data from the real world; ascertaining real objects detected by the sensor from the raw data, and generating route data sets, which each describe a scene including images of these real objects at consecutive points in time. providing a library of synthetic objects; assigning synthetic objects to detected images of the real objects, replacing the detected images with synthetic objects; supplementing temporally consecutive route data sets with supplementary data sets before the first route data set; and generating test data by converting the route data sets supplemented by the supplementary data sets into raw data that would have been recorded in the real world by the sensor during the introductory period and the recording period.

Claims

1. A method for processing raw data recorded from the real world with the aid of a sensor into test data for stimulating a control unit to be tested, the method comprising: providing recorded raw data from the real world, which were recorded along a recording route from at least one portion of the surroundings of the recording route with the aid of a sensor for a recording period, and which include temporally consecutive data value sets which resulted from real objects detected by the sensor; ascertaining real objects detected by the sensor from the temporally consecutive data value sets and generating temporally consecutive route data sets, which each describe a scene, including images of these real objects at consecutive points in time; providing a library of synthetic objects; ascertaining an absolute velocity of the sensor along the recording route at point in time zero of the recording period and in an event that the absolute velocity of the sensor along the recording route at point in time zero is other than at point in time zero of the recording period, supplementing the temporally consecutive route data sets with temporally consecutive supplementary data sets before the first route data set in such a way that the sensor has an absolute velocity of zero at point in time zero of the introductory period, and the temporally consecutive supplementary data sets include synthetic objects which demonstrate a quasi-continuous temporal sequence of movements between temporally directly consecutive supplementary data sets, the sequence of movements quasi-continuously resulting in the images of the real objects of the first route data set at point in time zero of the recording period; and generating the test data for stimulating a control unit to be tested by converting the supplementary data sets into raw data which would have been recorded in the real world with the aid of the sensor during the introductory period if the sensor had detected the synthetic objects in the temporally consecutive supplementary data sets as real objects; and supplementing recorded raw data with the raw data obtained by converting the supplementary data sets before the first route data set at point in time zero of the recording period.

2. The method according to claim 1, wherein the route data sets, which each describe a scene including images of these real objects at consecutive points in time, are formed by temporally consecutive frames, which each contain a virtual representation of the scene.

3. The method according to claim 1, wherein the synthetic objects, which show, between temporally directly consecutive supplementary data sets, a quasi-continuous temporal sequence of movements, which quasi-continuously lead to the images of the real objects of the first route data set at point in time zero of the recording period, are selected in such a way that they are seen as being in a predetermined range of similarities with the images of the real objects to which they lead within the scope of an assessment with the aid of a predetermined metric or within the scope of an assessment by a correspondingly trained artificial neural network.

4. The method according to claim 3, wherein a synthetic object is seen as being within a predetermined range of similarities with the image of a real object, if the raw data which was recorded from a real object with the aid of a sensor correspond to the synthetic object within a predetermined range of similarity with the raw data which was recorded by the sensor from the real object to whose image the synthetic object leads.

5. The method according to claim 1, wherein the sensor includes a radar sensor, a LIDAR sensor, a camera, and/or a ultrasonic sensor.

6. The method according to claim 1, further comprising: testing a control unit using test data which were obtained according to one of the preceding claims; and repeating the method steps of claim 1 if the control unit demonstrates, during the application of the test data, that it does not accept the test data as raw data recorded in the real world.

7. The method according to claim 6, wherein, during the repetition of the method steps from FIG. 1, images of virtual objects are made more similar to the real objects corresponding to them.

8. The method according to claim 6, wherein, during the repetition of the method steps, images of virtual objects of this type are inserted which were previously not used, due to too small a size of the real objects corresponding to them.

9. The method according to claim 1, wherein the test data obtained is for stimulating a control unit to be tested.

10. A nonvolatile, computer-readable memory medium having commands stored thereon which, when executed on a processor, effectuate a method according to claim 1.

11. A non-volatile, computer-readable memory medium having test data stored thereon, which were obtained according to claim 1.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present invention will become more fully understood from the detailed description given hereinbelow and the accompanying drawings which are given by way of illustration only, and thus, are not limitive of the present invention, and wherein:

[0019] FIG. 1 schematically shows a flowchart of a method for processing raw data recorded from the real world with the aid of a sensor into test data for stimulating a control unit to be tested according to an example;

[0020] FIG. 2 schematically shows a flowchart of a method for processing raw data recorded from the real world with the aid of a sensor into test data for stimulating a control unit to be tested according to an example;

[0021] FIG. 3 shows a semantic description of an initial real traffic situation, which corresponds to the target situation of the introductory scenario;

[0022] FIG. 4 shows a semantic description of the layout of the road in the introductory scenario;

[0023] FIG. 5 shows a semantic description of dynamic objects in the introductory scenario; and

[0024] FIG. 6 shows a semantic description of the entire introductory scenario.

DETAILED DESCRIPTION

[0025] A flowchart of a method for processing raw data recorded from the real world with the aid of a sensor into test data for stimulating a control unit to be tested is schematically apparent from FIG. 1. This control unit may be, for example, a control unit which performs tasks in the area of “advanced driver assistance systems” (ADAS) and “autonomous driving” (AD) in a motor vehicle, and which, for example, intervenes in the steering and velocity control of the motor vehicle.

[0026] This method includes the following method steps:

[0027] In method step S1, previously recorded raw data are provided from the real world, which were recorded with the aid of a sensor for a certain recording period along a certain recording route from a portion of the surroundings of the recording route. The recording route was thus previously driven with a motor vehicle, which includes a corresponding sensor for detecting at least one portion of the surroundings of the recording route. In this way, temporally consecutive data value sets were generated, which have resulted on the basis of real objects detected by the sensor. It is, of course, possible and also preferred within the scope of the invention that the motor vehicle, with which the route was previously driven for the purpose of the recording, includes more than one sensor for detecting the surroundings of the recording route.

[0028] In method step S2, real objects detected by the sensor are subsequently ascertained from the temporally consecutive data value sets, and temporally consecutive route data sets are generated, which each describe a scene containing images of these real objects at consecutive points in time. These route data sets thus represent a reproduction of the surroundings detected while driving along the recording route.

[0029] In method step S3, a library of synthetic objects is then provided. These synthetic objects are, in principle, also images of real objects, but they are not a direct reproduction of a particular real object. If a synthetic object represents, for example, another motor vehicle, the library of synthetic objects will include only a finite number of synthetic objects of this type, each of which is to represent one motor vehicle. synthetic objects of this type typically each describe a class of motor vehicles, such as “compact car,” “station wagon,” sedan,” “SUV,” “minivan,” “truck.” In practical terms, the synthetic objects thus represent simplified images of the real objects. This applies similarly to synthetic objects which represent static objects, for example buildings or vegetation, in the surroundings of the recording route. The synthetic objects may be stored, in particular, as 3D objects, based on which a graphic engine may render a two-dimensional view of a given object from a arbitrary perspective in a data format corresponding to the recorded raw data.

[0030] In method step S4, the absolute velocity of the sensor along the recording route is subsequently ascertained at point in time zero of the recording period. In the event that the absolute velocity of the sensor along the recording route is other than zero at point in time zero of the recording period, the temporally consecutive route data sets are supplemented with temporally consecutive supplementary data sets for an introductory period before the first route data set in such a way that the sensor has an absolute velocity of zero at point in time zero of the introductory period, and the temporally consecutive supplementary data sets include synthetic objects which demonstrate, between directly consecutive supplementary data sets, a quasi-continuous temporal sequence of movements which quasi-continuously leads to the images of the real objects of the first route data set at point in time zero of the recording period. This is therefore the method step in the method described above, in which a plausible introductory scenario is created to avoid implausible states, which would not be accepted by the control unit to be tested later on.

[0031] In method step S5, the test data are then generated for stimulating a control unit to be tested by converting the supplementary data sets into raw data which would have been recorded in the real world during the introductory period with the aid of the sensor if the sensor had detected the synthetic objects as real objects in the temporally consecutive supplementary data sets. In step S6, the recorded raw data are then supplemented with the raw data obtained by converting the supplementary data sets before the first route data set at point in time zero of the recording period. In this way, implausible states during the testing of the control unit may be avoided, which would otherwise result in an aborting of the test of the control unit.

[0032] This ends the method currently under discussion for processing the raw data recorded from the real world with the aid of the sensor into test data for stimulating the control unit to be tested. The test data generated in this way may be used multiple times and also for different control units to be tested. In addition, different sets of test data may be generated by driving along different recording routes for the purpose of testing control units of this type. The invention thus makes it possible to generate different sets of test data, which are stored in a separate library and may be used as needed for testing control units. It is, of course, equally possible, as shown schematically in the flowchart illustrated in FIG. 2, to subsequently test a control unit after generating the test data directly in a method step S7.

[0033] A method of this type is explained as an example below, based on the creation of an introductory scenario for a recording of radar raw data.

[0034] FIG. 3 shows a semantic description of the traffic situation as present in the first route data set at point in time zero of the recording period of the raw data recording. Instead of the term “route data set,” the term “frame” will be used below. This situation in the first route data set or in the first frame, is the target situation, in which the introductory scenario is to be introduced as seamlessly as possible. To reconstruct the traffic scenario, not only the radar sensor but also other sensors may be used as support, for example a forward-directed camera, which also runs during the test drive. The semantic description of the target situation comprises a designation (label), a position, and a velocity for each object detected. The introductory scenario comprises the position of real vehicle 1, with which the route data sets were ascertained, images of other passenger cars 2, images of trucks 3, images of vegetation 4, images of lane boundaries 5, if the data do to originate exclusively from a radar sensor, and an image of a guardrail 6. For illustrative purposes, view cone 7 of radar sensor 8 is also drawn in FIG. 3, which is carried by vehicle 1 and was recorded together with the data.

[0035] To set up the introductory scenario in a first step, a virtual reconstruction of a road section 10 takes place, which matches the road in the target situation, i.e., including a corresponding number of lanes of a suitable width. This is apparent schematically from FIG. 4. The length of road section 10 should be selected in such a way that, in each case, enough room remains for all synthetic objects which are to lead to the images of the real objects of the first route data set at point in time zero of the recording period, in particular to such images which represent the dynamic objects and which move absolutely within the space of the scenario, A virtual test vehicle 9, also referred to as an ego vehicle, is positioned on the road at the transverse position, which corresponds to the position of real test vehicle 1 in the target scenario, and it then accelerates from zero to the target velocity, i.e., the velocity of the test vehicle in the target scenario (in the first route data set). Virtual test vehicle 9 is equipped with a virtual radar sensor 11 at the exact position where real radar sensor 8 was mounted in the test vehicle.

[0036] If virtual test vehicle 9 is adjusted to the target velocity, a suitable virtual object, which generates a similar radar image, is positioned at the same relative position for each dynamic object from the target situation, as illustrated schematically in FIG. 5. Each dynamic object is then moved to a starting position in the target scenario by backward extrapolation, illustrated by arrows in the figure, of the particular dynamic object, based on its absolute velocity. The extrapolation is continued until all initial positions are outside view cone 12 of virtual radar sensor 11.

[0037] If the relative velocity of a dynamic object is equal to zero or approximately equal to zero, this object is positioned just outside view cone 12 of virtual radar sensor 11 and provides a controller, which activates the object, with the relative target position of the object as a setpoint value.

[0038] The scenes from FIG. 4 and FIG. 5 are now combined into a scenario, as shown in FIG. 6, until all dynamic objects are at the correct target position. It is also apparent from FIG. 6 that, starting at the position where virtual test vehicle 9 was placed, the introductory scenario is now filled with static objects from the target scenario, so that, in the final frames of the introductory scenario, at least the most important static objects are already visible which are also visible in the first frame, i.e., in the first route data set, which corresponds to the target scenario, and no static objects appear out of nowhere. It is shown by way of example that, in the immediate surroundings of the target position of virtual test vehicle 9, which the latter occupies in the last frame of the introductory scenario, the introductory scenario is filled with synthetic counter-pieces of guardrail 6 and vegetation 4.

[0039] The introductory scenario is then simulated, this simulation is converted into a synthetic radar raw data recording, and the real recording is then appended in front thereof, so that they both merge with each other seamlessly. This is followed by a test phase, in which it is tested whether a control unit to be tested accepts the introductory scenario, i.e., whether the transition from the introductory scenario into the real recording takes place without an error message or unusual behavior on the part of the control unit.

[0040] If this is the case, the method is concluded. If not, it is improved. For example, virtual objects, i.e., static and/or dynamic objects, are structured in a similar way as their real counter-pieces from the recording, and/or further static objects are inserted, which were previously ignored due to their small radar cross-section. The following principle applies: Greater care is taken in the case of the objects which are of interest from the perspective of the particular control unit. For example, an algorithm of a lane keeping assistant will pay special attention to boundaries of the lane, such as edge lines, guardrails, curbs, etc., and other objects tend to be ignored, such as vehicles, pedestrians, and vegetation. Improvements continue to be made until the particular control unit accepts the introductory scenario.

[0041] The invention being thus described, it will be obvious that the same may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are to be included within the scope of the following claims.