COMPUTER-IMPLEMENTED METHOD FOR AUTOMATICALLY PROVIDING AN ADVICE FOR TEST PROCESSES

Abstract

A computer-implemented method for the automatic provision of an Advice for test processes, wherein the Advice is determined by at least one Advice-generator and the Advice-generator for tests and/or simulations is selected manually and/or automatically, wherein the Advice-generator monitors at least two test runs, so that at least one event is detected in the test runs and at least one Advice is derived, wherein the Advice-generator is executed automatically by an Advice-generator mechanism during test runs and an Advice determined by the Advice-generator is provided to the test system and/or a user.

Claims

1. A computer-implemented method for an automatic provision of an Advice for test processes, the method comprising: determining the Advice by at least one Advice-generator; selecting the Advice-generator manually or automatically for tests or simulations; monitoring, via the Advice-generator, at least two test runs so that at least one event is detected in the test runs and at least one Advice is derived; automatically executing the Advice-generator by an Advice-generator mechanism during the test runs; and providing an Advice determined by the Advice-generator to the test system and/or a user.

2. The computer-implemented method according to claim 1, wherein an Advice is used for test processes at least in scenario-based testing and/or homologation and/or further test methods or in requirement-based testing and/or signal-based testing.

3. The computer-implemented method according to claim 2, wherein a scenario in scenario-based testing comprises at least one test of a device for the at least partially autonomous Advice of a transporter and/or road user.

4. The computer-implemented method according to claim 1, wherein at least one test and/or one simulation is determined by at least one parameter, wherein the at least one parameter includes: a scenario parameter comprising at least one of the features: the number of and/or the width of a lane and/or curves and/or road restrictions and/or the ambient temperature; or a driving situation parameter that describe the number and characteristics of moving objects in the scenario, comprising at least one of the features: the number of road users and/or the number of lane changes in a traffic situation and/or the speed of the road user and/or transporter.

5. The computer-implemented method according to claim 1, wherein the transporter and/or road users includes at least an Ego vehicle and/or a Fellow vehicle, wherein an Ego vehicle is the vehicle with the system under test (SUT) and a Fellow vehicle is any other vehicle.

6. The computer-implemented method according to claim 1, wherein an Advice-generator is designed as an Advice-generator plugin and an Advice-generator plugin has a defined interface to an Advice-generator plugin mechanism and an Advice-generator plugin comprises a configuration description and a script executable by a computer.

7. The computer-implemented method according to claim 6, wherein the Advice-generator plugins is selected dynamically and reusable for tests and/or simulations.

8. The computer-implemented method according to claim 1, wherein the Advice-generator mechanism is an Advice-generator plugin mechanism and contains at least one Advice-generator plugin, and wherein all Advice-generator plugins selected in the Advice-generator plugin mechanism are automatically executed for testing.

9. The computer-implemented method according to claim 1, wherein the Advice-generator plugins monitor test results and/or test data over at least two test runs of at least one test case and/or detect events in test results and/or test data and/or determine an Advice value.

10. The computer-implemented method according to claim 1, wherein an Advice-generator plugin determines an Advice value, and if in the configuration description of the Advice-generator plugin a threshold value for the Advice value is specified and if the Advice value exceeds the threshold value, the generation of an Advice is triggered.

11. The computer-implemented method according to claim 1, wherein the Advice specified according to an Advice value includes an Advice to a user and/or a termination of the test process and/or a modification of the test process by the test system, in particular by changing at least one parameter of a simulation and/or test.

12. The computer-implemented method according to claim 1, wherein an event detected by an Advice-generator plugin is identified in test results of at least two test runs and/or in test data of the at least one test run.

13. A test unit for an automatic provision of an Advice for test processes, wherein the Advice is determined by at least one Advice-generator and the Advice-generator for tests and/or simulations is selected manually and/or automatically, wherein the Advice-generator monitors at least two test runs, so that at least one event is identified in the test runs and at least one Advice is derived, wherein the Advice-generator is executed automatically by an Advice-generator mechanism during the test runs and an Advice determined by the Advice-generator is provided to the test system and/or a user.

14. A computer program comprising program code to perform the method according to claim 1 when the computer program is executed on a computer.

15. A computer-readable data carrier comprising program code of a computer program to perform the method according to claim 1 when the computer program is executed on a computer.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention will become more fully understood from the detailed description given hereinbelow and the accompanying drawings which are given by way of illustration only, and thus, are not limitive of the present invention, and wherein:

[0039] FIG. 1 is a schematic representation for the inventive differentiation between scenarios,

[0040] FIG. 2 is a schematic view that indicates a boundary between critical and non-critical test results,

[0041] FIG. 3 is an inventive representation of an Advice-generator plugin

[0042] FIG. 4 is a schematic representation for the inventive description of the Advice-generator plugin mechanism and the Advice-generator plugin,

[0043] FIG. 5 is a schematic representation for the inventive description of the use of Advice-generator plugins,

[0044] FIG. 6 is a schematic representation for the inventive description of the use of Advice-generator plugins

[0045] FIG. 7 shows a sequence of the inventive application of the Advice-generator plugin mechanism and Advice-generator plugins, and

[0046] FIG. 8 shows possible manifestations of an inventive Advice.

DETAILED DESCRIPTION

[0047] FIG. 1 describes two different scenarios S1 and S2. An intersection is displayed in each case. In both scenarios S1 and S2, an Ego vehicle (Ego) is depicted. In S1, the Ego vehicle (Ego) performs a turning maneuver. The Ego vehicle is also the “Subject under Test” or the “System under Test” (SUT). Four Fellow vehicles (F1 to F4) are involved. In S2, the Ego vehicle follows the vehicle course straight ahead without the participation of Fellow vehicles. Thus, there are differences in the scenario parameters as well as driving situation parameters. The goal in the scenarios can be, for example, the testing and simulation of adaptive cruise control.

[0048] The testing and simulation of such adaptive cruise control may require multiple test follow-ups in order to obtain a valid test result. The number of test runs yield many test results, so that relevant events can be shown over the course of the results. Relevant events can be detected via an Advice-generator plugin (API), as already presented. The Advice-generator plugin (API-1) mentioned can detect correlations in collision rates, such as: [0049] When at least 10 (N) test drives have been completed, examine the collision rate in relation to the simulated test kilometers driven. [0050] With at least an average of 2 (X) collisions per 1,000 km, give the user an Advice of an increased collision occurrence. [0051] In the event of at least 3 (Y) collisions per 1,000 km, alert the user to the increased collision occurrence. [0052] In the event of at least 4 (Z) collisions per 1,000 km, abort the test process with the scenario and start the analysis with the next test scenario. [0053] If all test drives with the first scenario were simulated and no (0, A) collision occurred, skip the analyses of the subsequent test scenarios.

[0054] With this Advice-generator plugin (API-1), for example, the variables N, X, Y, Z and A can be selected in the configuration file. Thus, different variables can be used for scenario S1 than scenario S2, but for both scenarios relevant relationships can be recognized and treated.

[0055] FIG. 2 shows a function that indicates a boundary between critical and non-critical test results. The items shown are simulated test results. Alternatively, these can also be approximated test results.

[0056] The function shown is the safety objective function, which has a numerical value that has a minimum value at a safety distance between the Ego vehicle (Ego) and the other motor vehicle, the Fellow vehicle, of ≥VFELLOW×0.55, in a collision between the Ego vehicle (Ego) and the other motor vehicle has a maximum value, and at a safety distance between the motor vehicle and the other motor vehicle of ≤VFELLOW×0.55 has a numerical value which is greater than the minimum value. Results for a safety objective function can be monitored by an Advice-generator (i.e., also by an Advice-generator plugin (API)) over several test runs, and events can be detected. For this purpose, the Advice-generator, i.e., also the Advice-generator plugin (API), is automatically executed during the test run by an Advice-generator mechanism (i.e., also by an Advice-generator plugin mechanism).

[0057] As an alternative to the safety objective function, for example, a comfort objective function or an efficiency objective function can be simulated and/or approximated, which has a numerical value with a minimum value in the event of no change in the acceleration of the motor vehicle, has a maximum value in the event of a collision between the Ego vehicle (Ego) and the other motor vehicle, and in the event of a change in the acceleration of the Ego vehicle (Ego) has a numerical value between the minimum value and the maximum value, depending on the amount of the change in acceleration. The majority of driving situation parameters, in particular the speed VEGO of the Ego vehicle (Ego) and the speed VFELLOW of the other motor vehicle, the Fellow vehicle, are generated within the given definition range, e.g., by means of a simulation.

[0058] For evaluation, variables can be defined in the configuration file of an Advice-generator plugin and also threshold values beginning with which an Advice is to be triggered. Accordingly, for example, an Advice can be triggered if an objective function has a high value over several test runs. The Advice is also configurable and can therefore be automatically executed for the test run.

[0059] FIG. 3 shows a representation of an Advice-generator plugin (API) according to the invention. Advice-generator plugins (APIs) have an interface to the Advice-generator plugin mechanism. In addition, an Advice-generator plugin (API) consists of a configuration file (C) and an executable script (S).

[0060] The configuration file (C) provides information for monitoring different variables in the test run results and/or the test data.

[0061] In addition, the configuration file (C) determines at which Advice value an Advice should be triggered and how the Advice is configured. For example, if a defined limit is exceeded, a warning can be issued to the user and/or an Advice can be given to the test system to terminate the current test process and/or optimize storage space so that test data is not lost. This list is not considered to be exhaustive, but merely serves to clarify the possibilities of an Advice-generator plugin (API) and an Advice. The executable script (S) contains the information for determining an Advice value, which in turn determines whether and which Advice should be provided to a user and/or a test system.

[0062] FIG. 4 shows a schematic representation for the inventive description of an Advice-generator plugin mechanism (API-M). The Advice-generator plugin mechanism (API-M) controls the automatic execution of the selected Advice-generator plugins (API). All Advice-generator plugins (APIs) have a defined interface to the Advice-generator plugin mechanism (API-M). Thus, in a preferred embodiment, before starting the test and/or simulation, the user will be able to easily select Advice-generator plugins (API) for the test without further configuring the interface. The interface also ensures a simple transfer of test results and/or other test data from the respective test run to the Advice-generator plugin mechanism (API-M) and then to the selected Advice-generator plugins (API). This ensures reusability of the Advice-generator plugins (API), and the automatic execution of the Advice-generator plugins (API) represents significant time and cost savings. In addition, the test process can be optimized by early detection of relevant events in test results and/or other test data and the provision of a corresponding Advice.

[0063] FIG. 5 shows a schematic representation for the inventive description of the use of Advice-generator plugins (API) for a concrete scenario and thus in a defined simulation/test.

[0064] For this purpose, FIG. 5 shows the Advice-generator plugin mechanism (API-M) with two Advice-generator plugins (API) API-1 and API-3. The Advice-generator plugin mechanism (API-M) is shown in conjunction with a test T1 in a scenario S1. The Advice-generator plugins (API) API-1 and API-3 are not directly linked to the scenario or the test. Their generic and dynamic definition allows for Advice-generator plugins (APIs) to be reused for various scenarios and tests. Therefore, no new Advice-generator plugin mechanism (API-M) or new Advice-generator plugin (API) is created for each new scenario or for a different test situation and/or simulation.

[0065] FIG. 6 also shows a further schematic representation for the inventive description of the use of Advice-generator plugins (API) for a concrete scenario and thus in a defined simulation/test.

[0066] Here, the Advice-generator plugin mechanism (API-M) is shown in conjunction with a test T2 in a scenario S2. FIG. 6 shows the Advice-generator plugin mechanism (API-M) with the Advice-generator plugins (API) API-2 and API-3. The Advice-generator plugins (APIs) are not directly linked to the scenario or test. It should be made clear that the Advice-generator plugin (API) API-3 can be used both in scenario S1 and test T1, as shown by FIG. 5, and in scenario S2 and thus in test T2, as shown in FIG. 6. This reuse occurs without modifying the corresponding Advice-generator plugin (API) and the associated script (S) of the Advice-generator plugin (API).

[0067] It is possible, for example, that in both the test situation shown in FIG. 5 and the test situation shown in FIG. 6, the Advice-generator plugin (API) API-3 monitors test framework conditions such as storage capacity of the storage medium which receives the test data in order to give an Advice to the test system if necessary, so that the test system can enable a further release of storage capacities. The Advice-generator plugin (API) API-1, for example, can recognize correlations in collision rates in the test results, as already presented. In addition, the Advice-generator plugin (API) API-2 can monitor the runtime of test runs in order to be able to determine cost estimates as an Advice for the user.

[0068] FIG. 7 shows the process of the inventive use of the Advice-generator plugin mechanism and Advice-generator plugins.

[0069] The Advice-generator plugin mechanism (API-M) receives test results from the test run (E-J), for example, for a test T1 which is executed for scenario S1. In addition, the Advice-generator plugin mechanism (API-M) can obtain additional test data (T-D) from at least one test case. This means that data from multiple test cases can also be made available to an Advice-generator plugin (API). All available information is collected and, if necessary, processed and automatically transmitted to the Advice-generator plugins (API) according to their respective configuration. Advice-Generator Plugin Mechanism (API-M) and Advice-Generator Plugin (API) have defined interfaces. The Advice-generator plugins (API) run automatically and determine an Advice value. An Advice is determined according to the Advice value and the corresponding configuration of the Advice-generator plugin (API). The Advice is implemented by the Advice-generator plugin mechanism (API-M), such as a configuration change (C-J) for future test runs.

[0070] FIG. 8 shows possible manifestations of an Advice according to the invention. The executable script (S) of an Advice-generator plugin (API) determines an Advice value. In the configuration file (C) of the Advice-generator plugin (API), both the threshold value and the characteristics of the Advice are defined.

[0071] The Advice-generator plugin mechanism (API-M) controls the automatic execution of the Advice-generator plugins (API), for example, the Advice-generator plugins API-1 and API-3 shown in FIG. 8. Furthermore, the Advice-generator plugin mechanism (API-M) takes over the execution of the determined Advice.

[0072] One manifestation of an Advice can be an alert or output to the user (A) of the test system (TS). For example, the Advice-generator plugin API-3 can monitor the runtime and thus the cost of a test process and issue an alert if a limit specified in the configuration file (C) of the Advice-generator plugin (API) is exceeded.

[0073] In addition, the manifestation of the Advice can have an effect on the configuration (C-J) of new test runs. This allows for parameter configurations to be optimized for future test runs based on events in previous test runs. Even the termination of the running test process, so that no further configuration (C-J) of test runs takes place, can be transmitted as an Advice by the Advice-generator plugin mechanism (API-M).

[0074] In addition to an Advice for the current test process, the Advice can also refer to future test processes. For example, for scenario-based testing, the selection for scenarios to be tested can be determined by an Advice-generator plugin such as API-1 from relevant events in the test results of a test process and be output either to the test system (TS) for automatic execution or to the user (A) for manual confirmation. The configuration of the scenario (S-C) can also be affected by an Advice.

[0075] The embodiments of the manifestation of the Advice is not exhaustive and further manifestations are possible according to the inventive method.

[0076] The invention being thus described, it will be obvious that the same may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are to be included within the scope of the following claims.