COMPUTER-IMPLEMENTED METHOD FOR LOW-LATENCY GENERATION AND INPUT OF SENSOR DATA INTO A CONTROL UNIT OR INTO CONTROL UNIT NETWORKS

Abstract

A method for low-latency generation and input of sensor data for control units or control unit networks includes: computing, by a simulator system or device, environmental data sets at a simulation frequency using an environment model of a virtual environment scene, wherein each environmental data set is associated with a respective point in time in an overall simulation time at which point in time the respective environment data set is available for further processing; and executing, by the simulator system or device or by at least one additional system or device, at least one sensor simulation model, wherein the at least one sensor simulation model generates sensor data sets at a sampling frequency, based on the environmental data sets, for further processing by at least one control unit, wherein each sensor data set is associated with a respective start time and respective time of completion in the overall simulation time.

Claims

1. A method for low-latency generation and input of sensor data for control units or control unit networks, comprising: computing, by a simulator system or device, environmental data sets at a simulation frequency using an environment model of a virtual environment scene, wherein each environmental data set is associated with a respective point in time in an overall simulation time at which point in time the respective environment data set is available for further processing; and executing, by the simulator system or device or by at least one additional system or device, at least one sensor simulation model, wherein the at least one sensor simulation model generates sensor data sets at a sampling frequency, based on the environmental data sets, for further processing by at least one control unit, wherein each sensor data set is associated with a respective start time and a respective time of completion in the overall simulation time, and wherein a respective sensor data set is stored in an output memory of the simulator system or device or of the at least one additional system or device, while waiting for transmission to the at least one control unit for further processing for a time interval between the respective time of completion and a respective time of transmission for the respective sensor data set, and wherein the respective time of transmission is associated with a point in time in the overall simulation; wherein the method further comprises: in each computation cycle in the overall simulation time, measuring and reducing or increasing a time interval between time of completion and time of transmission relative to a previous time interval between time of completion and time of transmission.

2. The method according to claim 1, wherein a respective time interval between time of completion and time of transmission is reduced relative to a respective previous time interval between time of completion and time of transmission by using, for generation of a respective sensor data set, a respective environmental data set whose point in time in the overall simulation time is closest to a start time of the generation of the respective sensor data set; and wherein the reduced time interval is greater than a risk buffer.

3. The method according to claim 2, wherein the risk buffer is adjustable by a user or provided through software.

4. The method according to claim 1, wherein a respective computation cycle in the overall simulation time has a fixed cycle length in which a sensor data set is generated based on environmental data sets and transmitted to a control unit that performs further processing.

5. The method according to claim 1, wherein the sensor data sets include radar data, lidar data, or camera image data.

6. The method according to claim 1, wherein sensor data of a previous computation cycle in the overall simulation time is transmitted based on a time of transmission being before a time of completion in a current computation cycle.

7. The method according to claim 1, wherein a step size for reducing the time interval is adjustable by a user or provided by the at least one additional system or device through software.

8. One or more non-transitory computer-readable mediums having processor-executable instructions stored thereon for low-latency generation and input of sensor data for control units or control unit networks, wherein the processor-executable instructions, when executed, facilitate: computing environmental data sets at a simulation frequency using an environment model of a virtual environment scene, wherein each environmental data set is associated with a respective point in time in an overall simulation time at which point in time the respective environment data set is available for further processing; and executing at least one sensor simulation model, wherein the at least one sensor simulation model generates sensor data sets at a sampling frequency, based on the environmental data sets, for further processing by at least one control unit, wherein each sensor data set is associated with a respective start time and a respective time of completion in the overall simulation time, and wherein a respective sensor data set is stored in an output memory while waiting for transmission to the at least one control unit for further processing for a time interval between the respective time of completion and a respective time of transmission for the respective sensor data set, and wherein the respective time of transmission is associated with a point in time in the overall simulation; wherein the processor-executable instructions, when executed, further facilitate: in each computation cycle in the overall simulation time, measuring and reducing or increasing a time interval between time of completion and time of transmission relative to a previous time interval between time of completion and time of transmission.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Subject matter of the present disclosure will be described in even greater detail below based on the exemplary figures. All features described and/or illustrated herein can be used alone or combined in different combinations. The features and advantages of various embodiments will become apparent by reading the following detailed description with reference to the attached drawings, which illustrate the following:

[0014] FIG. 1 illustrates a basic sequence of sensor data generation in an exemplary embodiment of the invention;

[0015] FIG. 2 shows a detailed sequence of sensor data generation in an exemplary embodiment of the invention;

[0016] FIG. 3 shows an example of a configuration of processing units in an exemplary embodiment of the invention; and

[0017] FIG. 4 shows another exemplary embodiment of a sequence of sensor data generation.

DETAILED DESCRIPTION

[0018] In an exemplary embodiment, the invention provides a computer-implemented method for low-latency generation and input of sensor data into a control unit or into control unit networks. In another exemplary embodiment, the invention provides a computer program product and a computer-readable storage medium containing instructions which, when executed by a computer, cause the computer to execute a method in accordance with an exemplary embodiment of the invention.

[0019] In an exemplary embodiment, the invention provides a method which provides for continuously executing an environment model of a virtual environment scene at a simulation frequency on a simulator processing unit, and for obtaining environmental data sets as output from the environment model. The simulation frequency for the environmental data should be greater than or equal to the highest sampling frequency of at least one sensor simulation model. The sensor simulation models output sensor data of a nature that is expected by the respective sensor to be served and models the environmental data as the respective sensor would perceive the virtual environment scene. The simulator processing unit (which may also be referred to as a simulator system or device) may be a specialized HIL simulator system having a real-time operating system, but may also be a commercially available computer.

[0020] The environment model of a virtual environment scene includes all the geometric information of the environmental objects present in the surroundings relative to the position of the control unit(s) to be tested, such as information about the roadway, including guardrails and roadside trees, obstacles, and vehicles creating a traffic situation. The environment model is executed periodically continuously at a simulation frequency, and thus generates a sequence of changing discrete states of the environment model, which are reflected in the environmental data sets. The sequence of environmental data sets follow one another in succession along the overall simulation time, which is the time that forms the basis for all computations. Accordingly, each individual environmental data set is associated with a point in time in the overall simulation time.

[0021] The generated environmental data sets are executed in at least one sensor simulation model on the simulator processing unit or on at least one additional processing unit, and the at least one sensor simulation model continuously generates sensor data at a sampling frequency, based on the environmental data sets, for further processing by a control unit. In this process, each generated sensor data set is associated with a start time at which the generation begins and a time of completion at which the generation is completed in the respective computation cycle in the overall simulation time. The sensor simulation models output sensor data of a nature that is expected by the respective sensor to be served and models the environmental data as the respective sensor would perceive the virtual environment scene.

[0022] The generated sensor data is stored in the output memory of the simulator processing unit or on at least one additional processing unit (which may also be referred to as at least one additional system or device) for a time interval until the generated sensor data is transmitted to a processing unit that performs further processing (which may be the simulator processing unit or the at least one additional processing unit or another system or device).

[0023] In order to keep the time interval as short as possible and thereby provide the processing unit that performs further processing with sensor data based on the most up-to-date environmental data possible, the time interval is gradually reduced with each computation cycle. The gradual reduction is performed until only a risk buffer remains as the time interval. If no new sensor data should be available at the time of transmission, the data from the previous computation cycle is transmitted. In this case, the time interval is then gradually increased in the next computation cycle to ensure the generation of up-to-date sensor data.

[0024] Exemplary embodiments of the present invention achieve low-latency generation and input of sensor data into a control unit or into control unit networks.

[0025] As indicated above, an advantageous embodiment provides for gradually reducing the time interval between the time of completion and the time of transmission of the sensor data generated based on environmental data. The gradual reduction is performed until only a risk buffer remains as the time interval.

[0026] In another preferred embodiment of the invention, the size of the risk buffer is adjustable by a user. This makes it possible to adjust the extent to which low-latency input of sensor data into a processing unit that performs further processing becomes more relevant, or to reduce the likelihood that no new sensor data is available in a computation cycle and data from the previous computation cycle must be transmitted.

[0027] In a further advantageous embodiment of the method, the time interval between the completion of the generated sensor data and the transmission of the sensor data is varied for testing purposes. For testing, accurate simulation of the sensor data is advantageous. This includes, for example, the computation of the virtual environment scene in a form expected by the sensor control unit. In this process, the sensor data is computed from the virtual environment scene using sensor simulation models, as mentioned earlier.

[0028] In order to enable test scenarios with small latencies between the completion of the sensor data and the transmission of the sensor data and with large latencies between the completion of the sensor data and the transmission of the sensor data in the following computation cycle, the time interval should be configurable in each computation cycle. This makes it possible to test thresholds of the respective latencies and effects for the control unit to be tested.

[0029] In another advantageous embodiment of the invention, the environment model is executed in response to a request by the sensor simulation model, and thus models only the relevant discrete state of the environment model, thereby making available the environmental data set that is utilized for the generation of the sensor data by the sensor simulation model at the desired point in time. This approach also has the advantage of preventing unnecessary transmission of environmental data sets that cannot be processed.

[0030] The transmission of environmental data sets may involve a high data-processing load, especially if the generation of environmental data sets is executed on a different computer system than the sensor simulation models.

[0031] The invention will now be described in more detail with reference to the drawings, in which like elements are designated by identical reference numerals. The embodiments shown are highly schematic.

[0032] FIG. 1 shows a basic sequence of sensor data generation in an exemplary embodiment of the invention. More particularly, FIG. 1 shows a timing diagram of the overall simulation time, which includes a plurality of computation cycles (8), each ending with a time of transmission (st).

[0033] FIG. 2 shows a detailed sequence of sensor data generation in an exemplary embodiment of the invention. In this process, an environment model (1) of a virtual environment scene is computed on a simulator processing unit, and environmental data sets (x,t) are continuously generated at the simulation frequency.

[0034] Sensor data (w) is generated based on an environmental data set (x,t) at a time (t) in the overall simulation. The computation (2) of sensor data (w) is completed at a time of completion (ft) in the overall simulation. The completed sensor data is then present in an output memory (m) of the simulator processing unit or of the at least one additional processing unit. The waiting time of sensor data (w) in output memory (m) is made up of the sum of time interval (wt) and risk buffer (r). A computation cycle (8) ends with the transmission of sensor data (w) at a time of transmission (st), which represents a point in time (t) in the overall simulation.

[0035] Then, in the next computation cycle (8), the computation (2) of sensor data (w) is delayed for a time (5), so that in the current computation cycle (8), a specific data set (x,t) of the continuously generated environmental data sets (x,t) that is generated at a later time (t) is used for the computation (2) of sensor data (w). This shortens time interval (wt) and reduces the latency for a processing unit that performs further processing.

[0036] This procedure is continued in the next computation cycle until only a risk buffer (r) remains as the waiting time in output memory (m) after the time of completion (ft) of sensor data (w).

[0037] FIG. 3 shows an example of a configuration of processing units in an exemplary embodiment of the invention. Here, an environment model of a virtual environment scene is computed on a simulator processing unit, and environmental data sets (x,t) are continuously generated at the simulation frequency.

[0038] The generation of sensor data (w) based on an environmental data set (x,t) is performed on another processing unit. Thus, this processing unit performs the steps of delaying (5) the computation of sensor data (w), generating (2) sensor data (w), storing sensor data (w) in output memory (m), and waiting the time interval (wt) and the time of risk buffer (r) until the time of transmission (st). At the time of transmission (st), sensor data (w) is transmitted to a processing unit that performs further processing.

[0039] Another embodiment of the invention is shown in FIG. 4. Here, initially, environmental data sets (x,t) are continuously generated at the simulation frequency in first computation cycle (8). This is followed by initiating the computation (2) of sensor data (w) based on an environmental data set (x,t). This results in a long time interval (wt) in the first computation cycle, as a result of which sensor data (w) remains longer in output memory (m) of the simulator processing unit or of the at least one additional processing unit. In the next computation cycle, a longer delay (5) of the computation (2) of sensor data (w) is seen. In order to change, for test cases, between large and small latencies for the processing unit that performs further processing, both the time interval (wt) and the risk buffer (r) can be adaptively varied, for example, by a user.

[0040] Limit values for the waiting time of sensor data (w) in output memory (m) can be examined, for example, by large changes between computation cycles (8). Relevant test cases can also be generated by the reduction of risk buffer (r) to zero.

[0041] While subject matter of the present disclosure has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. Any statement made herein characterizing the invention is also to be considered illustrative or exemplary and not restrictive as the invention is defined by the claims. It will be understood that changes and modifications may be made, by those of ordinary skill in the art, within the scope of the following claims, which may include any combination of features from different embodiments described above.

[0042] The terms used in the claims should be construed to have the broadest reasonable interpretation consistent with the foregoing description. For example, the use of the article “a” or “the” in introducing an element should not be interpreted as being exclusive of a plurality of elements. Likewise, the recitation of “or” should be interpreted as being inclusive, such that the recitation of “A or B” is not exclusive of “A and B,” unless it is clear from the context or the foregoing description that only one of A and B is intended. Further, the recitation of “at least one of A, B and C” should be interpreted as one or more of a group of elements consisting of A, B and C, and should not be interpreted as requiring at least one of each of the listed elements A, B and C, regardless of whether A, B and C are related as categories or otherwise. Moreover, the recitation of “A, B and/or C” or “at least one of A, B or C” should be interpreted as including any singular entity from the listed elements, e.g., A, any subset from the listed elements, e.g., A and B, or the entire list of elements A, B and C.

LIST OF REFERENCE CHARACTERS

[0043] 1 computation of an environmental data set (x,t) [0044] 2 computation of sensor data (w) [0045] 5 delay of the computation of sensor data (w) [0046] 8 computation cycle [0047] ft time of completion of the computation of sensor data (w) [0048] m output memory of the simulator processing unit or of at least one additional processing unit [0049] r risk buffer [0050] sst start time of the computation of sensor data (w) [0051] st time of transmission of sensor data (w) [0052] t point in time in the overall simulation [0053] w sensor data [0054] wt time interval that the sensor data waits in output memory (m) [0055] X,t environmental data set