METHOD FOR THE PREDICTION OF TRAJECTORIES FOR A VEHICLE

Abstract

A method for a trajectory prediction for a vehicle. The method includes receiving trajectory data of a travel trajectory of a further vehicle driving in a traffic lane within a surroundings of a vehicle; detecting at least one control action performed by the further vehicle based on the trajectory data, the control action representing at least part of a driving maneuver, by which the further vehicle is controlled along the travel trajectory; ascertaining a future travel trajectory of the vehicle by taking into account the detected control action executed by the further vehicle; and providing the future travel trajectory of the further vehicle.

Claims

1-15. (canceled)

16. A method for trajectory prediction for a vehicle, comprising the following steps: receiving trajectory data of a travel trajectory of a further vehicle driving in a traffic lane within a surroundings of the vehicle; detecting at least one control action performed by the further vehicle based on the trajectory data, the control action representing at least part of a driving maneuver by which the further vehicle is controlled along the travel trajectory; ascertaining a future travel trajectory of the further vehicle by taking into account the detected control action executed by the further vehicle; and providing the future travel trajectory of the further vehicle.

17. The method as recited in claim 16, wherein the ascertainment of the future travel trajectory is carried out by a trained artificial neural network, and wherein the detected at least one control action is used as input data for the artificial neural network.

18. The method as recited in claim 17, wherein the artificial neural network includes a fusion layer, the fusion layer being configured to take into account a plurality of control actions as a representation of complex driving maneuvers of a vehicle for ascertaining trajectories.

19. The method as recited in claim 17, wherein the at least one control action is detected from a plurality of previously known control actions, and wherein the previously known control actions are integrated as independent units into the neural network.

20. The method as recited in claim 19, wherein the artificial neural network is trained in accordance with a multi-task learning approach in that for each control action represented in a unit of the neural network, an ascertainment of a future trajectory is independently trained.

21. The method as recited in claim 17, wherein semantic information regarding the travel trajectory executed by the further vehicle is integrated into the neural network via the control actions.

22. The method as recited in claim 17, wherein the neural network is trained using a slice-based learning approach, in which selected data of respective training data sets are taken into account with heightened or lowered priority.

23. The method as recited in claim 16, wherein based on the detected at least one control action, a plurality of future travel trajectories is ascertained and/or provided, and the future travel trajectories are provided with reliability values.

24. The method as recited in claim 16, wherein the detection of the at least one control action is performed by user-defined determined detection functions.

25. The method as recited in claim 16, wherein each of the at least one control action describes and includes a general action of a vehicle including at least one of: acceleration, braking, straight-ahead driving, cornering, changing lanes to the right and to the left, turning off to the right and to the left, driving off, stopping.

26. The method as recited in claim 25, wherein each of the at least one control is adapted to actions of a vehicle in city traffic and/or in overland traffic, and to right-hand traffic and/or left-hand traffic.

27. The method as recited in claim 16, wherein the trajectory data include position data, speed data, acceleration data, steering angle data, and wherein the trajectory data are based on sensors of the further vehicle and/or on driving environment sensor data of the vehicle.

28. The method as recited in claim 16, wherein map data are taken into account for detecting the at least one control action, and wherein the map data includes a course of the traffic lane and/or a position of the traffic lane and/or a location of the traffic lane with respect to further traffic lanes, and/or wherein the ascertainment of the future travel trajectory is performed by taking the trajectory data into account.

29. A processing unit configured to predict a trajectory prediction for a vehicle, the processing unit configured to: receive trajectory data of a travel trajectory of a further vehicle driving in a traffic lane within a surroundings of the vehicle; detect at least one control action performed by the further vehicle based on the trajectory data, the control action representing at least part of a driving maneuver by which the further vehicle is controlled along the travel trajectory; ascertain a future travel trajectory of the further vehicle by taking into account the detected control action executed by the further vehicle; and provide the future travel trajectory of the further vehicle.

30. A non-transitory computer-readable medium on which is stored a computer program for trajectory prediction for a vehicle, the computer program, when executed by a data processing unit, causing the data processing unit to perform the following steps: receiving trajectory data of a travel trajectory of a further vehicle driving in a traffic lane within a surroundings of the vehicle; detecting at least one control action performed by the further vehicle based on the trajectory data, the control action representing at least part of a driving maneuver by which the further vehicle is controlled along the travel trajectory; ascertaining a future travel trajectory of the further vehicle by taking into account the detected control action executed by the further vehicle; and providing the future travel trajectory of the further vehicle.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Exemplary embodiments of the present invention are explained with reference to the figures.

[0041] FIG. 1 shows a schematic representation of a traffic situation of a driving assistance system according to one specific example embodiment of the present invention.

[0042] FIG. 2 shows a flow chart of a method for the trajectory prediction for a vehicle according to one specific example embodiment of the present invention.

[0043] FIG. 3 shows a schematic representation of an artificial neural network for carrying out the method for the trajectory prediction for a vehicle according to one specific example embodiment of the present invention.

[0044] FIG. 4 shows a schematic representation of a model for carrying out the method for the trajectory prediction for a vehicle according to one specific example embodiment of the present invention.

[0045] FIG. 5 shows a schematic representation of a computer program product, in accordance with an example embodiment of the present invention.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0046] FIG. 1 shows a schematic representation of a traffic situation.

[0047] FIG. 1 shows a traffic situation including a vehicle 200 and a further vehicle 201, which are respectively moved along a traffic lane 217. In the illustrated example, traffic lane 217 is developed as an intersection. This is merely by way of example and shall not limit the invention.

[0048] In the illustrated specific embodiment, vehicle 200 comprises a processing unit 300, which is configured to carry out the method according to the invention for predicting trajectories for a vehicle.

[0049] FIG. 1 shows a snap-shot of a traffic situation, in which the further vehicle 201 was moved along a traveled travel trajectory 203 along traffic lane 217 and is now, at the time of the illustrated traffic situation, located at the entrance of the crossing traffic lane 217. To move vehicle 201 along the traveled travel trajectory 203, the further vehicle 201 performed a plurality of past control actions 204. The past control actions 204 performed in temporal succession are designed here in such a way that by performing the past control actions 204 it was possible to move the further vehicle 201 along the traveled travel trajectory 203.

[0050] In the sense of the present application, control actions 204, 205, 206, 207, 208, 209, 210 are general actions of a vehicle, by which a vehicle is controllable along a predetermined travel trajectory. Control actions in this connection may comprise acceleration, braking, straight-ahead driving, cornering, lane changing both toward the right as well as toward the left, turning off, driving off or stopping. The listed examples of control actions are not conclusive and may additionally comprise further functions of the vehicle.

[0051] In the snap-shot of FIG. 1, the travel trajectory 203 traveled by the further vehicle 201 comprises further an executed control action 205, 206, 207. The executed control action 205, 206, 207 here represents the most recently executed control action of the traveled travel trajectory 203 at the time of the preparation of the snap-shot of the illustrated traffic situation.

[0052] For predicting the trajectory in accordance with the method of the invention, vehicle 200 first receives the trajectory data of the traveled travel trajectory 203 of the further vehicle 201. For this purpose, the trajectory data may be exchanged via the data communication units 221 of vehicles 200, 201 in accordance with a vehicle-to-vehicle communication between the further vehicle 201 and vehicle 200. Alternatively, the trajectory data may be based on driving environment sensor data of at least one driving environment sensor 219 of vehicle 200. The trajectory data may in this case comprise at least position data, speed data, acceleration data and steering angle data of the further vehicle 201.

[0053] On the basis of the trajectory data, at least one executed control action 205, 206, 207 is subsequently detected. According to the examples of control actions listed above, these may be executed simultaneously by a vehicle during the driving process in order to execute accordingly a complex driving maneuver. FIG. 1 shows an example of such a complex driving maneuver 215, which comprises a plurality of consecutively executed control actions 204, 206, 209, and which is executed in the illustrated exemplary representation as a left turn. For reasons of illustration, FIG. 1 shows only consecutively executed control actions. Alternatively, it is possible to execute at any time a plurality of control actions simultaneously and at the same time.

[0054] By taking into account the control action 205, 206, 207 executed by the further vehicle 201, a future travel trajectory 211, 212, 213 of the further vehicle 201 is subsequently ascertained. The future travel trajectory 211, 212, 213 is here configured in such a way that it represents an uninterrupted continuation of the most recently executed control actions 205, 206, 207.

[0055] FIG. 1 shows as the most recently executed control action 205, 206, 207 three alternative control actions, control action 205 representing straight-ahead driving of the further vehicle 201, control action 206 representing a left turn of the further vehicle 201 and control action 207 representing a right turn of further vehicle 201. In accordance with the three alternative control actions 205, 206, 207, a future travel trajectory 211 associated with control action 205 represents straight-ahead driving of vehicle 201, a future travel trajectory 211 associated with control action 206 represents a left turn of further vehicle 201 and a future travel trajectory 213 associated with control action 207 represents a right turn of further vehicle 201. The future travel trajectories 211, 212, 213 are to be executed by appropriate future control actions 208, 209, 210.

[0056] Following the ascertainment of the future travel trajectory, the travel trajectory is provided to vehicle 200 so that vehicle 200 may be appropriately controlled by taking the ascertained future travel trajectories 211, 212, 213 of further vehicle 201 into account.

[0057] In the illustrated example, only one future travel trajectory 211, 212, 213 is ascertained for each detected executed control action 205, 206, 207. Alternatively, it is possible to ascertain a plurality of alternative future travel trajectories 211, 212, 213 for each detected executed control action 205, 206, 207. In the illustrated example, the future travel trajectories 211, 212, 213 are shown merely as position information of the further vehicle 201. Alternatively, the future travel trajectories 211, 212, 213 may comprise, as indicated above, speed information, acceleration information and steering angle information of the further vehicle 201. A plurality of alternative future travel trajectories may vary in terms of the position data, acceleration data, speed data or steering angle data. When indicating a plurality of alternative future travel trajectories, the individual travel trajectories may be provided with corresponding reliability values, so that a control of vehicle 200 is able to take into account the plurality of indicated future travel trajectories 211, 212, 213 in accordance with the reliability values.

[0058] Apart from the actions of a vehicle 201, 202 indicated above, which are described by the control actions, the control actions may additionally be adapted to actions of a vehicle 201 in city traffic or in overland traffic and both to right-hand traffic as well as to left-hand traffic.

[0059] For detecting the executed control actions 205, 206, 207 based on the trajectory data of the traveled travel trajectory 203 of the further vehicle 201, it is possible to detect the control actions 205, 206, 207 from a plurality of previously known control actions by executing user-defined and determined detection functions. The determined detection functions may be trained or configured to detect, from a set of previously known control actions that describe a driving behavior of a vehicle, based on corresponding trajectory data, control actions actually executed by a vehicle in a driving situation observed via the trajectory data.

[0060] The alternatives shown in FIG. 1 of an executed control action 205, 206, 207 are intended to represent actual alternative situations. In the illustrated situation, the vehicle may execute one of the illustrated control actions 205, 206, 207, which is detected via the user-defined functions. The representation of the three alternative control actions 205, 206, 207 merely serves to illustrate the dependence of the detected control action 205, 206, 207 and of the prediction of the corresponding future travel trajectory 211, 212, 213 based on it. Different detected control actions 205, 206, 207 result in different predicted future travel trajectories 211, 212, 213. The control actions 205, 206, 207 represented as alternative control actions are here previously known to the user-defined detection functions, so that it is possible to detect, on the basis of the respective trajectory data, the corresponding control action 205, 206, 207 as present and therefore executed.

[0061] In the illustrated representation, the three alternative executed control actions 205, 206, 207, which are taken into account in determining the future travel trajectories 211, 212, 213, are represented as mutually distinguishable control actions. For reasons representability, the executed control actions 205, 206, 207 are oriented along the future descriptive travel trajectories 211, 212, 213. Control action 206 indicates a left-directed change in direction and control action 207 indicates a right-directed change in direction of vehicle 201, while control action 205 indicates straight-ahead driving. This is not to imply, however, that the executed control actions 205, 206, 207 detected by vehicle 200 comprise exclusively changes in direction of vehicle 201.

[0062] Additionally or alternatively, the executed control actions 205, 206, 207 detected by vehicle 200 may comprise control actions that are executed while the direction of vehicle 201 remains the same, for example braking actions or activations of a turn indicator. On the basis of these control actions, it is possible already prior to the initiation of a change of direction to infer corresponding driving maneuvers, for example turn-off processes.

[0063] According to one specific embodiment, the ascertainment of the future travel trajectory based on the detected executed control action 205, 206, 207 and the provision of the future travel trajectory 211, 212, 213 may be performed by an appropriately trained neural network. For this purpose, the executed control actions 205, 206, 207 detected via the user-defined and determined detection functions may be used as input data for the artificial neural network. For this purpose, the detected control actions may be represented in a numerical vector representation, and may be provided in the numerical vector representation as input data to the artificial neural network.

[0064] The artificial neural network may here comprise a fusion layer, which is designed to take into account a plurality of control actions as a representation of complex driving maneuvers 207 of a vehicle 200, 201 for ascertaining trajectories. The fusion layer allows for a vector representation of the previously known control actions or the detected control actions. The plurality of previously known control actions may for this purpose be integrated in independent units (batches) into the neural network, so that the independent units (batches) of the neural network respectively comprise exclusively data of individual previously known control actions.

[0065] In particular, the artificial neural network may be trained in accordance with a multi-task learning approach in that for each control action represented in a unit of the neural network the ascertainment of a future trajectory is trained. The individual units or batches of the artificial neural network, which respectively represent individual control actions, may thus be trained independently on respective training data of the artificial neural network to predict corresponding future travel trajectories by taking into account the respective control action of the unit or batch of the neural network. The multi-task learning approach thus makes it possible to determine a plurality of alternative future travel trajectories simultaneously.

[0066] When executing the appropriately trained artificial neural network for predicting trajectories, all future travel trajectories ascertained via the respective individual units or batches of the neural network, which respectively represent the individual previously known control actions, may subsequently be taken into account with a corresponding weighting. It is possible for example, to use only the future travel trajectories provided with a high weighting, for example with a high reliability value, for providing the future travel trajectories to the control of vehicle 200.

[0067] The artificial neural network may furthermore be trained using a slice-based learning approach, in which selected data of a respective training data set having a heightened or a lowered priority are taken into account for training the neural network. For example, driving situations that are represented in the respective training data set with a low frequency, such as for example drives in specific weather, drives at increased/reduced traffic volume, or other infrequently occurring driving situations, may be taken into account with an accordingly higher priority in the training of the neural network.

[0068] Besides the detected control actions, the trajectory data of the traveled travel trajectory 203 of the further vehicle 201 may also be used as input data for the artificial neural network for ascertaining future travel trajectories.

[0069] The integration of the previously known control actions makes it possible to integrate into the artificial neural network, in addition to the raw data of the trajectory data, also semantic information regarding driving maneuvers executed by the respective vehicle. The semantic information of the previously known control actions makes it possible to render the prediction of the future travel trajectories more precise in that the known control actions integrated into the artificial neural network define a bias function, which has the effect that future travel trajectories are preferred in the prediction that result in meaningful driving maneuvers of the vehicle in accordance with the detected executed control actions. This makes it possible to reduce the number of possible future travel trajectories to those travel trajectories that result in meaningful driving maneuvers, and that accordingly correspond with greater probability to a travel trajectory actually executed by the respective vehicle at a future point in time.

[0070] FIG. 2 shows a flow chart of a method 100 for the trajectory prediction for a vehicle 200 according to one specific embodiment.

[0071] The method 100 according to the invention for the trajectory prediction for a vehicle 200, 201 is applicable to the driving situation shown in FIG. 1 by way of example.

[0072] In a first method step 101, vehicle 200 initially receives trajectory data of a traveled travel trajectory 203 of a further vehicle 201 driving in traffic lane 217 within the surroundings of vehicle 200.

[0073] In a further method step 103, at least one control action 205, 206, 207 executed by the further vehicle 201 is detected on the basis of the trajectory data. The executed control action 205, 206, 207 in this case represents at least a portion of a driving maneuver 215 executed or to be executed by the further vehicle 201, by which the further vehicle 201 is controllable along the traveled travel trajectory 203 or along a future travel trajectory 211, 212, 213.

[0074] In a further method step 105, a corresponding future travel trajectory 211, 212, 213 is ascertained by taking the detected control actions 205, 206, 207 executed by the further vehicle into account. The future travel trajectory 211, 212, 213 in this case represents a steady continuation of the executed control actions 205, 206, 207 and of the corresponding driving maneuver 215.

[0075] Subsequently, in a further method step 107, the future travel trajectory 211, 212, 213 of the further vehicle 201 is provided.

[0076] FIG. 3 shows a schematic representation of an artificial neural network 504 for carrying out the method 100 for the trajectory prediction for a vehicle 200, 201 according to one specific embodiment.

[0077] FIG. 3 shows by way of example a structure of a neural network 504 according to one specific embodiment of the present invention. Neural network 504 comprises a plurality of lower layers 507. Above the lower layers 507, neural network 504 comprises the fusion layer 509. A plurality of separate and independent units or batches 510 are situated within fusion layer 509. The independent units 510 respectively include data of the previously known control actions, each unit 510 exclusively comprising one individual control action.

[0078] The structure of the neural network 504 is here designed in such a way that the lower layers 507 of neural network 504 or the data integrated there are jointly available to all layers. The separate units 510 of fusion layer 509, by contrast, are operated separately. This ensures the multi-task learning approach of neural network 504.

[0079] Jointly taking into account the individual control actions of the independent units or batches 510 within fusion layer 509 ensures a semantic representation of respective driving maneuvers of a vehicle in the prediction of future travel trajectories.

[0080] In the execution of the appropriately trained neural network 504, the individual units or batches 510 of the respectively independently represented control actions are activated in accordance with the existence of a respective control action in a driving situation of a vehicle observed on the basis of corresponding trajectory data and are accordingly prompted to predict a future travel trajectory. By contrast, control actions that are not present in the observed driving situation do not result in an activation of the respective unit 510 of neural network 504 and accordingly do not contribute to the prediction of a corresponding future travel trajectory.

[0081] FIG. 4 shows a schematic representation of a model for carrying out the method 100 for the trajectory prediction for a vehicle 200, 201 according to a specific embodiment.

[0082] FIG. 4 shows a schematic representation of a model 500 for carrying out the trajectory prediction. Via the defined and deterministic detection functions 501, corresponding control actions are detected on the basis of the trajectory data. The detected control actions are converted into corresponding numeric vector representations via a corresponding function for the numeric transformation 503. The totality of the corresponding vector representations of the previously known control actions forms a corresponding representation 505 of corresponding driving maneuvers comprising the respectively executed control actions. These vector representations of the driving maneuvers are combined with the data of the lower layers 507 of the neural network 504 in the fusion layer 509 and are arranged in linear blocks 511 of fusion layer 509. The data arranged in the linear blocks 511 may be rearranged in corresponding stack blocks 513 and may be used in accordance with the slice-based learning approach in slicing blocks 515 with corresponding priority for training. In the linear blocks 511, furthermore, a linear weighting of the driving maneuvers of representation 505 and of the data of lower layers 507 may be performed.

[0083] FIG. 5 shows a schematic representation of a computer program product 400.

[0084] FIG. 5 shows a computer program products 400, comprising commands, which, when the program is executed by a processing unit, prompt the latter to carry out the method 100 as recited in one of the aforementioned specific embodiments. In the illustrated specific embodiment, computer program product 400 is stored on a memory medium 401. The memory medium 301 may be any memory medium from the related art.