METHOD AND COMPUTER SYSTEM FOR CONTROLLING THE MOVEMENT OF A HOST VEHICLE
20230286535 · 2023-09-14
Inventors
Cpc classification
B60W2552/53
PERFORMING OPERATIONS; TRANSPORTING
B60W60/0011
PERFORMING OPERATIONS; TRANSPORTING
B60W30/18163
PERFORMING OPERATIONS; TRANSPORTING
B60W60/001
PERFORMING OPERATIONS; TRANSPORTING
B60W50/0097
PERFORMING OPERATIONS; TRANSPORTING
B60W2556/50
PERFORMING OPERATIONS; TRANSPORTING
International classification
B60W60/00
PERFORMING OPERATIONS; TRANSPORTING
Abstract
A method is provided for controlling the movement of a host vehicle. Lane data related to road items are determined from a data source, and object data acquired from objects are determined via a sensor system. The road items and the objects are located in an external environment of the host vehicle. A machine-learning algorithm is fed with the lane data and the object data to generate a set of controlling functions for controlling the movement of the host vehicle by performing the steps of: classifying a predefined set of maneuvers for the host vehicle based on the lane data and the object data, and determining a set of controlling functions related to kinematics of the host vehicle. The movement of the host vehicle is controlled based on the classified maneuvers and the determined controlling functions.
Claims
1. A computer implemented method for controlling movement of a host vehicle, the method comprising: determining lane data related to road items from a data source and object data acquired via a sensor system from objects, wherein the road items and the objects are located in an external environment of the host vehicle, feeding a machine-learning algorithm with the lane data and the object data to generate a set of controlling functions for controlling the movement of the vehicle by performing the steps of: classifying a predefined set of maneuvers for the host vehicle based on the lane data and the object data, and determining a set of controlling functions related to kinematics of the host vehicle, and controlling the movement of the host vehicle based on the classified maneuvers and the determined controlling functions.
2. The method according to claim 1, wherein the step of determining the set of controlling functions includes: predicting control parameters of an acceleration and of a steering angle for the host vehicle.
3. The method according to claim 1 wherein the step of determining the set of controlling functions includes: predicting parameters of a planned curvature for the host vehicle.
4. The method according to claim 1, wherein the data source for determining the lane data includes the sensor system and/or a predetermined map.
5. The method according to claim 1, wherein the set of controlling functions includes continuous functions in time.
6. The method according to claim 1, wherein for training the machine-learning algorithm, common ground truth data is applied to classify the predefined set of maneuvers and to determine the controlling functions.
7. The method according to claim 6, wherein the common ground truth data is acquired as data recorded for the host vehicle over a predetermined time period.
8. The method according to claim 7, wherein: additional pre-classified data is generated by pre-classifying the predefined set of maneuvers using the data recorded for the host vehicle, and the additional pre-classified data is applied for the training of determining the controlling functions.
9. The method according to claim 1, wherein the road items include lane markings, road boundaries and/or guard rails.
10. The method according to claim 1, wherein the object data include predefined physical properties of the objects.
11. The method according to claim 1, wherein the predefined set of maneuvers for the host vehicle includes a lane change to the left, keeping the lane and a lane change to the right as lateral maneuvers, and acceleration, keeping speed and deceleration as longitudinal maneuvers.
12. A computer system, the computer system being communicatively coupled to a sensor system for providing object data acquired from objects, wherein the computer system is provided with lane data related to road items and determined from a data source, wherein the road items and the objects are located in an external environment of a host vehicle, wherein the computer system comprises a machine-learning algorithm which is fed with the lane data and the object data to generate a set of controlling functions for controlling the movement of the vehicle by: a classification decoder classifying the predefined set of maneuvers for the host vehicle based on the lane data and the object data, and a regression decoder determining the set of controlling functions related to kinematics of the host vehicle, and wherein the computer system comprises a vehicle control unit configured to control the movement of the host vehicle based on the classified maneuvers and the determined controlling functions.
13. The computer system according to claim 12, wherein the machine-learning algorithm includes an encoder which is used when classifying the predefined set of maneuvers and determining the set of controlling functions.
14. A vehicle comprising the computer system of claim 12.
15. Non-transitory computer readable medium comprising instructions for carrying out the computer implemented method of claim 1.
Description
DRAWINGS
[0041] Exemplary embodiments and functions of the present disclosure are described herein in conjunction with the following drawings, showing schematically:
[0042]
[0043]
[0044]
[0045]
[0046]
[0047]
[0048]
DETAILED DESCRIPTION
[0049]
[0050] An arrow 140 indicates the present driving direction and the present speed of the host vehicle 110. According to the related art, a future trajectory of the host vehicle 110 may be predicted, for example, based on given physical properties (like position, velocity, and/or acceleration) of the host vehicle 110 and of further objects located in the external environment of the host vehicle 110 including the further vehicle 120, for example. The future trajectory of the host vehicle 110 may also be influenced by its static context which may for example include the course of the lanes 130, 131, or the like. For example, the vehicle 110 may perform a breaking maneuver for keeping a safe distance to the further vehicle 120, or perform a lane change maneuver to the left (like indicated by arrow 150).
[0051] The lane markings 132 and other similar road structures like the guard rails, the road boundaries 135, 136, barriers, are the most important road items which may influence the road users' behavior regarding the future trajectory of the host vehicle 110. Such road items 132, 135, 136 may at least partly represent the traffic rules and may regulate the road user's individual kind of driving behavior into systematic patterns. But even for a relatively simple highway road, modeling such a behavior of road users within their driving environment may already not be easy.
[0052] A prediction of the future movements of the host vehicle 110 is a necessary component for many of the vehicle's safety features and for autonomous driving.
[0053]
[0054] As an input, the encoder 210 receives lane data 211 related to road items, and object data 212 related to properties of objects in the environment of the host vehicle 110. The lane data 211 describes the position of the lane markings 132 (see
[0055] The lane data 211 related to the road items like the lane markings 132 and/or the road boundaries 135, 136 is determined via a sensor system which is installed in the host vehicle 110. The sensor system includes a visual sensor. Additionally or alternatively, the sensor system may include a radar sensor and/or a Lidar sensor. For an alternative embodiment, the lane data 211 related to the road items may be acquired from a map, wherein map data may be stored in a data base of the vehicle 110.
[0056] The lane data 211 is represented as a third order polynomial depending from time. That is, the y-coordinate of a position of a respective road item may be represented by a third order polynomial with the x-coordinate as the only variable, wherein the x- and y-coordinates refer to a vehicle coordinate system having the x-axis along the longitudinal axis of the vehicle 110 and the y-axis along the lateral axis of the vehicle 110.
[0057] The lane data 211 also includes the specific type of the road items which are monitored by the sensor system of the vehicle 110, and in addition predetermined ranges for the validity of the lane data 211. In addition or as alternative to the road boundaries 135, 136, the position of guard rails may be observed and tracked by the sensor system of the vehicle 110. The position of the guard rails may also be included in the lane data 211. However, the lane data 211 generally describes the course of the lane 130 in which the vehicle 110 is currently moving, and optionally the course of neighbouring lanes like the lane 131 (see
[0058] The object data 212 is also detected or determined via the sensor system of the vehicle 110, i.e. via a visual sensor, a radar sensor and/or a Lidar sensor. The object data 212 includes at least the position of objects located in the external environment of the host vehicle 110. Depending from the specific sensor which is applied to acquire the object data, the object data 212 may further include the velocity, the heading angle and further physical properties of the objects like the further vehicle 120 (see
[0059] The computer system 200 feeds a machine-learning algorithm with the lane data 211 and the object data 212 in order to perform a classification task of classifying a predefined set of maneuvers for the host vehicle 110, and for performing a regression task of determining continuous functions related to the kinematics of the host vehicle 110. The machine-learning algorithm is realized as a neural network which includes the encoder 210 as well as the classification and regression decoders 220, 230.
[0060] The classification task and the regression task both use the common encoder 210 which receives the lane data 211 and the object data 212 as an input. Furthermore, the classification task is performed via the classification decoder 220, whereas the regression task is performed via the regression decoder 230. The output of the classification decoder 220 includes respective probabilities 225 for each of a predefined set of maneuvers which are to be applied for controlling the host vehicle 110.
[0061] For the present embodiment, the predefined set of maneuvers for the host vehicle 110 includes a lane change to the left, keeping the lane and a lane change to the right as lateral maneuvers as well as acceleration, keeping speed and deceleration as longitudinal maneuvers. That is, a predefined set of six maneuvers is used by the classification decoder 220 in order to perform the classification task which outputs a respective probability for each of these maneuvers as a function in time for a predetermined time period which is related to the future movement of the vehicle 110.
[0062] When performing the regression task, the regression decoder 230 outputs continuous controlling functions 235 in time for kinematic parameters of the host vehicle 110. For the present embodiment, these kinematic parameters are the acceleration and the steering angle of the host vehicle 110. Therefore, the regression decoder 230 outputs a respective continuous controlling function 235 in time for the acceleration and for the steering angle, respectively, in order to control the future movement of the host vehicle 110.
[0063] The output of the classification decoder 220, i.e. the probabilities 225 for the vehicle maneuvers, and the output of the regression decoder 230, i.e. the continuous functions 235 for the future acceleration and steering angle, are provided to a vehicle control unit 240 which applies the output 225, 235 of the decoders, i.e. the probabilities 225 and the continuous functions 235, in order to control the future trajectory of the host vehicle 110. As will be explained below in context of
[0064] In detail, the output of the classification decoder 220, i.e. the probabilities 225 for the vehicle maneuvers, may be used in different ways by the regression decoder 230 and/or the vehicle control unit 240. For example, the most probable longitudinal and lateral maneuvers, having the highest respective probability 225 for a certain point in time, may be selected and used as an input for the regression decoder 230. Alternatively, the respective most probable maneuvers may be compared with an external input for the maneuver to be performed, e.g. provided by a driver of the vehicle, in order to decide which maneuver may be used as an input for the regression decoder 230 and/or the vehicle control unit 240.
[0065] In addition, the continuous controlling functions 235 for the acceleration and for the steering angle can optionally be used for a visualization 250 of the host vehicle's future trajectory. For such a visualization 250, a kinematic bicycle model for the vehicle is applied which is generally known in the art. In
[0066] In detail, the continuous functions 235 for the acceleration a and the steering angle δ for the future time t are represented by a respective second order polynomial as follows:
α(t)=α.sub.αt.sup.2+b.sub.αt+α.sub.0 (1)
δ(t)=α.sub.δt.sup.2+b.sub.δt+δ.sub.0 (2)
[0067] The coefficients a.sub.δ, b.sub.δ and a.sub.δ, b.sub.δ are the output of the regression decoder 230 for determining the continuous controlling functions 235. The constants α.sub.0 and δ.sub.0 represent the acceleration and the steering angle at the current time, and therefore they are given parameters and no part of the regression task. As an alternative to the second order polynomials, polynomials of a higher order may be used to improve the smoothness of the continuous controlling functions 235.
[0068] For another embodiment, the future trajectory of the host vehicle 110 may be modelled directly as fifth order polynomial. In this case, x and y represent the longitudinal position and the lateral position, respectively, of the vehicle 110. The trajectory may be modelled as follows:
y(x)=a.sub.yx.sup.5+b.sub.yx.sup.4+c.sub.yx.sup.3+d.sub.yx.sup.2+e.sub.yx+f.sub.y (3)
[0069] In this case, the required coefficients a.sub.y, b.sub.y, c.sub.y, d.sub.y, e.sub.y and f.sub.y are output b.sub.y the regression decoder 230 in order to define the continuous controlling function 235 which is given by formula (3). In practice, f.sub.y can be ignored and set to 0 since the coordinate system in which x and y are defined usually has its origin at the center of gravity (or another reference position) of the host vehicle 110.
[0070] For a further embodiment, the curvature of the future trajectory may be used for a continuous controlling function 235 which is output by the regression decoder 230. The curvature of the future trajectory is defined as the second derivative of the trajectory as given e.g. by formula (3) and may be represented as follows:
y″(x)=20a.sub.yx.sup.3+12b.sub.yx.sup.2+6c.sub.yx+2d.sub.y (4)
[0071] To train the neural network which realizes the machine-learning algorithm for performing the classification task and the regression task, respective kinematic parameters of a real vehicle 110 are used as ground truth. The kinematic parameters, e.g. the acceleration and the steering angle for the present embodiment, are acquired for a certain predetermined time period. Simultaneously, corresponding lane data 211 and object data 212 are acquired and used as input for the encoder 210. The output 225 and 235 of the classification decoder 220 and the regression decoder 230, respectively, is compared with the acquired or logged data for the probabilities 225 of the respective vehicle maneuvers and for the continuous functions 235 of the acceleration and the steering angle in order to determine a respective loss function 226, 236 for the classification task and the regression task, respectively. During training, these loss functions 226, 236 have to be minimized.
[0072] For the training of the neural network, one has also to deal with a multi-modality problem if data of a real vehicle 110 are used which are related to a real driver. Different drivers have different driving styles such that one driver will prefer following a slow leading vehicle, whereas another driver will change the lane and perform an overtaking maneuver. This will lead to great differences between the acquired or logged real data for the acceleration and the steering angle as a function in time. For considering such a multi-modality properly, the set of vehicle maneuvers includes the three longitudinal and the three lateral maneuvers only, as described above, i.e. acceleration, keep speed and deceleration for the longitudinal direction and lane change to the left, keeping the lane and lane change to the right for the lateral direction. For considering the multi-modality, the computer system 200 outputs the probabilities 225 for this set of six vehicle maneuvers such that the sum for the probabilities for the longitudinal maneuvers and the sum of probabilities for the lateral maneuvers will each add to 1, respectively, for a certain point in time.
[0073] In addition, a so-called “teacher forcing” is applied for the training of the neural network of the computer system 200. The “teacher forcing” uses specific training data with pre-classified longitudinal and lateral maneuvers, and this pre-classified data for the two groups of maneuvers is used as an additional input 260 for the regression decoder 230. The additional pre-classified data 260 is provided to the regression decoder 230 during the training of the neural network only. By this means, the training data or ground truth has one modality only for a certain point in time. This supports the convergence of the regression task which led to the term “teacher forcing” if such a technique is applied. The pre-classified data 260 is fed to the regression decoder 230 only in order not to disturb the classification task which is performed by the classification decoder 220.
[0074] The application of the additional pre-classified data 260 may also be regarded as asking the neural network to perform a certain maneuver and to predict the corresponding continuous functions 235 accordingly. That is, a separated and independent maneuver control model is provided as input for training the neural network. Alternatively, the predicted future maneuvers may be used as an output from the classification decoder 220, i.e. provided by the probabilities 225 for the respective maneuvers, which are based on real logged data for the vehicle 110.
[0075]
[0076] In detail, the steering angle in rad and the acceleration in m/s.sup.2 are predicted by the computer system 200, i.e. by the regression decoder 230 (see
[0077] For two different current points in time, the respective upper left diagrams of
[0078] At the same time, the classification decoder 220 outputs probabilities 225 for the three lateral maneuvers and for the three longitudinal maneuvers as described above. The probabilities 225 are provided as function in time by the classification decoder 220 for the next 5.6 seconds in the present example which corresponds to the “future time period” for which the steering angle and the acceleration are also predicted by the regression decoder 230.
[0079] In the respective lower diagrams of
[0080] For the situation as shown in
[0081] However, after about 1.5-2.0 s with respect to the current point in time of 0.0 as shown in
[0082] For the scenario as shown in
[0083] With respect to the current point in time which is shown at 0.0 in the lower part of
[0084] As can be seen in
[0085]
[0086] According to various embodiments, the step of determining the controlling functions may include predicting control parameters of an acceleration and of a steering angle for the host vehicle.
[0087] According to various embodiments, the step of determining the controlling functions may include predicting parameters of a planned curvature for the host vehicle.
[0088] According to various embodiments, the data source for determining the lane data may include the sensor system and/or a predetermined map.
[0089] According to various embodiments, the set of controlling functions includes continuous functions in time.
[0090] According to various embodiments, for training the machine-learning algorithm, common ground truth data may be applied to classify the predefined set of maneuvers and to determine the controlling functions.
[0091] According to various embodiments, the common ground truth data may be acquired as data recorded for the host vehicle over a predetermined time period.
[0092] According to various embodiments, additional pre-classified data may be generated by pre-classifying the predefined set of maneuvers using the data recorded for the host vehicle, and the additional pre-classified data may be applied for the training of determining the controlling functions.
[0093] According to various embodiments, the road items may include lane markings, road boundaries and/or guard rails.
[0094] According to various embodiments, the object data may include predefined physical properties of the objects.
[0095] According to various embodiments, the predefined set of maneuvers for the host vehicle may include a lane change to the left, keeping the lane and a lane change to the right as lateral maneuvers, and acceleration, keeping speed and deceleration as longitudinal maneuvers.
[0096] Each of the steps 502, 504, 506, and the further steps described above may be performed by computer hardware components.
[0097]
[0098] The data determination circuit 602 may be configured to determine lane data related to road items from a data source and object data acquired via a sensor system from objects. The road items and the objects may be located in an external environment of the host vehicle.
[0099] The circuit for the machine-learning algorithm 604 may be configured to feed the machine-learning algorithm with the lane data and the object data to generate a set of controlling functions for controlling the movement of the host vehicle b.sub.y performing the steps of: classifying a predefined set of maneuvers for the host vehicle based on the lane data and the object data, and determining a set of controlling functions related to kinematics of the host vehicle. For performing the above steps of classifying the predefined maneuvers and of determining the control functions, the circuit 604 may include the encoder 25 (see also
[0100] The movement controlling circuit 606 may be configured to control the movement of the host vehicle based on the classified maneuvers and the determined controlling functions.
[0101] The data determination circuit 602, the circuit for the machine-learning algorithm 604, and movement controlling circuit 606 may be coupled with each other, e.g. via an electrical connection 608, such as e.g. a cable or a computer bus or via any other suitable electrical connection to exchange electrical signals.
[0102] A “circuit” may be understood as any kind of a logic implementing entity, which may be special purpose circuitry or a processor executing a program stored in a memory, firmware, or any combination thereof.
[0103]
[0104] The processor 702 may carry out instructions provided in the memory 704. The non-transitory data storage 706 may store a computer program, including the instructions that may be transferred to the memory 704 and then executed by the processor 702.
[0105] The processor 702, the memory 704, and the non-transitory data storage 706 may be coupled with each other, e.g. via an electrical connection 708, such as e.g. a cable or a computer bus or via any other suitable electrical connection to exchange electrical signals. As such, the processor 702, the memory 704 and the non-transitory data storage 706 may represent the circuit for the machine-learning algorithm 604 as described above, i.e. including the common encoder 25, the decoder for classification 27 and the decoder for regression 29 for performing the steps of classifying the predefined maneuvers and of determining the controlling functions, respectively.
[0106] The terms “coupling” or “connection” are intended to include a direct “coupling” (for example via a physical link) or direct “connection” as well as an indirect “coupling” or indirect “connection” (for example via a logical link), respectively.
[0107] It will be understood that what has been described for one of the methods above may analogously hold true for the vehicle controlling system 600 and/or for the computer system 700.
REFERENCE NUMERAL LIST
[0108] 110 host vehicle [0109] 120 further preceding vehicle [0110] 130 current lane [0111] 131 neighboring left lane [0112] 132 lane markings [0113] 135, 136 road boundaries [0114] 140 arrow indicating a present speed of the host vehicle [0115] 150 arrow indicating a lane change to the left [0116] 200 computer system [0117] 210 encoder [0118] 211 lane data [0119] 212 object data [0120] 220 classification decoder [0121] 225 probabilities for vehicle maneuvers [0122] 226 loss function for the classification decoder [0123] 230 regression decoder [0124] 235 controlling functions [0125] 236 loss function for the regression decoder [0126] 240 vehicle control unit [0127] 250 trajectory visualization [0128] 260 pre-classified data [0129] 310 ground truth for steering angle [0130] 320 predicted steering angle [0131] 330 ground truth for acceleration [0132] 340 predicted acceleration [0133] 350 predicted trajectory [0134] 360 trajectory based on ground truth [0135] 410 ground truth for steering angle [0136] 420 predicted steering angle [0137] 430 ground truth for acceleration [0138] 440 predicted acceleration [0139] 450 predicted trajectory [0140] 460 trajectory based on ground truth [0141] 500 flow diagram illustrating a method for controlling the movement of a host vehicle according to various embodiments [0142] 502 step of determining data lane data related to road items from a data source, and object data acquired from objects via a sensor system [0143] 504 step of feeding a machine-learning algorithm with the lane data and the object data to generate a set of controlling functions for controlling the movement of the host vehicle be performing the steps of: classifying a predefined set of maneuvers for the host vehicle based on the lane data and the object data, and determining a set of controlling functions related to kinematics of the host vehicle [0144] 506 step of controlling the movement of the host vehicle based on the classified maneuvers and the determined controlling functions [0145] 600 vehicle controlling system [0146] 602 data determination circuit [0147] 604 circuit for machine-learning algorithm [0148] 606 movement controlling circuit [0149] 608 connection [0150] 700 computer system according to various embodiments [0151] 702 processor [0152] 704 memory [0153] 706 non-transitory data storage [0154] 708 connection