METHOD USED FOR DERIVING A CONTROL VARIABLE FOR LATERAL GUIDANCE OF A MOTOR VEHICLE

Abstract

A method used for deriving a control variable for lateral guidance of a motor vehicle by means of images of at least one camera of the motor vehicle, wherein the at least one camera captures images of a roadway. A more precise and/or resource-saving derivation of the control variable, and thus lateral guidance, of the motor vehicle is achieved by projecting a traffic lane to be followed as well as a travel trajectory of the motor vehicle onto the images, and by a control variable for lateral guidance of the motor vehicle being derived by comparing the traffic lane projected onto the images with the travel trajectory projected onto the images. A motor vehicle in which the method is performed is also described.

Claims

1. A method for deriving a control variable for lateral guidance of a motor vehicle using images of at least one camera of the motor vehicle, wherein the at least one camera captures images of a roadway, the method comprising the following steps: projecting a traffic lane of the roadway to be followed onto the images; determining a travel trajectory of the motor vehicle in reference to the images based on a self-motion estimate and projecting the determined travel trajectory onto the images; and deriving a control variable for lateral guidance of the motor vehicle from a comparison between the traffic lane projected onto the images and the determined travel trajectory projected onto the images.

2. The method according to claim 1, wherein the travel trajectory is determined using a single lane model and a ground plane estimate, the single lane model including an Ackermann model.

3. The method according to claim 1, further comprising: determining various time constants for a collision between the traffic lane and the travel trajectory from the projected traffic lane and the projected travel trajectory, the control variable being derived as a projection in the images for at least a portion of the various points in time.

4. The method according to claim 1, wherein the control variable is derived by a line-by-line comparison between the projected traffic lane and the projected travel trajectory.

5. The method according to claim 1, further comprising: determining a trajectory vanishing point of the self-motion estimate is determined, and projecting the trajectory vanishing point onto the images as the travel trajectory; determining at least one lane vanishing point of the traffic lane and projecting the at least one lane vanishing point of the traffic lane onto the images; and wherein the control variable is derived by comparing the trajectory vanishing point and the at least one lane vanishing point.

6. The method according to claim 5, wherein, for a curved traffic lane, for each of a plurality of points along the traffic lane, an respective associated lane vanishing point is determined, and the control variable is derived by comparing the respective associated lane vanishing points with the trajectory vanishing point.

7. The method according to claim 1, wherein the traffic lane to be followed is determined from the images using road boundary features.

8. The method according to claim 1, wherein the traffic lane to be followed is determined by determining a drivable surface and/or using a geographic map and/or using semantic segmentation and/or by a comparison with trajectories of other vehicles.

9. A non-transitory computer-readable storage medium on which is stored a computer program for deriving a control variable for lateral guidance of a motor vehicle using images of at least one camera of the motor vehicle, wherein the at least one camera captures images of a roadway, the computer program, when executed by a computer or control device, causing the computer or control device to perform the following steps: projecting a traffic lane of the roadway to be followed onto the images; determining a travel trajectory of the motor vehicle in reference to the images based on a self-motion estimate and projecting the determined travel trajectory onto the images; and deriving a control variable for lateral guidance of the motor vehicle from a comparison between the traffic lane projected onto the images and the determined travel trajectory projected onto the images.

10. A motor vehicle, comprising: at least one camera which, during operation, captures images of a roadway; and a control device communicatively connected to the at least one camera, wherein the control device is configured to derive a control variable for lateral guidance of a motor vehicle using images of the at least one camera of the motor vehicle, the control device configured to: projecting a traffic lane of the roadway to be followed onto the images; determine a travel trajectory of the motor vehicle in reference to the images based on a self-motion estimate and projecting the determined travel trajectory onto the images, and derive a control variable for lateral guidance of the motor vehicle from a comparison between the traffic lane projected onto the images and the determined travel trajectory projected onto the images.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0042] FIG. 1 shows a highly simplified top view of a motor vehicle having a camera, according to an example embodiment of the present invention.

[0043] FIG. 2 shows a flow chart for deriving a control variable for lateral guidance of the motor vehicle, according to an example embodiment of the present invention.

[0044] FIG. 3 shows an image captured by the camera.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0045] For lateral guidance, e.g., for lane center guidance and/or for lane keeping in a motor vehicle 1, as shown by way of example in

[0046] FIG. 1, images 3 (see FIG. 3) of an upcoming roadway 4 are recorded using at least one camera 2 of the motor vehicle 1, as indicated in FIG. 1. The images 3 are used to derive a control variable for lateral guidance of the motor vehicle 1, e.g., a steering angle and/or a steering torque, according to the following explanations.

[0047] In this context, the projections and derivations explained below are performed within the images 3. Compared to projections between the images 3 and a coordinate system, errors and accuracy losses caused by the transformations are therefore prevented or at least reduced. Doing so leads to a more precise derivation of the control variable, hence to a more precise lateral guidance of the motor vehicle 1. A traffic lane 5 to be followed as well as a known, extrinsic calibration of the at least one camera 2 are required for this purpose. In the exemplary embodiment shown in FIG. 1, the motor vehicle 1 features a single camera 2. The traffic lane 5 to be followed and the calibration of the camera 2 are presupposed in the following description.

[0048] According to FIG. 2, the traffic lane 5 to be followed along the roadway 4 is projected onto the images 3 during a method step 20. The method step 20 is also referred to hereinafter as the traffic lane projection step 20. In FIG. 3, the projection of the traffic lane 5 is shown in an image 3 from the camera 2. The projected traffic lane 5 is shown as unbroken lines. As shown in FIG. 3, the traffic lane 5 to be followed is bounded by, e.g., road boundary features 9, which in the exemplary embodiment shown are road markings 12. The road boundary features 9 are shown as dashed lines in FIG. 3. In addition, during a method step 21 as shown in FIG. 2, a trajectory 6 of the motor vehicle 1 is determined in reference to the images 3 on the basis of a self-motion estimate and projected onto the images 3. This trajectory 6 is also hereinafter referred to as the travel trajectory 6. The travel trajectory 6 is shown as lines of alternating dots and dashes in FIG. 3. The method step 21 is also referred to hereinafter as the travel trajectory step 21. The self-motion estimate is advantageously camera-based and/or image-based. In a subsequent method step 22, the control variable for lateral guidance of the motor vehicle 1 is derived based on a comparison between the traffic lane 5 projected onto the images and the travel trajectory 6 projected onto the images. The method step 22 is also referred to hereinafter as the derivation step 22. In the derivation step 22, the traffic lane 5 corresponds in this case to the target state of lateral guidance, and the travel trajectory 6 corresponds to the actual state of lateral guidance.

[0049] In the flow chart shown in FIG. 2, it is assumed that the travel trajectory step 21 is performed after the traffic lane projection step 20. Of course, these method steps 20, 21 can also be performed in reverse order, or at the same time.

[0050] Determination of the travel trajectory 6 can be determined by means of an Ackermann model and a ground plane estimate. The travel trajectory 6 thus determined is then projected onto the images 3 during the travel trajectory step 21. The travel trajectory step 21 can in this case also include determining the travel trajectory 6

[0051] To derive the control variable (hence during the derivation step 22), various time constants regarding a collision between the travel lane 5 and the travel trajectory 6 can be determined based on the projected traffic lane 5 and the projected travel trajectory 6. These time constants are also known to the person skilled in the art as “Time To Contact” (abbreviated as “TTC”) and are indicated as “TTC” in FIG. 3. As indicated in FIG. 3, the control variable can in this case be derived as a projection in image 3 for at least a portion of the various time constants. Said derivation is advantageously performed by means of a line-by-line comparison between the projected traffic lane 5 and the projected travel trajectory 6, the corresponding lines in FIG. 3 being indicated by transverse and parallel lines. Accordingly, the control variable is advantageously derived by means of a line-by-line comparison between the projected traffic lane 5 and the projected travel trajectory 6. In other words, the control variable can be derived by means of a line-by-line comparison between the actual state and the target state. The change in the control variable can be determined for various time constants as a projection onto the images 3 in order to determine a correspondingly suitable control variable, or a change of control variable.

[0052] Alternatively or additionally, it is possible that a vanishing point-based approach be used to derive the control variable. For this purpose, according to FIG. 3, a vanishing point 7 for the self-movement of the motor vehicle 1 (hereinafter also referred to as a trajectory vanishing point 7), as well as at least one vanishing point 8 of the traffic lane 5 to be followed (hereinafter also referred to as a lane vanishing point 8), is projected onto the images 3. More specifically, during the travel trajectory step 21, the trajectory vanishing point 7 is determined based on the self-movement of the motor vehicle 1 and projected onto the images 3. In addition, at least one lane vanishing point 8 is the traffic lane 5 is determined and projected onto the images 3. Alternatively or additionally, during the traffic lane projection step 20, at least one lane vanishing point 8 can be determined and projected onto the images 3. The control variable is in this case derived by comparing the vanishing points 7, 8. In other words, during the derivation step 22, the control variable is determined by comparing the trajectory vanishing point 7 with the at least one lane vanishing point 8. In the exemplary embodiment shown in FIG. 3, a straight traffic lane 5 is shown. The use of a single lane vanishing point 8 is thus sufficient. In contrast, if the traffic lane 5 is curved and/or has bends (not shown), it is advantageous for several points along the traffic lane 5 to respectively determine an associated lane vanishing point 8, and for the control variable to be derived by comparing the lane vanishing points 8 with the trajectory vanishing point 7.

[0053] It is possible that the traffic lane 5 be determined from the images 3 from the camera 2. Doing so has the advantage that the required steps are performed in the images 3. This results in a further reduction of possible errors and/or inaccuracies. In other words, the traffic lane 5 to be followed is extracted and thus determined from the images 3, e.g., by means of the road boundary features 9. However, it is also alternatively or additionally possible that the traffic lane 5 to be followed be determined by means of determining a drivable surface and/or by means of a geographic map and/or by means of semantic segmentation and/or by comparison with the trajectories of other vehicles.

[0054] It is understood that the method steps 20, 21, 22 will be repeated continuously in order to achieve an appropriate and continuous derivation of the control variable.

[0055] The method is performed in an automated manner and, advantageously, by means of a computer program product, e.g., an appropriately configured software and/or algorithm.

[0056] In order to perform the method, the motor vehicle 1 comprises a control device 10 as indicated in FIG. 1, which is communicatively connected to the camera 2 and configured accordingly. In this context, the control device 10 can comprise a computer system 11, in particular the computer program product, which is configured to perform the method.

[0057] Although only one control variable for lateral guidance of the vehicle 1 has been addressed in the foregoing description of the figures, it is to be understood that two or more control variables can also be derived using the method. It is further understood that deriving the control variable also includes changes to an existing and/or provided control variable.

[0058] The at least one derived control variable is in this case provided to a driving assistance system of the motor vehicle 1 (not shown). The driving assistance system is preferably able to drive, in particular steer, the motor vehicle 1 at least partially autonomously.