METHOD FOR ASCERTAINING A DYNAMIC FOREIGN OBJECT-DRIVING CORRIDOR ASSOCIATION

Abstract

A method for ascertaining a dynamic foreign object-driving corridor association with the aid of an image sensor of an ego vehicle, in which images of the environment of the ego vehicle are generated with the aid of the image sensor. A driving corridor of the ego vehicle is ascertained with the aid of the images in the native measuring space of the image sensor from roadway information and/or with the aid of the odometry of the ego vehicle, foreign objects are detected, and at least one kinematic variable is ascertained for at least one of the foreign objects, and at least one dynamic foreign object-driving corridor association for the lateral movement of the foreign object relative to the driving corridor is ascertained based on the at least one kinematic variable and the driving corridor.

Claims

1. A method for ascertaining a dynamic foreign object-driving corridor association using an image sensor of an ego vehicle, the method comprising the following steps: generating images of an environment of the ego vehicle using the image sensor; using the images in a native measuring space of the image sensor, performing: ascertaining a driving corridor of the ego vehicle along which the ego vehicle will be moving from roadway information and/or using odometry, detecting foreign objects, and ascertaining at least one kinematic variable is ascertained for at least one of the foreign objects; and based on the at least one kinematic variable and the driving corridor, ascertaining at least one dynamic foreign object-driving corridor association for a lateral movement of the foreign object.

2. The method as recited in claim 1, wherein a lateral velocity of the foreign object relative to the driving corridor is ascertained as one of the at least one kinematic variable.

3. The method as recited in claim 2, wherein a bounding box is assigned to the foreign object in the native measuring space, a movement of the bounding box is tracked over time and the lateral velocity of the foreign object relative to the driving corridor is ascertained from the movement.

4. The method as recited in claim 2, wherein an ego movement of the ego vehicle is utilized, and based on the velocity, a relative movement of the foreign object with regard to the driving corridor is ascertained as a dynamic foreign object-driving corridor association of the at least one dynamic foreign object-driving corridor association.

5. The method as recited in claim 1, wherein a distance of the foreign object from the driving corridor is ascertained as a kinematic variable of the at least one kinematic variable.

6. The method as recited in claim 1, wherein: foreign objects that are located outside of the driving corridor are detected in the native measuring space, a time-to-enter corridor at which the foreign object enters the driving corridor is ascertained as a dynamic foreign object-driving corridor association of the at least one dynamic foreign object-driving corridor association.

7. The method as recited in claim 1, wherein: foreign objects that are located within the driving corridor are detected in the native measuring space, a time-to-leave corridor at which the foreign object leaves the driving corridor is ascertained as a dynamic foreign object-driving corridor association of the at least one dynamic foreign object-driving corridor association.

8. The method as recited in claim 6, wherein a ratio of s distance of the foreign object from a boundary of the driving corridor and the lateral velocity is ascertained as the time-to-enter corridor.

9. The method as recited in claim 7, wherein: at least one geometrical variable of the foreign object is used, an exit probability of the foreign object leaving the driving corridor is ascertained using the at least one geometrical variable and the time-to-leave corridor.

10. The method as recited in claim 1, wherein the at least one dynamic foreign object-driving corridor association is made available to a driver assistance system of the ego vehicle.

11. A non-transitory computer-readable medium on which is stored a computer program for ascertaining a dynamic foreign object-driving corridor association using an image sensor of an ego vehicle, the computer program, when executed by a computer, causing the computer to perform the following steps: generating images of an environment of the ego vehicle using the image sensor; using the images in a native measuring space of the image sensor: ascertaining a driving corridor of the ego vehicle along which the ego vehicle will be moving from roadway information and/or using odometry, detecting foreign objects, ascertaining at least one kinematic variable is ascertained for at least one of the foreign objects; and based on the at least one kinematic variable and the driving corridor, ascertaining at least one dynamic foreign object-driving corridor association for a lateral movement of the foreign object.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0070] FIG. 1 shows a greatly simplified representation of a motor vehicle having an image sensor.

[0071] FIG. 2 shows an image recorded with the aid of the image sensor.

[0072] FIG. 3 shows a flow diagram to describe a method for ascertaining a dynamic foreign object-driving corridor association with the aid of the images generated by the image sensor, according to an example embodiment of the present invention.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0073] A motor vehicle 1, as shown in simplified form in FIG. 1, has at least one image sensor 2. In the exemplary embodiment illustrated in FIG. 1, it is assumed for the sake of simplicity that motor vehicle 1 has a single such image sensor 2. In the illustrated exemplary embodiment, image sensor 2 is part of a camera 3. While in operation, image sensor 2 generates images 4 of the environment of motor vehicle 1 (compare FIGS. 2 and 3). With the aid of these images 4, a dynamic association between a driving corridor 5 of motor vehicle 1 and foreign objects 6 is ascertained according to the method described in the following text. Motor vehicle 1 thus is ego vehicle 1. Hereinafter, the dynamic association will also be referred to as a dynamic foreign object-driving corridor association.

[0074] FIG. 2 shows an image 4 from image sensor 2 to which information has been inserted in addition. FIG. 3 shows a flow diagram to describe the method.

[0075] As may be gathered from FIGS. 2 and 3, images 4 are utilized to ascertain the dynamic foreign object-driving corridor association, and such an image 4 is symbolically shown in FIG. 4 by dashed lines. The detection of foreign objects 6 and the ascertainment of driving corridor 5 as well as the kinematics of foreign objects 6 take place with the aid of images 4 and in the native measuring space of image sensor 2, and thus in the two-dimensional image plane. To ascertain the dynamic association, driving corridor 5 of ego vehicle 1 is ascertained from roadway information with the aid of images 4 from image sensor 2 and in the native measuring space of image sensor 2. Driving corridor 5 is the particular corridor 5 along which ego vehicle 1 will move. As a rule, driving corridor 5 is restricted by roadway boundaries, which are not shown, as roadway information and/or is able to be estimated from the kinematic movement of ego vehicle 1 and thus the odometry of ego vehicle 1. In addition, foreign objects 6 are detected with the aid of images 4 from image sensor 2 and in the native measuring space of image sensor 2. For the sake of simplicity, a single foreign object 6 can be seen in the example shown in FIG. 2. To simplify matters, it is also assumed hereinafter that this single foreign object 6 is detected in images 4 to simplify matters. The foreign object involves a foreign object 6 that moves on its own accord. Purely by way of example, foreign object 6 is a person 7. A not depicted third-party vehicle may likewise be a foreign object 6. In addition, at least one kinematic variable is ascertained with the aid of images 4 from image sensor 2 and in the native measuring space from image sensor 2 for at least one of foreign objects 6, which is the only foreign object 6 in the illustrated exemplary embodiment. Based on the at least one kinematic variable and driving corridor 5, at least one dynamic foreign object-driving corridor association for the lateral movement of foreign object 6 relative to driving corridor 5 will then be ascertained. The dynamic foreign object-driving corridor association thus describes and/or encompasses the lateral movement of foreign object 6 relative to driving corridor 5. The dynamic foreign object-driving corridor association expediently also includes a static association between foreign objects 6 and driving corridor 5. A static association is to be understood as the allocation of foreign objects 6 that are static relative to driving corridor 5.

[0076] The roadway information may be available in point lists or as polygon chains, also known as splines, which describe the roadway extension and driving corridor 5 in corresponding image coordinates. As shown in FIG. 2, foreign objects 6 are able to be represented by associated so-called bounding boxes 8, which make the position and extension of associated foreign object 6 available in image coordinates.

[0077] In the following text, the method for ascertaining the dynamic association will be described by way of example with reference to FIG. 3. In an advantageous manner, the method is executed using an appropriately developed computer program product.

[0078] As shown by dashed lines in FIG. 3, images 4 generated by image sensor 2 are supplied on a continuous basis. In a method measure 100, images 4 are used to ascertain driving corridor 5 in the native measuring space, as described above. Method measure 100 will also be referred to as track measure 100 in the following text. In addition to ascertaining driving corridor 5, track measure 100 advantageously also includes the ascertainment of the entire drivable area as well as its topology. In a further method measure 101, images 4 are utilized and, as described above, foreign objects 6 are detected in the native measuring space. This method measure 101 will also be referred to as a foreign object measure 101 in the following text. In addition, in a method measure 102, at least one kinematic variable is ascertained for foreign objects 6 with the aid of images 4 and in the native measuring space. This method measure 102 is also referred to as kinematic measure 102 in the following text. In the illustrated exemplary embodiment, the ego movement of ego vehicle 1 is furthermore ascertained in a method measure 103 with the aid of images 4 and in the native measuring space. Hereinafter, this method measure 103 is also referred to as ego movement measure 103. In an advantageous manner, ego movement measure 103 includes the ascertainment of translatory and rotatory ego movements of ego vehicle 1. The result of ego movement measure 103 is taken into consideration in kinematic measure 102. This means that the at least one kinematic variable is ascertained in the native measuring space, for which the ego movement of ego vehicle 1 is considered. The results of method measures 100 to 103 are made available to a following method measure 104 for ascertaining at least one dynamic foreign object-driving corridor association, as described above. Method measure 104 will also be referred to as association measure 104 in the following text. The result of association measure 104, as described above and shown in FIG. 3 by two fields within association measure 104, includes both the association of foreign objects 6 that are moving relative to driving corridor 5 and the static association. As shown in FIG. 3, the result of association measure 104 is able to be made available to a driver assistance system 9 of ego vehicle 1. With the aid of driver assistance system 9, assistance for a vehicle driver (not shown) can be realized and an at least semiautomated driving of ego vehicle 1 is able to be implemented. Driver assistance system 9, for example, may include an adaptive cruise control and/or an autonomous emergency braking function.

[0079] At least one of the ascertained kinematic variables is advantageously the lateral velocity of foreign object 6 relative to driving corridor 5, preferably relative to a boundary of driving corridor 5. That means that a velocity of foreign object 6 transversely or at an angle to driving corridor 5 is ascertained with the aid of images 4 and in the native measuring space. To ascertain the lateral velocity, a velocity signal is advantageously derived from the movement of bounding box 8 associated with foreign object 6 in the image plane over time. Taking the ego movement of ego vehicle 1 into account, the relative movement of foreign object 6 with respect to driving corridor 5, especially to boundaries of driving corridor 5, and thus the lateral velocity, is able to be ascertained. In the same way, a movement direction 10 (see FIG. 2) of foreign object 6 is ascertainable as a kinematic variable. In an advantageous manner, a distance of foreign object 6 from driving corridor 5, in particular from boundaries of driving corridor 5, is ascertained as a further kinematic variable. For instance, this is accomplished by measuring the distance in pixels between the splines and the associated bounding box 8. This may involve the lateral distance of foreign object 6 from the boundary of driving corridor 5. With the aid of the distance, the static association can particularly also be implemented in a simplified manner. As an alternative or in addition, kinematic variables can also be ascertained with the aid of the optical flow.

[0080] The respective dynamic association may be one such form, provided it describes and/or includes a dynamic relation between driving corridor 5 and associated foreign object 6.

[00004]$TTEC=\frac{d}{v_l},$

[0081] In the exemplary embodiment shown in FIG. 2, a time period in which foreign object 6 enters driving corridor 5 is ascertained as a dynamic association. The entry period is also known under its English name “time-to-enter corridor”, abbreviated as “TTEC”. Foreign object 6 is located outside of driving corridor 5. In a simple case of a uniform movement of foreign object 6, the time-to-enter corridor corresponds to a uniform movement of foreign object 6 in relation to the distance of foreign object 6 from a boundary of driving corridor 5 and the lateral velocity:

[00005]$TTEC=\frac{d}{v_l},$

where d is the distance, and v.sub.l is the lateral velocity.

[0082] In a similar manner, a time-to-leave corridor in which foreign object 6 leaves driving corridor 5 is able to be ascertained for a foreign object 6 (not shown) which is located within driving corridor 5. The time-to-leave is also known under its English name “time-to-leave corridor”, abbreviated as “TTLC”.

[00006]$TTEC=\frac{d}{v_l},$

[0083] In an advantageous manner, based on the time-to-enter corridor or the time-to-leave corridor, probabilities of foreign object 6 entering driving corridor 5 or of foreign object 6 leaving driving corridor 5 are ascertained. For this purpose, at least one geometrical variable of foreign object 6 is advantageously utilized, and an entry probability of foreign object 6 entering driving corridor 5 is ascertained with the aid of the at least one geometrical variable and the time-to-enter corridor, and/or an exit probability of foreign object 6 exiting driving corridor 5 is ascertained with the aid of the at least one geometrical variable and the time-to-leave corridor. Such a geometrical variable, for example, is the angle between the predicted object trajectory of foreign object 6 and driving corridor 5, in particular a boundary of driving corridor 5.

[0084] In addition, a plausibility check for the entering or leaving of foreign object 6 is able to be utilized. Such a plausibility check advantageously includes further information relating to foreign object 6 such as a turn signal of a third-party vehicle (not shown) as a foreign object 6.