Method for environmental acquisition, data processing unit

Abstract

A method for environmental acquisition for acquiring the surrounding environment of a vehicle. Using a plurality of cameras and/or sensor devices, from at least two positions, images of the surrounding environment are produced, and on the basis of the images a free surface, as well as a contour bounding the free surface, are ascertained. Separately for each image and the contour determined on the basis of this image, for a multiplicity of points along the contour, feature vectors are calculated and entered into a one-dimensional array. Through matching of the one-dimensional arrays, correspondences are sought for which the three-dimensional position is subsequently ascertained by triangulation. A computing unit for carrying out at least individual method steps of the method is also described.

Claims

1. A method for environmental acquisition for acquiring a surrounding environment of a vehicle, the method comprising the following steps: using a plurality of cameras and/or sensor devices, from at least two positions, producing images of the surrounding environment, and ascertaining, based each of the images, a respective free surface and a respective contour bounding the respective free surface; separately, for each of the images, for a multiplicity of points along the respective contour, calculating feature vectors and entering the features vectors into a respective one-dimensional array; determining correspondences by matching the respective one-dimensional arrays; and ascertaining, based on the correspondences, a three-dimensional position using triangulation.

2. The method as recited in claim 1, wherein the ascertained three-dimensional position is stored in an environment model using a specified reference point.

3. The method as recited in claim 2, wherein the specified reference point is a vehicle reference point.

4. The method as recited in claim 1, wherein the images of the surrounding environment are produced using a set of calibrated cameras and/or sensor devices.

5. The method as recited in claim 1, wherein at least one sensor is used to ascertain the free surface.

6. The method as recited in claim 5, wherein the sensor is a radar sensor or a lidar sensor.

7. The method as recited in claim 1, wherein from the images of the surrounding environment and/or sensor data, information is derived about a local surrounding environment of a point, and is stored as additional information.

8. The method as recited in claim 1, wherein a number of points along each respective contour that bounds the respective free surface is quantified.

9. The method as recited in claim 1, wherein a number of points along each respective contour that bounds the respective free surface is quantified by horizontal view rays configured at the same angular distance from one another.

10. The method as recited in claim 1, wherein in the matching of the respective one-dimensional arrays, epipolar geometry is used in the form of an epipolar line.

11. The method as recited in claim 1, wherein, for error correction, a post-estimation is carried out in a local surrounding environment of a found correspondence.

12. The method as recited in claim 1, wherein in the determining of the correspondences, a previously defined ground plane is used as a starting point.

13. The method as recited in claim 1, wherein, to distinguish between static and dynamic objects in the surrounding environment, it is investigated whether the three-dimensional position in the environment model, derived from the correspondences, remains constant over time.

14. A data processing unit configured for environmental acquisition for acquiring a surrounding environment of a vehicle, the data processing unit configured to: using a plurality of cameras and/or sensor devices, from at least two positions, produce images of the surrounding environment, and ascertain, based each of the images, a respective free surface and a respective contour bounding the respective free surface; separately, for each of the images, for a multiplicity of points along the respective contour, calculate feature vectors and entering the features vectors into a respective one-dimensional array; determine correspondences by matching the respective one-dimensional arrays; and ascertain, based on the correspondences, a three-dimensional position using triangulation; wherein the data processing unit is connected to or capable of being connected to, at least one camera and/or sensor device in such a way as to communicate data.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 shows a schematic representation of a vehicle having a set of calibrated cameras for acquiring the surrounding environment.

(2) FIG. 2a shows a first image and FIG. 2b) a second image, of a surrounding environment of a vehicle, produced using different cameras.

(3) FIG. 3 shows a schematic representation of a vehicle in a typical surrounding environment.

(4) FIG. 4 shows a schematic representation of the region of acquisition of a camera with view ray division.

(5) FIG. 5 shows a schematic representation of two one-dimensional arrays of feature vectors during the matching.

(6) FIG. 6 shows a schematic representation for the explanation of a method sequence in accordance with an example embodiment of the present invention.

(7) FIG. 7 shows a further schematic representation for the explanation of the example method sequence.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

(8) FIG. 1 shows a vehicle 1 that is equipped with four cameras 2, having a front camera, a rear camera, and two side cameras, the side cameras being situated on different sides of vehicle 1. Thus, each two cameras 2 have an overlapping region of acquisition, so that an object 8 present in the region of acquisition is visible to both cameras 2, from two different positions (A, B).

(9) With the camera setup shown in FIG. 1, a distance estimation using a stereo approach, for example “wide-baseline stereo,” is difficult. Using stereo algorithms, local image regions from a first image of a first camera 2 are supposed to be located again in a second image of a second camera 2. Due to the strong geometrical distortions between the different views, which result from the large camera distances, stereo algorithms function here only with limited success.

(10) FIGS. 2a) and 2b) show, as an example, two different views or images of a surrounding environment of a vehicle 1, the upper image of FIG. 2a) having been produced by a front camera and the lower image in FIG. 2b) having been produced by a side camera. In each image, a free surface 3 is bounded by a contour 4. Free surface 3 is the drivable surface between vehicle 1 (home vehicle) and at least one object 8, 8′, 8″, or obstacle. An object 8 situated in the region of acquisition of both cameras 2 can be acquired through points P.sub.1, P.sub.2, P.sub.3, P.sub.4 situated on the respective contour 4. Thus, the points P.sub.1, P.sub.2, P.sub.3, P.sub.4 are found in both images. In order to determine the spatial position of these points P.sub.1, P.sub.2, P.sub.3, P.sub.4, a matching has to be carried out, i.e. correspondences must be sought. With the use of the method according to the present invention, the search can be limited to a line, namely contour 4. That is, the entire image does not have to be searched for correspondences, which significantly reduces the computing expense. If in addition an epipolar line 7 is used in the correspondence search, then the search can be limited to the points of intersection of epipolar line 7 with contour 4.

(11) FIG. 3 shows a traffic situation in a top view, corresponding approximately to the views or images of FIG. 2. (Home) vehicle 1 is situated in the center. The surface between vehicle 1 and surrounding objects 8, 8′, 8″, or obstacles, corresponds to free surface 3. Here it does not matter whether the surface is situated in front of, behind, or next to vehicle 1. In order to acquire all the objects 8, 8′, 8″, vehicle 1 has to be equipped with a plurality of cameras 2.

(12) An object 8 situated in the region of acquisition of a camera 2 is determined by a multiplicity of points P.sub.1, P.sub.2, P.sub.3, . . . , P.sub.n along contour 4. In order to reduce the number of points for which a correspondence has to be sought in another image, the region of acquisition of a camera 2 is divided by view rays 6. This is shown as an example in FIG. 4. The point of intersection of a view ray 6 with contour 4 then determines in each case a point to which the correspondence search is limited.

(13) On the basis of FIGS. 6 and 7, in the following the sequence of a method according to the present invention is described.

(14) FIG. 6 shows a system 200 in which an input image 210, for example a first image of the surrounding environment taken with a camera, is given to a subsystem 100. As output of subsystem 100, system 200 receives a one-dimensional array of feature vectors 220. Subsystem 100, to which input image 210 is given, is made up of a free surface determination 110 on the basis of which features 120 are calculated and points are assigned that are situated along a contour that bounds the free surface.

(15) FIG. 7 shows an overall system 400 that is based on the system 200 of FIG. 6 and that is used for the determination of three-dimensional environmental information 410 in the form of a compact one-dimensional array. Previously, system 200 of FIG. 6 was applied to two input images 210, so that for each input image 210 there is a one-dimensional array of feature vectors 220 as output. The outputs, in turn, are used as inputs 200.1 and 200.2 of the overall system 400. These inputs are given to a subsystem 300, in which first a matching 310 is carried out of the feature vectors from the one-dimensional arrays. Through triangulation 320, for each found correspondence the position in space is subsequently determined, and is stored at a specified reference point in an environmental model.

(16) A way of visualizing the matching 310 is shown as an example in FIG. 5 for two one-dimensional arrays 5 having a plurality of feature vectors. The individual feature vectors are indicated by individual boxes that can be assigned, individually or in groups, to different objects 8, or obstacles. For each feature factor of upper array 5, a correspondence is sought in the array 5 below it. Arrows 9 indicate correspondences that have been found. In the matching, the following cases can occur: there is a one-to-one assignment a plurality of feature vectors of upper array 5 are assigned to one feature vector of the array 5 below it there is no assignment for a feature vector in one of the two arrays 5.

(17) Here, the correspondence search is represented as a orderly problem, because arrows 9 do not cross one another in the assignment.