Camera monitoring system

Abstract

A camera monitoring system for a side region of a vehicle, includes an image-capturing unit for simultaneously capturing image data relating to a first ground region and relating to a second ground region on at least one side of the vehicle, and an image-analyzing unit, which is designed to separate image data relating to the first ground region from image data relating to the second ground region so as to enable independent display.

Claims

1. A camera monitoring system for a side region of a vehicle, comprising: an image capture unit for simultaneous capture of image data for a first ground region and for a second ground region on at least one side of the vehicle, wherein the image capture unit includes camera unit with a wide-angle lens configured to simultaneously capture an image of the first ground region and the second ground region; and an image analysis unit configured to separate, in the captured image, image data for the first ground region from image data for the second ground region to enable an independent display, wherein the first ground region is defined as follows: starting with a first line which extends perpendicularly to a direction of movement of the vehicle and is offset rearwards by 4 m from a position of a driver, the first ground region extends rearwards parallel to a vehicle edge with a laterally variable width, wherein the variable width increases linearly from 1 m to a width of 5 m to a distance of 26 m from the first line and then remains constant; and the second ground region is defined as follows: starting with a second line, which extends perpendicularly to a direction of movement of the vehicle and which is offset rearwards by 1.5 m from a position of a driver, the second ground region extends rearwards parallel to a vehicle edge to a length of 23.5 m with a laterally variable width, wherein the variable width increases linearly from 4.5 m to a width of 15 m up to a distance of 8.5 m from the second line and then remains constant.

2. The camera monitoring system as claimed in claim 1, wherein the image capture unit comprises at least one of the following camera units: a first camera unit with a fish-eye lens to simultaneously detect a front region in front of the vehicle, the first ground region, the second ground region and a third ground region when attached to a front corner of the vehicle on the passenger side; a second camera unit for capturing image data on a passenger side of the vehicle; a third camera unit for capturing image data on a driver side of the vehicle; a fourth camera unit with a fish-eye lens for simultaneously capturing the front region, the first ground region, the second ground region and a third ground region when attached to a front corner of the vehicle on the driver side.

3. The camera monitoring system as claimed in claim 2, wherein the image analysis unit is configured to compensate for distortions in the captured image data.

4. The camera monitoring system as claimed in claim 2, wherein the third ground region is defined as follows: starting with a third line which extends perpendicularly to a direction of movement of the vehicle through a position of a driver, the third ground region extends parallel to a vehicle edge rearwards by 1.75 m and forwards by 1 m at a width of 2 m; and the front region is defined as follows: starting with a front vehicle boundary, the front region extends in the longitudinal direction of the vehicle up to at least 2 m over the entire width of the cab and sideways 2 m beyond the passenger side.

5. The camera monitoring system as claimed in claim 1, further comprising: at least one display unit with at least a first section for displaying the first ground region and a second section for displaying the second ground region.

6. The camera monitoring system as claimed in claim 1, wherein the image analysis unit is configured to verify and provide at least one of the following environment information items in the captured image data: a road marking; an object; another vehicle; a roadside.

7. A commercial vehicle comprising a camera monitoring system according to claim 1.

8. A method for supporting a driver assistance system or autonomous driving of a vehicle, the method comprising: analyzing image data collected by a camera monitoring system, the camera monitoring system being for a side region of a vehicle and having: an image capture unit for simultaneous capture of image data for a first ground region and for a second ground region on at least one side of the vehicle, wherein the image capture unit includes camera unit with a wide-angle lens configured to simultaneously capture an image of the first ground region and the second ground region; and an image analysis unit configured to separate, in the captured image, image data for the first ground region from image data for the second ground region to enable an independent display based on the collected image data, identifying environment information of the vehicle; based on the collected image data, identifying environment information of the vehicle, wherein the environment information includes at least one of the following: a detected road marking, a detected roadside, a detected other vehicle, a detected object in the vicinity of the vehicle; and providing the environment information, wherein the first ground region is defined as follows: starting with a first line which extends perpendicularly to a direction of movement of the vehicle and is offset rearwards by 4 m from a position of a driver, the first ground region extends rearwards parallel to a vehicle edge with a laterally variable width, wherein the variable width increases linearly from 1 m to a width of 5 m to a distance of 26 m from the first line and then remains constant; and the second ground region is defined as follows: starting with a second line, which extends perpendicularly to a direction of movement of the vehicle and which is offset rearwards by 1.5 m from a position of a driver, the second ground region extends rearwards parallel to a vehicle edge to a length of 23.5 m with a laterally variable width, wherein the variable width increases linearly from 4.5 m to a width of 15 m up to a distance of 8.5 m from the second line and then remains constant.

9. A computer product comprising a non-transitory computer readable medium having stored therein a program code which, when executed on a processor, carries out the acts of: analyzing image data collected by a camera monitoring system, the camera monitoring system being for a side region of a vehicle and having: an image capture unit for simultaneous capture of image data for a first ground region and for a second ground region on at least one side of the vehicle, wherein the image capture unit includes camera unit with a wide-angle lens configured to simultaneously capture an image of the first ground region and the second ground region; and an image analysis unit configured to separate, in the captured image, image data for the first ground region from image data for the second ground region to enable an independent display based on the collected image data, identifying environment information of the vehicle; based on the collected image data, identifying environment information of the vehicle, wherein the environment information includes at least one of the following: a detected road marking, a detected roadside, a detected other vehicle, a detected object in the vicinity of the vehicle; and providing the environment information, wherein the first ground region is defined as follows: starting with a first line which extends perpendicularly to a direction of movement of the vehicle and is offset rearwards by 4 m from a position of a driver, the first ground region extends rearwards parallel to a vehicle edge with a laterally variable width, wherein the variable width increases linearly from 1 m to a width of 5 m to a distance of 26 m from the first line and then remains constant; and the second ground region is defined as follows: starting with a second line, which extends perpendicularly to a direction of movement of the vehicle and which is offset rearwards by 1.5 m from a position of a driver, the second ground region extends rearwards parallel to a vehicle edge to a length of 23.5 m with a laterally variable width, wherein the variable width increases linearly from 4.5 m to a width of 15 m up to a distance of 8.5 m from the second line and then remains constant.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 shows a camera monitoring system according to an exemplary embodiment of the present invention.

(2) FIG. 2 shows further details of the camera monitoring system according to further exemplary embodiments.

(3) FIG. 3A,3B show ground regions which are captured by the capture unit simultaneously according to exemplary embodiments.

(4) FIG. 4A,4B show further ground regions which are captured by the capture unit simultaneously according to exemplary embodiments.

(5) FIG. 5A,5B show a display for ground regions from FIGS. 3A, 3B.

(6) FIG. 6A,6B show an exemplary display for ground regions from FIGS. 4A, 4B.

DETAILED DESCRIPTION OF THE DRAWINGS

(7) FIG. 1 shows a camera monitoring system according to an exemplary embodiment of the present invention. It is particularly suitable for use in commercial vehicles, but also suitable for other vehicles, for example to replace existing mirror arrangements. The camera monitoring system comprises an image capture unit 110 for the simultaneous capture of image data for a first ground region 210 and for a second ground region 220 on at least one side (especially on the passenger side) of the exemplary commercial vehicle 10. The image capture unit 110 is, for example, a suitably mounted camera, which captures both ground regions 210, 220 simultaneously with one image. In addition, the camera monitoring system includes an image analysis unit 120, which is designed to separate image data for the first ground region 210 from image data for the second ground region 220 to enable an independent display.

(8) Optionally, the camera monitoring system comprises at least one display unit 131 for visualizing the images captured by the image capture unit 110. However, the display unit 131 does not have to be part of the camera monitoring system. It is also possible to use an existing display for this purpose.

(9) For example, it is possible to display the first ground region 210 in a first section of the display unit 131 and the second ground region 220 in another section. In the simplest example, the image capture unit 110 is a camera with a suitable lens (for example, a wide-angle lens), which can detect multiple regions simultaneously without having to move the camera. The multiple regions are separated by the image processing unit 120 to allow the separate, simultaneous display on the display unit 131. The ground regions 210, 220 can be displayed on the display but also one after the other by switching.

(10) FIG. 2 shows another exemplary embodiment of the camera monitoring system, which differs from the system shown in FIG. 1 in that the image capture unit 110 comprises a first camera unit 121, a second camera unit 122, a third camera unit 123 and a fourth camera unit 124, for example.

(11) The first camera unit 121 and the second camera unit 122 are, for example, designed to be mounted in a front corner region or on a side region of the cab (for example on the passenger side 16) and to capture the side region and rear region of the commercial vehicle independently of each other. In particular, the first camera unit 121 also captures a ground region directly in front of and next to the commercial vehicle and the side region behind the driver's cab. To achieve this, the first camera unit 121 can include a suitable wide-angle lens such as a fish-eye lens. The second camera unit 122 captures a side and rear region (first and second ground regions 210, 220) of the exemplary commercial vehicle. For this purpose, a suitable wide-angle lens may be provided.

(12) In addition, the third camera unit 123 and the fourth camera unit 124 are provided on the driver side 17, which can be attached in a corner region of the cab 10. The third camera unit 123 can be in a similar or identical form to the second camera unit 122 and can capture a side region next to or behind the driver's cab 10 on the driver side 17. The fourth camera unit 124 may be of a similar or identical form to the first camera unit 121 and can capture a ground region that is directly adjacent to the commercial vehicle. To achieve this, the fourth camera unit 124 can also include a suitable wide-angle lens such as a fish-eye lens.

(13) This offers the advantage that different sensor regions and in particular ground regions around the vehicle are captured in parallel. For example, the first camera unit 121 and the second camera unit 122 can simultaneously capture the first and second ground regions 210, 220 on the side next to and behind the commercial vehicle. Due to this redundancy, exemplary embodiments are particularly suitable for commercial vehicles where visual checking by the driver is generally not possible on the passenger side.

(14) Exemplary embodiments of the present invention are therefore particularly suitable for replacing ordinary mirror arrangements such as rear-view mirrors of commercial vehicles, wherein additional functionalities are provided.

(15) FIG. 3A, FIG. 3B, FIG. 4A and FIG. 4B illustrate the ground regions which are captured simultaneously as side regions for example by the camera units 121, 122, 123, 124 according to exemplary embodiments. Here and below, all length specifications are understood to have a tolerance of up to ±10% or ±5%. In addition, the following ground regions are minimal regions, which are captured as a minimum. It is understood that the camera units will also capture other regions.

(16) FIGS. 3A, 3B illustrate the first ground region 210 and the second ground region 220, which are captured simultaneously by the second and third camera units 122, 123 on their respective sides according to exemplary embodiments.

(17) FIG. 3A shows the first ground region 210, which extends on both sides of the vehicle and can be defined as follows: it starts with a first line 211, which extends perpendicularly to a direction of movement of the commercial vehicle and is offset rearwards by 4 m from a position 15 of a driver (or his eye points) and extends rearwards parallel to both vehicle edges 16, 17 with a variable lateral width. The variable width increases linearly from 1 m to a width of 5 m rearwards up to a distance of 26 m from the first line 211 and then remains constant. The first ground region 210 thus has a maximum width of approx. 5 m, wherein this maximum width is reached at a distance of approx. 30 m starting at the driver's position.

(18) FIG. 3B shows the second ground region 220, which extends on both sides of the vehicle and can be defined as follows: the second ground region 220 begins with a second line 221, which extends perpendicularly to a direction of movement of the commercial vehicle and is offset rearwards by 1.5 m from a position 15 of a driver (or his eye points) and extends rearwards parallel to the edges of the vehicle 16, 17 to a length of 23.5 m with a laterally variable width. The variable width increases linearly from 4.5 m to a width of 15 m up to a distance of 8.5 m from the second line 221 and then remains constant rearwards. It thus widens from approx. 4.5 m to a maximum width of 15 m, which is reached at a distance of approx. 10 m from the driver's position.

(19) The first ground region 210 from FIG. 3A thus essentially represents a viewing region rearwards, while the second ground region 220 from FIG. 3B represents a side view region, which is important, for example, for a lane change in order to detect whether there is a vehicle in one of the adjacent lanes next to the vehicle, for example. The second camera unit 122 (and analogously the third camera unit 123 on the driver side 16) is primarily used for capturing the first ground region 210 and simultaneously the second ground region 220.

(20) FIGS. 4A, 4B illustrate the third ground regions 230 and the front region 240, which according to exemplary embodiments are primarily captured simultaneously by the first camera unit 121 on the passenger side 16 (and analogously by the fourth camera unit 124 on the driver side 17).

(21) FIG. 4A illustrates the third ground region 230, which extends from a third line 231, which runs perpendicularly to a direction of movement of the commercial vehicle rearwards by 1.75 m and approx. 1 m forwards through the position 15 of the driver. The width of the third ground region 230 is about 2 m, measured from the right edge of the vehicle 16. The third ground region 230 is defined for the driver side 17 in the same or a similar way.

(22) FIG. 4B shows a fourth ground region 240 (front region), which is also captured by the first camera unit 121, for example. The fourth ground region 240 extends from the front 241 of the cab 10 over the width of the vehicle up to a distance of approx. 2 m forwards and up to 2 m beyond the vehicle edge 16 on the passenger side. Optionally, the fourth ground region 240 is rounded at the front right corner (from the driver's point of view) with a radius of curvature of approx. 2 m. The third ground region 230 and the fourth ground region 240 therefore cover in particular regions which are visible with difficulty or not visible at all to drivers of commercial vehicles (as opposed to passenger cars).

(23) For example, in order to achieve redundancy of the camera monitoring system, the first camera unit 121 is mounted in an upper corner region of the cab 10 on the passenger side 16, so that the first camera unit 121 can capture the third ground region 230, the fourth ground region 240, the first ground region 210 and the second ground region 220. In order to be able to visually represent all ground regions as close to reality as possible, for example, images by the first camera unit 121 (for example with the fish-eye lens) from the first ground region 210 and from the second ground region 220 can be processed or transformed accordingly, in order to eliminate possible distortions by the fish-eye lens of the first camera unit 121.

(24) Also on the driver side, the fourth camera unit 124 is advantageously a wide-angle or fish-eye camera, which also captures image data, which are suitable after corresponding transformation to capture the first and second ground regions 210, 220 and to visualize these to the driver with minimal distortion. For this purpose, the fourth camera unit 124 can again be mounted in an upper corner region of the cab 10 of the commercial vehicle, so that this camera unit can detect not only the third ground region 230 and the fourth ground region 240 (in front of the cab), but also the rear-facing first ground region 210 and the second ground region 220.

(25) The distortion correction can also be done using lookup tables to adjust the image for a special display that is already present in the vehicle.

(26) Exemplary examples thus allow the driver to constantly monitor what is happening both on the left side of the vehicle and on the right side of the vehicle. For example, camera units with a high resolution and a wide-angle lens can be used to ensure good coverage by the camera units. Compared to conventional mirror arrangements, such camera systems offer a very high degree of flexibility, since camera images can be displayed separately on different sections of a display. For example, it is possible to split a display to show the different ground regions separately.

(27) FIG. 5A and FIG. 5B show by way of example how a captured image from the driver side (on the left) can be displayed on the corresponding display unit 131. In FIG. 5A, the first ground region 210 is highlighted, which is shown in an upper region of the display unit 131, for example. FIG. 5B shows the second ground region 220, which is shown on the display unit 131 in a lower region, for example.

(28) FIG. 6A shows by way of example a captured image of the third ground region 230 as it is captured by the first camera unit 121 on the passenger side 16. FIG. 6B shows by way of example the front region 240 as it is captured by the first camera unit 121 from the passenger side 16, for example. Both captured regions 230, 240 can be displayed in a similar way to the first ground region 210 and the second ground region 220 (see FIGS. 5A, 5B).

(29) The first camera unit 121 (and similarly the fourth camera unit 124) comprises by way of example a fish-eye lens and is arranged in a corner region of the vehicle cabin, for example on the passenger side 16. In this way, both the third ground region 230 and the front region 240 can be captured. The first and third camera units 121, 124 are directed vertically downwards and can also capture regions in the horizontal (i.e. the first and second ground regions 210, 220) due to the fish-eye lens. It is advantageous if these fish-eye cameras 121, 124 have the highest possible resolution, in order to be able to recognize many details after distortion correction (image transformation) of the captured images. Therefore, the first camera unit 121 and/or the fourth camera unit 124 may have a higher resolution than the second camera unit 122 and/or the third camera unit 123, for example.

(30) The road markings 20 shown (see for example FIG. 6B) can be captured or detected by any of the mentioned camera units 121, 122, 123, 124. Due to the defined ground regions, the road markings 20 or other objects can almost always be captured by the camera monitoring system. In contrast to conventional systems, which are directed forwards or directly rearwards, in exemplary embodiments of the present invention the view cannot be blocked by other objects (such as vehicles). In particular, the fish-eye cameras (first or fourth camera unit 121, 124) are suitable for detecting the road markings 20 and/or objects in an environment of the vehicle since they capture the ground region on all sides. However, the road markings 20 or other structures on the road near the vehicle can be captured and continuously tracked using the rear-facing second and/or third camera units 122, 123.

(31) Therefore, exemplary embodiments can also be used in a driver assistance system or in autonomous driving, in order to take over the detection of road markings 20 or of objects in the environment of the vehicle by means of additional algorithms.

(32) Exemplary embodiments of the present invention relate in particular to the following subject matter.

(33) A camera monitoring system for a commercial vehicle which comprises at least one rear-facing wide-angle camera 122, 123, which covers the first ground region 210 and the second ground region 220.

(34) According to other advantageous embodiments, the camera monitoring system comprises at least two rear-facing wide-angle cameras 122, 123, which can be mounted on the right and left sides of the vehicle.

(35) According to other advantageous embodiments, the camera monitoring system comprises at least one downward-facing fish-eye camera 121, which can be mounted on the passenger side 16 to capture a front region 240 and a third ground region 230.

(36) According to other advantageous embodiments, the camera monitoring system comprises an additional downward-facing fish-eye camera 124 on the driver side 17 of the cab 10 to provide further similar functionalities as provided on the passenger side 16. Thus, the first ground region 210 and the second ground region 220 can also be captured on the driver side 17.

(37) According to other advantageous embodiments, the images which are captured by the camera monitoring system, for example by the rear-facing wide-angle cameras 122, 123, can be used for the detection of road markings 20 and/or for object detections.

(38) According to other advantageous embodiments, the downward-facing fish-eye camera 121, 124 can be used to recognize the road markings 20 and/or to detect objects.

(39) Exemplary embodiments thus also include a method for the use of the camera monitoring system. This method or these steps of the method can also be computer-implemented, i.e. can be implemented by instructions stored on a memory medium that are able to perform the steps of the method when it is running on a processor. The instructions typically include one or more statements, which may be stored in different ways on different media which are in or peripheral to a control unit (with a processor) which, when read and executed by the control unit, cause the control unit to perform functions, functionalities, and operations necessary to perform a method according to the present invention.

(40) The features of the invention disclosed in the description, the claims and the figures may be essential for the realization of the invention both individually and in any combination.

REFERENCE CHARACTER LIST

(41) 10 Cab 15 Driver position (eye positions) 16 Passenger side (vehicle side on passenger side) 17 Driver side 20 Road markings 110 Image capture unit(s) 120 Image processing unit 121,122, . . . Camera units 131 Display unit(s) 210,220, . . . Ground region 211,121, . . . Front reference lines (first line, second line, . . . ) 241 Front vehicle edge/vehicle boundary.