MOTOR VEHICLE CAMERA DEVICE WITH HISTOGRAM SPREADING

Abstract

The invention relates to methods for operating a camera device (4) for a motor vehicle (3). An image sensor (5) of the camera device (4) provides a raw image (R) of an environment (8) of the camera device (4). An image processing device (6) generates an output image (I) from the raw image (R) by means of a spreading function (13) of a histogram spreading (12). The spreading function (13) generates from a respective input pixel value (Li) of each pixel of the raw image (R) each one output pixel value (Lo) of a corresponding pixel of the output image (I). It is the object of the invention to adapt the histogram spreading (12) to the capturing situation. At least one parameter value (S, G) of the camera device (4) depending on a brightness (B1, B2) of the environment (8) is acquired and a limit value (L) for the output pixel values (Lo) is set in the spreading function (13) depending on the at least one acquired parameter value (S, G).

Claims

1. A method for operating a camera device for a motor vehicle, comprising: providing a raw image of an environment of the camera device by an image sensor of the camera device; and generating an output image from the raw image by an image processing device, which determines from a respective input pixel value of each pixel of the raw image each one output pixel value of a corresponding pixel of the output image by a spreading function of a histogram spreading, wherein at least one parameter value of the camera device depending on a brightness of the environment is acquired and a limit value for the output pixel values is set in the spreading function depending on the at least one acquired parameter value.

2. The method according to claim 1, wherein the limit value of the spreading function is set such that a histogram of the pixel values of the output image has a dynamic range, which is smaller than the maximally possible dynamic range of the pixel values.

3. The method according to claim 1, wherein the limit value is set independently of a histogram of the input pixel values of the raw image.

4. The method according to claim 1, wherein by the spreading function, as the input pixel value, a brightness value and/or a color value of the pixel of the raw image are respectively mapped to an output pixel value of the corresponding pixel of the output image.

5. The method according to claim 1, wherein a minimum value is set for the output pixel values by the limit value.

6. The method according to claim 5, wherein the minimum value is set the greater, the darker the environment is.

7. The method according to claim 1, wherein the limit value is stepwise switched.

8. The method according to claim 7, wherein a first switching step is in a range from 0.3 lux to 0.8 lux, a second switching step is in a range from 1.5 lux to 2,5 lux, and a third switching step is in a range from 8 lux to 12 lux, wherein in the third switching step, the limit value is set to a minimum value or a maximum value.

9. The method according to claim 1, wherein the at least one parameter value includes a sensor value of a brightness sensor and an acquisition range of the brightness sensor is directed into the environment.

10. The method according to claim 1, wherein the at least one parameter value includes an exposure period adjusted in generating the raw image.

11. The method according to claim 1, wherein the at least one parameter value includes an image sensor gain adjusted in generating the raw image.

12. The method according to claim 1, wherein the at least one parameter value includes an aperture value adjusted in generating the raw image.

13. The method according to claim 1, wherein a temperature of the image sensor is acquired and the limit value is set depending on the acquired temperature.

14. A camera device for a motor vehicle, comprising: an image sensor for providing a raw image of an environment of the camera device; an image processing device for generating an output image from the raw image based on a histogram spreading, wherein the camera device is configured to perform a method according to claim 1.

15. A motor vehicle with at least one camera device according to claim 14.

Description

[0027] In the following, an embodiment of the invention is described. There show:

[0028] FIG. 1 a diagram for illustrating a histogram spreading according to the prior art,

[0029] FIG. 2 a schematic illustration of an embodiment of the motor vehicle according to the invention,

[0030] FIG. 3 a diagram for illustrating a histogram spreading according to the method according to the invention,

[0031] FIG. 4 a diagram for illustrating the method, wherein a limit value other than in FIG. 3 is set,

[0032] FIG. 5 a diagram with an exemplary histogram for a raw image.

[0033] The embodiment explained in the following is a preferred embodiment of the invention. However, in the embodiment, the described components of the embodiment each represent individual features of the invention to be considered independently of each other, which each develop the invention also independently of each other and thereby are also to be regarded as a component of the invention in individual manner or in another than the shown combination. Furthermore, the described embodiment can also be supplemented by further features of the invention already described.

[0034] In FIG. 1, a histogram spreading is illustrated, as it is known from the prior art. An input histogram Hi is illustrated, that is a histogram, which can be formed of pixel values of pixels of a raw image of a camera sensor. In the shown example and also in the further examples, it is assumed that each pixel can have a pixel value, which can be in a range from 0 to 255, that is an 8-bit pixel value. The pixel value can for example be a luminance value Li or hue value of the pixel. The smallest brightness value is denoted by Lmin. The greatest brightness value is denoted by Lmax. The dynamic range Di of the raw image is defined by the brightness values Lmin and Lmax.

[0035] The input histogram Hi can be formed in the manner illustrated in FIG. 5. In FIG. 5, it is presented how for the input luminance values Li of the raw image, for each possible luminance value between 0 and 255, a number C (count) of those pixels is determined, which have the corresponding luminance value Li. Hereby, the input histogram Hi results.

[0036] In FIG. 1, it is exemplified, how input luminance values Li can be mapped to output luminance values Lo by a spreading function 1 to generate from the raw image by pixel-wise transformation of the respective pixel value of each pixel of the raw image a pixel value for a corresponding pixel of the output image. Hereby, the output image arises. This output image has an output histogram Ho depending on the spreading function 1.

[0037] According to the prior art, the spreading function 1 is selected such that the smallest brightness value Lmin of the raw image is mapped to the minimum luminance value 0. The greatest brightness value Lmax of the raw image is mapped to the maximally possible luminance value, here 255. For better orientation, the identity function 2 is also registered in the diagram. By the spreading function 1, the output histogram Ho has luminance values having the full dynamic range Dmax. In other words, the maximum dynamic range Dmax includes the value 0 and the value 255.

[0038] In FIG. 2, a motor vehicle 3 is illustrated, which can for example be a passenger car. The motor vehicle 3 can have a camera device 4, which can include a camera sensor 5 and an image processing device 6. A capturing range 7 of the camera sensor 5 can for example be oriented into an environment 8 of the motor vehicle 3, for example into a front area of the motor vehicle 3 (as illustrated) or (not illustrated) into a rear area of the motor vehicle 3 or into a vehicle interior of the motor vehicle 3. In a manner known per se, the image sensor 5 can be adapted to generate an image sequence of individual raw images R. For generating the raw image R, thus a capture of the environment 8, the exposure period S (shutter speed), an aperture A (aperture) and/or an image sensor gain or gain G can be set, which then represent capturing parameters for the raw image R. In the shown example, in the raw image R of the environment 8, a road 9 located in front of the motor vehicle 3, a house 10 and a tree 11 are imaged. In the shown example, it is to be assumed that the raw image R is relatively dark. In other words, a brightness in the environment 8 can be relatively low as it can for example occur in night drive or in a drive in a tunnel. From this, an input histogram Hi for the raw image R results as it is also exemplarily illustrated in FIG. 2. The luminance values Li do not reach the maximally possible luminance value, which can here be assumed as 255.

[0039] A histogram spreading 12 can be provided by the image processing device 6. The histogram spreading 12 can for example be realized by a program module of a processor of the image processing device 6. The processor 6 can for example be a digital signal processing processor (DSP). The histogram spreading can include a parameterizable spreading function 13, which can have a limit value L (limit) as a parameter. By means of the spreading function 13, in a manner as was already described in connection with the spreading function 1 in FIG. 1, an output image I can be generated from the raw image R. Herein, the output image I can have an overall brightness, which is greater than the overall brightness of the raw image R. An output histogram Ho is exemplarily also illustrated in FIG. 2 for the output luminance values Lo of the output image I. Herein, the limit value L of the spreading function 13 constitutes a minimum value for the histogram values of the output histogram Ho.

[0040] In the motor vehicle 3, it can be provided that the output image I is transmitted to a further vehicle component 14. The vehicle component 14 can for example be a display device such as for example a screen. A driver of the motor vehicle 3 can then view the output image I on the display device and hereby see the environment 8. However, the vehicle component 14 can for example also be a driver assistance system, which can assist a driver in driving the motor vehicle 3. For example, the vehicle component 14 can then include an image recognition device, by which the road 8 and/or the house 10 and/or the tree 11 can be recognized, i.e. generally objects, in the output image I.

[0041] In FIG. 3 and FIG. 4, the operation of the histogram spreading 12 is explained for two capturing situations.

[0042] To this, in FIG. 3 and FIG. 4, each in the same manner as already explained in connection with FIG. 1, an input histogram Hi of the raw image R and an output histogram Ho of the output image I after histogram spreading 12 are shown. The histogram spreading 12 is performed by means of the spreading function 13, which can have the limit value Las a curve parameter. In the shown example, the limit value L is the minimum value for the output luminance values Lo of the pixels of the output image I. In other words, by the limit value L, an ordinate setting or ordinate displacement 15 of a lower limit U of the spreading function 13 can be determined. In the spreading function 13, an abscissa position 16 of the lower limit U and an abscissa position 17 of an upper limit O of the spreading function 13 can be set such that the lower limit U as the abscissa value corresponds to the minimum value Lmin of the input histogram Hi and the abscissa value 17 of the upper limit O corresponds to the maximum value Lmax of the input histogram Hi.

[0043] In the motor vehicle 3, the limit value L can be set depending on a brightness of the environment 8 or depending on a parameter value correlated with the brightness, for example the exposure period S and/or the gain G. To this, the exposure period S or the gain G can for example be transmitted from the image sensor 5 to the image processing device 6, as it is illustrated in FIG. 2. In the shown example, for an environmental brightness value B1, values for the closure period S and/or the gain G have arisen, by which the limit value L has been set to a value L1 by the image processing device 6. Correspondingly, an output histogram Ho results, which extends from the lower limit L1 to the maximum value, here 255, and hereby has a dynamic range Do, which is smaller than the maximum dynamic range Dmax.

[0044] In FIG. 4, a capturing situation is shown, in which a brightness value B2 of the capturing situation is less than the brightness value B1 of the capturing situation illustrated in FIG. 3. In other words, it is a capture of the raw image R at lower brightness. By the image processing device 6, in the example, the limit value was correspondingly switched from the value L1 to the value L2, which is greater than the value L1, for example by a switching operation 18. Hereby, the dynamic range Do of the output histogram Ho decreases.

[0045] Thus, limits are applied to the histogram stretching that adjust for the light levels of the environment that the imager sees. The light levels of the scene can be determined by monitoring the exposure and gain controls of the image, based on these readings we can adjust the histogram stretching limits of the image to give a brighter image at very low light and less bright and higher contrast at slightly higher light levels.

[0046] Histogram stretching increases the contrast of the image. Histogram stretching can be either enabled or disabled on the system. Enabling applies a histogram stretching to the image across the max to min grayscale. Limits can be applied to the scale of the histogram. The image processing device applies limits on the histogram that can be dynamically updated based on the exposure gain of the imager (in other words based on the light level of the scene that the imager sees). By limiting the lower level of the histogram we can make the image brighter at very low light levels (reducing contrast). The side effect of this is as the light level increases the images at light level 10 lux.

[0047] This invention will provide a limit at the lower level of the histogram at very low light (brightening the image) but dynamically updates the limit as the scene gets brighter, therefore keeping the contrast of the scene.

[0048] The desired effect of the invention is to have limit L2 at very low lightlevels (approx. 0.5 lux), and reading the exposure and gain controls of the imager dynamically update the histogram to limit 1 at a light level of 2 lux. Then once the exposure gain of the image is a value corresponding to a light level ˜10 lux or above (when brightening the image is no longer required for example) switch to an unlimited histogram stretching image (increasing contrast to the max possible).

[0049] Example Use Cases of Desired Achievement:

[0050] At a very low light level (0.5 lux) (exposure and gain values would be high) we would like to apply histogram A (limit L2). At 0.5 lux, unlimited histogram stretching has a high contrast but the image is not very bright. We would like to apply histogram A with limit 2. This applies a limit to how dark pixels can be, pulling the darker luminance values to a brighter luminance value. The desired effect of this invention is to have limit 2 Histogram at very low light levels (brightens the image). We can only brighten to a certain extent without reducing the contrast too much).

[0051] At a higher light level (2 lux) (exposure and gain values would be less than at 0.5 lux) we would like to apply histogram B (Limit L1): At 2 lux, unlimited histogram stretching provides an image with a high contrast but still a little dark. We would like to apply histogram B with limit 1 that brightens the image slightly while a high contrast level is kept up.

[0052] To dynamically update histogram limits we use the gain and exposure values of the image: [0053] imager exposure=e [0054] imager gain=g [0055] Limit 1 exposure/gain Threshold=E1 or G1 [0056] Limit 2 exposure/gain Threshold=E2 or E2 [0057] Unlimited=E3 or G3 [0058] Histogram Stretching Limit=L

[0059] Sample Logic:

TABLE-US-00001 If e > E1 or g > G1 { L = limit_1 } else if e > E2 or g > G2 { L = limit_2 } else if e > E3 or g > G3 { L = no limit = 0 }

[0060] In addition, the invention can adjust the limit in response to the imager temperature to mitigate the influence of noise into the limit calculation.

[0061] Overall, the example shows, how a histogram spreading with dynamic limitation can be provided by the invention to adjust the limitation depending on the image sensor exposure period and/or the image sensor gain.