STREET MARKING COLOR RECOGNITION

Abstract

The present invention relates to a method and an image processing system for determining the color of a street marking, by: capturing an image of a street, detecting a street marking as a set of pixels provided by the image, wherein the pixels include at least two different pieces of color information, determining a color score for the street marking by comparing said at least two different pieces of color information, and determining the color of the street marking by comparing the color score to at least one threshold value.

Claims

1. A method for determining the color of a street marking, comprising: capturing an image of a street; detecting a street marking as a set of pixels provided by the image, wherein the pixels include at least two different pieces of color information; determining a color score for the street marking by comparing said at least two different pieces of color information; and determining the color of the street marking by comparing the color score to at least one threshold value.

2. The method of claim 1, wherein a red-clear camera is used for capturing the image such that the image has pixels with red color information and clear color information, wherein the clear color information corresponds to pixels having no red, green or blue color distinction.

3. The method of claim 1, wherein the color score is determined by calculating proportions based on a comparison of the at least two different pieces of color information.

4. The method of claim 3, wherein the color score is determined by using a training-based classifier.

5. The method of any of claim 1, further comprising: gathering the color scores determined for a plurality of street markings; and determining the at least one threshold value based on the statistical distribution of the color scores.

6. The method of claim 5, wherein the color scores are gathered based on a plurality of street markings detected in a plurality of images, wherein the color scores are gathered and the at least one threshold value is determined during capturing of the video stream taken from a vehicle.

7. The method of claim 5, wherein the threshold value is determined based on the statistical distribution being modelled as a superposition of a plurality of statistical color distributions, wherein each of the statistical color distributions corresponds to a color of a street marking, wherein the statistical distribution is modelled as a superposition of a plurality of Gaussian statistical color distributions.

8. The method of claim 1, wherein the color of the street marking is determined to correspond to a first color if the color score is greater than a threshold value, and the color of the street marking is determined to correspond to a second color if the color score is smaller than the threshold value.

9. The method of claim 8, wherein the first color and the second color correspond to the colors of a street lane, wherein the first color and the second color correspond to the colors yellow and white, or white and yellow, respectively.

10. An image processing system, comprising: a camera adapted to capture an image of a street; an image processing means coupled to the camera; wherein the image processing means: detects a street marking as a set of pixels provided by the image, wherein the pixels include at least two different pieces of color information; determines a color score for the street marking by comparing said at least two different pieces of color information; and determines the color of the street marking by comparing the color score to at least one threshold value.

11. The image processing system of claim 10, wherein the camera is a red-clear camera for capturing images having pixels with red color information and clear color information, wherein the clear color information corresponds to pixels having no red, green or blue color distinction.

12. The image processing system of claim 10, wherein the image processing means is adapted to gather the color scores determined for a plurality of street markings, and to determine the at least one threshold value based on the statistical distribution of the color scores

13. The image processing system of claim 12, wherein the image processing means is adapted to gather the color scores based on a plurality of street markings detected in a plurality of images captured by the camera, wherein the color scores are gathered and the at least one threshold value is determined during capturing of the video stream taken from a vehicle.

14. The image processing system of claim 12, wherein the image processing means is adapted to determine the at least one threshold value based on modelling the statistical distribution as a superposition of a plurality of statistical color distributions, wherein each of the statistical color distributions corresponds to a color of a street marking, wherein the statistical distribution is modelled as a superposition of a plurality of Gaussian statistical color distributions.

15. The image processing system of claim 10, wherein the image processing means is adapted to determine that the color of the street marking corresponds to a first color if the color score is greater than the threshold value, and to determine that the color of the street marking corresponds to a second color if the color score is smaller than the threshold value

16. The image processing system of claim 15, wherein the first color and the second color correspond to the colors yellow and white, or white and yellow, respectively.

17. The image processing system of claim 10, wherein the camera is configured to be mounted on a vehicle.

18. The image processing system of claim 10, wherein the color of a street marking is the color of a street lane marking

19. The image processing system of claim 18, wherein the color of a street marking is a yellow color or a white color.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0049] Further embodiments of the invention are described in the following description of the Figures. The invention will be explained in the following by means of embodiments and with reference to the drawings in which is shown:

[0050] FIG. 1 is a street marking scenario;

[0051] FIG. 2 is a method for determining the color of a street marking;

[0052] FIG. 3 is a red-clear type camera pixel arrangement;

[0053] FIG. 4 is color score statistical distribution functions; and

[0054] FIG. 5 is an image processing system.

DETAILED DESCRIPTION

[0055] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0056] One or more includes a function being performed by one element, a function being performed by more than one element, e.g., in a distributed fashion, several functions being performed by one element, several functions being performed by several elements, or any combination of the above.

[0057] It will also be understood that, although the terms first, second, etc. are, in some instances, used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact.

[0058] The terminology used in the description of the various described embodiments herein is for describing embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms a, an and the are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term and/or as used herein refers to and encompasses all possible combinations of one or more of the associated listed items. It will be further understood that the terms includes, including, comprises, and/or comprising, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

[0059] As used herein, the term if is, optionally, construed to mean when or upon or in response to determining or in response to detecting, depending on the context. Similarly, the phrase if it is determined or if [a stated condition or event] is detected is, optionally, construed to mean upon determining or in response to determining or upon detecting [the stated condition or event] or in response to detecting [the stated condition or event], depending on the context.

[0060] FIG. 1 shows a street marking scenario 100, wherein a vehicle 110 having a camera 120 is travelling along a road with white street markings 130. The camera 120 is mounted behind the rear-view mirror of the vehicle 110, such as to be provided with an unobstructed front view. The camera could also be mounted on a front part, rear part or side part of the vehicle 110, for example to provide a different viewpoint on the street markings.

[0061] The white street markings 130 are arranged to define the road edges and to separate two road lanes. Thus, the white street markings 130 provide lane markings. The white street markings can also indicate road exits, road crossings or parking zones, depending on the road scenario under consideration.

[0062] In FIG. 1, a road construction zone 140 with a construction machine 150 is shown to overlap one of the white street marking 130, and to extend inside the street lane being used by the vehicle 110.

[0063] Such as to improve safety in the road construction zone 140, yellow street markings 160 have been provided to override the white street markings 130. Thus, the yellow street markings are arranged to guide the vehicle 110 to pass the construction zone 140 at a safe distance. For example, according to European traffic rules, when yellow and white color lane markings are being used, the yellow lane 160 marking should be treated with higher priority to override directions provided by the white lane marking.

[0064] It follows that a correct color extraction is required to distinguishing between the different street markings based on their colors, for example to enhance safety where yellow street markings 160 have been provided to override white street markings 130, for example at construction zones 140.

[0065] FIG. 2 shows a corresponding method for determining the color of a street marking 200. An image of the street is captured 210, and a street marking 130, 160 is detected as a set of pixels provided by the image 220, wherein the pixels include at least two different pieces of color information.

[0066] For example, the set of pixels provided by the image can be provided by an RGB camera 120, i.e. a camera for capturing images having pixels with red, green and blue color information. Alternatively, the set of pixels provided by the image can be provided by a red-clear type camera 120, i.e. a camera for capturing images having pixels with red color information and clear color information.

[0067] For example, FIG. 3 shows a pixel arrangement of a red-clear camera, with pixels 310 which are adapted to capture red light information, whereas the remaining pixels 320 have no R/G/B filters provided on top of them. In other words, images captured by red-clear cameras do not provide full R/G/B color information.

[0068] In step 230 shown in FIG. 2, the at least two different pieces of color information are compared to determine a color score for the street marking. The color score can represent a value indicating the detected color, for example, as detailed above, indicating a proportion of color information to describe the yellowness of the street marking.

[0069] For example, the color score can be determined by calculating proportions or other dependencies, based on a comparison of the different pieces of color information provided by the set of pixels, for example by using a training-based classifier, such as for example a neural network or a Fuzzy logic based classifier. In an example, the classifier can be trained by using a machine learning based training data set, for example with labeled white and yellow lane markings, such as to train the classifier to calculate a color score based on the different pieces of color information provided by the set of pixels.

[0070] Then, in step 240 shown in FIG. 2, the color score is compared to at least one threshold value to determine the color of the street marking.

[0071] For example, when using the red-clear camera, it is not possible to distinguish all visible spectrum colors. For example, it is not possible to distinguish the color yellow from pure red, or, e.g. magenta, which is a combination of red and blue.

[0072] However, as the street markings 130, 160 shown in FIG. 1 are either white or yellow, the red-channel information can be compared to the clear color information, for example to determine the proportion of the color information, and the resulting color score can used to distinguish between the two colors.

[0073] More specifically, such as to distinguish between the white and yellow colors of the street markings, the color score is compared to a threshold value.

[0074] In this way, the color score is determined based on a comparison of the different pieces of color information, and thus may compensate illumination affects that affect the different colors in the same or similar way. For example, changes in brightness and illumination that affect the different colors in the same or similar way can be compensated by comparing the different pieces of color information, for example by determining the proportion of color information.

[0075] Moreover, by gathering color scores determined for the street marking, the threshold value can be determined based on the statistical distribution of the gathered color scores. For example, the color scores can be gathered based on street markings detected in a plurality of images, for example in a plurality of image frames of a video stream captured by the camera 120.

[0076] In this way, the statistical distribution of the determined color score can be analyzed and estimated. For example, if in multiple frames, there is only one color (e.g. either white or yellow) of captured lane markings, the statistical distribution of the determined color score can be efficiently modelled for classification purposes, for example as a single Gaussian statistical distribution.

[0077] FIG. 4 shows color score statistical distribution functions 400 wherein a Gaussian statistical distribution for the color white 410 is defined as a Gaussian probability density function 420 of the color score. By repeating the analysis for a yellow lane marking that is captured in multiple frames of a video stream, the corresponding Gaussian statistical distribution for the color yellow 423 results. Thus, the two different colors correspond to the two different Gaussian statistical distribution functions 410, 420 shown in FIG. 4.

[0078] It follows that the relative difference between the two different colors can be described in statistical terms, and the corresponding threshold value 440 can be determined based on probability calculations derived from the different color score statistical distributions 410, 430.

[0079] In other words, the threshold value can be determined based on the statistical distribution of the color score being modelled as a superposition of a plurality of statistical color distributions, wherein each of the statistical color distributions corresponds to a color of a street marking.

[0080] Then, the color of the street marking is determined to correspond to a yellow color if the color score is greater than the threshold value, and the color of the street marking is determined to correspond to a white color if the color score is smaller than the threshold value, or vice-versa.

[0081] As mentioned above, if more than two colors need to be detected, the color score is compared to n threshold values such as to decide between n+1 different colors.

[0082] Accordingly, at least one threshold value is used to correctly distinguish between the different colors based on a comparison with a color score. Hence, the color score is derived by comparison of different pieces of color information, and thus can, in combination with the comparison with the at least one threshold value, enhance the robustness and efficiency of the color detection, in particular when changing image conditions affect the intensities of each of the pieces of color information in the same or similar manner.

[0083] FIG. 5 shows an image processing system 500 that is adapted to perform the method 200 shown in FIG. 2. For this purpose, the image processing system 500 comprises the camera 510, 120 for capturing the image of the street, and an image processing means 520 coupled to the camera 510. The image processing means is adapted to perform the above steps of: detecting a street marking as a set of pixels provided by the image, wherein the pixels include at least two different pieces of color information, determining a color score for the street marking by comparing said at least two different pieces of color information, and determining the color of the street marking by comparing the color score to at least one threshold value 440.

[0084] Here, the image processing means 520 includes computing means 530, such as for example a microprocessor, coupled to storage means 540, wherein the storage means 540 includes software adapted to be executed by the microprocessor, such as to perform the method steps shown in FIG. 2.

[0085] While this invention has been described in terms of the preferred embodiments thereof, it is not intended to be so limited, but rather only to the extent set forth in the claims that follow.