SYSTEM AND METHOD FOR OBJECT RECOGNITION UNDER NATURAL AND/OR ARTIFICIAL LIGHT

Abstract

Described herein are a system and a method for object recognition via a computer vision application, the system including at least the following components: at least one object to be recognized, the object having object specific reflectance and luminescence spectral patterns, a light source which is configured to illuminate a scene including the at least one object, the light source being designed to omit at least one spectral band of a spectral range of light when illuminating the scene, the at least one omitted spectral band being in the luminescence spectral pattern of the at least one object, at least one sensor which is configured to exclusively measure radiance data of the scene in at least one of the at least one omitted spectral band when the scene is illuminated by the light source, a data storage unit, and a data processing unit.

Claims

1. A system for object recognition via a computer vision application, the system comprising at least the following components: at least one object to be recognized, the object having object specific reflectance and luminescence spectral patterns, a light source which is configured to illuminate a scene including the at least one object, the light source being designed to omit at least one spectral band of a spectral range of light when illuminating the scene, the at least one omitted spectral band being in the luminescence spectral pattern of the at least one object, at least one sensor which is configured to exclusively measure radiance data of the scene in at least one of the at least one omitted spectral band when the scene is illuminated by the light source, a data storage unit which comprises luminescence spectral patterns together with appropriately assigned respective objects, and a data processing unit which is configured to extract the object specific luminescence spectral pattern of the at least one object to be recognized out of the measured radiance data of the scene and to match the extracted object specific luminescence spectral pattern with the luminescence spectral patterns stored in the data storage unit, and to identify a best matching luminescence spectral pattern and, thus, its assigned object.

2. The system according to claim 1, wherein the light source is a LED light source which is configured to intentionally and intrinsically leave out the at least one spectral band of the spectral range of light when illuminating the scene.

3. The system according to claim 2, wherein the LED light source is configured to omit a plurality of spectral bands and composed of a plurality of narrow band LEDs, each LED being configured to emit light in a narrow spectral band, the spectral bands of the LEDs being spaced apart from each other with the omitted spectral bands in between them.

4. The system according to claim 1, wherein the light source is equipped with at least one light filter, the at least one light filter being designed to block the at least one spectral band of the spectral range of light from entering the scene.

5. The system according to claim 4, wherein the at least one light filter is designed as a dynamic light filter which is configured to block at least one spectral band of light from entering the scene at a time and to change the at least one spectral band which is to be blocked dynamically, thus blocking at least one portion of the spectral range of light over time.

6. The system according to claim 5, wherein the dynamic light filter is configured to continuously operate over the light spectral range of interest and to provide blocking of at least one of the at least one spectral band of interest on demand.

7. The system according to claim 4 which comprises a plurality of dynamic light filters on the same natural and/or artificial light source and/or on multiple natural and/or artificial light sources illuminating the scene, wherein the filters are configured to be synchronized with each other to block at least a portion of the same one of the at least one spectral band simultaneously.

8. The system according to claim 4, wherein the at least one light filter is designed as a notch filter which is configured to block light entering the scene from a window as in natural lighting or an artificial lighting element at the at least one distinct spectral band continuously.

9. The system according to claim 8, wherein the notch filter is designed to block a plurality of distinct spectral bands within the spectral range of light.

10. The system according to claim 1, wherein the at least one sensor is a camera which is configured to image the scene and to record radiance data over the scene exclusively at different spectral bands of the at least one spectral band of the spectral range of light at time intervals when the scene is illuminated by the light source.

11. The system according to claim 10, wherein the sensor is a hyperspectral camera or a multispectral camera.

12. The system according to claim 1, wherein the data processing unit is configured to calculate the object specific luminescence spectral pattern of the at least one object to be recognized based on the spectral distribution of the measured radiance data of the scene and to match the calculated object specific luminescence spectral pattern with the luminescence spectral patterns stored in the data storage unit, and to identify a best matching luminescence spectral pattern and, thus, its assigned object.

13. A method for object recognition via a computer vision application, the method comprising at least the following steps: providing an object to be recognized, the object having object specific reflectance and luminescence spectral patterns, illuminating a scene including the object using a light source, the light source being designed to omit at least one spectral band of a spectral range of light when illuminating the scene, the at least one omitted spectral band being in the luminescence spectral pattern of the at least one object, measuring, by means of at least one sensor, radiance data of the scene exclusively at the at least one omitted spectral band when the scene is illuminated by the light source, providing a data storage unit which comprises luminescence spectral patterns together with appropriately assigned respective objects, extracting, by means of a data processing unit, the object specific luminescence spectral pattern of the object to be recognized out of the measured radiance data of the scene, matching the extracted object specific luminescence spectral pattern with the luminescence spectral patterns stored in the data storage unit, and identifying a best matching luminescence spectral pattern and, thus, its assigned object.

14. The method according to claim 13, wherein the light source is chosen as a LED light source which is configured to intentionally and intrinsically leave out the at least one spectral band of the spectral range of light when illuminating the scene.

15. The method according to claim 13, wherein the light is source is equipped with at least one light filter, the at least one light filter being designed to block the at least one spectral band of the spectral range of light from entering the scene.

16. The method according to claim 15, further comprising choosing the at least one light filter as a dynamic filter and operating over the light spectral range of interest and providing blocking of at least one of the at least one spectral band of interest on demand.

17. The method according to claim 15, further comprising choosing the at least one light filter as a notch filter which is configured to block light from entering the scene from a window as in natural lighting or an artificial lighting element at the at least one distinct spectral band continuously.

18. The method according to claim 13, further comprising calculating the object specific luminescence spectral pattern of the at least one object to be recognized based on the spectral distribution of the at least one omitted spectral band and the measured radiance data of the scene and matching the calculated object specific luminescence spectral pattern with the luminescence spectral patterns stored in the data storage unit, and identifying a best matching luminescence spectral pattern and, thus, its assigned obj ect.

19. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause a machine to: provide an object to be recognized, the object having object specific reflectance and luminescence spectral patterns, illuminate a scene including the object using a light source, the light source being designed to omit at least one spectral band of a spectral range of light when illuminating the scene, the at least one omitted spectral band being in the luminescence spectral pattern of the at least one object, measure radiance data of the scene exclusively at the at least one omitted spectral band when the scene is illuminated by the light source, provide a data storage unit which comprises luminescence spectral patterns together with appropriately assigned respective objects, extract the object specific luminescence spectral pattern of the object to be recognized out of the measured radiance data of the scene, match the extracted object specific luminescence spectral pattern with the luminescence spectral patterns stored in the data storage unit, and identify a best matching luminescence spectral pattern and, thus, its assigned object.

20. The computer-readable medium according to claim 19, further storing instructions to calculate the object specific luminescence spectral pattern of the at least one object to be recognized based on the spectral distribution of the at least one omitted spectral band and the radiance data of the scene and to match the calculated object specific luminescence spectral pattern with the luminescence spectral patterns stored in the data storage unit, and to identify a best matching luminescence spectral pattern and, thus, its assigned object.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0076] FIG. 1a shows a schematic diagram of an unfiltered illuminant spectrum and a notch filter transmission spectrum;

[0077] FIG. 1b shows a schematic diagram of a resulting illuminant spectrum after filtration, i.e. a superimposing of the unfiltered illuminant spectrum and the notch filter transmission spectrum of FIG. 1a.

[0078] FIG. 2 shows a schematic diagram of a notch filter transmission spectrum and one sensor band being located within each notch filter blocking band,

[0079] FIG. 3 shows a schematic diagram of a notch filter transmission spectrum and multiple sensor bands being located within each notch filter blocking band

[0080] FIG. 4 shows schematically one embodiment of the proposed system.

DETAILED DESCRIPTION OF THE DRAWINGS

[0081] FIG. 1 a shows a diagram with a horizontal axis 101 and two vertical axes 102 and 103. The diagram is shown for an embodiment of the proposed system for object recognition via a computer vision application. The system comprises at least a natural and/or artificial light source which comprises at least one illuminant to illuminate a scene including at least one object to be recognized.

[0082] The at least one object to be recognized has object-specific reflectance and luminescence spectral patterns. The light source is equipped with at least one notch filter which is designed to block at least one predefined spectral band within a spectral range of light from entering the scene wherein the at least one filtered spectral band lies within the luminescence spectral pattern, i.e. the luminescent spectral range of the at least one object. The wavelength of the spectral range of light is plotted along the horizontal axis 101. The transmission of the notch filter is plotted along the vertical axis 103, wherein the transmission is given in percent. A radiance intensity of the light source, i.e. of the illuminant comprised by the light source, is plotted along the vertical axis 102. The curve 110 indicates the developing of radiance intensity values of the light source as a function of the wavelength, and the curve 111 indicates the transmission of the notch filter as a function of the wavelength. Thus, in the diagram of FIG. 1a an unfiltered illuminant spectrum and a notch filter transmission spectrum are plotted as respective functions of the wavelength independently from each other.

[0083] FIG. 1b shows a diagram wherein the curves 110 and 111 from FIG. 1a are superimposed with each other forming curve 120, thus, indicating which spectral bands are filtered/blocked from entering the scene. As already indicated above, the filtered spectral bands are chosen as being correlated with the luminescence spectral pattern of the object to be recognized so that radiance data resulting from those spectral bands (blocked from illumination) and measured by the sensor have to be unambiguously assigned to the luminescence spectral pattern of the at least one object and, therefore, give clear indication of the at least one object. The notch filter shown here blocks five spectral bands along the wavelength range which is plotted along the horizontal axis 101. As no light within the blocked spectral bands can enter the scene, no light within those spectral bands can be reflected and, therefore, all light which can be sensed/measured by the sensor within those spectral bands has to be resulting from the luminescence spectral pattern of the at least one object.

[0084] FIG. 2 shows a schematic diagram for a system which comprises the light source, the notch filter and a respective sensor which is configured to measure radiance data of the scene including the at least one object when the scene is illuminated by the light source. The diagram has a horizontal axis 201 and two vertical axes 202 and 203. The wavelength of light entering the scene and of light being radiated from the scene is plotted along the horizontal axis 201. A sensor sensitivity is plotted along the vertical axis 202. A transmission capability of the notch filter is plotted along the axis 203. The transmission is given in percent. The notch filter is chosen as a multiband notch filter, i.e. the notch filter is configured to block multiple spectral bands of the spectral range of light from entering the scene, the spectral range of light being defined by the beginning and the end of the horizontal axis 201. In the case shown here, the notch filter blocks, as indicated by curve 210, five spectral bands along the spectral range of light defined by the horizontal axis 201. The sensor is configured, as indicated by curve 220, to particularly measure radiance data whithin exactly the five spectral bands of the spectral range of light which are blocked by the notch filter from entering the scene. Therefore, the sensor is explicitly configured to sense only light which is emitted from the scene as luminescent response to the entering light. The reflected response of the scene is masked as the sensor is not configured to measure radiance data within the spectral bands which are not blocked by the notch filter. Therefore, it is possible to focus the measurement made by the sensor on the luminescence response of the scene. If the spectral bands of the notch filter which are blocked are adapted to the luminescence spectral pattern of the at least one object to be recognized, the sensor can clearly measure the radiance data resulting from the luminescence spectral pattern of the object and enables to clearly identify the object due to its measured luminescence spectral pattern.

[0085] FIG. 3 shows a further example of diagram. The wavelength of light entering a scene or emitting from the scene is plotted along the horizontal axis 301. A sensor sensitivity is again plotted along vertical axis 302. A transmission capacity of a notch filter is again plotted again a vertical axis 303. Within the wavelength range defined by the horizontal axis 301, the notch filter has two spectral bands which are blocked and three spectral bands which are not blocked as indicated by curve 310. In the example shown here, the sensor is configured to measure two spectral bands within each blocked spectral band of the notch filter as indicated by curve 320. That means that multiple sensor bands are located within each notch filter band, i. e. within each spectral band which is blocked by the notch filter. In the case that the sensor with its sensor bands is chosen to be adapted/to correlate with the luminescence spectral pattern of the object to be recognized and the notch filter with its blocking spectral bands is also adapted to the luminescence spectral pattern of the object, the object can be unambiguously identified due to its luminescence spectral pattern which can be measured in detail by the respective sensor.

[0086] Methods for measuring a fluorescence emission spectrum from an object containing fluorescence emission and reflectance are already known. Most of these methods rely on measuring a radiance spectrum of the object under two or more lighting conditions which have to be known and using various calculations to separate out the reflection and emission contribution to the total radiance of the object. However, using multiple lighting conditions is not ideal for non-laboratory environments, as the additional lighting conditions increase the cost of the light sources and add complexity challenges in syncing the light source to the sensors used. There is one paper describing a separation fluorescence emission and reflectance under a single lighting condition (Zheng, Fu, Lam, Sato, and Sato, ICCV2015 3523-3531). Within this paper, a “spiky” illumination source, i.e. a high-intensity discharge bulb used principally for automotive headlights, is used. Thus, there is still a need for generalizable methods and systems for separating reflectance and fluorescence emission under single light source conditions.

[0087] The proposed system and method enable to intentionally create dark regions in an illumination spectrum and to then measure a radiance within those dark regions. Objects with no fluorescence will not register a radiance within the dark regions, as there is no illumination for them to reflect at these wavelengths. Objects with fluorescence emission that overlaps the dark regions will have a radiance due to the conversion of higher energy light. These dark regions can be created by the application of notch filters, which are filters that transmit most of the light over their effective range with an exception of a relatively small portion of the spectrum, which should be as close to zero transmission as possible. Notch filters, including filters with multiple “notches” in a single filter, are commercially available. It is proposed to apply notch filters to illumination sources such as light bulbs and outside windows to create an environment/a scene in which an object is to be recognized. A sensor, particularly a camera with spectral sensitivity within the dark regions of the illuminant spectrum is also required. To get a fluorescence spectral shape, either multiple dark regions (FIG. 2) or a larger dark region with multiple sensor bands within that region (FIG. 3) will be required. Additionally, dynamic notch filters, where the “notch” portion of the spectrum is changeable over time, may be available. With dynamic notch filters an entire spectrum can be scanned over time, allowing for better identification of a fluorescence spectrum of a respective object to be recognized.

[0088] FIG. 4 shows an embodiment of the proposed system. The system 400 comprises an object to be recognized 420, a light source 410, a sensor 440, a data storage unit 460 and a data processing unit 450. The object 420 has an object specific reflectance spectral pattern and an object specific luminescence spectral pattern. The light source 410 is configured to emit UV, visible or infrared light in a spectral range of light. Generally, it is possible that the light source 410 is configured to emit light spanning the whole spectral range of light. In that case, the light source is coupled/equipped with a light filter 415 which is designed to block at least one individual spectral band of the spectral range of light from entering a scene 430 including the object 420 when the light source 410 emits light towards the scene 430. The light source 410 may also be the sun and the filter 415 can be a window fitted with filters and optionally with a sensor, such as a camera 440 (see FIG. 4). The at least one individual spectral band which is blocked lies within the luminescence spectral pattern of the object 420. Alternatively, the light source 410 is designed to leave out intrinsically at least one individual spectral band, i.e. the light source 410 does not emit light within said individual spectral band when illuminiating the scene 430 including the object 420. According to one possible embodiment of the system, the light source is a LED light source which is configured to intentionally and intrinsically leave out (omit) the at least one spectral band of the spectral range of light when illuminating the scene. The LED light source can be composed of a plurality of narrow band LEDs, each LED being configured to emit light in a narrow spectral band, the spectral bands of the LEDs being spaced apart from each other with omitted spectral bands in between them.

[0089] A combination of such light source with a filter is also possible. The system 400 shown in FIG. 4 further comprises a sensor 440 which is configured to sense/record radiance data/responses over the scene 430 at the at least one spectral band which has been left out when illuminating the scene 430. That means that only a fluorescent response of the scene 430 including the object 420 to be recognized is recorded, i.e. the fluorescent response of the object 420 provided that no further item with a similar fluorescence spectral pattern is present within the scene. The system 400 further comprises a data processing unit 450 and a data storage unit 460. The data storage unit comprises a database of fluorescence spectral patterns of a plurality of different objects. The data processing unit is in communicative connection with the data storage unit and also with the sensor 440. Therefore, the data processing unit 450 can calculate the luminescence emission spectrum of the object 420 to be recognized and search the database 460 of the data storage unit for a match with the calculated luminescence emission spectrum. Thus, the object 420 to be recognized can be identified if a match within the database 460 can be found.

LIST OF REFERENCE SIGNS

[0090] 101 horizotal axis

[0091] 102 vertical axis

[0092] 103 vertical axis

[0093] 110 curve

[0094] 111 curve

[0095] 120 curve

[0096] 201 horizontal axis

[0097] 202 vertical axis

[0098] 203 vertical axis

[0099] 210 curve

[0100] 220 curve

[0101] 301 horizontal axis

[0102] 302 vertical axis

[0103] 303 vertical axis

[0104] 310 curve

[0105] 320 curve

[0106] 400 system

[0107] 410 light source

[0108] 415 filter

[0109] 420 object to be recognized

[0110] 430 scene

[0111] 440 sensor

[0112] 450 data processing unit

[0113] 460 data storage unit