Multispectral or Hyperspectral Imaging and Imaging System Based on Birefringent Subwavelength Resonating Structure
20180107015 · 2018-04-19
Inventors
Cpc classification
G02F1/137
PHYSICS
G02B27/288
PHYSICS
G02F1/0136
PHYSICS
G02B26/007
PHYSICS
International classification
Abstract
An angle-stable, miniaturized and integrate-able imaging system based on plasmon resonances or dielectric resonances for multispectral imaging maintaining full spatial resolution of the image sensor. Active tunability of the filter allows color recording, estimation of unknown spectra and determination of spectral singularities, for example laser lines, with the use of a conventional B/W camera. The system is characterized by high angular acceptance, cost-efficient fabrication and ease-of-use. This system can be used in conjunction with other commercial multispectral imaging systems such as RGB cameras to further enhance the spectral resolution. It can be adapted to different spectral ranges, depending on the application.
Claims
1. Tunable optical filter comprising: an input polarizer for receiving incident electromagnetic radiation incident from an object, the input polarizer being configured to produce linearly polarized electromagnetic radiation; a phase retarder arranged to receive the linearly polarized electromagnetic radiation having first and second polarization components on two orthogonal axes, the phase retarder being configured to carry out a wavelength dependent shift of a phase of the first and/or second polarization components; wherein the phase retarder is a phase retarder comprising at least a plurality of periodically substantially aligned subwavelength elongated structures configured to generate localized surface plasmon resonances or subwavelength dielectric resonances at a first resonance frequency when the first or second polarization component is incident on the phase retarder, and an analyzing polarizer arranged to receive the electromagnetic radiation output from the phase retarder, the analyzing polarizer being configured to filter said received electromagnetic radiation at a plurality of different polarization state angles .sub.m to output electromagnetic radiation having distinct spectral content at each polarization state angle .sub.m.
2. Tunable optical filter comprising: an analyzing polarizer for receiving electromagnetic radiation incident from an object, the analyzing polarizer being configured to filter said incident electromagnetic radiation at a plurality of different polarization state angles .sub.m and to output electromagnetic radiation at each polarization state angle .sub.m to output electromagnetic radiation from the tunable optical filter having distinct spectral content at each polarization state angle .sub.m; a phase retarder arranged to receive, from the analyzing polarizer, linearly polarized electromagnetic radiation having first and second polarization components on two orthogonal axis, the phase retarder being configured to carry out a wavelength dependent shift of a phase of the first and/or second polarization components; wherein the phase retarder is a phase retarder comprising at least a plurality of periodically substantially aligned subwavelength elongated structures configured to generate localized surface plasmon resonances or subwavelength dielectric resonances at a first resonance frequency when the first or second polarization component is incident on the phase retarder, and an output polarizer arranged to receive the electromagnetic radiation output from the phase retarder, the output polarizer being configured to filter said received electromagnetic radiation to output linearly polarized electromagnetic radiation.
3. Tunable optical filter according to claim 1, wherein the phase retarder is a phase retarder comprising a plurality of periodically substantially aligned subwavelength elongated structures configured to generate localized surface plasmon resonances or subwavelength dielectric resonances at a first resonance frequency and/or at a second resonance frequency.
4. Tunable optical filter according to claim 1, wherein the phase retarder is a plasmonic phase retarder comprising at least a first plurality of periodically substantially aligned subwavelength elongated structures configured to generate localized surface plasmon resonances at first and second resonance frequencies and a second plurality of periodically substantially aligned subwavelength structures configured to generate localized surface plasmon resonances at third and fourth resonance frequencies.
5. Tunable optical filter according to claim 1, wherein the plurality of periodically substantially aligned subwavelength elongated structures includes subwavelength metallic nanostructures having a metallic thickness value in the range of 5 nm to 100 nm, and a separation trench separating the metallic nanostructures having a depth in the range of 30 nm to 250 nm to provide filtering in the visible spectrum.
6. Tunable optical filter according to claim 1, wherein the plurality of periodically substantially aligned subwavelength elongated structures includes a plurality of periodically substantially aligned subwavelength metallic nanostructures including subwavelength metallic nanostructures having a periodic separation in the range of 140 nm to 200 nm to provide an angle independent response of the phase retarder.
7. Tunable optical filter according to claim 1, wherein the plurality of periodically substantially aligned subwavelength elongated structures includes a plurality of periodically substantially aligned subwavelength metallic nanostructures including a first set and a second set of periodically aligned subwavelength metallic nanostructures, the first and second set of periodically aligned subwavelength metallic nanostructures comprising subwavelength metallic nanostructures having a different orientation, and a different metallic thickness value, and a different periodic separation, and a different separation trench depth.
8. Tunable optical filter according to claim 1, wherein the input polarizer, the phase retarder and the analyzing polarizer are connected together to form an integrated compact device; or the analyzing polarizer, the phase retarder and the output polarizer are connected together to form an integrated compact device.
9. Tunable optical filter according to claim 2, wherein the analyzing polarizer, the phase retarder and the output polarizer are connected together to form an integrated compact device.
10. Smart phone or imaging device including the tunable optical filter according to claim 1.
11. Multispectral or hyperspectral imaging system comprising: a tunable optical filter according to claim 1; and an image sensor including a plurality of pixels, the image sensor being arranged downstream from the tunable optical filter and is configured to record the electromagnetic radiation intensity V.sub.pixel (.sub.m) of the electromagnetic radiation output from the output polarizer or the analyzing polarizer at each polarization state angle.
12. Multispectral or hyperspectral imaging system according to claim 11, wherein the tunable optical filter is configured to homogenously filter electromagnetic radiation incident on a plurality of pixels of the image sensor.
13. Multispectral or hyperspectral imaging system according to claim 12, wherein the tunable optical filter is configured to homogenously filter electromagnetic radiation incident on the full field of view of the image sensor.
14. Multispectral or hyperspectral imaging system according to claim 10, further including a processor configured to calculate a reflected spectra R.sub.pixel(.sub.n) of the incident electromagnetic radiation from the object for each pixel of the image sensor and for a plurality of wavelengths .sub.n based on the following equation:
15. Multispectral or hyperspectral imaging system according to claim 14, further including a memory configured to store values for (i) the sensitivity of the image sensor S(.sub.n), (ii) the object illuminant I(.sub.n), and (iii) the transmission spectra of the phase retarder T.sub.SRTF (.sub.n, .sub.m) as a function of polarization state angle .sub.m; and wherein the processor is further configured to calculate the equation A(.sub.n, .sub.m)=S(.sub.n)*I(.sub.n)*T.sub.SRTF(.sub.n, .sub.m) based on these stored values.
16. Multispectral or hyperspectral imaging system according to claim 15, wherein the processor is further configured to carry out the calculation of solving the equation Vpixel(.sub.m)=A(.sub.n, .sub.m)*Rpixel (.sub.n) to determine the spectra R.sub.pixel(.sub.n).
17. Multispectral or hyperspectral imaging system according to claim 16, wherein the processor is further configured to calculate a multispectral or hyperspectral image based on the determined spectra R.sub.pixel(.sub.n).
18. Multispectral or hyperspectral imaging system according to claim 10, wherein the image sensor comprises a one-channel or Black and white camera, a RGB camera or a multispectral camera.
19. Multispectral or hyperspectral imaging system according to claim 10, wherein the imaging system is configured to determine the polarization state angles for a plurality of wavelengths .sub.n.
20. Spectrometer comprising: a tunable optical filter according to claim 1; and a photodiode arranged downstream from the tunable optical filter and configured to record the electromagnetic radiation intensity of the electromagnetic radiation output from the analyzing polarizer or output polarizer at each polarization state angle.
21. Spectrometer according to claim 20, further including a plurality of photodiodes each associated with a broadband optical filter.
Description
A BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
[0067]
[0068]
[0069]
[0070]
[0071]
[0072]
[0073]
[0074]
[0075]
[0076]
[0077]
[0078]
[0079]
[0080]
[0081]
[0082]
[0083]
[0084]
[0085]
[0086]
[0087]
[0088]
[0089]
[0090]
[0091]
[0092]
[0093]
[0094] Herein, identical reference numerals are used, where possible, to designate identical elements that are common to the figures.
DETAILED DESCRIPTION OF THE SEVERAL EMBODIMENTS
[0095]
[0096] The imaging system SRMSI can be, for example, a multispectral imaging or hyperspectral imaging system. The tunable optical filter SRTF can include an entrance or input polarizer 1, a phase retarder or a birefringent subwavelength resonating structure (BSRS) 2, and an analyzing polarizer 3. The imaging system SRMSI includes the tunable optical filter SRTF and an imaging sensor, imaging system or device, camera, or photodiode 4.
[0097] The input polarizer 1 is configured to receive incident electromagnetic radiation IR incident from an object OB, and is configured to produce, for example, linearly polarized electromagnetic radiation.
[0098] The phase retarder or birefringent subwavelength resonating structure (BSRS) 2 includes an anisotropic metallic or dielectric structure, for example, periodic and with spatial periodicity well below the wavelength or wavelength range of the incident light.
[0099] The phase retarder or BSRS 2 receives, for example, linearly polarized electromagnetic radiation having first and second polarization components on, for example, two orthogonal axes from the input polarizer 1. The phase retarder or BSRS 2 is configured to carry out a wavelength dependent shift of a phase of the first and/or second polarization components.
[0100] The phase retarder or BSRS 2 may comprise a plurality of periodically aligned subwavelength structures (for example, elongated structures) configured to generate localized surface plasmon resonances or subwavelength dielectric resonances at a first resonance frequency when the first or second polarization component is incident on the phase retarder or BSRS 2.
[0101] The analyzing polarizer 3 receives the electromagnetic radiation output from the phase retarder 2, and is configured to filter the received electromagnetic radiation at a plurality of different polarization state angles .sub.m (see for example
[0102] The analyzing polarizer 3 can however take a plurality of states, at least three, but can be in the range of ten to one hundred or hundreds. The plurality of states can be obtained either by mechanically rotating the polarizer 3 or using a voltage in the case of liquid crystals (see, for example,
[0103] The imaging sensor 4 records an image for a plurality or each of the orientations of the analyzing polarizer 3. The entrance polarizer 1 and the phase retarder or BSRS 2 may not share a common symmetry axis when the entrance polarizer 1 is fixed compared to the BSRS 2, in order to generate a set of sharply modulated transmission states. The polarization transmitted through the entrance polarizer 1 can take two different states, either by mechanically rotating the polarizer or using a voltage in the case of liquid crystals. The control of the states of the entrance polarizer 1 provides information on the polarization of the incident light.
[0104] A broadband or narrowband light source (thermal or solid state) providing for example illumination I() can be used.
[0105] The BSRS 2 and the polarizers 1, 3 are in a preferable homogeneous, the BSRS or phase retarder 2 is designed to have substantially angle-independent transmission, so that it can be used for any numerical aperture of the imaging system. The imaging system 4 can therefore comprise or consist of, but is not limited to, an imaging sensor and an objective lens. However, other optical components may also be used.
[0106] The tunable optical filter SRTF or system SRMSI can be used in conjunction with other commercial multispectral imaging systems 4 such as e.g. RGB cameras to further enhance the spectral resolution. The tunable optical filter SRTF and the imaging system SRMSI can be adapted to different spectral ranges, depending on the application.
[0107] The phase retarder or birefringent subwavelength resonating structure 2 supports localized resonances. At the resonance frequency, light is absorbed or reflected, the remaining light being transmitted through the structure 2. The excitation efficiency of plasmons or dielectric resonance modes depends on the polarization of the incident light. Combined excitation of two different polarizations (see for example 2a and 2b in
[0108] The analyzing polarizer 3 filters out such a wavelength-dependent intensity distribution, by analyzing the output 2c at a certain polarization .sub.m. This leads to the plurality of states (at least 3, but usually in the range of 10 to one hundred or hundreds), where 3a in
[0109] Alternatively, in another non-illustrated embodiment, the analyzing polarizer 3 and the input polarizer 1 can be inverted in position. The analyzing polarizer 3 receives electromagnetic radiation incident from an object OB, and is configured to filter the incident electromagnetic radiation at a plurality of different polarization state angles .sub.m and to output electromagnetic radiation at each polarization state angle .sub.m. The phase retarder or BSRS 2 receives for example, from the analyzing polarizer 3, linearly polarized electromagnetic radiation having first and second polarization components on, for example, two orthogonal axis. The output polarizer (polarizer 1) receives the electromagnetic radiation output from the phase retarder 2, and filters the received electromagnetic radiation to output linearly polarized electromagnetic radiation.
[0110] The phase retarder or birefringent subwavelength resonating structure 2 comprises a plasmonic or dielectric resonating subwavelength structure providing a plasmonic or dielectric resonating filter having a phase shift at resonance (strongly increasing contrast) over different axis, making it birefringent with a strongly wavelength dependent birefringence, providing angle-independent properties (high field-of-view), and broadband transmission (high photon efficiency). The phase retarder or birefringent subwavelength resonating structure 2 can, for example, have the following features or properties: [0111] Effect: phase shift, zero order transmission, broadband, chiral [0112] Phase Shift: 1 or 2 phase shifts for each 1 or 2 polarizations [0113] Broadband: absorbing or reflecting light at resonance [0114] Structure: 1D or 2D [0115] Exemplary Structure Type or form: stack of lines, lines, rods, ellipse, pillars, or any resonators procuring a phase shift [0116] Angle-independency (field-of-view): Exemplary period: 160 nm, 180 nm; Exemplary period range: 140-200 nm, or 160 to 180 nm [0117] Exemplary Depth: 70 nm, 90 nm; Exemplary depth range 30-120 nm [0118] Exemplary Materials for structure(s): Ag, Al, Au, Ni for metals inducing phase shift in the visible range, however other metals can be used; dielectric materials, especially high refractive index dielectrics and semiconductors [0119] Exemplary Thickness of Metal for structure(s): range 10 to 30 nm or 40 nm; range 5-50 nm [0120] Acting Spectral Range: can be modified and determined by values of depth, period, the structure material used, thickness, the underlying and/or covering material for operation, for example, in the visible, Infrared, Mid-infrared, UV [0121] Exemplary Underlying (and Covering) Material: Transparent sol-gel, plastic, polymer, or other materials [0122] Covering (Embedding): with (better protection, ambient condition) or without [0123] Area: homogenous (covering several pixels of field-of-view), line-wise varying, pixel-wise varying, patches for different cameras [0124] Spectral Range: homogenous for filter or pixelization could be used to enhance spectral resolution or dynamic range
[0125]
[0126] The phase retarder or birefringent subwavelength resonating structure 2 can include subwavelength metallic nanostructures having a metallic thickness value in the range of 5 nm to 100 nm, preferably 10 nm to 50 nm. A separation trench separating the metallic nanostructures may have a depth in the range of 30 nm to 250 nm, preferably 30 nm to 120 nm. The subwavelength metallic nanostructures can have, for example, a periodic separation in the range of 140 nm to 200 nm to provide an angle independent response of phase retarder. It is noted that all range values mentioned throughout this application include the boundary values.
[0127] This plasmonic phase retarder 2 including the plurality of metal nanostructures (nanowires) produce localized surface plasmon resonances (LSPR) permitting broadband wavelength filtering. Parameters of the nanostructures, such as the period and depth, can be varied to optimize angle- and orientation-dependency of the generated color and to tune the localized surface plasmon resonance (LSPR) to a desired wavelength range. The thickness of the silver nanowires also influences the position of the LSPR. The above-mentioned plasmonic phase retarder comprising a periodic array of silver nanowires can be used to carry out color filtering in the visible spectral range.
[0128] The plasmonic phase retarder 2 supporting localized surface plasmon resonances alters the polarization of incident light and induces a strong phase shift for a polarization setting. As a result, the rotation of the analyzing polarizer 3 gives rise to distinct colors.
[0129] For example, to generate distinct colors, the input polarizer 1 can be set for example at angle =45 (diagonally polarized) and the analyzing polarizer 3 angle set at =0, 45, 90, and 135 as illustrated in
[0130] This above described exemplary plasmonic phase retarder 2 includes a plurality of periodically substantially aligned subwavelength metal elongated structures configured to generate localized surface plasmon resonances (at a first resonance frequency and/or at a second resonance frequency) to produce the above-mentioned wavelength-dependent phase shift. Alternatively, the phase retarder 2 may be configured to generate subwavelength dielectric resonances at a first resonance frequency and/or at a second resonance frequency to produce the above-mentioned wavelength-dependent phase shift.
[0131] Subwavelength dielectric resonances are resonances occurring in a subwavelength dielectric structure, for example, a geometrically defined dielectric structure, that can be repetitive and plural, having an abrupt change in permittivity. This creates in the electromagnetic field, reflected at dielectric/dielectric interface(s) (and possibly partially trapped in high refractive index subwavelength volumessuch as in subwavelength gratings acting as local resonator), constructive or destructive interferences.
[0132] The dielectric structure preferably comprises or consists of a high refractive index dielectric material or layer located, for example directly, on a low refractive index dielectric material or layer. Many of these materials, such as titanium dioxide/titania can also be called semiconductors or high refractive index semiconductors, depending on the wavelength range considered, because of their existing bandgap. Subwavelength dielectric resonances therefore also can include resonances due to a subwavelength semiconductor structure, substantially transparent in a part of the wavelength range of interest. Such high refractive index material have preferably a refractive index in the wavelength range of interest higher than 1.8, preferably higher than 2. As examples, TiO2, ZnS, Ta2O5, HfO2, ZrO2, AlN and Al2O3 can be used. For a wavelength range in the infrared, other materials transparent in this wavelength range are suitable such as silicon, chalcogenide glasses or gallium compounds. The refractive index of the material surrounding the high refractive index material in this embodiment in preferably lower than 1.6. It can as example be air on one side of the subwavelength grating, with a refractive index of 1, and around 1.55 on its other side, with a glass or polymer supporting structure.
[0133] The phase retarder 2 configured to generate subwavelength dielectric resonances can have, for example, a structure similar to that of the plasmonic phase retarder 2 shown in
[0134] As previously mentioned, the phase retarder 2 can comprise a plurality of periodically substantially aligned subwavelength elongated structures configured to generate localized surface plasmon resonances or subwavelength dielectric resonances at a first resonance frequency and/or at a second resonance frequency. The phase retarder can alternatively comprise a first plurality (or first set) of subwavelength structures, for example, periodically substantially aligned subwavelength elongated structures configured to generate localized surface plasmon resonances or subwavelength dielectric resonances at first and second resonance frequencies, and further include a second plurality (or second set) of subwavelength structures, for example periodically substantially aligned subwavelength structures, configured to generate localized surface plasmon resonances or subwavelength dielectric resonances at third and fourth resonance frequencies. The first to fourth frequencies are different frequency values. The first plurality of subwavelength structures and the second plurality of subwavelength structures are configured, (for example, via their structural form, thickness, periodicity materials etc. as previously mentioned) to produce resonances at these different frequencies.
[0135] The first and second set of subwavelength structures, for example, periodically aligned subwavelength metallic nanostructures comprising subwavelength metallic nanostructures can have a different orientation, and/or a different (metallic) thickness value, and/or a different periodic separation, and/or a different separation trench depth.
[0136] The resonance (localized surface plasmon resonance or subwavelength dielectric resonance) can also consist of a plurality of resonances, in spectral proximity, or within a coupling regime, forming a coupled resonance system. This coupled resonance system acts as one resonance.
[0137] The phase retarder 2 generally is located between the input polarizer 1 and the tunable or analyzing polarizer 3. The input polarizer 1 can be a conventional polarizer for producing, for example, linearly polarized electromagnetic radiation. In other words, an optical filter letting electromagnetic waves of a specific polarization pass and blocks electromagnetic waves of other polarizations. Light of undefined or mixed polarization s filtered to provide light of well-defined polarization or polarized light. The input polarizer 1 can be a built-in polarizer integral (forming one object) with the phase retarder 2. It can be tunable as well to further enhance detection capability. Direct excitation of two states by other filtering methods or adapted plasmonic structures 2 can also be envisaged. The analyzing polarizer 3 can also be a conventional polarizer.
[0138] The polarizers 1, 3 can include liquid crystals and a liquid crystal polarization rotator to allow electrical tuning of the polarization. Alternatively, mechanically tunable polarizers can be used. The SRTF can be rotated in respect to the imaging axis, e.g. for polarization sensing.
[0139] Stacking of the elements 1, 2 and 3 can be carried out with, for example, adhesive (e.g. glue), lamination, hot bonding, direct integration, spin coat of liquid crystal, etc. For example, after fabrication of the phase retarder 2, a wire grid polarizer 1 (for example, ITOS, XP44) can be glued diagonally to the nanowires lines of the phase retarder 2. This can then be fixed onto a holder just in front of an automatic rotation stage (for example, Thorlabs, PRM1/MZ8) containing a wire grid polarizer 3 (for example, ITOS, XP44). Rotation of this polarizer 3 enables different transmission spectra to be obtained from this plasmonic active tunable filter SRTF.
[0140] The incident polarizer 1 can be tunable to either analyze the polarization of the incoming light or can be used as the analyzing polarizer, where in this case the second polarizer is preferably static. As an example, the incident polarizer 1 can analyze the orientation of linear polarized incoming light. Combined with a retarder plate, circular and ellipsoidal polarization chirality can be analyzed.
[0141] The imaging system SRMSI (see for example
[0142] The image sensor 4 includes at least one or a plurality of pixels. The image sensor 4 is arranged downstream from the tunable optical filter SRTF and is configured to record the electromagnetic radiation intensity of the electromagnetic radiation output from the output polarizer or the analyzing polarizer 3 at each polarization state angle. The tunable optical filter SRTF can be configured to homogenously filter electromagnetic radiation incident on a plurality of pixels of the image sensor 4, or to homogenously filter electromagnetic radiation incident on the full field of view of the image sensor 4.
[0143] The imaging system SRMSI can include optical components such as lens, objective lens, or microlens. Such components may be positioned between the tunable filter SRTF and the image sensor 4. The imaging system SRMSI can include a light source such as a LED, xenon lamp, tunable light source, or use sunlight as a light source. The tunable optical filter SRTF can be located in the imaging system in different positions, it can be flexible in front of the image sensor 4, closely attached, or attached onto the imaging chip 4. The imaging system SRMSI can be miniaturized, lightweight, electrically addressable, adaptable, and exchangeable. Spectral recording can be carried out using the full spatial resolution of the camera 4 since the filter SRTF is located before the imaging sensor 4.
[0144]
[0145] To calculate the reflected spectra (R.sub.pixel(.sub.n)) from the object OB, the continuous spectral range is divided into N discrete parts. The recorded intensity of each pixel (V.sub.pixel (.sub.m)) can be defined as a discrete function of the wavelength .sub.n as:
with the sensitivity of the camera 4 being S(.sub.n), the illuminant being I(.sub.n), the transmission spectra of the filter SRTF being T.sub.SRTF and the reflection of the target or object OB being R.sub.pixel(.sub.n). It is noted that * with represents multiplication. V.sub.pixel (.sub.m) and R.sub.pixel(.sub.n) include spatial position, and are evaluated for each pixel of the imaging sensor or system 4.
[0146] S, I and T.sub.SRTF only depend on the wavelength .sub.n and polarization angle of the filter .sub.m respectively and need to be evaluated only once, thus (A(.sub.n, .sub.m)) can be evaluated only once where A(.sub.n, .sub.m)=S(.sub.n)*I(.sub.n)*T.sub.SRTF (.sub.n, .sub.m).
[0147] By solving the equation Vpixel(.sub.m)=A(.sub.n, .sub.m)*Rpixel(.sub.n), the spectra R.sub.pixel(.sub.n) can be determined. Generally, the number of used filters .sub.m=n. Forn>m estimation methods (e.g. Wiener estimation) can be utilized, for n<m, the solution should be addressable or reduction methods (e.g. principal component analysis PCA) can be used.
[0148] In other words, a source with spectrum I() illuminates the sample or object OB to be measured. The reflected light, with intensity R.sub.pixel(.sub.n), transmits trough the filter SRTF with transmission T.sub.SRTF (which depends on the rotation angle of the polarizer), before an intensity image, V.sub.pixel (.sub.m), is recorded by a camera 4 with for example a plurality of pixels and a given sensitivity S().
[0149] A black and white camera 4 can, for example, be used, but the following also applies for the usage of multispectral cameras, where multiple spectral sensitivities S are used. The reflected spectra R.sub.pixel(.sub.n) is calculated by reconstruction. The recorded intensity of each pixel V.sub.pixel can be determined as a discrete function of the wavelength (Equation 1 above). V.sub.pixel and R.sub.pixel depend on each recorded pixel and have to be evaluated individually for each sample or object. S, I and T.sub.SRTF only depend on the wavelength and the rotation angle of the filter .sub.m respectively and are evaluated only once for a given illuminant.
[0150] The number of discretization parts N defines the number of unknowns to determine the reflected spectrum R.sub.pixel(.sub.n). The number of filters M, expressed by the rotation angle .sub.m, determines the number of known variables. This leads to a linear equation system in matrix form of Vpixel(.sub.m)=A(.sub.n, .sub.m)*Rpixel(.sub.n), with M knowns and N unknowns.
[0151] Solving this linear equation gives the spectra R.sub.pixel of a given pixel discretized by .sub.n. To directly relate the measured intensity to spectral information a linear least-squares approximation can, for example, be used (more details can be found for example in Strang, G. Introduction to linear algebra. Wellesley-Cambridge Press (2016), the entire contents thereof being herewith incorporated by reference). The spectra R(.sub.n) can be determined by minimizing the squared Euclidean norm of:
[0152] However, other reconstruction algorithms can also be used, such as for example, principle component analysis (PCA) or Wiener estimation. Solving of the algorithm can be done in different ways, and can be optimized depending on the application, the number of filters used and a referenced database. [0153] Solving algorithm: Least-squares (generalized, regularized, weighted), L1-Norm, QR decomposition [0154] Reduction methods: PCA, SVD [0155] Estimation algorithm: Eigenvector, Wiener filtering, orthogonal projection [0156] Possible improvement using: dark current, cf. known database, error matrix, calibration matrix, etc. [0157] Required parameters: Spectra of filter, sensitivity of each channel, ev. light source [0158] Correction algorithms: Gamma correction, white balance, linear matrix, etc. [0159] In combination with multispectral imaging system: demosaicing, etc.
[0160] The imaging system SRMSI includes a processor or calculator P and a memory DS containing stored algorithms or programs to carry out the above calculations to determine the reflected spectra, as well as stored data used during these calculations. The processor P is configured to calculate the reflected spectra R.sub.pixel(.sub.n) of the incident electromagnetic radiation from the object OB for each pixel of the image sensor 4 and for a plurality of wavelengths .sub.n based on Equation 1:
where S(.sub.n) is the sensitivity of the image sensor, I(.sub.n) is the object illuminant, T.sub.SRTF (.sub.n, .sub.m) is the transmission spectra of the phase retarder 2 as a function of polarization state angle .sub.m and V.sub.pixel (.sub.m) is the pixel recorded intensity at each polarization state angle .sub.m, where A(.sub.n, .sub.m)=S(.sub.n)*I(.sub.n)*T.sub.SRTF(.sub.n, .sub.m).
[0161] The memory or data storage DS, (for example semiconductor memory, HDD, flash memory) is configured to store values for (i) the sensitivity of the image sensor S(.sub.n), (ii) the object illuminant I(.sub.n), and (iii) the transmission spectra of the phase retarder also called subwavelength resonating tunable filter T.sub.SRTF (.sub.n, .sub.m) as a function of polarization state angle .sub.m. The processor P is further configured to calculate the equation A(.sub.n, .sub.m)=S(.sub.n)*I(.sub.n)*T.sub.SRTF (.sub.n, .sub.m) based on these stored values, and to carry out the calculation of solving the equation Vpixel(.sub.m)=A(.sub.n, .sub.m)*Rpixel(.sub.n) to determine the reflected spectra R.sub.pixel(.sub.n).
[0162] The processor P is further configured to calculate a multispectral or hyperspectral image based on the determined spectra R.sub.pixel(.sub.n). A program can be included to convert into RGB values (CIE 1931 color space), which were then plotted as RGB image.
[0163] The processor P is additionally configured to determine the polarization state angles for a plurality of wavelengths .sub.n using a corresponding program stored in the memory DS.
[0164] The processor P is also configured to solve the equation Vpixel(.sub.m)=A(.sub.n, .sub.m)*Rpixel(.sub.n), to determine spectra R.sub.pixel(.sub.n). The processor is configured to use the least-squares solving algorithm previously mentioned (equation 2) to determine spectra R.sub.pixel(.sub.n), but can alternatively be configured to use any one of the other solving algorithms disclosed herein.
[0165] The memory DS also includes a control program or software which the processor P is configured to execute to controlling rotation of the polarizers and/or imagining device 4.
[0166] The tunable optical filter SRTF with angle-independent and broadband transmission properties can thus be used for multispectral or hyperspectral imaging by determining the reflected spectra as set out above.
[0167] Multispectral or hyperspectral imaging can thus be carried out by: [0168] providing incident electromagnetic radiation IR from an object OB to the input polarizer 1 configured to produce linearly polarized electromagnetic radiation; [0169] providing the linearly polarized electromagnetic radiation having first and second polarization components on two orthogonal axes to the phase retarder 2 configured to carry out a wavelength dependent shift of a phase of the first and/or second polarization component; [0170] providing the electromagnetic radiation output from the phase retarder 2 to the analyzing polarizer 3 configured to filter the provided electromagnetic radiation at a plurality of different polarization state angles .sub.m to output electromagnetic radiation having distinct spectral content at each polarization state angle .sub.m; and [0171] recording, using the image sensor 4 containing a plurality of pixels, an electromagnetic radiation intensity V.sub.pixel (.sub.m) of the electromagnetic radiation output from the analyzing polarizer 3 at a plurality of different polarization state angles.
[0172] In the case where the analyzing polarizer 3 and the input polarizer 1 are inverted, multispectral or hyperspectral imaging can be carried out by: [0173] providing incident electromagnetic radiation from an object OB to the analyzing polarizer 3 configured to filter said provided electromagnetic radiation at a plurality of different polarization state angles .sub.m to output electromagnetic radiation at each polarization state angle .sub.m; [0174] providing the output electromagnetic radiation having first and second polarization components on two orthogonal axes to the phase retarder 2 configured to carry out a wavelength dependent shift of a phase of the first and/or second polarization component; [0175] providing the electromagnetic radiation output from the phase retarder 2 to an output polarizer 1 configured to produce linearly polarized electromagnetic radiation; and [0176] recording, using the image sensor 4, an electromagnetic radiation intensity V.sub.pixel (.sub.m) of the electromagnetic radiation output from the output polarizer 1 at a plurality of different polarization state angles .sub.m.
[0177] The reflected spectra R.sub.pixel(.sub.n) of the incident electromagnetic radiation from the object OB can be calculated for each pixel of the image sensor and for a plurality of wavelengths .sub.n based on Equation 1. Values for (i) the sensitivity of the image sensor S(.sub.n), (ii) the object illuminant I(.sub.n), and (iii) the transmission spectra of the phase retarder T.sub.SRTF (.sub.n, .sub.m) as a function of polarization state angle .sub.m are stored and the equation A(.sub.n, .sub.m)=S(.sub.n)*I(.sub.n)*T.sub.SRTF(.sub.n, .sub.m) calculated based on these stored values. The calculation of solving the equation Vpixel(.sub.m)=A(.sub.n, .sub.m)*Rpixel(.sub.n) can be carried out to determine the reflected spectra R.sub.pixel(.sub.n). The multispectral or hyperspectral image can then be calculated based on the determined spectra R.sub.pixel(.sub.n).
[0178]
[0179] The proposed system can be used for many kinds of application areas such as: [0180] Type: color recording (at full resolution), spectral estimation, singularities detection, imaging at specific wavelengths, polarization imaging. [0181] Resolution: using a B/W, RGB or multispectral imaging system defines the ratio of spectral versus spatial resolution (requirements depend on applications). [0182] Applications: Spectral imaging of objects, Monitoring of nutrition, remote sensing, art conservation, medical monitoring, spectral imaging in space, military applications.
[0183] A miniaturized SRTF can, for example, be built and used in combination with a smart phone. The integrated flash lamp can serve as a known source, exposure time can be adapted and fixed, and algorithms or programs can determine or estimate the spectral information. The reconstruction range can be selected via touch display to decrease computation time. The SRTF can be powered by the phone itself. This provides a cost-efficient sensor that can be utilized for food monitoring, art analysis, etc.
[0184] The system can be configured so that a certain number of filters can be used to measure just certain transmission spectra. For example, by comparing two or more specific filters, one can obtain a pronounced spectral transmission, which can be used to exactly detect certain traces or amounts of reflected light or specific wavelengths, permitting for example to detect specific events, for example, if a plant or plants are alive or not.
[0185] The SRTF can, for example, provide many phase shifts at different polarizations. This can be provided by, for example, 2D rectangular plasmonic structures, having different plasmon resonances, each inducing a phase shift. This can enhance the spectral resolution. Furthermore, it could limit certain measurements to specific ranges. This could be used with different imaging devices acting at different ranges, e.g. with a beam splitter.
[0186] The disclosed system may also be used as a miniaturized spectrometer by using a photodiode 4 for intensity measurement. To increase spectral resolution, multiple photodiodes 4 with each a broadband filter (e.g. quantum dot, dyes, plasmonic filter) can be used. This provides be a very precise, cost-efficient and angle-independent spectrometer.
[0187] The disclosed filtering device SRTF can also be used for calibration of a camera sensor. A light source with known spectra illuminates the camera through the filter system SRTF. This gives a very distinct pattern depending on the light source and could be used in combination with pixelization.
[0188] To verify the spectral reconstruction of the imaging system, a Macbeth ColorChecker was used as a test target. B/W images of colored patches were recorded using the SRFT and a LED light source. The reconstructed spectra were then compared to the spectra measured directly with a conventional spectrometer and indicated as ref. in
[0189] The color representation is very well preserved also for higher tilt angles showing the potentially high FOV and corresponding large aperture. Overall these results indicate that the imaging system functions at least up to an incident angle of 25, therefore enabling a possible large acceptance angles of at least about 50. This would correspond to a minimum f-number of f/1.07, which covers the range of conventional lenses.
[0190] Imaging experiments were performed with the imaging system SRMSI containing a B/W camera 4 demonstrating the color image recording capability (see
[0191] In contrast to conventional imaging systems, no further post-processing (e.g. white balance, gamma correction, etc.) on the reconstructed image has been done. Good color representation of the fruits is present in the reconstructed image of
[0192] For image recording, the filter SRTF was mounted in front of a black/white camera (Baumer TXG14, silicon sensor) with configurable macro lens (Opto Engineering, MC3-03X). The setup was facing an object at a distance of about 50 cm, which was illuminated with a LED screen (Drr, LP400). Images were recorded with an inhouse LabVIEW (version 2015) script, while controlling the filter angle. To compare the accuracy of the spectral reconstruction, the colored patches of the Macbeth ColorChecker (X-Rite) were measured with the spectrometer directly.
[0193] Concerning the reconstruction program or software, the recorded images were imported into an inhouse a MATLAB (version 2016) script. The intensity of the pixels of interest were extracted correspondingly for each used filter angle. For analyzing the color patches, an average of 100100 pixels was taken. Additionally, the measured spectra of the angle-dependent filter, light source and the sensitivity of the camera was imported. For simplicity reasons, a linear behavior of the camera sensor response was assumed.
[0194] Then MATLAB (version 2016) was used to compute an iterative leastsquares fit (command: lsqlin, see for example mathworks https.//ch.mathworks.com/help/optim/ug/lsqlin.html) with boundary conditions of a certain wavelength range scaling. The solution was limited to positive values only, with the upper boundary limiting it to physical useful values. A smoothness filter was applied to the reconstructed spectra to reduce oscillation effects arising from the illcondition problem.
[0195] The resulting spectra can be and were converting into RGB values (CIE 1931 color space, MATLAB script), which were then plotted as RGB image.
[0196] To further highlight the spectral recording capabilities, the imaging system was used to measure laser peaks at different spectral positions. Some reconstructed laser lines are shown in
[0197] The imaging system of the present disclosure can provide spectral estimation of colored samples and spectral singularities. The accuracy of the spectral estimation depends intrinsically on the design of the filter system. The system can be described by mainly 3 eigenvectors, which would make a larger number of filters redundant. Nevertheless, calculations show that, depending on the measured objects, a higher number of eigenvectors or distinct eigenvectors are useful for a reconstruction with higher accuracy. Therefore, using a large set of filters can add value to the system, besides reducing the noise. On the other hand, research has demonstrated that already 5-8 eigenvectors are sufficient to completely reproduce artworks or e.g. 1269 Munsell chips. Ultimately, there will be a tradeoff between the spectral accuracy and number of filters in relation to the type of sample, which should be measured.
[0198] The eigenvectors and corresponding eigenvalues of the filter combined with a B/W camera and a RGB camera are shown in
[0199] A functional multispectral imaging system based on homogenous active tunable plasmonic filters and a commercially available black/white camera has thus been demonstrated. Recording of colored objects and laser lines at full spatial resolution and without prior knowledge has been carried out. This includes an estimation of the spectra for each pixel, capable of distinguishing two laser lines separated by 1 nm. A high angle-stability of the plasmonic filter has been shown, allowing recording for example at a large FOV of about 50 by simply mounting the filter SRFT in front of an imaging system. The filter SRFT can be fabricated with standard roll-to-roll techniques, enabling cost-effective manufacturing for a wide range of applications. Combined with a camera containing multispectral arrays (e.g. RGB camera), the spectral resolution and dynamic range can be strongly increased. Non-inverse and adaptable filters for existing imaging system could be a key factor for a wide range of customer applications (e.g. smart phones). The number of used filters in combination with an appropriate spectral estimation algorithm can be adapted in situ depending on the kind of application, e.g. analysis of artworks, remote sensing. Applying this principle as pixelization of the plasmonic filter could increase the spectral resolution while maintaining a high spatial resolution. The rotating polarizer could be exchanged by electrical tunable liquid crystal (see for example
[0200] While the invention has been disclosed with reference to certain preferred embodiments, numerous modifications, alterations, and changes to the described embodiments, and equivalents thereof, are possible without departing from the sphere and scope of the invention. Accordingly, it is intended that the invention not be limited to the described embodiments, and be given the broadest reasonable interpretation in accordance with the language of the appended claims.