Imager and spot sampler with translatable stage
12271004 ยท 2025-04-08
Assignee
Inventors
Cpc classification
G02B6/4298
PHYSICS
H04N23/55
ELECTRICITY
G01J3/021
PHYSICS
G01J3/0243
PHYSICS
G02B27/108
PHYSICS
International classification
H04N23/55
ELECTRICITY
G02B7/00
PHYSICS
Abstract
Optical devices that combine imaging with spectral detection usually require bulky or expensive optics or are limited in the spectral measurements that can be made. The disclosed device uses a lens and mirror to provide two image planes: one for a pixelated detector and the other for an optical fiber assembly. The optical fiber assembly may be scanned over most or all of the field of view, or this may be achieved with a fixed fiber assembly and translatable mirror. Multiple fibers may be included in the assembly and multiple measurement or other devices may be connected to the remote ends of the optical fibers.
Claims
1. An optical device comprising: one or more lenses providing a first image focal plane; a mirror providing a second image focal plane of the one or more lenses; a pixelated detector at the first image focal plane, the pixelated detector being taller, wider or both taller and wider than the mirror; a translation stage upon which the mirror is mounted; and an optical fiber positioned so that an entrance face of the optical fiber is in the second image focal plane; wherein the translation stage translates the mirror between a position in which it provides the second image focal plane and a position in which it does not provide the second image focal plane.
2. An optical device comprising: one or more lenses providing a first image focal plane; a mirror providing a second image focal plane of the one or more lenses; a pixelated detector at the first image focal plane; a translation stage upon which the mirror is mounted; an optical fiber positioned so that an entrance face of the optical fiber is in the second image focal plane; and on the translation stage, one or any of: a prism, a beam splitter, clear glass, a thin film filter, an absorbing filter, a polarizing filter, a linear variable bandpass filter, an opaque panel; wherein the translation stage translates the mirror between a position in which it provides the second image focal plane and a position in which it does not provide the second image focal plane.
3. The optical device of claim 1 or 2, wherein the translation stage is translatable from a position where it does not occlude the pixelated detector to a position where it partially or completely occludes the pixelated detector.
4. The optical device of claim 1 or 2, wherein the translation stage is translatable from a position where it does not occlude the optical fiber to a position where it partially or completely occludes the optical fiber.
5. The optical device of claim 1, wherein: there is exactly one lens that provides the first image focal plane; and the second image focal plane is a focal plane of said exactly one lens.
6. The optical device of claim 1 or 2, comprising a filter wheel with one or more optical filters, the filter wheel located between the one or more lenses and the mirror.
7. The optical device of claim 1, comprising one or more further optical fibers each having a further entrance face positioned in the second image focal plane.
8. The optical device of claim 1 or 2 wherein there are no lenses between a plane of translation of the translation stage and the pixelated detector.
9. The optical device of claim 1 or 2, wherein the translatable stage supports at least one color filter, at least one neutral density filter, or at least one color filter and at least one neutral density filter in front of either or both of the pixelated detector and the optical fiber.
10. The optical device of claim 1 or 2, wherein a location of the one or more lenses is adjustable depending on a wavelength of light captured by the optical fiber.
11. The optical device of claim 1 or 2, comprising a reference light source that directs light into the optical fiber.
12. The optical device of claim 1 comprising, on the translation stage, one or any of: a prism, a beam splitter, clear glass, a thin film filter, an absorbing filter, a polarizing filter, a linear variable bandpass filter, an opaque panel.
13. The optical device of claim 1 or 2, comprising a further translation stage on which an exit end of the optical fiber is mounted.
14. The optical device of claim 1 or 2, wherein the mirror is curved or segmented.
15. An optical device comprising: one or more lenses providing an image focal plane; a mirror providing another image focal plane of the one or more lenses; a pixelated detector at a first one of said image focal planes; a translation stage that translates parallel to a second one of said image focal planes; and an optical fiber mounted on the translation stage so that an entrance face of the optical fiber is translatable in the second one of said image focal planes; wherein there are no lenses between a plane of translation of the translation stage and the pixelated detector.
16. The optical device of claim 15, wherein: there is exactly one lens that provides the image focal plane; and the other image focal plane is a focal plane of said exactly one lens.
17. The optical device of claim 16, wherein said exactly one lens is a compound lens.
18. The optical device of claim 16, wherein: the mirror is mounted on the translation stage; and the optical device comprises a selectable filter between said exactly one lens and the mirror.
19. The optical device of claim 15, comprising a filter wheel with one or more optical filters, the filter wheel located between the one or more lenses and the mirror.
20. The optical device of claim 15, wherein the mirror is in a fixed position relative to the one or more lenses.
21. The optical device of claim 20, wherein: the mirror is a 100% reflecting mirror; or the mirror is a beam-splitter; or said second one of said image focal planes is provided by the one or more lenses and the mirror has a reflective surface in which there is a central transparent aperture; or said second one of said image focal planes is provided by the mirror and the mirror has a central reflective spot surrounded by a transparent surface; or said second one of said image focal planes is provided by the one or more lenses and the mirror has a reflective surface in which there is are multiple transparent apertures; or said second one of said image focal planes is provided by the mirror and the mirror has multiple reflective spots surrounded by a transparent surface.
22. The optical device of claim 15, comprising: one or more focusing lens elements between the mirror and the entrance face of the optical fiber; one or more other focusing lens elements between the mirror and the pixelated detector; wherein: the mirror is tilted relative to an optical axis of the one or more lenses; light from the one or more lenses focuses at an intersection of the optical axis and a surface of the mirror; the mirror has an aperture therein at the optical axis or within 10% of a width of the mirror from the optical axis; a minor fraction of the light passes through the aperture and is focused on the entrance face of the optical fiber, and the remaining fraction of the light is focused onto the pixelated detector.
23. The optical device of claim 22, wherein the mirror has one or more further apertures, the mirror translatable so that each of the further apertures is positionable on the optical axis or within 10% of the width of the mirror from the optical axis.
24. The optical device of claim 15, wherein: said first one of said image focal planes is provided by the one or more lenses; the pixelated detector is taller, wider or both taller and wider than the mirror; and the mirror is mounted on the translation stage.
25. The optical device of claim 24, wherein the translation stage is translatable from a position where it does not occlude the pixelated detector to a position where it partially or completely occludes the pixelated detector.
26. The optical device of claim 24, wherein the translation stage is translatable from a position where it does not occlude the optical fiber to a position where it partially or completely occludes the fiber optical fiber.
27. The optical device of claim 15, comprising one or more further optical fibers, each having an entrance face translatable in the second one of said image focal planes.
28. The optical device of claim 27, wherein all of the optical fibers are connected to a common optical measurement instrument or to different optical measurement instruments.
29. The optical device of claim 15, wherein the translatable stage supports at least one color filter, at least one neutral density filter, or at least one color filter and at least one neutral density filter in front of either or both of the pixelated detector and the optical fiber.
30. The optical device of claim 15, wherein the translation stage translates in two dimensions parallel to the second one of said image focal planes.
31. The optical device of claim 15, wherein a location of the one or more lenses is adjustable depending on a wavelength of light captured by the optical fiber.
32. The optical device of claim 15, comprising a reference light source that directs light into the optical fiber.
33. The optical device of claim 15, comprising, on the translation stage, one or any of: a prism, a beam splitter, clear glass, a thin film filter, an absorbing filter, a polarizing filter, a linear variable bandpass filter, an opaque panel.
34. The optical device of claim 15, comprising a further translation stage on which an exit end of the optical fiber is mounted.
35. The optical device of claim 34, comprising further optical fibers, wherein exit faces of all the optical fibers are mounted on the further translation stage, which translates so that one or some of the exit faces are aligned with a light-limiting aperture, an entrance slit of a spectrometer, a fiber assembly, one of multiple light-limiting apertures, or a further light-limiting aperture that has a lens assembly on one or both sides.
36. The optical device of claim 15, comprising a micro lens mounted on an exit face of the optical fiber.
37. The optical device of claim 15, wherein an exit face of the optical fiber couples to an optical measurement instrument.
38. The optical device of claim 15, wherein the mirror is curved or segmented.
39. A method for calibrating an optical device that comprises: one or more lenses providing an image focal plane; a mirror providing another image focal plane of the one or more lenses; a pixelated detector at a first one of said image focal planes; a translation stage that translates parallel to a second one of said image focal planes; and an optical fiber mounted on the translation stage so that an entrance face of the optical fiber is translatable in the second one of said image focal planes; the method comprising: focusing the pixelated detector onto a white screen; translating the entrance face of the optical fiber to a desired location; back-illuminating the optical fiber to illuminate a spot on the white screen; applying a mark to the white screen at the illuminated spot; translating the entrance face of the optical fiber outside of a field of view of the pixelated detector; acquiring an image of the white screen and the mark; defining pixels of the pixelated detector that correspond to the mark; and recording the defined pixels in association with the desired location.
40. The method of claim 39, wherein the optical device comprises a filter in an optical path from the one or more lenses to the pixelated detector, the method further comprising: determining a spectral responsivity of each pixel of the pixelated detector.
41. The method of claim 39, wherein the optical device comprises one or more further optical fibers, the method comprising: connecting any of the optical fiber and the further optical fibers to a reference light source after calibration of the optical device.
42. The optical device of claim 2, wherein: there is exactly one lens that provides the first image focal plane; and the second image focal plane is a focal plane of said exactly one lens.
43. The optical device of claim 2, comprising one or more further optical fibers each having a further entrance face positioned in the second image focal plane.
44. The optical device of claim 5 or 42, wherein said exactly one lens is a compound lens.
45. The optical device of claim 7 or 43, wherein all of the optical fibers are connected to a common optical measurement instrument or to different optical measurement instruments.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
(26)
(27)
DETAILED DESCRIPTION OF THE INVENTION
A. Glossary
(28) ADU/sanalog to digital units per second Back-focusrefers to the distance between the camera lens mount and the sensor plane. CCDcharge-coupled device Compound lensa series of simple lenses of different shapes and different refractive indices, arranged adjacent or one after the other on a common axis. Fiber guidea thin fiber with a core through which light can pass with very little loss of intensity. The core can be made of glass, plastic or transparent fluid. Fiber guide assemblythis can be one fiber or a multitude of fibers. It may also include optics in front of the fibers to improve the light collection characteristics including Fresnel, circular or cylindrical lenses. It may also include diffractive optical elements (DOEs). Fiber OpticA fiber guide assembly Filteran optical device through which some light passes and through which some light is blocked. Types of filters include, but are not limited to, bandpass, polarizing, short-pass, long-pass or neutral density filters. A filter may also be an electronically tunable color filter (e.g., Sharp et al. 1990. Continuously Tunable Smectic A* Liquid-Crystal Color Filter, Optics Letters 15(10):523-525). It may also be a Linear Variable Bandpass Filter (LVBF), in which the center wavelength of the pass band varies with position along the length of the filter by, for example, up to a spectral octave. LEDlight emitting diode LVBFlinear variable bandpass filter Lit regionthe volume comprising all rays of light from a scene passing through a lens and imaged onto a camera sensor. Measurement surfacethe reader should understand that the image of the measurement system includes whatever is in the 3-dimensional volume, or field of view of the instrument, and that by extension, all references to measurement surface, can reasonably be understood to be readily extendable to measurement field of view. Optical spectrometerthis term includes any electro-optical instrument that measures the relative or absolute optical power of the radiation incident upon the instrument's input optics as a function of wavelength. Pixelated detectora two-dimensional array of sensors that is used to capture an image of a scene. Reference light sourcethis may be a laser, LED, tungsten lamp, stimulated phosphor lamp, HgAr, or any other light source. The reference light source may be used to check the transmission of light through to the measurement instrument, or it may be used to check the wavelength scale of a spectrometer. RGBRed, green, blue Simple lensa single lens of a single material, such as a convex lens. Other simple lenses may be concave, plano-convex, plano-concave or meniscus lenses. Spectrographa spectrometer that can simultaneously measure a multitude of spectra on one array detector. Spectrometersee Optical Spectrometer. Spectroradiometera spectrometer (optical spectrometer) that has been calibrated in terms of radiometric units, i.e. calibrated to measure the spectral power distribution of electromagnetic radiation incident upon the instrument's input optics. Spot photometeran image forming photometer that measures the light for a single area in a scene. The area measured is usually a circle or rectangle. Turning mirrorthis may be a reflecting surface or beam splitting optic. is used for denoting a specific wavelength in a continuous or a nearly continuous spectrum. is used for denoting the central wavelength of a wide spectral band within a larger spectrum.
B. Exemplary Embodiments
(29)
(30) The turning mirror in the invention is readily realized using a front surface mirror or other mirror. For example, a thin transparent mirror or an optical pellicle may be used. Other reflecting components may be substituted for the beam splitter. If the surface of the mirror 73 had a mirrored spot at 74 and was transparent elsewhere, then the positions of the translation stage 78 with fiber guide assembly 70 and pixelated detector 80 could be swapped, as in
(31)
(32) In some embodiments, the imaging system includes a translating stage on which is mounted both a mirror and fiber(s). The stage moves in the plane perpendicular to the optical axis, in one or two dimensions.
(33)
(34)
(35) Note that in the systems of
(36) The fiber optic assembly 106, 126 may be translated to a multitude of positions and collect light from a corresponding multitude of locations in the scene that is imaged. There may be just one fiber or a multitude of fibers in the assembly, for example in a linear array or a 2D array. The fibers may be coupled to one or more spectrometers or other light measurement instruments.
(37)
(38) If the angles of illumination to the edges of the focal plane are higher than can be accepted by the fiber assembly 144 then there may be additional strategies deployed to improve fiber collection efficiency including light shaping optics in front of the fiber assembly. For example, a curved turning mirror may be used instead of planar turning mirror 146, or a turning mirror with multiple planar segments which are at different orientations may be used instead.
(39)
(40)
(41) Also shown in
(42) In another example, if the fiber and turning mirror are translating with the stage, then at some position when the mirror is not in the light path between the lens and pixelated sensor, a fixed position light source may illuminate the fiber via the turning mirror. Other possibilities to achieve illumination of the fiber with a reference lamp via translation of the stage may be devised using a multitude of methods and are not limited to the methods above.
(43)
(44)
(45) The filters in
(46) In embodiments where they are present, the multiple turning mirror elements may also be at different distances such that different object locations are imaged onto the fiber with one or more different turning mirror elements.
(47) Through translation of the fiber and mirror, one location in the scene may be sampled by a multitude of fiber-based instruments. For example, two spectrometers at different wavelength ranges and a flicker sensor are all connected by the fiber assembly and may, via translation of the stage, sample the same location sequentially. Software may stitch the two spectra together to provide a wider spectrum. It can be readily imagined that there may be a multitude of fiber-based instruments that could be combined to increase the measurements at one particular location. Additionally, users skilled in the art will recognize that lenses are generally not in focus for all wavelengths. If different fibers are used to collect light over different wavelength ranges, then the fibers may be oriented at various distances from the exit pupil to optimize in-focus imaging for each wavelength range and fiber combination.
(48) In another embodiment, there are a multitude of fibers in the fiber guide assemblies of
(49) The fibers need not be positioned very near to the entrance slit if there is a lens in front of the entrance slit that focuses the fiber's light onto the entrance slit. In another embodiment, not shown, each fiber has a miniature lens (micro lens) coupled to it and the fiber and the mated miniature lens translate together.
(50)
(51) In another embodiment, where the spectrometer entrance slit is taller than the fiber diameter, the fibers are arranged in a two-dimensional, MN array, of M rows and N columns such that the stage can be arranged to present M fibers of a column to a spectrometer entrance slit. In this way the light from M fibers can be measured simultaneously. The stage can then be stepped so that another column of M fibers illuminates the spectrometer entrance slit and can be measured.
(52) In other embodiments, instead of, or in addition to a spectrometer entrance slit, there is some other limiting aperture that permits the light from only one or a limited number of fibers in the array to illuminate some other light measurement instrument such as a flicker detector.
(53)
(54) The propagation of light through the optical fibers can be in either direction. Also, there can one input and multiple outputs or multiple inputs and one output. It is also possible that there are multiple inputs and multiple outputs via multiple limiting apertures. In one example there are a multitude of optical fibers coming from the device of, for example, any of
(55)
(56)
(57)
(58)
(59) The invention discloses a mechanical stage supporting one or more optical fibers and optional filters that can be translated in one or two dimensions across the focal plane, but there are other possible approaches. For example,
(60)
C. Calibration of Fiber Location
(61) The location and translation properties of the fiber sampling areas are mapped to pixel locations in the pixelated detector. One way to establish the location of a fiber is shown in
(62) In step 538, the back-illumination of the fiber(s) is turned off and in step 540 the optical fiber assembly is translated to be outside of the light path between the lens assembly and the pixelated detector. In step 542, the pixelated detector is used to acquire an image of the white screen with the mark(s) representing the locations of the fiber spot(s). In step 544, pixels corresponding to the mark(s) are defined, by using image analysis software to define aggregates of camera pixels corresponding to the imaged fiber spot area(s).
(63) Step 546 involves recording the location of the center of each fiber spot in pixelated detector pixels (X.sub.ref, Y.sub.ref) in association with the corresponding stepper motor position, encoder reading, or some similar position sensor.
(64) In one example, reference positions were located using the method above, (X.sub.ref1, Y.sub.ref1) and (X.sub.ref2, Y.sub.ref2) the respective number of steps S.sub.1 and S.sub.2 to go to these two locations would also be known. The rate of change of X pixel location, R.sub.x, and Y pixel location Ry, as a function of motor steps, can now be calculated:
(65)
And for any other position of the stage with steps S, the position of the fiber is:
(66)
Using the equations above, a multitude of fiber locations can now be predicted.
(67) In one embodiment of the invention, one or more of the fibers in the fiber assembly are dedicated for backlighting the optical system and locating the fiber locations. In one embodiment, the fiber that is illumination is electronically controlled within the instrument enclosure. In another embodiment, the fiber may be illuminated by a user-supplied illumination. Types of illumination suitable for this include, but are not limited to lasers, LEDs or VCSELs (vertical-cavity surface-emitting lasers).
(68) In the manufacturing process, the locations of a multitude of the fibers for measurement and illumination in the assembly may be located using the methods above. In service or periodically by the user, the dedicated fiber(s) for illumination may be used to validate the fiber assembly is in the assumed location.
D. Data Augmentation
(69) In the case of multi-spectral imaging where the imaging system spectral sampling interval is less than that used by the spot spectrometer, the imaging data can be augmented by the spectrometer. For example, referring to
(70) In another embodiment, the spot spectrometer measures a multitude of locations in the field of view to create the dataset S(X,Y,). By expanding upon the example detailed above, one can derive a scalar dataset C(X,Y,) which has been made more spatially accurate by considering the spectral measurements from multiple locations of the scene rather than only considering one location, (X.sub.1, Y.sub.1). Similarly, a three-dimensional fitting function could be created to express this dataset mathematically where the fitting domains are spatial (X, Y), and spectral (A).
(71) The methods in the example above are able to generate a more detailed and wider range spectrum with no gaps in the data at all pixelated detector measurement locations. From R(X, Y,), new evaluations of the data may be computed such as CIE 1931 chromaticity x,y.
(72) Persons skilled in the art will know that these methods are equally valid for three-dimensional space; that, when using other coordinate systems such as spherical polar, there are many methods to fit datasets to a mathematical function; and that the explained methods above are example embodiments.
(73) Alternately, in the case of multi-spectral imaging where the imaging system spectral sampling interval is less than that used by the spot spectrometer, the pixelated imaging data can be augmented by the spectrometer. For example, referring to
(74) R1(X1,Y1,) can be thought of as seven data points of the spectrum, and if the imager and spectrometer are properly calibrated, the values for each wavelength should have the same radiometric values when sampling the same spot, i.e. the value from the imager R1(X1,Y1,400) matches the spectrometer S(X1,Y1,400) and R1(X1,Y1,450) matches the spectrometer S(X1,Y1,450), and so on. At any specific pixel (X, Y) in the field of view, the relative values of R at the seven wavelengths will vary. A computation can be made to increase the spectral data interval measured by the pixelated detector by referencing the spectrometer's higher resolution spectral data. For example, if we want to estimate n1 equally spaced spectral data points between two radiance measurements made by the pixelated imaging detector at wavelengths A and B, where B>A, and the radiance values can be denoted by R(Xi, Yj,A) and R(Xi, Yj, B) where Xi and Xj denote a specific pixel in the image.
(75) The measured, high resolution spectrum from the spectrometer S(X1, Y1, A) should be smoothed to create S(X1,Y1, A) using boxcar filtering covering a wavelength range of (BA)/n. The values needed are of for wavelengths A+(nm)*(BA)/n, where m has values from 1 to n1. The spectrum S(X1,Y1, A) can report radiance values at A+(nm)(BA)/n, where m has values from 1 to n1,
(76)
(77) By expanding upon the example detailed above, one can derive a scalar dataset C(X, Y,) which has been made more spatially accurate by considering the spectral measurements from multiple locations of the scene rather than only considering one location, (X1,Y1). Similarly, three-dimensional fitting function could be created to express this dataset mathematically where the fitting domains are spatial (X, Y), and spectral (A).
(78) The methods in the example above are able to generate a more detailed and wider range spectrum with no gaps in the data at all pixelated detector measurement locations. From R(X, Y,), new evaluations of the data may be computed such as CIE 1931 chromaticity x,y. Persons skilled in the art will know that these methods are equally valid for three-dimensional space; that, when using other coordinate systems such as spherical polar, there are many methods to fit datasets to a mathematical function; and that the explained methods above are example embodiments.
E. Omitting Measurements
(79) Many spectral measurements of real-world phenomena contain regions of smoothly varying data while also having low dynamic range. These spectra may be accurately modeled using simple reconstruction algorithms. In the embodiments wherein the imaging instrument contains hyperspectral or multispectral filters as well as a spot spectrometer, a first analysis of the spot spectral measurement can provide detail as to where there is critical spectral content and where there is none.
(80) Let us consider the example of measurements of a white LCD display which is backlit with a multitude of red, green and blue LEDs. Across the display surface and as a result of dimming, diffuser non-uniformities, etc., variation in the relative heights of the measured red, green and blue LED spectral peaks may be observed, and the location of the spectral peaks may shift to shorter or longer wavelengths depending upon the temperatures of the LEDs. However, the shape of the red, green and blue LED spectra may be quite consistent. When the full spectrum at one location on the display is characterized, a reconstruction model can then be created for portions of the LED spectra relative to the measured peak wavelengths. To illustrate, a reconstruction model for the portion of the spectrum measured by the spot spectrometer from 350 nm to 480 nm may be created when the LCD is set to emit blue. If the blue LCD setting is also measured by the imaging system but only in the range from 450 nm to 480 nm, estimates for the unmeasured imaging system range from 380 nm to 450 nm may be generated by referencing the reconstruction model. Similarly, other sections of the spectrum may be similarly omitted from the imaging measurement and then later derived from reconstruction models.
(81) Additionally, consider the application where the user wishes to measure the luminance and chromaticity of an LED RGB display with the hyperspectral or multispectral imaging system accurately, but in the least amount of time possible. An evaluation of spot spectrometer measurements acquired of the display reveals that the spectral regions more than 30 nm shorter than a blue peak at 465 nm have a negligible contribution to the luminance and a negligible contribution to the chromaticity results. In this case, pre-assessment of the display by the spot spectrometer can be used to devise a hyperspectral or multispectral imaging system measurement recipe wherein the wavelength bands below 435 nm are omitted, thus achieving the required accuracy in fewer measurements and hence shorter overall measurement time than if data below 435 nm were acquired. The exposures required to measure all remaining wavelengths of interest with the hyperspectral or multispectral imager could be estimated by comparison of the spot spectral measurement data and the sensitivity of the hyperspectral or multispectral imager.
(82) From this brief case study, a strategy to improve the measurement times for the imaging system is apparent: Firstly, obtain advance knowledge of the scene to be measured by the imaging system via first measuring the scene with the spot spectroradiometer. Secondly, analyze the spectrum to determine which wavelengths or ranges of wavelengths are not needed to be measured for the intended application. They may be of very low signal strength and inconsequential in the desired integral results such as luminance and chromaticity. In other cases, the signal may be low and ignored for the spectral measurement, but added in as a correction to the integral measurement such as luminance, radiance or tristimulus X etc. In other cases, the data to be disregarded may be adequately predicted from other spectral data as in the case described above where only the LED peak region needs to be sampled to derive the entire LED spectrum. Thirdly, utilize knowledge of the spectral sensitivity ranges at various exposures and using various filters of the imaging system to predict a time-optimized set of imaging system settings to use to measure the scene.
(83) By extension, by acquiring spot spectroradiometer measurements of the brightest and dimmest areas of interest in a scene, reasonable determinations of the light level ranges that are important to quantify for each imaging system setting (i.e. filter selection) may be obtained. Thus, an imaging system measurement sequence may be derived with a sufficient set of short and long exposures, and potentially neutral density filters, to measure the required dynamic range of interest in the scene with all imaging system filters required in the sequence.
F. Filter Error Correction
(84) It is a common technique to correct the filtering error of the entire area of the pixelated detector within an imaging colorimeter by referencing a measurement made by a spectroradiometer of a location in a scene measured by both devices. However, while this correction is valid for the commonly sampled measurement area, it is deficient when the spectral responsivity of the imaging system is not spatially uniform across the entire area of the pixelated detector. In many imaging colorimeter designs, the filters are located between a lens and the camera sensor, and not located in collimated space (embodiments disclosed in U.S. Pat. No. 8,482,652B2 are examples). In these instruments, light from the lens exit pupil goes through the filter(s) at a range of angles to each location on the sensor. For a large 35 mm camera sensor and an electronic focus (EF) lens set to a small iris, light on the optical axis is nearly collimated, and the path length through the filter to the center of the sensor will be minimized. However, at the corners of the camera sensor, the light rays could be 23 or more from normal, depending upon the exact position of the exit pupil of the lens. The path length of the light rays travelling through the glass filter at 23 away from the optical axis is over 8% longer than the path of the light rays traversing the filter along the optical axis. This causes significant spectral filtering disparity at the corners as compared to light rays exiting the lens normal to the pixelated detector's center.
(85) Consider that the transmission, T(A) of the glass is described by the Beer-Lambert Law:
(86)
where E(A) is attenuation of the material as a function of wavelength, and L is the thickness of the material. For example, a colored glass has transmission values of 75% and 5% at wavelengths .sub.A and .sub.B respectively when the light rays traverse the glass at normal angles. That same colored glass would have transmission values reduced to 73% and 3.7% for wavelengths .sub.A and .sub.B respectively if the angles were tilted such that the path lengths of those light rays were increased by 10%. In general, the spectral transmission of the filter is path length dependent.
(87)
(88) Considering the errors that can result in an imaging colorimeter measurement as a result of the wavelength-dependent transmission variability for light rays traversing a colored glass filter, one method to generate a spatially dependent spectral responsivity of an imaging system is shown in
(89) Step 560 is to illuminate an integrating sphere with monochromatic light at wavelength .sub.min, where .sub.min is the shortest wavelength to be measured by the system. Step 562 is to align the imaging system to be calibrated such that its full field of view measures the uniform radiance light field (with wavelength .sub.min) emanating from the measurement port of the integrating sphere. Step 564 is to configure the imaging system such that each image pixel reports analog to digital units per second (ADU/s). This requires that each pixel in each image acquired by the system must be divided by the exposure time used, in seconds.
(90) Step 566 is to acquire a measurement of light from the integrating sphere measurement port using the imaging system and note the wavelength of the monochromatic light emanating from the measurement port. Step 568 is to align a calibrated spectroradiometer, providing spectral radiance in units of W/(sr.Math.m.sup.2)/nm, to the another measurement port of the integrating sphere. Step 570 is to acquire a measurement of the second sphere measurement port using the spectroradiometer and note the wavelength of the monochromatic light emanating from the measurement port.
(91) Step 572 is to increase the wavelength of the light illuminating the sphere by (for example =10 nm). If, in step 574, the longest (desired) wavelength (.sub.max) is not exceeded (for example .sub.max=1100 nm), then optionally adjust the lens focus in step 576 to keep the field of view constant. Track the lens focus setting for each wavelength if it is changed. Steps 566, 570, 572 are repeated until the longest (desired) wavelength (.sub.max) has been achieved (for example .sub.max=1100 nm) in step 574.
(92) Step 580 is to combine the measurements from step 566 into a wavelength-ordered 3D spectral responsivity array with dimensions of horizontal image pixel index (X), vertical image pixel index (Y), and monochromatic wavelength (A), r(X,Y,). Each element in this 3D array is in units of ADU/s, which can be optionally normalized to the maximum ADU/s value in the dataset. Should absolute units be preferred, each imaging system measurement should first be divided by the corresponding spectral radiance measured in step 570, thus providing units of (ADU/s)/(W/(sr.Math.m.sup.2)/nm).
(93) The process above is then repeated with all filters in the imaging system, following step 582, if there are any more filters, and step 584 in which the filter is changed. For example, an imaging colorimeter with three filters would have three corresponding wavelength-ordered 3D spectral responsivity arrays: r.sub.tri_X(X, Y,), r.sub.tri_Y(X, Y,) and r.sub.tri_Z(x,y,), where subscripts tri_X, tri_Y, and tri_Z identify the imaging system filter. The process may also be repeated for different lenses or lens settings to generate other spatially dependent spectral responsivity 3D arrays for the imaging system. In step 586, it is determined whether there is another lens setting to be calibrated, and if so, the lens setting is changed in step 588. If lens focus adjustments are done in step 576, above, then those same lens settings should be used for the measurements using the above calibration.
(94) Alternately, step 566 is to acquire a 2D, pixelated measurement of light from the integrating sphere measurement port using the imaging system and note the wavelength of the monochromatic light emanating from the measurement port. Step 568 is to align a calibrated spot spectroradiometer, providing spectral radiance in units of W/(sr.Math.m.sup.2)/nm, to another or the same measurement port of the integrating sphere. Step 570 is to acquire a measurement of the sphere using the spot spectroradiometer and note the wavelength and radiance of the monochromatic light emanating from the measurement port.
(95) Step 572 is to increase the wavelength of the light illuminating the sphere by (for example =10 nm). If, in step 574, the longest (desired) wavelength (.sub.max) is not exceeded (for example .sub.max=1100 nm), then if the focus is normally adjusted as a function of wavelength and corrected for registration errors, adjust the lens focus as needed use these functions and apply the registration corrections in step 576. Steps 566, 570, 572 are repeated until the longest (desired) wavelength (.sub.max) has been achieved (for example .sub.max=1100 nm) in step 574.
(96) In step 580 each imaging system measurement should first be divided by the corresponding spectral radiance measured in step 570, thus providing the responsivity r, with units of (ADU/s)/(W/(sr.Math.m.sup.2)/nm) at every pixel and at the wavelength stepping interval . The resulting dataset is a wavelength-ordered 3D spectral responsivity array with dimensions of horizontal image pixel index (X), vertical image pixel index (Y), and monochromatic wavelength (), r(X, Y,). Each element in this 3D array is in units of ADU/s/(W/(sr.Math.m.sup.2)/nm) or just ADU/s or even % if the dataset is normalized is normalized to the maximum responsivity or ADU/s value in the dataset.
(97) Following step 582, the process above is repeated in step 584 with all filters in the imaging system. For example, an imaging colorimeter with three filters would have three corresponding wavelength-ordered 3D spectral responsivity arrays: r.sub.tri_X(X, Y,), r.sub.tri_Y(X,Y,) and r.sub.tri_Z(X,Y,), where subscripts tri_X, tri_Y, and tri_Z identify the imaging system filter. The process may also be repeated for different lenses or lens settings to generate other spatially dependent spectral responsivity 3D arrays for the imaging system. In step 586, it is determined whether there is another lens setting to be calibrated, and if so, the lens setting is changed in step 588. If lens focus adjustments are done in step 576, above, then those same lens settings should be used for the measurements using the above calibration. Then compute F1 as a function at each pixel or for regions in the imager field of view. Then predict colorimetry errors as a function of field of view for the instrument. If the spot spectroradiometer can scan across the field of view, then a color correction can be made as a function of pixel number from the optical axis.
(98) The process may be applied to other filtered systems including, but not limited to imaging photometers, imaging radiometers, imaging colorimeters, multi-spectral imagers and hyperspectral imagers.
(99) Here is one method to apply a spatially dependent spectral responsivity correction to a measurement acquired by an imaging colorimeter with an integrated spot spectroradiometer: Measure an unknown source (such as an LCD display) with the integrated spot spectroradiometer, giving S(), the measured spectral radiance and calculated tristimulus integrals sX, sY and sZ. Measure the same unknown source with the imaging colorimeter, giving icX(X,Y), icY(X, Y) and icZ(X,Y), the imaging colorimeter's tristimulus measurements at each pixelated detector's pixel location, (X,Y). Within these measurements, the pixels corresponding to the spot on the unknown source sampled by the spectroradiometer, are used to calculate tristimulus values ic_sX, ic_sY and ic_sZ as measured by the imaging colorimeter. A simple correction is derived which is the scalar needed to adjust the imaging colorimeter measurement to match that of the spot spectroradiometer, for each imaging colorimeter filter. Such correction scalars are:
(100)
These first order scalar values are then applied to the respective imaging colorimeter measurements, giving corrected imaging colorimeter measurements icXcorr(X,Y), icY corr(X,Y), and icZcorr(X,Y):
(101)
(102) If the spatially dependent spectral responsivity of the imaging system is known for a given filter assembled within the system, then the spatial variation in detector response may be compensated by deriving and applying a unique scalar for each pixelated detector pixel location (X,Y), using the following steps: Using the imaging system pixels corresponding to the spot on the unknown source also sampled by the spectroradiometer, the average spectral responsivity is calculated for the imaging system at a location coincident with the spot spectroradiometer location, giving:
(103)
A scalar is then computed for every pixel in the imaging system, per filter assembled therein:
(104)
These scalars are then applied to the measurements obtained by the imaging colorimeter, per respective filter. The corrected imaging colorimeter tristimulus values at each pixel (X,Y) are thus:
(105)
Note that the spatial correction is weighted by the spectral power distribution of the source being measured and the spectral responsivity at each pixel.
(106) For some imaging systems the spatially dependent spectral responsivity may be rotationally symmetric (perhaps about the optical axis) and as a result, the spatially dependent spectral responsivity may be expressed as a function of distance from the optical axis.
(107) With the translating spot spectroradiometer, the measurement of spectral radiance S(), calculation of tristimulus integrals sX, sY and sZ, and derivation of all corresponding corrections, can be performed at one or more locations in the scene. By extension, further translation of the spot spectroradiometer to locations not yet measured allows additional spectroradiometer measurements of the scene to be obtained for the purpose of verifying any corrections derived for, and applied to, the imaging system.
G. Measuring RGB Displays
(108) The method may be further extended for measurements of RGB displays. If the process above is repeated such that S() and scalar corrections are created for every pixel when an RGB display is set to red, green, blue and white, then a matrix color correction can be created at each pixel location using the methods described by Yoshi Ohno et al., National Institute of Standards and Technology, Gaithersburg, Maryland: Four-Color Matrix Method for Correction of Tristimulus Colorimeters, Color Imaging Conference 1997: 301-305 and Four-Color Matrix Method for Correction of Tristimulus ColorimetersPart 2, Published in Proc., IS&T Sixth Color Imaging Conference (1998).
(109) For the case of a scene comprised of more than one light source to be measured, using the aforementioned imaging system correction processes allows each light source to have its own color correction and acceptable chromaticity tolerance (i.e. color region). An uncorrected imaging colorimeter measurement may be suitably segmented and compared against a library of stored color regions with associated color corrections, and thus a multitude of color corrections may be applied to the imaging colorimeter measurement.
(110) The method may also be expanded so that the correction is only applied if the uncorrected chromaticity is close to spectroradiometer chromaticity. For example, the uncorrected chromaticity coordinates (u,v) at any pixel location must each be within the color region with a radius of 0.09 and centered at chromaticity coordinates (u,v) by the spectroradiometer. Other shapes and definitions for color regions are possible. A set of unique spatial corrections may be made for a multitude of color or regions and applied selectively in a measurement. For example, green areas may have a different correction than the correction applied to red regions.
(111) Some scenes being imaged may have more than one light source to be measured. Each light source may have its own spatial correction and its own color region. An uncorrected imaging colorimeter measurement may be compared against a library of color regions and a multitude of color corrections may be applied to a single measurement. These methods may be applied to other imaging instruments such as multispectral imagers and hyperspectral imagers.
H. Further Variations
(112) Persons skilled in the art will readily understand that in the described embodiments above it is also possible to add various optical filters, either fixed or translated into the optical path at almost any position in the optical system. These filters can transform the measurement in many ways including, but not limited to: narrowing of the spectral measurement range, attenuation and/or polarization of light incident to the system, spectral filtering (e.g. tristimulus filters), etc.
(113) Persons skilled in the art will also recognize that the optical fiber sub-assembly may be configured with a multitude of fibers, each with different attributes such as numerical aperture, core shape, core and cladding diameters, spectral transmission range, polarization properties, etc.
(114) Persons skilled in the art will also recognize that all values, angles, dimensions, proportions, configurations etc., are given as examples and do not definitively define the properties of a particular embodiment.
(115) The prism mirror may be replaced with any mirror or mirrored glass with a reflection angle of not necessarily 45 degrees, to deflect the light to the optical fiber.
(116) Persons skilled in the art will readily understand that where a spectroradiometer has been described, a non-radiometrically calibrated spectrometer may also be substituted for some measurement applications and vice-versa.
(117) When combined with an electronic focusing lens, the system may be refocused for different wavelength bands and the mirror/fiber assembly in turn may be translated to compensate for image shifting due to re-focusing. This procedure keeps the measurement spot(s) in focus for the same location(s) in the scene.
(118) If the spectral spot measurement is calibrated at a multitude of locations for particular quantities such as luminance or chromaticity, then the spot spectrometer can acquire spatially differentiated measurements, which may in turn be used to provide a spatially variant correction of the pixelated detector's measurements. For example, the luminance correction could be fit to be a function of just one spatial variable such as the distance from the optical axis; or a function with two spatial variables: rectilinear pixel space (X,Y) or polar coordinate space (,) for example. The calibrations and corrections may be extended to include a multitude of measurement quantities.
(119) In some implementations of a spot spectroradiometer, there may also be an integrated 2D spectral imager in the same instrument. For example, a filter wheel may contain a selection of spectrally selective filters that enable a multispectral or hyperspectral imager (e.g. patent U.S. Pat. No. 9,909,920B2 and patent application published with number US2022/0146309).
(120) Alternatively, there may be an LVBF in addition to, or instead of the filter, 148 in
(121) The spot spectroradiometer measurement results may be used to enhance the 3D hyperspectral or 2D filtered or unfiltered measurements of the pixelated detector in multiple ways. For example, a measurement by the spot spectroradiometer may be used to improve the accuracy of the multi-spectral data cube acquired by the pixelated detector by any or all of: filling in gaps where there is missing, noisy or corrupted spectral data; providing a more detailed model of the spectrum; extending the spectral data set to cover a wider wavelength range; providing a reference spectrum which can be used to select a set of exposure times to use for the multi-spectral measurements to be obtained by the pixelated detector.
(122) Some or all of the functions of the device with an imaging detector may be controlled with one or more processors that execute computer-readable instructions stored in a non-transitory computer-readable memory.
(123) Some of the embodiments of the invention have the 2D imager on the same optical axis as the lens or, the imager is on the axis via a mirror or beam splitter. Users familiar with optics will understand that, the embodiments are essentially equivalent if the positions of the imager and fiber optic assembly with stage are swapped relative to the mirror or beam splitter.
I. Industrial Applicability
(124) The features and assemblies disclosed herein may provide one or more of the following advantages, depending on the configuration of the particular embodiment of the invention.
(125) Obstruction-free and distortion-free imaging with the two dimensional pixelated detector: When the mirror and fiber sub-assembly are being used for measurement with a spectrometer, they occlude the two-dimensional pixelated detector. When the pixelated detector is being used, then the fiber and mirror sub-assembly is translated so as not to obstruct the illumination of the pixelated detector. By comparison, fixed-position beam splitters (which are always simultaneously present in the optical paths to the two dimensional pixelated detector and spectrometer) can cause unwanted image distortions, light scatter and ghosting artifacts.
(126) Facilitating the measurement of multiple locations within the field of view: Whereas the prior art systems have a fixed measurement location and one optical fiber attached measuring device, the new invention can measure a multitude of locations by having a multitude of fibers, or by translating the stage, or by both methods.
(127) Multiple spectra from a fiber array: Instead of a single optical fiber, there may be a rigid fiber sub-assembly with a multitude of optical fibers that are all aligned to the focal plane of the lens. The fibers may be coupled to multiple single-channel spectrometers, or to one or more multi-channel spectrometers. Optical fibers in the sub-assembly may additionally be connected to other light measurement devices.
(128) Optimum focus: In another manifestation of the invention, there are a plurality of optical fibers connected to a plurality of spectrometers or other light measurement devices which may all have different spectral sensitivity ranges. To compensate for chromatic aberration of the lens, the fibers may be positioned at varying distances from the turning mirror depending upon the measurement wavelength range of the instrument connected to the end of each fiber.
(129) Extended wavelength range measurement of a location within the lens field of view: Two or more optical fiber assemblies may be translated in some sequence such that they all measure (in focus) the same location within the field of view of the lens. If the fiber assemblies are connected to different spectrometers that have different wavelength ranges, then the wavelength measurement range is that formed by the union of the wavelength ranges from the individual spectrometers.
(130) High positioning accuracy: Differing from common filter wheel systems, long travel translation slides can have accuracy specifications which achieve micron levels. Linear translation slides paired with high resolution encoders are available with 1 m positioning accuracy. In comparison, the inventor has not evaluated a filter wheel positioning system with better accuracy than 20 m. Furthermore, to date, the drive mechanisms evaluated for filter wheels have backlash and hysteresis in their motion, leading to rotation in only one direction being recommended for best rotational positioning accuracy.
(131) Compact: Because the fiber guide is very close to the turning mirror and sensor, the mirror need not be very large. As a result, a smaller mirror means a reduced mass to translate and thus lower-cost components can be used for this sub-assembly. The inventor(s) have recognized that there are many applications that may benefit from an imaging spot spectrometer that consumes a small volume of space within the back focus region found between the camera lens and camera sensor. For example, an instrument could be created with two filter wheels in the optical path of the camera and there would still be enough volume remaining in front of the camera sensor within which a translating mirror assembly may be installed. For the application where a translation of the measurement spot(s) is desired, translation of the fiber(s) is a simple and economical solution. If the fiber(s) were fixed, the translation could alternatively be achieved by translating the entire camera assemblywhich would be much more expensive and complicated.
(132) Fast: Typically, as filter wheel size, mass, and rotation speed increase, so too do the rotational impulses exerted on the filter wheel as it is accelerated to leave a position or decelerated to arrive at a position. For a mirror assembled onto such a filter wheel along with other optical filters, these rotational impulses or vibrations can lead to mirror and filter positioning errors and other mechanical issues in the system. To counteract this problem: filter wheel rotation speeds, accelerations, and decelerations may be reduced, thus slowing the overall rotation speed of the filter wheel; or the size and mass of the entire instrument and corresponding mounting may need to be increased to reduce the system sensitivity to such vibrations and impulses. As mentioned above, the components which are assembled onto the linear slide are typically much smaller in size and mass due to the proximity of the linear slide to the camera sensor. Thus the distance the linear translation slide assembly must travel is shorter which depending upon its speed may yield positioning time savings.
(133) Translation Stage Supports More Features: The mirror assembled onto the linear slide need only be large enough to couple and focus the light cone from the optical system onto the optical fiber at the desired position(s) in the field of view. Suitable linear translation slides exist that can support linear travel exceeding 300 mmwhich is several times the width of a 35 mm camera sensor. The linear translation slide thereby can be used to insert other optical components into the optical path between the lens and the camera sensor such as Linear Variable Bandpass Filters (LVBFs), density filters, polarizers or tristimulus filters. Additional turning mirrors may be added to the stage to couple light sources not within the lens field of view to the pixelated detector or spectrometer, or to redirect light rays from sources not within the lens field of view such that they traverse through the instrument and exit the lens, for calibration and verification purposes.
(134) Simple components: Whereas the turning mirror in the invention is realized using a simple, low cost rectangular prism mirror; the optimal shape for a turning mirror mounted in a filter wheel would likely need expensive, custom machining to optimize its function.
(135) Again, depending on the configuration of the particular embodiment of the invention, one or more of the following disadvantages of the prior art are not present:
(136) Beam splitter: Since the camera sensor, optical fiber, and lens optical axis are all fixed in position relative to one another, the fiber guide assembly must be positioned such that it does not occlude the light cone from the lens to the camera sensor. As a result, a beam splitter or mirror would need to be positioned sufficiently far from the camera sensor to provide a focal plane for the optical fiber which is outside of the light cone from the lens to the camera. This restriction would also force the capturing of a larger cone of light rays, and the size of the beam splitter or mirror would always be the same size as the camera sensor or larger. Consider a 43 mm diagonal or more for modern high resolution scientific cameras. With a back focus of 44.5 mm, fitting a beam splitting cube behind an EF lens and in front of the sensor leaves little remaining space. By comparison the height of a turning mirror in the proposed invention could consume 10 mm of space or less.
(137) Rotating mirrors: For a system with a fixed optical fiber and turning mirror assembled on a rotating filter wheel, the fixed fiber must be positioned so as not to occlude the light cone from the lens to the camera sensor. As a result, the turning mirror may need to be quite far from the sensor and thus be quite large. This is especially true for fast lenses and large sensors. The large mirror will take up more space in the lens back-focus region, may be heavier, and thus the system may require mounting hardware and associated motors of increased size and cost.
(138) Expensive Optics: Whereas in some of the novel embodiments described above, the translating mirror size is practically invariant with the camera sensor size, the fixed-position optical fiber designs utilizing turning mirror or beam-spitting optics may be larger, heavier and more expensive.
(139) Image artifacts: Careful design, mounting and alignment of beam splitters would be needed to avoid certain types of image distortion, polarization problems, ghost images and light scatter.
(140) Complicated spectral scanning across the field of view: Traditionally, to scan a spectral spot(s) across the scene, the entire instrument would be translated or rotated in one or two dimensions. The external mechanical stages, robots, etc., required to facilitate such motion need significant time to set up and may be very expensive. By comparison, a feature of some embodiments of the invention is a system to translate an internal turning mirror and fiber to facilitate spectral spot measurements at multiple locations across the scene, eliminating the requirement for one dimension of external scanning hardware.
J. Numbered Embodiments
(141) 1. An imaging system comprising: a lens; an pixelated detector; an optic fiber assembly mounted on the translation stage, which moves an entrance surface of the optic fiber(s) into one or more positions within a focal plane of the lens; and wherein there is a mechanical stage and mirror between the lens and fiber optic assembly such that the stage can position the mirror so that an image is formed at the pixelated detector plane or at the plane of the optic fiber assembly; or wherein there is a fixed beam splitter and between the lens and fiber optic assembly such that an image is formed at both the pixelated detector plane and at the plane of the optic fiber assembly; and wherein there are no lenses between a plane in which the translation stage is moved and the pixelated detector. 2. An imaging system comprising: a lens; a two-dimensional array detector; a translation stage; and an optic fiber assembly mounted on the translation stage, which moves an entrance surface of the optic fiber assembly into one or more positions within a focal plane of the lens; a beam splitter between the lens and a plane in which the translation stage is moved; wherein part of the light that passes through the lens is focused onto a two-dimensional array detector and part of the light is imaged onto the optic fiber assembly. 3. (