Computational High-Speed Hyperspectral Infrared Camera System

Abstract

A hyperspectral infrared imaging system includes optical components, multi-color focal plane array or arrays, readout electronics, control electronics, and a computing system. The system measures a limited number of spatial and spectral points during image capture and the full dataset is computationally generated.

Claims

1. A hyperspectral infrared imaging system comprised of optical components, multi-color focal plane array or arrays, readout electronics, control electronics, and a computing system, which measures a limited number of spatial and spectral points during image capture and the full dataset is computationally generated.

2. A system in claim 1 that includes a coded aperture.

3. A system in claim 1 that includes a set of spectral filters.

4. A system in claim 1 that includes an interferometer.

5. A system in claim 1 that includes a comb filter.

6. A system as in claim 1 in which a subset of pixels is not read during operation and replaced by the computing system.

7. A system in claim 1 in which a subset of the spectral wavelengths is not read during operation and are replaced by the computing system.

8. A system in claim 1 in which the spectral sensitivity of the pixels is known but not uniform and/or constant in space and/or time.

9. A system in claim 1 that includes at least one of dispersive, translational, or focusing elements such that the input light to the system can contain multiple wavelengths possibly overlapping in space but can be moved with respect to the image plane and acquire images with different levels of defocus.

10. A system in claim 1 in which the optics provide each pixel with more than 1 spectral band which is later spectrally reconstructed by the computing system.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0028] FIG. 1 is a schematic view of an exemplary system of the invention.

[0029] FIG. 2 is a demonstrative figure showing a method of inpainting using best pixels, some portions of the images have been redacted;

[0030] FIG. 3 is a demonstrative figure showing a method of inpainting using best pixels for three wavelengths of light, red, blue and green;

[0031] FIG. 4 is a demonstrative figure showing inpainted images using the masks shown in FIG. 3 and a combined image, some portions of the images have been redacted.

DETAILED DESCRIPTION

[0032] While this invention is susceptible of embodiment in many different forms, there are shown in the drawings, and will be described herein in detail, specific embodiments thereof with the understanding that the present disclosure is to be considered as an exemplification of the principles of the invention and is not intended to limit the invention to the specific embodiments illustrated.

[0033] This application incorporates U.S. provisional Application Ser. No. 63/185,940 filed May 7, 2021; U.S. Provisional Application Ser. No. 63/185,934 filed May 7, 2021; U.S. Pat. No. 10,256,072 and U.S. Ser. No. 17/687,170 filed Mar. 4, 2022 in their entireties to the extent that they are not contrary to the teaching of the present disclosure.

[0034] FIG. 1 illustrates in schematic form an exemplary embodiment of a hyperspectral infrared imaging system 10. The system 10 includes optical components 16 receiving infrared radiation 14, multi-color focal plane array or arrays (Infrared Focal Plane Array or “IRFPA”) 20 receiving infrared radiation from the optical components 16, readout electronics 22, such as a read-out integrated circuit (ROIC), generating signals 24, control electronics 26 receiving the signals, and a computing system 28. The system measures a limited number of spatial and spectral points during image capture and the full dataset is computationally generated in the computing system 28. IRFPA structures are described in U.S. Pat. Nos. 4,956,686; 5,479,032; 5,518,934; 6,657,194 and 9,883,178 herein incorporated by reference to the extent they are not contrary to the present disclosure. ROIC structures are described in U.S. Pat. Nos. 9,410,850; 5,431,328; 7,462,831; 5,196,703; 6,657,194 and 5,581,084, herein incorporated by reference to the extent they are not contrary to the present disclosure.

[0035] The system can include a coded aperture.

[0036] The system can include a set of spectral filters.

[0037] The system can include an interferometer.

[0038] The system can include a comb filter.

[0039] The system can operate in a manner in which a subset of pixels is read and a subset of pixels is not read during operation and replaced by the computing system. The computing system can use inpainting to construct a full dataset from the subset of pixels read.

[0040] The system can operate in a manner in which the spectral sensitivity of the pixels is known but not uniform and/or constant in space and/or time.

[0041] The system can include at least one of dispersive, translational, or focusing elements such that the input light to the system can contain multiple wavelengths possibly overlapping in space but can be moved with respect to the image plane and acquire images with different levels of defocus.

[0042] The system can operate in a manner in which the optics provide each pixel with more than one spectral band which is later spectrally reconstructed by the computing system.

[0043] FIGS. 2 and 3 demonstrate a three-color example of the use of hyperspectral images.

[0044] As shown in FIG. 3, the same array IRFPA can be illuminated with multiple wavelengths to identify the best performing pixels for SWIR, LWIR and MWIR, for example. These pixels form the masks for the individual color images that can be combined together to form an inpainted hyperspectral image shown in FIG. 4.

[0045] The initial step is to determine which pixels in the camera are most sensitive to each selected wavelength—this is done by illuminating the camera with a single wavelength and mapping the response. This only needs to be done once per wavelength and can be done during the manufacture of the camera. Once the best pixels for each wavelength are identified, those pixels become the sub-sampled mask for that wavelength. The software correlates that wavelength to a color, e.g., red, green or blue.

[0046] From the total image, a red image is inpainted from the best red pixels, a green image is inpainted from the best green pixels and a blue image is inpainted from the best blue pixels. The separation of the pixels into colors is performed by the software, as is the inpainting.

[0047] When the whole image is read out, it contains red, blue and green sub-sampled images that are reconstructed using inpainting methods. Accordingly, scanning over a large range of different images for the different energy range is not required, all the information is in a single image.

[0048] If the whole imaging array is used for the hyperspectral data cube then the detector will have the regular read-out performance, i.e., speed. If a smaller number of pixels is used for read-out, then the detector can balance read-out speed against the breadth of the hyperspectral datacube.

[0049] Sub-sampling and inpainting algorithms are disclosed in U.S. Pat. Nos. 7,840,086; 10,224,175 and 10,256,072 herein incorporated by reference to the extent that they are not contrary to the present disclosure. Other inpainting techniques are known and examples are described in U.S. Pat. Nos. 9,467,628; 10,431,419; 11,056314; 10,740,881 and 11,080,833 herein incorporated by reference to the extent that they are not contrary to the present disclosure.

[0050] From the foregoing, it will be observed that numerous variations and modifications may be effected without departing from the spirit and scope of the invention. It is to be understood that no limitation with respect to the specific apparatus illustrated herein is intended or should be inferred.