Turbulence-free camera system and related method of image enhancement

Abstract

A turbulence-free CCD camera system with nonclassical imaging resolution, for applications in long-distance imaging, such as satellite and aircraft-to-ground based distant imaging, utilizing an intensity-fluctuation correlation measurement of thermal light. The proposed camera system has the following advantages over classic imaging technology: (1) it is turbulence-free; (2) its spatial resolution is mainly determined by the angular diameter of the light source. For example, using sun as the light source, this camera may achieve a spatial resolution of 200 micrometer for any object on Earth. 200-micrometer resolution is insignificant for short distance imaging, however, taking a picture of a target at 10-kilometer, a classic camera must have a lens of 90-meter diameter in order to achieve 200-micrometer resolution. Unlike a classic camera, the proposed turbulence-free CCD camera system has adequate spatial resolution zoom a long distance and still distinguish the objects within a small area, even with a small lens.

Claims

1. A camera system, comprising: a beam splitter receiving thermal light photons from an object or area to be imaged, said thermal light photons passing through the beam splitter to either an image sensor or a bucket photodetector, wherein the image sensor passes information to a first Pos-Neg identifier and the bucket photodetector passes information to a second Pos-Neg identifier and each of the first and second Pos-Neg identifiers pass information to a positive-negative fluctuation circuit (PNFC) comprising at least one processor, at least one memory operatively associated with the processor, such memory containing a positive-negative fluctuation protocol algorithm, and wherein the PNFC detects photon coincidence from intensity fluctuation correlation of sunlight, and passes the resulting correlation information to an image viewer.

2. The camera system of claim 1 wherein the information passed from the image sensor and the bucket photodetector to the respective Pos-Neg identifier comprises a registration time of each photodetection event.

3. The camera system of claim 1, wherein the information passed from each Pos-Neg identifier to the PNFC comprises counting numbers classified as positive and negative fluctuations.

4. The camera system of claim 1 wherein the PNFC and associated software records a registration time of each photodetection event from the image sensor and the bucket photodetector in two independent event timers, which are divided in to sequences of short time windows.

5. The camera system of claim 4, wherein the PNFC and associated software analyzes photon counting number fluctuations in each short time window, calculating an average counting numbers per short time window.

6. The camera system of claim 1 wherein an image produced on the image viewer is a series of images comprising a video sequence.

7. A method of use of the camera system of claim 1, comprising the following processor implemented steps not necessarily in sequential order: providing a series of frames of a given region of interest; determining a value of each pixel at each location within each frame to form a pixel value array for each frame; summing the pixel values in each frame to obtain a frame intensity for each frame; multiplying the pixels in the pixel value array by the frame intensity to produce a frame intensity multiplied pixel value array; summing the frame intensity multiplied pixel value arrays together and dividing by the number of frames to obtain an average of the frame intensity multiplied pixel value arrays; using the pixel value arrays, creating an array of average pixel values; determining an average frame intensity for the series of frames; multiplying the array of average pixel values by the average frame intensity for the series of frames; and subtracting the array of average pixel values multiplied by average frame intensity from the average of the frame intensity multiplied pixel value arrays to provide an array of modified pixel values to form an improved image.

8. The method of claim 7, wherein the camera system is located on a satellite, airplane, helicopter, drone or other aerial vehicle.

9. The method of claim 7, wherein the thermal light photons inputted into the camera system were influenced by effects of turbulence or adverse weather effects.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is a schematic drawing of the experimental setup for studying intensity-fluctuation and second-order spatial correlation and anti-correlation by scanning point-like photodetector DA transversely.

(2) FIG. 2 is a general schematic diagram for measuring statistical correlation. In this experiment, the observable are the positive (above mean value) and negative (below mean value) intensity fluctuations.

(3) FIGS. 3A, 3B, and 3C are schematic drawings of a turbulence-free imaging system.

DETAILED DESCRIPTION OF THE INVENTION

(4) The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.

(5) Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout. As used herein the term and/or includes any and all combinations of one or more of the associated listed items.

(6) The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the full scope of the invention. As used herein, the singular forms a, an and the are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms comprises and/or comprising, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

(7) The following terms, as used herein, have the following meanings:

(8) A charge-coupled device (CCD) is a device for the movement of electrical charge, usually from within the device to an area where the charge can be manipulated, for example conversion into a digital value. This is achieved by shifting the signals between stages within the device one at a time. CCDs move charge between capacitive bins in the device, with the shift allowing for the transfer of charge between bins.

(9) A coincidence circuit is an electronic device with one output and two (or more) inputs. The output is activated only when signals are received within a time window accepted as at the same time and in parallel at both inputs. Coincidence circuits are widely used in particle physics experiments and in other areas of science and technology.

(10) Photon Noise (aka Shot noise) is a type of electronic noise which originates from the discrete nature of electric charge. The term also applies to photon counting in optical devices, where photon noise is associated with the particle nature of light.

(11) Ghost imaging is a technique that produces an image of an object by combining information from two light detectors: a conventional, multi-pixel detector that does not view the object, and a single pixel (bucket) detector that does view the object.

(12) Type-one ghost imaging is ghost imaging in which the imaging resolution is determined by the angular diameter, , of the thermal source, and not by the diameters of the two imaging lenses.

(13) Object means: an area, a scene, an object or objects, a landscape, overhead view of land or an object or objects, or a combination thereof.

(14) Frame means: a picture, an image or one of the successive pictures on a strip of film or video.

(15) Process means an algorithm, software, subroutine, computer program, or methodology.

(16) Algorithm means: sequence of steps using computer software, process, software, subroutine, computer program, or methodology.

(17) Image sensor means: a camera, charge coupled device (CCD), video device, spatial sensor, or range sensor. The image sensor may comprise a device having a shutter controlled aperture that, when opened, admits light enabling an object to be focused, usually by means of a lens, onto a surface, thereby producing a photographic image OR a device in which the picture is formed before it is changed into electric impulses.

(18) PNFC protocol means the novel photon number fluctuation correlation measurement protocol as fully disclosed herein.

(19) PNFC Circuit (PNFC) means a coincidence circuit employing the PNFC protocol.

(20) Processor or image processor as used in the following claims includes a computer, multiprocessor, CPU, minicomputer, microprocessor or any machine similar to a computer or processor which is capable of processing algorithms.

(21) Operations as used in the following claims includes steps, a series of operations, actions, processes, subprocesses, acts, functions, and/or subroutines.

(22) Array refers to a systematic arrangement of data in rows and columns. An example of an array is a matrix which is a rectangular array of numbers, symbols, or expressions.

(23) Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

(24) Our ghost imaging takes advantage of the nonlocal second-order point-to-point correlation of either entangled light or thermal light. In the case of thermal light, the useful point-to-point correlation is measured from the intensity fluctuations. We may call it intensity-fluctuation correlation. The intensity-fluctuation correlation of sunlight is turbulence-free, meaning it is not subject to distortion from atmospheric turbulence when traveling through the air. It is necessary to emphasize that other types of ghost imaging technologies may not be turbulence-free. For example, a different type of thermal light ghost imaging uses a so-called speckle-to-speckle correlation to form a ghost image in coincidences. In these kind experiments two sets of classical imaging systems are employed to image the speckles of the source onto the object plane and the image plane, respectively. The speckle-to-speckle correlation is produced by the first-order classical images of the speckle of the source, which is turbulence-sensitive, and thus this kind of ghost imaging. To distinguish our ghost imaging from the speckle-to-speckle ghost imaging, we name ours as type-one and the other type-two.

(25) In our newly developed ghost imaging process, a positive-negative fluctuation (PNF) protocol is employed to distinguish the positive and negative intensity fluctuations of each CCD element and the bucket detector within a small time window. With this protocol, we are able to measure and calculate the correlations between positive-positive, negative-negative, positive-negative and negative-positive intensity fluctuations between the CCD and the bucket detector. This kind of correlation is referred to as second order intensity-fluctuation correlation, and produces the ghost imaging process with a 100% contrast ghost image. This original ghost imaging process is further enhanced by the novel PNFC protocol newly disclosed herein below.

(26) Our newly designed PNFC protocol divides the timely accumulative measurement into short time windows, t, and calculates the mean intensity of the measurement. The PNFC protocol is then make a positive-negation intensity-fluctuation judgment for the measured intensity within each short time window t.sub.j. If the measured intensity I.sub.j is greater than the mean intensity, we name it positive-fluctuation. If the measured intensity I.sub.j is smaller than the mean intensity, we name it negative-fluctuation. With this protocol, we are able to separately record the measured positive-fluctuation and negative-fluctuation of the CCD and the bucket detector into two sets of sequences of short time windows t.sub.j. We are thus able to calculate the intensity-fluctuation correlation between the positive-positive, negative-negative, positive-negative, and negative-positive intensity fluctuations of the CCD and the bucket detector. Significantly, the PNFC protocol brings us a turbulence-free 100% intensity-fluctuation correlation, which does not contain the turbulence-sensitive one-half that is indistinguishable in the traditional second-order correlation measurements of two identical classical images.

(27) In our novel PNFC protocol, the 100% intensity-fluctuation correlation of thermal light, and the turbulence-free CCD camera have been experimentally demonstrated.

(28) The use of intensity-fluctuation correlation for ghost imaging, especially sunlight ghost imaging, provides an avenue for enhancement of the utility of long-distance photography due to the following advantages over classical imaging technology: (1) it is nonlocal, i.e., the ghost camera can see targets that cannot be resolved by classical cameras; (2) it is turbulence-free; and (3) its spatial resolution is mainly determined by the angular diameter of the light source. These characteristics are especially advantageous in the areas of satellite and aerial photography.

(29) For example, a sunlight ghost image of any object on earth may achieve a spatial resolution of 200 micrometer because the sun has an angular diameter of 0.53 degree relative to Earth. Such level of spatial resolution is insignificant for short distance imaging, but to take a picture of a target at 10 kilometers, a classical camera must have a lens 90 meters in diameter to achieve 200-micrometer resolution.

(30) In addition to enhanced spatial resolution, the disclosed turbulence-free CCD camera system uses natural sunlight for turbulence-free distance imaging applications with full color reproduction.

(31) The invention comprises an enhancement of ghost imaging from its classical simulation. There are currently two known types of thermal light ghost imaging. Type-one ghost imaging, which is used in the present invention, uses the intensity-fluctuation correlation <I.sub.o I.sub.i>, or photon number fluctuation correlation <n.sub.o n.sub.i>, between the object-plane and the image-plane to produce the second-order image in the joint measurement between a bucket photodetector and a CCD array. Type-two can be considered as a classical simulation of the original ghost imaging. A type-two ghost image is produced by two sets of identical intensity distributions or the so called speckle-to-speckle correlation of thermal light. Mathematically, the two are easily recognized from their image-forming correlation function:

(32) Type-One:

(33) $\begin{array}{c}{R}_c\left({\stackrel{.fwdarw.}{}}_i\right)=d{\stackrel{.fwdarw.}{}}_o{.Math.A\left({\stackrel{.fwdarw.}{}}_o\right).Math.}^2.Math.I_o{I}_i.Math.\\ =d{\stackrel{.fwdarw.}{}}_o{.Math.A\left({\stackrel{.fwdarw.}{}}_o\right).Math.}^2.Math.n_o{n}_i.Math.\\ =d{\stackrel{.fwdarw.}{}}_o{.Math.A\left({\stackrel{.fwdarw.}{}}_o\right).Math.}^2{\mathrm{somb}}^2\frac{}{}\left({\stackrel{.fwdarw.}{}}_0-{\stackrel{.fwdarw.}{}}_i\right)\end{array}$

(34) where R.sub.c is the coincidence counting rate; {right arrow over ()}.sub.o and {right arrow over ()}.sub.i are the transverse coordinates of the object-plane and the image-plane, respectively; A({right arrow over ()}.sub.o) is the aperture function of the object; somb(x)=2J.sub.1(x)/x with J.sub.1(x) the first-order Bessel function and the wavelength.

(35) Type-Two:

(36) $\begin{array}{c}{R}_c\left({\stackrel{.fwdarw.}{p}}_i\right)=d{\stackrel{.fwdarw.}{}}_o{.Math.A\left({\stackrel{.fwdarw.}{}}_o\right).Math.}^2.Math.I_o.Math..Math.I_i.Math.\\ =d{\stackrel{.fwdarw.}{}}_o{.Math.A\left({\stackrel{.fwdarw.}{}}_o\right).Math.}^2.Math.n_o.Math..Math.n_i.Math.\\ =d{\stackrel{.fwdarw.}{}}_o{.Math.A\left({\stackrel{.fwdarw.}{}}_o\right).Math.}^2\mathrm{somb}\frac{D_o}{s_o^0}\left({\stackrel{.fwdarw.}{}}_s-{\stackrel{.fwdarw.}{}}_o/m_o\right)\\ \mathrm{somb}\frac{D_i}{s_i^0}\left({\stackrel{.fwdarw.}{}}_s-{\stackrel{.fwdarw.}{}}_i/m_i\right)\end{array}$

(37) where {right arrow over ()}.sub.s is the transverse coordinate of the source-plane; D.sub.o(D.sub.i) is the diameter of the imaging lens on the object (image) side; s.sub.o.sup.0(s.sub.i.sup.0) is the distance between the source to the imaging lens on the object (image) side; m.sub.o(m.sub.i) is the magnification factor of the classical imaging system on the object (image) side. The two sorb-functions are made by two sets of classical imaging systems, which produce two sets of identical speckles of the source-plane onto the object-plane and the image-plane.

(38) In a preferred embodiment, the HBT intensity correlation used in our variety of ghost imaging is naturally turbulence-free, and thus the software associated with the PNFC is not required to perform corrections.

DETAILED DESCRIPTION OF THE FIGURES

(39) FIG. 1 schematically illustrates the photon number fluctuation correlation measurement protocol, namely the positive-negative fluctuation correlation measurement circuit (PNFC). The PNFC protocol records the registration time of each photodetection event of D.sub.A and D.sub.B in two independent Event Timers along their time axes t.sub.A and t.sub.B, which are divided into sequences of short time windows t.sub.j, j=A, B. The circuit and the associated software analyze the photon counting number fluctuations in each time window t.sub.j. The software first calculates the average counting numbers per short time window, n.sub.A, and n.sub.B. Two virtual logic circuits (Pos-Neg identifiers) classify the counting numbers per window as positive and negative fluctuations based on n.sub.A, and n.sub.B,
n.sub.jAn.sub.jAn.sub.A; n.sub.jBn.sub.jBn.sub.B,

(40) Where j=1 to N.sub.win labeling the jth short time window. N.sub.win is the total number of time windows in a measurement. After synchronizing the time windows of channel-A and channel-B, the fluctuation-fluctuation coincidences in the jth time window is calculated,
R.sub.j=n.sub.jAn.sub.jB.

(41) The statistical Fluctuation-fluctuation correlation is thus calculated as follows:

(42) $.Math.n_A{n}_B.Math.=\underset{j}{.Math.}{R}_j=\underset{j}{.Math.}{n}_{\mathrm{jA}}{n}_{\mathrm{jB}}.Math.I_A{I}_B.Math..$

(43) FIG. 2 is a schematic drawing evidencing one embodiment of the disclosed turbulence-free camera system 100 that uses natural sunlight for turbulence-free distant imaging applications with enhanced spatial resolution and full color reproduction. In FIG. 2, correlated photons 110 pass through beam splitter 120 and are split, with split photons 111 passing to bucket photodetector 130 and thereby to Pos-Neg Identifier 140 and thereby to a PNF circuit (PNFC) 170, while split photons 112 pass to CCD Camera 150 and thereby to Pos-Neg Identifier 160 and thereby to a PNFC 170, with PNFC 170 finally passing reconstituted imaging information to final image viewing device 180.

(44) The operations employed by camera system 100, and illustrated by FIG. 2, are as follows. CCD camera 150 takes a picture through atmospheric turbulence. The image of the target on the CCD array will be blurred due to the turbulence or due to the poor spatial resolution of the camera, if its imaging lens is not large enough for the distance to the imaged object 200. However, with the help of the bucket photodetector 130 and a novel coincidence detection protocol, namely the PNF protocol, a turbulence-free image of the target of 200 micrometer imaging resolution can be observed from the intensity fluctuation correlation measurement. The positive-negative fluctuation protocol distinguishes the positive and negative intensity fluctuations, or photon number fluctuation, of each CCD element and bucket photodetector 130 within a small time window. With the novel protocol, we are able to measure and calculate the correlations between the positive-positive, negative-negative, and negative-positive intensity fluctuations, or photon number fluctuations between the CCD camera 150 and the bucket photodetector 130. This second order correlation produces an image based on the coincidences.

(45) FIGS. 3(a)-(c) comprise a series of graphs indicating the spatial resolution of various types of imaging. FIG. 3(a) evidences a classical image of a spatial resolution gauge directly observed from the CCD camera. FIG. 3(b) evidences classically correlated image of the same gauge which is the result of the formula
I({right arrow over ()}.sub.o)I({right arrow over ()}.sub.o).

(46) FIG. 3(c) evidences the image of the same gauge as FIGS. 3(a) and 3(b), but taken by the new turbulence-free ghost imaging camera system. Comparison of the graphs indicates that: the image of the classical CCD camera in FIG. 3(a) is almost invisible; the classically correlated image of FIG. 3(b) is improved (roughly a 1.414 gain), but remains blurred; while the type-one ghost image of the gauge illustrated in FIG. 3(c) remains 100% resolvable.

Example 1Construction of Prototype Turbulence-Free CCD Camera

(47) In this example a prototype turbulence-free CCD camera is developed that is able to take second-order turbulence-free images.

(48) Materials

(49) The prototype unit consists of a commercial research grade CCD camera, and hardware and software to interface with the PNFC system. Intensity fluctuation-fluctuation correlation calculations, involving large-dimensional matrix algebra, are conducted using commercial PCs and multi-crosstalk graphic cards.

(50) CCD Camera

(51) Based on the optical and electronic parameters of the CCD camera, the hardware parts are designed for a ghost imaging system including the PNFC system and the hardware-interface between the CCD camera and the PNFC protocol as well as coincidence-measurement circuits. This results in a working PNFC system and hardware interface between the CCD camera and the PNFC.

(52) Multi-Crosstalk Graphic Cards

(53) The software is developed for multi-crosstalk graphic cards for the system for the coincidence-measurement circuit and positive-negative fluctuation analysis. The production of this software results in working software for taking data from the CCD and the bucket detector, analyzing and identifying the positive-negative fluctuations, calculating the correlations between the positive-negative fluctuations, producing the ghost image from the fluctuation-fluctuation correlation.

(54) Interface Software and Hardware

(55) The above individual working software is then interfaced with the individual working hardware systems to produce a working ghost imaging system with the PNFC.

(56) Testing

(57) The constructed turbulence-free PNFC camera is then tested comparing it with a classic camera that has similar lens and zooming ability. The main test shall be the spatial resolution of the prototype system that results in a working prototype system and a testing report on its turbulence-free nature and nonclassical spatial resolution of a 200 micrometer grid at a distance of 100 m.

(58) The references recited herein are incorporated herein in their entirety, particularly as they relate to teaching the level of ordinary skill in this art and for any disclosure necessary for the commoner understanding of the subject matter of the claimed invention. It will be clear to a person of ordinary skill in the art that the above embodiments may be altered or that insubstantial changes may be made without departing from the scope of the invention. Accordingly, the scope of the invention is determined by the scope of the following claims and their equitable Equivalents.