DATA DIGITIZATION AND DISPLAY FOR AN IMAGING SYSTEM

Abstract

A system and method for digitizing data from an imaging system includes sampling a signal from an optical detector with a first circuit having a first attenuation and with a second circuit having a second attenuation different than the first attenuation. The system and method further includes digitizing the sampled signal at a predetermined number of bits desired for an analog to digital conversion of the sampled signal by allocating a first portion of bits to digitizing a signal from the first circuit and allocating a second portion of bits to digitizing a signal from the second circuit. The system and method further includes encoding the first and second portion of bits into one monotonic digital word corresponding to a range of the sampled signal.

Claims

1. A method for digitizing data from an imaging system, comprising; sampling a signal from an optical detector with a first sample and hold, S/H, for a first sample time t.sub.s/h2 and with a second S/H for a second sample time t.sub.s/h2 where t.sub.s/h1>t.sub.s/h2, producing a digital counter signal and a ramp signal wherein the counter and the ramp are timed such that the counter counts from 0 to a predetermined number of bits, C.sub.max-x, wherein C.sub.max corresponds to the number of bits desired for an analog to digital conversion of the detector signal, in approximately the time taken for the ramp to ramp from 0 to a predetermined voltage, v.sub.s1, sending the first S/H output to a comparator and comparing it to the ramp signal, starting the counter and the ramp, after a time>t.sub.s/h, latching the count value if the ramp signal meets a threshold defined by the output of the first S/H before the counter counts to C.sub.max-x, sending the second S/H signal to the comparator if the latching does not happen before C.sub.max-x is reached, resetting the ramp to zero and restarting the ramp at a slope sufficient to ramp from 0 to a predetermined voltage v.sub.s2 in the time required for x counts, and latching the count value if the output of the ramp signal meets a threshold defined by the output of the second S/H, wherein the count latched is a digitization of the detector signal.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The invention will be better understood by referring to the following figures.

[0012] FIG. 1 shows an A/D integrator in block diagram form, according to an illustrative embodiment.

[0013] FIGS. 2a and 2b illustrate the operation of an A/D integrator, according to an illustrative embodiment.

[0014] FIGS. 3a and 3b illustrate the operation of an A/D integrator, according to an illustrative embodiment.

[0015] FIG. 4 illustrates a transfer function of an A/D converter, according to an illustrative embodiment.

[0016] FIG. 5 illustrates a histogram based image display function, according to an illustrative embodiment.

[0017] FIG. 6 illustrates a flowchart of a method of digitizing an infrared optical signal, according to an illustrative embodiment.

[0018] FIG. 7 illustrates a flowchart of a method of conditionally combining data from successive frames, according to an illustrative embodiment.

DETAILED DESCRIPTION

[0019] One or more embodiments may provide for mass-produced Infrared cameras which are low cost and have a simple implementation.

[0020] One or more embodiments may produce a thermal image that shows temperature differences in a scene in a visually clear manner.

[0021] One or more embodiments may reduce the component count while increasing the functionality of the readout.

[0022] One or more embodiments may provide an efficient design for signal processing electronics in combination with image display techniques that produces a visually satisfying IR image while simplifying camera system design.

[0023] One or more embodiments may apply slope/latch count analog to digital conversion to FPA design.

[0024] One or more embodiments may avoid design and implementation complexity associated with designs concerned with preserving resolution and linearity across an entire range.

[0025] One or more embodiments may avoid performing operations and producing data in a form not needed or desirable for low cost IR cameras.

[0026] One or more embodiments may utilize a novel approach to slope/latch count analog to digital conversion that achieves dynamic range enhancement (but not with uniform conversion resolution), utilizes a simpler electrical design and/or produces data better suited for the visual image enhancement desirable for a mass-produced IR/thermal imaging camera.

[0027] One or more embodiments may provide dynamic range extension that produces monotonic data in the form of a single digital word per conversion.

[0028] One or more embodiments may provide on-chip conversion of image data from bolometer based infrared focal plane arrays used in IR cameras.

[0029] One or more embodiments may apportion resolution to image display applications, allocating higher A/D resolution to smaller, lower noise signals.

[0030] One or more embodiments may be used in conjunction with a Histogram Based Equalization Method to produce superior thermal image quality with a minimal component count and efficient image data handling.

[0031] One or more embodiments may be used to combine data from successive frames which may originate from both the long and short integration signal paths conditionally.

[0032] Referring to FIG. 1, a digitization system in the form of a novel variation of a slope/latch count A/D converter is shown in block diagram form, according to an illustrative embodiment. A detector signal, corresponding to intensity, v.sub.a, suitably buffered and integrated (not shown) is connected to a first sample and hold element or circuit 1, for a long integration time t.sub.s/h1 and to a second sample and hold element or circuit 2 for a shorter integration time t.sub.s/h2. This arrangement effectively applies a first attenuation (lower gain) on the short sample time signal path. The signal from second sample/hold circuit 2, v.sub.s/h2, will be t.sub.s/h2/t.sub.s/h1 times the first integrator signal v.sub.s/h1. Thus a low and a high gain signal path are established.

[0033] Outputs of first and second sample and hold circuits 1 and 2 are sequentially switched to a comparator 3 using the switches shown, where they are compared to a voltage ramp from a programmable ramp generator circuit 5. The output of the comparator 3 is used to trigger a latch or other storage element 4, which latches a count from a digital counter 6. The counter 6 may count from 0 to C.sub.max, where C.sub.max is the A/D conversion resolution. For example, if the counter counts from 0 to C.sub.max=16383, the A/D conversion is a 14 bit conversion. The latched count is the digitized signal v.sub.d.

[0034] The processing circuit components of FIG. 1 may be implemented using any of a variety of analog and/or digital discrete circuit components (transistors, resistors, capacitors, inductors, diodes, etc.), programmable logic, microprocessors, microcontrollers, application-specific integrated circuits, or other circuit elements. A memory configured to store a computer program may be implemented along with discrete circuit components to carry out one or more of the processes described herein. The processes of one or more of the circuit elements shown in FIG. 1 may be combined with one another or separated out into separate circuits in various alternative embodiments. Each block in FIG. 1 may represent one or a plurality of circuits. For example, ramp generator 5 may comprise one ramp generator used for comparison or a plurality of ramp generators, one each for comparison to the first sampled signal and the second sampled signal (and optionally, additionally sampled signals). A third S/H circuit may be used with an integration time longer or shorter than both of S/H circuits 1 and 2, as well as a fourth, etc., for additional range and/or resolution. Processing circuitry may be added to FIG. 1 to control timing of the switches, latching, inputs and outputs of each of the elements of FIG. 1.

[0035] Referring to FIGS. 2a and b, operation of an A/D converter is shown, according to an illustrative embodiment. V.sub.s/h1 is sent to, connected to, or received by the comparator after a time equal to or greater than t.sub.s/h1. The counter and the ramp signal are started together, substantially simultaneously. Alternatively, the signals may be started with a fixed delay corresponding to a known voltage offset. The ramp generator ramps from zero or another initial voltage to a predetermined voltage v.sub.s1 in approximately the time it takes the counter to count from zero to C.sub.max?x, where x is preferably many times smaller than C.sub.max. V.sub.s1 can be any of a variety of values depending on the system constraints and performance as will be discussed below.

[0036] FIG. 2a illustrates the case where v.sub.s/h1 is less than or equal to the first sampled signal v.sub.s1. The comparator will trigger before C.sub.max?x is reached and the count will be latched as the digitized output v.sub.d. In this case, the first sampled signal is within the voltage v.sub.s1 and the corresponding range of counts C.sub.max?x defined by the counter and the ramp. FIG. 2b illustrates the case where v.sub.s/h1 is greater than v.sub.s1 so the comparator will not trigger before the count reaches C.sub.max?x. In this case, the ramp generator is reprogrammed to ramp from zero to predetermined voltage v.sub.s2 in the time the counter counts from C.sub.max?x to C.sub.max. At approximately the same time as the ramp restarts, the output of the second S/H v.sub.s/h2 is switched to the comparator in place of v.sub.s/h1. v.sub.s/h2 is equal to v.sub.s/h1 times y where y=t.sub.s/h2/t.sub.s/h1. Thus v.sub.s/h2 is lower in amplitude than v.sub.s/h1, i.e. more attenuated, for the same input signal, and therefore the dynamic range of input signals v.sub.a that can be converted is extended. The comparator triggers when the ramp reaches V.sub.s/h2 and the count is latched as the digitized signal v.sub.d. Thus the two signal paths, of differing attenuation (gains) are each converted using two portions of the full count C.sub.max and the result is the conversion is encoded as one monotonic digital word of length C.sub.max.

[0037] In this embodiment, x is chosen to be much smaller than C.sub.max, which results in the higher amplitude detector signals being converted at a lower resolution than the lower amplitude signals. This is suitable for a micro0bolometer based FPA where the higher signal amplitude (hotter) signals do not benefit greatly from high resolution conversion. Also in this embodiment, the first part of the second conversion cycle (FIG. 2b) is basically excluded. Since v.sub.s/h1 is greater than v.sub.s1 for this case, v.sub.s/h2 must be greater than v.sub.s1 times y for any possible latch condition, so no latching takes place until the second steeper slope ramp voltage exceeds (v.sub.s1)y. This exclusion region is shown in grey in FIG. 2b. In other words, in this embodiment the counts in this exclusion zone will not be a resultant conversion v.sub.d.

[0038] Although a variety of values for the various voltage levels, counts and times will produce acceptable results depending on the details of the FPA design and desired performance, some implementations are more desirable than others. A particularly beneficial implementation is illustrated in FIGS. 3a and b. For this case v.sub.s1=v.sub.s2=v.sub.max, a voltage chosen to correspond to the maximum range (full count) desired for the digital conversion, C.sub.max. V.sub.max can be determined in a variety of ways known in the art, but preferably depends on the voltage rails allowable in the circuit design and the noise floor. Typically 1 count (1 LSB) is chosen to represent the noise floor so as not to waste conversion resolution on noise, and the conversion size C.sub.max is chosen to correspond to an expected maximum level of the detector and any buffer electronics. For a particular tested implementation, the inventor chose C.sub.max to be 14 bits (16384) which limits v.sub.max to 1.6 V since the noise floor was ?100 microvolts. Since the swing possible on the detector signal is higher than 1.6 V, the A/D shown achieves the full useful noise floor limited resolution for lower amplitude signals but extends the dynamic range of the conversion at lower resolution for higher amplitude signals. In this system, x was chosen to be 1024, so the ramp slope for the second conversion cycle is ?16 times the first slope, y was chosen to be 4. The actual sample times and count rates depend on desired image frame rate, detector response times and allowable clock speeds for the readout.

[0039] An A/D converter may be implemented on the readout integrated circuit (IC) associated with or coupled to an FPA, which is typically an array of N pixels, each pixel consisting of an IR detector and associated interface electronics. The A/D components include components dedicated on a per pixel basis such as the comparators and latches, and components shared across many or all pixels such as the ramp generator and counter. The A/D converters presented herein are implemented efficiently in terms of power consumption, component count and complexity which is beneficial to the readout design and manufacture for a low cost IR camera. Moreover, the nature of the data and how it is encoded lends itself very well, as will be shown, to producing unexpected results in visual image quality for thermal imaging.

[0040] An FPA typically includes a two dimensional array of N detectors, which can provide a two-dimensional image of a scene. For imaging purposes, image frames, typically data from all or some of the detectors N.sub.f, are produced by the FPA, each successive frame containing data from the array captured in successive time windows. Thus, a frame of data delivered by the FPA will consist of N.sub.f digital words, representing each pixel in the image, of length C.sub.max. These words typically correspond to an intensity of radiation measured by each pixel in the array. The intensity per pixel, for a bolometer IR FPA , corresponds to temperature of the corresponding part of the scene, with lower values corresponding to colder regions and higher values to hotter regions. With the A/D disclosed, N.sub.f monotonic words encoded as described are produced per frame. It may be desirable to display this data on a visual display. The quality of the display and the ability to distinguish intensities visually lead to the desirable result of a user visually able to identify regions of differing temperature in a scene.

[0041] FIG. 4 illustrates a transfer function of an A/D converter, according to an illustrative embodiment. Input signals below a certain value are converted in a first cycle and are thus subject to a higher gain (a first attenuation) and are converted with higher resolution, v.sub.max/(C.sub.max?x). Input signals above a certain level are subject to a lower gain (a second attenuation) and are converted at lower resolution, v.sub.max/x. There is a dead zone or discontinuity in the count as explained above. Thus, one monotonic digital word is generated for the entire extended dynamic range; however the word may not be a linear function of the input. In a preferred embodiment, the digital counter may be a Gray code (reflected binary) counter. This is advantageous because the analog detector signal may be an analog signal that is not synchronized with the digital counter. Thus, the comparator triggering is also un-synchronized, and may trigger in the middle of a count transition. If many bits are changing in a particular count transition, (such as 01111 to 10000) the data may be rippling at the latch point, which can result in conversion error. In Gray code counting, only one bit changes per transition, minimizing the possible transition error.

[0042] If the purpose of the system is to visually display an infrared scene, the transfer function shown in FIG. 4 turns out to be unexpectedly advantageous. For an image display application, the FPA may be a square or rectangular array of N pixels, and the FPA may output successive frames of data from some or all of the pixels, N.sub.f pixels, in successive time windows, creating a video frame rate that is in turn displayed on a monitor or screen (single frames or averaged groups of frames may be used to display still images as well), such as a screen on a mobile computing device, such as a smartphone. Each frame consists of N.sub.f digital words (samples) of C.sub.max length, where the count value corresponds to the intensity for that pixel. For example, if the FPA is a 128?256 pixel array, for a full frame, N.sub.f would be 32768 samples per frame, each sample for a 14 bit conversion would be 14 bits in length. In an embodiment, the number of samples with a given count value (intensity) is plotted vs. counts (intensity), or range of counts, in a histogram. Intensity corresponds to temperature of various scene elements imaged by IR camera with higher count values representing higher temperature. An illustrative histogram is shown in FIG. 5 for a given frame of IR image date, such as would be expected if an image was acquired of a room with a person working on electronics with a soldering iron for instance. Such a scene will have a variety of temperatures. There will be a large number of pixels showing a relatively cool background for elements such as the walls, which is represented by the largest bins in the histogram at area A. There may be some cooler objects, such as objects recently brought into the room represented by the group of lower count values. There may be equipment which will be warmer than the background or a person also warmer than the walls, represented by the two groups of hotter pixels at areas C and D, and there will be a very hot group of pixels representing the soldering iron at area E. There will also be count values with no or very small numbers of pixels, such as the exclusion zone discussed above; and in general scenes imaged in the IR often have gaps as not all temperatures exist in many scenes.

[0043] The digital sample words may be converted into display compatible form to be shown on a screen. The display may have a digital brightness scale, which for an IR image corresponds to temperature, brighter being hotter. However, the brightness scale, for example grey scale, is generally a much shorter digital word than the pixel sample words. For instance the sample word may be 14 bits while a display range such as grey scale is typically 8 bits. So for display purposes, the higher resolution image data may be compressed down to fit the display range. How this is accomplished can effect visual image quality.

[0044] One way to allocate the samples to the display is to perform one of several standard image processing algorithms known as Histogram Equalization Based Methods, using the histogram of FIG. 5. These methods, including Contrast Limited Histogram Equalization (CLHE), Contrast Limited Adaptive Equalization (CLAHE) and many variations, may be used to allocate the display range 0 to B to the pixel intensities, 0 to Cmax. These methods have the affect of balancing the histogram, so that intensities with very few samples are not allocated significant fractions of the display range, while intensities with moderate number of samples are given enhanced fractions of display range, while very high count intensity bins are attenuated in display range, balancing the allocation of display range. Thus, the encoding of the current A/D may be directly compatible with HEBM. Also, HEBM by its nature adjusts for the exclusion zone and lower number of counts allocated to the higher intensity signals inherent in the current A/D.

[0045] Thus, in one embodiment, the highly efficient, low-part-count A/D, because of the nature of how it encodes the output data, combined with an HEBM conversion to display, results in very high visual image quality. The result for the class of uncooled IR cameras can be superior image quality at a low cost implementation with little penalty for the A/D data characteristics. For instance, a scene such as a soldering iron next to a person's face has excellent clarity for all features, which is excellent performance for thermal imaging.

[0046] One or more of the simplified equalization techniques disclosed herein may provide high contrast. They also may automatically account for the difference in gain between the two (or more) conversion ranges by allocating the majority of the brightness bits to the intensity count values corresponding to non-zero signals, thus ensuring that even though the numbers of pixels per count at the high end is smaller, the display bits are not wasted on zero or low data counts.

[0047] Referring now to FIG. 6, a flowchart illustrating a method of digitizing an infrared optical signal will be described, according to an illustrative embodiment. Step 60 is sample Signals corresponding to intensity from the pixels from an array of Optical detectors, each signal sampled with different attenuations. For example, the attenuation can be due to an integration time or sample/hold time such as t.sub.s/h1. Step 61 is for each signal, allocate a first portion of conversion bits to a first attenuated signal and a second portion of bits to a second attenuated signal. The allocation can be preconfigured by the arrangement of circuit elements or per a programmed computing device. The digitization may be done using a ramp and counter as described above, or using other analog-to-digital techniques. Step 62 is encode the first and second conversion bits into single monotonic intensity words. Step 63 is plot the number of intensities vs. intensity to form a histogram, and step 64 is assign the intensity values to image display values using Histogram Equalization Based Methods. Steps 62 through 64 may be accomplished by the arrangement of circuit elements or per a programmed computing device as well. One or more of the steps described in FIG. 6 may be rearranged, omitted or replaced with other processing steps.

[0048] Lower amplitude signals may benefit from integration and averaging techniques to increase the apparent signal to noise. Such techniques may involve combination of successive frames of digitized image data, often before equalization, to form a digital integration, which may then be averaged or otherwise processed before equalized. For the two signal path A/D, combining successive pixels digitized on the different gain signal paths may be problematic.

[0049] For instance, one integration technique may include adding, on a per pixel basis, the pixels from M successive frames, where M is an integer, such as 4 for example, to form an integration frame where each pixel is the sum of the M corresponding pixels. It may be desirable to divide each integrated pixel by M, averaging, for instance to provide better signal to noise. If any of the successive pixels has been digitized on the low gain signal path, that successive pixel data cannot be summed and averaged with high gain data as is clear from the transfer function.

[0050] FIG. 7 illustrates a flow chart of an exemplary embodiment for combining digitized data from successive frames where such data may derive from differing gain digitization. In step 70, data is digitized from M successive frames. In step 71 any successive pixel from low gain conversion is detected. For each successive high gain pixel, the successive pixels are summed in step 72b. When a low gain pixel is detected, the sum if any is discarded, and the value of the first detected low gain pixel is multiplied by M in step 72a. In step 73 an M frame integration is produced where the pixels in the integrated frame are either the results of step 72a or 72b.

[0051] Thus in this illustrative embodiment, the data from successive frames may be combined conditionally depending on whether or not there is any low gain signal path data in successive pixels. For successive pixels, where all of the data is high gain, corresponding to lower amplitude detector signals, the data may be combined directly and lower amplitude signals may benefit most from techniques that increase signal to noise. For successive pixels with any low gain data, corresponding to higher amplitude signals, signal to noise may not be as important, so one data point, the first low gain pixel detected, is used, not integrated but simply multiplied by M to match the encoding of the summed high gain pixels. Detection of the low gain signals may be accomplished in a variety of ways. For instance for a 14 bit conversion, where the high gain conversion is done in counts 0 to 15359, and the low gain conversion is done in counts 15360 to 16383, the top 4 most significant bits are all 1 (or the grey scale equivalent) for any low gain digitization.

[0052] While an infrared optical device is used in the illustrative embodiments described herein, the teachings may be applied to other types of optical devices, such as cameras sensing electromagnetic radiation in the visible light spectra and/or other spectra. References in the claims to a or an element or limitation are meant to be open-ended, allowing a plurality of such elements, meaning the same as at least. References to a first and a second element are meant to broadly include systems or methods having three or more of such elements. The teachings herein are not limited to the specific embodiments disclosed, but may be applied to other alternative embodiments.