Optical frequency imaging

Abstract

Frequency imaging of different areas or object in an image is created by a visible light, infrared or other cameras taking multiple sequential images. The images are recorded and stacked. Pixels that vary in the images yield time varying data on a pixel by pixel basis. The time varying data is processed to extract pixel by pixel signal spectrum or another similar signal metric. Frequency at each pixel is displayed and distinguished, such as by recoloring the pixels based on spectral power rather than intensity contrast.

Claims

1. Apparatus comprising: a camera configured for taking multiple sequential images of a scene, a digital recorder connected to the camera and adapted for recording the multiple sequential images of the scene and adapted for stacking the recorded multiple sequential images in a time sequence, a computer connected to the digital recorder and adapted for obtaining intensity versus time for each pixel on the recorded multiple sequential images of the scene, the computer being configured for transforming a time series of data from pixel points to frequency information at the same pixel points, the computer being adapted to indicate where energy lies in the pixels, the computer being adapted to display an energy spectrum according to frequency content of each pixel and to distinguish features of each pixel according to the frequency content of pixels or frequency narration in the same pixels, wherein the computer is configured to: convert pixel image characteristics of each pixel in the multiple sequential images to a specific color, and reconstruct each of the multiple sequential images to a colored image displaying a plurality of colors, each of the colored images thereby displaying frequency behavior of each of the pixels.

2. The apparatus of claim 1, wherein the camera is a visible light camera.

3. The apparatus of claim 1, wherein the camera is an infrared camera.

4. The apparatus of claim 1, wherein the camera is a video camera.

5. The apparatus of claim 1, wherein the camera is capable of operating at speeds in excess of 1000 frames per second.

6. The apparatus of claim 1, wherein the scene to which the camera is directed is operating machinery.

7. The apparatus of claim 1, wherein the scene to which the camera is directed to make the multiple sequential images is infrastructure.

8. The apparatus of claim 1, wherein the scene to which the camera is directed to make the multiple sequential images is a jet engine.

9. The apparatus of claim 1, wherein the scene to which the camera is directed to make the multiple sequential images is a jet engine exhaust plume.

10. The apparatus of claim 1, wherein the multiple sequential images are taken of one or more distant objects.

Description

BRIEF DESCRIPTION OF THE DRAWING

(1) The accompanying drawing, which are incorporated herein and form a part of the specification, illustrate exemplary embodiments and, together with the description, further serve to enable a person skilled in the pertinent art to make and use these embodiments and others that will be apparent to those skilled in the art.

(2) FIG. 1 shows the new frequency imaging.

DETAILED DESCRIPTION

(3) Frequency Image Reconstruction

(4) As shown in FIG. 1, Frequency Imaging 1 works by using a visible or infrared camera 10 to take images of an object 12, in this case a machine, taking multiple frames 14 of image data. Camera 10 provides 11 frames 14 to a computer for recording and “stacking” 16 them in a time sequence 18. The computer then obtains 20 intensity versus time 22 for each pixel point. The time series of data 22 at each pixel is then transformed 24 from the time domain 22 to frequency domain 26 through Fourier transform 24 or related technique to yield frequency information 28 at that pixel. The spectrum at each pixel is then subdivided 30 into spectral bins which indicates where the energy in the pixel power spectrum lies. This binning structure can then be used to color the pixel in an image display 32 according to the frequency content of the pixel. This scheme allows an image to be reconstructed according to a scene's frequency content and not pixel intensity.

(5) The general frequency image steps are as follows:

(6) Step 1: Video output streams data into a computer.

(7) Step 2: Data from each pixel in a frame is stored in “time slice” groups in computer memory.

(8) Example: for 1000 Hz video, collect 2000 data points over 2 second “time slice” interval and store in array.

(9) Step 3: signals process pixel data in each “time slice”. One can use Fast Fourier Transform, as an example, to convert intensity versus time data per pixel into Frequency Power Spectrum per pixel per time slice.

(10) Step 4: In a context scaling step, each pixel spectrum is assessed to determine where the signal power lies in each pixel spectrum (i.e. if majority of signal power lies between 100-200 Hz or if there is a spike in frequency at 1000 Hz) or if there are characteristic frequency lines that pop up in the time slice of a given pixel. These characteristics are then related to a color or intensity for visualization.
Step 5: an image is reconstructed using the color scales determined in step 4, thereby providing a new “image” that shows where the strongest frequency pixels and areas are.

(11) Examples of potential application areas for Frequency Imaging include:

(12) 1. Medical imaging.

(13) 2. Commercial unmanned aerial vehicle (UAV) detection—air safety, privacy.

(14) 3. Defense UAV detection—vehicle identification.

(15) 4. Maintenance diagnosis for industrial machinery—looking for damaged/failing parts of operating machinery.

(16) 5. Electrical maintenance—looking for electrically induced vibrations from short circuits or failing circuitry.

(17) 6. Biometric identification.

(18) 7. Electronic entertainment—frimage apps.

(19) 8. Aircraft detection and identification.

(20) 9. Satellite and space detection and identification.

(21) 10. Geologic monitoring.

(22) 11. Windmill monitoring for renewable energy.

(23) 12. Rig, site or facility monitoring in the oil and gas or related industries.

(24) 13. Large network activity monitoring in an installation or facility (i.e., military base, large company, etc.).

(25) By noninvasive video visual and infrared imaging of body parts or of whole bodies, storing time sliced groups of frames and pixels and extracting data from individual pixels, differences in intensity versus time and scaling characteristic frequency lines and reconstructing the image frame showing in selected characteristic colors and intensity problem areas requiring interventions or changes.

(26) UAVs may be identified as to type, model and age and individual characteristics using frequency imaging and the steps of the invention.

(27) Aircraft and UAV may have periodic, preflight and inflight evaluations using high speed cameras and producing data streams of exhausts, engines, control surfaces and bodies and parts thereof. The same steps are used for storing data from each pixel in a frame stored in time slice groups. Intensity versus time data for each pixel is converted into frequency power spectrum per pixel time slice. Context scaling accesses each pixel for normal frequencies and converts the pixel image characteristics to a color and intensity for visualizing, and the image is reconstructed using the color scales to show the strongest frequency pixels locations.

(28) Context Auto-Scaling

(29) The display of a Frequency Image has meaning if it is viewed in the context of a specific application. Different viewing criteria will determine what frequency ranges or thresholds would need to be marked to provide value and utility to a user. Examples of this would be imaging rotating machinery to examine for out of tolerance components during operation or typing aircraft from a long distance. Maintaining arbitrary color or intensity scale settings for adjustment is necessary to convert the general tool of frequency imaging into a specific tool for a given application. Embedding this feature into the Frequency Imaging method is key to providing value to a user.

(30) The invention provides a frequency imaging method of visualizing video from visible or infrared cameras to extract new and different information from scenes.

(31) The invention also provides an approach to tailor the video display output to give context to the frequency image for defense, commercial and industrial applications.

(32) While the invention has been described with reference to specific embodiments, modifications and variations of the invention may be constructed without departing from the scope of the invention, which is defined in the following claims.