MULTI-WAVELENGTH LIDAR AND THERMAL IMAGER
20210190962 · 2021-06-24
Inventors
- Shimon Maimon (Rochester, NY, US)
- Itai MAIMON (Rochester, NY, US)
- Ory Amos-Haim MAIMON (Rochester, NY, US)
- Ronit SASON-MAIMON (Rochester, NY, US)
Cpc classification
Y02A90/10
GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
G01S17/42
PHYSICS
G01S17/894
PHYSICS
G01S7/4865
PHYSICS
International classification
G01S17/86
PHYSICS
G01S17/894
PHYSICS
G01S7/481
PHYSICS
G01S7/4865
PHYSICS
Abstract
Several embodiments of an apparatuses and method for sensing objects in a volume of interest are provided, including optionally sensing weather phenomena such as snow, ice, fog and the like by illuminating the volume with a laser pulse which is highly absorbed by the phenomenon and another laser pulse which is highly reflected, and sensing phenomenon presence by the returned pulses. Another aspect involves LIDAR utilizing a plurality of wavelengths. Further aspect include apparatus combining LIDAR and thermal imaging utilizing a single photodetector biased at different polarities to switch between the LIDAR and thermal imaging modes, providing a 3D thermal map of the volume of interest. All features may be combined in a single embodiment of the invention, or various aspects may be combined at will. A dual polarity photodetector for use in such combined LIDAR and thermal imager is also provided. Methods of using various embodiments are also provided.
Claims
1. An apparatus for detecting a weather phenomenon in a volume of interest, the apparatus comprising: a first laser emitter capable of illuminating the volume of interest with laser light pulses having a first wavelength; a second laser emitter capable of illuminating the volume of interest with laser light pulses having a second wavelength; a focal plan photodetector comprising a plurality of pixels each pixel being capable of sensing reflected laser light pulses from the first and second emitters respectively and of receiving light from a respective portion of the volume of interest; a focusing mechanism capable of focusing reflected light from the volume of interest on the photodetector, such that each pixel receives light from a respective portion of the volume of interest; a comparator configured to compare relative amplitude of the reflected light of first and second wavelengths detected by the photodetector; the first wavelength having a higher absorption by the weather phenomenon than the second wavelength, and the second wavelength having a higher reflectance rate by the weather phenomenon than the first wavelength.
2. The apparatus of claim 1, wherein at least one of the plurality of pixels is capable of detecting infrared energy when the photodetector is biased at a first polarity, and of detecting light at least at the first and second wavelengths when the photodetector is biased at a second polarity opposite the first polarity.
3. The apparatus of claim 2, wherein the infrared energy is a black body energy.
4. A method of detecting the existence of a weather phenomenon in an area or volume of interest, the method comprising: illuminating the area or volume of interest by a laser light of a first wavelength; sensing the laser light of first wavelength reflected from the area or volume of interest; illuminating the area or volume of interest by a laser light pulse of a second wavelength; sensing the laser light of second wavelength reflected from the area or volume of interest; the first wavelength having a higher absorption by the weather phenomenon than the second wavelength, and the second wavelength having a higher reflectance rate by the weather phenomenon than the first wavelength; and indicating the presence of a weather phenomenon upon sensing a difference between the sensed level of the first wavelength is lower by at least a predetermined amount than the sensed level of the second wavelength.
5. The method of claim 4 wherein the illuminating by the first and second laser lights is performed by sequentially illuminating the area or volume of interest by a first pulse of laser light of the first wavelength followed by illuminating the area or volume of interest by a second laser pulse of the second wavelength.
6. The method of claim 5 further comprising measuring a time difference between a pulse initiation time where time the first or the second pulse is initiated and a detection time of the respective pulse where the reflected respective pulse is sensed.
7. The method of claim 4 wherein the step of sensing is performed by a focal plan photodetector comprising a plurality of pixels each pixel being capable of sensing reflected laser light pulses from the first and second emitters respectively, and of receiving light from a respective portion of the volume of interest
8. A LIDAR (Laser Imaging Detection and Ranging) device comprising: a first laser emitter capable of illuminating the volume of interest with laser light pulses having a first wavelength; a focal plan photodetector comprising a plurality of pixels each pixel being capable of sensing laser light reflected from the volume of interest and of receiving light from a respective portion of the volume of interest; a focusing mechanism capable of focusing reflected light from the volume of interest onto the photodetector, such that each pixel receives light from a respective portion of the volume of interest; at least one timer capable of measuring time difference between an initiating of a laser pulse and a time when the laser pulse reflected from the volume of interest is sensed by at least one pixel; a second laser emitter capable of illuminating the volume of interest with laser light pulses having a second wavelength, and, at least one pixel of the photodetector is capable sensing laser pulses of the second wavelength.
Description
SHORT DESCRIPTION OF DRAWINGS
[0044] The summary above, and the following detailed description will be better understood in view of the enclosed drawings which depict details of preferred embodiments. It should however be noted that the invention is not limited to the precise arrangement shown in the drawings and that the drawings are provided merely as examples.
[0045]
[0046]
[0047]
[0048]
[0049]
[0050]
[0051]
DETAILED DESCRIPTION
[0052]
[0053] Certain embodiments shall now be described in detail in order to exemplify to the skilled in the art some of the advantages provided by the invention, and assist in its understanding and utilization.
[0054]
[0055] An optional scanning mechanism 220 such as rotating mirrors and/or prisms directs the laser pulses from the emitters 210, 215 to a volume of interest, limited by a field of view of interest. Alternatively, the scanning mechanism is obviated by utilizing a beam expander such as a diverging lens, to illuminate the whole volume of interest in a single light pulse, colloquially referred to as a “flash”. Utilizing a flash allows illuminating a complete scene with a single pulse. Preferably the pulse is of high intensity, such as 1 micro-joule, 1 mili-joule or even 1 joule by way of example. Judicious selection of the wavelength allow utilization of eye safe wavelength and the fact that the energy is distributed over a larger area further reduces risks to an eye within the volume of interest. A beam expander may comprise at least one diverging lens to spread the laser light to fit the Field of View. Notably the volume of interest may be of any desired size.
[0056] Laser pulses may be of wide variety of durations such as a pulse shorter than 1 ns or even shorter than 0.5 ns, to pulses shorter than 10 ns, 100 ns, 500 ns, and 1 μs, and even to pulses longer than 1 μs, such as 2 μs, 5 μs, and more. In certain embodiments the laser pulses may be AC modulated.
[0057] Generally, transmitter logic 206 controls the firing rate, pulse length, intensity, and other parameters of the emitted laser pulses. The emitters are operated at different times, so that the receiver control circuitry 255 may differentiate the source of any reflected laser light. While the figure is schematic and not to scale in general, the squiggle lines marked SD indicates a scale discontinuity, as commonly the distance to the target is far greater than the distance between the emitters and detectors.
[0058] When the laser pulse of the highly reflected emitter hits a target, such as 225, at least a portion of the transmitted energy is returned to the receiver 245 and after being focused onto photodetector 250 by focus element 240, is detected by photodetector 250, the output thereof is read by receiver control circuitry 255.
[0059] In certain embodiments the LIDAR system merely detects the fact that a reflected pulse exceeding a certain threshold occurred, and measures the time between the pulse transmission and the detection of the reflected pulse. In other embodiments the LIDAR system detects the amplitude of the reflected pulse returned. In flash type LIDAR the photodetector comprises a plurality of pixels, and the system is capable of detecting the time difference between the time a laser pulse is transmitted and the time a reflected laser pulse is detected for each of the plurality of pixels.
[0060] Multiple wavelength LIDAR is advantageous for numerous applications, including by way of example allowing operations with high intensity at wavelengths which may be harmful to humans and animals when no risk of such operation exist, while switching to different wavelength when such risk exists.
[0061] Yet another advantage of multiple wavelengths LIDAR is directed at detection of weather phenomena such as snow, cloud, fog, rain, ice, and the like. In such embodiments the wavelength emitted by the first emitter 210 is selected to be highly absorbed by the phenomenon to be detected, while the wavelength of the second emitter 215 is selected to be highly reflected by the weather phenomenon. Clearly the terms highly absorbed and highly reflected should be construed as opposing relative to each other—a highly absorbed wavelength implies lower reflected wavelength and vice versa, thus the selection criteria may be differently stated as the first wavelength being highly absorbed than the weather phenomenon than the second wavelength, and conversely the second wavelength being highly reflected, more than the first wavelength. It is noted that the terms highly absorbed and highly reflected are not a measure of degree but a relative measure of the relative absorption and reflectivity respectively of any two selected frequencies, by the environment in the volume of interest. By way of a non-limiting example the highly absorbed wavelength may be about 1550 nm or about 1800 nm while the highly reflected wavelength may be about 900 nm or about 1800 nm. Generally transmitter logic 205 controls the firing rate, pulse length, intensity, and other parameters of the emitted laser pulses. The emitters are operated at different times, so that the receiver control circuitry 255 may differentiate the source of any reflected laser light.
[0062] If both emitters transmit towards the target but only the highly reflected laser light wavelength is detected, it is indicative that the direction from which the highly reflected pulse is detected contains the weather phenomenon, while reflectance of light from both emitters would indicate lack of the phenomenon. Range to the phenomenon may be estimated by the time of flight of the reflected laser pulse. It is important to note that the results may be binary or more likely differential, as some energy of the reflected wavelength shall be absorbed while some energy of the absorbed wavelength may be reflected back and detected by the photodetector. Therefore the determination of the existence or non-existence of the weather phenomenon in the field of view of interest is related to the differences between the detected first and second reflected wavelengths.
[0063] In certain embodiments the principle of detection of the weather phenomenon is utilized at a small area. By way of example it may be desired to detect the presence of ice on an aircraft wings, control surfaces, and/or fuselage. To that end the scanning mechanism 220 or a beam expander is optional, and in some embodiments even the focus mechanism 240 is optional. The detection of ice may be carried out on a single spot. Alternatively the scanning mechanism 220 may form a line scan or be directed to scan the entire volume of interest as done in a LIDAR described above. Similarly flash type detection mechanism may be utilized.
[0064] Focus mechanism 240 may take any desired form, from a ‘pinhole aperture’ to lenses, fiber bundle, and the like. The focus mechanism directs light and/or thermal energy from the volume of interest such that each pixel receives such light and/or thermal energy from a differing portion of the volume of interest. While the portions that each pixel receives may overlap in some areas, different pixels receive, as a whole, energy from a different portion from the adjacent pixels.
[0065] The operation of the LIDAR and/or the weather phenomena device is controlled by system controller 260. System controller 260 controls the transmitter 205 and the receiver 245, as well as communications with systems such as display, vehicle control systems, general communications, and the like. It is noted that the division between the transmitter 206 and receiver 245 controls may be integrated within the system controller 260 at various levels of integration from having all functionality within the system controller, to merely being housed in communication therewith. Furthermore system controller 260 may be located remotely to the LIDAR device. In some embodiments system controller 260 also creates the two or three dimensional representation of the volume of interest, colloquially known as a ‘map’ or an ‘image’ of the LIDAR.
[0066] In flash type LIDAR the Read-Out Circuit (ROC) of each pixel in the photodetector is capable of either measuring, or storing a charge proportional to, the time between the transmitted laser pulse and the detection of its reflected representation.
[0067]
[0068] By way of example, highly absorbed wavelength may include wave of about 1550 nm and 2000 nm, while highly reflected wavelength may include waves of about 905 nm and 1800 nm. While the above wavelengths are indicated as a single wavelength, it is noted that those wavelengths are approximate and encompass wavelengths longer or shorter than indicated, which provide similar functionality. Similarly, other wavelengths may be selected, as long as the selection allows operation of the phenomenon detection according to the principles disclosed herein. The selection of wavelength is primarily a matter of technical choice as long as the absorbance of one wavelength as compared to the reflectance of the other provide sufficient difference to discern the existence of the weather phenomenon, or lack thereof. Furthermore, the system controller may alternate between scanning an entire field of view by one emitter and only then scan the field of view by the other emitter causing an interleaved frame scan between the two wavelengths. Alternatively, any desired scanning order may be utilized, such as alternating between the emitters for each direction during the scan of a single orientation, a single line, half a frame or a full frame, by way of example.
[0069] The utilization of LIDAR to provide three dimensional representation of the field of view of interest may be further enhanced by thermal imaging, as identifying certain types of objects utilizing thermal data is relatively simple. By way of example, since living creatures exert dark thermal energy, they are relatively easy to discern. As different materials absorb heat differently, the thermal image allows differentiating between different materials such as asphalt and sand, by way of example. However thermal images are generally two dimensional and lack depth information. Combining the information derived from LIDAR 3D representation and thermal imaging information enabled formation of a three dimensional (3D) thermal map. Such 3D thermal map may utilized for various purposes such as creating a combined actual image for display, analysis by software for collecting information about the field of view, selection of a path, object avoidance, and the like.
[0070] Combining such LIDAR and thermal images is a complex process when the information is derived by separate detectors, due to parallax. However if the detection is done by a single sensor the combination of thermal and LIDAR data becomes far simpler—the same pixel will generally generate the thermal information for the object detected by the LIDAR information. This may be achieved by an aspect of the invention that utilizes a single detector to detect LIDAR and thermal image data. As previously stated, the fusion of the LIDAR and the Thermal information forms an apparatus which is very useful for applications within moving vehicles, and more so within an autonomous vehicle. The thermal image information facilitates discerning animate from inanimate objects while the LIDAR information provides an exact distance from the object. Therefore, the data transmitted from such apparatus offers significant advantageous to autonomous vehicles or any autonomous device as it allows discerning humans from surroundings as well as discerning the difference between objects of different temperature or emissivity. The ability to detect snow and hazardous conditions makes such combined LIDAR and thermal apparatus even more useful.
[0071]
[0072] When the laser pulse of the emitter 410 hits a target, such as 425, at least a portion of the transmitted laser energy is returned to the receiver 445 and after being focused onto a photodetector by focus element 240.
[0073] The photodetector 450 in this embodiment, while occupying a combined volume, may be considered as two superimposed photodetectors 450A and 450B respectively. Receiver control 455 controls the operation of the receiver 445 and includes inter alia a readout circuitry. When the photodetector 450 is biased at a first polarity the detector is operative to detect at least the wavelength of the laser pulse emitted from emitter 410, and when biased at a second polarity, the detector is operative to detect wavelengths at mid IR range, at a wavelength for example between 4400 nm-5200 nm, a range which for brevity shall be equivalently referred to as HOT-MW and/or MWIR.
[0074] The use of a single detector detection of detect both the LIDAR wavelength and the thermal HOT-MW wavelength provides yet another advantage, according to an aspect of the invention, namely the ability to receive thermal imaging and LIDAR information with a small temporal distance therebetween, thus facilitating formation of 3D thermal with increased accuracy. Further, in such embodiment the LIDAR may be a single wavelength or utilize a plurality of wavelengths, all of which are detected by the same photodetector together with black body energy in the HOT-MW range.
[0075] Combining the thermal and LIDAR information involve two potential sources of parallax. The first is the common optical parallax stemming from one sensor being disposed at a different physical location than a second sensor. The second ‘parallax’ source shall be termed a temporal parallax, does not strictly stemming from distance between the sensors, and thus strays somewhat from the strict definition of parallax, stems from motion over a time difference between the time an image is obtained from the first sensor and an image obtained by the second sensor. The resultant displacement of similar objects is somewhat similar to the optical parallax, and thus this phenomenon shall be termed a ‘temporal parallax’.
[0076] The use of a single photodetector in certain aspects of the invention overcomes the optical parallax for most practical purposes. However reducing the temporal parallax requires minimizing the time difference between acquiring the two images—namely the LIDAR and thermal images.
[0077] Information obtained by photodetectors is transformed by circuitry colloquially known as a Digital or non-Digital Read-Out circuit (DROIC, ROIC). In common LIDAR and thermal imaging systems the readout circuitry of the detector stores a single frame of reflected laser's pulses, and then output the frame data to other electronic circuitry for further processing and/or storage. For brevity the other electronic circuitry shall be termed ‘processing circuitry’ which extends to a local as well as remote circuitry, and which may perform other functions. The time difference between frames in the above described method may be 200 microseconds or larger, since communicating each frame requires time for the readout to communicate with the processing circuitry.
[0078] An aspect of the invention involves the read-out circuitry storing a plurality of individual frames, each representing differing wavelength, prior to transferring the stored data to the processing circuitry. Thus if the apparatus utilizes a thermal data frame and a LIDAR data frame, a plurality of LIDAR data frames each from a different emitters, or a combination multi-wavelength LIDAR and thermal frames. Since no communication is required between the DROIC circuitry and the processing circuitry is eliminated between capture of subsequent frames of a different wavelength the temporal parallax is greatly reduced, and in certain applications may be completely ignored. By way of example the time difference between frame capture may be reduced to 5 microseconds and bellow. While transferring the data between the RDOIC and the processing circuit may stay the same, the effects of motion of the sensing apparatus and/or objects in the field of view of interest are greatly diminished.
[0079]
[0080] The system than begins the capture of the second frame, where the system controller activates the first laser emitter, and sends a sync signal to the controller 510. Controller 510 in turn commands the switching unit 515 to bias the photodetector pixel 505 in a second polarity, at which it can detect the laser wavelengths. Simultaneously, the switching unit 515 triggers a timer 535 which begins charging a timing capacitor 540 at a known rate. The detected signal is again integrated by the intensity integration capacitor 520, and charge is collected therein. When the charge reaches a predetermined or selectable level, a trigger 560 is activated and indicates to the timer that a reflected pulse has been detected. The timer evaluates the time since being triggered, and stores this timing information in a second storage unit 545. The timing capacitor and the intensity integration capacitors are again discharged to ready the system for the third frame. The third frame operation is identical is identical to the second frame except for storing the timing information in a third storage unit 555.
[0081] In certain embodiments the PROC also detects and stores the intensity of the reflected signal in the second and third frames. In such embodiments the intensity integration capacitor 520 is read at the end of an integration period, and the intensity information is stored at the respective storage unit. Notably the integration period may vary between frames.
[0082] After obtaining information for the three frames the controller 510 may transfer information regarding the three frames to a processor for creating a 3D thermal map and the like.
[0083] Another aspect of the present invention involves a single photodetector capable of detecting wavelengths in the thermal and LIDAR ranges, or more precisely of a focal plane array photodetector comprising a plurality of pixels capable of such detection. It is noted that the photodetector disclosed herein is provided as an example, however the aspects of the invention disclosed herein may also be applied to other types of photodetectors.
[0084]
[0085] The detector is grown on a substrate 650 of GaSb, onto which a first absorber 630 is deposited the first absorber 630 is directed to detecting the laser LIDAR pulses, with a wavelength sensitivity cutoff of 3.5 μm, which can absorb both 905 nm and 1.55 μm laser pulses as well as other wavelengths which may be selected for the LIDAR laser emitters. The first photoabsorber 630 is deposited onto the substrate 650 beginning with depositing InAs/InAsSb sls-II for a resulting absorber initial layer having an initial layer of Eg=0.35 eV at a temperature of 150K. The deposition continues with linear grading over 0.5 μm to Eg=0.3 eV. The linear grading is done by changing the period composition of the InAs/InAsSb sls-II. The linear grading of the LIDAR absorber is made to increase the response time of the detector, making it faster. To understand this, the following equations show the diffusion time for holes.
v.sub.drift=μ*E
[0086] Assuming a 0.5 μm absorber, a grading of 0.05 eV, and a hole mobility of 200 cm2/Vs at 150K, the drift velocity due to grading is 2×10{circumflex over ( )}5 cm/s, which results in a cross time of the LIDAR absorber of 0.25 ns resulting in a very fast detector.
[0087] A barrier 610 comprising AlAsSb is then deposited on top of the first absorber 630. The second absorber 620 is then deposited on top of the barrier. The second absorber is not contiguous, and comprises an individual absorber for each pixel. The separation between the second detectors 620 of each individual pixel detectors may be achieved by selective deposition or by subsequent selective etching.
[0088] Notably, the present photodetector utilizes some technologies disclosed in U.S. Pat. No. 7,687,871 to one of the present inventors, and which is incorporated herein by reference. U.S. Pat. No. 7,687,871 describes inter alia a photodetector with reduced dark current utilizing a barrier which eliminates the requirement for etching of the barrier 610 and the bottom absorber 630, thus resulting in the barrier passivating the bottom 630 and top 620 absorbers.
[0089] The second absorber 620 is utilized for sensing thermal energy (MWIR) and is constructed by depositing InAs/InAsSb (3 nm/2 nm with 25% Sb concentration, for example) on top of the barrier, with a thickness in the order of 2 μm. The Absorber 620 shall have a cutoff wavelength of about 5.2 μm, and is well suited for the Hot MW, MWIR range.
[0090] A metal contact 640 is then deposited on top of the second absorber 620. Optionally a small amount of indium, known colloquially as an ‘indium bump’ 660 is deposited on top of the metal layer to ease connection to the ROIC, such as by a ‘flip-chip’ connection method.
[0091] Optionally a process of substrate 650 removal or thinning is utilized to minimize blockage of light which may prevent light from penetrating into the sls-II, in a bottom-illumination system.
[0092] The ROIC can then selectively activate the first and second photoabsorber. Activation of the top absorber 620 is done by applying a negative charge to the contact 640 and a positive charge to the bottom absorber 630 resulting in negatively biasing the top absorber, and causing sensing of LIDAR laser wavelengths. Conversely, connecting a positive charge to the contact 640 and a negative charge to the bottom absorber 630 would result in positively biasing the top absorber, and causing thermal MWIR sensing. Optionally, peripheral indium bumps are utilized for achieving the electrical connection to the bottom absorber 630.
[0093]
[0094] Unless otherwise specified, relational terms used in these specifications should be construed to include certain tolerances that the skilled in the art would recognize as providing equivalent functionality. By way of example the term perpendicular is not necessarily limited to 90.0°, but also to any slight variation thereof that the skilled in the art would recognize as providing equivalent functionality for the purposes described for the relevant member or element. Terms such as “about” and “substantially” in the context of configuration relate generally to disposition, location, or configuration that is either exact or sufficiently close to the location, disposition, or configuration of the relevant element to preserve operability of the element within the invention which does not materially modifies the invention. Similarly, unless specifically specified or clear from its context, numerical values should be construed to include certain tolerances that the skilled in the art would recognize as having negligible importance as it does not materially change the operability of the invention.
[0095] In these specifications reference is often made to the accompanying drawings which form a part of the disclosure, and in which are shown by way of illustration and not of limitation, exemplary implementations and embodiments. Further, it should be noted that while the description provides various exemplary embodiments, as described below and as illustrated in the drawings, this disclosure is not limited to the implementations described and illustrated herein, but can extend to other embodiments as would be known or as would become known to those skilled in the art. Reference in the specification to “one embodiment”, “this embodiment”, “these embodiments”, “several embodiments”, “selected embodiments” or “some embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiment(s) may be included in one or more implementations, and the appearances of these phrases in various places in the specification are not necessarily all referring to the same embodiment(s). Additionally, in the description, numerous specific details are set forth in order to provide a thorough disclosure, guidance and/or to facilitate understanding of the invention or features thereof. However, it will be apparent to one of ordinary skill in the art that these specific details may not all be needed in each implementation. In certain embodiments, well-known structures, materials, circuits, processes and interfaces have not been described in detail, and/or may be illustrated schematically or in block diagram form, so as to not unnecessarily obscure the disclosure.
[0096] For clarity the directional terms such as ‘up’, ‘down’, ‘left’, ‘right’, and descriptive terms such as ‘upper’ and ‘lower’, ‘above’, ‘below’, ‘sideways’, ‘inward’, ‘outward’, and the like, are applied according to their ordinary and customary meaning, to describe relative disposition, locations, and orientations of various components. When relating to the drawings, such directional and descriptive terms and words relate to the drawings to which reference is made. Notably, the relative positions are descriptive and relative to the above described orientation and modifying the orientation would not change the disclosed relative structure.
[0097] It will be appreciated that the invention is not limited to what has been described hereinabove merely by way of example. While there have been described what are at present considered to be the preferred embodiments of this invention, it will be obvious to those skilled in the art that various other embodiments, changes, and modifications may be made therein without departing from the spirit or scope of this invention and that it is, therefore, aimed to cover all such changes and modifications as fall within the true spirit and scope of the invention, for which letters patent is applied.