TIME OF FLIGHT SENSOR RECORDED WITH COMPENSATION PARAMETERS
20230236299 · 2023-07-27
Inventors
- TSO-SHENG TSAI (Hsin-Chu County, TW)
- YUEH-LIN CHUNG (Hsin-Chu County, TW)
- SHIN-LIN WANG (Hsin-Chu County, TW)
Cpc classification
G01S7/4915
PHYSICS
G01S17/36
PHYSICS
International classification
G01S7/481
PHYSICS
Abstract
There is provided a time of flight sensor including a light source, a first pixel, a second pixel and a processor. The first pixel generates a first output signal without receiving reflected light from an external object illuminated by the light source. The second pixel generates a second output signal by receiving the reflected light from the external object illuminated by the light source. The processor calculates deviation compensation and deviation correction associated with temperature variation according to the first output signal to accordingly calibrate a distance calculated according to the second output signal.
Claims
1. A time-of-flight (TOF) sensor, comprising: a light source, arranged in a first accommodation space, and configured to illuminate light according to a light source driving signal; a light sensor, comprising a first pixel, arranged in the first accommodation space, and a second pixel, arranged in a second accommodation space different from the first accommodation space, and configured to generate output signals according to a sampling signal; a memory, configured to previously record a detection compensation and a detection correction that are obtained according to a second reference phase-distance relationship associated with the second pixel at a reference temperature; and a processor, configured to calibrate, at an operating temperature, a current distance using the detection compensation and the detection correction.
2. The TOF sensor as claimed in claim 1, further comprising an encapsulation having the first accommodation space and the second accommodation space separated by an isolation wall, wherein the first pixel is configured to receive reflected light from a top cover of the encapsulation illuminated by the light source, and the second pixel is configured not to directly receive emission light from the light source.
3. The TOF sensor as claimed in claim 1, wherein the second reference phase-distance relationship is obtained at the reference temperature according to a first detection output signal generated by the second pixel when the sampling signal and the light source driving signal has a first time delay therebetween, and a second detection output signal generated by the second pixel when the sampling signal and the light source driving signal has a second time delay therebetween.
4. The TOF sensor as claimed in claim 3, wherein the first time delay is zero, and the second time delay is larger than zero.
5. The TOF sensor as claimed in claim 3, wherein the second time delay is larger than the first time delay.
6. The TOF sensor as claimed in claim 1, wherein the current distance is obtained according to an operation detected signal of the second pixel at the operating temperature.
7. The TOF sensor as claimed in claim 1, wherein the second reference phase-distance relationship is not previously recorded in the memory.
8. A time-of-flight (TOF) sensor, comprising: a light source, arranged in the first accommodation space, and configured to illuminate light according to a light source driving signal; a light sensor, comprising a first pixel, arranged in the first accommodation space, and a second pixel, arranged in a second accommodation space different from the first accommodation space, and configured to generate output signals according to a sampling signal; a memory, configured to previously record a first reference phase-distance relationship associated with the first pixel at a reference temperature; and a processor, configured to, at an operating temperature, calculate an operation phase-distance relationship associated with the first pixel, calculate a temperature compensation and a temperature correction according to the first reference phase-distance relationship and the operation phase-distance relationship, and calibrate a current phase using the temperature compensation and the temperature correction.
9. The TOF sensor as claimed in claim 8, further comprising an encapsulation having the first accommodation space and the second accommodation space separated by an isolation wall, wherein the first pixel is configured to receive reflected light from a top cover of the encapsulation illuminated by the light source, and the second pixel is configured not to directly receive emission light from the light source.
10. The TOF sensor as claimed in claim 8, wherein the first reference phase-distance relationship is obtained at the reference temperature according to a first reference output signal generated by the first pixel when the sampling signal and the light source driving signal has a first time delay therebetween, and a second reference output signal generated by the first pixel when the sampling signal and the light source driving signal has a second time delay therebetween.
11. The TOF sensor as claimed in claim 10, wherein the first time delay is zero, and the second time delay is larger than zero.
12. The TOF sensor as claimed in claim 10, wherein the second time delay is larger than the first time delay.
13. The TOF sensor as claimed in claim 8, wherein the operation phase-distance relationship is obtained at the operating temperature according to a first operation output signal generated by the first pixel when the sampling signal and the light source driving signal has a first time delay therebetween, and a second operation output signal generated by the first pixel when the sampling signal and the light source driving signal has a second time delay therebetween.
14. A time-of-flight (TOF) sensor, comprising: an encapsulation, comprising a first accommodation space and a second accommodation space, wherein the first accommodation space is connected to a first opening and covered by a top cover surrounding the first opening, and the second accommodation space is connected to a second opening; a light source, arranged in the first accommodation space; a first pixel, arranged in the first accommodation space, and configured to receive reflected light projected by the light source to the top cover; and a second pixel, arranged in the second accommodation space, and configured to receive reflected light formed by emission light projected by the light source to penetrate the first opening of the first accommodation space, reflected by an external object, and then penetrating the second opening of the second accommodation space.
15. The TOF sensor as claimed in claim 14, further comprising an isolation wall extending downward from the top cover to separate the first accommodation space and the second accommodation space.
16. The TOF sensor as claimed in claim 14, further comprising a time control circuit configured to generate a light source driving signal, delayed or not delayed, to the light source, generate a sampling signal to the light sensor to cause the first pixel and the second pixel to respectively generate output signals according to the sampling signal, and control a time delay between the sampling signal and the light source driving signal.
17. The TOF sensor as claimed in claim 16, further comprising a memory configured to previously record a first reference phase-distance relationship associated with the first pixel at a reference temperature, and a second reference phase-distance relationship associated with the second pixel at the reference temperature.
18. The TOF sensor as claimed in claim 17, wherein the first reference phase-distance relationship is obtained at the reference temperature according to a first reference output signal generated by the first pixel when the sampling signal and the light source driving signal has a first time delay therebetween, and a second reference output signal generated by the first pixel when the sampling signal and the light source driving signal has a second time delay therebetween.
19. The TOF sensor as claimed in claim 17, wherein the second reference phase-distance relationship is obtained at the reference temperature according to a first detection output signal generated by the second pixel when the sampling signal and the light source driving signal has a first time delay therebetween, and a second detection output signal generated by the second pixel when the sampling signal and the light source driving signal has a second time delay therebetween.
20. The TOF sensor as claimed in claim 17, wherein the memory is further configured to record an operation phase-distance relationship associated with the first pixel at an operation temperature.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Other objects, advantages, and novel features of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
[0022]
[0023]
[0024]
[0025]
[0026]
DETAILED DESCRIPTION OF THE EMBODIMENT
[0027] It should be noted that, wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
[0028] The time of flight (TOF) sensor of the present disclosure is to adopt an additional reference pixel that is used to previously record a temperature compensation and a temperature correction in the memory. The temperature compensation and the temperature correction are used to calibrate a phase and a distance obtained in actual operation. There are two stages before the actual operation. The first stage is a setting stage before shipment in which a reference phase-distance relationship associated with the reference pixel is recorded under a reference temperature. The second stage is a booting stage before each actual operation in which an operation phase-distance relationship associated with the reference pixel is recorded under an operating temperature. The temperature compensation and the temperature correction are calculated according to the reference phase-distance relationship and the operation phase-distance relationship for calibrating a temperature variation between the reference temperature and the operating temperature. In this way, the deviation caused by the temperature variation is effectively eliminated.
[0029] Referring to
[0030] The light source 31 emits light of an identifiable spectrum to illuminate an object outside the encapsulation 39. The light source 31 is a coherent light source or a non-coherent light source, e.g., a light emitting diode or a laser diode.
[0031] The light sensor 33 includes, e.g., a CMOS sensor which has a first pixel 331 and a second pixel 332, wherein the first pixel 331 and the second pixel 332 respectively include at least one photodiode or at least one single photon avalanche diode (SPAD) used to detect optical energy and output electrical signals. It should be mentioned that although
[0032] The encapsulation 39 has a first accommodation space 391 and a second accommodation space 393. The first accommodation space 391 is connected to a first opening and covered by a top cover surrounding the first opening, and is used to accommodate the first pixel 331 and the light source 31, wherein the first pixel 331 receives reflected light from the top cover (e.g., located upon the first pixel 331) of the encapsulation 39 illuminated by the light source 31. As the first pixel 331 and the light source 31 are both disposed in the first accommodation space 391 and close to each other, a time of flight of emission light from being emitted from the light source 331 till propagating to the first pixel 331 is considered substantially identical to zero. The second accommodation space 393 is connected to a second opening, and is used to accommodate the second pixel 332.
[0033] More specifically, a part of emission light from the light source 31 is reflected inside the first accommodation space 391 to be received by the first pixel 331, and another part of emission light is reflected, after penetrating the first opening of the first accommodation space 391, by an external object outside the encapsulation 39 and then received by the second pixel 332 after entering the second opening of the second accommodation space 393.
[0034] The encapsulation 39 preferably has an isolation wall 395, e.g., extending downward from the top cover to a surface of the base layer as shown in
[0035] In some aspects, the first opening of the first accommodation space 391 is further arranged with a filter to block light outside the emission spectrum of the light source 31 to reduce the interference from ambient light to the light detection of the first pixel 331; and the second opening of the second accommodation space 393 is further arranged with a filter to block light outside the emission spectrum of the light source 31 to reduce the interference from ambient light to the light detection of the second pixel 332.
[0036] Please referring to
[0037] The time control circuit further generates a sampling signal S.sub.d1 to the light sensor 33. The sampling controller 47 of the light sensor 33 reads charges (e.g., by a correlated double sampling, but not limited thereto) in the first pixel 331 and the second pixel 332 according to the sampling signal S.sub.d1 to generate output signals S.sub.O1 and S.sub.O2, respectively. The time control circuit further controls a time delay between the sampling signal S.sub.d1 and the light source driving signal S.sub.d2_d and S.sub.d2_nd. That is, the delay in the present disclosure is referred to whether the light source driving signal has a time delay with respect to the sampling signal. The processor 49 calculates the compensation, the correction, the phase and the distance according to the output signals S.sub.O1 and S.sub.O2. An example will be illustrated hereinafter.
[0038] In the present disclosure, the processor 49 is selected from an application specific integrated circuit (ASIC) and a digital signal processor (DSP) that is arranged, for example, in the light sensor 33 to perform the calculation using software, firmware and/or hardware.
[0039] In one aspect, the time control circuit includes, for example, a time controller 41, a first delay circuit 431 and a second delay circuit 433. The time controller 41 generates a timing signal to the first delay circuit 431 and the second delay circuit 433. For example referring to
[0040] In one aspect, the sampling controller 47 or other circuit of the light sensor 33 further generates an inverse sampling signal S.sub.d1_inv (e.g., referring to
[0041] It is appreciated that although the timing signal S.sub.syn, the sampling signal S.sub.d1 as well as the light source driving signal S.sub.d2_nd and S.sub.d2_d are illustrated by square waves in the drawings, the present disclosure is not limited thereto. In other aspects, the timing signal S.sub.syn, the sampling signal S.sub.d1 as well as the light source driving signal S.sub.d2_nd and S.sub.d2_d respectively have other different waveforms according to different applications.
[0042] In the present disclosure, the TOF sensor 300 preferably includes a memory (e.g., volatile memory and/or non-volatile memory) used to previously (e.g., a setting stage before shipment) record and store (1) a first reference phase-distance relationship associated with the first pixel 311 at a reference temperature; and (2) a detection compensation and a detection correction that are obtained according to a second reference phase-distance relationship associated with the second pixel 332 at the reference temperature. The memory further records and stores before actual operation (e.g., a booting stage) (3) a temperature compensation and a temperature correction calculated according to an operation phase-distance relationship (obtained at an operating temperature) associated with the first pixel 331 and the first reference phase-distance relationship. These recorded parameters (1)-(3) are used to compensate and calibrate a detected phase and a detected distance during actual operation. An example will be given below for illustration purposes.
[0043] Please referring to
[0044] At the reference temperature, when the sampling signal S.sub.d1 and the light source driving signal S.sub.d2_nd and S.sub.d2_d have the first time delay T.sub.d0, the first pixel 331 generates a first reference output signal S.sub.O1_nd as shown in
[0045] For example, when the first pixel 331 is a single pixel, the first reference output signal S.sub.O1_nd and the second reference output signal S.sub.O1_d are respectively an output signal of the single pixel; whereas, when the first pixel 331 includes multiple pixels, the first reference output signal S.sub.O1_nd and the second reference output signal S.sub.O1_nd are respectively a summation or an average of output signals of the multiple pixels, wherein the summation and the average are implemented by the circuit of the first pixel 331.
[0046] The processor 49 of the light sensor 33 or an external processor (e.g., an external computer may be used before shipment) obtains a first reference phase-distance relationship, e.g., a line L.sub.Rr shown in
[0047] In one aspect, the line L.sub.Rr is determined by using two reference points RP.sub.r1 and RP.sub.r2. For example, the processor 49 or the external processor obtains a first reference phase according to the first reference output signal S.sub.O1_nd, and obtains a first distance according to the first time delay T.sub.d0 so as to obtain a first reference point RP.sub.r1 at a phase-distance plane as shown in
[0048] Similarly, the processor 49 or the external processor obtains a second reference phase (e.g., using an equation: area of B2/(area of A2+area of B2)) according to the second reference output signal S.sub.O1_d, and obtains a second distance (e.g., using an equation: velocity of light×T.sub.delay/2) according to the second time delay T.sub.delay so as to obtain a second reference point RP.sub.r2 at the phase-distance plane as shown in
[0049] Next, the processor 49 or the external processor obtains the first reference phase-distance relationship L.sub.Rr on the phase-distance plane according to a line connecting the first reference point RP.sub.r1 and the second reference point RP.sub.r2. The first reference phase-distance relationship L.sub.Rr is then recorded in the memory.
[0050] At the reference temperature, when the sampling signal S.sub.d1 and the light source driving signal S.sub.d2_nd and S.sub.d2_d have the first time delay T.sub.d0, the second pixel 332 generates a first detection output signal S.sub.O2_nd as shown in
[0051] Similarly, the second pixel 332 is a single pixel or includes multiple pixels, and used to directly output the first detection output signal S.sub.O2_nd and the second detection output signal S.sub.O2_d, or output a summation or an average of output signals of the multiple pixels to form the first detection output signal S.sub.O2_nd and the second detection output signal S.sub.O2_d.
[0052] Next, the processor 49 of the light sensor 33 or an external processor obtains a second reference phase-distance relationship according to the first detection output signal S.sub.O2_nd and the second detection output signal S.sub.O2_d, e.g., a line L.sub.Ar shown in
[0053] Similarly, the line L.sub.Ar is determined by using two detected points AP.sub.r1 and AP.sub.r2. For example, the processor 49 or the external processor obtains a first detected phase (e.g., a longitudinal axis value of first detected point AP.sub.r1) according to the first detection output signal S.sub.O2_nd, and obtains a first detected distance (e.g., a transverse axis value of first detected point AP.sub.r1) according to the first time delay T.sub.d0 and the predetermined time of flight T.sub.tof (e.g., T.sub.overall=T.sub.d0+T.sub.tof) so as to obtain a first detected point AP.sub.r1 at a phase-distance plane as shown in
[0054] Similarly, the processor 49 or the external processor obtains a second detected phase (e.g., a longitudinal axis value of second detected point AP.sub.r2) according to the second detection output signal S.sub.O2_d, and obtains a second detected distance (e.g., a transverse axis value of second detected point AP.sub.r2) according to the second time delay T.sub.delay and the predetermined time of flight T.sub.tof (e.g., T.sub.overall=T.sub.delay+T.sub.tof) so as to obtain a second detected point AP.sub.r2 at the phase-distance plane as shown in
[0055] Next, the processor 49 or the external processor obtains a second reference phase-distance relationship L.sub.Ar according to a line connecting the first detected point AP.sub.r1 and the second detected point AP.sub.r2 on the phase-distance plane. In one aspect, the second reference phase-distance relationship L.sub.Ar is recorded in a memory. In another aspect, the processor 49 or the external processor obtains a detection compensation (e.g., a phase-axis intercept of L.sub.Ar) and a detection correction (e.g., the correction to calibrate a slope of L.sub.Ar to a slope of ideal line, e.g., dotted line shown in
[0056] After the above parameters are recorded or stored in the memory of the TOF sensor 300 before shipment, the setting stage is accomplished. It is appreciated that although the above descriptions are described in the way using two points to determine L.sub.Rr and L.sub.Ar as an example, the present disclosure is not limited thereto. In other aspects, by setting multiple time delays T.sub.delay, it is possible to use multiple reference points to determine L.sub.Rr and use multiple detected points to determine L.sub.Ar.
[0057] Please referring to
[0058] Similarly, the time control circuit controls the sampling signal S.sub.d1 and the light source driving signal S.sub.d2_nd and S.sub.d2_d to sequentially have a first time delay T.sub.d0 and the second time delay T.sub.delay according to
[0059] At the operating temperature, the first pixel 331 generates a first operation output signal, which is similar to S.sub.O1_nd in
[0060] The processor 49 of the light sensor 33 obtains an operation phase-distance relationship, e.g., a line L.sub.Ro in
[0061] Similarly, the line L.sub.Ro is determined by using two reference operation points RP.sub.O1 and RP.sub.O2. For example, the processor 49 obtains a first operation phase (e.g., a longitudinal axis value of first reference operation point RP.sub.O1) according to the first operation output signal, and obtains a first distance (e.g., a transverse axis value of first reference operation point RP.sub.O1) according to the first time delay T.sub.d0 so as to obtain a first reference operation point RP.sub.O1 at a phase-distance plane as shown in
[0062] Next, the processor 49 obtains the operation phase-distance relationship L.sub.Ro according to a line connecting the first reference operation point RP.sub.O1 and the second reference operation point RP.sub.O2. The operation phase-distance relationship L.sub.Ro is then recorded in the memory.
[0063] Please referring to
[0064] When the sampling signal S.sub.d1 and the light source driving signal S.sub.d2_nd and S.sub.d2_d have the first time delay T.sub.d0, the second pixel 332 generates an operation detected signal, similar to S.sub.O2_nd as shown in
[0065] Using the same method mentioned above, the processor 49 calculates a current phase (e.g., a longitudinal axis value of current operation point AP.sub.O) according to an ratio of area of the operation detected signal, and calculates a current distance (e.g., a traverse axis value of current operation point AP.sub.O) according to T.sub.overall=T.sub.tof so as to obtain a current operation point AP.sub.O. In this stage, the object distance is a value to be measured. As shown in
Compensated distance=velocity of light×detection correction×[(current phase−Δoffset)/Δslope−detection compensation]/2
[0066] That is, the processor 49 calibrates the current distance according to the recorded temperature compensation, temperature correction, detection compensation and detection correction, wherein the current distance is calculated according to a time of flight currently detected. In other words, if the object is arranged just at the predetermined distance (i.e. the distance D for obtaining L.sub.Rr and L.sub.Ar before shipment), an operation line L.sub.Ao shown in
[0067] In another aspect, the processor 49 calculates a current phase and a current distance according to the operation detected signal generated by the second pixel 332 when the sampling signal S.sub.d1 and the light source driving signal S.sub.d2_nd and S.sub.d2_d) have the second time delay T.sub.delay. That is, in the present disclosure the time delay used for calculating the current phase and the current distance is not particularly limited.
[0068] Referring to
[0069] Details of these steps of the temperature compensation method have been illustrated above, wherein the Steps S112-S114 are executed, for example, in a setting stage before shipment; the Steps S115-S118 are executed, for example, in a booting stage before accrual operation; the Step S111 is executed in both the setting stage and the booting stage; and the Step S119 is executed according to a current phase measured according to a current time of flight in actual operation.
[0070] In addition, the above steps are mainly used to eliminate the phase deviation caused by temperature variation. If it is desired to eliminate the distance deviation caused by other factors, the temperature compensation method of the present disclosure preferably further includes the steps of: generating a first detection output signal S.sub.O2_nd by the second pixel 332 at the reference temperature and the first time delay T.sub.d0; generating a second detection output signal S.sub.O2_d by the second pixel 332 at the reference temperature and the second time delay T.sub.delay; obtaining a second reference phase-distance relationship L.sub.Ar according to the first detection output signal S.sub.O2_nd and the second detection output signal S.sub.O2_d; calculating and recording a detection compensation and a detection correction according to the second reference phase-distance relationship L.sub.Ar; and calibrating a current distance obtained according an operation detected signal using the detection compensation and the detection correction. Details of these steps are also described above, and thus are not repeated herein.
[0071] In the present disclosure, it is assumed that the operating temperature does not have apparent change within a short time, and thus the operation phase-distance relationship L.sub.Ro is described in the way being obtained in the booting stage. However, if the operating temperature can change significantly within a short time, the operation phase-distance relationship L.sub.Ro is calculated and recorded anytime according to the requirement of the user as long as being recorded before the actual operation.
[0072] As mentioned above, the object distance detected by the conventional TOF sensor is deviated due to the environmental temperature change, and the conventional method for compensating this deviation is not effective to the TOF sensor with modulated light source. Therefore, the present disclosure further provides a time of flight sensor for compensating the temperature deviation (as shown in
[0073] Although the disclosure has been explained in relation to its preferred embodiment, it is not used to limit the disclosure. It is to be understood that many other possible modifications and variations can be made by those skilled in the art without departing from the spirit and scope of the disclosure as hereinafter claimed.