Time of flight camera system
09726762 · 2017-08-08
Assignee
Inventors
Cpc classification
G01S17/894
PHYSICS
G01S7/4915
PHYSICS
International classification
G01S17/36
PHYSICS
Abstract
A light transit time camera system and method for operating such a system which can be operated with at least three modulation frequencies, having the steps a) determining a phase shift (φ.sub.i) of an emitted signal (Sp1) and a received signal (Sp2) for a modulation frequency (f.sub.1, f.sub.2, f.sub.3) in a phase-measuring cycle (PM.sub.1, PM.sub.2, . . . ), b) carrying out a plurality of phase-measuring cycles (PM.sub.1, PM.sub.2, . . . ), c) determining a distance value (d.sub.n,n+1) on the basis of the phase shifts (φ.sub.n, φ.sub.n+1) determined in two successive phase-measuring cycles (PM.sub.n, PM.sub.n+1), in a distance-measuring cycle (M.sub.1, M.sub.2, . . . ), d) carrying out a plurality of distance-measuring cycles (M.sub.1, M.sub.2, . . . ), e) determining a distance deviation (Δd) between the distance values of successive distance-measuring cycles, f) outputting of a distance value (d.sub.n,n+1) as a valid distance value if the distance deviation (Δd) is within a tolerance limit (Δd.sub.tol) is provided.
Claims
1. A method for operating a TOF camera system which is operable with at least three modulation frequencies, the method comprising: a) implementation of a plurality of phase measurement cycles with different modulation frequencies; b) determination of a phase shift of an emitted and received signal for different modulation frequencies in each phase measurement cycle of the plurality of phase measurement cycles; c) implementation of a plurality of distance measurement cycles; d) determination of a distance value based on the phase shifts determined according to step b) in two subsequent phase measurement cycles in each distance measurement cycle of the plurality of distance measurement cycles; e) determination of a distance deviation between distance values according to step d) of successive distance measurement cycles; and f) output of a distance value according to step d) as a valid distance value if the distance deviation according to step e) falls within a tolerance limit.
2. The method according to claim 1, wherein the value of the tolerance limit is greater than a signal noise of the distance value and smaller than a minimum possible distance jump.
3. A TOF camera system including a modulator which is connected to a lighting means for transmitting an intensity modulated signal and a receiver of the TOF camera system, wherein a modulation control unit is connected to the modulator, and is designed such that the modulator is operable with at least three modulation frequencies, and that successive distance measurements have different modulation frequency pairings, and the one evaluation unit is designed such that: a phase shift is determined for each modulation frequency, a distance is determined for each two successive phase shift measurements, and a distance value is only output as valid if a distance deviation of successive determined distance values falls within a tolerance limit.
4. The TOF camera system including a modulator which is connected to a lighting means for transmitting an intensity modulated signal and a receiver of the TOF camera system, wherein a modulation control unit is connected to the modulator, and is designed such that the modulator is operable with at least three modulation frequencies, and that successive distance measurements have different modulation frequency pairings, and the one evaluation unit is designed such that: a phase shift is determined for each modulation frequency, a distance is determined for each two successive phase shift measurements, and a distance value is only output as valid if a distance deviation of successive determined distance values falls within a tolerance limit that is designed for carrying out a method according to claim 1.
5. The TOF camera system including a modulator which is connected to a lighting means for transmitting an intensity modulated signal and a receiver of the TOF camera system, wherein a modulation control unit is connected to the modulator, and is designed such that the modulator is operable with at least three modulation frequencies, and that successive distance measurements have different modulation frequency pairings, and the one evaluation unit is designed such that: a phase shift is determined for each modulation frequency, a distance is determined for each two successive phase shift measurements, and a distance value is only output as valid if a distance deviation of successive determined distance values falls within a tolerance limit that is designed for carrying out a method according to claim 2.
Description
BRIEF DESCRIPTION
(1) Some of the embodiments will be described in detail, with reference to the following figures, wherein like designations denote like members, wherein:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
DETAILED DESCRIPTION
(12)
(13) The TOF camera system 1 comprises a transmitting unit or a lighting module 10 including a lighting means 12 and associated beam-shaping optics 15 as well as a receiving unit or TOF camera 20 including receiving optics 25 and a TOF photo sensor 22.
(14) The TOF photo sensor 22 includes at least one transit time pixel, preferably also a pixel array, and is formed in particular as a PMD sensor. For improving the imaging properties, the receiving optics 25 are typically comprised of a plurality of optical elements. The beam-shaping optics 15 of the transmitting unit 10 can, for example, be formed as reflector or lens optics. In a very simple design, receiving- and transmitting-side optical elements can also optionally be omitted.
(15) The measurement principle of this arrangement is substantially based on the fact that starting from the phase shift of the emitted and received light, the transit time, and thus the distance traveled by the received light, can be determined. For this purpose, the light source 12 and the TOF photo sensor 22 are together subjected via a modulator 30 to a certain modulation signal M.sub.0 having a base phase length φ.sub.0. In the example depicted, a phase shifter 35 is further provided between the modulator 30 and the light source 12, using which phase shifter 35 the base phase φ.sub.0 of the modulation signal M.sub.0 of the light source 12 can be shifted by defined phase lengths φ.sub.var. For typical phase measurements, phase lengths of φ.sub.var=0°, 90°, 180°, 270° are preferably used. According to the set modulation signal, the light source 12 transmits an intensity modulated signal S.sub.p1 having the first phase length p1 or p1=φ.sub.0+φ.sub.var. In the case depicted, this signal S.sub.p1 or the electromagnetic radiation is reflected by an object 40 and, due to the distance traveled, meets correspondingly phase-shifted Δφ(t.sub.L) with a second phase length p2=φ.sub.0+φ.sub.var+Δφ(t.sub.L) as received signal S.sub.p2 on the TOF photo sensor 22. In the TOF photo sensor 22 the modulation signal M.sub.0 is mixed with the received signal S.sub.p2, wherein the phase shifting or the object distance d is determined from the resulting signal.
(16) In order to improve the measuring accuracy and/or expand the uniqueness range, it is advantageous to carry out the time of flight measurements with different modulation frequencies. For this purpose the modulator 30 is connected to a modulation control device 38, which can preferably specify modulation frequencies within a prespecified frequency spectrum.
(17) The modulator 30 could, for example, be formed as a frequency synthesizer that is controlled via the modulation control device 38 for the respective measurement task. An exchange between quartz oscillators having fixed frequencies is also conceivable.
(18) Furthermore, the receiving unit 20 is connected to an evaluation unit 27. The evaluation unit 27 can optionally also be a component of the receiving unit 20, and in particular also part of the TOF photo sensor 22. The object of the evaluation unit 27 is to determine and/or evaluate phase shifts based on the received signal in relation to the modulation frequency. The mixing of the received light beams with the modulation frequency preferably occurs in the TOF photo sensor 22 or PMD sensor. Furthermore, the modulation control unit 38 can also be a component of the evaluation unit 27. In particular, it can also be provided that the evaluation unit 27 fully or partially assumes the function of the modulation control unit 38.
(19) Infrared light-emitting diodes are preferably suited as an illumination source or light source 12. Of course, other radiation sources in other frequency ranges are conceivable; in particular, light sources in the visible frequency range might also be considered.
(20) The basic principle of phase measurement is depicted schematically in
(21)
(22) When there is impact by the signal S.sub.p2 without phase shift, i.e. Δφ(t.sub.L)=0°, for example, if the transmitted signal S.sub.p1 is guided directly onto the sensor, the phases of the modulation M.sub.0 and those from the received signal S.sub.p2 are identical, so that all generated charge carriers are captured phase-synchronously at the first gate Ga and thus a maximum difference signal having Δq=1 is present.
(23) With increasing phase shift, the charge at the first accumulation gate Ga decreases and increases at the second accumulation gate Gb. With a phase shift of Δφ(t.sub.L)=90°, the charge carriers qa, qb are equally distributed at the two gates Ga, Gb and the difference is thus 0, and after a 180° phase shift, “−1.” With further increasing phase shift, the charge at the first gate Ga increases again, so that as a result the charge difference increases once more, again reaching a maximum at 360° or 0°.
(24) Mathematically there is a correlation function here of the received signal S.sub.p2 to the modulating signal M.sub.0.
(25)
(26) With a modulation using a square signal, as already shown, a delta function results as correlation function. With a modulation using, for example, a sinusoidal signal, the result would be a cosine function.
(27) As
(28) For maximum detection of the phase shift, the IQ (inphase quadrature) method, for example, is known, wherein two measurements are carried out having phase angles shifted by 90°, i.e. for example with the phase angle φ.sub.var=0° and φ.sub.var=90° The result of a measurement with the phase angle φ.sub.var=90° is depicted in
(29) The phase angle can then be determined in a known manner via an arctan function:
(30)
(31) In order, for example, to compensate for asymmetries of the sensors, additional phase measurements shifted by 180° can be carried out, so that in the result, the phase angle can be determined as follows:
(32)
(33) From the propagation-time dependent phase shift Δφ(t.sub.L) depicted in
(34)
(35) For distances d>λ/2, as a rule, there is no possibility to measure the phase shift absolutely, so the determined phase shift can no longer be uniquely associated with a distance value.
(36)
(37)
wherein of course the distance traveled up to the receiver 20 is twice as large, namely D=2d=4λ+R.
(38) To increase the uniqueness range it is provided, as schematically depicted in
(39)
(40) Wherein the following applies for the relative phase shift φ.sub.i(f.sub.i,D) dependent on the modulation frequency and the object distance:
(41)
(42) The relative phase shift φ.sub.i(f.sub.i,D) is thus a measure of the residual R.sub.i remaining in the distance measurement. For the distance measurement, a solution for the distance comparison represented above can now be found using two phase shifts φ.sub.1/2(f.sub.1/2,D) recorded for different modulation frequencies f.sub.1, f.sub.2.
(43) One possible solution path is depicted schematically in
(44) In one possible application case it could be provided, for example, that an appropriate number of phase differential pairs are recorded together with their associated distance value D or object distance d in a table of values. With a distance measurement, it can then be determined, for example, which tabulated phase difference pair having a correspondingly associated distance value is closest to the determined phase difference pair. Alternatively, the object distance can also be calculated each time.
(45)
(46)
(47) In the example depicted, the two residuals are of the same length up to a total wavelength D, which corresponds to the smallest wavelength, i.e., here 40 m.
(48) However, for a total distance D of, for example, 70 m, the residuals are of different sizes.
(49) The relationship between phase and distance depicted in
(50)
(51) However, if a detected phase value pair falls, for example, in the center of the adjacent distance line or curve sections, the distance association is unclear. In such a case, the phase value pair could be associated with a distance of 93 m instead of the actual distance of 23 m.
(52) However, mismatches arise not only due to unclear phase value pairs, but also due to overreach of objects outside the uniqueness range. In the depicted case, the uniqueness range EB extends to 120 m total distance D, i.e. to a maximum object distance d of 60 m. A TOF camera system is typically designed such that objects outside the uniqueness range provide only a low-level signal and are not taken into consideration in the evaluation. However, objects having a higher reflectivity can generate a sufficiently high signal strength at the sensor and are recognized as objects.
(53) If, for example, such an object is at an actual distance of d=71.5, i.e. a total travel distance of 143 m, due to the uniqueness range bounded at 120 m, such an object would be captured with a total distance D of 23 m. With an overreach with the total distance of 213 m, a distance value D of 93 m results. In order to detect such mismatches, it is inventively provided that the distance measurement is repeated with additional different modulation frequencies and correspondingly different uniqueness ranges, and that distance values are only allowed when preferably all or a predetermined number of distance measurements within tolerated limits lead to the same result.
(54) In
(55) In
(56) Some examples of mismatches and measurement noise are shown in the following table:
(57) TABLE-US-00001 Distance measurement cycle 1 2 3 4 5 6 7 8 9 EB 120 200 300 120 200 300 120 200 300 Distance d.sub.12 d.sub.23 d.sub.31 d.sub.12 d.sub.23 d.sub.31 d.sub.12 d.sub.23 d.sub.31 Unclear phase value pairs D = 23 m 23 178 267.5 93 23 23 93 23 267.5 Overreaches D = 143 m 23 143 143 23 143 143 23 143 143 D = 213 m 93 13 213 93 13 213 93 13 213 D = 340 m 100 140 40 100 140 40 100 140 40 Signal noise D = 23 m 23.2 22.5 23.0 23.4 23.4 22.8 22.7 23.1 22.9
(58) In the example depicted it can be seen that measurement fluctuations caused by signal noise are smaller by orders of magnitude than the mismatches generated due to overreaches or due to unclear measurement values. While fluctuations originating due to signal noise can be readily smoothed out by averaging, incorrectly assigned distance values must be recognized and, as necessary, discarded.
(59) With regard to overreaches, it could, for example, be decided that a distance measurement value is only valid if substantially the same distance value is determined in two successive distance measurement cycles. According to such a provision, in accordance with the above table, a distance value of 143 m would be recognized as valid for an overreach of 143 m of the second and third distance measurement cycle. If at least three identical distance values are required, then, in the example depicted, all overreaches greater than 120 m are discarded.
(60)
(61) In a first phase measurement cycle PM.sub.1 a first phase shift φ.sub.1 is determined for a first modulation frequency f.sub.1, and for the subsequent phase measurement cycles PM.sub.2/3 a second and third phase shift φ.sub.2, φ.sub.3 is determined for a second and third modulation frequency f.sub.2, f.sub.3. After the third phase measurement cycle PM.sub.3 the phase measurements begin again with the first modulation frequency f.sub.1 and so forth. With more than three modulation frequencies, other frequency sequences, in particular also random sequences, can be selected.
(62) Each two successive phase measurement cycles PM.sub.n,n+1 form a distance measurement cycle M.sub.n from the one phase value pair (φ.sub.n, φ.sub.n+1) and a distance value d.sub.n,n+1 associated with this pair.
(63) According to embodiments of the invention, it is provided here that a distance value d is only considered valid if the distance difference of successive distance measurement cycles falls within tolerated limits.
(64) A possible inventive procedure is exemplarily illustrated in the following table:
(65) TABLE-US-00002 Distance measurement cycle 1 2 3 4 5 6 7 8 9 EB 120 200 300 120 200 300 120 200 300 Distance d.sub.12 d.sub.23 d.sub.31 d.sub.12 d.sub.23 d.sub.31 d.sub.12 d.sub.23 d.sub.31 D = 23 m 23.4 22.5 23.0 23.7 178 23.1 23.2 24 267 Difference 23.4 0.9 0.5 0.7 155.3 145.9 0.1 0.8 243 Tolerance 50 m — < < < > > < < > Valid value (−) (+) (+) (+) (−) (−) (+) (+) (−) Output 22.5 23.0 23.7 23.2 24
(66) In order to detect mismatches, the tolerance limit is preferably set significantly higher than a typical signal noise and below a minimum possible distance jump due to mismatch. A typical signal noise can, for example, be assumed in the angular range of π/100 and thus in the distance range of EB/100. In the case according to
(67) The distance values determined in the second, third, and fourth measurement cycle fall within the tolerance limits, are thus valid, and are output. For certain evaluation methods it could also be optionally provided to recursively evaluate and output the most recent evaluation value if it falls within the tolerance, as is the case in the depicted first measurement cycle.
(68) In the fifth measurement cycle, a mismatch occurred with a distance jump to 178 m, which exceeds the tolerance limit. This distance value is invalid and is not output. A valid distance value is not available until the seventh distance measurement cycle. Here too, it might be possible to recursively output the most recent sixth distance value falling within the tolerance as valid.
(69) According to embodiments of the invention it is provided that the tolerance limits are principally set such that misallocations are reliably detected. Of course the tolerance limits can be fixed more narrowly depending on the accuracy requirement of the distance measurements.
(70) With a moving camera and/or moving object, the movement speeds must also be considered.
(71) If, for example, the camera is moved with a maximum speed of 10 m/s and a detection rate of 1/50 s is assumed for each phase measurement, then each distance point shifts by 0.2 m between two measurements. This means that with three successive measurements, a distance point will shift by 0.6 m due to the camera movement alone. For the determination of the tolerance limits it would thus be advantageous to consider a camera proper motion and optionally also an expected object movement and possible measurement error. For example, in the present example, the tolerance limits, in particular the lower threshold for the tolerance limits, could for example be extended by 1.2 m.