ESTIMATION OF A DISTANCE OFFSET IN AN INDIRECT TIME-OF-FLIGHT MEASUREMENT DEVICE, AND CORRESPONDING DEVICE

Abstract

A method for determining a distance offset between a depth map of an indirect time-of-flight measurement device and a reference ground truth map includes emitting emission optical radiation by an emission circuit of the indirect time-of-flight measurement device. The emission optical radiation is sequentially modulating by N modulation frequencies. N is greater than or equal to three. The method further includes receiving, by a reception circuit of the indirect time-of-flight measurement device, reception optical radiation generated by the emission optical radiation reflected off objects of a scene and by optical crosstalk between the emission optical radiation and the reception optical radiation, and estimating the distance offset based on the reception optical radiation.

Claims

1. A method for determining a distance offset between a depth map of an indirect time-of-flight measurement device and a reference ground truth map, the method comprising: emitting emission optical radiation by an emission circuit of the indirect time-of-flight measurement device, the emission optical radiation being sequentially modulated by N modulation frequencies, wherein N is greater than or equal to three; receiving, by a reception circuit of the indirect time-of-flight measurement device, reception optical radiation generated by the emission optical radiation reflected off objects of a scene and by optical crosstalk between the emission optical radiation and the reception optical radiation; and estimating the distance offset based on the reception optical radiation.

2. The method according to claim 1, wherein the optical crosstalk is generated by a glass pane covering the emission circuit and the reception circuit.

3. The method according to claim 1, further comprising generating the depth map using successive groups of N demodulations of the reception optical radiation with the N respective modulation frequencies so as to acquire, in response to each group of demodulations and for each pixel of a set of pixels of the reception circuit, components of demodulated optical radiation associated with the N frequencies.

4. The method according to claim 1, wherein estimating the distance offset comprises dynamically estimating the distance offset group by group by estimating a current output distance offset after a current group of demodulations and in response to the preceding group of demodulations, the current output distance offset being based on current components of the demodulated optical radiation obtained by the current group of demodulations, and on a current input distance offset taken from a preceding output distance offset.

5. The method according to claim 4, wherein estimating the distance offset further comprises recursively filtering the preceding output distance offset, the current input distance offset being the filtered preceding output distance offset.

6. The method according to claim 4, wherein estimating the current output distance offset comprises: preparing a current set of reference distance values located on either side of a current input distance value; for each pixel of the set of pixels and for each reference distance value of the current set, estimating at least three crosstalk values based on the current components of the demodulated optical radiation, assigned to the pixel, and respectively associated with at least three pairs of modulation frequencies taken from the N modulation frequencies, and selecting a reference distance value minimizing error between the estimated crosstalk values; and determining the current output distance offset based on the selected reference distance values.

7. The method according to claim 6, wherein the reference distance values are integer values, and wherein determining the current output distance offset comprises preparing a current histogram for the set of pixels including, for each selected reference distance value, the number of pixels having resulted in the selected reference distance value.

8. The method according to claim 7, wherein preparing the current histogram comprises including each pixel for which the error between the estimated crosstalk values is minimized with a single local minimum in the current histogram, and excluding each pixel for which the error between the estimated crosstalk values is minimized with more than one local minimum from the current histogram.

9. The method according to claim 7, further comprising analyzing the current histogram using a sliding analysis window by selecting a window including the greatest number of pixels, and determining the current output distance offset in the selected window.

10. An indirect time-of-flight measurement device comprising an emission circuit configured to emit emission optical radiation sequentially modulated by N modulation frequencies, N being greater than or equal to three; a crosstalk region configured to generate optical crosstalk between the emission radiation and reflected radiation generated by the emission optical radiation reflecting off objects of a scene; a reception circuit configured to receive a reception optical radiation from the reflected radiation and the optical crosstalk; and a processing circuit configured to generate a depth map and estimate a distance offset between the depth map and a reference ground truth map based on the reception optical radiation.

11. The indirect time-of-flight measurement device according to claim 10, wherein the crosstalk region comprises a glass pane covering the emission circuit and the reception circuit.

12. The indirect time-of-flight measurement device according to claim 10, wherein the processing circuit is configured to generate the depth map using successive groups of N demodulations of the reception optical radiation with the N respective modulation frequencies, so as to acquire, in response to each group of demodulations and for each pixel of a set of pixels of the reception circuit, components of the demodulated optical radiation associated with the N frequencies.

13. The indirect time-of-flight measurement device according to claim 10, wherein the processing circuit is configured to dynamically estimate the distance offset group by group by estimating a current output distance offset after a current group of demodulations and in response to the preceding group of demodulations, the current output distance offset being based on current components of the demodulated optical radiation obtained by the current group of demodulations, and on a current input distance offset taken from a preceding output distance offset.

14. The indirect time-of-flight measurement device according to claim 12, wherein the processing circuit comprises a filtering circuit configured to recursively filter the preceding output distance offset, the current input distance offset being the filtered preceding output distance offset.

15. The indirect time-of-flight measurement device according to claim 12, wherein to estimate the current output distance offset, the processing circuit is configured to: prepare a current set of reference distance values located on either side of the current input distance offset value; for each pixel of the set of pixels and for each reference distance value of the current set, estimate at least three crosstalk values based on the components of the demodulated optical radiation, assigned to the pixel and respectively associated with at least three pairs of modulation frequencies taken from said N modulation frequencies, and select a reference distance value minimizing error between the estimated crosstalk values; and determine the current output distance offset based on the selected reference distance values.

16. The indirect time-of-flight measurement device according to claim 15, wherein the reference distance values are integer values, and wherein, for the selection of the reference distance value minimizing the error, the processing circuit is configured to prepare a current histogram for the set of pixels including, for each selected reference distance value, the number of pixels having resulted in the selected reference distance value.

17. The indirect time-of-flight measurement device according to claim 16, wherein the processing circuit is further configured to include each pixel for which the error between the estimated crosstalk values is minimized with a single local minimum in the current histogram, and exclude each pixel for which the error between the estimated crosstalk values is minimized with more than one local minimum from the current histogram.

18. The indirect time-of-flight measurement device according to claim 16, wherein the processing circuit is further configured to analyze the current histogram using a sliding analysis window by selecting a window including the greatest number of pixels, and determine the current output distance offset in the selected window.

19. A communication device comprising: an indirect time-of-flight measurement device comprising an emission circuit configured to emit emission optical radiation sequentially modulated by N modulation frequencies, N being greater than or equal to three; a crosstalk region configured to generate optical crosstalk between the emission radiation and reflected radiation generated by the emission optical radiation reflecting off objects of a scene; a reception circuit configured to receive a reception optical radiation from the reflected radiation and the optical crosstalk; and a processing circuit configured to generate a depth map and estimate a distance offset between the depth map and a reference ground truth map based on the reception optical radiation.

20. The communication device according to claim 19, wherein the communication device is a cellular mobile phone, a phablet or a digital tablet.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Further advantages and features of the invention will emerge upon studying the detailed description of embodiments and implementations, and of the appended drawings schematically illustrating embodiments of the invention, wherein:

[0049] FIG. 1 illustrates an example iToF device;

[0050] FIG. 2 illustrates another example iToF device;

[0051] FIG. 3 illustrates an example method of generating a depth map;

[0052] FIG. 4 illustrates an example method of estimating of the current output distance offset associated with the current group of demodulations of the reception optical radiation;

[0053] FIG. 5 illustrates an example method of preparing a histogram;

[0054] FIG. 6 illustrates an example method of determining validity of a pixel; and

[0055] FIG. 7 illustrates an example method of analyzing the histogram.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

[0056] FIG. 1 illustrates an indirect time-of-flight measurement device (iToF device) 1, for example incorporated into a communication device APP such as a smartphone type cellular mobile phone, a phablet, a digital tablet, and the like, without these examples of devices being limiting.

[0057] The iToF device 1 comprises an emission circuit 11 including, for example, an array of vertical-cavity surface emitting laser (VCSEL) diodes.

[0058] This emission circuit 11 is configured to emit emission optical radiation RYE sequentially modulated by N modulation frequencies. In this example, N is equal to 3.

[0059] By way of non-limiting example, a first modulation frequency f.sub.1 can be equal to 200 MHz, a second modulation frequency f2 can be equal to 177.77 MHz, and a third modulation frequency f3 can be equal to 133.33 MHz.

[0060] The emission optical radiation RYE is intended to be reflected on objects OBJ of a scene so as to generate a reflected radiation RYRF.

[0061] The iToF device 1 also includes crosstalk region 14, returned to in detail hereinafter in a structure example, configured to generate optical crosstalk DP between the emission radiation RYE and the reflected radiation RYRF.

[0062] The iToF device 1 also includes a reception circuit 12 including here a pixel array.

[0063] This reception circuit 12 is configured to receive a reception optical radiation RYR which is obtained from the reflected radiation RYRF and the crosstalk DP.

[0064] The iToF device 1 also includes a processing circuit MT configured to generate a depth map and estimate a distance offset between the depth map and a reference ground truth map based on the reception optical radiation RYR.

[0065] The processing circuit MT may be of conventional structure and include, for example, a modulator/demodulator and a computing circuit. The processing circuit MT can be embodied in a software and/or hardware manner using, for example, a microcontroller and/or hardware components.

[0066] Moreover, as will be seen in more detail hereinafter, the processing circuit MT may advantageously include a filtering circuit MFLT, for example of recursive type.

[0067] A memory circuit MM is also provided to, for example, temporarily store certain data such as demodulated signal components.

[0068] FIG. 2 illustrates more precisely but still schematically, an example of iToF device 1.

[0069] It is seen here that the emission circuit 11 and the reception circuit 12 are located in a substrate 10.

[0070] The reception circuit 12 is covered with a lens 13. Moreover, the iToF device 1 includes a glass pane 14 (e.g. for protection) covering the emission circuit 11 and the reception circuit 12.

[0071] The glass pane 14 forms here the crosstalk region referred to in FIG. 1.

[0072] Indeed, this crosstalk is here generated by the signal reflections on and inside the glass pane 14. Thus, a portion of the emission radiation RYE is reflected in the glass pane 14 in a secondary radiation RYER, which will be combined with the reflected radiation RYRF from objects of the scene.

[0073] It is also possible that a portion of the reflected radiation RYRF is also reflected within the glass pane 14. That being said, this reflection phenomenon of the reflected signal is secondary with respect to the reflection of the emission radiation RYE as the intensity of the emission radiation RYE is markedly greater than the intensity of the reflected radiation RYRF.

[0074] Under noise-free conditions and assuming a sinusoidal modulation, the demodulated complex signal S.sub.i at the frequency f.sub.i, delivered by the reception circuit 12 is given by the following formula (I).

S.sub.i=I.sub.i+jQ.sub.i=(Aexp(jφ.sub.i)+X)exp(jφ.sub.off,i)  (I)

[0075] In formula (I), I.sub.i is the in-phase component of the signal and Q.sub.i is the quadrature component (i.e. phase-shifted by 90 degrees) of the signal.

[0076] The term A denotes the amplitude of the signal and X the amplitude of the crosstalk.

[0077] Moreover, the phase φ.sub.i of the signal (without crosstalk) is given by formula (II).

φ.sub.i=4πd.sub.tf.sub.i/c

[0078] The phase shift φ.sub.off,i at the frequency f.sub.i is given by formula (III).

φ.sub.off,i=4πd.sub.0f.sub.i/c  (III)

[0079] In formulas (I), (II) and (III), j denotes the complex number in which the square is equal to −1, d.sub.t is the target distance and do is the distance offset (“zero distance”) between the depth map and the reference map, and c is the speed of light.

[0080] Reference is now made more specifically to FIG. 3 and the following to describe an implementation of the method according to an embodiment of the invention.

[0081] As illustrated very schematically in FIG. 3, the generation of the depth map comprises successive groups GR.sub.i of N (here 3) demodulations of the reception optical radiation with the three respective modulation frequencies f.sub.1, f.sub.2, f.sub.3.

[0082] Each group of three demodulations make it possible to generate an image of the depth map and also make it possible, as will be seen, to estimate the distance offset (zero distance) dynamically, demodulation group by demodulation group, based on the preceding estimated value of this distance offset.

[0083] Although it is possible to account for all the pixels of the reception circuit, it is also possible to account for only a smaller set of pixels of this reception circuit, for example pixels uniformly distributed in the reception circuit.

[0084] And each group of three demodulations makes it possible to acquire, for each pixel of the set of pixels, components I and Q of the demodulated optical radiation associated with the three modulation frequencies.

[0085] Thus, as illustrated in FIG. 3, the acquisition ST1 of the components I and Q of the pixels of the set in question and resulting from the first GR1 of demodulation at the frequencies f.sub.1, f.sub.2 and f.sub.3 makes it possible, using the input distance offset do(init) which is here an initial distance offset, for example a static distance offset, to obtain a first output distance offset do.sub.1(out).

[0086] Then, in this example, this distance offset do1(out) is filtered in a step ST11 by the recursive filtering circuit MFLT so as to supply a first filtered output distance offset do1f(out) which will be used as a second input distance offset do2(in) for the following processing operation.

[0087] This first filtered output distance offset do.sub.1f(out) is compensated in a step ST12 by a conventional compensation block, for example software, for the generation ST13 of the first image IMG1 of the depth map.

[0088] The second processing operation is similar to the first processing operation described above.

[0089] More specifically, the second group GR.sub.2 of three demodulations at the frequencies f.sub.1, f.sub.2 and f.sub.3 makes it possible to determine for each of the pixels of the set in question, the components I and Q of the demodulated reception optical radiation.

[0090] The second output distance offset do.sub.2(out) is then estimated in the step ST2 based on these components I and Q and the input distance offset do.sub.2(in) then filtered in the step ST22 by the recursive filtering circuit MFLT so as to supply the second filtered distance offset do.sub.2f(out) which will serve as a third input distance offset do.sub.3(in) to enable the estimation of the third output offset do.sub.3(out) based on the components I and Q obtained in the step ST3 in response to the third group GR.sub.3 of three demodulations.

[0091] This second filtered output distance offset do.sub.2f(out) is also delivered to the compensation block to enable a compensation ST23 with a view to the generation ST24 of the second image IMG2 of the depth map.

[0092] Reference is now made more specifically to FIG. 4 to describe an example of estimation of the current output distance offset do.sub.gf(out) associated with the current group GR.sub.g of three successive demodulations of the reception optical radiation at the frequencies f.sub.1, f.sub.2 and f.sub.3.

[0093] More specifically, these three demodulations make it possible to acquire, for the PS pixels of the set ENS in question taken from the P pixels of the reception circuit 12, the components I.sub.1, Q.sub.1 of the demodulated signal at the frequency f.sub.1, the components I.sub.2, Q.sub.2 of the demodulated signal at the frequency f.sub.2 and the components I.sub.3, Q.sub.3 of the demodulated signal at the frequency f.sub.3.

[0094] As will be detailed hereinafter with reference to FIG. 5, in the step ST40, based on all the components I.sub.1, Q.sub.1, I.sub.2, Q.sub.2, I.sub.3, Q.sub.3 of all PS pixels, the current input distance offset do.sub.g(in), equal to the filtered preceding output offset do.sub.g−1f (out), a length L and a step s, a histogram H is prepared which will be analyzed in the step ST41 to obtain the current output offset do.sub.g(out).

[0095] Then, this current output offset is filtered in the step ST42 by the recursive filtering circuit MFLT using the coefficient α less than 1, equal for example to 0.5.

[0096] The filtered current output offset do.sub.gf(out) which will be equal to the following input distance offset do.sub.g+1 (in) is then obtained.

[0097] A recursive filtering method, which may be well-known to a person skilled in the art, is defined by formula (IV).

do.sub.gf(out)=αdo.sub.g(out)+(1−α)do.sub.g(in)  (IV)

[0098] Reference is now made more specifically to FIG. 5 to describe in more detail the operations performed in the step ST40 of FIG. 4, for the current group GR.sub.g of demodulations, and for each pixel PX of the set ENS.

[0099] A range INT of integer values centered around the current input distance offset do.sub.g(in) and of length 2L is prepared.

[0100] More specifically, as illustrated in FIG. 5, this range INT is equal to [do.sub.g(in)−L; do.sub.g(in)+L].

[0101] The integer values of this range INT define so-called reference distance values, located on either side of the value of current input distance offset do.sub.g(in).

[0102] L is the maximum offset of the zero distance with respect to the last estimated value of the distance offset, namely the value do.sub.g(in).

[0103] The integer values of this range is spaced by the step s.

[0104] By way of example, for a broad range, L can be selected as equal to 300 mm and s to 8 mm.

[0105] Such a broad range is used, for example, for determining the first output distance offset do.sub.1(out) corresponding to the first group GR.sub.1 of demodulations making it possible to obtain the first image IMG1 of the depth map.

[0106] For the other images of the depth map (i.e., for the other groups of three demodulations) a narrower range can be used where L is equal, for example, to 12 mm and s is equal to 2 mm.

[0107] The processing circuit MT scans the range INT and for each value k of this range INT, the processing circuit determine for the pixel in question the three crosstalk values X.sub.12(k), X.sub.13(k) and X.sub.23(k) based on the current components I.sub.1, Q.sub.1, I.sub.2, Q.sub.2, I.sub.3, Q.sub.3 assigned to this pixel and respectively associated with the three pairs of modulation frequencies (f.sub.1, f.sub.2), (f.sub.1,f.sub.3), (f.sub.2,f.sub.3) (step ST50).

[0108] The crosstalk value X.sub.ij(k) for i,j=1.2, 1.3 and 2.3 is taken here equal to 0.5U/(V-W+10.sup.−8) where U, V and W are defined by formulas (V), (VI) and (VII) below, where the in-phase components I.sub.i and I.sub.j and quadrature components Q.sub.i and Q.sub.j at the frequency f.sub.i and at the frequency f; are expressed in MHz, and c is expressed in km/second.

U=(I.sub.i.sup.2+Q.sub.i.sup.2)−(I.sub.j.sup.2+Q.sub.j.sup.2)  (V)

V=I.sub.i cos(2πf.sub.ik/c)+I.sub.i sin(2πf.sub.ik/c)  (VI)

W=I.sub.j cos(2πf.sub.jk/c)+I.sub.j sin(2π.sub.fjk/c)  (VII)

[0109] It should be noted that the crosstalk value X.sub.ij(k) is equal in theory to 0.5U/(V−W). However, to avoid any division by zero should V=W, a very low-powered term, for example 10.sup.−8, maybe added.

[0110] Then, the processing circuit determines, in the step ST51, the error (k) between the three crosstalk values.

[0111] This error (k) is defined by the formula (VIII) where ABS denotes the absolute value.

Error(k)=ABS(X.sub.12(k)−X.sub.23(k))+ABS(X.sub.12(k)−X.sub.13(k))+ABS(X.sub.13(k)−X.sub.23(k))  (VIII)

[0112] The operations ST50 and ST51 are resumed for all the k values of the range INT.

[0113] When this is performed, in step ST52, the reference value {circumflex over (k)} of the range INT minimizing said error is determined.

[0114] The current histogram H includes for each selected reference distance value {circumflex over (k)} the number of pixels having resulted in this selected reference value.

[0115] When the processing is initialized, the histogram H is set to zero.

[0116] In the step ST53, the value of the histogram h({circumflex over (k)}) is incremented by 1.

[0117] It is possible in certain cases, that there is more than one local minimum in the range INT.

[0118] In other words, it is possible that two reference values of the range {circumflex over (k)} and {circumflex over (k)}′ minimize the error (k).

[0119] In this case, as illustrated in FIG. 6, after having determined in the step ST60 a second selected reference value {circumflex over (k)}′ minimizing the error, the processing circuit determines the absolute value of the difference between these two values {circumflex over (k)}′ and {circumflex over (k)} and compare this absolute difference value to a threshold TH (step ST61).

[0120] If this difference is less than or equal to the threshold TH taken, for example, equal to 9 mm, this means that there is only one local minimum, the pixel PX is then considered as being valid and the incrementation of the step ST53 can be performed.

[0121] If however the absolute value of the difference, in the step ST61, is greater than the threshold TH, then the pixel PX is considered to be invalid as this means that there are two local minima for the error. Consequently, there is no incrementation of the value h({circumflex over (k)}) of the histogram H (step ST62).

[0122] It should be noted that this determination of the validity or invalidity of a pixel is performed generally in wide ranges.

[0123] Once the current histogram H has been obtained, the analysis ST70 thereof illustrated in FIG. 7 is performed.

[0124] More specifically, an analysis of the histogram was performed using a sliding analysis window Wi to WL between the two bounds do.sub.g(in)−L and do.sub.g(in)+L preferably excluding the limits of the range.

[0125] The size M of this sliding window is taken, for example, to be equal to 3.

[0126] The processing circuit MT then selects the window counting the greatest number of pixels.

[0127] In the example illustrated in FIG. 7 this is the window Wopt.

[0128] The current output distance offset do.sub.g(out) is then determined in this window using for example the weighted formula illustrated in step ST71.