Distance measuring apparatus which detects optical system abnormality

Abstract

A distance measuring apparatus includes an optical system abnormality detection section for detecting abnormalities of an optical system of the distance measuring apparatus by comparing a relationship between a measured distance value d and a light intensity Ls with a reference value K.sub.ρ, K.sub.ρ1, K.sub.Ls0d0, or K.sub.Ab0.

Claims

1. A distance measuring apparatus, comprising an irradiation section which irradiates an object with distance measurement light, and a light receiving section which receives reflected light from the object, wherein the distance measuring apparatus outputs a distance measurement value to the object calculated based on a phase difference between the distance measurement light and the reflected light, and a light intensity of the reflected light, the distance measurement apparatus further comprising: an optical system abnormality detection section which detects an abnormality of an optical system of the distance measuring apparatus by comparing a relationship Ls.Math.d.sup.2 between the distance measurement value d and the light intensity Ls with a reference value.

2. The distance measuring apparatus according to claim 1, wherein the reference value is determined using a reflectivity of the object.

3. The distance measuring apparatus according to claim 1, wherein the reference value is determined using the relationship between the distance measurement value and the light intensity acquired when the distance measuring apparatus is normal.

4. The distance measuring apparatus according to claim 1, further comprising: a two-dimensional image generation section which generates a two-dimensional image of a space observed by the distance measuring apparatus based on a light reception amount of the light receiving section, wherein an abnormality of the optical system is determined using, in place of the distance measurement value, a distance to the object calculated based on the two-dimensional image in which the object, which has a plurality of feature points having obvious three dimensional coordinate correlations, is captured.

5. The distance measuring apparatus according to claim 1, wherein the reference value is determined using an abnormality of the optical system detected using another object which is different from the object, and the relationship between the distance measurement value to the object, which is captured simultaneously with another object, and the light intensity.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is a block diagram of the distance measuring apparatus according to one embodiment.

(2) FIG. 2A is an explanatory view showing an optical system abnormality detection principle.

(3) FIG. 2B is an explanatory view showing an optical system abnormality detection principle.

(4) FIG. 2C is an explanatory view showing an optical system abnormality detection principle.

(5) FIG. 3A is a view showing detection of abnormalities according to other embodiment applied to an object having a known reflectivity.

(6) FIG. 3B is a view showing abnormality detection according to yet another embodiment applied to an object having an unknown reflectivity.

(7) FIG. 4 is an explanatory view showing an example in which optical system abnormality is detected using the distance to an object calculated based on a two-dimensional image.

(8) FIG. 5A is an explanatory view showing a method for determining a reference value according to yet another embodiment.

(9) FIG. 5B is an explanatory view showing a method for determining a reference value according to yet another embodiment.

(10) FIG. 6A is an explanatory view showing that the adhesion of spot-like dirt can be regarded as optically equivalent to uniform soiling.

(11) FIG. 6B is a view showing that the adhesion of spot-like dirt can be regarded as optically equivalent to uniform soiling.

(12) FIG. 7A is an explanatory view showing distance measurement theory according to prior art.

(13) FIG. 7B is an explanatory view showing an S/N ratio when an optical system is normal according to prior art.

DESCRIPTION OF PREFERRED EMBODIMENTS

(14) The embodiments of the present disclosure will be described in detail below with reference to the attached drawings. In the drawings, identical or similar constituent elements have been assigned the same or similar reference numerals. Furthermore, the embodiments described below do not limit the technical scope of the inventions described in the claims or the definitions of the terms.

(15) FIG. 1 is a block diagram of the distance measuring apparatus 10 according to one embodiment. The distance measuring apparatus 10 is a TOF camera, laser scanner, etc., which measures the distance to an object based on, for example, a phase difference method, and comprises an irradiation section 11 which irradiates an object O with intensity-modulated distance measurement light L1, a light receiving section 12 which receives reflected light L2 from the object O, and an output control section 13 which outputs distance measurement values to the object and the light intensity of the reflected light based on the phase difference between the distance measurement light and the reflected light.

(16) The irradiation section 11 is constituted by a distance measurement light source such as a light-emitting diode or a semiconductor laser which emits, for example, near-infrared (NIR) light, and a diffusion plate which diffuses the NIR light as the distance measurement light L1. The irradiation section 11 illuminates the object O with the distance measurement light L1 which is intensity modulated on a predetermined cycle based on a light emission timing signal from a light emission/image capture timing control section 14.

(17) The light receiving section 12 is constituted by, for example, a condenser lens which focuses reflected light L2 from the object O, a photoelectric element which receives the reflected light L2 via an RGB filter, an NIR filter, etc., and a CCD imaging sensor, a CMOS imaging sensor, etc. The reflected light L2 includes reflected ambient light in addition to the distance measurement light L1 reflected by the object O. The light receiving section 12 may include, per pixel, four light receiving elements for receiving red light, blue light, green light, and NIR light, or may include, per pixel, a single light receiving element for receiving only NIR light. In some cases, the light receiving section 12 includes a plurality of two-dimensionally arranged light receiving elements 15.

(18) In laser scanners, in general, the light receiving section 12 performs direct measurement of the delay Δt of the reflected light with respect to the irradiation timing of the reference light shown in FIGS. 7A and 7B or the light intensity of the reflected light components. The present embodiment will be described hereinafter under the assumption that the distance measuring apparatus 10 is a TOF camera. The light receiving element 15 is constituted by, for example, a photo diode, a capacitor, etc. A light receiving element 15 which receives NIR light acquires light reception amounts Q.sub.1 to Q.sub.4 at a plurality of image capture timings which are out of phase by 0°, 90°, 180°, and 270°, respectively, with respect to, for example, the light emission timing of the distance measurement light L1 based on an image capture timing signal from the light emission/image capture timing control section 14. A light receiving element 15 which receives red light, blue light, and green light acquires the respective light reception amounts over a predetermined image capture interval. The acquired light reception amounts are amplified by an amplification section 16, A/D converted by an A/D conversion section 17, and the A/D converted values are stored in a buffer memory 18.

(19) A distance image generation section 19 generates a distance image 30 including the distance measurement value to the object O for each pixel based on the A/D converted values of the light reception amounts Q.sub.1 to Q.sub.4 of the NIR light. The distance measurement value d is calculated from, for example, the known formula below. Td is the phase difference between the distance measurement light and the reflected light, c is the speed of light, and f is the frequency of the distance measurement light. The generated distance image 30 is stored in a buffer memory 20, and is output to the outside of the distance measuring apparatus 10 via the output control section 13.

(20) $\begin{array}{cc}\mathrm{Td}=\mathrm{arc}\tan \left(\frac{Q_2-Q_4}{Q_1-Q_3}\right)& \left[\mathrm{Formula}1\right]\\ d=\frac{c}{4\pi f}\mathrm{Td}& \left[\mathrm{Formula}2\right]\end{array}$

(21) A two-dimensional image generation section 21 generates a two-dimensional image 31 based on the A/D converted values of the light reception amounts of the RGB light or the NIR light. In other words, the two-dimensional image 31 may be an RGB image (color image), or may be an NIR image (monochrome image). The two-dimensional image 31 is stored in the buffer memory 20, and is output to the outside of the distance measuring apparatus 10 via the output control section 13. Note that when the distance measuring apparatus 10 is a TOF camera, the light intensity Ls output from the light receiving element 15 is calculated using, for example, the known formula below, based on the light reception amounts Q.sub.1 to Q.sub.4.

(22) $\begin{array}{cc}\mathrm{Ls}=\frac{\sqrt{{\left(Q_1-Q_3\right)}^2+{\left(Q_2-Q_4\right)}^2}}{2}& \left[\mathrm{Formula}3\right]\end{array}$

(23) As described above with reference to FIG. 7B, when the optical system is not normal, the reflected light becomes weak, whereby the S/N ratio is reduced, erroneous detection or undetectability of signal components of the distance measurement light occurs, and the distance measurement accuracy is reduced or loss of the distance measurement capability occurs. In order to detect such abnormalities of the optical system, and to maintain the distance measurement accuracy or distance measurement capability, the distance measuring apparatus 10 comprises an optical system abnormality detection section 22 which detects abnormalities of the optical system. The optical system abnormality detection section 22 can be constituted as software which causes, for example, a processor such as a central processing unit (CPU) to function. Alternatively, for example, the processing of at least some of such software may be executed as hardware such as an executable processor.

(24) FIGS. 2A to 2C are explanatory views of the principle of detection of abnormalities of the optical system. As shown in FIG. 2A, the illuminance E from the distance measurement light source P toward the object is generally inversely proportional to the square of distance d, as in the following formula. I is the light intensity of the distance measurement light source P in the direction of observation.

(25) $\begin{array}{cc}E=\frac{I}{d^2}& \left[\mathrm{Formula}4\right]\end{array}$

(26) Furthermore, as shown in FIG. 2B, when the object O has a uniformly diffusing reflective surface having a reflectivity of ρ, the luminance Lr of the reflective surface can be generally represented by the following formula.

(27) $\begin{array}{cc}\mathrm{Lr}=\frac{\rho }{\pi }.Math.E& \left[\mathrm{Formula}5\right]\end{array}$

(28) Further, as shown in FIG. 2C, the light receiving element 15 outputs an output current which is generally proportional to the incident luminous flux. Since the luminous flux received by the light receiving element 15, i.e., the light intensity Ls which is output by the light receiving element 15, is proportional to the luminance Lr of the surface of the light receiving element 15 searched for by the condenser lens 22, the following formula is obtained.

(29) $\begin{array}{cc}\mathrm{Ls}\propto \frac{\rho }{\pi }.Math.\frac{I}{d^2}& \left[\mathrm{Formula}6\right]\end{array}$

(30) When the above formula is expressed using the constant k, the following formula is obtained. The constant k is a correction value which offsets differences in apparatus structure, differences in the characteristic of each element with regards to light emission or reception, and the individual characteristic variations of each element.

(31) $\begin{array}{cc}\mathrm{Ls}=k.Math.\frac{\rho .Math.I}{d^2}& \left[\mathrm{Formula}7\right]\end{array}$

(32) The above formula can be changed to the following basic formula. Thus, a state in which the intensity I of the distance measurement light source in the direction of observation is constant and the distance of an object O having a reflectivity of ρ is measured indicates that the relationship between the distance measurement value d to the object O on the left side of the following basic formula and the light intensity Ls received by the light receiving element 15 becomes a constant value. In other words, when detecting optical system abnormality, if the calculated results of the relationship between the distance measurement value d to the object O having a reflectivity of ρ and the light intensity Ls received by the light receiving element 15 differs from a predetermined value, there is something wrong with the optical system of the distance measuring device 10.
Ls.Math.d.sup.2=k.Math.p.Math.I  [Formula 8]

(33) Though the right side of the above basic formula includes the intensity I of the distance measurement light source in the direction of observation, this value is different for each distance measuring apparatus 10 due to characteristic variations of the parts constituting the distance measuring apparatus 10. When, for example, the distance measuring apparatus 10 is normal (e.g., at the time of shipment, at the time of installation, etc.), the reference value K.sub.ρ for detecting abnormality of the optical system is obtained from the distance measurement value d obtained by calibration using an object having a reflectivity of ρ.sub.cb and the light intensity Ls, as described in the following formula. The reference value K.sub.ρ is stored in a non-volatile memory 23 (refer to FIG. 1) of the distance measuring apparatus 10 along with the reflectivity ρ.sub.cb (like the reference value, which will be described later).
K.sub.ρ=Ls.Math.d.sup.2  [Formula 9]

(34) When detecting abnormality of the optical system, the optical system abnormality detection section 22 calculates the relationship between the light intensity Ls of the object O, which has a known reflectivity ρ.sub.cb, and the distance measurement value d, and detects the abnormality Ab of the optical system of the distance measuring apparatus 10 by comparing such relationship with the reference value K.sub.ρ. The abnormality Ab of the optical system may be detected as a ratio of the relationship between the light intensity Ls and the distance measurement value d to the reference value K.sub.ρ as in, for example, the following formula, or may be detected as a difference. However, since the following formula is the theoretical principle of detection of abnormality of the optical system, it should be noted that the relationship between the light intensity Ls and the distance measurement value d of the object O may require correction due to various circumstances.

(35) $\begin{array}{cc}\mathrm{Ab}=\frac{\mathrm{Ls}.Math.d^2}{K_{\rho }}& \left[\mathrm{Formula}10\right]\end{array}$

(36) When the abnormality Ab of the optical system in the above formula is less than 1, a decrease in light intensity due to the adhesion of dirt to the optical window (lens, diffuser, etc.), a decrease in received intensity, or a decrease in output of the distance measurement light source, etc., can be suspected. By using an object O having a known reflectivity ρ.sub.cb as such a reference value, it is possible to detect abnormality of the optical system regardless of the installation environment of the distance measuring apparatus 10. Furthermore, by using the relationship between the light intensity Ls of the object O and the distance measurement value d as the measurement value of the optical system abnormality (the numerator of the above formula), detection of optical system abnormalities is possible regardless of the distance to the object O.

(37) FIG. 3A is a view showing abnormality detection according to other embodiment when an object having a known reflectivity is used. The reference value K.sub.ρ1 in the case in which an object O having a reflectivity pi, which is different from the reflectivity ρ.sub.cb when the reference value K.sub.ρ is acquired, is used is determined from the above formula using the ratio of the reflectivity ρ.sub.cb to the reflectivity ρ.sub.1, as in, for example, the following formula.

(38) $\begin{array}{cc}{K}_{\rho 1}=\frac{{\rho }_1}{{\rho }_{\mathrm{cb}}}.Math.K_{\rho }& \left[\mathrm{Formula}11\right]\end{array}$

(39) When detecting optical system abnormality, the optical system abnormality detection section 22 calculates the relationship between the light intensity Ls of the object O and the distance measurement value d, and by comparing this relationship with the reference value K.sub.ρ1, the abnormality Ab of the optical system of the distance measuring apparatus 10 is detected. The abnormality Ab of the optical system may be detected as the ratio of the relationship of the light intensity Ls and the distance measurement value d to the reference value K.sub.ρ1, as in, for example, the following formula, or may be detected as a difference.

(40) $\begin{array}{cc}\mathrm{Ab}=\frac{\mathrm{Ls}.Math.d^2}{K_{\rho 1}}& \left[\mathrm{Formula}12\right]\end{array}$

(41) FIG. 3B is a view showing abnormality detection according to another embodiment when an object having an unknown reflectivity is used. The above basic formula demonstrates that even for an object O having an unknown reflectivity, a reference value K.sub.Ls0d0, as in, for example, the following formula, can be determined using the relationship between the measured distance value d.sub.0 of the object O acquired when the distance measuring apparatus 10 is normal (e.g., at the time of installation, when maintenance has been completed, etc.) and the light intensity Ls.sub.0. Thereafter, abnormality can be detected using identical objects O (i.e., objects which have unknown reflectivities but which have the same reflectivity).
K.sub.Ls0d0=Ls.sub.0.Math.(d.sub.0).sup.2  [Formula 13]

(42) When detecting optical system abnormality, the optical system abnormality detection section 22 calculates the relationship between the light intensity Ls of the object O and the distance measurement value d, and abnormality Ab of the optical system of the distance measuring apparatus 10 is detected by comparing such relationship with the reference value K.sub.Ls0d0. The abnormality Ab of the optical system may be detected as the ratio of the relationship between the light intensity Ls and the distance measurement value d to the reference value K.sub.Ls0d0, as in, for example, the following formula, or may be detected as a difference.

(43) 0$\begin{array}{cc}\mathrm{Ab}=\frac{\mathrm{Ls}.Math.d^2}{K_{\mathrm{Ls}0d0}}& \left[\mathrm{Formula}14\right]\end{array}$

(44) As may be understood from the foregoing, the distance measuring apparatus 10 can determine the reference value from the relationship between the distance measurement value do of the object O having an unknown reflectivity and the light intensity Ls.sub.0. In other words, the object may be a unique fixed object (e.g., a unique floor, desk, machine, etc.) the image of which is captured continuously or periodically by the distance measuring apparatus 10. By using the relationship between the light intensity of such a fixed object and the distance measurement value as the measured value of the optical system abnormality (the numerator of the above formula) and the reference value (the denominator of the above formula), optical system abnormality can be detected. In other words, the distance measuring apparatus 10 can automatically maintain the distance measurement accuracy or distance measurement capability without the preparation of a special object having a known reflectivity.

(45) FIG. 4 is an explanatory view showing an example of optical system abnormality detection in which a distance d.sub.ref to the object O geometrically calculated based on the two-dimensional image 31 is used. When the distance measuring apparatus 10 includes a plurality of light receiving elements 15 arranged two-dimensionally, as in, for example, a TOF camera, a two-dimensional image constituted by the light intensity values of each of the light receiving elements 15 is output. A so-called “reference marker” is used as the object O. The reference marker is, for example, a square, white, plate-like member having a known size on which a perfect circle, a square, and a rhombus having known sizes and positional relationships are arranged, and includes a large number of feature points having known three-dimensional coordinate correlations. For example, the feature points may be central portions 32a, 32b, and 32c of the perfect circle, square, and rhombus, respectively, and the corners 32d, 32e, 32f, 32g of the square. Further, the center 32b of the perfect circle of the reference marker is a representative feature point 32b, and the inside of the perfect circle is a portion having a predetermined reflectivity ρ. Note that in another embodiment, optical system abnormality may be detected using arbitrary objects having obvious three-dimensional coordinate correlations in place of the reference marker. Note that “obvious three-dimensional coordinate correlations” means that the relative positional relationships between the plurality of feature points can be known, and it is not always necessary that the correlations of three-dimensional coordinates be known. Specifically, it is not necessary that they be stored in advance in a memory or the like of the distance measuring apparatus 10.

(46) The optical system abnormality detection section 22 detects the reference marker from the two-dimensional image 31 using known image processing, specifies the position coordinates of the various feature points of the reference marker in the image at the sub-pixel level, and geometrically calculates the distance d.sub.ref to the center 32b of the perfect circle, which is the representative feature point, from the combination of the position coordinates of the plurality of feature points (generally, four or more) in the image. In order to calculate the distance d.sub.ref with higher accuracy, a plurality of values of d.sub.ref may be calculated from different combinations of the plurality of feature points, and the average thereof may be used. The optical system abnormality detection section 22 can detect the abnormality Ab of the optical system using the geometrically calculated distance d.sub.ref to the reference marker (representative feature point) based on the two-dimensional image 31 in which the reference marker is captured, in place of the measured distance value d, as in, for example, the following formula.

(47) $\begin{array}{cc}\mathrm{Ab}=\frac{\mathrm{Ls}.Math.d_{\mathrm{ref}}^2}{K_{\rho }}& \left[\mathrm{Formula}15\right]\end{array}$

(48) By using the distance d.sub.ref to the object O calculated based on the two-dimensional image 31 in this manner, even when distance measurement accuracy is low due to optical system abnormality, it is possible to accurately detect the abnormality Ab of the optical system.

(49) FIGS. 5A and 5B are explanatory views showing methods for determining the reference value according to yet another embodiment. As shown in FIG. 5A, when another object O′ such as a reference marker is used, it can be accurately observed that the abnormality Ab.sub.0 of the optical system has advanced to some extent. Thus, the reference value K.sub.Ab0 may be determined using the degree of abnormality Ab.sub.0 of the optical system, which has advanced halfway to the observation target object O. The reference value K.sub.Abo is determined using the abnormality Ab.sub.0 of the optical system detected using another object O′ and the relationship between the measured distance value d.sub.bref and the light intensity L.sub.bref, which are related to the observation target object O which is captured simultaneously with another object O′, as in, for example, the formula below.

(50) $\begin{array}{cc}{K}_{\mathrm{Ab}0}=\frac{L_{\mathrm{bref}}.Math.d_{\mathrm{bref}}^2}{{\mathrm{Ab}}_0}& \left[\mathrm{Formula}16\right]\end{array}$

(51) As shown in FIG. 5B, when detecting optical system abnormality, the optical system abnormality detection section 22 calculates the relationship between the light intensity Ls of the object O and the distance measurement value d, and the abnormality Ab of the optical system of the distance measuring apparatus 10 is detected by comparing such relationship with the reference value K.sub.Ab0. The optical system abnormality Ab may be detected as the ratio of the relationship between the light intensity Ls and the measured distance value D to the reference value K.sub.Ab0, as in, for example, the formula below, or may be detected as a difference.

(52) $\begin{array}{cc}\mathrm{Ab}=\frac{\mathrm{Ls}.Math.d^2}{K_{\mathrm{Ab}0}}& \left[\mathrm{Formula}17\right]\end{array}$

(53) When the reference value Ab.sub.0 is determined using the degree of abnormality Ab.sub.0 of such an optical system which has advanced halfway to the observation target object O, an object having a known reflectivity, a reference marker, or the like is temporarily used, whereby abnormality of the optical system can be determined using only the observation target object.

(54) FIGS. 6A and 6B are explanatory views showing that spotty soiling can be regarded as optically equivalent to uniform soiling. Examples of the adhesion of dirt to the optical window (e.g., the lens 33, the diffusion plate 34, etc.) among the causes of optical system abnormalities include uniform adhesion such as dust and misting, and spot-like adhesion. As shown in FIG. 6A, when spot-like dirt adheres to the surface of the lens 33, a portion of the image does not become dark, but rather, the entire image becomes dark. The same applies to the diffusion plate 34 shown in FIG. 6B. Thus, it should be noted that when an observation target object having a defined reference value is captured in only a portion of the image, as in the present application, such a situation can be regarded as equivalent to detecting an anomaly of the entire image.

(55) When an optical system abnormality is detected, the optical system abnormality detection section 22 issues a warning command to a warning section 24 (refer to FIG. 1), and outputs the optical system abnormality Ab to the warning section 24. The warning section 24 is constituted by, for example, a speaker, an LED lamp, a display device, a communication control unit, or the like, and outputs the abnormality Ab of optical system to the outside of distance measuring apparatus 10 in accordance with the warning command from the optical system abnormality detection section 22.

(56) According to the above embodiment, the distance measuring apparatus 10 can easily detect abnormality of the optical system from the relationship between the distance measurement value d and the light intensity Ls, and the appropriate timing of cleaning and part replacement of the distance measuring apparatus 10 can be understood based on such abnormality Ab. Additionally, the distance measurement accuracy and distance measurement capability can be maintained.

(57) Though various embodiments have been described in the present description, the present invention is not limited to the embodiments described above, and it can be recognized that various modifications can be made within the scope of the claims described below.