Method for calibrating camera measurement system

Abstract

In a calibration method a ring-shaped jig is disposed on a machine tool workpiece. The optical axis of a camera is aligned parallel to an axis of the machine tool. The jig is photographed with the horizontal or the vertical direction of the camera aligned with an axial direction other than the signal direction of the machine tool. The circumferential shape of the jig in the photograph is extracted as a contour. The center position of the jig in the image is calculated from the contour while all distortion correction coefficients in tangential and radial directions are ignored and set to zero. The displacements of the main point of the camera are set to zero. The translation distance, which is an external parameter of calibration, is calculated based on the center position of the jig in the image and the known three-dimensional center position of the jig.

Claims

1. A method for calibrating a camera measurement system having a single camera comprising: setting a ring-shaped calibration jig at a workpiece on a machining tool; aligning an optical axis of said camera parallel to a single axis of a coordinate system of said machining tool and also aligning a horizontal direction or a vertical direction of said camera with an axial direction other than a direction of said single axis of said machining tool wherein an image of said calibration jig is aligned at a central portion of an image and photographed using said camera; extracting a circumferential shape of said calibration jig in said central portion of said image as a contour; calculating a center position (U.sub.Ring, V.sub.Ring) of said calibration jig in said image from said contour; and calculating a translation distance (tx, ty, tz) as an external parameter of calibration on the basis of said center position (U.sub.Ring, V.sub.Ring) of said calibration jig in said central portion of said image and a known three-dimensional center position (X.sub.r, Y.sub.r, Z.sub.r) of said calibration jig, wherein all the distortion correction coefficients in a tangential direction of said camera and all the distortion correction coefficients in a radial direction of said camera are ignored and set to zero and displacement of the main point of a camera are all set to zero and are included in the translation distance of the camera as $\begin{array}{c}S\left[\begin{array}{c}{U}_{\mathrm{image}}\\ V_{\mathrm{image}}\\ 1\end{array}\right]=\left[\begin{array}{ccc}{k}_x& 0& 0\\ 0& k_y& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{camera}}\\ Y_{\mathrm{camera}}\\ Z_{\mathrm{camera}}\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}{\mathrm{fk}}_x& 0& 0& 0\\ 0& {\mathrm{fk}}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{camera}}\\ Y_{\mathrm{camera}}\\ Z_{\mathrm{camera}}\\ 1\end{array}\right]\\ S\left[\begin{array}{c}{U}_{\mathrm{image}}\\ V_{\mathrm{image}}\\ 1\end{array}\right]=\left[\begin{array}{cccc}{\mathrm{fk}}_x& 0& 0& 0\\ 0& {\mathrm{fk}}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{camera}}\\ Y_{\mathrm{camera}}\\ Z_{\mathrm{camera}}\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}{\mathrm{fk}}_x& 0& 0& 0\\ 0& {\mathrm{fk}}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_x\\ r_{21}& r_{22}& r_{23}& t_y\\ r_{31}& r_{32}& r_{33}& t_z\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{world}}\\ Y_{\mathrm{world}}\\ Z_{\mathrm{world}}\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}{\mathrm{fk}}_x& 0& 0& 0\\ 0& {\mathrm{fk}}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cccc}1& 0& 0& t_x\\ 0& 1& 0& t_y\\ 0& 0& 1& t_z\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{world}}\\ Y_{\mathrm{world}}\\ Z_{\mathrm{world}}\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}{\mathrm{fk}}_x& 0& 0& {\mathrm{fk}}_x{t}_x\\ 0& {\mathrm{fk}}_y& 0& {\mathrm{fk}}_y{t}_y\\ 0& 0& 1& t_z\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{world}}\\ Y_{\mathrm{world}}\\ Z_{\mathrm{world}}\\ 1\end{array}\right]\\ S\left[\begin{array}{c}{U}_{\mathrm{image}}\\ V_{\mathrm{image}}\\ 1\end{array}\right]=\left[\begin{array}{cccc}{\mathrm{fk}}_x& 0& 0& {\mathrm{fk}}_x{t}_x\\ 0& {\mathrm{fk}}_y& 0& {\mathrm{fk}}_y{t}_y\\ 0& 0& 1& t_z\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{world}}\\ Y_{\mathrm{world}}\\ Z_{\mathrm{world}}\\ 1\end{array}\right]\\ S\left[\begin{array}{c}{U}_{\mathrm{Ring}}\\ V_{\mathrm{Ring}}\\ 1\end{array}\right]=\left[\begin{array}{cccc}{\mathrm{fk}}_x& 0& 0& {\mathrm{fk}}_x{t}_x\\ 0& {\mathrm{fk}}_y& 0& {\mathrm{fk}}_y{t}_y\\ 0& 0& 1& t_z\end{array}\right]\left[\begin{array}{c}\mathrm{Xr}\\ \mathrm{Yr}\\ \mathrm{Zr}\\ 1\end{array}\right]\end{array}$ wherein, k.sub.x=1/s.sub.x, k.sub.y=1/s.sub.y, s.sub.x and s.sub.y indicates image pixel size and focus distance (fk.sub.x, fk.sub.y) is known based on the specification of a lens and the camera; and wherein
{right arrow over (X)}.sub.world=[X.sub.world,Y.sub.world,Z.sub.world,1].sup.T: Coordination of world coordination system
{right arrow over (X)}.sub.camera=[X.sub.camera,Y.sub.camera,Z.sub.camera,1].sup.T: Coordination of Camera coordination system
{right arrow over (U)}.sub.image=[U.sub.image,V.sub.image,1].sup.T: Coordination projected on image plane and {right arrow over (R)} and {right arrow over (T)} are exterior parameters of the calibration, {right arrow over (R)} indicating a rotational angle of a camera and {right arrow over (T)} indicating a setting position as $\begin{array}{c}\mathrm{Exterior}\mathrm{paramter}\left(\mathrm{rotation}\right)\text{:}\\ \stackrel{.fwdarw.}{R}=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]\\ =\left[\begin{array}{ccc}\cos & -\sin z& 0\\ \sin z& \cos z& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}\cos y& 0& \sin y\\ 0& 1& 0\\ -\sin y& 0& \cos y\end{array}\right]\\ \left[\begin{array}{ccc}1& 0& 0\\ 0& \cos x& -\sin x\\ 0& \sin x& \cos x\end{array}\right]\\ \mathrm{Exterior}\mathrm{parameter}\left(\mathrm{translation}\right)\text{:}\\ \stackrel{.fwdarw.}{T}=\left[\begin{array}{c}{t}_x\\ t_y\\ t_z\end{array}\right]\end{array}$ where .sub.x is the rotational angle with respect to X-axis, .sub.y is the rotational angle with respect to Y-axis and .sub.z is the rotational angle with respect to Z-axis.

2. A method for calibrating a camera measurement system having a single camera comprising: setting a ring-shaped calibration jig at a workpiece on a machining tool; moving a camera parallel to a X-axis, Y-axis and Z-axis of the coordinate system of said machining tool, while a plurality of images of said calibration jig are aligned at a central portion of an image and photographed by said camera; extracting a circumferential shape of said calibration jig in said central portion of said image as a contour; calculating a center position (U.sub.image, V.sub.mage) of said calibration jig in said image from said contour; and calculating a rotation angle (x, y, z) and a translation distance (tx, ty, tz) as an external parameter of calibration on the basis of said center position (U.sub.image, V.sub.image) of said calibration jig in said central portion of said image and a known three-dimensional center position (X.sub.world, Y.sub.world, Z.sub.world) of said calibration jig, wherein all the distortion correction coefficients in a tangential direction of said camera and all the distortion correction coefficients in a radial direction of said camera are ignored and set to zero and displacement of the main point of a camera are all set to zero and are included in the translation distance of the camera as $\begin{array}{c}S\left[\begin{array}{c}{U}_{\mathrm{image}}\\ V_{\mathrm{image}}\\ 1\end{array}\right]=\left[\begin{array}{cccc}{\mathrm{fk}}_x& 0& 0& 0\\ 0& {\mathrm{fk}}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{camera}}\\ Y_{\mathrm{camera}}\\ Z_{\mathrm{camera}}\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}{\mathrm{fk}}_x& 0& 0& 0\\ 0& {\mathrm{fk}}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_x\\ r_{21}& r_{22}& r_{23}& t_y\\ r_{31}& r_{32}& r_{33}& t_z\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{world}}\\ Y_{\mathrm{world}}\\ Z_{\mathrm{world}}\\ 1\end{array}\right]\end{array}$ wherein, k.sub.x=1/s.sub.x, k.sub.y=1/s.sub.y, s.sub.x and s.sub.y indicates image pixel size and focus distance (fk.sub.x, fk.sub.y) is already known based on the specification of a lens and the camera, and wherein
{right arrow over (X)}.sub.world=[X.sub.world,Y.sub.world,Z.sub.world,1].sup.T: Coordination of world coordination system
{right arrow over (X)}.sub.camera=[X.sub.camera,Y.sub.camera,Z.sub.camera,1].sup.T: Coordination of Camera coordination system
{right arrow over (U)}.sub.image=[U.sub.image,V.sub.image,1].sup.T: Coordination projected on image plane and {right arrow over (R)} and {right arrow over (T)} are exterior parameters of the calibration, {right arrow over (R)} indicating a rotational angle of a camera and {right arrow over (T)} indicating a setting position as $\begin{array}{c}\mathrm{Exterior}\mathrm{paramter}\left(\mathrm{rotation}\right)\text{:}\\ \stackrel{.fwdarw.}{R}=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]\\ =\left[\begin{array}{ccc}\cos & -\sin z& 0\\ \sin z& \cos z& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}\cos y& 0& \sin y\\ 0& 1& 0\\ -\sin y& 0& \cos y\end{array}\right]\\ \left[\begin{array}{ccc}1& 0& 0\\ 0& \cos x& -\sin x\\ 0& \sin x& \cos x\end{array}\right]\\ \mathrm{Exterior}\mathrm{parameter}\left(\mathrm{translation}\right)\text{:}\\ \stackrel{.fwdarw.}{T}=\left[\begin{array}{c}{t}_x\\ t_y\\ t_z\end{array}\right].\end{array}$

Description

BRIEF DESCRIPTION OF THE DRAWING

(1) FIG. 1(a) is a perspective view for showing how a calibration jig using a camera according to the present invention measures. FIG. 1(b) shows a contour of a circumferential of a circle extracted from an image and a center position calculated from the circumferential of the circle extracted in the present invention.

(2) FIG. 2 is a perspective view for measuring a center position of a hole by a touch sensor in a conventional art.

(3) FIG. 3(a) is a perspective view for showing a front side of a boom of a hydro pressure shovel and FIG. 3(b) is a perspective view for showing a rear side of the boom of the hydro pressure shovel. FIG. 3(c) is an enlarged perspective view of a portion in broken circles in FIGS. 3(a) and 3(b).

(4) FIG. 4 shows a displacement between a focus point and a main point.

(5) FIG. 5(a) is a sketched model of a digital camera, FIG. 5(b) explains a focus distance and FIG. 5(c) shows a displacement of the main point.

(6) FIG. 6(a) explains a distortion correction coefficient in a radial direction. FIG. 6(b) explains a sketch for showing an image before correcting the distortion in the radial direction and an image after correcting the distortion in the radial direction. FIG. 6(c) explains a distortion correction coefficient in a tangential direction. FIG. 6(d) explains a sketch for showing an image before correcting the distortion in the tangential direction and an image after correcting the distortion in the tangential direction.

(7) FIG. 7 is a perspective view of a calibration board with a checked pattern.

(8) FIG. 8 is a perspective view of a calibration board with circles arranged in a checked pattern.

(9) FIG. 9(a) through FIG. 9(e) show how necessary interior parameters are extracted from each obtained image and the respective obtained image.

(10) FIG. 10 shows how characteristic points are extracted from the obtained image.

(11) FIG. 11(a) shows a calibration board before processing a calibration operation. FIG. 11(b) shows the calibration board after processing the calibration operation.

(12) FIG. 12 explains how a necessary image to extract exterior parameter is obtained.

(13) FIG. 13(a) through FIG. 13(d) show images from which necessary exterior parameters are extracted.

(14) FIG. 14 explains how the characteristic points are extracted from the obtained image.

(15) FIG. 15 is a perspective view for showing hoe a camera is arranged with respect to a calibration jig in the present invention.

(16) FIG. 16 is a flow chart for explaining a calibration method in the third embodiment according to the present invention.

EMBODIMENT TO DO THE PRESENT INVENTION

(17) In the present invention, a ring-shaped calibration jig 1 as shown in FIG. 1(a) is photographed by a camera 2. A contour 1a of a circumference of a circle of an image photographed (shown as an alternate long and short dash line in FIG. 1(b)) is extracted. A center position 1b of the calibration jig 1 is calculated based on the extracted contour 1a of the circumference of the circle.

(18) As a ring-shaped calibration jig 1, for example, a cylindrical member 3a provided at a boom 3 of a hydro pressure shovel as shown in FIG. 3 can be utilized.

(19) The boom 3 of the hydro pressure shovel is simultaneously machined at the both sides of the boom 3 by an opposed arranged machine utilized for processing a construction equipment (a moving column type opposed arranged boring milling machine, a moving table type opposed arranged horizontal boring milling machine and so on). In the conventional art as shown in FIG. 3, a center position of a hole is measured by attaching four touch sensors 4 along an inner peripheral surface of a cylindrical member 3a. Therefore, a measurement operation becomes more complicatedly.

(20) In the present invention, a ring-shaped calibration jig 1 is employed. On the other hand, a calibration board as shown in FIG. 7 and FIG. 8 is not utilized since such a calibration board has some difficulty in view of a manufacture and an attaching process. Concerning with the ring-shaped calibration jig 1, it is possible to use a cylindrical member 3a into which an axis is completely penetrated of a boom 3 of a hydro pressure shovel and a cylindrical member into which an axis is not penetrated.

(21) In the present invention, the ring shaped calibration jig 1 is photographed by a camera 2. A shape to be measured is only suitable for a hole shape, so that a camera calibration can be omitted. That is, it is possible to ignore distortion correction coefficients in a tangential direction and distortion correction coefficients in a radial direction and set to zero. A displacement of a main point of the camera may be included in a translation distance of a camera and set to zero.

(22) As described above, the interior parameters of the camera are determined so as to correct a lens distortion in the camera calibration. In the case that the measured object is a circular shape and a center position of the circular shape is measured, an influence caused by the calibration is little. Therefore, the distortion correction can be omitted.

(23) In the present invention, when the camera 2 is attached, an optical axis of the camera 2 is aligned parallel to a z-axis of the machine coordination. Further, a perpendicular line of the camera is aligned along a x-axis of the machine coordination and a parallel line of the camera is aligned along a y-axis of the machine coordination so that it is unnecessary to correct three parameters (rotation) in the camera calibration Thus, by specifically setting a relation between the coordination of the camera 2 and a machine coordination as described above, within the exterior parameters (x, y, z, tx, ty, tz) of the camera, three exterior parameters can be set to zero (x=0, y=0, z=0). The determined parameters are only three translate parameters (tx, ty, tz).

(24) Further, if the parameter tx is already known, two unknown parameters (tx, ty) are determined.

(25) Herein, a center position 1b of the calibration jig 1 in an image has been already determined. If a known center position (Zr, Yr, Zr) of the calibration ring is designated as a center position, the unknown parameters (tx, ty) still remained can be determined.

(26) In the present invention, if the coordination of the camera 2 is unrelated with respect to the machine coordination in the case that the camera 2 is set, it is necessary to determine six exterior parameters (x, y, z, tx, ty, tz) of the camera 2.

(27) In such a case, the camera 2 is translated parallel to the X-axis, Y-axis and Z-axis of the machine coordination while a plurality of images of the calibration jig 1 are photographed. Then, a contour of the calibration jig 1 is extracted as a circumference of the circle 1a in each photographed image so as to calculate a center position 1b of the calibration jig 1 based on the circumferential of the circle 1a. By repeating such a process, a known center position 1b of the calibration jig 1 in the photographed image is utilized to a center position (Xr, Yr, Zr) of the calibration jig 1 so that six exterior parameters (x, y, z, tx, ty, tz) can be determined.

(28) As described above, in the present invention, by utilizing a ring-shaped calibration jig, an influence of a camera calibration can be omitted and an operation of the calibration can be simplified.

Embodiment 1

(29) 1) The present invention is exclusively used to measure a shape of a hole, interior parameters in a camera (focus distance: (fk.sub.x, fk.sub.y), displacement of a main point: (c.sub.x, c.sub.y), distortion correction coefficient in a radial direction: (k.sub.1, k.sub.2, k.sub.3), distortion correction coefficient in a tangential direction: (p.sub.1, p.sub.2)) are specifically set as follows: (1) Regarding a camera and a lens within the camera, a distortion in a tangential direction is little so that distortion correction coefficient in a tangential direction: (p1, p2) can be ignored and the all set to zero. (2) A purpose of the measurement according to the present invention is to measure a shape of a hole wherein a measured shape of the hole is aligned at a central portion of an image. Therefore, an influence of the distortion in the radial direction is not occurred. Therefore, in the system according to the present invention, there is no problem even if the all distortion correction coefficients in the radial direction: (k.sub.1, k.sub.2, k.sub.3) are set to zero. (3) The displacement of the main point: (c.sub.x, c.sub.y) is included in the exterior parameter of the camera (translation distance: (t.sub.x, t.sub.y, t.sub.z)) so that the displacement of the main point can be set as zero. (4) The focus distance: (fk.sub.x, fk.sub.y) depends on a specification of the lens and the camera. Therefore, necessary values are known in a specification listed by those manufactures. Based on the above situation,

(30) $\begin{array}{cc}\left[\mathrm{No}.8\right]& \\ \begin{array}{c}S\left[\begin{array}{c}{U}_{\mathrm{image}}\\ V_{\mathrm{image}}\\ 1\end{array}\right]=\left[\begin{array}{ccc}{k}_x& 0& 0\\ 0& k_y& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{camera}}\\ Y_{\mathrm{camera}}\\ Z_{\mathrm{camera}}\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}f{k}_x& 0& 0& 0\\ 0& f{k}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{camera}}\\ Y_{\mathrm{camera}}\\ Z_{\mathrm{camera}}\\ 1\end{array}\right]\end{array}\mathrm{And},& \left(21\right)\\ \left[\mathrm{No}.9\right]& \\ X_{\mathrm{camera}}=\left(1+k_1{r}^2+k_2{r}^4+k_3{r}^6\right)X_c+p_1\left(r^2+2X_c^2\right)+2p_2{X}_c{Y}_c=X_c\left(k_1=0,k_2=0,k_3=0,p_1=0,p_2=0\right)& \left(22\right)\\ Y_{\mathrm{camera}}=\left(1+k_1{r}^2+k_2{r}^4+k_3{r}^6\right)Y_c+p_2\left(r^2+2Y_c^2\right)+2p_1{X}_c{Y}_c=Y_c\left(k_1=0,k_2=0,k_3=0,p_1=0,p_2=0\right)& \left(23\right)\\ Z_{\mathrm{camera}}=Z_c& \left(24\right)\end{array}$

(31) Under the above condition, the calibration is processed. As shown in FIG. 1(a), a ring-shaped calibration jig 1 is photographed by a camera 1. As shown an alternate long and short dash line in FIG. 1(b), a contour 1a of a circumference of a circle is extracted from a photographed image. Judging from the contour 1a of an extracted circumference, a center position 1b of the calibration jig 1 is calculated.

(32) Further, focus distance: (fk.sub.x, fk.sub.y) is already known based on the specification of a lens and the camera. 2) As shown in FIG. 15, an optical axis of the camera 2 is aligned on the Z-axis of the machine coordination and a horizontal direction of the camera 2 is arranged on the X-axis of the machine coordination to mount the camera 2. A distance between the camera and the calibration jig 1 is substantially constant and then the parameter: tz is recognized as known. Thus, x=0, y=0, and z=0.

(33) $\begin{array}{cc}\left[\mathrm{No}.10\right]\mathrm{Exterior}\mathrm{paramater}\left(\mathrm{rotation}\right)\text{:}& \\ \begin{array}{c}\stackrel{.fwdarw.}{R}=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]\\ =\left[\begin{array}{ccc}\cos z& -\sin z& 0\\ \sin z& \cos z& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}\cos y& 0& \sin y\\ 0& 1& 0\\ -\sin y& 0& \cos y\end{array}\right]\\ \left[\begin{array}{ccc}1& 0& 0\\ 0& \cos x& -\sin x\\ 0& \sin x& \cos x\end{array}\right]\\ =\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]\\ =\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]\end{array}& \left(25\right)\\ \left[\mathrm{No}.11\right]& \\ \begin{array}{c}S\left[\begin{array}{c}{U}_{\mathrm{image}}\\ V_{\mathrm{image}}\\ 1\end{array}\right]=\left[\begin{array}{cccc}f{k}_x& 0& 0& 0\\ 0& f{k}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{camera}}\\ Y_{\mathrm{camera}}\\ Z_{\mathrm{camera}}\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}f{k}_x& 0& 0& 0\\ 0& f{k}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_x\\ r_{21}& r_{22}& r_{23}& t_y\\ r_{31}& r_{32}& r_{33}& t_z\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{world}}\\ Y_{\mathrm{world}}\\ Z_{\mathrm{world}}\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}f{k}_x& 0& 0& 0\\ 0& f{k}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cccc}1& 0& 0& t_x\\ 0& 1& 0& t_y\\ 0& 0& 1& t_z\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{world}}\\ Y_{\mathrm{world}}\\ Z_{\mathrm{world}}\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}f{k}_x& 0& 0& f{k}_x{t}_x\\ 0& f{k}_y& 0& f{k}_y{t}_y\\ 0& 0& 1& t_z\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{world}}\\ Y_{\mathrm{world}}\\ Z_{\mathrm{world}}\\ 1\end{array}\right]\end{array}& \left(26\right)\end{array}$

(34) Accordingly, determined parameters are

(35) $\begin{array}{cc}\left[\mathrm{No}.12\right]& \\ \begin{array}{c}\mathrm{Exterior}\mathrm{parameter}\left(\mathrm{translation}\right)\text{:}\\ \stackrel{.fwdarw.}{T}=\left[\begin{array}{c}{t}_x\\ t_y\\ t_z\end{array}\right]\end{array}& \end{array}$

(36) Further, the parameter tz can be recognized as known. Therefore, two parameters (tx, ty) are determined. 3) By setting the center position (X.sub.r, Y.sub.r, Z.sub.r) of the calibration ring already known as a center position, the parameters (tx, ty) are calculated so as to match with a coordination (U.sub.ring, V.sub.ring) of the center position 1b in a plane of an image of the calibration jig 1 photographed by the camera 2.

(37) $\begin{array}{cc}\left[\mathrm{No}.13\right]& \\ S\left[\begin{array}{c}{U}_{\mathrm{image}}\\ V_{\mathrm{image}}\\ 1\end{array}\right]=\left[\begin{array}{cccc}f{k}_x& 0& 0& f{k}_x{t}_x\\ 0& f{k}_y& 0& f{k}_y{t}_y\\ 0& 0& 1& t_z\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{world}}\\ Y_{\mathrm{world}}\\ Z_{\mathrm{world}}\\ 1\end{array}\right]S\left[\begin{array}{c}{U}_{\mathrm{Ring}}\\ V_{\mathrm{Ring}}\\ 1\end{array}\right]=\left[\begin{array}{cccc}f{k}_x& 0& 0& f{k}_x{t}_x\\ 0& f{k}_y& 0& f{k}_y{t}_y\\ 0& 0& 1& t_z\end{array}\right]\left[\begin{array}{c}\mathrm{Xr}\\ \mathrm{Yr}\\ \mathrm{Zr}\\ 1\end{array}\right]& \left(27\right)\end{array}$

Embodiment 2

(38) 1) The present invention is exclusively used to measure a shape of a hole, interior parameters in a camera (focus distance: (fk.sub.x, fk.sub.y), displacement of a main point: (cx, cy), distortion correction coefficient in a radial direction: (k.sub.1, k.sub.2, k.sub.3), distortion correction coefficient in a tangential direction: (p.sub.1, p.sub.2)) are specifically set as follows: (1) Regarding a camera and a lens within the camera, a distortion in a tangential direction is little so that distortion correction coefficient in a tangential direction: (p1, p2) can be ignored and the all set to zero. (2) A purpose of the measurement according to the present invention is to measure a shape of a hole wherein a measured shape of the hole is aligned at a central portion of an image. Therefore, an influence of the distortion in the radial direction is not occurred. Therefore, in the system according to the present invention, there is no problem even if the all distortion correction coefficients in the radial direction: (k1, k2, k3) are set to zero. (3) The displacement of the main point: (c.sub.x, c.sub.y) is included in the exterior parameter of the camera (translation distance: (tx, ty, tz)) so that the displacement of the main point can be set as zero. (4) The focus distance: (fk.sub.x, fk.sub.y) depends on a specification of the lens and the camera. Therefore, necessary values are known in a specification listed by those manufactures.

(39) Based on the above situation,

(40) 0$\begin{array}{cc}\left[\mathrm{No}.14\right]& \\ \begin{array}{c}S\left[\begin{array}{c}{U}_{\mathrm{image}}\\ V_{\mathrm{image}}\\ 1\end{array}\right]=\left[\begin{array}{ccc}{k}_x& 0& 0\\ 0& k_y& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{camera}}\\ Y_{\mathrm{camera}}\\ Z_{\mathrm{camera}}\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}f{k}_x& 0& 0& 0\\ 0& f{k}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{camera}}\\ Y_{\mathrm{camera}}\\ Z_{\mathrm{camera}}\\ 1\end{array}\right]\end{array}\mathrm{And},& \left(31\right)\\ \left[\mathrm{No}.15\right]& \\ X_{\mathrm{camera}}=\left(1+k_1{r}^2+k_2{r}^4+k_3{r}^6\right)X_c+p_1\left(r^2+2X_c^2\right)+2p_2{X}_c{Y}_c=X_c\left(k_1=0,k_2=0,k_3=0,p_1=0,p_2=0\right)& \left(32\right)\\ Y_{\mathrm{camera}}=\left(1+k_1{r}^2+k_2{r}^4+k_3{r}^6\right)Y_c+p_2\left(r^2+2Y_c^2\right)+2p_1{X}_c{Y}_c=Y_c\left(k_1=0,k_2=0,k_3=0,p_1=0,p_2=0\right)& \left(33\right)\\ Z_{\mathrm{camera}}=Z_c& \left(34\right)\end{array}$

(41) Under the above condition, the calibration is processed. As shown in FIG. 1(a), a ring-shaped calibration jig 1 is photographed by a camera 1. As shown an alternate long and short dash line in FIG. 1(b), a contour 1a of a circumference of a circle is extracted from a photographed image. Judging from the contour 1a of an extracted circumference, a center position 1b of the calibration jig 1 is calculated.

(42) Further, focus distance: (fk.sub.x, fk.sub.y) is already known based on the specification of a lens and the camera. 2) As shown in FIG. 15, an optical axis of the camera 2 is closely aligned on the Z-axis of the machine coordination (it is unnecessary to match the optical axis with the Z-axis completely) so as to set the camera 2. A horizontal direction of the camera 2 is closely aligned on the X-axis of the machine coordination (x0, y0, z0). Accordingly, determined parameters are six exterior parameters (x, y, z, tx, ty, tz).

(43) $\begin{array}{cc}\left[\mathrm{No}.16\right]& \\ \begin{array}{c}S\left[\begin{array}{c}{U}_{\mathrm{image}}\\ V_{\mathrm{image}}\\ 1\end{array}\right]=\left[\begin{array}{cccc}f{k}_x& 0& 0& 0\\ 0& f{k}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{camera}}\\ Y_{\mathrm{camera}}\\ Z_{\mathrm{camera}}\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}f{k}_x& 0& 0& 0\\ 0& f{k}_y& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_x\\ r_{21}& r_{22}& r_{23}& t_y\\ r_{31}& r_{32}& r_{33}& t_z\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{world}}\\ Y_{\mathrm{world}}\\ Z_{\mathrm{world}}\\ 1\end{array}\right]\end{array}& \left(35\right)\\ \left[\mathrm{No}.17\right]\mathrm{Exterior}\mathrm{paramter}\left(\mathrm{rotation}\right)\text{:}& \\ \begin{array}{c}\stackrel{.fwdarw.}{R}=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]\\ =\left[\begin{array}{ccc}\cos z& -\sin z& 0\\ \sin z& \cos z& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}\cos y& 0& \sin y\\ 0& 1& 0\\ -\sin y& 0& \cos y\end{array}\right]\\ \left[\begin{array}{ccc}1& 0& 0\\ 0& \cos x& -\sin x\\ 0& \sin x& \cos x\end{array}\right]\end{array}& \\ \mathrm{Exterior}\mathrm{parameter}\left(\mathrm{translation}\right)\text{:}\stackrel{.fwdarw.}{T}=\left[\begin{array}{c}{t}_x\\ t_y\\ t_z\end{array}\right]& \end{array}$ 3) By setting the center position (X.sub.r, Y.sub.r, Z.sub.r) of the calibration ring already known as a center position, the parameters (x, y, z, tx, ty, tz) are calculated.

(44) In a specific case as shown in FIG. 15, several images are photographed by the calibration jig 1 while a camera 2 is moved parallel to the X-axis, Y-axis and the Z-axis while. In each photographed image, a contour 1a of a circumference of a circle is extracted. Based on the contour 1a of an extracted circumferential of the circle, a center position 1b of the calibration jig 1 is calculated.

(45) A center position of the calibration ring is already known so that the parameters (x, y, z, tx, ty, tz) can be determined by the each center position 1b of the calibration jig 1 calculated respectively in the least square method.

Embodiment 3

(46) FIG. 16 shows the third embodiment of the present invention. FIG. 16 is a flow chart for showing a calibration method according to the present invention.

(47) As shown in FIG. 1(a), a camera 2 is positioned near a center position of a calibration jig 1 (STEP S1). The calibration jig 1 is previously set on a workpiece on a machining tool, for instance a boom 3 of a hydro pressure shovel as shown in FIG. 3. As shown in FIG. 15, a relation between the coordination of the camera 2 and a machine coordination may be specifically related (Embodiment 1) or may be unrelated (Embodiment 2).

(48) In the next, an image of the calibration jig 1 is photographed by a camera 2 (STEP S2).

(49) Then, as shown in FIG. 1(b), a contour 1a of a circumferential of a circle setting a ring-shaped calibration jig at a workpiece on a machining tool is extracted by processing an image photographed by the camera (STEP S3).

(50) Then, as shown in FIG. 1(b), a center position 1b of the calibration jig 1 is calculated judging from the contour 1a of the circumferential of the circle (STEP S4).

(51) Concerning with calibration parameters excluding the interior parameter of the camera, position calibration parameters (the exterior parameters) of the camera is detected judging from the center position (Xr, Yr, Zr) of the calibration jig 1 already known as the position of the machine coordination and the center position 1b of the calibration jig calculated in the STEP S4 (STEP S5).

(52) In the case that a relation between the coordination system of the camera 2 and the machine coordination system is a specifically related, x=0. y=0 and z=0. The detected exterior parameters are only translation distance. In the case that the parameter: tz is already known, the parameters (tx, ty) can be determined based on one image.

(53) On the other hand, in the case that a relation between the coordination system of the camera 2 and the machine coordination system is unrelated, detected parameters are six exterior parameters (x, y, z, tx, ty, tz). Therefore, several images of the calibration jig 1 are photographed so as to determine each exterior parameter, respectively.

INDUSTRIAL USE OF THE INVENTION

(54) The present invention can be broadly utilized as a calibration method for a system for measuring a shape of a hole (a center position of a hole) in a tool machine industry.

EXPLANATION OF NUMERAL

(55) 1 calibration jig 1a contour of circumference of circle 1b center position of calibration jig 2 camera 3 boom of hydro pressure shovel 3a cylindrical portion 10 sensor 11, 12 calibration board 20 lens 30 camera