CALIBRATION METHOD FOR MULTI-DEGREE-OF-FREEDOM MOVABLE VISION SYSTEM

Abstract

Disclosed is a calibration method for a multi-degree-of-freedom movable vision system. A calibration template is placed in front of a camera component, each degree of freedom of movement of the camera component is rotated, several images including features of the calibration template are recorded using the camera component, position information of each degree of freedom of movement when the corresponding images are acquired is also recorded, and calibration results of the camera component and each degree of freedom of movement are calculated using a calculation component. The movable vision system comprises the camera component, the calculation component and a control component. The method has good calculation real-time performance, parameter information of position changes of the camera component can be acquired in real time by means of the calculation of the position information of each degree of freedom of movement, the problem of errors, caused by machining and assembly, with a theoretical design can be solved, and the method has broad application prospects.

Claims

1. A calibration method of a multi-degree-of-freedom movable vision system, wherein the multi-degree-of-freedom movable vision system comprises: an imaging component comprising at least one imaging element capable of capturing consecutive images, the imaging component having at least one motional degree of freedom and mounted on a base, wherein each motional degree of freedom is provided with a position acquisition device capable of acquiring rotation or translation information; a calculation component capable of calculating and processing information of images and information of movement in each motional degree of freedom; and a control component capable of controlling movement in each motional degree of freedom, and wherein the calibration method comprises: placing a calibration template in front of the imaging element; driving the imaging element to move in each motional degree of freedom; capturing a number of images containing features of the calibration template by the imaging element and simultaneously recording position information of each motional degree of freedom when capturing corresponding images; and calculating calibration results for the imaging element and each motional degree of freedom by the calculation component.

2. The calibration method of a multi-degree-of-freedom movable vision system according to claim 1, comprising following steps: (1) recording position information of each motional degree of freedom at a reference position; (2) placing the calibration template in front of the imaging component, and for each motional degree of freedom, causing several times of changes in a motional degree of freedom, and capturing corresponding images and collecting corresponding position information of the motional degree of freedom; and (3) calculating calibration results for each motional degree of freedom using a calibration algorithm, based on the captured images and the collected position information of each motional degree of freedom.

3. The calibration method of a multi-degree-of-freedom movable vision system according to claim 2, wherein the calibration template includes one of a natural static scene and an artificial standard target, wherein the natural static scene when serving as the calibration template provides invariant features and known information of position relationship between the features, and wherein the artificial standard target includes at least one of a two-dimensional (2D) planar target and a three-dimensional (3D) stereoscopic target.

4. The calibration method of a multi-degree-of-freedom movable vision system according to claim 2, wherein the calibration results comprise rotation and translation matrices of a rotation axis in each motional degree of freedom relative to an imaging element coordinate system.

5. The calibration method of a multi-degree-of-freedom movable vision system according to claim 2, wherein a relative positional relationship between a current position of the imaging element and the reference position is calculated from the calibration results and a current position information of the imaging element in each motional degree of freedom and an established relationship model between a rotation axis coordinate system for each motional degree of freedom and the imaging element coordinate system, and wherein the relative positional relationship is expressed by rotation and translation matrices.

6. The calibration method of a multi-degree-of-freedom movable vision system according to claim 5, wherein the relationship model is $W_a={\left[\begin{array}{cc}{R}_{\mathrm{BCa}}& T_{\mathrm{BCa}}\\ O& 1\end{array}\right]}^{-1}\left[\begin{array}{cc}{R}_{\mathrm{pa}}& 0\\ O& 1\end{array}\right]\left[\begin{array}{cc}{R}_{\mathrm{BCa}}& T_{\mathrm{BCa}}\\ O& 1\end{array}\right],a=1\mathrm{.Math.}N\left[\begin{array}{cc}{R}^{\prime }& T^{\prime }\\ O& 1\end{array}\right]=W_1{W}_2\mathrm{.Math.}{W}_N,$ where R′ and T′ are rotation and translation matrices of the current position of the imaging element relative to the reference position, respectively, R.sub.BCa,T.sub.BCa are rotation and translation matrices of a rotation axis in an a-th motional degree of freedom relative to the imaging element coordinate system, N is a number of motional degree(s) of freedom of the movable vision system, and R.sub.pa is a rotation matrix converted from a rotation angle of a rotation axis in the a-th motional degree of freedom relative to the reference position.

7. The calibration method of a multi-degree-of-freedom movable vision system according to claim 1, wherein the calibration template includes one of a natural static scene and an artificial standard target, wherein the natural static scene when serving as the calibration template provides invariant features and known information of position relationship between the features, and wherein the artificial standard target includes at least one of a two-dimensional (2D) planar target and a three-dimensional (3D) stereoscopic target.

8. The calibration method of a multi-degree-of-freedom movable vision system according to claim 1, wherein the calibration results comprise rotation and translation matrices of a rotation axis in each motional degree of freedom relative to an imaging element coordinate system.

9. The calibration method of a multi-degree-of-freedom movable vision system according to claim 1, wherein a relative positional relationship between a current position of the imaging element and the reference position is calculated from the calibration results and a current position information of the imaging element in each motional degree of freedom and an established relationship model between a rotation axis coordinate system for each motional degree of freedom and the imaging element coordinate system, and wherein the relative positional relationship is expressed by rotation and translation matrices.

10. The calibration method of a multi-degree-of-freedom movable vision system according to claim 9, wherein the relationship model is $W_a={\left[\begin{array}{cc}{R}_{\mathrm{BCa}}& T_{\mathrm{BCa}}\\ O& 1\end{array}\right]}^{-1}\left[\begin{array}{cc}{R}_{\mathrm{pa}}& 0\\ O& 1\end{array}\right]\left[\begin{array}{cc}{R}_{\mathrm{BCa}}& T_{\mathrm{BCa}}\\ O& 1\end{array}\right],a=1\mathrm{.Math.}N\left[\begin{array}{cc}{R}^{\prime }& T^{\prime }\\ O& 1\end{array}\right]=W_1{W}_2\mathrm{.Math.}{W}_N,$ where R′ and T′ are rotation and translation matrices of the current position of the imaging element relative to the reference position, respectively, R.sub.Bca, T.sub.Bca are rotation and translation matrices of a rotation axis in an a-th motional degree of freedom relative to the imaging element coordinate system, N is a number of motional degree(s) of freedom of the movable vision system, and R.sub.pa is a rotation matrix converted from a rotation angle of a rotation axis in the a-th motional degree of freedom relative to the reference position.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0020] FIG. 1 is a schematic block diagram of a multi-degree-of-freedom (DOF) movable vision system of the present application.

[0021] FIG. 2A is a calibration flowchart of a multi-degree-of-freedom (DOF) movable vision system of the present application.

[0022] FIG. 2B is a flowchart for real-time calculation of extrinsic parameters of a multi-degree-of-freedom (DOF) movable vision system according to the present application.

[0023] FIG. 3 shows a calibration template of a checkerboard type used in a calibration algorithm of a particular embodiment of the present application.

[0024] FIG. 4 schematically illustrates the camera coordinate system and the rotation axis coordinate system in each motional degree of freedom according to the present application.

DETAILED DESCRIPTION

[0025] The present application will be described in greater detail blow with reference to particular embodiments and the accompanying drawings. For the sake of brevity, when describing the various processes, conditions and experimental methods of those embodiments, some details well known in the art are omitted, and the present application is not particularly limited to such details. A more detailed description of the calibration method for a multi-degree-of-freedom movable vision system according to the present application will be set forth below with reference to FIGS. 1 to 4. It should be noted that the calibration method can be applied to a multi-degree-of-freedom movable vision systems with any number of motional degrees of freedom, and the following embodiments are provided solely for the purpose of exemplification and are not intended to limit the present application in any sense.

Embodiment 1

[0026] In Embodiment 1 of the present application, a movable vision system with three degrees of freedom will be described in detail as an example.

[0027] As shown in FIG. 1, the multi-degree-of-freedom (DOF) movable vision system according to the present application includes an imaging component, a calculation component and a control component. The calculation component may be coupled to the imaging component by a signal connection and configured to calibrate influences of movements in each degree of freedom on the imaging component based on images captured by the imaging component in various poses and corresponding pose information. The control component may be coupled to both the imaging component and the calculation component by signal connections and configured to control the imaging component to capture image at various poses and control the calculation component to perform calibration calculation.

[0028] As shown in FIG. 2A, the calibration method of a multi-degree-of-freedom movable vision system in present application mainly includes the flowing calibration steps:

[0029] (Step S11) recording position information of each motional degree of freedom at an initial position to determine a reference position;

[0030] (Step S12) driving each motional degree of freedom successively and capturing images by the imaging component for each motional degree of freedom; and recording position information of each motional degree of freedom at the time of capturing the corresponding images; and

[0031] (Step S13) calculating calibration results by the computing component.

[0032] Once achieving the calibration, a pose of the imaging component relative to the initial position can be precisely calculated based on both the calibration results and position information of each motional degree of freedom. As shown in FIG. 2B, the above calculation process in practical use mainly includes the steps of:

[0033] (Step S21) obtaining information on the calibration results of the multi-degree-of-freedom movable vision system;

[0034] (Step S22) obtaining information on current position of each motional degree of freedom; and

[0035] (Step S23) calculating information on current rotation and translation of the imaging component relative to the recorded reference position by substituting the information obtained in Steps S21 and S22 into a model for solving for extrinsic parameters of the movable vision system.

[0036] Take the movable vision system having three degrees of freedom as an example. The system has three motional degrees of freedom, which are defined respectively as pitch, yaw and roll motional degrees of freedom. In a three-dimensional rectangular coordinate system, pitch refers to rotation about X axis, yaw referring to rotation about Y axis, roll referring to rotation about Z axis. In this example, movement in the three degrees of freedom may be enabled by three motion elements each for a respective one of the degrees of freedom. In other examples, movement in the three degrees of freedom may also be enabled by one motion element for one of the degrees of freedom and another motion element for the other two degrees of freedom, or by a single motion element for all of the three degrees of freedom. Preferably, the three motion elements are rigidly connected to one another so that movements in one of the degrees of freedom do not affect movements in the other two degrees of freedom. Preferably, the motion element for the pitch degree of freedom may be fixed to a base via a rigid connection, and one or more imaging elements capable of capturing consecutive images may be attached to the motion element for the roll degree of freedom. Movements in above three degrees of freedom are driven by respective motors, each provided with a position recorder for obtaining real-time output position information from the motor. In this example, the output position information from each motor is expressed in the form of an angular value. In particular, the output position information from the motor for driving movements in the roll degree of freedom (also referred to as the “first degree of freedom” hereinafter) is expressed as θ1, indicating the rotation angle of the spindle of the imaging element relative to the Z axis. The output position information from the motor for driving movement in the yaw degree of freedom (also referred to as the “second degree of freedom” hereinafter) is expressed as θ2, indicating the rotation angle of the spindle of the imaging element relative to the Y axis. The output position information from the motor for driving movement in the pitch degree of freedom (also referred to as the “third degree of freedom” hereinafter) is expressed as θ3, indicating the rotation angle of the spindle of the imaging element relative to the X axis.

[0037] The calibration method includes the following steps.

[0038] 1. Acquisition of Reference Position Information

[0039] With a calibration template held in front of the imaging in the movable vision system in such a manner that the camera is able to capture complete images of the calibration template, initial position information (θ.sub.I1, θ.sub.I2, θ.sub.I3) of the three motors is recorded at this time as the reference position information.

[0040] 2. Calibration of Motion Shafts

[0041] Each motional degree of freedom is driven successively with the calibration template being kept stationary. That is, when one of the degrees of freedom changes, the other two degrees of freedom do not change. Taking the first (roll) degree of freedom as an example, for each time of rotation of the motion axis (corresponding to the i-th position), Q1 valid images M.sub.1ij (j=1, 2, . . . , Q1) are captured by the camera at the corresponding position, and current position information of the motion axis is recorded and get {M.sub.1ij, θ.sub.11i, θ.sub.12i, θ.sub.12i} (i=1, 2, . . . , P1), where P1 represents the number of rotation of the motion axis in the calibration process of the first degree of freedom. A calculation is then performed based on all sets of images captured at respective positions using a calibration algorithm (e.g., Zhang Zhengyou's calibration algorithm) of the planar checkerboard calibration template shown in FIG. 3, to determine rotation and translation results {R.sub.1Ci, T.sub.1Ci} of the images captured at respective rotation positions relative to the calibration template. Intermediate calibration results {R.sub.1Ci, T.sub.1Ci, θ.sub.11i, θ.sub.12i, θ.sub.13i} (i=1 . . . P.sub.1) for the first degree of freedom are then derived from both the above rotation and translation results and the position information of the motion axis recorded at the time of capturing the images. The above process is repeated to yield intermediate calibration results for the second (yaw) and third (pitch) degrees of freedom, and all the intermediate calibration results are integrated into {R.sub.aCi, T.sub.aCi, θ.sub.a1i, θ.sub.a2i, θ.sub.a3i} (i=1 . . . P.sub.a, a=1, 2, 3).

[0042] Limited by the mechanical machining techniques, it is difficult to either ensure that the optical center of the movable vision system is positioned on each rotation axis or ensure that each rotation axis is parallel to a corresponding coordinate axis of the camera coordinate system. Therefore, it is necessary to calculate pose variation of the camera coordinate system based on outputs of the rotation axis encoders. For a rigid body, relative positional relationship between the camera spindle and rotation axes of respective degrees of freedom remains unchanged. In order to determine the positional relationship between each rotation axis and the camera coordinate system, a mathematical model is created as shown in FIG. 4. The camera coordinate system (also called the “system C” hereinafter for short) is denoted as C{O.sub.c−x.sub.cy.sub.cz.sub.c}. The vertical line starting from the optical center O.sub.c of the camera coordinate system is drawn and intersects the rotation axis 1 of the roll degree of freedom at point O.sub.b. The perpendicular line extends to get a ray O.sub.bx.sub.b. The rotation axis coordinate system B {O.sub.b−x.sub.by.sub.bz.sub.b} (also called the “system B” hereinafter for short) is established, with O.sub.b as an origin, the ray O.sub.bx.sub.b as an x axis, the rotation axis as a y axis, and a z axis determined according to the right-hand rule.

[0043] If O.sub.cO.sub.b=d (which is a constant determined by the mechanical machining process), then coordinate of O.sub.c in the system B can be represented as t=(−d, 0,0).sup.T. Let R.sub.BC be a rotation matrix from the camera coordinate system C to the rotation axis coordinate system B. For any point P in space, its coordinate PC in the system C and coordinate PB in the system B satisfies the transformation equation P.sub.B=R.sub.BCP.sub.C+t, expressed in homogeneous coordinates as

[00003]$\begin{array}{cc}\left[\begin{array}{c}{x}_B\\ y_B\\ z_B\\ 1\end{array}\right]=\left[\begin{array}{cc}{R}_{BC}& t\\ O& 1\end{array}\right]\left[\begin{array}{c}{x}_C\\ y_C\\ z_C\\ 1\end{array}\right],\mathrm{where}{T}_{\mathrm{BC}}=\left[\begin{array}{cc}{R}_{BC}& t\\ O& 1\end{array}\right]& \left(1\right)\end{array}$

is the desired calibration result, which is a transformation matrix from the system B to C for any point P in space.

[0044] After a rotation about the rotation axis by an angle θ, the rotation axis coordinate system B and the camera coordinate system C become new rotation axis coordinate system B′ and new camera coordinate system C′. The rotation about the rotation axis is equivalent to a corresponding rotation of the system B to B′ about the Z.sub.b axis by the angle θ. For the same point P, its coordinate P.sub.B in the system B and coordinate P.sub.B′ in system B′ sastisfy:

[00004]$\begin{array}{cc}{P}_{B^{\prime }}=\left[\begin{array}{cc}{R}_{B^{\prime }B}& O\\ O& 1\end{array}\right]P_B,\mathrm{where}{R}_{B^{\prime }B}=\left[\begin{array}{ccc}\cos \theta & 0& \sin \theta \\ 0& 1& 0\\ -s\mathrm{in}\theta & 0& \cos \theta \end{array}\right]& \left(2\right)\end{array}$

(this value depends on the angle of rotation).

[0045] Similarly, the camera coordinate system rotates by the angle θ into the new camera coordinate system C′. During calibration, the transformation of the camera coordinate system can be calculated using a fixed checkerboard. Assuming a point P is represented by a coordinate x.sub.w in a world coordinate system of the checkerboard, calculated extrinsic parameters of the checkerboard in the systems C and C′ will be T.sub.CW and T.sub.C′W, respectively. Since P.sub.C=T.sub.CWx.sub.w and P.sub.C′=T.sub.C′Wx.sub.w, the following equation can be get:

P.sub.C′=T.sub.C′WT.sub.CW.sup.−1P.sub.C=T.sub.C′CP.sub.C   (3)

[0046] Because of the nature of a rigid body, the positional relationship T.sub.BC between the systems B and C remains unchanged in the rotating process. Thus, the same point in the space still satisfies Eqn. (1) in the new coordinate systems B′ and C′, and the following equation is get.

[00005]$\begin{array}{cc}{P}_{B^{\prime }}=\left[\begin{array}{cc}{R}_{\mathrm{BC}}& t\\ O& 1\end{array}\right]P_C& \left(4\right)\end{array}$

[0047] From Eqns. (2), (3) and (4), the following equation is get.

[00006]$\begin{array}{cc}{\left[\begin{array}{cc}{R}_{\mathrm{BC}}& t\\ O& 1\end{array}\right]}^{-1}\left[\begin{array}{cc}{R}_{B^{\prime }B}& O\\ O& 1\end{array}\right]\left[\begin{array}{cc}{R}_{\mathrm{BC}}& t\\ O& 1\end{array}\right]=\left[\begin{array}{cc}{R}_{C^{\prime }C}& t_{C^{\prime }C}\\ O& 1\end{array}\right]\mathrm{where}\left[\begin{array}{cc}{R}_{\mathrm{BC}}& t\\ O& 1\end{array}\right]& \left(5\right)\end{array}$

is the desired calibration result for the rotation axis,

[00007]$\hspace{1em}\left[\begin{array}{cc}{R}_{B^{\prime }B}& O\\ O& 1\end{array}\right]$

is a matrix output from position sensors for each rotation, and

[00008]$\hspace{1em}\left[\begin{array}{cc}{R}_{C^{\prime }C}& t_{C^{\prime }C}\\ O& 1\end{array}\right]$

is a matrix T.sub.C′C calculated by the camera for each rotation. T.sub.BC is calculated for each rotation to calibrate relationship between the rotation axis coordinate system and the camera coordinate system.

[0048] According to the present application, for each degree of freedom, an individual calibration process is performed. For a given set of data {R.sub.aCi, T.sub.aCi, θ.sub.a1i, θ.sub.a2i, θ.sub.a3i} (i=1 . . . P.sub.a, a=1 . . . 3), the motors' rotation angles relative to the reference position can be calculated as {θ.sub.a1i-θ.sub.I1, θ.sub.a2i-θ.sub.I2, θ.sub.a3i-θ.sub.I3} (i=1 . . . P.sub.a, a=1 . . . 3) based on this set of data, and the rotation matrices can be calculated as

[00009]$\begin{array}{cc}\{\begin{array}{c}{R}_{B^{\prime }B1i}=\left[\begin{array}{ccc}\cos \left({\theta }_{a1i}-{\theta }_{l1}\right)& -\sin \left({\theta }_{a1i}-{\theta }_{l1}\right)& 0\\ -\sin \left({\theta }_{a1i}-{\theta }_{l1}\right)& \cos \left({\theta }_{a1i}-{\theta }_{l1}\right)& 0\\ 0& 0& 1\end{array}\right]\\ R_{B^{\prime }B2i}=\left[\begin{array}{ccc}1& 0& 0\\ 0& \cos \left({\theta }_{a2i}-{\theta }_{l2}\right)& -\sin \left({\theta }_{a2i}-{\theta }_{l2}\right)\\ 0& \sin \left({\theta }_{a2i}-{\theta }_{l2}\right)& \cos \left({\theta }_{a2i}-{\theta }_{l2}\right)\end{array}\right]\\ R_{B^{\prime }B3i}=\left[\begin{array}{ccc}\cos \left({\theta }_{a3i}-{\theta }_{l3}\right)& 0& \sin \left({\theta }_{a3i}-{\theta }_{l3}\right)\\ 0& 1& 0\\ -\sin \left({\theta }_{a3i}-{\theta }_{l3}\right)& 0& \cos \left({\theta }_{a3i}-{\theta }_{l3}\right)\end{array}\right]\end{array},i=1\mathrm{.Math.}{P}_a,a=1\mathrm{.Math.}3& \left(6\right)\end{array}$

[0049] When substituting this into Eqn. (5), the following set of equations can be get:

[00010]$\begin{array}{cc}{W}_{\mathrm{ai}}={\left[\begin{array}{cc}{R}_{\mathrm{BCa}}& T_{\mathrm{BCa}}\\ O& 1\end{array}\right]}^{-1}\left[\begin{array}{cc}{R}_{B^{\prime }\mathrm{Bai}}& 0\\ O& 1\end{array}\right]\left[\begin{array}{cc}{R}_{\mathrm{BCa}}& T_{\mathrm{BCa}}\\ O& 1\end{array}\right],a=1\mathrm{.Math.}3,i=1\mathrm{.Math.}P{W}_{1i}{W}_{2i}{W}_{3i}=\left[\begin{array}{cc}{R}_a& T_a\\ O& 1\end{array}\right],i=1\mathrm{.Math.}{P}_a.& \left(7\right)\end{array}$

[0050] P sets of equations can be obtained when all the data is substituted into Eqn. (7), and an optimal solution {R.sub.BCa, T.sub.BCa} (a=1 . . . 3) can be obtained by solving those equations.

[0051] 3. Calculation of Real-Time Calibration Results

[0052] The overall calibration results include the reference position information (θ.sub.I1, θ.sub.I2, θ.sub.I3) and the motion axis calibration results {R.sub.BCa, T.sub.BCa} (a=1 . . . 3). Position information (θ.sub.p1, θ.sub.p2, θ.sub.p3) of the three motional degrees of freedom is obtained after the imaging component of the movable vision system moves. Rotation angles of the three degrees of freedom are obtained as (θ.sub.p1-θ.sub.I1, θ.sub.p2-θ.sub.I2, θ.sub.p3-θ.sub.I3), which are then converted into rotation matrices (R.sub.p1, R.sub.p2, R.sub.p3,) and fed into the model. As a result, post-movement extrinsic parameters of the movable vision system relative to the reference camera position are obtained as

[00011]$W_a={\left[\begin{array}{cc}{R}_{\mathrm{BCa}}& T_{\mathrm{BCa}}\\ O& 1\end{array}\right]}^{-1}\left[\begin{array}{cc}{R}_{\mathrm{pa}}& 0\\ O& 1\end{array}\right]\left[\begin{array}{cc}{R}_{\mathrm{BCa}}& T_{\mathrm{BCa}}\\ O& 1\end{array}\right],a=1\mathrm{.Math.}3\left[\begin{array}{cc}{R}^{\prime }& T^{\prime }\\ O& 1\end{array}\right]=W_1{W}_2{W}_3.$

[0053] The obtained R′ and T′ represent rotation and transition matrices of the camera relative to the reference position after the three-degree-of-freedom movable vision system has moved.

[0054] The calibration method of a movable vision system proposed in the present application is not limited to the foregoing embodiments, and various variations and modifications can be made without departing from the principles of the present application. The scope of the present application is not limited to the embodiments disclosed hereinabove. Rather, it embraces any and all changes or advantages that can be devised by those skilled in the art without departing from the spirit and scope of conception of this application. Thus, the protection scope of present application is defined by the appended claims.