Binocular-vision-based method for tracking fruit space attitude and fruit space motion

Abstract

A binocular-vision-based method for tracking fruit space attitude and fruit space motion, the method comprising: establishing a connected base coordinate system by taking a junction of a fruit and a fruit stem as an origin; statically photographing a feature point on the surface of the fruit and a point of the connected base coordinate system established at the junction of the fruit and the fruit stem; storing a photographed image; acquiring an inherent relationship between the feature point and the connected base coordinate system; photographing dynamic motion of the fruit; acquiring absolute coordinates of the feature point on the surface of the fruit; calculating, according to the inherent relationship between the feature point and the connected base coordinate system, absolute coordinates of a point of the connected base coordinate system at each moment corresponding to each frame of image; and respectively calculating the displacement, instantaneous speed and instantaneous acceleration of the fruit, calculating swing angular displacement and swing angular acceleration of the fruit, and calculating a fruit torsion angular speed and a fruit torsion angular acceleration at the moment t. The study of a fruit motion state in the field of forest fruit harvest through vibration is performed, so that the motion of fruits can be better tracked.

Claims

1. A computer implemented method for tracking fruit space attitude and fruit space motion, wherein the computer is configured to carry out the steps comprising: (1) marking three feature points C.sub.1, C.sub.2, C.sub.3 on a surface of a fruit; (2) establishing a connected base coordinate system by taking a junction of the fruit and a fruit stem as an origin, a X axis, a Y axis, and a Z axis of the connected base coordinate system are marked as a X connected-axis, a Y connected-axis and a Z connected-axis, respectively, and the origin of the connected base coordinate system is marked as O.sub.1, an end point of the unit vector in a positive direction of the X connected-axis is marked as X.sub.1, an end point of the unit vector in a positive direction of the Y connected-axis is marked as Y.sub.1, an end point of the unit vector in a positive direction of the Z connected-axis is marked as Z.sub.1; (3) establishing a public reference base coordinate system; (4) statically photographing the three feature points on the surface of the fruit and connected base coordinate system points c established at the junction of the fruit and the fruit stem, using software to store the photographed images, and using software to process the feature points and connected base coordinate system points O.sub.1, X.sub.1, Y.sub.1, Z.sub.1 in the images, acquiring a absolute coordinate of the feature points C.sub.1, C.sub.2, C.sub.3 and the connected base coordinate system points O.sub.1, X.sub.1, Y.sub.1, Z.sub.1, and then acquiring an inherit relationship between the feature points and the connected base coordinate system; (5) photographing the fruit dynamic motion, using software to store each frame of the image in the photographed video, using software to process the feature points in the images to acquiring the absolute coordinate of the feature points on the fruit surface, according to an inherent relationship between the feature points and the connected base coordinate system, the absolute coordinate of each frame of image corresponding to the connected base coordinate system points O.sub.1, X.sub.1, Y.sub.1, Z.sub.1 at each moment being calculated by inverse rotation transformation, and calculating an angle between the vector {right arrow over (O.sub.1Z.sub.1)} at each moment and the Z axis in the public reference base coordinate system through the space vector angle formula, and the angle representing a swing posture of the fruit; (6) rotating the {right arrow over (O.sub.1Z.sub.1)} at each moment to a position that coincides with the Z axis in the public reference base coordinate system, and calculating an angle between the rotated {right arrow over (O.sub.1Z.sub.1)} and the X axis in the public reference base coordinate system, the angle representing a twisting posture of the fruit; and (7) setting moments corresponding to two adjacent frames to t+1 moment and t moment, and calculating a displacement of the point O.sub.1 along the X axis, Y axis and Z axis of the public reference base coordinate system at t+1 moment and t moment respectively, calculating an instantaneous speed along the X axis, an instantaneous speed along the Y axis, and an instantaneous speed along the Z axis of the fruit at t moment according to the displacement, calculating an instantaneous acceleration along the X axis, an instantaneous acceleration along the Y axis and an instantaneous acceleration along the Z axis of the fruit at t moment according to the instantaneous speed, calculating a swing angular displacement of the fruit in a time interval between the two adjacent frames of images, sequentially calculating a swing angular speed of the fruit at t moment and a swing angular acceleration of the fruit at t moment via the swing angular displacement, and calculating a torsion angular speed of the fruit at t moment and a torsion angular acceleration of the fruit at t moment according to the torsion angular degree from t moment to t+1 moment.

2. The method according to claim 1, wherein the step (2) includes: establishing the connected base coordinate system by taking a junction of the fruit and the fruit stem as the origin, and taking a line between the junction of the fruit and the fruit stem and a center point of the fruit as a center line, the center line being the Z axis of the connected base coordinate system, which being marked as the Z connected-axis, taking a direction from the central point pointing to the joining point as the positive direction of the Z connected-axis, establishing the X axis and Y axis of the connected base coordinate system on a plane perpendicular to the Z axis of the connected base coordinate system, and marking them as X connected-axis and Y connected-axis respectively; marking the origin of the connected base coordinate system as O.sub.1, marking the end point of the unit vector in the positive direction of the X connected-axis as X.sub.1, marking the end point of the unit vector in the positive direction of the Y.sub.1 connected-axis as Y.sub.1, and marking the end point of the unit vector in the positive direction of the Z connected-axis as Z1.

3. The method according to claim 2, wherein the absolute coordinates are spatial coordinates in the public reference base coordinate system, and the unit vectors of the coordinate axis of the public reference base coordinate system are:
X=(1 0 0).sup.T, Y=(0 1 0).sup.T, Z=(0 0 1).sup.T.

4. The method according to claim 3, wherein the step (4) includes: (a) under static conditions, using two high-speed cameras to statically photograph the three feature points on the surface of the fruit and the points O.sub.1, X.sub.1, Y.sub.1, Z.sub.1 of the connected base coordinate system established at the junction of the fruit and the fruit stem, using software to store the photographed images, and using software to process the feature points C.sub.1, C.sub.2, C.sub.3 and the connected base coordinate system points O.sub.1, X.sub.1, Y.sub.1, Z.sub.1 in the images, acquiring the absolute coordinate of the feature points C.sub.1, C.sub.2, C.sub.3 and the absolute coordinate of the connected base coordinate system points O.sub.1, X.sub.1, Y.sub.1, Z.sub.1; (b) creating vectors {right arrow over (C.sub.1C.sub.2)} and {right arrow over (C.sub.1C.sub.3)} via absolute coordinate, unitizing {right arrow over (C.sub.1C.sub.2)} and {right arrow over (C.sub.1C.sub.3)} creating vector {right arrow over (C.sub.1O.sub.1)}, vector {right arrow over (C.sub.1X.sub.1)}, vector {right arrow over (C.sub.1Y.sub.1)} and vector {right arrow over (C.sub.1Z.sub.1)}; (c) doing a vector product of unitized {right arrow over (C.sub.1C.sub.2)} and {right arrow over (C.sub.1C.sub.3)} to get: {right arrow over (Y)}={right arrow over (C.sub.1C.sub.2)}×{right arrow over (C.sub.1C.sub.3)}, doing a vector product of {right arrow over (X)}={right arrow over (C.sub.1C.sub.2)} and {right arrow over (Y)} to get: {right arrow over (Z)}={right arrow over (X)}×{right arrow over (Y)}, thus, establishing a coordinate system C.sub.xyz by taking feature point C.sub.1 as an origin, the coordinate axis vector of the coordinate system C.sub.xyz is: {right arrow over (X.sub.C.sub.xyz)}=(x.sub.C.sub.x, y.sub.C.sub.x, z.sub.C.sub.x).sup.T, {right arrow over (Y.sub.C.sub.xyz)}=(x.sub.C.sub.y, y.sub.C.sub.y, z.sub.C.sub.y).sup.T, {right arrow over (Z.sub.C.sub.xyz)}=Z.sub.C.sub.z, y.sub.C.sub.z, z.sub.C.sub.z).sup.T, calculating angles α.sub.x, β.sub.x, γ.sub.x between the vector {right arrow over (X.sub.C.sub.xyz)} and the X axis, Y axis and Z axis of the public reference base coordinate system through the calculation formula of the space vector angle, calculating angles α.sub.y, β.sub.y, γ.sub.y between the vector $\stackrel{.fwdarw.}{Y_{C_{{\mathrm{xyz}}_t}}}$ and the X axis, Y axis and Z axis of the public reference base coordinate system, calculating angles α.sub.z, β.sub.z, γ.sub.z between the vector $\stackrel{.fwdarw.}{Z_{C_{\mathrm{xyz}}}}$ and the X-axis, Y-axis and Z-axis of the public reference base coordinate system, then the coordinate conversion matrix is: $\begin{array}{cc}A=\left(\begin{array}{ccc}\cos {\alpha }_x& \cos {\beta }_x& \cos {\gamma }_x\\ \cos {\alpha }_y& \cos {\beta }_y& \cos {\gamma }_z\\ \cos {\alpha }_z& \cos {\beta }_z& \cos {\gamma }_z\end{array}\right);& \left(1\right)\end{array}$ the coordinates of point O.sub.1 under the coordinate system C.sub.xyz are obtained by formula (2): $\begin{array}{cc}{O}_{1_{C_{\mathrm{xyz}}}}=A*\stackrel{.fwdarw.}{C_1{O}_1}=\left(\begin{array}{ccc}\cos {\alpha }_x& \cos {\beta }_x& \cos {\gamma }_x\\ \cos {\alpha }_y& \cos {\beta }_y& \cos {\gamma }_z\\ \cos {\alpha }_z& \cos {\beta }_z& \cos {\gamma }_z\end{array}\right)\left(\begin{array}{c}{x}_{O_1}^{\prime }\\ y_{O_1}^{\prime }\\ z_{O_1}^{\prime }\end{array}\right)=\left(\begin{array}{c}{x}_{O_{1C_{\mathrm{xyz}}}}\\ y_{O_{1C_{\mathrm{xyz}}}}\\ z_{O_{1C_{\mathrm{xyz}}}}\end{array}\right);& \left(2\right)\end{array}$ wherein, {right arrow over (C.sub.1O.sub.1)}=(x.sub.O.sub.1′ y.sub.O.sub.1′ z.sub.O.sub.1′).sup.T, and obtained from (b) in step (4); similarly, $X_{1_{C_{\mathrm{xyz}}}}=A*\stackrel{.fwdarw.}{C_1{X}_1}{Y}_{1_{C_{\mathrm{xyz}}}}=A*\stackrel{.fwdarw.}{C_1{Y}_1}\mathrm{and}{Z}_{1_{C_{\mathrm{xyz}}}}=A*\stackrel{.fwdarw.}{C_1{Z}_1}$ can be obtained, wherein the coordinates of $X_{1_{C_{\mathrm{xyz}}}}{Y}_{1_{C_{\mathrm{xyz}}}}{Z}_{1_{C_{\mathrm{xyz}}}}$ are the coordinates of points X.sub.1, Y.sub.1, Z.sub.1 under the coordinate system C.sub.xyz; the coordinates of $O_{1_{C_{\mathrm{xyz}}}}{X}_{1_{C_{\mathrm{xyz}}}}{Y}_{1_{C_{\mathrm{xyz}}}}{Z}_{1_{C_{\mathrm{xyz}}}}$ are the coordinates of points O.sub.1, X.sub.1, Y.sub.1, Z.sub.1 under the coordinate system C.sub.xyz, that is, representing the inherent relationship between the feature point and the connected base coordinate system.

5. The method according to claim 4, wherein the step (5) includes: (a) using two high-speed cameras to photograph the fruit dynamic motion, using software to store each frame of image of the photographed video, using software to process the feature points in the images, and obtaining the absolute coordinate of the feature points on the surface of the fruit; (b) calculating angles α.sub.x.sub.1, β.sub.x.sub.1, γ.sub.x.sub.1 between the vector $\stackrel{.fwdarw.}{X_{C_{{\mathrm{xyz}}_t}}}$ and the X axis, Y axis, and Z axis of the public reference base coordinate system at t moment through the calculation formula of the space vector angle, calculating angles α.sub.y.sub.1, β.sub.y.sub.1, γ.sub.y.sub.1 between the vector $\stackrel{.fwdarw.}{Y_{C_{{\mathrm{xyz}}_t}}}$ and the X axis, Y axis and Z axis of the public reference base coordinate system, calculating angles α.sub.z.sub.1, β.sub.z.sub.1, γ.sub.z.sub.1 between the vector $\stackrel{.fwdarw.}{Z_{C_{{\mathrm{xyz}}_t}}}$ and the X axis, Y axis and Z axis of the public reference base coordinate system, wherein, $\stackrel{.fwdarw.}{X_{C_{{\mathrm{xyz}}_t}}}\stackrel{.fwdarw.}{Y_{C_{{\mathrm{xyz}}_t}}}\stackrel{.fwdarw.}{Z_{C_{{\mathrm{xyz}}_t}}}$ are the coordinate axis vector of the coordinate system C.sub.xyz established by taking the feature point C.sub.1 in one frame of image corresponding to the t moment as an origin, then the coordinate conversion matrix is: $\begin{array}{cc}{A}_t=\left(\begin{array}{ccc}\cos {\alpha }_{xt}& \cos {\beta }_{\mathrm{xt}}& \cos {\gamma }_{\mathrm{xt}}\\ \cos {\alpha }_{y_t}& \cos {\beta }_{yt}& \cos {\gamma }_{\mathrm{zt}}\\ \cos {\alpha }_{\mathrm{zt}}& \cos {\beta }_{zt}& \cos {\gamma }_{\mathrm{zt}}\end{array}\right);& \left(3\right)\end{array}$ the coordinates of point O.sub.1 in the public reference base coordinate system, that is, the absolute coordinate is: $\begin{array}{cc}\left(\begin{array}{c}{x}_{O_1}\\ y_{O_1}\\ z_{O_1}\end{array}\right)=\left(\begin{array}{c}{x}_{C_1}\\ y_{C_1}\\ z_{C_1}\end{array}\right)+A_t^{-1}\left(\begin{array}{c}{x}_{O_{1C_{\mathrm{xyz}}}}\\ y_{O_{1C_{\mathrm{xyz}}}}\\ z_{O_{1C_{\mathrm{xyz}}}}\end{array}\right);& \left(4\right)\end{array}$ wherein (x.sub.O.sub.1, y.sub.O.sub.1, z.sub.O.sub.1) are the absolute coordinate of the point O.sub.1 under t moment, (x.sub.C.sub.1, y.sub.C.sub.1, z.sub.C.sub.1) are the absolute coordinate of the point C.sub.1 under t moment, $\left(x_{O_{1C_{\mathrm{xyz}}}}{y}_{O_{1C_{\mathrm{xyz}}}}{z}_{O_{1C_{\mathrm{xyz}}}}\right)$ are the coordinate of point O.sub.1 in the coordinate system C.sub.xyz; similarly, calculating the absolute coordinate of the points X.sub.1, Y.sub.1, Z.sub.1 under t moment; each frame of image is independent of each other, and corresponds to a moment respectively, and solves the absolute coordinates of the points O.sub.1, X.sub.1, Y.sub.1, Z.sub.1 of the connected base coordinate system frame by frame; calculating the angle θ between the vector {right arrow over (O.sub.1Z.sub.1)} at each moment and the Z axis in the public reference base coordinate system through the space vector angle formula $\begin{array}{cc}\theta =\arccos \left(\frac{.Math.x_{Z_1}{x}_Z+y_{Z_1}{y}_Z+z_{Z_1}{z}_Z.Math.}{\sqrt{x_{Z_1}^2+y_{Z_1}^2+z_{Z_1}^2}\times \sqrt{x_Z^2+y_Z^2+z_Z^2}}\right);& \left(5\right)\end{array}$ wherein {right arrow over (O.sub.1Z.sub.1)}=(x.sub.z.sub.1, y.sub.z.sub.1, z.sub.z.sub.1).sup.T (x.sub.Z, y.sub.Z, z.sub.Z).sup.T is the unit vector of the Z axis in the public reference base coordinate system, the angle θ between the vector {right arrow over (O.sub.1Z.sub.1)} and the Z axis in the public reference base coordinate system represents a swing posture of the fruit.

6. The method according to claim 5, wherein the step (6) includes: (a) using the axis rotation matrix to rotate {right arrow over (O.sub.1Z.sub.1)} at the moment corresponding to a certain frame of image to a position that coincides with the Z axis in the public reference base coordinate system, calculating the angle between the rotated {right arrow over (O.sub.1X.sub.1)} and the X axis in the public reference base coordinate system, the angle represents an twisting posture of the fruit at the moment corresponding to the frame image, detail as follows: the Z connected-axis direction vector of the frame of image is: $\stackrel{.fwdarw.}{Z_{}}=\stackrel{.fwdarw.}{O_1{Z}_1}={\left(x_{z_1}{y}_{z_1}{z}_{z_1}\right)}^T,.Math.\stackrel{.fwdarw.}{O_1{Z}_1}.Math.=\sqrt{x_{Z_1}^2+y_{Z_1}^2+z_{Z_1}^2},$ calculating the angle φ between the Z connected-axis and the Z axis in the public reference base coordinate system: $\begin{array}{cc}\phi =\arccos \left(\frac{Z_{Z_1}}{.Math.\stackrel{.fwdarw.}{O_1{Z}_1}.Math.}\right);& \left(6\right)\end{array}$ calculating the angle ψ between the projection of the Z connected-axis in the X axis and Y axis of the public reference base coordinate system and the Y axis of the public reference base coordinate system: $\begin{array}{cc}\psi =\arccos \left(\frac{y_{Z_1}}{\sqrt{y_{Z_1}^2+x_{Z_1}^2}}\right);& \left(7\right)\end{array}$ calculating the axis rotation matrix: $\begin{array}{cc}{T}_1=\left(\begin{array}{ccc}\cos \phi & 0& \sin \phi \\ 0& 1& 0\\ -\sin \phi & 0& \cos \phi \end{array}\right);& \left(8\right)\\ T_2=\left(\begin{array}{ccc}\cos \psi & -\sin \psi & 0\\ \sin \psi & \cos \psi & 0\\ 0& 0& 1\end{array}\right);& \left(9\right)\end{array}$ the absolute coordinate of the rotated point X.sub.1 is consistent with the coordinate of the rotated vector {right arrow over (O.sub.1X.sub.1)}, and the rotated vector {right arrow over (O.sub.1X.sub.1)} is: $\begin{array}{cc}\left(\begin{array}{c}{x}_{X_1^{\prime }}\\ y_{X_1^{\prime }}\\ z_{X_1^{\prime }}\end{array}\right)=T_2^{-1}*T_1^{-1}*\left(\begin{array}{c}{x}_{X_1}\\ y_{X_1}\\ z_{X_1}\end{array}\right);& \left(10\right)\end{array}$ wherein (x.sub.X.sub.1, y.sub.X.sub.1, z.sub.X.sub.1).sup.T is the vector {right arrow over (O.sub.1X.sub.1)} before the {right arrow over (O.sub.1Z.sub.1)} of the frame of image is rotated; the torsion angular degree is: $\begin{array}{cc}\varphi =\arccos \left(\frac{.Math.x_{X_1^{\prime }}{x}_X+y_{X_1^{\prime }}{y}_X+z_{X_1^{\prime }}{z}_X.Math.}{\sqrt{x_{X_1^{\prime 2}}+y_{X_1^{\prime 2}}+z_{X_1^{\prime 2}}}\times \sqrt{x_{X^2}+y_{X^2}+z_{X^2}}}\right);& \left(11\right)\end{array}$ the torsion angular degree is the angle between the rotated vector {right arrow over (O.sub.1X.sub.1)} and the X axis in the public reference base coordinate system, which represents a torsion posture of the fruit at the moment corresponding to the frame image, wherein (x.sub.X, y.sub.X, z.sub.X).sup.T is the unit vector of the X axis in the public reference base coordinate system; and (b) similarly, in (a) of step (6), calculating the twisting posture of the fruit at each moment corresponding to each frame image.

7. The method according to claim 6, wherein the step (7) includes: (a) setting moments corresponding to two adjacent frames of images to t+1 moment and t moment, based on the point O.sub.1 of the connected base coordinate system in adjacent two frames of images, and calculating the displacement of the point O.sub.1 along the X axis of the public reference base coordinate system at t+1 moment and t moment respectively:
S.sub.x=x.sub.O.sub.t+1−x.sub.O.sub.1  (12); wherein x.sub.O.sub.t+1, x.sub.O.sub.1 are the coordinates x of the point O.sub.1 under the public reference base coordinate system at t+1 moment and t moment; similarly, calculating the displacement S.sub.y along the Y axis and the displacement S.sub.z along the Z axis of the point O.sub.1 of the public reference base coordinate system at t+1 moment and t moment, and the combined displacement is:
S=√{square root over (S.sub.x.sup.2+S.sub.y.sup.2+S.sub.z.sup.2)}  (13); (b) using the average speed of the point O.sub.1 moving along the X axis of the public reference base coordinate system from t moment to t+1 moment to express the instantaneous speed v.sub.x(t) of the fruit along the X axis at t moment: $\begin{array}{cc}{v}_{x\left(t\right)}=\frac{S_x}{\Delta t};& \left(14\right)\end{array}$ wherein Δt is the interval time between two frames of images; similarly, the instantaneous speed v.sub.y(t) of the fruit along the Y axis of the public reference base coordinate system at t moment and the instantaneous speed v.sub.z(t) of the fruit along the Z axis of the public reference base coordinate system at t moment can be obtained, then the combined speed of the fruit at t moment is:
v=√{square root over (v.sub.x(t).sup.2+v.sub.y(t).sup.2+v.sub.z(t).sup.2)}  (15); (c) using the acceleration of the point O.sub.1 moving along the X axis of the public reference base coordinate system from t moment to t+1 moment to express the instantaneous acceleration a.sub.x(t) of the fruit along the X axis at the t moment: $\begin{array}{cc}{a}_{x\left(t\right)}=\frac{v_{x\left(t+1\right)}-v_{x\left(t\right)}}{\Delta t};& \left(16\right)\end{array}$ wherein v.sub.x(t+1) is the instantaneous speed of the fruit along the X axis at t+1 moment, and v.sub.x(t) is the instantaneous speed of the fruit along the X axis at t moment; similarly, the instantaneous acceleration a.sub.y(t) of the fruit along the Y axis of the public reference base coordinate system at t moment and the instantaneous acceleration a.sub.z(t) of the fruit along the Z axis of the public reference base coordinate system at t moment can be obtained; then the combined acceleration of the fruit at t moment is:
a=√{square root over (a.sub.x(t).sup.2+a.sub.y(t).sup.2+a.sub.z(t).sup.2)}  (17); (d) using the angle from the Z connected-axis at t moment to the Z connected-axis at t+1 moment to represent the swing angular displacement of the fruit in Δt: $\begin{array}{cc}\Delta \theta =\arccos \left(\frac{.Math.x_{Z_{1_{t+1}}}{x}_{Z_{1_t}}+y_{Z_{1_{t+1}}}{y}_{{Z_1}_t}+z_{Z_{1_{t+1}}}{z}_{Z_{1_t}}.Math.}{\sqrt{x_{Z_{1_{t+1}}}^2+y_{Z_{1_{t+1}}}^2+z_{Z_{1_{t+1}}}^2}\times \sqrt{x_{Z_{1_t}}^2+y_{Z_{1_t}}^2+z_{Z_{1_t}}^2}}\right);& \left(18\right)\end{array}$ wherein ${\left(x_{Z_{1_t}}{y}_{Z_{1_t}}{z}_{Z_{1_t}}\right)}^T$ the direction vector of Z connected-axis at t moment, that is t moment vector {right arrow over (O.sub.1Z.sub.1)}; ${\left(x_{Z_{1_{t+1}}}{y}_{Z_{1_{i+1}}}{z}_{Z_{1_{t+1}}}\right)}^T$ is the direction vector of Z connected-axis at t+1 moment, that is t+1 moment vector {right arrow over (O.sub.1Z.sub.1)}; (e) using the average swing angular speed from t moment to t+.sup.1 moment to express the t swing angular speed ω.sub.θ.sub.t of the fruit at t moment: $\begin{array}{cc}{\omega }_{{\theta }_t}=\frac{\Delta \theta }{\Delta t};& \left(19\right)\end{array}$ (f) using the amount of change of the swing angular speed from t moment to t+1 a moment to express the swing angular acceleration a.sub.θ.sub.t of the fruit at t moment: $\begin{array}{cc}{a}_{{\theta }_t}=\frac{{\omega }_{{\theta }_{t+1}}-{\omega }_{{\theta }_t}}{\Delta t};& \left(20\right)\end{array}$ (g) using the axis rotation matrix to rotate the Z connected-axis at t moment and the Z connected-axis at t+1 moment to the position that coincides with the Z connected-axis in the public reference base coordinate system, using formula (6) to formula (10) in step (a) of step (6) to calculate the rotated vector {right arrow over (O.sub.1X.sub.1)} at t moment and the rotated vector {right arrow over (O.sub.1X.sub.1)} at t+1 moment, then the torsion angular degree from t moment to t+1 moment is: $\begin{array}{cc}\Delta \varphi =\arccos \left(\frac{.Math.x_{X_{1_{t+1}}^{\prime }}{x}_{X_t}+y_{X_{1_{t+1}}^{\prime }}{y}_{X_t}+z_{X_{1_{t+1}}^{\prime }}{z}_{X_t}.Math.}{\sqrt{x_{X_{1_{t+1}}^{\prime 2}}+y_{X_{1_{t+1}}^{\prime 2}}+z_{X_{1_{t+1}}^{\prime 2}}}\times \sqrt{x_{X_t^2}+y_{X_t^2}+z_{X_t^2}}}\right);& \left(21\right)\end{array}$ wherein ${\left(x_{X_{1_t}}^{\prime }{y}_{X_{1_t}}^{\prime }{z}_{X_{1_t}}^{\prime }\right)}^T$ is the rotated vector {right arrow over (O.sub.1X.sub.1)} at t moment, ${\left(x_{X_{1_{t+1}}}^{\prime }{y}_{X_{1_{t+1}}}^{\prime }{z}_{X_{1_{t+1}}}^{\prime }\right)}^T$ is the rotated vector {right arrow over (O.sub.1X.sub.1)} at t+1 moment; the torsion angular speed of the fruit at t moment is: $\begin{array}{cc}{\omega }_{{\varphi }_t}=\frac{\Delta \varphi }{\Delta t};& \left(22\right)\end{array}$ the torsion angular acceleration of the fruit at t moment is: $\begin{array}{cc}{a}_{{\varphi }_t}=\frac{{\omega }_{{\varphi }_{t+1}}-{\omega }_{{\varphi }_t}}{\Delta t}.& \left(23\right)\end{array}$

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Various other features and attendant advantages of the present invention will be more fully appreciated from the following detailed description when considered in connection with the accompanying drawings in which like reference characters designate like or corresponding parts throughout the several views, and wherein:

(2) FIG. 1 shows a flowchart of the method;

(3) FIG. 2 shows a relationship diagram of a coordinate system in a fruit space;

(4) FIG. 3 shows a coordinate curve of a feature point x;

(5) FIG. 4 shows a coordinate curve of a feature point y;

(6) FIG. 5 shows a coordinate curve of a feature point z;

(7) FIG. 6 shows a schematic diagram of a fruit space motion;

(8) FIG. 7 shows a translation trajectory of a fruit spatial;

(9) FIG. 8 shows a position diagram of a conjoined base coordinate of two adjacent moments; and

(10) FIG. 9 shows a schematic diagram of torsion angular.

DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENT

(11) The following further describes the specific implementation of the present invention based on FIGS. 1 to 9.

(12) This embodiment provides a binocular-vision-based method for tracking fruit space attitude and fruit space motion. FIG. 1 is an application step diagram of this embodiment. The content of FIG. 1 will be expanded in detail below.

(13) Mark three feature points C.sub.3 on the surface of the fruit.

(14) Establish a method for setting the fruit posture in space. The fruit is regarded as a rigid body, which does not deform in any state, and the fruit is regarded as a standard rotating body. establishing the connected base coordinate system by taking a junction of the fruit and the fruit stem as the origin, and taking a line between the junction of the fruit and the fruit stem and a center point of the fruit as a center line, the center line being the Z axis of the connected base coordinate system, which being marked as the Z connected-axis, taking a direction from the central point pointing to the joining point as the positive direction of the Z connected-axis, establishing the X-axis and Y-axis of the connected base coordinate system on a plane perpendicular to the Z-axis of the connected base coordinate system, and marking them as X connected-axis and Y connected-axis respectively; marking the origin of the connected base coordinate system as O.sub.1, marking the end point of the unit vector in the positive direction of the X connected-axis as X.sub.1, marking the end point of the unit vector in the positive direction of the Y connected-axis as Y.sub.1, and marking the end point of the unit vector in the positive direction of the Z connected-axis as Z.sub.1. Using this method, the movement of the fruit in space can be decomposed into translation, swinging around the junction of the fruit stem and the fruit, and twisting around the centerline. Using this method, the movement of the fruit in space can be decomposed into translation, swinging around the junction of the fruit stem and the fruit, and twisting around the centerline. Using this method, the movement of the fruit in space can be decomposed into translation, swinging around the junction of the fruit stem and the fruit, and torsion movement around the centerline.

(15) Establish a public reference base coordinate system, the unit vector of the coordinate axis of the public reference base coordinate system is:

(16) Z=(0 0 1).sup.T.

(17) The absolute coordinates described in this article are all spatial coordinates in the public reference base coordinate system. The vectors described in this article are all vectors in the public reference base coordinate system.

(18) Since the feature points on the fruit surface tracked by the binocular camera cannot directly reflect the spatial posture of the fruit, this embodiment invented a method for creating a fixed relationship between the conjoined base and the feature points on the fruit surface. First, using two high-speed cameras 1-1 (M310, VEO 410) to statically photograph the three feature points on the surface of the fruit and the points Z.sub.1 of the connected base coordinate system established at the junction of the fruit and the fruit stem, using Phantom software to store the photographed images, and using TEMA software to process the feature points and the connected base coordinate system points Z.sub.1 in the images, export the spatial absolute coordinate of the feature points C.sub.3 and the points Z.sub.1 of the connected base coordinate system and then establish an inherent relationship between the feature points and the connected base coordinate system. Referring to FIG. 2, this embodiment uses two high-speed cameras 1-1 to photograph the fruit 2-1. Two high-speed cameras 1-1 can obtain depth information by binocular visual tracking, and obtain the spatial coordinates of the fruit 2-1. Reference numeral 1-1 in FIG. 2 represents a high-speed camera, reference numeral 2-1 represents a fruit, X, Y, and Z in FIG. 2 represent a public reference base coordinate system, and X connected-axis, Y connected-axis, Z connected-axis represent a connected base coordinate system, X.sub.cxyz, Y.sub.cxyz, Z.sub.cxyz represent coordinate system C.sub.xyz. Specifically, it includes the following steps:

(19) (a) under static conditions, using two high-speed cameras (M310, VEO 410) to statically photograph the three feature points C.sub.3 on the surface of the fruit and the points Z.sub.1 of the connected base coordinate system established at the junction of the fruit and the fruit stem, using Phantom software to store the photographed images, and using TEMA software to process the feature points C.sub.3 and the connected base coordinate system points Z.sub.1 in the images, acquiring the absolute coordinate of the feature points C.sub.3 and the absolute coordinate of the connected base coordinate system points Z.sub.1; the absolute coordinate of feature point C.sub.2 is ( z.sub.C.sub.2).sup.T, the absolute coordinate of feature point C.sub.1 is ( z.sub.C.sub.1).sup.T, the

(20) absolute coordinate of feature point C.sub.3 is ( z.sub.C.sub.3).sup.T;

(21) (b) creating vectors {right arrow over (C.sub.1C.sub.2)} and {right arrow over (C.sub.1C.sub.3)} via absolute coordinate, unitizing {right arrow over (C.sub.1C.sub.2)} and {right arrow over (C.sub.1C.sub.3)}, {right arrow over (C.sub.1C.sub.2)} and {right arrow over (C.sub.1C.sub.3)} are:

(22) {right arrow over (C.sub.1C.sub.2)}=(X.sub.C.sub.2− y.sub.C.sub.2− z.sub.C.sub.2−Z.sub.C.sub.1).sup.T, {right arrow over (C.sub.1C.sub.3)}=(x.sub.C.sub.3− y.sub.C.sub.3− z.sub.C.sub.3−z.sub.C.sub.1).sup.T; establish vector {right arrow over (C.sub.1O.sub.1)}, {right arrow over (C.sub.1O.sub.1)}=(x.sub.O.sub.1− y.sub.O.sub.1− z.sub.O.sub.1−z.sub.C.sub.1).sup.T=(x.sub.O.sub.1′ y.sub.O.sub.1′ z.sub.O.sub.1′).sup.T;

(23) is the coordinate of the point O.sub.1 of the connected base coordinate system in the public reference base coordinate system; and use the same method to create vector vector {right arrow over (C.sub.1Y.sub.1)} and vector {right arrow over (C.sub.1Z.sub.1)};

(24) (c) doing a vector product of unitized {right arrow over (C.sub.1C.sub.2)} and {right arrow over (C.sub.1C.sub.3)} to get: {right arrow over (Y)}={right arrow over (C.sub.1C.sub.2)}× {right arrow over (C.sub.1C.sub.3)}, doing a vector product of {right arrow over (X)}={right arrow over (C.sub.1C.sub.2)} and {right arrow over (Y)} to get: {right arrow over (Z)}={right arrow over (X)}×{right arrow over (Y)}, thus, establishing a coordinate system C.sub.xyz by taking feature point C.sub.1 as an origin, the coordinate axis vector of the coordinate system C.sub.xyz is: X.sub.C.sub.xyz===( z.sub.C.sub.z).sup.T, calculating angles γ.sub.x between the vector {right arrow over (X.sub.C.sub.xyz)} and the X axis, Y axis and Z axis of the public reference base coordinate system through the calculation formula of the space vector angle, calculating angles γ.sub.y between the vector Y.sub.C.sub.xyz and the X axis, Y axis and Z axis of the public reference base coordinate system, calculating angles γ.sub.z between the vector {right arrow over (Z.sub.C.sub.xyz )} and the X axis, Y axis and Z axis of the public reference base coordinate system; wherein the calculation formula for the angle of the space vector is:

(25) $\cos \phi =\frac{.Math.m_1{m}_2+n_1{n}_2+p_1{p}_2.Math.}{\sqrt{m_1^2+n_1^2+p_1^2}\times \sqrt{m_2^2+n_2^2+p_2^2}};$

(26) wherein p.sub.2 respectively correspond to the X coordinate, Y coordinate, and Z coordinate of the space vector;

(27) the coordinate conversion matrix is:

(28) $\begin{array}{cc}A=\left(\begin{array}{ccc}\cos {\alpha }_x& \cos {\beta }_x& \cos {\gamma }_x\\ \cos {\alpha }_y& \cos {\beta }_y& \cos {\gamma }_z\\ \cos {\alpha }_z& \cos {\beta }_z& \cos {\gamma }_z\end{array}\right);& \left(1\right)\end{array}$

(29) the coordinates of point O.sub.1 under the coordinate system C.sub.xyz are obtained by formula (2):

(30) $\begin{array}{cc}{O}_{1_{C_{\mathrm{xyz}}}}=A^{*}\stackrel{.fwdarw.}{C_1{O}_1}=\left(\begin{array}{ccc}\cos {\alpha }_x& \cos {\beta }_x& \cos {\gamma }_x\\ \cos {\alpha }_y& \cos {\beta }_y& \cos {\gamma }_z\\ \cos {\alpha }_z& \cos {\beta }_z& \cos {\gamma }_z\end{array}\right)\left(\begin{array}{c}{x_{o_I}}^{{}^{\prime }}\\ {y_{o_1}}^{{}^{\prime }}\\ {z_{o_1}}^{{}^{\prime }}\end{array}\right)=\left(\begin{array}{c}{x}_{O_{1C_{\mathrm{xy}}}}\\ y_{O_{{\mathrm{IC}}_{\mathrm{xyz}}}}\\ z_{O_{1C_{\mathrm{xyz}}}}\end{array}\right);& \left(2\right)\end{array}$

(31) wherein, {right arrow over (C.sub.1O.sub.1)}=(x.sub.O.sub.1′ y.sub.O.sub.1′ z.sub.O.sub.1′).sup.T, and obtained from step (b);

(32) similarly,

(33) $X_{1_{C_{\mathrm{xyz}}}}=A^{*}{\stackrel{.fwdarw.}{C_{1}X}}_1,Y_{1_{C_{\mathrm{xyz}}}}=A^{*}\stackrel{.fwdarw.}{C_1{Y}_1}\mathrm{and}{Z}_{1_{C_{\mathrm{xyz}}}}=A^{*}\stackrel{.fwdarw.}{C_1{Z}_1},$
wherein the coordinates corresponding to

(34) $X_{1_{C_{\mathrm{xyz}}}},Y_{1_{C_{\mathrm{xyz}}}},Z_{1_{C_{\mathrm{xyz}}}}$
are the coordinates of points X.sub.1, Z.sub.1 under the coordinate system C.sub.xyz; the vector {right arrow over (C.sub.1X.sub.1)}, the vector {right arrow over (C.sub.1Y.sub.1)}, and the vector {right arrow over (C.sub.1Z.sub.1)} are all obtained from the above-mentioned step (b); the coordinates corresponding to

(35) $O_{1_{C_{\mathrm{xyz}}}}{X}_{1_{C_{\mathrm{xyz}}}}{Y}_{1_{C_{\mathrm{xyz}}}}{Z}_{1_{C_{\mathrm{xyz}}}}$
are use coordinates of points Z.sub.1 under the feature point coordinate system C.sub.xyz, that is, it represents the inherent relationship between the feature point and the connected base coordinate system. The inherent relationship in this embodiment is:

(36) $S_e=\left(O_{1_{C_{\mathrm{xyz}}}}{X}_{1_{C_{\mathrm{xyz}}}}{Y}_{1_{C_{\mathrm{xyz}}}}{Z}_{1_{C_{\mathrm{xyz}}}}\right)=\left(\begin{array}{cccc}8.3580& 7.3782& 8.3534& 8.2740\\ -12.4324& -12.3469& -12.4959& -13.4267\\ -2.3046& -2.3054& -3.2860& -2.2420\end{array}\right)\circ$

(37) The fruit space motion process is divided into translation, swing and torsion, and the motion process is shown in FIG. 6. The binocular-vision-based method of tracking fruit space attitude first uses two high-speed cameras 1-1 (M310, VEO 410) to photograph the fruit dynamic motion, and the movement of the fruit under a certain frequency of excitation. Use Phantom software to store each frame of image in the photographed video, and use TEMA software to process the feature points in each frame of image, export the absolute coordinate of the feature point on the fruit surface of each frame of image in the video, as shown in FIGS. 3 to 5, FIG. 3 is the x-coordinate diagram of the feature point C.sub.1, FIG. 4 is the y-coordinate diagram of the feature point C.sub.1, and FIG. 5 is the z-coordinate diagram of the feature point C.sub.1, which represents the feature point motion curve in the video. The abscissas in FIGS. 3 to 5 represent each moment corresponding to each frame of image. According to the inherent relationship between the feature points and the connected base coordinate system, the absolute coordinate of each frame of image corresponding to the connected base coordinate system points Z.sub.1 at each moment being calculated by the inverse rotation transformation, and calculating an angle between the vector {right arrow over (O.sub.1Z.sub.1)} at each moment and the Z axis in the public reference base coordinate system through the space vector angle formula, and the angle representing the swing posture of the fruit. Rotate the {right arrow over (O.sub.1Z.sub.1)} at each moment to a position that coincides with the Z axis in the public reference base coordinate system, and calculating an angle between the rotated {right arrow over (O.sub.1X.sub.1)} and the X axis in the public reference base coordinate system, the angle representing the twisting posture of the fruit, details as follows:

(38) (a1) Use two high-speed cameras 1-1 to photograph fruit dynamic motion, use Phantom software to store each frame of image of the photographed video, use TEMA software to process the feature point in the image, and export the absolute coordinate of the feature point on the fruit surface frame by frame;

(39) (b1) Follow the above-mentioned steps (b) and (c) to establish a coordinate system C.sub.xyz by taking the feature point C.sub.1 in one frame of image corresponding to t moment as an origin;

(40) $\stackrel{.fwdarw.}{X_{C_{{\mathrm{xyz}}_t}}}\stackrel{.fwdarw.}{Y_{C_{{\mathrm{xyz}}_t}}},\stackrel{.fwdarw.}{Z_{C_{{\mathrm{xyz}}_t}}}$
are the coordinate axis vector of the coordinate system C.sub.xyz established by taking the feature point C.sub.1 in one frame of image corresponding to t moment as an origin;

(41) calculating angles γ.sub.x.sub.t between the vector

(42) 0$\stackrel{.fwdarw.}{X_{C_{{\mathrm{xyz}}_t}}}$
and the X axis, Y axis, and Z axis of the public reference base coordinate system at t moment through the calculation formula of the space vector angle, calculating angles γ.sub.y.sub.t between the vector

(43) $\stackrel{.fwdarw.}{Y_{C_{{\mathrm{xyz}}_t}}}$
and the X axis, Y axis and Z axis of the public reference base coordinate system, calculating angles γ.sub.z.sub.t between the vector

(44) $\stackrel{.fwdarw.}{Z_{C_{{\mathrm{xyz}}_t}}}$
and the X axis, Y axis and Z axis of the public reference base coordinate system, then the coordinate conversion matrix at t moment is:

(45) $\begin{array}{cc}{A}_t=\left(\begin{array}{ccc}\cos {\alpha }_{\mathrm{xt}}& \cos {\beta }_{\mathrm{xt}}& \cos {\gamma }_{\mathrm{xt}}\\ \cos {\alpha }_{\mathrm{yt}}& \cos {\beta }_{\mathrm{yt}}& \cos {\gamma }_{\mathrm{yt}}\\ \cos {\alpha }_{\mathrm{zt}}& \cos {\beta }_{\mathrm{zt}}& \cos {\gamma }_{\mathrm{zt}}\end{array}\right);& \left(3\right)\end{array}$

(46) the coordinates of point O.sub.1 in the public reference base coordinate system, that is, the absolute coordinate is:

(47) $\begin{array}{cc}\left(\begin{array}{c}{x}_{O_1}\\ y_{O_1}\\ z_{O_1}\end{array}\right)=\left(\begin{array}{c}{x}_{C_1}\\ y_{C_1}\\ z_{C_1}\end{array}\right)+A_t^{-1*}\left(\begin{array}{c}{x}_{O_{1C_{\mathrm{xyz}}}}\\ y_{O_{1C_{\mathrm{xyz}}}}\\ z_{O_{1C_{\mathrm{xyz}}}}\end{array}\right);& \left(4\right)\end{array}$

(48) wherein ( z.sub.O.sub.1) are the absolute coordinate of the point O.sub.1 under t moment, ( Z.sub.O.sub.1) are the absolute coordinate of the point C.sub.1 under t moment,

(49) $\left(x_{O_{1C_{\mathrm{xyz}}}}{y}_{O_{1C_{\mathrm{xyz}}}}{z}_{O_{1C_{\mathrm{xyz}}}}\right)$
are the coordinate of point O.sub.1 in the coordinate system C.sub.xyz; similarly, calculating the absolute coordinate of the points Z.sub.1 under t moment;

(50) wherein, the vector corresponding to the absolute coordinate of point X.sub.1 at t moment is equal to the vector corresponding to the absolute coordinate of point C.sub.1 at t moment plus the vector of A.sub.t.sup.−1 multiplied by the coordinates of point X.sub.1 under the coordinate system C.sub.xyz; the absolute coordinate calculation method of point Y.sub.1 and point Z.sub.1 can be deduced by analogy;

(51) each frame of image is independent of each other, and corresponds to a moment respectively, and solves the absolute coordinates of the points Z.sub.1 of the connected base coordinate system at each moment frame by frame; calculate the angle θ between the vector {right arrow over (O.sub.1Z.sub.1)} at each moment and the Z axis in the public reference base coordinate system through the space vector angle formula:

(52) $\begin{array}{cc}\theta =\arccos \left(\frac{.Math.x_{Z_1}{x}_Z+y_{Z_1}{y}_Z+z_{Z_1}{z}_Z.Math.}{\sqrt{x_{Z_1}^2+y_{Z_1}^2+z_{Z_1}^2}\times \sqrt{x_Z^2+y_Z^2+z_Z^2}}\right);& \left(5\right)\end{array}$

(53) wherein {right arrow over (O.sub.1Z.sub.1)}=(x.sub.z.sub.1 y.sub.z.sub.1 z.sub.z.sub.1).sup.T, is the unit vector of the Z axis in the public reference base coordinate system, the angle θ between the vector {right arrow over (O.sub.1Z.sub.1)} and the Z axis in the public reference, base coordinate system represents a swing posture of the fruit.

(54) (c1) using the axis rotation matrix to rotate {right arrow over (O.sub.1Z.sub.1)} (that is, Z connected-axis) at the moment corresponding to a certain frame of image to a position that coincides with the Z axis in the public reference base coordinate system, calculating the angle between the rotated {right arrow over (O.sub.1X.sub.1)} and the X axis in the public reference base coordinate system, the angle represents a twisting posture of the fruit at the moment corresponding to the frame image, detail as follows:

(55) the Z connected-axis direction vector of the frame of image is: ={right arrow over (O.sub.1Z.sub.1)}=(x.sub.z.sub.1 y.sub.z.sub.1 z.sub.z.sub.1).sup.T, |{right arrow over (O.sub.1Z.sub.1)}|=√{square root over (x.sub.Z.sub.1.sup.2+y.sub.Z.sub.1.sup.2+z.sub.Z.sub.1.sup.2)}, calculating the angle φ between the Z connected-axis and the Z axis in the public reference base coordinate system:

(56) $\begin{array}{cc}\phi =\arccos \left(\frac{Z_{Z_1}}{.Math.\stackrel{.fwdarw.}{O_1{Z}_1}.Math.}\right);& \left(6\right)\end{array}$

(57) calculating the angle ψ between the projection of {right arrow over (O.sub.1Z.sub.1)} (that is, Z connected-axis) in the X axis and Y axis of the public reference base coordinate system and the Y axis of the public reference base coordinate system:

(58) $\begin{array}{cc}\psi =\arccos \left(\frac{y_{Z_1}}{\sqrt{y_{Z_1}^2+x_{Z_1}^2}}\right);& \left(7\right)\end{array}$

(59) calculating the axis rotation matrix:

(60) $\begin{array}{cc}{T}_1=\left(\begin{array}{ccc}\cos \phi & 0& \sin \phi \\ 0& 1& 0\\ -\sin \phi & 0& \cos \phi \end{array}\right);& \left(8\right)\end{array}$

(61) 0$\begin{array}{cc}{T}_2=\left(\begin{array}{ccc}\cos \psi & -\sin \psi & 0\\ \sin \psi & \cos \psi & 0\\ 0& 0& 1\end{array}\right);& \left(9\right)\end{array}$

(62) the absolute coordinate of the rotated point X.sub.1 is consistent with the coordinate of the rotated vector {right arrow over (O.sub.1X.sub.1)}, and the rotated vector {right arrow over (O.sub.1X.sub.1)} is:

(63) $\begin{array}{cc}\left(\begin{array}{c}{x}_{X_1}^{\prime }\\ y_{X_1}^{\prime }\\ z_{X_1}^{\prime }\end{array}\right)=T_2^{-1*}{T}_1^{-1*}\left(\begin{array}{c}{x}_{X_1}\\ y_{X_1}\\ z_{X_1}\end{array}\right);& \left(10\right)\end{array}$

(64) wherein, is the vector {right arrow over (O.sub.1X.sub.1)} before the {right arrow over (O.sub.1Z.sub.1)} of the frame of image is rotated;

(65) the torsion angular degree is:

(66) $\begin{array}{cc}\varphi =\arccos \left(\frac{.Math.x_{X_1^{\prime }}{x}_X+y_{X_1^{\prime }}{y}_X+z_{X_1^{\prime }}{z}_X.Math.}{\sqrt{x_{X_1^{\prime 2}}+y_{X_1^{\prime 2}}+z_{X_1^{\mathrm{\prime 2}}}}\times \sqrt{x_{X^2}+y_{X^2}+z_{X^2}}}\right);& \left(11\right)\end{array}$

(67) the torsion angular degree is the angle between the rotated vector {right arrow over (O.sub.1X.sub.1)} and the X axis in the public reference base coordinate system, which represents the torsion posture of the fruit at the moment corresponding to the frame image, wherein is the unit vector of the X axis in the public reference base coordinate system.

(68) (d1) Similarly, in step (c1), calculating the twisting posture of the fruit at each moment corresponding to each frame image.

(69) A study method of fruit space motion, the space motion of fruit needs to use public reference base coordinates to express its absolute motion. Setting the moments corresponding to two adjacent frames of images to t+1 moment and t moment, based on the point O.sub.1 of the connected base coordinate system in adjacent two frames of images, and calculating the displacement of point O.sub.1 along the X axis, Y axis and Z axis of the public reference base coordinate system at t+1 moment and t moment respectively. calculating an instantaneous speed along the X axis, an instantaneous speed along the Y axis, and an instantaneous speed along the Z axis of the fruit at t moment according to the displacement, calculating an instantaneous acceleration along the X axis, an instantaneous acceleration along the Y axis and an instantaneous acceleration along the Z axis of the fruit at t moment according to the instantaneous speed, calculating a swing angular displacement of the fruit in the time interval between two adjacent frames of images, sequentially calculating a swing angular speed of the fruit at t moment and a swing angular acceleration of the fruit at t moment via the swing angular displacement, and calculating the torsion angular speed of the fruit at t moment and the torsion angular acceleration of the fruit at t moment according to the torsion angular degree from t moment to t+1 moment. The fruit space motion process is divided into translation, swing and torsion, and the motion process is shown in FIG. 6. The movement of the point O.sub.1 along the X Y Z axis represents the translation of the fruit, the swing around the junction of the fruit stem and the fruit represents the swing of the fruit, and the twisting movement around the centerline (that is, Z connected-axis) represents the twist of the fruit. FIG. 7 shows the spatial translation trajectory of fruit, and the curve in FIG. 7 shows the movement of the point O.sub.1 in the space coordinates of the public reference base coordinate system.

(70) Wherein, the instantaneous speed moving along the X Y Z axis, the instantaneous acceleration moving along the X Y Z axis, the swing angular displacement, the swing angular speed, and the swing angular acceleration within the frame time interval, the torsion angular degree, the torsion angular speed, and the torsion angular acceleration within the frame time interval all indicate the tracked fruit space motion status. The specific calculation method is as follows:

(71) (a2) setting moments corresponding to two adjacent frames of images to t+1 moment and t moment, based on the point O.sub.1 of the connected base coordinate system in adjacent two frames of images, and calculating the displacement of the point O.sub.1 along the X axis of the public reference base coordinate system at t+1 moment and t moment respectively:
S.sub.x=x.sub.O.sub.t+1−x.sub.O.sub.1  (12);

(72) wherein x.sub.O.sub.1 are the coordinates x of the point O.sub.1 under the public reference base coordinate system at t+1 moment and t moment; similarly, calculating the displacement S.sub.y along the Y axis and the displacement S.sub.z along the Z axis of the point O.sub.1 of the public reference base coordinate system at t+1 moment and t moment, and the combined displacement is:
S=√{square root over (S.sub.x.sup.2+S.sub.y.sup.2+S.sub.z.sup.2)}  (13);

(73) (b2) using the average speed of the point O.sub.1 moving along the X axis of the public reference base coordinate system from t moment to t+1 moment to express the instantaneous speed v.sub.x(t) of the fruit along the X axis at t moment:

(74) $\begin{array}{cc}{v}_{x\left(t\right)}=\frac{S_x}{\Delta t};& \left(14\right)\end{array}$

(75) wherein Δt is the interval time between two frames of images; similarly, the instantaneous speed v.sub.y (t) of the fruit along the Y axis of the public reference base coordinate system at t moment and the instantaneous speed v.sub.z (t) of the fruit along the Z axis at t moment can be obtained, then the combined speed of the fruit at t moment is:
v=√{square root over (v.sub.x(t).sup.2+v.sub.y(t).sup.2+v.sub.z(t).sup.2)}  (15);

(76) (c2) using the acceleration of the point O.sub.1 moving along the X axis of the public reference base coordinate system from t moment to t+1 moment to express the instantaneous acceleration a.sub.x(t) of the fruit along the X axis at the t moment:

(77) $\begin{array}{cc}{a}_{x\left(t\right)}=\frac{v_{x\left(t+1\right)}-v_{x\left(t\right)}}{\Delta t};& \left(\mathrm{l6}\right)\end{array}$

(78) wherein v.sub.x (t+1) is the instantaneous speed of the fruit along the X axis at t+1 moment, and v.sub.x(t) is the instantaneous speed of the fruit along the X axis at t moment; similarly, the instantaneous acceleration a.sub.y (t) of the fruit along the Y axis of the public reference base coordinate system at t moment and the instantaneous acceleration a.sub.z (t) of the fruit along the Z axis of the public reference base coordinate system at t moment can be obtained; then the combined acceleration of the fruit at t moment is:
a=√{square root over (a.sub.x(t).sup.2+a.sub.y(t).sup.2+a.sub.z(t).sup.2)}  (17);

(79) (d2) since there are too many solving moments, this embodiment lists the positions of the connected base coordinate system of adjacent t+1 moment and t moment corresponding to two adjacent frames of images to indicate the swing and twist of the fruit. For convenience of representation, the origins of the conjoined base coordinates of two adjacent moments are overlapped, as shown in FIG. 8, the angle Δθ in FIG. 8 is the angle from t moment Z connected-axis to t+1 moment Z connected-axis, and Δθ represents the swing angular displacement of the fruit in Δt:

(80) $\begin{array}{cc}\Delta \theta =\arccos \left(\frac{.Math.x_{Z_{1_{t+1}}}{x}_{Z_{1_t}}+y_{Z_{1_{t+1}}}{y}_{Z_{1_t}}+z_{Z_{1_{t+1}}}{z}_{Z_{1_t}}.Math.}{\sqrt{x_{Z_{1_{t+1}}}^2+y_{Z_{z_{t+1}}}^2+z_{Z_{1_{t+1}}}^2}\times \sqrt{x_{Z_{1_t}}^2+y_{Z_{1_t}}^2+z_{Z_{1_t}}^2}}\right);& \left(18\right)\end{array}$

(81) wherein

(82) ${\left(x_{Z_{1_t}}{y}_{Z_{1_t}}{z}_{Z_{1_t}}\right)}^T$
is the direction vector of Z connected-axis at t moment, that is t moment vector {right arrow over (O.sub.1Z.sub.1)};

(83) ${\left(x_{Z_{1_{t+1}}}{y}_{Z_{1_{t+1}}}{z}_{Z_{1_{t+1}}}\right)}^T$
is the direction vector of Z connected-axis at t+1 moment, that is t+1 moment vector {right arrow over (O.sub.1Z.sub.1)};

(84) (e2) using the average swing angular speed from t moment to t+1 moment to express the t swing angular speed ω.sub.θ.sub.t of the fruit at t moment:

(85) $\begin{array}{cc}{\omega }_{{\theta }_t}=\frac{\Delta \theta }{\Delta t};& \left(19\right)\end{array}$

(86) (f2) using the amount of change of the swing angular speed from t moment to t+1 moment to express the swing angular acceleration a.sub.θ.sub.t of the fruit at t moment:

(87) $\begin{array}{cc}{a}_{{\theta }_t}=\frac{{\omega }_{{\theta }_{t+1}}-{\omega }_{{\theta }_t}}{\Delta t};& \left(20\right)\end{array}$

(88) (g2) FIG. 8 is a schematic diagram of the overlapping conjoined base coordinate origins of two adjacent moments. The angle Δθ between the Z connected-axis of t moment and t+1 moment in FIG. 8 represents the swing angular displacement of the fruit in Δt. However, the angle between the X connected-axis of t moment and t+1 moment, and the angle between the Y connected-axis of t moment and t+1 moment in FIG. 8 are not the torsion angular. Therefore, it is necessary to rotate the Z connected-axis of t moment (also expressed as {right arrow over (O.sub.1Z.sub.1)}) and the Z connected-axis of t+1 moment to the position that coincides with the Z connected-axis in the public reference base coordinate system through the axis rotation matrix. As shown in FIG. 9, the two Z connected-axes in this FIG. 9 coincide with the Z axis in the public reference base coordinate system. The angle between the two X connected-axis at this time is the torsion angular degree from t moment to t+1 moment, and the calculation method is: use the formula (6) to formula (10) in the above-mentioned step (c1) respectively calculate the rotated vector {right arrow over (O.sub.1X.sub.1)} at t moment and the rotated vector {right arrow over (O.sub.1X.sub.1)} at t+1 moment.

(89) similarly, in formula (10) of step (c1), the vector {right arrow over (O.sub.1X.sub.1)} rotated at t moment is:

(90) 0$\begin{array}{cc}\left(\begin{array}{c}{x}_{X_{1_t}}^{\prime }\\ y_{X_{1_t}}^{\prime }\\ z_{X_{1_t}}^{\prime }\end{array}\right)=T_{2_t}^{-1}*T_{1_t}^{-1}*\left(\begin{array}{c}{x}_{X_{1_t}}\\ y_{X_{1_t}}\\ z_{X_{1_t}}\end{array}\right);& \left(21\right)\end{array}$

(91) wherein

(92) ${\left(x_{X_{1_t}}{y}_{X_{1_t}}{z}_{X_{1_t}}\right)}^T$
is the vector {right arrow over (O.sub.1X.sub.1)} before the Z connected-axis rotation at t moment;

(93) ${\left(x_{X_{1_t}}^{\prime }{y}_{X_{1_t}}^{\prime }{z}_{X_{1_t}}^{\prime }\right)}^T$
is the vector {right arrow over (O.sub.1X.sub.1)} after the Z connected-axis rotation at t moment; T.sub.2.sub.t.sup.−1 and T.sub.1.sub.t.sup.−1 are the axis rotation matrices under t moment;

(94) the torsion angular speed degree of the fruit from t moment to t+1 moment is:

(95) $\begin{array}{cc}\Delta \varphi =\arccos \left(\frac{.Math.x_{X_{1_{t+1}}^{\prime }}{x}_{X_t}+y_{X_{1_{t+1}}^{\prime }}{y}_{X_t}+z_{X_{1_{t+1}}^{\prime }}{z}_{X_t}.Math.}{\sqrt{x_{X_{1_{t+1}}^{\prime 2}}+y_{X_{1_{t+1}}^{\prime 2}}+z_{X_{1_{t+1}}^{\prime 2}}}\times \sqrt{x_{X_t^2}+y_{X_t^2}+z_{X_t^2}}}\right);& \left(22\right)\end{array}$

(96) wherein

(97) ${\left(x_{X_{1_t}}^{\prime }{y}_{X_{1_t}}^{\prime }{z}_{X_{1_t}}^{\prime }\right)}^T$
is the rotated vector {right arrow over (O.sub.1X.sub.1)} at t moment,

(98) ${\left(x_{X_{1_{t+1}}}^{\prime }{y}_{X_{1_{t+1}}}^{\prime }{z}_{X_{1_{t+1}}}^{\prime }\right)}^T$
is the rotated vector {right arrow over (O.sub.1X.sub.1)} at t+1 moment;

(99) the torsion angular speed of the fruit at t moment is:

(100) $\begin{array}{cc}{\omega }_{{\varphi }_t}=\frac{\Delta \varphi }{\Delta t};& \left(23\right)\end{array}$

(101) the torsion angular acceleration of the fruit at t moment is:

(102) $\begin{array}{cc}{a}_{{\varphi }_t}=\frac{{\omega }_{{\varphi }_{t+1}}-{\omega }_{{\varphi }_t}}{\Delta t}.& \left(24\right)\end{array}$

(103) When the inertial force generated by the vibration of vibrating fruit picking exceeds the separation force between the fruit and the fruit stem, the fruit is separated from the fruit branch or the fruit stem at the weakest connection point to complete the picking. The effect of fruit separation ultimately depends on the maximum inertial force generated when the fruit vibrates. Inertial force comes from acceleration, so acceleration response is one of the most important characteristics in fruit tree dynamics. This method can construct the trajectory of the fruit in the vibration process through the relationship between the conjoined base at the junction of the fruit stem and the fruit, and then solve the displacement, speed and acceleration of fruit at each moment, which is beneficial to explore the law of fruit movement. The binocular-vision-based method for tracking fruit space attitude and fruit space motion, the system and analysis software for taking fruit vibration photography belong to the prior art, such as binocular cameras and spatial position synthesis software.

(104) The scope of protection of the present invention includes, but is not limited to, the above-mentioned embodiments. The scope of protection of the present invention is subject to the claims. Any substitutions, modifications, and improvements that can be easily conceived by those skilled in the art made to the present invention fall within the scope of protection of the present invention.