Abstract
Provided is a method for generating teacher data for image recognition while reducing the number of images used as the basis. A captured image is obtained by imaging an object by an imaging device C arranged at a first designated position P.sub.i. A basic image region S.sub.iis extracted from the captured image. The teacher data is generated as a result of coordinate transformation of the basic image region S.sub.i from one image coordinate system to a coordinate system of a captured image by the imaging device C on the assumption that the imaging device C is arranged at a second designated position P.sub.j which is different from the first designated position P.sub.i.
Claims
1. A method for generating teacher data for image recognition of an object, comprising: a step of obtaining a plurality of captured images by imaging the object by an imaging device arranged at each of a plurality of first designated positions; a step of extracting a basic image region from each of the plurality of captured images; and a step of generating the teacher data as a result of coordinate transformation of the basic image region from one image coordinate system to each coordinate system of the plurality of captured images by the imaging device on an assumption that the imaging device is arranged at each of a plurality of second designated positions which are different from the plurality of first designated positions.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0009] FIG. 1 illustrates an explanatory diagram showing a producing method of teacher data as an embodiment of the present invention.
[0010] FIG. 2 is an explanatory diagram illustrating an imaging position of an imaging device C.
[0011] FIG. 3A and FIG. 3B are explanatory diagrams in which FIG. 3A illustrates a method of changing a camera position, and FIG. 3B illustrates internal parameter setting of the camera.
[0012] FIG. 4A and FIG. 4B are explanatory diagrams in which FIG. 4A illustrates images of an object actually captured from a plurality of different positions, and FIG. 4B illustrates estimation images of the object on an assumption that the object is captured from the plurality of different positions.
[0013] FIG. 5 is an explanatory diagram illustrating an evaluation result of a deviation between the actual captured images and the estimation images.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] A method for generating teacher data for image recognition as an embodiment of the present invention will be described.
[0015] First, a first index i which expresses the number of captured images used as the basis is set to “1” (FIG. 1/STEP11). The position P of an imaging device C is adjusted to a first designated position P.sub.i (FIG. 1/STEP12). The posture of the imaging device C is adjusted so that the optical axis thereof is directed to one point of an actual space polar coordinate system.
[0016] For example, as shown in FIG. 2, on a hemisphere face of a radius R, having the origin of the three-dimensional polar coordinate system as the center and having a pole above the center, in addition to the pole P.sub.1, four points P.sub.2 to P.sub.5 which are arranged at equal intervals in the longitude direction at a latitude expressed as θ.sub.i=θ(20°≦θ≦70°), are defined as the first designated positions. In this case, the first designated position P.sub.i is expressed by coordinate values of the three-dimensional polar coordinate system (r.sub.i sin θ.sub.i cos φ.sub.i, r.sub.i sin θ.sub.i sin φ.sub.i, r.sub.i cos θ.sub.i). The position and the posture of the imaging device C is adjusted manually by the operator, and then fixed by an appropriate tool such as the platform or the like, or may be automatically adjusted by a driving device such as a robot arm.
[0017] Next, the Object is captured by the imaging device C at position P to obtain the captured image (FIG. 1/STEP13),
[0018] A basic image region S.sub.i is extracted from the captured image (FIG. 1/STEP14). For example, a person (an operator) manipulates through an input interface while visually recognizing the captured image displayed through an output interface to extract a region recognized as being a substantially flat surface as the basic image region S.sub.i. The extraction of the basic image region S.sub.i is performed only for a few numbers of arbitrary captured images. The distance r between the optical center of the imaging device and the basic image region S.sub.i is measured by an appropriate range sensor.
[0019] It is determined whether or not the first index i is a first specified number N.sub.1 or more (FIG. 1/STEP15). The first specified number N.sub.1 is, for example, set to “5”. If it is determined that the first index i is less than the first specified number N.sub.1 (FIG. 1/STEP15 . . . NO), the first index i is increased by “1” (FIG. 1/STEP16). And then, the processes after the adjustment of the position and the posture of the imaging device C are repeated (FIG. 1/STEP12 to STEP15). Each of a plurality of imaging devices C can be arranged at each of the plurality of designated positions, and used.
[0020] If it is determined that the first index i is equal to or more than the first specified number N.sub.1 (FIG. 1/STEP15 . . . YES), a second index j expressing the number of generated estimation images, is set to “1” (FIG. 1/STEP21). The position P of a virtual imaging device C is adjusted to a second designated position P.sub.j which is different from the first designated position P.sub.i (FIG. 1/STEP22). For example, the second designated position P.sub.j is also defined on a hemisphere face similar to the first designated position P.sub.i (refer to FIG. 2).
[0021] More specifically, in addition to the optical center P=e (three-dimensional vector) of the imaging device C as shown in FIG. 3A, by using unit vector 1=(c-e)/|c-e| expressing the azimuth of the center of the image (basic image region S.sub.i) with the optical center P of the image device C as the reference, unit vector u′=s×1 and unit vector s=1×u expressing the upper direction of the imaging device C after the optical center of the imaging device C is moved, the coordinate transformation matrix M.sub.1 which changes the optical center position P of the imaging device C is defined by expression (1).
[00001]
[0022] In addition to the lower limit value d.sub.n and the upper limit value d.sub.f of the depth of field of the imaging device C as shown in FIG. 3B, by using the angle of view θ.sub.y in the vertical direction, the aspect ratio a of the angle of view θ.sub.x in the horizontal direction with respect to the angle of view θ.sub.y in the vertical direction, and f=1 / tan (θ.sub.y / 2), the transformation matrix M.sub.2 based on internal parameter of the imaging device C is defined by expression (2).
[00002]
[0023] As a result of coordinate transformation of the basic image region S.sub.i according to expression (3), S.sub.î is generated as one of the teacher data (FIG. 1/STEP23).
[0024] [Expression 3]
S.sub.î=M.sub.2M.sub.1S.sub.i (3)
[0025] It is determined whether or not the second index j is a second specified number N.sub.2 or more (FIG. 1/STEP24). The second specified number N.sub.2 is set to a sufficient number necessary as the teacher data for image recognition, for example “7000”. If it is determined that the second index j is less than the second specified number N.sub.2 (FIG. 1/STEP24 . . . NO), the second index j is increased by “1” (FIG. 1/STEP25). And then, the processes after the adjustment of the position and the posture of the imaging device C are repeated (FIG. 1/STEP22 to STEP24). Then, if the second index j is determined to be equal to or more than the second specified number N.sub.2 (FIG. 1/STEP24 . . . YES), the series of processing terminates. In addition to the basic image region obtained as above, the estimation image group as calculated above is accumulated in the database as the teacher data.
[0026] After that, the basic image region is extracted from the captured image obtained under an arbitrary environment, and the extraction result is collated with or used for pattern matching with the teacher data accumulated in the database, thereby recognizing that an object related to the teacher data exists in the actual space corresponding to the basic image region.
EXAMPLE
[0027] In a three-dimensional coordinate system having a surface center of an object having a substantially rectangular flat plate shape as the origin and its surface being a part of the x-y plane, the object was imaged by intermittently changing the latitude (or the elevation angle in the three-dimensional polar coordinate system) of the position of the imaging device C on the hemisphere face having a pole on the z axis, while keeping the longitude constant. FIG. 4A shows the actual captured images in such case in order from the left. FIG. 4B shows the estimation images according to the above method in the case it is assumed that the same object is imaged by changing the position of the imaging device C similarly from the left.
[0028] FIG. 5 shows by plotting the relation between an elevation angle θ expressing an actual (or virtual) position of the imaging device C and an angle in the image coordinate system of one corner angle of the substantially rectangular flat plate shaped object in each of the actual captured image and the estimation image. It is assumed to generate the estimation image in the range of 20°≦θ≦160°. As shown in FIG. 5, although there is a tendency that the deviation of the corner angles gradually becomes larger as θ becomes larger (as the position of the imaging device C comes closer to the x-y plane), the maximum relative deviation is 1.9%. Therefore, it is verified that the recognition accuracy of the object is improved by using the estimation image group as the teacher data.
EXPLANATION OF THE REFERENCE SIGNS
[0029] C . . . imaging device, P.sub.1 to P.sub.5 . . . first designated positions, S.sub.i . . . basic image region
