Collecting device for training data

Abstract

A first standard deviation of feature quantities of all pieces of non-expert data stored in a non-expert data storage unit 13 is calculated, and a second standard deviation of feature quantities of all pieces of expert data stored in an expert data storage unit 14 is calculated. In addition, a first rank sum of the feature quantities of all pieces of non-expert data stored in the non-expert data storage unit 13 is calculated, and a second rank sum of the feature quantities of all pieces of expert data stored in the expert data storage unit 14 is calculated. Then, a continuation and an end of acquisition of defective product data by an expert are determined, based on the first standard deviation and the second standard deviation , and the first rank sum and the second rank sum .

Claims

1. A collecting device for training data that collects defective product data including an external appearance image of an inspected object to be an abnormal product, as the training data for use in learning by a predetermined learning model, the collecting device comprising: a processor; and a memory, including instructions stored thereon, which when executed by the processor, cause the collecting device to perform the functions of: a non-expert defective product data acquisition unit configured to acquire the defective product data in accordance with a selection by a non-expert; an expert defective product data acquisition unit configured to acquire the defective product data in accordance with a selection by an expert; a non-expert data storage unit configured to store, as non-expert data, the defective product data that has been acquired by the non-expert defective product data acquisition unit; an expert data storage unit configured to store, as expert data, the defective product data that has been acquired by the expert defective product data acquisition unit; a first standard deviation calculation unit configured to calculate, as a first standard deviation, a standard deviation of feature quantities of all pieces of the non-expert data stored in the non-expert data storage unit; a second standard deviation calculation unit configured to calculate, as a second standard deviation, a standard deviation of feature quantities of all pieces of the expert data stored in the expert data storage unit; a first rank sum calculation unit configured to calculate, as a first rank sum, a rank sum of the feature quantities of all pieces of the non-expert data stored in the non-expert data storage unit; a second rank sum calculation unit configured to calculate, as a second rank sum, a rank sum of the feature quantities of all pieces of the expert data stored in the expert data storage unit; a determination unit configured to determine a continuation and an end of acquisition of the defective product data by the expert defective product data acquisition unit, based on the first and second standard deviations and the first and second rank sums, wherein the determination unit is configured to calculate a determination coefficient based on the first and second standard deviations and the first and second rank sums, and to terminate the acquisition of defective product data by the expert defective product data acquisition unit when the determination coefficient is equal to or greater than a predetermined value; and a generation unit configured to generate the learning model as a classification model based on the defective product training data and the non-defective product training data.

2. The collecting device for the training data according to claim 1, wherein in a case where denotes the first standard deviation, denotes the second standard deviation, denotes the first rank sum, and denotes the second rank sum, the determination unit determines the end of the acquisition of the defective product data by the expert defective product data acquisition unit, in a case where a determination coefficient Vrank calculated in a following expression (1) is equal to or greater than a predetermined value: $\left[\mathrm{Expression}1\right]$ $\begin{array}{cc}\mathrm{Vrank}=1-\frac{1}{{\left(\frac{}{}\right)}^2{\left(\frac{}{}\right)}^2}.& \left(1\right)\end{array}$

Description

BRIEF DESCRIPTION OF DRAWINGS

(1) FIG. 1 is a diagram for describing an outline of an inspection system in which training data that has been collected by a collecting device for training data according to an embodiment of the present invention is used for learning;

(2) FIG. 2 is a block diagram illustrating the collecting device for training data according to an embodiment of the present invention;

(3) FIG. 3 is a flowchart illustrating collection processing of training data by the collecting device for training data; and

(4) FIG. 4 is a flowchart illustrating generation processing of a classification model.

DETAILED DESCRIPTION

(5) Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 illustrates an inspection system including a learning model by which learning has been performed by use of data of a large number of defective product images (defective product data) that has been collected by a collecting device 11 for training data to be described later and data of a large number of non-defective product images (non-defective product data). Such an inspection system 1 is installed in, for example, a manufacturing factory of vehicle components, and by inspecting the appearance of a vehicle component, automatically determines whether a manufactured vehicle component (for example, a cylinder block) is a normal product (non-defective product) or an abnormal product (defective product). Hereinafter, a vehicle component to be inspected will be referred to as an inspected object.

(6) As illustrated in FIG. 1, the inspection system 1 includes a conveyor 2 for conveying an inspected object G in a predetermined direction at a predetermined speed, and an inspection device 3 for determining the quality of the inspected object G, when the inspected object G reaches a predetermined inspection position. Note that the illustration is omitted, but the inspected object G that has been determined to be a defective product by the inspection device 3 is removed from the conveyor 2, or is conveyed to a storage place dedicated to the defective products.

(7) The inspection device 3 is configured with an information processing device mainly including a computer, and includes a control unit 4, an image acquisition unit 5, a storage unit 6, a learning unit 7, an input unit 8, an output unit 9, and a camera 10.

(8) The control unit 4 includes a CPU, and controls the above respective units 5 to 9 of the inspection device 3, and the camera 10. The image acquisition unit 5 acquires, as digital data, an external appearance image of the inspected object G that has been imaged by the camera 10. The storage unit 6 includes a ROM and a RAM, stores various programs to be used in the control of the inspection device 3, and also stores various types of data. The learning unit 7 includes a learning model by which criteria for determining the quality of the inspected object G have been learned. The input unit 8 includes a keyboard and/or a mouse to be operated by an operator, and in addition, is configured so that data and/or signals can be input from the outside. The output unit 9 includes a display device such as a display on which a determination result of the inspected object G is displayed.

(9) FIG. 2 illustrates a collecting device 11 for training data according to an embodiment of the present invention. The collecting device 11 for training data is operated by an operator who performs an inspection operation of the inspected object G so as to collect defective product data. Similarly to the inspection device 3 described above, the collecting device 11 for training data is configured with an information processing device including a computer, and includes a defective product image acquisition unit 12 (non-expert defective product data acquisition unit, expert defective product data acquisition unit), a non-expert data storage unit 13, an expert data storage unit 14, a feature quantity conversion unit 15, a standard deviation calculation unit 16 (first standard deviation calculation unit, second standard deviation calculation unit), a rank sum calculation unit 17 (first rank sum calculation unit, second rank sum calculation unit), a determination coefficient Vrank calculation unit 18, and a defective product image acquisition continuation and end determination unit 19 (determination unit).

(10) Regarding an external appearance image of the inspected object G that has been imaged by a camera similar to the camera 10 of the inspection device 3 described above, the defective product image acquisition unit 12 acquires, as defective product data, the external appearance image that has been determined to be a defective product by the operator.

(11) The non-expert data storage unit 13 stores defective product data (non-expert data) that has been selected by non-experts (newcomers or operators with short years of experience in the inspection operation). On the other hand, the expert data storage unit 14 stores defective product data (expert data) that has been selected by experts (skilled persons or operators with long years of experience in the inspection operation).

(12) The feature quantity conversion unit 15 converts the defective product data into a predetermined feature quantity. Specifically, the defective product data is converted into the feature quantity by use of, for example, scale-invariant feature transform (SIFT) or convolution neural network (CNN).

(13) The standard deviation calculation unit 16 calculates a standard deviation of feature quantities for many pieces of defective product data respectively stored in the non-expert data storage unit 13 and the expert data storage unit 14. In addition, the rank sum calculation unit 17 calculates a rank sum of the feature quantities for many pieces of defective product data described above.

(14) As will be described later, the determination coefficient Vrank calculation unit 18 calculates a determination coefficient Vrank for determining a continuation or an end of acquisition of the defective product image by using four parameters including a standard deviation and a rank sum of the non-expert data and a standard deviation and a rank sum of the expert data.

(15) The defective product image acquisition continuation and end determination unit 19 determines whether to continue or end the acquisition of the defective product image in accordance with the determination coefficient Vrank that has been calculated.

(16) FIG. 3 illustrates collection processing of a defective product image serving as training data by the collecting device 11 for training data described above. In the present processing, first, step 1 (indicated as S1 in the drawing. Hereinafter, the same will apply), a defective product image is acquired.

(17) Next, in step 2, it is determined whether the defective product image that has been acquired is a selection by a non-expert. For example, while a non-expert is operating the collecting device 11 for training data, it is determined that the defective product image that has been acquired is the selection by the non-expert, and while an expert is operating the collecting device 11 for training data, it is determined that the defective product image that has been acquired is the selection by the expert. In a case where a determination result in step 2 is YES, the image that has been acquired is stored in the non-expert data storage unit 13 (step 3). Note that in a case where a non-expert selects a defective product image, and has never seen the selected defective product image before, such a defective product image is regarded as an outlier, and is not stored in the non-expert data storage unit 13 in order to adjust the condition with the expert data.

(18) Next, the feature quantity conversion unit 15 converts all the images in the non-expert data storage unit 13 respectively into feature quantities (step 4). Note that the defective product image other than the defective product image currently stored in the non-expert data storage unit 13 has already been converted into the feature quantity beforehand. Therefore, in step 4, only the defective product image acquired this time is converted into a feature quantity. Then, the standard deviation calculation unit 16 and the rank sum calculation unit 17 respectively calculate a standard deviation (first standard deviation) and a rank sum (first rank sum), based on the feature quantities of all the images stored in the non-expert data storage unit 13 (step 5).

(19) On the other hand, in a case where the determination result in step 2 is NO and the defective product image acquired this time is the selection by an expert, the acquired image is stored in the expert data storage unit 14 (step 6). Next, all the images in the expert data storage unit 14 are respectively converted into feature quantities in the same manner as in step 4 (step 7). Then, the standard deviation calculation unit 16 and the rank sum calculation unit 17 respectively calculate a standard deviation (second standard deviation) and a rank sum (second rank sum), based on the feature quantities of all the images stored in the expert data storage unit 14 (step 8).

(20) After step 5 or step 8 ends, the determination coefficient Vrank calculation unit 18 uses the four parameters calculated in steps 5 and 8, that is, the standard deviation , the standard deviation , the rank sum , and the rank sum , calculates a determination coefficient Vrank by using a following expression (1).

(21) $\left[\mathrm{Expression}1\right]$$\begin{array}{cc}\mathrm{Vrank}=1-\frac{1}{{\left(\frac{}{}\right)}^2{\left(\frac{}{}\right)}^2}& \left(1\right)\end{array}$

(22) The determination coefficient Vrank, which is calculated by use of a standard deviation ratio (/) between the standard deviations and in the above expression (1) and a rank sum ratio (/) between the rank sums and , approaches 1 in value, as the standard deviation ratio and the rank sum increase. In a case where the determination coefficient Vrank becomes equal to or greater than a predetermined value VREF to be described later, it becomes possible to determine that the non-expert data and the expert data that have been collected as the training data are not concentrated in the vicinity of the average in the entire data and are in an appropriately distributed state.

(23) In step 10, it is determined whether the determination coefficient Vrank is equal to or greater than the predetermined value VREF. In a case where the determination result is NO and the determination coefficient Vrank is smaller than the predetermined value VREF, it is determined that the expert data has not been sufficiently prepared yet, and the processing returns to step 1 to continue acquiring the defective product image.

(24) On the other hand, in a case where the determination result in step 10 is YES and VrankVREF is satisfied, the expert data has been sufficiently prepared, the collection of the defective product image by the expert should be completed, 1 is set to the expert data acquisition completion flag F_COMP (step 11), and the collection processing for training data ends. Note that by setting 1 to the flag F_COMP, in the collecting device 11 for training data, the collection of the training data having been completed is notified on a display unit, not illustrated, or the like.

(25) FIG. 4 illustrates generation processing of a classification model serving as a learning model to be mounted on the learning unit 7 of the inspection device 3 in the inspection system 1 described above. In the present processing, all pieces of data (expert data) are output from the expert data storage unit 14 of the collecting device 11 for training data (step 21), and in addition, predetermined pieces of data (non-expert data) that are all or more than the pieces of expert data are output from the non-expert data storage unit 13 (step 22). Next, the expert data and the non-expert data that have been output are combined (step 23). Accordingly, many pieces of defective product training data to be used for generating the classification model are created.

(26) Then, learning of the classification model is performed by use of many pieces of defective product training data that have been created and many pieces of non-defective product data that have already been collected (non-defective product training data) (step 24). Accordingly, the classification model with high classification accuracy is obtainable, and in the inspection system 1, the quality of the inspected object G can be determined with accuracy.

(27) Heretofore, as described in detail, according to the present embodiment, the continuation and the end of the acquisition of the defective product image to be selected by the expert are determined by use of the standard deviation and the rank sum based on the feature quantities of the non-expert data and the standard deviation and the rank sum based on the feature quantities of the expert data. Therefore, it is possible to collect the training data for generating the learning model with high classification accuracy, while minimizing expert data.

(28) Note that the present invention is not limited to the above-described embodiments, and can be implemented in various modes. For example, in an embodiment, the continuation and the end of the collection of the defective product image by the expert is determined by use of the determination coefficient Vrank. However, the continuation and the end of the collection of the defective product image by the expert may be determined by respectively comparing the standard deviation ratio (/) between the first standard deviation and the second standard deviation ratio and the rank sum ratio (/) between the first rank sum and the second rank sum with predetermined reference values. In addition, the detailed configuration and the like of the collecting device 11 for training data, which have been described in the embodiments are merely examples, and can be appropriately changed within the scope of the gist of the present invention.