Method for checking for completeness

Abstract

A method for checking the completeness of a container provided with a plurality of objects by means of a 3D camera includes providing a three-dimensional image of the objects and a multi-ROI. The multi-ROI includes a plurality of ROIs which have a user-settable shape and are arranged within the multi-ROI in a user-settable number of rows, columns and raster type, including a learning mode and a temporally subsequent working mode.

Claims

1. A method for completeness check of a container provided with a plurality of objects by means of a 3D camera which provides a three-dimensional image of the objects and a multi-region of interest, wherein the multi-region of interest includes a plurality of regions of interest comprising a user-settable shape and arranged within the multi-region of interest in a user-settable number of rows, columns and raster type, including a learning mode and a temporally subsequent working mode, wherein the learning mode comprises the following steps: capturing an image of the objects in the form of a three-dimensional pixel matrix by means of the 3D camera, wherein the pixel matrix in one dimension comprises distance values of the detected objects with respect to the 3D camera and in the two other dimensions comprises position values in respective planes perpendicular thereto; reproducing the captured image as a two-dimensional image with the position values in a first image area; displaying input fields for the shape or the number of rows, columns and raster type of the regions of interest; displaying a two-dimensional multi-region of interest with a user-entered number of rows, columns and raster type of the regions of interest and a user-entered shape of the regions of interest in the first image area; adjusting the size, position and skew of the multi-region of interest in response to at least one user input; adjusting the size of the regions of interest in response to at least one user input; displaying height values derived from the distance values for each region of interest in a second image area which is different from the first image area (8); displaying at least one of a lower limit and an upper limit for the height values in the second image area; adjusting at least one of the lower limit and the upper limit for the height values in response to at least one input of the user; and switching into the working mode, wherein the working mode comprises the following step: indicating a state of a respective region of interest, wherein the state of the respective region of interest is at least one of the states overfilled, underfilled and good, and wherein the state overfilled is indicated when the height value of the respective region of interest is above the upper limit, the state underfilled is indicated when the height value of the respective region of interest is below the lower limit, and the state good is indicated when the height value is not outside an intended limit.

2. The method according to claim 1, wherein the height values derived from the distance values for each region of interest in the second image area are displayed as lines with lengths corresponding to the respective height value.

3. The method according to claim 2, wherein the lower limit and the upper limit of the height values are displayed as boundaries extending perpendicular to the lines.

4. The method according to claim 3, wherein the boundaries can be shifted by the user by means of a cursor or a finger gesture.

5. The method according to claim 1, wherein in the learning mode when reproducing the captured image as a two-dimensional image with the position values in the first image area the distance data are indicated coded as colors or gray values.

6. The method according to claim 1, wherein in the learning mode the two-dimensional multi-region of interest is automatically generated based on the number of rows and columns input by the user such that all rows have the same row width and all columns have the same column width.

7. The method according to claim 1, wherein the shape of the regions of interest is the same for all regions of interest.

8. The method according to claim 1, wherein in the working mode the states of the respective regions of interest in the first image area are indicated coded as colors or gray values.

9. The method according to claim 1, wherein in the working mode the states of the respective regions of interest are displayed in an image area different from the first image area, preferably in the form of a list.

10. The method according to claim 1, wherein a time of flight camera is used as a 3D camera.

Description

DRAWINGS

(1) The drawings described herein are for illustrative purposes only of selected embodiments and not all possible implementations, and are not intended to limit the scope of the present disclosure.

(2) In the drawings:

(3) FIG. 1 schematically shows the setup for carrying out a method for completeness check according to a preferred exemplary embodiment of the disclosure;

(4) FIG. 2 shows a screen display for the method for completeness check according to the preferred exemplary embodiment of the disclosure in a first method step;

(5) FIG. 3 shows a screen display of the method for completeness check according to the preferred exemplary embodiment of the disclosure in a further method step;

(6) FIG. 4 shows a screen display of the method for completeness check according to the preferred exemplary embodiment of the disclosure in a further method step;

(7) FIG. 5 shows a screen display of the method for completeness check according to the preferred exemplary embodiment of the disclosure in a further method step;

(8) FIG. 6 shows a screen display for the method for completeness check according to the preferred exemplary embodiment of the disclosure in a further method step;

(9) FIG. 7 shows a screen display of the method for completeness check according to the preferred exemplary embodiment of the disclosure in a further method step;

(10) FIG. 8 shows a screen display for the method with a honeycomb-like arrangement of the detection areas and a short, second honeycomb row;

(11) FIG. 9 shows a screen display for the method with a honeycomb-like arrangement of the detection areas and a long, second honeycomb row;

(12) FIG. 10 shows a screen display for the method with a honeycomb-like arrangement of the detection areas and a second honeycomb row displaced to the right;

(13) FIG. 11 shows a screen display for the method with a honeycomb-like arrangement of the detection areas and a second honeycomb row displaced to the left;

(14) FIG. 12 shows a screen display for the method with a honeycomb-like arrangement of the detection areas and a plurality of selection fields for rows, columns and raster type.

(15) Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings.

DETAILED DESCRIPTION

(16) Example embodiments will now be described more fully with reference to the accompanying drawings.

(17) FIG. 1 schematically shows the setup for performing a method for completeness check according to a preferred exemplary embodiment of the disclosure. On a conveyor belt 1 containers 2 are conveyed, each of which should be filled correctly with twelve objects 3. The containers 2 are open boxes filled with objects 3 in the form of milk cartons.

(18) By means of a 3D camera 4, which is arranged above the conveyor belt 1, a respective container 2 provided with objects 3 can be optically detected. Here, the 3D camera 4 provides a three-dimensional image in the form of a three-dimensional pixel matrix, wherein the pixel matrix includes distance values of the detected objects 3 with respect to the 3D camera 4 in one dimension and position values in respective planes perpendicular thereto in the other two dimensions. This three-dimensional image captured by the 3D camera 4 is transmitted to a display device 5 in the form of a screen and can be displayed there, as explained in detail below. According to the preferred embodiment of the disclosure described herein, a keyboard 6 and a pointing device 7, such as a mouse, are connected to the display device 5 in order to enable a user to make inputs. Alternatively, the display device 5 may be configured as a touch display, which makes a keyboard and a mouse dispensable.

(19) The sequence of a method according to the presently described preferred embodiment of the disclosure is as follows:

(20) First, the learning mode of the method is carried out. For this purpose, in a state where the conveyor belt is switched off, that is to say when the container 2 is stationary, an image 9 of the objects 3 and the container 2, which are located below the 3D camera 4, is captured. The captured image 9 of the objects 3 and the container 2 is then displayed in a first image area 8 on the display device 5, as shown in FIG. 2. FIG. 2 also shows that a field 10 for indicating the columns and a field 11 for indicating the rows is shown for the user of the present method. Due to the present input of 4 for the columns and 3 for the rows, a multi-ROI 12 with a total of twelve ROIs 13 corresponding to the four columns and three rows is automatically created and displayed in the first image area 8. In this case, the column widths are automatically selected to be all identical, and also the row widths are automatically set to be all identical. That is, according to the presently described preferred exemplary embodiment of the disclosure, the multi-ROI is symmetrically divided into ROIs. In other words, in this step, the ROIs are optimally distributed across the multi-ROI.

(21) FIGS. 1 to 7 show the method according to the disclosure on the basis of a regular structure, i.e. a structure where the number of columns is the same for all rows. However, the method according to the disclosure can also be applied in the same way to a honeycomb structure according to FIGS. 8 to 12. In addition, the method is also applicable to structures with a different number of columns per row, wherein the ROI may also be distributed irregularly. Such arbitrary structures can then be detected, for example, by individually shifting the ROIs 13.

(22) In the next step, which can be extracted from FIG. 3, the user of the method can select the shape of the ROIs. To this end, the shapes of a square, a rectangle, a circle and an ellipse within a corresponding button are offered to the user. By selecting the circular shape by means of the mouse pointer 15, the ROIs 13 in the first image area 8 are given a circular shape.

(23) In the next step of the method shown in FIG. 4, the position and size of the multi-ROI 12 are now adapted to the reproduced image 9. This is done with the aid of the mouse pointer 15, by means of which by clicking on the multi-ROI 12 a shifting of the multi-ROI 12 and a change of its size is possible.

(24) In the subsequent step, which is shown in FIG. 5, following the adaptation of the size and the position of the multi-ROI the skew of the multi-ROI 12 is now adapted to the image 9, too. The edge of the multi-ROI 12 thus corresponds to the edge contour of the image 9, which is formed by the container 2. The individual ROIs 13 now lie exactly above the areas of the images of the objects 3 in the container 2 to be detected. In addition to the size of the multi-ROIs 12, moreover, the sizes of the individual ROIS 13 can be changed by means of the mouse pointer 15. The size of the individual ROIs 13 is preferably dimensioned such that they cover 80% of the images of the objects 3 to be captured.

(25) It can now be seen from FIG. 6 that in a second image area 16 different from the first image area 8 for each ROI 13 height values derived from the distance values are displayed, namely in the form of lines corresponding to the height values 17. Perpendicularly to these lines 17 an upper boundary 18 for an upper limit of the height values and a lower boundary 19 for a lower limit for the height values are displayed. These boundaries 18, 19 can be moved up and down by the user by means of the mouse pointer 15. In this way, the switching points can be adjusted in order to indicate different states of a respective ROI 13. If a respective height value of a ROI 13 lies on the upper or the lower limit or in between, this corresponds to the state good. However, if the height value is above the upper limit, the state overfilled is present, and if the height value is below the lower limit, the state underfilled is present. In addition, a state invalid is possible, namely when no reliable height value has been detected.

(26) In the present case, which can be seen in FIG. 6, an object is missing in the leftmost column in the second row. For this reason, only a small height value is determined for the corresponding ROI, which is reflected in a correspondingly shorter line 17 in the second image area 16. This line 17 is below the lower boundary 19, so that the state underfilled is detected here.

(27) As finally shown in FIG. 7, this can be visualized in the working mode by displaying the corresponding ROI 13 in the first image area 8 in a different color. In addition, a table representation 20 is provided here outside the first image area 8, from which it can be seen that the ROI 13 with the consecutive number 5 has the state underfilled.

(28) FIG. 8 shows a multi-ROI 12 for a honeycomb-like structure. The ROIs 13 are already displayed circularly in the image area 8. The illustrated structure has four rows, wherein the first and third rows each have 5 columns, and the second and fourth rows each have 4 columns. Such a structure is particularly advantageous for round products in order to achieve a high packing density. In order to specify such a honeycomb-like structure it is sufficient, for example, to specify the number of rows, the maximum number of columns, and preferably a characteristic of the second row. In the case illustrated in FIG. 8, the second row is characterized in that it is shorter than the first row. The structure can then be specified as follows: rows 4, columns 5, raster type short.

(29) FIG. 9 shows a distribution of the ROIs 13 which is complementary to FIG. 8 and in which the second row is longer than the first one. This structure can preferably be specified as follows: rows 4, columns 5, raster type long.

(30) FIGS. 10 and 11 show further embodiments in which all rows have the same number of columns, but in FIG. 10 the second row is offset to the right and in FIG. 11 to the left. The structures according to FIGS. 10 and 11 accordingly are denoted by the specification rows 4, columns 5, raster type, right or raster type left.

(31) FIG. 12 shows a display device analogous to the display device according to FIG. 2 in which, in addition to the fields for columns 10 and rows 11, a further field 21 for selecting the present structure or raster type is provided. As already described above, options for the raster type are: short, long, right, left and regular for the case where no honeycomb structure is present.

(32) The remaining elements correspond to those already presented in FIG. 2, and the subsequent adaptation of the ROI, too, is implemented analogously to the procedure for the regular structure in accordance with the explanations relating to FIGS. 2 to 7.

(33) The selection options of the raster type are of course not restricted to the examples shown. It is also conceivable to provide other arrangements in the selection menu. In particular, it is also conceivable to provide a selection arbitrary which, for example, in further method steps, offers the possibility to place the ROIs individually.

(34) The foregoing description of the embodiments has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but, where applicable, are interchangeable and can be used in a selected embodiment, even if not specifically shown or described. The same may also be varied in many ways. Such variations are to be regarded as a departure from the disclosure, and all such modifications are intended to be included within the scope of the disclosure.