READING AN OPTICAL CODE

Abstract

A method of reading an optical code (20) is provided in which image data with the code (20) are recorded by an image sensor (24), a first processing unit (30) reads the image data from the image sensor (24) and generates metadata with respect to the image data and/or to the code (20) in the course of a pre-processing of the image data, and a second processing unit (32) reads the code (20) after the pre-processing by the first processing unit (30), The second processing unit (32) reads the code (20) solely from the metadata without making use of the image data.

Claims

1. A method of reading an optical code in which image data with the code are recorded by an image sensor, a first processing unit reads the image data from the image sensor and generates metadata with respect to the image data and/or to the code in the course of a pre-processing of the image data, and a second processing unit reads the code after the pre-processing by the first processing unit, wherein the second processing unit reads the code solely from the metadata without making use of the image data.

2. The method in accordance with claim 1, wherein the metadata comprise edge positions of black and white transitions in the image data.

3. The method in accordance with claim 2, wherein the metadata comprise edge positions at least initially for a horizontal direction and a vertical direction.

4. The method in accordance with claim 2, wherein edge positions are counted in the horizontal direction and in the vertical direction and a preferred direction of the code is determined using the edge positions with the higher number.

5. The method in accordance with claim 2, wherein the edge positions are determined multiple times.

6. The method in accordance with claim 5, wherein the edge positions are determined multiple times once per line and/or column of the image data.

7. The method in accordance with claim 5, wherein respective representative edge positions for the horizontal and/or vertical directions are determined from the multiple edge positions with respect to a plurality of lines and/or columns.

8. The method in accordance with claim 7, wherein the lines and/or columns are displaced against one another for the location of representative edge positions to align the respective edge positions with one another.

9. The method in accordance with claim 8, wherein the respective required displacement is determined by a comparison of a respective line with two adjacent lines and/or a respective column with two adjacent columns.

10. The method in accordance with claim 7, wherein the representative edge positions are determined or corrected by forming and evaluating a position histogram.

11. The method in accordance with claim 1, wherein the first processing unit reads the image data in an image data stream.

12. The method in accordance with claim 11, wherein the first processing unit reads the image data in the image data stream and read image data are already pre-processed still while the first processing unit reads further image data.

13. The method in accordance with claim 1, wherein the first processing unit divides the image data into tiles, that is part regions of a predefined size, with the processing of the image data taking place tile-wise and with edge positions of black and white transitions in the image data being assembled over tiles in the metadata.

14. The method in accordance with claim 13, wherein the first processing unit classifies the tiles as to whether a part of a code is recorded therein and associates tiles with a region of interest having a code.

15. The method in accordance with claim 1, wherein the first processing unit has an FPGA (field programmable gate array) and/or the second processing unit has a microprocessor.

16. The method in accordance with claim 1, wherein the codes are attached to objects that are conveyed through a detection zone of the image sensor on a conveying device.

17. An optoelectronic code reading device having an image sensor for generating image data and having a control and evaluation unit that has a first processing unit and a second processing unit and in which a method for reading an optical code is implemented, in which method image data with the code are recorded by the image sensor, the first processing unit reads the image data from the image sensor and generates metadata with respect to the image data and/or to the code in the course of a pre-processing of the image data, and the second processing unit reads the code after the pre-processing by the first processing unit, wherein the second processing unit reads the code solely from the metadata without making use of the image data.

Description

[0029] The invention will be explained in more detail in the following also with respect to further features and advantages by way of example with reference to embodiments and to the enclosed drawing. The Figures of the drawing show in:

[0030] FIG. 1 a schematic overview representation of the exemplary installation of a camera based reading device above a conveyor belt;

[0031] FIG. 2 a schematic representation of a heterogeneous architecture for image evaluation and for code reading with an FPGA for streaming and pre-processing and a CPU for decoding;

[0032] FIG. 3 an exemplary image with optical codes, a division into tiles, and a region of interest with an optical code;

[0033] FIG. 4a a representation of an exemplary tile with a code fragment;

[0034] FIG. 4b a representation of the binarized tile in accordance with FIG. 4a;

[0035] FIG. 5 an exemplary flowchart for generating metadata with edge positions by an FPGA and a subsequent reading of an optical code solely on the basis of the metadata by a microprocessor;

[0036] FIG. 6 a tabular representation of exemplary edge positions over a plurality of tiles on evaluation of a barcode aligned in the X or Y direction;

[0037] FIG. 7 a tabular representation similar to FIG. 6, but now with a barcode aligned neither in the X direction nor in the Y direction;

[0038] FIG. 8 a matrix-like representation of a displacement error between lines of the recording of a barcode;

[0039] FIG. 9 a further tabular representation of exemplary edge positions of a recorded barcode over a plurality of tiles;

[0040] FIG. 10 a tabular representation in accordance with FIG. 9 after correction of displacements between the lines;

[0041] FIG. 11 an exemplary position histogram to explain an improved correction of displacements between the lines; and

[0042] FIG. 12 an illustration of the displacements detectable using the position histogram.

[0043] FIG. 1 shows a code reader 10 which is mounted above a conveyor belt 12 which conveys objects 14 through the detection zone 18 of the code reader 10, as indicated by the arrow 16. The objects 14 bear codes 20 on their outer surfaces which are detected and evaluated by the code reader 10. The codes 20 can only be recognized by the code reader 10 when they are affixed to the upper side or at least in a manner visible from above. Differing from the representation in FIG. 1, a plurality of code readers 10 can be installed from different directions for the reading of a code 22 affixed to the side or to the bottom, for instance, in order to permit a so-called omnireading from all directions. This stationary use of the code reader at a conveyor belt 12 is very common in practice. The invention, however, relates above all to the reading of codes 20 or to the code reader 10 itself so that this example may not be understood as restrictive.

[0044] The code reader 10 detects image data of the conveyed objects 14 and of the codes 20 by an image sensor 24 and said image data are further processed by a control and evaluation unit 26 by means of image evaluation and decoding processes. The control and evaluation unit 26 will be described in more detail immediately with reference to FIG. 2. The code reader 10 outputs information such as read codes or image data via an interface 28, possibly in different processing stages.

[0045] FIG. 2 shows the control and evaluation unit 26 and its integration in a schematic representation. The control and evaluation unit 26 in this preferred embodiment comprises a first processing unit 30 that will be explained in the following for the example of an FPGA (field programmable gate array) and a second processing unit 32 that will be explained in the following for the example of a microprocessor or a CPU (central processing unit).

[0046] The first processing unit 30 is, on the one hand, connected to the image sensor 24 and, on the other hand, has an interface in the direction of the second processing unit 32, preferably a high speed interface (PCI, PCIE, MIPI). Both processing units 30, 32 can access a memory 34 for image data and additional information, metadata, or processing results. The corresponding reading and writing procedures preferably take place by means of DMA (direct memory access). The memory 34 can be understood as preferably at least functional and, depending on the embodiment, also structurally as part of the second processing unit 32.

[0047] In operation, the image sensor 24 now respectively records a new image or a new image section. It can be a rectangular image of a matrix sensor, but individual or multiple image lines of a line sensor are also conceivable that then successively produce a total image in the course of the relative movement between the code reader 10 and the object 14. The image data of the image sensor 24 are read by the first control and evaluation unit 30. In this respect, metadata are preferably determined by pre-processing, and indeed advantageously on the fly, i.e. still while further image data of the image are to be read or are read by the image sensor 24. The pre-processing and the metadata thus acquired will be explained in more detail below with reference to FIG. 5. As represented by an arrow, the metadata are written to the memory 34. The image data themselves are not required for the code reading in accordance with the invention by the second processing unit 32, but are preferably likewise transferred or streamed directly and/or in a pre-processed manner into the memory 34, as shown by a dashed arrow.

[0048] The second processing unit 34 accesses the metadata in the memory 34 to further process them and to read the content of the recorded optical codes 20 solely from the metadata using a decoder 36 of the second processing unit 32. The decoder 36 works only with metadata for the method in accordance with the invention for reading codes and does not require any access to image data. This does not exclude the possible access to image data, in particular to read codes in at least one further reading attempt (retry) that could not be read solely from metadata. Since such uses of the image data are not part of the reading of codes in accordance with the invention, but are rather possible additions, only one arrow is shown for the reading of metadata and no arrow for a reading of image data.

[0049] The representation of FIG. 2 is to be understood as exemplary; only two processing units 30, 32 are generally required for the invention, with the first processing unit 30 reading image data and generating metadata by their pre-processing. The metadata are forwarded in any desired manner to the second processing unit 32 that reads codes solely from the metadata.

[0050] FIG. 3 shows an exemplary image with optical codes such as is recorded by the code reader 10. The image is preferably divided into tiles 38, preferably as shown in a uniform pattern of nm pixels, with n and m being predefined such that a plurality of tiles are produced per line and column. Metadata are determined with respect to the tiles 38. It can, on the one hand, be evaluated from these metadata whether the respective tile 38 contains a structure of interest such as an optical code, for instance with respect to a degree of contrast and in particular to a variation in the grayscale distribution. The metadata above all serve, as still to be explained with respect to FIG. 5, the reading of codes. It has already been mentioned that the processing steps for acquiring the metadata on-the-fly are possible during the streaming.

[0051] It is not necessary to evaluate all the tiles 38. The image of FIG. 3 contains two optical codes, with one of the two codes being marked as a region of interest 40 by way of example. The other image is of no interest for the code reading. Only a selection 42 of tiles 38 that corresponds to the region of interest 40 is therefore preferably further processed. If there are a plurality of regions of interest, the selection 42 is correspondingly multiplied. It depends on the image whether there are one or more regions of interest 40 or possibly no region of interest 40 at all, for instance because the image does not contain any optical codes. The tile-wise processing of the image data is particularly advantageous, but not absolutely necessary. Even without tiles, the whole image does not necessarily have to be evaluated; other segmentations are known to locate regions of interest.

[0052] FIG. 4a shows for illustration an exemplary tile 38 from the selection 42 with a code fragment. A tile can have 2424 or 4848 pixels, for example. FIG. 4b shows the tile after a binarization, the conversion into a purely black and white image, an advantageous pre-processing step of the first processing unit 30.

[0053] FIG. 5 shows an exemplary flowchart for generating metadata with edge positions by an FPGA as an example of a first processing unit 30 and a subsequent reading of an optical code solely on the basis of the metadata by a microprocessor as an example of a second processing unit 32. The respective responsible units 24, 30, 32 for the individual steps of the flowchart are entered on the right side. This is a preferred association; at least some of the steps can selectively run on the first processing unit 30, on the second processing unit 32, or distributed over both processing units 30, 32. Some of the steps shown are furthermore optional; the invention is by no means restricted to the complete working through of all the steps shown in FIG. 5, even if this is a preferred embodiment.

[0054] The basic idea comprises locating edge positions in the image data using the first processing unit 30 and forwarding them as metadata to the second processing unit 32 that reads codes solely on the basis of the metadata without accessing the image data for this purpose. This is particularly suitable for barcodes having a sufficient module size of, for example, at least two ppm (pixels per module). Tiles 38, such as introduced in FIG. 3, are not necessarily required; FIG. 5 does not contain any reference to tiles 38. However, the individual steps are frequently explained with reference to tiles 38 because this is a preferred embodiment. The steps are often more complex due to the greater data volumes without tiles 38; conversely, no transitions between tiles 38 then have to be considered.

[0055] In a step S1, the image sensor 24 records an image with a certain gray scale depth or color depth of, for example, eight, ten, or twelve bits. In a second step S2, the image data are read out by the first processing unit 30 (streaming). The further steps for evaluating already read image data and for generating metadata can already be performed on the fly still while further image data are being read in. Tiles 38 are preferably formed for the further processing, as explained with reference to FIG. 3 and shown by way of example for a tile in FIG. 4a, and metadata are determined tile-wise. In this respect, it may be necessary to wait until sufficient image data have been buffered, for example a sufficiently large number of image lines corresponding to the height of a tile 38 have been read in.

[0056] In an optional step S3, the image data are binarized, that is converted into a two-value black and white image. The result of the binarization is shown by way of example for the tile of FIG. 4a in FIG. 4b. Edge transitions are sharply defined in the binarized image, but an edge can also be localized in a gray scale image or a color image in principle. During the binarization, alternatively before or after it, a resolution increase can optionally take place, for example from 2424 pixels to 4848 pixels (interpolation, for example bilinearly or bicubically, upsampling). The resolution increase facilitates the dealing with small module sizes or expands the application area of the method in accordance with the invention to smaller module sizes.

[0057] In a step S4, edge positions are determined in the X direction and in the Y direction. For this purpose, the position at which a transition from black to white or vice versa takes place is respectively noted along the lines or columns of the image data. Relative to a tile, this produces a series of edge positions for an individual line or column such as (13, 27, 32, 41, 46). If a size of tiles 4848 is assumed, two times 48 such rows of edge positions are produced, once for the lines and once for the columns. This is a double line with 2448N edge positions, with N being able to vary with the lines and columns. A margin problem may occur if an edge is disposed just between two tiles. An additional line and column can be provided for this purpose or alternatively a tile size of 4949 pixels can be selected with a one pixel overlap.

[0058] The information relevant to the later decoding as to whether it is a transition from black to white or a transition from white to black is still missing in the edge positions. This information can be appended to every edge position or a second list in the size of the list of the edge positions can be made with the additional information. Alternatively, this information is only taken along for a single transition per line and column since the further edges are thus clearly defined. These are only examples; a number of other representations such as a sign or a higher value bit of the edge positions are likewise possible.

[0059] The edge positions can be transferred in this form as metadata to the second processing unit 32 to read the code solely from the metadata without making use of the image data. However, optional additional preparations are preferably still performed by the first processing unit, as now described as steps S5 and S6, with only one of the steps, both steps, or neither of the steps being performed depending on the embodiment.

[0060] In a step S5, a preferred direction of the code in the image is determined. This step is optional or can alternatively only be performed later by the second processing unit 32. The code is recorded with any desired orientation and accordingly generally not in the X direction corresponding to the lines and not in the Y direction corresponding to the columns. Either the X direction or the Y direction is more suitable for the actual alignment of the code, however; this is the preferred direction. A decision can be made on this via the number of edges that are disposed in the X direction in comparison with the Y direction. A line transversely to a barcode crosses all its bars with a maximum number of edges; a line longitudinal to a barcode is at best incident on the edges at the upper and lower ends of the same bar. It becomes clear from this that the higher number of edges corresponds to the preferred direction. To determine the number, a respective counter is increased in the associated direction with each edge position for the X direction and the Y direction, for example. The double list with 248N entries can now be further thinned out for the forwarding in the form of metadata to form a simple list with 148N entries only for the preferred direction; the edge positions of the other direction can be discarded.

[0061] In an optional step S6, a linear offset is determined by a comparison of adjacent lines. The term line here stands for a line or column depending on whether the X direction or the Y direction is looked at. Only one of the two directions is preferably left as the preferred direction due to step S5; this is now assumed. The determination of the linear offset is a preparatory step toward a mutual alignment and combination of the edge positions over a plurality of lines to form a representative line, as described below with reference to steps S8 and S10. The first processing unit 30 thus preferably prepares the combination in step S6 and the second processing unit 32 performs it in steps S8 and S10. Alternatively, all the steps can already be performed in the first processing unit 30 or only in the second processing unit 32.

[0062] Since a barcode has a certain extent, edge positions do not stand only for one line, but are rather provided redundantly for a plurality of adjacent lines. This redundancy can be utilized and eliminated relatively easily for the ideal case that the code is aligned in the X direction or in the Y direction. FIG. 6 shows an example of edge positions assembled over a plurality of tiles for a plurality of adjacent lines in this ideal case. The edge positions practically do not differ so that, for example, any line could be picked out as representative; alternatively, a mean value calculation or a majority decision could be used.

[0063] With a general orientation of the code, and therefore with any and an initially unknown orientation of the code, things become more complicated, as the example shown in FIG. 7 illustrates. The slanted orientation is not the only conceivable cause of inconsistencies; erroneous detections of edges in single lines can also be produced by contamination and other disruptions.

[0064] A method by which compensating displacements between the lines are located in an efficient processing manner compares a respective line with its two adjacent lines. The first and second lines are uploaded for a tile and successively displaced with respect to one another by [k, . . . , 0, . . . k] pixels, where k is a small natural number; k=2 is selected by way of example here. The difference of the two lines is formed pixel-wise for every displacement and is summed via the amount of these differences to obtain an error value: error=sum(abs(line1line2)). The third line is then uploaded and a corresponding error value is calculated for the comparison between the first line and the third line and between the second line and the third line. This is repeated for the further groups of three of lines, that is first the second to fourth lines up to the 48th-48th line of a tile. A total of 4625 error values results. FIG. 8 shows an exemplary 515 section thereof.

[0065] The error values are now initially evaluated linewise. Th maximum displacement window for the comparison of directly adjacent lines amounts to pixels since the slanted position of the code in the preferred direction is upwardly bounded by 45. Five error values are available, corresponding to the five displacements [k, . . . 0, . . . k] at k=2 that are designated by value(1) . . . value(5).

[0066] The following are fixed as rules for the displacement error shift:

[00001]$\mathrm{If}\mathrm{value}\left(3\right)=0,\mathrm{then}\mathrm{shift}=0.\mathrm{If}\mathrm{value}\left(4\right)=0,\mathrm{then}\mathrm{shift}=-1.\mathrm{If}\mathrm{value}\left(2\right)=0,\mathrm{then}\mathrm{shift}=+1.\mathrm{Otherwise}\mathrm{shift}=-\left(\frac{-2}{\mathrm{value}\left(1\right)}+\frac{-1}{\mathrm{value}\left(2\right)}+\frac{+1}{\mathrm{value}\left(4\right)}+\frac{+2}{\mathrm{value}\left(5\right)}\right).$

[0067] The first three rules take the maximum possible displacement error into account. The fourth rule otherwise approximates the displacement error in a manner that is simple to calculate corresponding to the deviation from a maximum displacement with a respective weighted contribution. To avoid resource intensive divisions in an FPGA, look up tables (LUTs) can instead be implemented that form an inverse (n>1/n) and do not have to take account of a large number of values since the error values do not become large. If more processing resources are available, on the implementation on the second processing 32 for example, a more complex calculation can take place instead of the rules, for example a parabolic approximation with sampling points around value(3).

[0068] If lines l, (i1) that are not directly adjacent are compared, but rather next but one adjacent lines l, (i2), the procedure is similar, but only some pre-factors change. It must be considered here that now the maximum displacement error can amount to two pixels. The rules here are:

[00002]$\mathrm{If}\mathrm{value}\left(3\right)=0,\mathrm{then}\mathrm{shift}=0.\mathrm{If}\mathrm{value}\left(4\right)=0,\mathrm{then}\mathrm{shift}=-1/2.\mathrm{If}\mathrm{value}\left(2\right)=0,\mathrm{then}\mathrm{shift}=+1/2.\mathrm{If}\mathrm{value}\left(1\right)=0,\mathrm{then}\mathrm{shift}=+1.\mathrm{If}\mathrm{value}\left(5\right)=0,\mathrm{then}\mathrm{shift}=-1.\mathrm{Otherwise}\mathrm{shift}=-0.5\left(\frac{-2}{\mathrm{value}\left(1\right)}+\frac{-1}{\mathrm{value}\left(2\right)}+\frac{+1}{\mathrm{value}\left(4\right)}+\frac{+2}{\mathrm{value}\left(5\right)}\right).$

[0069] A respective linewise displacement error is determined with shift, once for the comparison of the lines i, (i1) and once of the lines i, (i2). This can now be accumulated over i=1 . . . 45 lines of error values. shiftTotal(i)=shiftTotal(i1)+shift. In addition, the sum of the displacement errors can be determined as a quality feature and it can be assumed, for example, on an exceeding of a threshold value that no good displacement could be found. The displacement error shiftTotal and the quality feature can be determined for the case of the displacement of the line i with respect to the line (i1) and with respect to the line (i2) and can be forwarded as metadata.

[0070] In a step S7, a segmentation now takes place in which regions of interest with codes are found. This is described, for example, in EP 2 555 160 B1 named in the introduction and can alternatively use any other segmentation process known per se, including processes of machine learning, in particular neural networks. The code type is preferably also determined in the segmentation. The decoding in accordance with the invention preferably works with barcodes and even more preferably with those that have been recorded at a minimum resolution, that is have a certain module size of, for example, at least two pixels per module. In a preferred embodiment, a decoding in accordance with the invention is only attempted solely using the metadata when corresponding conditions have been satisfied. Otherwise, a conventional decoder can be used directly that, however, has to access the image data again. The use of a conventional decoder is also possible if the reading attempt solely with the metadata fails (retry).

[0071] The segmentation first takes place very late in the routine, which means that the previously described steps are performed for all the tiles with any desired image information irrespective of whether any relative code fragments are contained in the tile at all. The previous results can be used for this to decide which tiles belong to a region of interest. Alternatively, the segmentation can already take place in an earlier step.

[0072] After the segmentation, the second processing unit 32 takes over from a step S8 onward. The first processing unit 30 has determined metadata tile-wise, namely edge positions, optionally the preferred direction and shifts together with the quality feature and some should now only be consolidated over tiles.

[0073] It may occur that all the tiles for a region of interest are not uniform over the preferred direction. A global preferred direction can then be fixed, for example, via a majority decision. The majority should be unambiguous for if not even the preferred direction can be clearly determined, a simple code reading using metadata would probably anyway fail. The minority of the tiles with a differing preferred direction can therefore be discarded.

[0074] The edge positions are strung together over tiles in that a corresponding multiple of the tile extent is summed to an edge position depending on the position of the tile within the region of interest. FIG. 9 shows by way of example edge positions for a plurality of lines in the preferred direction after the stringing together, but still without a correction of displacements.

[0075] In the optional step S8, the lines are then aligned with one another while making use of the shifts transferred in the metadata and the quality features of step S6 or in that, alternatively, this step S6 is only now performed by the second processing unit 32. A mean value can be determined, for example, to consolidate the shifts calculated per tile over the tiles. Overshoots are preferably eliminated beforehand, with the respective quality feature contained in the metadata being able to provide an indication. The process can optionally be aborted and a conventional decoder can be started up if too many tiles having poor quality features are affected, i.e. if the code can no longer be covered by tiles in the preferred direction whose quality features are sufficiently high.

[0076] The consolidated shift called the meanShift is now respectively deduced from the jth line. Since it is a shift between adjacent lines, scaling takes place with the linear index j: pos=posmeanShift*j. FIG. 10 shows a result for the example of FIG. 9.

[0077] In an optional step S9, a position histogram is formed and evaluated. As can be recognized in FIG. 10, the targeted ideal case with the same lines in the table is actually not yet achieved solely by the shifts. The process described with respect to steps S6 and S8 is optimized for an FPGA and the shift over unavoidable processing errors of every algorithm really applied is therefore not exactly determined.

[0078] FIG. 11 shows an exemplary position histogram that is based, however, on different figures than the table in accordance with FIG. 10 for better statistics.

[0079] The position histogram collects the number entered on the Y axis as to how often these edge positions have been determined in its bins entered on the X axis corresponding to possible edge positions.

[0080] It can be determined during the building of the histogram line by line by the edge positions similar to the table in FIG. 10 whether a count is already present in an adjacent bin and an offset with respect to the adjacent bin is then noted. A list of offsets such as (0,0,1,0,1,0,1, . . . ) is then produced after every line. A mean value offsetMean can then in turn be determined from this that is taken into account and deduced or added for every new line. The global offset of the preceding line is furthermore taken into account so that the offset for every line changes as follows: offset=floor(offset)+round(offsetMean). This offset is corrected linewise and is preferably combined with the above-described shift. The offset compensates errors that are due to the shift not having been able to be exactly determined.

[0081] FIG. 11 in precise terms does not only show a position histogram but rather the overlap of the just described process (dark) and a comparison (light) in which the exact shift has been manually calculated. The additional offset that is determined in step S10 is moreover illustrated in FIG. 12 in which the line number is entered on the X axis and the offset is entered on the Y axis.

[0082] In a step S10, the edge positions redundantly determined over a plurality of lines are now combined. To return to the example of FIG. 10 again, the redundant two-dimensional table should be compressed to a one-dimensional line. Ideally, the lines are identical to one another, the optional steps S8 and/or S10 with a correction of the mutual displacement due to shifts and/or offsets contribute to better approaching this ideal case. In a manner similar to the ideal case in accordance with FIG. 4 it is thus now possible to pick out a line, to form a mean value, to make a majority decision, or to carry out a simple calculation to find representative edge positions.

[0083] In a final step S11, the acquired one-dimensional list of edge positions is used to read the code. The differences of adjacent edge positions correspond directly to the widths of the bars and gaps of the barcode so that a decoding is now easily possible.