IMAGE SENSOR FOR OPTICAL CODE RECOGNITION
20210392285 · 2021-12-16
Inventors
- Bruno DIASPARRA (Seyssins, FR)
- Thierry LIGOZAT (Quaix en Chartreuse, FR)
- Romain GUIGUET (Grenoble, FR)
- Gareth POWELL (Saint-Martin-d'Uriage, FR)
- Jérôme VANRUMBEKE (Saint-Étienne-de-Saint-Geoirs, FR)
- Arnaud FOUCHER (Meylan, FR)
Cpc classification
G06V10/267
PHYSICS
G06V10/44
PHYSICS
G06V10/23
PHYSICS
G06V10/26
PHYSICS
G06V10/467
PHYSICS
G06V10/36
PHYSICS
G06K7/1478
PHYSICS
H04N25/40
ELECTRICITY
G06V10/28
PHYSICS
G06V10/22
PHYSICS
International classification
Abstract
A CMOS image sensor for a code reader in an optical code recognition system incorporates a digital processing circuit that applies a calculation process to the capture image data as said data acquired by the sequential readout circuit of the sensor, in order to calculate a macro-image from the capture image data, which corresponds to location information of code(s) in the capture image, and transmit this macro-image in the image frame following the capture image data, in the footer of the frame.
Claims
1. A CMOS image sensor configured to implement the capturing of an image, upon external command (trig), and to output, towards an optical code recognition system, capture image data within a corresponding image frame (DATA-OUT), the image sensor being integrated into the same integrated circuit chip, a matrix (1) of pixels comprising N lines and M columns and a sequential read circuit (4) of the N lines of pixels of the matrix, characterised in that the sensor, moreover, wherein the sensor includes a digital processing circuit (7) configured to apply a calculation process (100) to the capture image data according to the acquisition of them by the sequential read circuit (4) and in a synchronous manner, in order to calculate, on the fly, a macro-image, taking as a starting point the capture image data corresponding to the code location information in said capture image, and to transmit it to the footer of said image frame (DATA-OUT), following said capture image data, said macro image being such that each macro-image point corresponds to a block (B) of points of the capture image, said calculation process of the macro-image comprising at least for each pixel of the matrix: a first stage (101) of calculation of at least one filter (F.sub.CV) defined by the coefficients of a convolution mask, which establishes, for the pixel, a first numeric value (V.sub.F), corresponding to high-frequency spatial information extracted from the capture image data of said pixel and the adjacent pixels in an adjacent window delimited by the convolution mask centered on the pixel, the calculation being carried out on the fly by the sequential read circuit (4) of said capture image data of the respective pixel and the adjacent pixels as the acquisition progresses, said first numeric value being encoded using the same number of bits (r) as said capture image data; a second stage (102) of binarisation (BIN1) of the first numeric value, establishing a first binary value (V.sub.B1) for each pixel of the matrix; and for each block (B) of pixels of the matrix: a third stage (103) of calculation of a second numeric value (VΣ) by adding up said first binary values (V.sub.B1) of the pixels in the block concerned in an accumulation register of Q=b1+b2 output bits, where b1 and b2 define the number n=2.sup.b1 of lines of pixels in the block and the number m=2.sup.b2 of columns of pixels in the block, n and m being greater than 1.
2. The image sensor of claim 1, wherein the second numeric value is a value encoded using q bits (VΣ.sub.q), where q=1 at Q.
3. The image sensor of claim 1, wherein the calculation process (100) includes a fourth stage (104), of binarisation (BIN2) of said second numeric value.
4. The image sensor of claim 3, wherein the calculation process (100) includes a fifth stage (105) of morphological processing by erosion followed by expansion, which is applied to the binary values established by the fourth stage.
5. The image sensor of claim 1, comprising a memory of configurable convolution masks.
6. The image sensor of claim 1, wherein the first numeric value is the absolute value of the result of the convolution mask calculation.
7. The image sensor of claim 1, wherein said first stage of calculation applies, in parallel, to at least a first and a second convolution mask, corresponding, respectively, to a horizontal gradient calculation and a vertical gradient calculation in each pixel of the matrix, wherein the first numeric value (V.sub.F) is the sum of the absolute values of said gradients calculated.
8. The image sensor of claim 1, wherein said first stage (101) corresponds to a filter calculation from among at least one filter from the following list: a non-directional high-frequency filter calculation combining one or more gradient calculations in a fixed direction, a filter corresponding to the spatial characteristics representing a type of code.
9. The image sensor of claim 8, wherein the sensor is configured to apply a first data filter calculation of a first capture image of a target object, and to apply a second filter calculation, different from the first, to data of a second capture image from the same target object, said first and second filter calculations being selected from said list.
10. The image sensor of claim 9, wherein the second capture image corresponds to zooming in on an area of interest determined taking the macro-image calculated using the data of the first capture image as a starting point.
11. The image sensor of claim 1, wherein the block size of pixels is a function of the size of code(s) in the capture image.
12. The image sensor of claim 11, wherein the block size is configurable.
13. The image sensor of claim 1, wherein the stages of binarisation make use of a respective threshold value (Vs1, Vs2), that is configurable.
14. The image sensor of claim 2, wherein the calculation process (100) includes a fourth stage (104), of binarisation (BIN2) of said second numeric value.
15. The image sensor of claim 4, comprising a memory of configurable convolution masks.
16. The image sensor of claim 5, wherein the first numeric value is the absolute value of the result of the convolution mask calculation.
17. The image sensor of claim 5, wherein said first stage of calculation applies, in parallel, to at least a first and a second convolution mask, corresponding, respectively, to a horizontal gradient calculation and a vertical gradient calculation in each pixel of the matrix, wherein the first numeric value (V.sub.F) is the sum of the absolute values of said gradients calculated.
18. The image sensor of claim 5, wherein said first stage (101) corresponds to a filter calculation from among at least one filter from the following list: a non-directional high-frequency filter calculation combining one or more gradient calculations in a fixed direction, a filter corresponding to the spatial characteristics representing a type of code.
19. The image sensor of claim 10, wherein the block size of pixels is a function of the size of code(s) in the capture image.
20. The image sensor of claim 12, wherein the stages of binarisation make use of a respective threshold value (Vs1, Vs2), that is configurable.
Description
[0029] Other features, details and benefits of the invention may emerge from reading the description below, with reference being made to the attached drawings, by way of example, and which respectively represent:
[0030]
[0031]
[0032]
[0033]
[0034]
[0035]
[0036]
[0037]
DETAILED DESCRIPTION
[0038] A code reader comprises a CMOS image sensor, a source of light to illuminate a target (one or more of the codes affixed to an object, a license plate . . . ) through a scanning window; and an optical system which allows for reflecting back the light reflected by the target onto the sensor's matrix of pixels. The light source is generally based on LEDs, to illuminate the target for taking the image, by flash. Some systems can, moreover, integrate a laser source associated with a diffractive optical device, with the aim of assisting the operator to centre the target code prior to taking the image, by projecting a particular motif (cross, angles, . . . etc.), This operation is known under the Anglo-Saxon term “aimer”. Said laser source is normally unlit, before the LED flash occurs in order to take the image. The taking of the image by the reader is triggered by an electronic or manual mechanism: the reader exits a standby mode (low consumption) and activates the light source and the sensor, in order to take the image and transmit it to an external code recognition system, external to the sensor.
[0039] The CMOS image sensor conventionally, as shown in
[0040] A classic embodiment of the capture image frame is illustrated in
[0041] The image data frame transmitted to the external recognition system in practice includes a frame header and a footer, which frames the flow of data of the N image lines. In particular, the header contains information such as the quantity of image data (size of the image). Actually, a full image capture, or one based only on areas of interest, can be implemented. The values of the image capture parameters of the image that has just been taken are likewise generally recorded. In particular, it concerns the TINT duration of the exposure time, the encoding dynamics (number of greyscale codes) and the conversion gain. These values are provided by a register 5 of parameters of the sensor (
[0042] The size (number of bits) of the frame header and the footer is small in relation to the frame size reserved for the actual image data. In one example, a number of bits is provided for the header or tooter of the page which corresponds to the number of bits of the data of an image line.
[0043] These general reminders being made, we are going to he able to describe the CMOS image sensor embodiment for a barcode reader proposed in the invention.
[0044] According to the invention, a CMOS image sensor integrates the means of calculation which allows for extracting the image data, the data which corresponds to information concerning the presence or location of the code(s) in the image, reliably. Said information is provided in a manner which complements the external recognition system in the footer of the image frame, in order to allow it to focus its computational and memory power on the capture image data in the image areas which it can identify, thanks to said complementary data.
[0045] The image sensor thus comprises digital processing means 7 (
[0046] Conventionally, the sensors generally comprise the digital processing means which the operators of elementary calculations use (adders, multipliers, dividers, comparators), which are, for example, used to calculate the histograms. The histogram calculations are useful to the sensor to allow it to adapt the exposure time of the pixels, set to a default value for a data application, when the histogram shows that the image is over or under-exposed. The sensor can also adapt the conversion gain in the reading stream to increase the contrast. The macro-image calculation in accordance with the invention uses the same elementary operators, so that space is saved and they are easy to integrate into the sensor chip, in order to calculate the value of each macro-image point, taking as a starting point the capture image data, at the point of the data being acquired, by way of a filter calculation that is representative of a spatial feature of the codes. Said circuit 7 comprises (
[0047] The calculation process on the fly in accordance with the invention, for extracting a spatial digital signature, implemented in the digital circuit 7, is now going to be explained with the aid of the chronogram shown in
[0048] Likewise, we could take, for example, a process which makes for extracting a high frequency signature of the capture image, with the aim of obtaining a corresponding macro-image which needs to allow for identifying the code regions which contain a high density of black and white alternations, or two very contrasting levels of grey, in at least one direction, while the rest of the image (in the relevant application context) contains little of it, in other words, one expects to be able to identify, in this macro-image, white “boxes”, the size of which could correspond to some code.
[0049] For this extraction, a gradient calculation is applied, in at least one direction, and the information that is retained is the magnitude of the gradient (i.e. the absolute value of the gradient, without considering the sign). A gradient calculation is preferably applied in at least two directions, in such a way as to obtain the greatest value possible at each point, by adding up their respective magnitudes. Actually, the “capture” codes in the image are not necessarily aligned, in relation to the pixel matrix axes. This will often be the case in applications in which the code reader is stationary, and the items to be scanned arrive on a conveyor belt in front of the camera of the reader. Certain codes likewise have a high frequency signature in different directions, in particular the symbolic 2D codes and the alphanumeric strings. In order to enable the most reliable location possible, it is of interest to obtain high frequency information at each point that is supposed to be the most complete (the most extensive).
[0050] The calculation process 100 thus comprises a first stage 101 of filter calculation. At this stage 101 (
[0051] In practice, the calculation is done on the fly in line with the provision of the pixel data by the reading circuit, line by line. For each pixel, the calculation result of each filter (a linear combination of the filter coefficients with the values of the image points in the adjacent window centred on the pixel) is obtained, once the process has been able to obtain the numerical values of the pixel, and of its adjacent pixels, defined by the mask window centred on the pixel concerned. It emerges from
[0052] V.sub.px, the numerical value of a capture image point, which is the value provided by the reading circuit 4, and which represents the amount of signal integrated by a corresponding signal into the matrix during the exposure time of the image taken, can be noted. Said value is encoded using r bits, in the example (
[0053]
[0054] The image I.sub.HD, corresponding to all the data V.sub.px, (collective DATA group formed by N DATA-L.sub.i) can be seen. The corresponding images calculated, having a horizontal gradient (I.sub.GH), vertical gradient (I.sub.GV) and high frequency, non-directional information (I.sub.F), at the stage 101 of the calculation process in accordance with the invention can be seen as well. All these images I.sub.HD, I.sub.GH, I.sub.GV and I.sub.F have N lines, M columns, and a depth of 12 bits.
[0055] The stage 102 following the calculation process corresponds to a binarisation of the high frequency image obtained, which makes for separating off the noise from the high frequency signal of the image. By improving the signal-to-noise ratio, the confidence index concerning the code location information which is provided at the output of the sensor is increased. In practice, the binarisation is performed as previously, on the fly, on each of the values Y.sub.F resulting from the filtration stage calculation process 101 (
[0056] A binary value V.sub.B1 per capture image point is obtained. The corresponding image simulated I.sub.B1 comprises N lines and M columns, at a depth of 1 bit. At this stage, point information that is not code location information emerges. It is necessary to be able to rely on this information in order to form significant code location information, taking into account the size of the codes that may be present in the image.
[0057] This concerns the function of the subsequent stages, 103 and 104, by which the binary code location macro-image has just been formed in the capture image.
[0058] These stages are, like the others, implemented on the fly, in line with the receipt of the data needed to calculate the value of each macro-image point. Figuratively speaking, this concerns relaying the information between them, in order to try to form the “significant boxes”, corresponding to the locations of code in the image, and, in a simple manner, by calculating on the fly, but reliably, meaning that the information output has an increased index of confidence.
[0059] For that, one proceeds in two stages, which are detailed in
[0060] The first stage 103 corresponds to a tessellation, which is going to make it possible to match a macro-image point with each block B of points of the capture image, for which a recorded value VΣ.sub.q is calculated, taking as a starting point the binary values V.sub.BIN1 of the points of block B. In other words, making reference to the image simulation I.sub.B1, the number of white points in the block in question is counted: the value VΣ.sub.q is quite indicative that the block B concerned could correspond to a code location in the image. This passage is implemented by an accumulation operation Σ.sub.q in a register element Acc corresponding to the block processed, meaning that the binary values V.sub.BIN1 of the points in block B concerned are added to their calculation on the fly (stages 101 and 102).
[0061] The tesselation operation corresponds to the capture image being divided by blocks of n=2.sup.b1 lines and m=2.sup.b2 columns. That defines the dimensions of the accumulation register element of the binary values of the n×m points of a block; this concerns a Q bits register, with Q=b1+b2.
[0062] Preferably, a tesselation consisting of squares, with b1=b2, adapted to all the codes and all the orientations, would be chosen. For example, if one takes b1=b2=6, one then has 4096 capture image points per block B and the accumulation register element Acc for each block allows for counting from 0 (all the points have a value of 0) to 4096 (all the points have a value of 1).
[0063] Advantageously, the value VΣ.sub.q provided within the output for each block may or may not be a truncated value. The index q is a parameter which indicates that, and it thus has a value which may be equal to 1, 2, 3, . . . , or Q bits. If q=4, for example, then the value VΣ.sub.q, provided as output for each block, will be a value encoded using only 4 bits.
[0064] The corresponding macro-image IΣ.sub.q (
[0065] The greater the value of q, the richer will be the location information contained in the macro-image.
[0066] The choice of values n, m and q defines the dimensions of the macro-image size IΣ.sub.q. Thus, depending upon the size of the block, smaller or larger, a value q can be chosen which can also be smaller (for a small block) or larger (for a larger block) so as to obtain a macro-image size (number of hits) that makes for calculating it on the fly at the time at which a capture image is acquired, as well as its integration into the footer of the frame. In other words, it is necessary to reach a balance between processing time, macro-image dimension and wealth of information.
[0067] The dimension of the block (number of lines and columns) firstly depends upon the number of lines and columns of the pixel matrix. Then, it must not be too small, in order to obtain the level of compression desired (calculating time, integration into the footer of the frame). However, it must also not be greater than the size of the codes in the image, as otherwise corresponding location information would be lost. We will return to these aspects further down below.
[0068] In a realistic example, for a pixel matrix of N=1920 lines and M=2560 columns, a block size of 64 lines×64 columns can be defined. A macro-image of n=30 lines and m=40 columns is obtained. If q=12 bits, the corresponding quantity of macro-image bits is 14,400 bits. If q=8 hits, it is 9600 bits, and if q=4, it is 4800 bits. The block size is adapted for the codes in the image of greater size, for example 100×100 pixels.
[0069] In one variant, we would like to provide a binary macro-image of I.sub.B2 (
[0070] It can be seen that the forms are not “perfect”, with serrated edges or small projections.
[0071] In one embodiment shown in
[0072] Ultimately, the calculation process implemented in the image sensor allows for obtaining a small image with significant content on the location of the codes in the capture image, which can be transmitted to the image transmission frame footer, following the capture image data. Said location data is directly usable by the external code recognition system, which can then focus its image processing algorithms on the capture image meats) identified in the macro-image as containing the high frequency information.
[0073]
[0074] In addition, if a block B is used that is greater in size than the small code, for instance 64×64, it can be seen from the image I1 which corresponds to the small binary image output from the process 100 (after the stage 104 or after the stage 105) that the small code, Code-1, has not been located. The image 1 simulates a superimposition of the information of the small image I1 upon the capture image 10. With a smaller block B, in the example 32×32, it can be seen that the small code is well located (image I2 and I′2).
[0075] The calculation process 100 for calculating the macro-image implemented by the digital processing circuit 7 just described takes place synchronously with the reading (the acquisition) of the capture image data by the reading circuit 4, in order to process the data of the pixel lines one by one, sequentially. In other words, the digital processing circuit 7 is clocked (
[0076] At each new frame (Fsync), the digital processing circuit is re-initialised, in particular the line memory M.sub.L and the accumulation register Arc are re-set to zero. The stages 101 to 103, 101 to 104 or 101 to 105 of the process then take place, according to the availability of the values involved in each calculation, for each pixel, on a rolling basis, in x and y (
[0077] The corresponding image frame is illustrated in
[0078] The calculation process that has just been described, for extracting a high frequency capture image signature, may be applied in the same way to extract the other spatial signatures (of repetition, direction, . . . etc.). For the same object, if a number of successive images taken are carried out, it is thus possible to calculate different spatial signatures, in regard to the entire image or part of it, which in turn allows for supplementing the location information by way of successive passes. In particular, it is possible to implement an image capture by zooming in (using a magnification lens belonging to the optical system of the code reader integrating the sensor) on a part of the scene which corresponds to an area of interest determined taking the small image as a starting point, extracted from the full image, and it is possible to obtain a more precise spatial signature, based on other geometric criteria. To take a numerical example: if the size of a code in a capture image taken for a given capture distance occupies 256×256 pixels, a corresponding “white” box will be obtained in the macro-image, which will not exceed 4×4 pixels, which, however, provides proper location information (position and size in the capture image). The recognition system may request that the image be taken a second time, zooming in on the corresponding area of interest, in order to obtain more precise spatial information, based on the spatial high frequency criteria, as described above, or on other geometric code criteria.
[0079] Each time, it remains a filter calculation (or multiple filter calculations, in parallel). What changes are the convolution masks used, in other words the coefficients of the filters, and possibly the size of the mask (5×5, for instance, instead of 3×3). A mask memory [a].sub.k can therefore easily be provided for in the sensor and for a given image taken one or more convolution masks are selected in said memory. The different thresholds of the process, namely Vs1, Vs2 and the value q of truncation, can be configured, as can the dimensions of block B. The latter can be done in different ways, by way of a pointer value for a memory of corresponding parameters (a mask memory, for example), or by way of the actual value of the parameter. The specialist skilled in the art knows how to implement that in practice. The sensor thus comprises corresponding parameters in one or more parameter registers (Register 5,