ELECTRONIC COUNTER SCALE INTELLIGENT VERIFICATION METHOD BASED ON DEEP LEARNING DETECTION AND IDENTIFICATION

Abstract

The present disclosure discloses an electronic counter scale intelligent verification method based on deep learning detection and identification, which includes: collecting a top image, top depth data, and a front image of the electronic counter scale, and selecting a reading display apparatus on the front image; inputting the top image into a scale pan detection and identification model to obtain a scale pan target detection result, verifying the result with the top depth data, obtaining a spatial position of a scale pan, and outputting the spatial coordinates of the four vertices of the scale pan; inputting the top image into an indicating value detection and identification model to obtain indicating value characters and a position of the display apparatus, correcting similar characters and low-confidence characters, and obtaining an indicating value of the electronic counter scale; and performing unmanned verification of weighing performance, repeatability, bias load, and discrimination.

Claims

1. An intelligent sensing method for verifying key information of an electronic counter scale based on deep learning, comprising the following steps: step A, collecting a top image, top depth data, and a front image of the electronic counter scale, and selecting a reading display apparatus on the front image of the electronic counter scale; step B, inputting the top image of the electronic counter scale into a scale pan detection and identification model to obtain a scale pan target detection result, verifying the scale pan target detection result with the top depth data to obtain a spatial position of a scale pan, and outputting the spatial coordinates of the four vertices of the scale pan; step C, inputting the top image of the electronic counter scale into an indicating value detection and identification model to obtain indicating value characters and a position of the display apparatus, correcting similar characters and low-confidence characters, and obtaining an indicating value of the electronic counter scale according to arrangement of horizontal coordinates of the indicating value characters; and step D, performing unmanned verification of weighing performance, repeatability, bias load, and discrimination by controlling a three-axis mechanical arm to load weights, based on the obtained spatial position of the scale pan and the indicating value of the display apparatus.

2. The intelligent sensing method for verifying key information of the electronic counter scale based on deep learning according to claim 1, wherein in the step A, the top image and the top depth data of the electronic counter scale are collected by a depth camera; and the front image of the electronic counter scale is collected by a color camera.

3. The intelligent sensing method for verifying key information of the electronic counter scale based on deep learning according to claim 1, wherein in the step B, the scale pan detection and identification model is a deep learning target detection model; the scale pan target detection result is a rectangular bounding box of the scale pan on the image, and the recording can be done in two ways: a top-left vertex and a length and width (u.sub.0, v.sub.0, w, d) of the bounding box or a top-left vertex and a bottom-right vertex (u.sub.0, v.sub.0, u.sub.2, v.sub.2) of the bounding box; the verification with the top depth data is to map the rectangular bounding box of the scale pan on the image to a coordinate space of the depth data, i.e., to map the top-left vertex of the bounding box (u.sub.0, v.sub.0).fwdarw.(x.sub.0, y.sub.0, z.sub.0), and the bottom-right vertex of the bounding box (u.sub.1, v.sub.1).fwdarw.(x.sub.1, y.sub.1, z.sub.1); and the scale pan is in a range of (x.sub.0, y.sub.0).fwdarw.(x.sub.0, y.sub.2).fwdarw.(x.sub.2, y.sub.2).fwdarw.(x.sub.1, y.sub.0).fwdarw.(x.sub.0, y.sub.0) in a point cloud, and depth of each point in the range is followed to obtain an accurate position of the scale pan, and output the spatial coordinates of the four vertices of the scale pan, i.e., (X.sub.0, Y.sub.0, Z.sub.0), (X.sub.1, Y.sub.1, Z.sub.1), (X.sub.2, Y.sub.2, Z.sub.2), and (X.sub.3, Y.sub.3, Z.sub.3).

4. The intelligent sensing method for verifying key information of the electronic counter scale based on deep learning according to claim 1, wherein in the step C, the indicating value detection and identification model is a deep learning target detection model; the scale pan target detection result is the indicating value characters and the position; the corrections include non-maximum suppression correction for the similar characters and decimal point complementing correction for the low-confidence characters, the non-maximum suppression correction for the similar characters is to perform a non-maximum suppression on the similar characters; the decimal point complementing correction for the low-confidence characters is to perform a decimal point complementing operation on the character detection results with low confidence, i.e., making 0.fwdarw.0. and 1.fwdarw.1. if the detection results have low confidence; and the horizontal coordinates of the indicating value characters are arranged from small to large.

5. The intelligent sensing method for verifying key information of the electronic counter scale based on deep learning according to claim 1, wherein in the step D, the spatial coordinates of the four vertices of the scale pan are obtained through a verification program, and thus calculating an off-loading area or center of the scale pan, and controlling the mechanical arm to place the weights.

Description

BRIEF DESCRIPTION OF FIGURES

[0012] FIG. 1 is a flow chart of an intelligent sensing method for verifying key information of an electronic counter scale based on deep learning; and

[0013] FIG. 2a and FIG. 2b are schematic diagrams of scale pan target detection results of a scale pan detection and identification model.

DETAILED DESCRIPTION

[0014] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly, the present disclosure will be described in further detail below in conjunction with embodiments and accompanying drawings.

[0015] As shown in FIG. 1, an intelligent sensing method for verifying key information of an electronic counter scale based on deep learning includes the following steps:

[0016] Step 10, verification of key information of an electronic counter scale, including a spatial position of a scale pan and an indicating value of a display apparatus, collecting a top image and top depth data of the electronic counter scale by a depth camera, collecting a front image of the electronic counter scale by a color camera, and manually selecting a reading display apparatus from the front image of the electronic counter scale.

[0017] Step 20, inputting the top image of the electronic counter scale into a scale pan detection and identification model to obtain a scale pan target detection result, verifying the scale pan target detection result with the top depth data to obtain an accurate scale pan position, and outputting the spatial coordinates of four vertices of the scale pan.

[0018] The scale pan detection and identification model is a deep learning target detection model with categories of a scale pan and a background, and the model structure may be YOLO, SSD, faster R-CNN, etc.

[0019] The scale pan target detection result is a rectangular bounding box of the scale pan on the image. The recording can be done in two ways: a top-left vertex and a length and width (u.sub.0, v.sub.0, w, d) of the bounding box or a top-left vertex and a bottom-right vertex (u.sub.0, v.sub.0, u.sub.2, v.sub.2) of the bounding box.

[0020] The verification with the top depth data is to map the rectangular bounding box of the scale pan on the image to a coordinate space of the depth data, i.e., to map the top-left vertex of the bounding box (u.sub.0, v.sub.0).fwdarw.(x.sub.0, y.sub.0, z.sub.0), and the bottom-right vertex of the bounding box (u.sub.1, v.sub.1)(x.sub.1, y.sub.1, z.sub.1) respectively.

[0021] The scale pan is in a range of (x.sub.0, y.sub.0).fwdarw.(x.sub.0, y.sub.2).fwdarw.(x.sub.2, y.sub.2).fwdarw.(x.sub.1, y.sub.0).fwdarw.(x.sub.0, y.sub.0) in a point cloud, and depth of each point in the range is followed to obtain an accurate position of the scale pan, and the spatial coordinates of the four vertices of the scale pan are output, i.e., (X.sub.0, Y.sub.0, Z.sub.0), (X.sub.1, Y.sub.1, Z.sub.1), (X.sub.2, Y.sub.2, Z.sub.2), and (X.sub.3, Y.sub.3, Z.sub.3).

[0022] As shown in FIG. 2a and FIG. 2b, the top image of the electronic counter scale (FIG. 2a) is first input into the scale pan detection and identification model to obtain a scale pan target detection result, which has a bounding box slightly larger than the scale pan; and the bounding box is [452.46,64.79,805.03,513.26].

[0023] Verify with the top depth data to obtain the accurate position of the scale pan, and output the spatial coordinates of the four vertices of the scale pan (FIG. 2b). This is also suitable for the situation where the scale pan is placed offset. [0024] The top-left vertex: [0.0436,?0.1128,0.581] m. [0025] The bottom-right vertex: [0.0436,0.0286,0.581] m. [0026] The bottom-left vertex: [?0.067,0.0286,0.581] m. [0027] The top-right vertex: [?0.067,?0.1128,0.581] m.

[0028] Step 30, inputting the top image of the electronic counter scale into an indicating value detection and identification model to obtain indicating value characters and a position of the display apparatus. Perform the following two corrections, non-maximum suppression for similar characters and decimal point complementing for low-confidence characters. Arrange the horizontal coordinate (u) of the indicating value characters from small to large to obtain an indicating value of the electronic counter scale.

[0029] The indicating value detection and identification model is a deep learning target detection model with 21 categories of 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0., 1., 2., 3., 4., 5., 6., 7., 8., 9., a background, etc., the model structure is faster R-CNN, etc., and the method helps to accurately recognize a decimal point.

[0030] The scale pan target detection result is the indicating value characters and the position, and two corrections are performed.

[0031] The similar characters are subjected to non-maximum suppression, which is to perform non-maximum suppression on ten pairs of similar characters such as 0-0., 1-1., etc. to prevent repeated detection.

[0032] The decimal point complementing for the low-confidence characters is a decimal point complementing operation on low-confidence character detection results, i.e., making 0.fwdarw.0. and 1.fwdarw.1. if the detection results have low confidence.

[0033] The horizontal coordinates (u) of the indicating value characters are arranged from small to large to obtain the indicating value of the electronic counter scale.

[0034] Step 40, performing unmanned verification of weighing performance, repeatability, bias load, and discrimination by a verification program to control a three-axis mechanical arm to load weights based on the obtained spatial position of the scale pan and the indicating value of the display apparatus and according to a verification procedure.

[0035] The verification program may obtain the spatial coordinates of the four vertices of the scale pan by adopting the method in Step 20, and thus calculating the off-loading area or center of the scale pan, and controlling the mechanical arm to place the weights.

[0036] After loading, use the method in step 30 to obtain the indicating value of the electronic counter scale.

[0037] In accordance with the requirements of the procedure, repeat Step 20 to Step 30 to complete the unmanned verification of the weighing performance, repeatability, bias load, and discrimination.

[0038] Compared with the prior art, one or more embodiments of the present disclosure may have the following advantages:

[0039] the intelligent sensing method for verifying key information of the electronic counter scale based on deep learning target detection obtains the spatial position of the scale pan and the indicating value of the display apparatus, cooperates with the verification apparatus and helps to realize unmanned verification of the electronic counter scale.

[0040] Although the disclosed implementations of the present disclosure are as above, the described contents are only used to facilitate the understanding of the present disclosure, and are not intended to limit the present disclosure. Any person skilled in the art to which the present disclosure belongs may, without departing from the spirit and scope disclosed by the present disclosure, make any modifications and changes in the form and details of the implementation, but the scope of the patent protection of the present disclosure shall still be subject to the scope defined in the appended claims.