Marker for occluding foreign matter in acquired image, method for recognizing foreign matter marker in image and book scanning method

Abstract

A marker for occluding a foreign matter in an acquired image contains a mark part whose surface is provided with a two-side continuous pattern formed by combining at least one or multiple primitives; and a fixing part which fixes a marker to a surface of foreign matter in the acquired image with the mark part to facilitate algorithm recognition and marking. The method for recognizing a foreign matter marker in an image includes the steps of performing edge detection on a planar image to acquire an edge map in the planar image; and extracting all contours in the edge map. A certain number of alternative straight-line segments are determined using an algorithm; a region is determined according to the position of each alternative straight-line segment; and finally the region of the approximate area above or below the marker is used.

Claims

1. A marker for labeling a foreign object in an acquired image for occlusion, comprising: a surface having a repeating pattern, comprising one or more geometric primitives, a fixing means configured to be detachably affixed to the foreign object, wherein the geometric primitive is a straight line or a circle so that the repeating pattern comprises a plurality of straight lines that are parallelly arranged or a plurality of circles, wherein the repeating pattern is disposed in a rectangular recognition region, each of the plurality of straight lines is perpendicular to a long side of the rectangular region, wherein a line connecting focal points of the plurality of circles is parallel to the long side of the rectangular region, and wherein a color of the rectangular recognition region is reverse to a color of the geometric primitive.

2. The marker according to claim 1, wherein the repeating pattern is composed of a plurality of geometric primitives.

3. The marker according to claim 2, wherein the plurality of geometric primitives are selected from straight lines, quarter circles, and hollow circles.

4. The marker according to claim 1, wherein the marker is a ring and the foreign object is a finger.

5. A method for recognizing a foreign object marker in an image, comprising the following steps: acquiring a planar image comprising the marker according to claim 3; and performing image preprocessing at least comprising binarization and denoising; performing edge detection on Gall the planar image to acquire an edge map in the planar image; extracting all contours in the edge map; performing straight-line screening on all contours to acquire alternative parallel straight-line segments; extracting an image of a region where the parallel straight-line segments are located from the acquired planar image; acquiring a local edge map by performing binarization and edge detection on the image of the region, extracting a local contour in the local edge map, and acquiring local contours of all ellipses as alternative circles by screening the local contours; calculating a focal point and lengths of long axis and short axis of each contour of each ellipse; examining median length and median angle of the straight-line segments near each alternative circle, and removing the alternative straight-line segments with a deviation exceeding a threshold range by comparing each straight-line segment with the median angle and the median length; calculating a pixel center C of all the straight-line segments meeting the condition as a center position reference of a marker; calculating an average length R of the long axes of all the ellipses as an index for judging a distance between the marker and a lens; calculating an average length L of all the straight-lines as a basis for a downward inclination degree of a finger; and calculating an image range of the marker according to the pixel center C, the average length R of the long axis, and the average length L.

6. The method for recognizing a foreign object marker in an image according to claim 5, wherein edge detection is performed on the planar image by Canny edge detection; and non-linear segment contours are eliminated by determining a threshold range of bounding area of the contour, a minimum dimension of a circumscribed rectangle of the contour, and a length-width ratio of the circumscribed rectangle.

7. The method for recognizing a foreign object marker in an image according to claim 5, further comprising eliminating an alternative straight-line segment when the alternative straight-line segment passes through a corresponding alternative circle; and eliminating an alternative circle when a focal point of the alternative circle deviates from a connecting line of the alternative circle and the deviation exceeds a threshold value.

8. The method for recognizing a foreign object marker in an image according to claim 5, further comprising calculating a center straightness of each of the associated straight-line segments before calculating the pixel center of all the straight-line segments, wherein the center straightness is an average distance from a center of each straight-line segments to a center line connecting all the centers of all the associated straight-line segments; and when the average distance is greater than 3 pixels, deleting the associated straight-line segments and the corresponding alternative circles at the same time.

9. A book scanning method, comprising the following steps: determining an image range of a marker on a planar image of a book page according to the method of claim 5; using a region of an approximate area above or below the marker to expand to the image range of the marker, and removing the image of the marker from book page.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) In order to more clearly illustrate the technical solutions in the embodiments of the invention or the prior art, the drawings to be used in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description merely indicate some embodiments of the invention, and those of ordinary skills in the art can further obtain other drawings according to these drawings without going through any creative work.

(2) FIG. 1 illustrates a diagram of a fingerstall as a marker in the embodiment of the invention;

(3) FIG. 2 illustrates a diagram of straight-line segment pattern in the embodiment of the invention;

(4) FIG. 3 illustrates a diagram of quarter circle pattern in the embodiment of the invention;

(5) FIG. 4 illustrates a diagram of pattern in the embodiment of the invention;

(6) FIG. 5 illustrates a diagram of planar image according to the invention;

(7) FIG. 6 illustrates a diagram of local image extracted according to the invention;

(8) FIG. 7 illustrates a diagram of a mask of an image range of the marker (fingerstall) in recognition method embodiment of the invention;

(9) FIG. 8 illustrates an analysis diagram of the mask in recognition method embodiment of the invention;

(10) FIG. 9 illustrates a detail structure diagram of the mask in recognition method embodiment of the invention;

(11) FIG. 10 illustrates a diagram of marker parameter calculation in recognition method embodiment of the invention;

(12) FIG. 11 illustrates a diagram of scanned image after removing the marker in recognition method embodiment of the invention;

(13) FIG. 12 illustrates a diagram of an application scenario according to embodiment 2 of the invention;

(14) FIG. 13 illustrates a diagram of recognition results in embodiment 2 of the invention;

(15) FIG. 14 illustrates a diagram of an imaging principle in recognition method embodiment of the invention; and

(16) FIG. 15 illustrates a flow chart of the recognition method for book scanning according to the invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

(17) In order to make the objects, the technical solutions and the advantages of the embodiments of the invention clearer, the technical solutions in the embodiments of the invention are clearly and completely described with reference to the drawings in the embodiments of the invention.

(18) As shown in FIG. 1, the embodiment provides a fingerstall solution for turning book pages by finger, the fixing part is a plastic/rubber fingerstall which is a circle similar to a rotator, and is convenient to be sleeved on a finger of a user turning the book page, and meanwhile, a larger friction force exists between plastic material and paper book page.

(19) A rectangular recognition region perpendicular to a center axis is arranged in a middle of the pattern shown in FIG. 1, a two-side continuous pattern formed by equal-length straight-line segments parallel to each other and two rows of two-side continuous patterns formed by hollow circles are arranged in the region, wherein the two rows of hollow circles are staggered. In the embodiment, only a solution of a circle as a special case of an ellipse is considered.

(20) In terms of color selection, for geometric primitives (as shown in FIGS. 2 and 3) i.e. the parallel equal-length line segments and the hollow circles are all white, and a background color of the rectangular recognition region is black, which is reverse to white, the fingerstall is yellow as a whole (in some implementations, other colors which are quite different from book paper can also be used, as shown in FIG. 4).

(21) In order to facilitate wearing, the inner surface of the fingerstall/marker is also provided with densely arranged rubber/plastic teeth, which act together with an elasticity of a rubber/plastic material to ensure firm wearing without causing discomfort to a finger due to excessive pressure (when a finger diameter is larger, the rubber teeth can deform themselves to reduce the pressure on the finger).

(22) Meanwhile, the rubber/plastic teeth are arranged along a direction where the finger enters the fingerstall to facilitate wearing and taking off the fingerstall, and a firmness in the process of turning book page by a lateral movement of the finger is ensured, especially a friction force between a sweaty finger and a rubber material may be smaller than a friction force between the rubber material and a paper book page, thus causing deflection of the fingerstall.

(23) Embodiment 1 is a book scanning application scenario, which mainly solves an influence on an image of a book in a process of turning a book page by a finger during image acquisition scanning, as shown in FIGS. 2 to 15.

(24) (I) straight-line segments with an appropriate size are found from an image:

(25) 1. the image is converted into a gray-scale map (as shown in FIG. 5);

(26) 2. median filtering [5,5] is performed to denoise;

(27) 3. Canny edge detection is performed, with a lower gradient limit of 75 and an upper gradient limit of 120, to acquire an edge map of the image;

(28) 4. a contour is extracted from the edge map;

(29) 5. each small contour is analyzed whether to meet dimension and shape requirements of a straight-line of a fingerstall, thus removing non-linear segment contour:

(30) a) an area enclosed by the contour shall meet requirements of upper and lower limits: a lower limit of area of 10 pixels and an upper limit of area of 500 pixels;

(31) b) width and height corresponding to a minimum circumscribed rectangle of the contour are acquired, and a long side of the circumscribed rectangle needs to meet requirements of upper and lower limits in a straight-line length: a lower length limit of 12 pixels and an upper length limit of 70 pixels; and

(32) c) a length-width ratio of the circumscribed rectangle shall be greater than three;

(33) according to the contour meeting the shape and dimension requirements above, two end points of the corresponding straight-line segment are acquired according to the circumscribed rectangle, and each straight-line segment represented by the two end points is stored as an alternative straight-line segment; and

(34) 6. an overlapping line of alternative straight-lines is removed: a judgment basis is that two straight-lines are considered to be overlapped if both the distance between the two end points of the two lines are less than 3 pixels.

(35) A yellow line segment in the drawing (FIG. 5) is a straight-line meeting the condition found in the step.

(36) (II) A local region where the fingerstall may exist is positioned according to a density of the straight-line segments, as shown in FIG. 6:

(37) 1. a center, a slope and a length of each straight-line segment are calculated;

(38) 2. the straight-line segment is compared with characteristics of every other straight-line segment in sequence to judge whether they meet all the following requirements:

(39) a) a distance between the centers of two straight-line segments meets requirements of upper and lower limits: a lower limit of 4 pixels and an upper limit of 60 pixels;

(40) b) a slope difference between the two straight-line segments is less than 0.05; and

(41) c) a length difference between the two straight-line segments is less than 0.3 times a length of either straight-line segment;

(42) if a relationship between some straight-line segment and the straight-line segment meets all the requirements above, a number of similar straight-line segments corresponding to the straight-line segment is increased by one; and

(43) 3. a region where a cumulative number of similar straight-line segments exceeds a threshold 5 is extracted to acquire an approximate fingerstall region.

(44) (III) A circle is found in the region above:

(45) 1. as shown in FIG. 6, a local positioning map above is converted into a gray-scale map;

(46) 2. Canny edge detection is performed on the gray-scale map, with a lower gradient limit of 50 and an upper gradient limit of 100 to acquire an edge map;

(47) 3. a contour is extracted for the edge map;

(48) 4. each contour is analyzed to find a contour conforming to characteristics of an ellipse:

(49) a) an area of the contour shall meet requirements of a threshold range: a lower limit of 200 pixels and an upper limit of 2500 pixels; and

(50) b) a ellipse is fitted according to a two-dimensional point set of the contour, a difference between an area of the fitted ellipse and an area of a real contour is calculated, and the difference between the two areas shall be less than 10 pixels. Otherwise, an ovality of the contour is considered to be insufficient.

(51) The contours meeting the two conditions above are retained and are considered as alternative circle patterns of the fingerstall, and a focal point of an ellipse and lengths of a long axis and a short axis of each contour are recorded.

(52) A blue part in FIG. 8 is an alternative circle found.

(53) (IV) Information of circles and straight-line segments are combined, and the alternative circles and the straight-line segments are screened.

(54) 1. First screening:

(55) a) each straight-line segment found is compared with each alternative circle to find associated straight-line segments meeting the condition around each circle, and the condition is that a distance between an end of straight-line segment (the end is a near end of each straight-line relative to a focal point) and the focal point meets requirements of upper and lower limits: a lower limit of 8 pixels and an upper limit of 45 pixels; and meanwhile, a distance between a far end (relative to the focal point) of each straight-line segment and the focal point meets a lower limit requirement of no less than 25 pixels;

(56) b) the associated straight-line segments around each circle are found according to the method above, and the alternative circle is deleted if a number of associated straight-line segments found around the circle is less than four; and

(57) c) angles, lengths and centers of the remaining associated straight-line segments around the alternative circle meeting the condition are stored.

(58) 2. Second screening:

(59) a) median length and median angle of the straight-line segments near each alternative circle are calculated, the angle and length of each straight-line segment are compared with the two medians, and the difference shall meet the requirements of upper and lower limits: a difference between the angle of each straight-line segment and the median angle cannot be greater than 5 degrees; and a difference between each straight-line segment and the median length cannot be greater than 5% of the median length;

(60) b) a line that does not meet the requirements of the upper and lower limits is deleted from an associated straight-line queue; and

(61) c) if a number of remaining associated straight-line segments around a certain circle is less than four after filtering in the previous step, the alternative circle is deleted.

(62) 3. Third screening:

(63) a) two end points of each straight-line segment cannot cross two ends of the associated alternative circle. Otherwise, it means the straight-line passes through the circle, and the circle of which the straight-line pass through will be deleted.

(64) 4. Fourth screening:

(65) a) center points of all associated straight-line segments of the alternative circles are calculated, and a center line is fitted by using the center points; and

(66) b) a distance between a center of each alternative circle and the above center line is calculated, and the distance shall be not less than 0.8 times the median length of the associated straight-line segments of the circle. The circle that does not meet the condition is deleted.

(67) 5. Fifth screening:

(68) a) a center straightness of all associated straight-line segments is calculated, i.e. an average distance from a center of each straight-line segment to a center line formed by the centers of all the associated straight-line segments; and

(69) b) if the average distance above is greater than 3 pixels, the associated straight-line group is deleted and the corresponding alternative circle is deleted at the same time.

(70) After the five rounds of screening above, the circle retained is a circle on the fingerstall finally determined, and meanwhile, information of the associated straight-line segments meeting the condition and corresponding to each circle is also saved.

(71) After completing positioning, a whole fingerstall image mask can be acquired, as shown in FIGS. 7-9, a typical mask range is composed of four parts: a rectangle (used to limit a width of the fingerstall), an ellipse in the middle of a finger, an ellipse of a fingertip, and an ellipse of a finger base.

(72) Short axes of the three ellipses in the drawing all follow a direction of an A-B line of the finger, and the direction is corresponding to a length or a length direction of a finger region. Lengths of the short axes of the three ellipses all depend on L: when an inclination degree of the finger is determined, and closer to the lens, the L is larger, and the length of the finger region is increased; and when a distance between the finger and the lens is determined, the greater the downward inclination of the fingertip is, the smaller the L is, and the shorter the length of the finger region is.

(73) The long axes of the three ellipses shown in the drawing and the width of the rectangle are all perpendicular to the direction of the A-B line (i.e. a direction of a center axis of the fingerstall, a cross section of the rubber/plastic fingerstall in a non-use state shows a symmetrical image as a whole, which is similar to an ellipse), and the direction is corresponding to the width of the finger region. All lengths of the patterns depend on R, because R is the long axis of the ellipse, thus having nothing to do with the inclination degree of the finger and being related to the distance between the finger and the lens only. When the finger is closer to the lens, the R is larger, and the width of the finger is increased at the moment.

(74) A relationship of a size and a position of each part corresponding to center C of the fingerstall, average length R of the long axes of the ellipses and average length L of the straight-lines is as follows:

(75) for the rectangle: C is used as a center, with a length of 15L and a width of 9.5R; for the ellipse in the middle of the finger: C is used as a center, with a radius of 4.3L of a short axis and a radius of 7R of a long axis; for the ellipse of the fingertip: A is used as a center, |AC|=3L, with a radius of 3L of a short axis and a radius of 3.5R of a long axis; for the ellipse of the figure base: B is used as a center, |BC|=5L, with a radius of 5L of a short axis and a radius of 6R of a long axis.

(76) As shown in FIG. 14, for the mathematical relationship described in the present invention, generally, for the lines of the same length in the image, a length of the corresponding actual parallel straight-line is proportional to a distance from the line to the optical center of a camera; and for parallel lines of the same length on a real object, a corresponding length is inversely proportional to a vertical distance from the line to the optical center on an imaged image.

(77) Embodiment 2 is a marker is used as an application scenario for general recognition to distinguish batch products. As shown in FIGS. 12 and 13, a marker is arranged on an upper surface of a mouse. In the embodiment, a fixing part of the marker is preferably in the form of adhesive plaster. The marker can be recognized by using the method in Embodiment 1.

(78) Due to primitive characteristics and reverse color characteristics of the marker itself, the marker can be recognized in application scenarios with various color backgrounds, thus ensuring the accuracy of recognition.

(79) The forgoing is only some embodiments of the invention without limiting the protection scope of the invention. The equal replacements or amendments made by those skilled in the art in the technical scope disclosed by the invention according to the technical solutions and the inventive concept of the invention shall fall within the protection scope of the invention.