Method and system for extracting image salient curve

Abstract

Provided is a method for extracting an image salient curve. The method comprises the following steps: drawing an approximate curve along a salient edge of an image from which a salient curve is to be extracted; obtaining short edges in the image; calculating a harmonic vector field by using the drawn curve as a boundary condition; filtering the short edges in the image by using the harmonic vector field; updating the vector field by using the short edges left in the image as boundary conditions; and obtaining an optimal salient curve of the image by using the energy of a minimized spline curve in the vector field. Also provided is a system for extracting an image salient curve. The image salient curve can ensure the smoothness and a bending characteristic.

Claims

1. A method for extracting image salient curve, comprising steps of: a. drawing a curve on an image from which a salient curve is to be extracted along a salient edge thereof; b. obtaining short edges in the image; c. calculating a harmonic vector field by using the drawn curve as a boundary condition; d. filtering the short edges in the image by using the harmonic vector field; e. updating the vector field by using the short edges left in the image as boundary conditions; and f. obtaining an optimal salient curve of the image by minimizing energy of a spline curve in the vector field.

2. The method of claim 1, wherein in the step b, the short edges in the image are obtained by canny operator, Prewitt operator, Sobel operator or Kirsch operator.

3. The method of claim 1, wherein the step c comprises deeming the curve drawn in the step a as a boundary of a two-dimensional manifold and solving u=0 (u is the vector field) by using tangential vectors at the boundary as boundary values to obtain the harmonic vector field.

4. The method of claim 1, wherein the step d comprises defining a score for each of the short edges: Score=1*a length of its own2*an average angle with respect to the harmonic vector field3*a mean curvature4*an average distance from the curve drawn by a user, wherein 1, 2, 3 and 4 are weights of items, respectively.

5. The method of claim 1, wherein the salient curve is an objective function as following:
f(C)=E.sub.distance(C)+E.sub.vector(C)+E.sub.smooth(C) wherein C is a cubic B-spline curve, E.sub.distance(C) defines a sum of distances of all pixels on the short edge from C, E.sub.vector(C) is an integration of angles between tangents of C and the vector field along C, and E.sub.smooth(C) is a smoothness of C which is measured by curvature.

6. A system for extracting image salient curve comprising an input unit, a short edge obtaining unit, a calculation unit, a filter unit, an update unit and a salient curve obtaining unit which are electrically connected sequentially, wherein: the input unit is used for receiving a curve drawn in an image from which the salient curve is to be extracted along a salient edge thereof; the short edge obtaining unit is used for obtaining short edges in the image; the calculation unit is used for calculating a harmonic vector field by using the curve drawn by a user as a boundary condition; the filter unit is used for filtering the short edges in the image by using the harmonic vector field; the update unit is used for updating the vector field by using the short edges left in the image as boundary conditions; the salient curve obtaining unit is used for obtaining an optimal salient curve of the image by minimizing energy of a spline curve in the vector field.

7. The system of claim 6, wherein the short edges in the image are obtained by canny operator, Prewitt operator, Sobel operator or Kirsch operator.

8. The system of claim 6, wherein the calculation unit is used for deeming the curve received by the input unit as a boundary of a two-dimensional manifold and solving u=0 (u is the vector field) by using tangential vectors at the boundary as boundary values to obtain the harmonic vector field.

9. The system of claim 6, wherein the filter unit is used for defining a score for each of the short edges: Score=1*a length of its own2*an average angle with respect to the harmonic vector field3*a mean curvature4*an average distance from the curve drawn by a user, wherein 1, 2, 3 and 4 are weights of items, respectively.

10. The system of claim 6, wherein the salient curve is an objective function as following:
f(C)=E.sub.distance(C)+E.sub.vector(C)+E.sub.smooth(C) wherein C is a cubic B-spline curve, E.sub.distance(C) defines a sum of distances of all pixels on the short edge from C, E.sub.vector(C) is an integration of angles between tangents of C and the vector field along C, and E.sub.smooth(C) is a smoothness of C which is measured by curvature.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is a flow chart of a method for extracting image salient curve according to the present disclosure;

(2) FIG. 2 is a hardware architecture diagram of a system for extracting image salient curve according to the present disclosure.

DETAILED DESCRIPTION

(3) The present disclosure will be further described with reference to the drawings and the specific embodiments.

(4) FIG. 1 shows a flow chart of a preferred embodiment of a method for extracting image salient curve according to the present disclosure.

(5) Step S401: drawing a curve on an image, from which a salient curve is to be extracted, along a salient edge thereof. Specifically, a user may manually draw a curve on the image approximately along the salient edge thereof using software which is developed based on, for example, opencv, MFC or QT library, etc.

(6) Step S402: obtaining short edges in the image. In the present embodiment, the short edges in the image may be obtained by Canny operator. Other edge detection operators may also be used to replace the Canny operator, such as Prewitt operator, Sobel operator, Kirsch operator, or the like.

(7) The specific steps may be as following:

(8) 1) filtering the image using a Gaussian filter to eliminate noise;

(9) 2) for each pixel in the image, calculating differential approximations in both transverse direction and longitudinal direction thereof to obtain magnitude and direction of the gradient of the pixel;

(10) 3) performing non-maxima suppression on the gradients of the pixels obtained above: when the gradient of a pixel is not a local maxima, it is set to zero; when the gradient of the pixel is a local maxima, it keeps unchanged. Local herein refers to a small neighborhood centered on the pixel, the radius of which may be set according to the requirements of the user.

(11) 4) filtering the pixels by means of their gradients to obtain potential edge pixels. I.e. the gradients of the pixels may be filtered by a high threshold and a low threshold and the remaining pixels may be the potential edge pixels. The gradients of the edge pixels must be between the high threshold and the low threshold.

(12) 5) connecting the edge pixels to obtain the short edges in the image.

(13) Step S403: calculating a harmonic vector field by using the curve drawn by the user as a boundary condition. The harmonic vector field may be an irrotational vector field, i.e. the curl throughout the vector field is zero. Specifically, deeming the curve drawn by the user in the step S401 as a boundary of a two-dimensional manifold, the harmonic vector field may be obtained by solving u=0 (u is the vector field) by using tangential vectors at the boundary as boundary values.

(14) When solved, the x component and the y component of u may be separately solved. This way, the problem may be converted into two partial differential equations under Dirichlet constraints.
argmin.sub.uu, s.t. u.sub.0=b

(15) Where u is a Laplace operator of the vector field u, u.sub.0 is a boundary value of the solved vector field, and b is a boundary value determined by the curve defined by the user.

(16) Step S404: filtering the short edges in the image by using the harmonic vector field. Specifically, a score may be defined for each of the short edges: Score=1*the length of its own2*the average angle with respect to the harmonic vector field3*the mean curvature4*the average distance from the curve drawn by the user, where 1, 2, 3 and 4 are weights of the items, respectively. By combination of 1, 2, 3 and 4, the short edges may be retained according to following rules: the retained short edges should be as long as possible; the tangential directions of the retained short edges should be as consistent with the harmonic vector field as possible; the curvatures of the retained short edges should be small; and the retained short edges should close to the curve drawn by the user.

(17) Step S405: updating the vector field by using the short edges left in the image as boundary conditions. Specifically, deeming the short edges left in the image as boundaries of a two-dimensional manifold, the harmonic vector field may be updated by solving u=0 (u is the vector field) by using the tangential vectors at the boundaries as the boundary values.

(18) Step S406: obtaining an optimal salient curve of the image by minimizing the energy of a spline curve in the vector field. The specific steps are as following:

(19) In the present embodiment, a cubic B-spline curve may be used to represent the salient curve. Other free curves may also be used to replace the cubic B-spline curve, such as Bezier curve, non-uniform rational B-spline, or the like.

(20) In the present embodiment, the curve drawn by the user may be used as initial values, and the salient curve may be defined as an objective function as following:
f(C)=E.sub.distance(C)+E.sub.vector(C)+E.sub.smooth(C)

(21) Where C is the cubic B-spline curve, E.sub.distance(C) defines a sum of the distances of all pixels on the short edge from C, E.sub.vector(C) is an integration of the angles between the tangents of C and the vector field along C, and E.sub.smooth(C) is the smoothness of C which may be measured by curvature.

(22) Minimizing f(C) is an unconstrained optimization problem. In the present embodiment, a BFGS operator may be used to solve the salient curve. Other optimization operators may also be used to replace the BFGS operator, such as steepest descent method, Newton method, or the like.

(23) FIG. 2 shows a hardware architecture diagram of a system for extracting image salient curve according to the present disclosure. The system may include an input unit, a short edge obtaining unit, a calculation unit, a filter unit, an update unit and a salient curve obtaining unit which are electrically connected sequentially.

(24) The input unit may be used for receiving a curve drawn in the image, from which the salient curve is to be extracted, along the salient edge thereof. Specifically, the user may manually draw a curve on the image approximately along the salient edge thereof using software which is developed based on, for example, opencv, MFC or QT library, etc, and the input unit may receive the curve drawn by the user.

(25) The short edge obtaining unit may be used for obtaining short edges in the image. In the present embodiment, the short edge obtaining unit may obtain the short edges in the image by Canny operator. Other edge detection operators may also be used to replace the Canny operator, such as Prewitt operator, Sobel operator, Kirsch operator, or the like.

(26) Specifically:

(27) 1) filtering the image using a Gaussian filter to eliminate noise;

(28) 2) for each pixel in the image, calculating differential approximations in both transverse direction and longitudinal direction thereof to obtain magnitude and direction of the gradient of the pixel;

(29) 3) performing non-maxima suppression on the gradients of the pixels obtained above: when the gradient of a pixel is not a local maxima, it is set to zero; when the gradient of the pixel is a local maxima, it keeps unchanged. Local herein refers to a small neighborhood centered on the pixel, the radius of which may be set according to the requirements of the user.

(30) 4) filtering the pixels by means of their gradients to obtain potential edge pixels. I.e. the gradients of the pixels may be filtered by a high threshold and a low threshold and the remaining pixels may be the potential edge pixels. The gradients of the edge pixels must be between the high threshold and the low threshold.

(31) 5) connecting the edge pixels to obtain the short edges in the image.

(32) The calculation unit may be used for calculating a harmonic vector field by using the curve drawn by the user as a boundary condition. The harmonic vector field may be an irrotational vector field, i.e. the curl throughout the vector field is zero. Specifically, deeming the curve drawn through the input unit by the user as a boundary of a two-dimensional manifold, the harmonic vector field may be obtained by solving u=0 (u is the vector field) by using tangential vectors at the boundary as boundary values.

(33) When solved, the x component and the y component of u may be separately solved. This way, the problem may be converted into two partial differential equations under Dirichlet constraints.
argmin.sub.uu, s.t. u.sub.0=b

(34) Where u is a Laplace operator of the vector field u, u.sub.0 is a boundary value of the solved vector field, and b is a boundary value determined by the curve defined by the user.

(35) The filter unit may be used for filtering the short edges in the image by using the harmonic vector field. Specifically, the filter unit may define a score for each of the short edges: Score=1*the length of its own2*the average angle with respect to the harmonic vector field3*the mean curvature4*the average distance from the curve drawn by the user, where 1, 2, 3 and 4 are weights of the items, respectively. By combination of 1, 2, 3 and 4, the short edges may be retained according to following rules: the retained short edges should be as long as possible; the tangential directions of the retained short edges should be as consistent with the harmonic vector field as possible; the curvatures of the retained short edges should be small; and the retained short edges should close to the curve drawn by the user.

(36) The update unit may be used for updating the vector field by using the short edges left in the image as boundary conditions. Specifically, deeming the short edges left in the image as boundaries of a two-dimensional manifold, the update unit may update the harmonic vector field by solving u=0 (u is the vector field) by using the tangential vectors at the boundaries as the boundary values.

(37) The salient curve obtaining unit may be used for obtaining an optimal salient curve of the image by minimizing the energy of a spline curve in the vector field. Specifically:

(38) In the present embodiment, a cubic B-spline curve may be used to represent the salient curve. Other free curves may also be used to replace the cubic B-spline curve, such as Bezier curve, non-uniform rational B-spline, or the like.

(39) In the present embodiment, the curve drawn by the user may be used as initial values, and the salient curve may be defined as an objective function as following:
f(C)=E.sub.distance(C)+E.sub.vector(C)+E.sub.smooth(C)

(40) Where C is the cubic B-spline curve, E.sub.distance(C) defines a sum of the distances of all pixels on the short edge from C, E.sub.vector(C) is an integration of the angles between the tangents of C and the vector field along C, and E.sub.smooth(C) is the smoothness of C which may be measured by curvature.

(41) Minimizing f(C) is an unconstrained optimization problem. In the present embodiment, a BFGS operator may be used to solve the salient curve. Other optimization operators may also be used to replace the BFGS operator, such as steepest descent method, Newton method, or the like.

(42) The present disclosure has been described with reference to the present preferred embodiments. However, a person skilled in the art should understand that the preferred embodiments above are only used to illustrate the present disclosure, but not to limit the scope of protection thereof. Any modification, equivalent, and improvement made within the spirit and principles of the present disclosure should be within the scope of the claims of the present disclosure.