Method for estimating parameters of an object which is to be estimated in a digital image, and method for removing the object from the digital image
11488301 · 2022-11-01
Assignee
Inventors
Cpc classification
G06V10/772
PHYSICS
G06F18/28
PHYSICS
International classification
Abstract
A method for estimating parameters of an object which is to be estimated in a digital image which represents real imaged content, comprising at least: a) an initial step comprising the production of a dictionary of content components and the production of a dictionary of object components, the content components and the object components having the same dimensions as the digital image; b) a step of establishing, at the same time, the magnitude of each of the content components of the dictionary of content components and of the object components of the dictionary of object components present in the digital image; c) a step of establishing, from the magnitude of each of the object components, the value of at least one parameter which characterizes the object to be estimated.
Claims
1. A method for estimating one or more parameters of an object to be estimated in a digital image representing an imaged real content, comprising at least: a) an initial step comprising creating a dictionary of content components and creating a dictionary of object components, the content components and the object components having the same dimensions as the digital image; b) a step of conjointly determining an amplitude of each of the content components of the dictionary of content components and of the object components of the dictionary of object components present in the digital image; c) a step of determining a value of at least one parameter characterizing the object to be estimated on the basis of the amplitude of each of the object components.
2. The method as claimed in claim 1, the step c) of determining the value of at least one parameter characterizing the object to be estimated comprising the sub-steps of: determining a sub-set of object components of highest amplitude; determining the value of said at least one parameter depending on a linear combination of the object components of said sub-set.
3. The method as claimed in claim 1, the creating of the content dictionary comprising generating a set of images of two-dimensional sinusoidal signals.
4. The method as claimed in claim 1, the creating of the content dictionary comprising generating a set of images representative of a modulation transfer function of a sensor that forms the digital image.
5. The method as claimed in claim 1, the content dictionary being created using a machine-learning algorithm.
6. The method as claimed in claim 5, the machine-learning algorithm being a K-SVD algorithm.
7. The method as claimed in claim 1, the creating of a dictionary of object components comprising generating a set of images each having a different value of the parameter.
8. The method as claimed in claim 1, the parameter being selected from a set comprising a frequency, a shape, a size and a location of the object.
9. The method as claimed in claim 1, the conjointly determining step b) being carried out by iteratively determining an estimation vector (β) containing various amplitude values of each of the components of the content dictionary and of the object dictionary present in the digital image, until convergence of the estimation vector (β).
10. The method as claimed in claim 9, the estimation vector (β) being considered to have converged when a norm of the estimation vector (β) remains below a preset threshold for two successive iterations.
11. The method as claimed in claim 9, the estimation vector (β) being considered to have converged at an end of a preset number of iterations.
12. The method as claimed in claim 9, comprising, at an end of each iteration, a step of modifying the components of the dictionary of content components and/or of the dictionary of object components so as to minimize a projection error (ε) defined by:
ε=|y−Dβ|.sup.2, where y is a vector representing the digital image, and D is a matrix representing components of the dictionary to be modified.
13. The method as claimed in claim 9, the conjointly determining step b) comprising iteratively determining an estimation of a noise variance (σ.sup.2), being determined on the basis of the estimation vector (β) and of a correlation matrix N of noise present in the digital image, until the estimation of the noise variance (σ.sup.2) converges.
14. The method as claimed in claim 13, wherein the noise variance (σ.sup.2) is defined by:
15. The method as claimed in claim 1, the conjointly determining step b) being carried out by iteratively determining a vector resulting from a concatenation of a content estimation vector (βc), containing various estimated amplitude values of each component of the imaged real content, and of an object estimation vector (βs), containing various estimated amplitude values of each object component.
16. The method as claimed in claim 15, comprising, at an end of each iteration, a step of modifying the components of the dictionary of content components and/or of the dictionary of object components so as to minimize a projection error (ε) defined by:
ε=|y−Dcβc−Ds(μ)βs|.sup.2, where y is vector representing the digital image, Dc is a matrix representing the dictionary of content components, and Ds(μ) is a matrix representing the dictionary of object components.
17. The method as claimed in claim 16, the conjointly determining step b) comprising iteratively determining an estimation of a noise variance (σ.sup.2), wherein the noise variance (σ.sup.2) is defined by:
18. The method as claimed in claim 13, the correlation matrix N of the noise being determined via a priori knowledge of a detector used to acquire the digital image.
19. The method as claimed in claim 1, comprising a step d) of reconstructing the digital image of content via linear combination of all the content components.
20. The method as claimed in claim 1, the digital image being acquired by X-ray imaging.
21. A method for removing an object present in a digital image representing an imaged real content, comprising the steps of: estimating a value of at least one parameter of the object and an amplitude of said value, using the method as claimed in claim 1; estimating the object on a basis of the value of the at least one parameter characterizing the object and said amplitude; subtracting pixel-by-pixel the estimated object from the digital image.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Other features, details and advantages of the invention will become more clearly apparent on reading the description, which is given with reference to the appended drawings, which are given by way of example and show, respectively:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
DETAILED DESCRIPTION
(12)
(13)
(14) The initial first step a) consists in creating a dictionary of content components and in creating a dictionary of object components.
(15) A first sub-step therefore consists in creating a dictionary of content components.
(16) According to a first embodiment, the dictionary of content components may be created without a priori, by generating a set of images of basic two-dimensional sinusoidal signals.
(17) According to another embodiment, the dictionary of content components may be created by generating a set of images representative of the modulation transfer function of the sensor that forms the digital image.
(18) According to another embodiment, the dictionary of content components may be created using a machine-learning algorithm, and in particular using the K-SVD algorithm. The K-SVD algorithm is particularly suitable for creating and learning dictionary components, this making it easy to implement. The dictionary of content components using such an algorithm may for example create the components from a bank of X-ray radiography images. Other machine-learning algorithms, for example neural networks or even an SVM algorithm (SVM being the acronym of Support Vector Machine) may be used.
(19) The dimensions of each content component may be the same as the dimensions of the digital image in which the object is found. The parameter estimation may also be carried out on one portion of the digital image, for example if the object is very localized. In this case, the dimensions of each content component are the same as the dimensions of the portion of the digital image.
(20) A second sub-step of the initial first step a) consists in creating a dictionary of object components. The dictionary of object components may be created by generating a set of images each having a different value of the parameter. The parameter may be, non-exhaustively, the frequency, shape, size and location of the object.
(21)
(22) The most frequent case is that in which there is one parameter to be estimated (for example the size of the square) and the amplitude of the corresponding object. Thus, the dictionary of object components comprises as many object components as there are possible values of the parameter. The object component having the highest amplitude corresponds to the correct parameter value. It is also possible to envision the case where there are a plurality of parameters to be estimated (for example, the location and size of the square) in addition to the amplitude of the corresponding object. If the object is a sum of known components of unknown amplitudes, it is then necessary to estimate the amplitudes of each of the components.
(23) Thus a dictionary of content components and a dictionary of object components are obtained. Each component is a matrix of I rows and J columns that will be written in the form of a vector of size M, M being the size of the image, or of the portion of the image in which the object is found (i.e. all the corresponding pixels).
(24) The amplitude of each of the content components of the dictionary of content components and of the object components of the dictionary of object components present in the digital image may be conjointly determined (step b) in
1.SUP.st .Embodiment
(25) In this first embodiment, the noise present in the digital image is processed separately from the content and from the object. This separate processing may be justified by the different nature of the processed data: the data of the content and of the object are not random, whereas those of the noise are.
(26) Let D be a matrix of M×L size containing the various components to be estimated (object and content). The matrix D is therefore a concatenation of the dictionary of content components and of the dictionary of object components. L is the number of components of the dictionary of content components and of the dictionary of object components. D.sup.i, which is a vector of size M, is the i.sup.th component of the matrix D.
(27) Information on the correlation of the noise present in the digital image is put into a matrix N, different from the matrix D. The noise present in the digital image is therefore processed separately from the content and from the object. The correlation matrix N of the noise may be determined via a priori knowledge of the detector used to acquire the digital image. Specifically, the noise may be specific to each imaging technique (X-rays, infrared, visible). In particular, in X-ray imaging, the photonic noise is generally not white noise. The correlation matrix N of the noise may be estimated using the Wiener-Khintchine theorem, by taking the inverse Fourier transform of the spectral power density of a “flat” image, i.e. an image with no content.
(28) As a variant, if it is assumed that there is no correlation between the pixels, N may be an identity matrix of M×M size.
(29) Let y be a vector representing the digital image, namely the value (also called brightness) of each pixel.
(30) Let β be an estimation vector, of size L, containing the estimated amplitudes of each component D.sup.i. Thus, β=[β.sub.1, β.sub.2, . . . , β.sub.L].
(31) Let a, b and q be parameters of the algorithm, a and b depending on the nature and on knowledge of the noise. If nothing in particular is known about the noise, then a=0 and b=0.
(32) The algorithm for conjointly determining the amplitude of each of the content components of the dictionary of content components and of the object components of the dictionary of object components present in the digital image comprises an initializing step, and six iterated steps, as illustrated in
(33) Initializing Step
(34) The value of the estimation vector β may be initialized in any manner (for example with random values) or indeed by determining the scalar product between each component D.sup.i and the vector y representing the digital image:
β.sub.i=D.sup.i.Math.y,for 1≤i≤L
(35) Moreover, an estimation of the noise variance (σ.sup.2) is initialized, in any manner (for example with random values), or with the following relationship:
(36)
(37) β being at its initial value.
(38) Iterated Steps
(39) 1. P=|β|.sup.qZ is computed, where q is a parameter set by the user; it is a question of a real number comprised between 1 and 2. The user may change the value of this parameter, depending on the final result, namely depending on the quality of the removal of the object from the image. Z is a diagonal matrix allowing the estimation of certain components to be privileged—in the general case Z will be an identity matrix. Z is parameterizable by the user. 2. An estimation matrix R is computed for the covariance matrix of the signal y of the digital image:
R=Ddiag(P)D.sup.H+σ.sup.2N 3. A normalization term is computed:
Nor=|β.sub.i|.sup.−q 4. Next, each estimated amplitude of component β.sub.i is computed, using the formula:
(40)
(41)
ε=|y−Dβ|.sup.2 Each dictionary component D.sup.i may in particular be updated with an update parameter comprised between 0 and 1:
(42)
(43) These six steps are repeated until convergence of the algorithm, namely until the estimation vector β and the estimation of the noise variance σ.sup.2 converge. Since the noise variance σ.sup.2 is directly dependent on β, convergence of one implies convergence of the other.
(44) The estimation vector β and the estimation of the noise variance σ.sup.2 may be considered to have converged when the norm of the estimation vector β remains below a preset threshold for two successive iterations, and when the value of the estimation of the noise variance σ.sup.2 remains below a preset threshold for two successive iterations, respectively.
(45) As a variant, convergence may be considered to have been reached at the end of a preset number of iterations. It is then possible to determine the value of the one or more parameters of the object (step c) in
(46) The sub-set of object components of highest amplitude may in particular comprise a single component.
(47) Using the algorithm described above, it is also possible to reconstruct the content, corresponding to the imaged real content (step d) in
2.SUP.nd .Embodiment
(48) In this second embodiment, the noise present in the digital image is processed with the content and object, i.e. in one and the same matrix.
(49) With respect to the first embodiment, the matrix D is therefore the result of the concatenation of the dictionary of object components, of the dictionary of content components, and of the information on the correlation of the noise present in the digital image (which forms the matrix N in the first embodiment).
(50) y is the vector representing the digital image, namely the value of each pixel.
(51) β is the estimation vector, of size L, containing the estimated amplitudes of each component D.sup.i. Thus, β=[β.sub.1, β.sub.2, . . . , β.sub.L].
(52) q is a parameter of the algorithm. With respect to the first embodiment, there is no computation of the estimation of the noise variance σ.sup.2; the parameters a and b are therefore not used.
(53) The conjointly determining algorithm also comprises an initializing step, and five iterated steps, as illustrated in
(54) Initializing Step
(55) The value of the estimation vector β may be initialized in any manner (for example with random values) or indeed by determining the scalar product between each component D.sup.i and the vector y representing the digital image:
β.sub.i=D.sup.i.Math.y,for 1≤i≤L
Iterated Steps 1. P=|β|.sup.qZ is computed, where q is a parameter set by the user; it is a question of a real number comprised between 1 and 2. The user may change the value of this parameter, depending on the final result, namely depending on the quality of the removal of the object from the image. Z is a diagonal matrix allowing the estimation of certain components to be privileged—in the general case Z will be an identity matrix. Z is parameterizable by the user. 2. An estimation matrix R is computed for the covariance matrix of the signal y of the digital image:
R=Ddiag(P)D.sup.H 3. A normalization term is computed:
Nor=D.sup.i.sup.
(56)
(57) These five steps are repeated until convergence of the algorithm, namely until the estimation vector β converges.
(58) The estimation vector β may be considered to have converged when the norm of the estimation vector β remains below a preset threshold for two successive iterations.
(59) As a variant, convergence may be considered to have been reached at the end of a preset number of iterations. Steps c) and d) are the same as for the first embodiment.
3.SUP.rd .Embodiment
(60) In this third embodiment, the noise, the content and the object are processed separately. This separate processing may be justified by the different nature of the processed data: the noise is random, the content is modeled by a dictionary of content components and the object is modeled by a dictionary of object parameter components.
(61) Let Dc be a matrix of M×Lc size containing the various components to be estimated of the content. Let Ds(μ) be a parameter matrix of size M×Ls containing the various components to be estimated of the object. The vector β is a parameter selected from a set comprising, non-exhaustively, the frequency, shape, size and location of the object. Lc is the number of components of the dictionary of content components and Ls is the number of components of the dictionary of object components. Dc.sup.i, which is a vector of size M, is the i.sup.th component of the matrix Dc, and Ds.sup.i(μ), which is a vector of size M, is the i.sup.th component of the matrix Ds(μ).
(62) Let D be the matrix resulting from the concatenation of the dictionary Ds(μ) of object components and of the dictionary Dc of content components.
(63) Information on the correlation of the noise present in the digital image is put into a matrix N, different from the matrices Dc and Ds(μ). The noise present in the digital image is therefore processed separately from the content and from the object. The correlation matrix N of the noise may be determined via a priori knowledge of the detector used to acquire the digital image. Specifically, the noise may be specific to each imaging technique (X-rays, infrared, visible). In particular, in X-ray imaging, the photonic noise is generally not white noise. The correlation matrix N of the noise may be estimated using the Wiener-Khintchine theorem, by taking the inverse Fourier transform of the spectral power density of a “flat” image, i.e. an image with no content.
(64) As a variant, if it is assumed that there is no correlation between the pixels, N may be an identity matrix of M×M size.
(65) Let y be a vector representing the digital image, namely the value (also called brightness) of each pixel.
(66) Let βc be a content estimation vector, of size Lc, containing the estimated amplitudes of each component Dc.sup.i of the content. Thus, βc=[βc.sub.1, βc.sub.2, . . . , βc.sub.Lc].
(67) Let βs be an object estimation vector, of size Lc, containing the estimated amplitudes of each component Ds.sup.i(μ) of the object. Thus, βs=[βs.sub.1, βs.sub.2, . . . , βs.sub.LS].
(68) Let β be the vector resulting from the concatenation of the vector βs containing the amplitudes of the object components and of the vector βc containing the amplitudes of the content components.
(69) Let a, b and q be parameters of the algorithm, a and b depending on the nature and on knowledge of the noise. If nothing in particular is known about the noise, then a=0 and b=0.
(70) The algorithm for conjointly determining the amplitude of each of the content components of the dictionary of content components and of the object components of the dictionary of object components present in the digital image comprises an initializing step, and six iterated steps, as illustrated in
(71) Initializing Step
(72) The values of the content estimation vectors βc may be initialized in any manner (for example with random values) or indeed by determining the scalar product between each component Dc.sup.i and the vector y representing the digital image:
βc.sub.i=Dc.sup.i.Math.y,for 1≤i≤Lc
(73) The values of the object estimation vectors βs may be initialized in any manner (for example with random values) or indeed by determining the scalar product between each component Ds.sup.i and the vector y representing the digital image:
βs.sub.i=Ds.sup.i(μ).Math.y,for 1≤i≤Ls
(74) Moreover, an estimation of the noise variance (σ.sup.2) is initialized, in any manner (for example with random values), or with the following relationship:
(75)
(76) βc and βs being at their initial value.
(77) Iterated Steps
(78) 1. P=|β|.sup.qZ is computed, where q is a parameter set by the user; it is a question of a real number comprised between 1 and 2. The user may change the value of this parameter, depending on the final result, namely depending on the quality of the removal of the object from the image. Z is a diagonal matrix allowing the estimation of certain components to be privileged—in the general case Z will be an identity matrix. Z is parameterizable by the user. 2. An estimation matrix R is computed for the covariance matrix of the signal y of the digital image:
R=Ddiag(P)D.sup.H+σ.sup.2N 3. A normalization term is computed:
Nor=|β.sub.i|.sup.−q 4. Next, each estimated amplitude of component β.sub.i is computed, using the formula:
(79)
(80)
ε=|y−Dcβc−Ds(μ)βs|.sup.2 Each dictionary component D.sup.i may in particular be updated with a new value of the parameter p using a gradient-descent method. To do this, J, called the Jacobian of the function |y−Dcβc−Ds(μ)βs| with respect to μ:
(81)
(82) The new value of the parameter p is then given by
μ.sub.new=μ.sub.old+[J.sup.T×J].sup.−1×J.sup.T×|y−Dcβc−Ds(μ)βs|
(83) These six steps are repeated until convergence of the algorithm, namely until the estimation vector β and the estimation of the noise variance σ.sup.2 converge. Since the noise variance σ.sup.2 is directly dependent on β, convergence of one implies convergence of the other.
(84) The estimation vector β and the estimation of the noise variance σ.sup.2 may be considered to have converged when the norm of the estimation vector β remains below a preset threshold for two successive iterations, and when the value of the estimation of the noise variance σ.sup.2 remains below a preset threshold for two successive iterations, respectively.
(85) As a variant, convergence may be considered to have been reached at the end of a preset number of iterations. It is then possible to determine the value of the one or more parameters of the object (step c) in
(86) determining the value of μ.
(87) Using the algorithm described above, it is also possible to reconstruct the content, corresponding to the imaged real content (step d) in
(88) The invention also relates to a method for removing an object present in a digital image representing an imaged real content, comprising the steps of: estimating the value of at least one parameter of the object and its amplitude using the aforementioned parameter-estimating method; determining the object on the basis of the parameter of the object and its amplitude; subtracting pixel-by-pixel the object from the digital image.
(89) Using the first embodiment of the algorithm, it is possible to obtain an estimation of the size and of the amplitude of the square.
(90) The method has been described for the entirety of an image formed by the detector. It may also be applied to one portion of the image formed by the detector, by adapting the dimensions of the matrices and of the vectors required to perform the parameter estimation.
(91) The method has been described in the case of an additive object:
Image=Object+Content
(92) It may be applied to the case of a multiplicative object:
Image=Object×Content
(93) For example, the anti-scatter grid is a multiplicative object, the value of the frequency and its amplitude then depending on the patient. To pass from the additive case to the multiplicative case, it is enough to compute the logarithm of the image (and therefore of the vector y):
(Image)=log(Object)+log(Content)
(94) Using the additive method described above, to obtain the estimation vector β, it is possible to estimate log(Object) and to deduce therefrom:
Object=10.sup.log(Object)
(95) The present invention may be used in any process requiring image quality to be improved. It is applicable, inter alia, to the field of medical imaging (X-ray radiography, scanners, MRI, etc.) and to any other type of imager (X-rays, visible, infrared).
(96) Concrete applications for example concern: The removal of a perturbation, different from noise, that degrades visibility and that may lead to a poor diagnosis in the case of medical imaging, as illustrated in
(97) The method according to the invention is thus capable of correctly estimating each of the components of the dictionaries even though the latter contain many components. The use of an iterative method allows the components to be estimated one by one while filtering out each time as best as possible the other components estimated beforehand.