Method for processing unmatched low-dose X-ray computed tomography image using neural network and apparatus therefor

Abstract

A method for processing an unmatched low-dose X-ray computed tomography (CT) image using a neural network and an apparatus therefor are provided. The method includes receiving a low-dose X-ray CT image and removing noise from the low-dose X-ray CT image using a unsupervised learning based neural network learned using unmatched data to reconstruct a routine-dose X-ray CT image corresponding to the low-dose X-ray CT image.

Claims

1. An image processing method, comprising: receiving a low-dose X-ray computed tomography (CT) image; removing noise from the low-dose X-ray CT image using a unsupervised learning based neural network learned using unmatched data to reconstruct a routine-dose X-ray CT image corresponding to the low-dose X-ray CT image; and learning a first neural network, a second neural network, and a discriminator based on an identity loss between a captured first low-dose X-ray CT image and a denoised first routine-dose X-ray CT image output using the first neural network, a cyclic loss between the first low-dose X-ray CT image and a second low-dose X-ray CT image including noise, output using the second neural network which receives the first routine-dose X-ray CT image, and an adversarial loss of the discriminator for discriminating between the first low-dose X-ray CT image and a third low-dose X-ray CT image for a captured second routine-dose X-ray CT image output using the second neural network; wherein the reconstructing comprises: reconstructing the routine-dose X-ray image corresponding to the low-dose X-ray CT image using the learned first neural network.

2. The image processing method of claim 1, wherein the neural network is learned based on a predefined identity loss, a predefined cyclic loss, and a predefined adversarial loss, with respect to the low-dose X-ray CT image and the routine-dose X-ray CT image.

3. The image processing method of claim 1, wherein the neural network comprises: a neural network based on a convolutional framelet.

4. An image processing method, comprising: receiving a low-dose X-ray computed tomography (CT) image; and removing noise from the low-dose X-ray CT image using a unsupervised learning based neural network learned using unmatched data to reconstruct a routine-dose X-ray CT image corresponding to the low-dose X-ray CT image; wherein the neural network comprises: a first neural network part configured to receive a first low-dose X-ray CT image and output a first routine-dose X-ray CT image corresponding to the first low-dose X-ray CT image; a second neural network part configured to receive an output image of the first neural network part and output a second low-dose X-ray CT image corresponding to the output image; a third neural network part configured to receive a second routine-dose X-ray CT image and output a third low-dose X-ray CT image corresponding to the second routine-dose X-ray CT image; a fourth neural network part configured to receive an output image of the third neural network part and output a third routine-dose X-ray CT image corresponding to the output image; a first discriminator configured to discriminate between the first low-dose X-ray CT image and the output image of the third neural network part; and a second discriminator configured to discriminate between the second routine-dose X-ray CT image and the output image of the first neural network part.

5. An image processing device, comprising: a reception unit configured to receive a low-dose X-ray CT image; and a reconstruction unit configured to remove noise from the low-dose X-ray CT image using a unsupervised learning based neural network learned using unmatched data to reconstruct a routine-dose X-ray CT image corresponding to the low-dose X-ray CT image; wherein the image processing device is configured to: learn a first neural network, a second neural network, and a discriminator based on an identity loss between a captured first low-dose X-ray CT image and a denoised first routine-dose X-ray CT image output using the first neural network, a cyclic loss between the first low-dose X-ray CT image and a second low-dose X-ray CT image including noise, output using the second neural network which receives the first routine-dose X-ray CT image, and an adversarial loss of the discriminator for discriminating between the first low-dose X-ray CT image and a third low-dose X-ray CT image for a captured second routine-dose X-ray CT image output using the second neural network; and wherein the reconstruction unit is further configured to: reconstruct the routine-dose X-ray image corresponding to the low-dose X-ray CT image using the learned first neural network.

6. The image processing device of claim 5, wherein the neural network is learned based on a predefined identity loss, a predefined cyclic loss, and a predefined adversarial loss, with respect to the low-dose X-ray CT image and the routine-dose X-ray CT image.

7. The image processing device of claim 5, wherein the neural network comprises: a neural network based on a convolutional framelet.

8. An image processing device, comprising: a reception unit configured to receive a low-dose X-ray CT image; and a reconstruction unit configured to remove noise from the low-dose X-ray CT image using a unsupervised learning based neural network learned using unmatched data to reconstruct a routine-dose X-ray CT image corresponding to the low-dose X-ray CT image; wherein the neural network comprises: a first neural network part configured to receive a first low-dose X-ray CT image and output a first routine-dose X-ray CT image corresponding to the first low-dose X-ray CT image; a second neural network part configured to receive an output image of the first neural network part and output a second low-dose X-ray CT image corresponding to the output image; a third neural network part configured to receive a second routine-dose X-ray CT image and output a third low-dose X-ray CT image corresponding to the second routine-dose X-ray CT image; a fourth neural network part configured to receive an output image of the third neural network part and output a third routine-dose X-ray CT image corresponding to the output image; a first discriminator configured to discriminate between the first low-dose X-ray CT image and the output image of the third neural network part; and a second discriminator configured to discriminate between the second routine-dose X-ray CT image and the output image of the first neural network part.

Description

BRIEF DESCRIPTION OF THE FIGURES

(1) The above and other objects and features will become apparent from the following description with reference to the following figures, wherein like reference numerals refer to like parts throughout the various figures unless otherwise specified, and wherein:

(2) FIG. 1 is a drawing illustrating an example of multiphase coronary CTA acquisition protocol;

(3) FIG. 2 is a flowchart illustrating an image processing method according to another embodiment of the inventive concept;

(4) FIG. 3 is a drawing illustrating a framework for a low-dose CT image denoising network architecture according to an embodiment of the inventive concept;

(5) FIG. 4 is a drawing illustrating an example of a network architecture of two generators;

(6) FIG. 5 is a drawing illustrating an example of a network architecture of two discriminators;

(7) FIG. 6 is a drawing illustrating an example of restoration results of a low-dose cardiac CT image by a method according to an embodiment of the inventive concept;

(8) FIG. 7 is a drawing illustrating another example of restoration results of a low-dose cardiac CT image by a method according to another embodiment of the inventive concept; and

(9) FIG. 8 is a block diagram illustrating a configuration of an image processing device according to an embodiment of the inventive concept.

DETAILED DESCRIPTION

(10) Advantages, features, and methods of accomplishing the same will become apparent with reference to embodiments described in detail below together with the accompanying drawings. However, the inventive concept is not limited by embodiments disclosed hereinafter, and may be implemented in various forms. Rather, these embodiments are provided to so that this disclosure will be through and complete and will fully convey the concept of the invention to those skilled in the art, and the inventive concept will only be defined by the appended claims.

(11) Terms used in the specification are used to describe embodiments of the inventive concept and are not intended to limit the scope of the inventive concept. In the specification, the terms of a singular form may include plural forms unless otherwise specified. The expressions “comprise” and/or “comprising” used herein indicate existence of one or more other components, steps, operations, and/or elements other than stated, components, steps, operations, and/or elements but do not exclude presence of additional elements.

(12) Unless otherwise defined herein, all terms (including technical and scientific terms) used in the specification may have the same meaning that is generally understood by a person skilled in the art. Also, terms which are defined in a dictionary and commonly used should be interpreted as not in an idealized or overly formal detect unless expressly so defined.

(13) Hereinafter, a description will be given in detail of exemplary embodiments of the inventive concept with reference to the accompanying drawings. Like reference numerals are used for the same components shown in each drawing, and a duplicated description of the same components will be omitted.

(14) In coronary CTA, even though the images at the low-dose and high-dose phases do not match each other exactly due to the cardiac motion, they are from the same cardiac volume so that they have important correspondence. Therefore, it may be conjectured that the correctly denoised low-dose phase should follow the routine-dose phase image distribution more closely and learning between two phase cardiac images is more effective than learning from totally different images.

(15) Embodiments of the inventive concept may be the gist of removing noise from a low-dose X-ray CT image using a unsupervised learning based neural network learned using unmatched data to reconstruct the low-dose X-ray CT image as a high-quality image, for example, a routine-dose X-ray CT image.

(16) Herein, an embodiment of the inventive concept may reconstruct an image using a neural network capable of cancelling noise of a CT image obtained at a low-dose phase by learning a CT image at a routine-dose phase.

(17) Although a supervised learning method is not facilitated due to a difference between basic hear structures of two phases, since two phase images have a close relation, an embodiment of the inventive concept may indeed improve the CT images at the low-dose phase by learning the distribution of the images at the high-dose phases using the cyclic consistency.

(18) An embodiment of the inventive concept may train two networks G.sub.AB: A.fwdarw.B and G.sub.BA: B.fwdarw.A where A and B are two different domains (low dose and routine dose). Herein, G.sub.AB refers to the network which has A as an input and has B as an output, and G.sub.BA refers to the network which has B as an input and has A as an output. Then, the training goal may be that G.sub.AB and G.sub.BA should be inverse of each other, that is, G.sub.BA (G.sub.AB(X.sub.A))≈X.sub.A and G.sub.AB (G.sub.BA(X.sub.B))≈X.sub.B. Thanks to the existence of inverse path, the degeneracy problem of mapping in the generative adversarial network (GAN) may be avoided. In addition, an embodiment of the inventive concept may use the identity loss. In contrast to the standard cycle GAN, the changes that an embodiment of the inventive concept expects from the generator are only the noise contributions that should be considered to avoid generating artificial features that are not present in the input images.

(19) Herein, a neural network used in an embodiment of the inventive concept may include a neural network based on a convolutional framelet.

(20) The convolutional framelet may be represented for the input signal f using the local basis ψ.sub.j and the non-local basis ϕ.sub.i and may be represented as Equation 1 below.

(21) $\begin{array}{cc}f=\frac{1}{d}\underset{i=1}{\overset{n}{.Math.}}\underset{j=1}{\overset{q}{.Math.}}.Math.f,{\varphi }_i.Math.{\psi }_j.Math.{\tilde{\varphi }}_i.Math.{\tilde{\psi }}_j& \left[\mathrm{Equation}1\right]\end{array}$

(22) Herein, ϕ.sub.i denotes the linear transform operator with the non-local basis vector, and ψ.sub.j denotes the linear transform operator with the local basis vector.

(23) In this case, the local basis vector and the non-local basis vector may have the dual basis vectors ϕ.sub.j and {tilde over (ψ)}.sub.j, respectively, which are orthogonal to each other. The orthogonal relationship between the basis vectors may be defined as Equation 2 below.

(24) $\begin{array}{cc}\tilde{\Phi }{\Phi }^T=\underset{i=1}{\overset{m}{.Math.}}{\tilde{\varphi }}_i{\varphi }_i^T=I_{n\times n},\Psi {\tilde{\Psi }}_i=\underset{j=1}{\overset{q}{.Math.}}{\psi }_j{\tilde{\psi }}_j^T=I_{d\times d}& \left[\mathrm{Equation}2\right]\end{array}$

(25) Using Equation 2 above, the convolutional framelet may be represented as Equation 3 below.
H.sub.d(f)=ΦΦ.sup.TH.sub.d(f)ΨΨ.sup.T=ΦCΨ.sup.T
C=Φ.sup.TH.sub.d(f)Ψ=Φ.sup.T(f.Math.Ψ)  [Equation 3]

(26) Herein, H.sub.d denotes the Hankel matrix operator, which may allow the convolutional operation to be represented as the matrix multiplication, and C denotes the convolutional framelet coefficient which is the signal transformed by the local basis and the non-local basis.

(27) The convolutional framelet coefficient C may be reconstructed as the original signal by applying the dual basis vectors ϕ.sub.j and {tilde over (ψ)}.sub.j. The reconstruction process may be represented as Equation 4 below.
f=(ΦC).Math.τ(Ψ)  [Equation 4]

(28) As such, the technique of representing the input signal through the local basis and the non-local basis may be the convolutional framelet.

(29) FIG. 2 is a flowchart illustrating an image processing method according to another embodiment of the inventive concept.

(30) Referring to FIG. 2, the method according to an embodiment of the inventive concept may include receiving (S210) a low-dose X-ray CT image and removing (S220) noise from the received low-dose X-ray CT image using a unsupervised based neural network learned using unmatched data to reconstruct the low dose X-ray CT image as the denoised routine-dose X-ray CT image.

(31) Herein, the low-dose X-ray CT image received in operation S210 may include a low-dose X-ray CT image of coronary CTA.

(32) Herein, the neural network used in operation S220 may include a neural network based on a convolutional framelet and may be learned based on a predefined identity loss, a predefined cyclic loss, and a predefined adversarial loss, with respect to the low-dose X-ray CT image and the routine-dose X-ray CT image.

(33) A description will be given in detail below of the method according to an embodiment of the inventive concept.

(34) Framework

(35) FIG. 3 is a drawing illustrating a framework for a low-dose CT image denoising network architecture according to an embodiment of the inventive concept.

(36) (A) shown in FIG. 3 denotes the low-dose CT domain and (B) denotes the routine-dose (normal-dose) CT domain. P.sub.A and P.sub.B denote the data distribution for each domain. The generator G.sub.AB denotes the network which receives (A) and outputs (B), and the G.sub.BA receives (B) and outputs (A). Herein, the generator G.sub.AB denotes the mapping from (A) to (B), and G.sub.BA are similarly defined as the mapping from (B) to (A).

(37) The generators G.sub.AB and G.sub.BA may be the neural network which reconstructs the low-dose CT image as the routine-dose CT image and the neural network which generate noise in the routine-dose CT image like the low-dose CT image, respectively.

(38) As for the generator, an embodiment of the inventive concept may employ the optimized network for a noise reduction in low-dose CT images. There may be two adversarial discriminators D.sub.A and D.sub.B which discriminate between measured input images and synthesized images from the generators.

(39) In other words, the framework shown in FIG. 3 may include the first generator G.sub.AB, the second generator G.sub.BA, the first discriminator D.sub.A, and the second discriminator D.sub.B. Since there is no accurately matched image, the two networks G.sub.AB and G.sub.BA may be simultaneously learned to match the two domain distributions.

(40) The first generator G.sub.AB may receive the captured low-dose X-ray CT image X.sub.A and may output the routine-dose X-ray CT image X.sub.AB denoised from the low-dose X-ray CT image. The first generator G.sub.AB may receive the low-dose X-ray CT image X.sub.BA output by the second generator G.sub.BA and may output the routine-dose X-ray CT image X.sub.BAB denoised from the low-dose X-ray CT image X.sub.BA.

(41) The second generator G.sub.BA may receive the captured routine-dose X-ray CT image x.sub.B and may output the low-dose X-ray CT image X.sub.BA in which noise is added to the routine-dose X-ray CT image x.sub.B. The second generator G.sub.BA may receive the routine-dose X-ray CT image x.sub.AB output by the first generator G.sub.AB and may output the routine-dose X-ray CT image X.sub.ABA in which noise is added to the routine-dose X-ray CT image X.sub.AB.

(42) The first discriminator D.sub.A may be the neural network which discriminates between the low-dose X-ray CT image X.sub.A and the low-dose X-ray CT image X.sub.BA. The second discriminator D.sub.B may be the neural network which discriminates between the routine-dose X-ray CT image x.sub.AB and the routine-dose X-ray CT image X.sub.B.

(43) L.sub.identity1 and L.sub.identity2 may denote the identity loss between the low-dose X-ray CT image X.sub.A and the routine-dose X-ray CT image X.sub.AB and the identity loss between the routine-dose X-ray CT image x.sub.B and the low-dose X-ray CT image x.sub.BA, respectively.

(44) L.sub.cycle1 and L.sub.cycle2 may denote the cyclic loss between the low-dose X-ray CT image X.sub.A and the routine-dose X-ray CT image X.sub.ABA and the cyclic loss between the routine-dose X-ray CT image x.sub.B and the routine-dose X-ray CT image x.sub.BAB, respectively.

(45) L.sub.adv1 and L.sub.adv2 may denote the adversarial loss of the first discriminator D.sub.A and the adversarial loss of the second discriminator D.sub.B, respectively.

(46) A description will be given in detail below of such an identity loss, a cyclic loss, and an adversarial loss.

(47) The generators and discriminators may be simultaneously trained. Specifically, an embodiment of the inventive concept aims to solve the following optimization problem:

(48) $\begin{array}{cc}\underset{G_{\mathrm{AB}},G_{\mathrm{BA}},D_A,D_B}{\min \max }\int \left(G_{\mathrm{AB}},G_{\mathrm{BA}},D_A,D_B\right)& \left[\mathrm{Equation}5\right]\end{array}$

(49) Herein, the overall loss may be defined by Equation 6 below.

(50) [Equation 6]

(51) $\int \left(G_{\mathrm{AB}},G_{\mathrm{BA}},D_A,D_B\right)={\int }_{\mathrm{GAN}}\left(G_{\mathrm{AB}},D_B,A,B\right)+{\int }_{\mathrm{GAN}}\left(G_{\mathrm{BA}},D_A,B,A\right)+\lambda {\int }_{\mathrm{cyclic}}\left(G_{\mathrm{AB}},G_{\mathrm{BA}}\right)+\gamma {\int }_{\mathrm{identity}}\left(G_{\mathrm{AB}},G_{\mathrm{BA}}\right)$

(52) Herein, λ and γ denote parameters which control the importance of the losses, and ∫.sub.GAN, ∫.sub.cyclic, and ∫.sub.identit denote the adversarial loss, cyclic loss, and identity loss, respectively.

(53) A description will be given in detail below of the adversarial loss, cyclic loss, and identity loss.

(54) Loss Formulation

(55) Adversarial Loss

(56) An embodiment of the inventive concept may employ adversarial losses using GAN. According to the original GAN, the generator G.sub.AB and the discriminator D.sub.B may be trained by solving the following min-max problem such as Equation 7 below.

(57) $\begin{array}{cc}\underset{G_{\mathrm{AB}}}{\min }\underset{D_B}{\max }{\int }_{\mathrm{GAN}}\left(G_{\mathrm{AB}},D_B,A,B\right)=E_{x_B~P_B}\left[\log D_B\left(x_B\right)\right]+E_{x_A~P_A}\left[\log \left(1-D_B\left(G_{\mathrm{AB}}\left(x_A\right)\right)\right)\right]& \left[\mathrm{Equation}7\right]\end{array}$

(58) Herein, in Equation 7 above, G.sub.AB is trained to reduce a noise in the low-dose CT image X.sub.A to make it similar to the routine-dose CT image x.sub.B, while D.sub.B is trained to discriminate between the denoised CT image G.sub.AB(x.sub.A) and the routine-dose CT image X.sub.B.

(59) However, since the original adversarial loss is unstable during training process, an embodiment of the inventive concept may change the log-likelihood function to a least square loss as in the least squares GAN (LSGAN). Then, the min-max problem may be changed to the two minimization problems as Equation 8 below.

(60) $\begin{array}{cc}\underset{G_{\mathrm{AB}}}{\min }{E}_{x_A~P_A}.Math.{\left(D_B\left(G_{\mathrm{AB}}\left(x_A\right)\right)-1\right)}^2.Math.\underset{D_B}{\min }{E}_{x_B~P_B}\left[{\left(D_B\left(x_B\right)-1\right)}^2\right]+E_{x_A~P_A}\left[{D_B\left(G_{\mathrm{AB}}\left(x_A\right)\right)}^2\right]& \left[\mathrm{Equation}8\right]\end{array}$

(61) The adversarial loss may cause the generator to generate the denoised images that may deceive the discriminator to classify them as the real images at routine doses. At the same time, the adversarial loss may guide the discriminator to well distinguish the denoised image and the routine-dose image. Similar adversarial loss may be added to the generator G.sub.BA, which generates noisy images.

(62) Cyclic Loss

(63) With the adversarial losses, an embodiment of the inventive concept could train the generator G.sub.AB and G.sub.BA to produce the realistic denoised CT images and noisy CT images, respectively, but this does not guarantee that they have an inverse relation described in FIG. 3.

(64) To enable one to one correspondence between the noisy and denoised images and to prevent from the degeneracy, the cycle which consists of two generators should be imposed to bring the input X.sub.A to the original image. The cyclic loss may be defined as Equation 9 below.
∫.sub.cyclic(G.sub.AB,G.sub.BA)=E.sub.x.sub.A.sub.˜P.sub.A└∥G.sub.BA(G.sub.AB(x.sub.A))−x.sub.A∥.sub.1┘+E.sub.x.sub.B.sub.˜P.sub.B└∥G.sub.AB(G.sub.BA(x.sub.B))−x.sub.B∥.sub.1┘  [Equation 9]

(65) Herein, ∥⋅∥.sub.1 denotes the 1.sub.1-norm. Then, the cyclic loss enforces the constraint that G.sub.AB and G.sub.BA should be inverse of each other, i.e. it encourages G.sub.BA(G.sub.AB(x.sub.A))≈X.sub.A and G.sub.AB (G.sub.BA(X.sub.B))≈X.sub.B.

(66) Identity Loss

(67) In multiphase CTA, there are often cases where the heart phase and dose modulation are not perfectly aligned as originally planned. For example, in the multiphase CTA acquisition, it is assumed that the systolic phase images should be obtained using low dose modulation, but due to the mismatch with the cardiac cycle from arrhythmia, the systolic phase image noise level may vary and even be in full dose. In this case, the input to the generator G.sub.AB may be at full dose, so it is important to train the generator so that it does not alter such clean images. Similarly, the generator G.sub.BA should not change the input images acquired at the low-dose level. To enforce the two generators G.sub.AB and G.sub.BA to satisfy these conditions, the following identity loss of Equation 10 below should be minimized.
∫.sub.identity(G.sub.AB,G.sub.BA)=E.sub.x.sub.B.sub.˜P.sub.B└∥G.sub.AB(x.sub.B)−x.sub.B∥.sub.1┘+E.sub.x.sub.A.sub.˜P.sub.A└∥G.sub.BA(x.sub.A)−x.sub.A∥.sub.1┘  [Equation 10]

(68) In other word, the generators should work as identity mappings for the input images at the target domain, as written in Equation 11 below.
G.sub.AB(x.sub.B)≈x.sub.B,G.sub.BA(x.sub.A)≈x.sub.A  [Equation 11]

(69) This identity loss may be similar to the identity loss for the photo generation from paintings in order to maintain the color composition between input and output domains. The constraints may ensure that the correctly generated output images no longer vary when used as inputs to the same network, i.e. the target domain should be the fixed points of the generator. This constraint may be important to avoid creating artificial features.

(70) Network Architecture

(71) FIG. 4 is a drawing illustrating an example of a network architecture of two generators. This network architecture shown in FIG. 4 is optimized for low-dose CT image denoising.

(72) As shown in FIG. 4, the first convolution layer may use 128 set of 3×3 convolution kernels to produce 128 channel feature maps. An embodiment of the inventive concept may have 6 sets of module composed of 3 sets of convolution layers for performing linear transform computation, batch normalization layers for performing normalization computation, and ReLU layers for performing nonlinear function computation, and one bypass connection with a ReLU layer.

(73) Convolution layers in the modules may use 128 sets of 3×3×128 convolution kernels. In addition, the proposed network has a concatenation layer that concatenates the inputs of each module and the output of the last module, which is followed by the convolution layer with 128 sets of 3×3×896 convolution kernels. Herein, this concatenation layer has a signal boosting effect using multiple signal representation and provides various paths for gradient backpropagation.

(74) The last convolution layer may use 15 sets of 3×3×128 convolution kernels. An embodiment of the inventive concept may add an end-to-end bypass connection to estimate the noise-free image while exploiting the advantages of bypass connection.

(75) FIG. 5 is a drawing illustrating an example of a network architecture of two discriminators D.sub.A and D.sub.B. The network architecture shown in FIG. 5 may include a network architecture in PatchGAN (P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.), which classifies 70×70 overlapping image patches to see whether they are real or synthesized.

(76) The network architecture shown in FIG. 5 may consist of 5 convolution layers including the last fully-connected layer. The first convolution layer may use 64 sets of 4×4 convolution kernels, and the number of convolution kernels in the following layers is twice that of the previous layer except the last fully connected layer. Arbitrary sized images may be applied to this discriminator network.

(77) Data: Cardiac CT Scans

(78) The study cohort may comprise 50 CT scans of mitral valve prolapse patients and 50 CT scans of coronary artery disease patients, and the CT scan protocols may be preset. The mean age of the population may be 58±13.2 years, and the mean body weight may be 66:2±12:6 kg. Using a second generation dual source CT scanner, electrocardiography (ECG)-gated cardiac CT scanning may be performed.

(79) Retrospective ECG-gated spiral scan with ECG-based tube current modulation may be applied to multiphase of 0-90% of the R-R interval which comprises with a full dose pulsing window of 30-80% of the R-R interval. The tube current may be reduced to 20% of the maximum outside the ECG pulsing window. A bolus of 70-90 mL of contrast material may be administered by a power injector at a rate of 4.0 mL/s and followed by 40 mL saline. The bolus tracking method including the region of interest (e.g., the ascending aorta), the attenuation threshold level (e.g., 100 HU), and the scan delay (e.g., 8 s) may be applied to determine scan time. In all CT scans, tube voltage and the tube current-exposure time product may be adjusted according to the patient's body size, and the scan parameters may include the tube voltage of 80-120 kV, the tube current-exposure time product of 185-380 mAs, the collimation of 128×0.6 mm, and the gantry rotation time of 280 s. Mean effective radiation dose of CCTA may be 11.4±6.2 mSv. Standard cardiac filter may be used for imaging reconstruction.

(80) Training

(81) Training may be performed by minimizing the loss function of Equation 6 above with λ=10 and γ=5. An embodiment of the inventive concept may apply the ADAM optimization method to train all networks with β.sub.1=0.5 and β.sub.2=0.999. The number of epochs (the number of training) may be 200, which is divided into two phases to control the learning rate during the training. In the first 100 training, an embodiment of the inventive concept may set the learning rate to 0.0002, and may linearly decrease it to zero over the next 100 training. The size of patch may be 56×56 and the size of mini-batch may be 10. Kernels may be initialized randomly from a Gaussian distribution. An embodiment of the inventive concept may alternately update the generator and the discriminator at each epoch.

(82) An embodiment of the inventive concept may normalize the intensity of the input low-dose CT images and the target routine-dose CT image using the maximum intensity value of the input low-dose CT images, and may subtract 0.5 and may multiply two to make the input image intensity range as [−1, 1].

(83) Evaluation

(84) Visual Grading Analysis

(85) An embodiment of the inventive concept may assess qualitative image quality using relative visual grading analysis (VGA). This VGA method may be planned to be related to the clinical task to evaluate any structural abnormality that may present at specific anatomical structures in CT images. Two expert cardiac radiologists establish a set of anatomical structures to evaluate image quality. Table 1 below demonstrates the 13 anatomical structures used in an embodiment of the inventive concept. The VGA scoring scale is shown in Table 2 below.

(86) TABLE-US-00001 TABLE 1 Organ Structure Left/right coronary artery LCA ostium LCA distal 1.5 cm LCA distal RCA ostium RCA 1.5 cm RCA distal Cardiac wall LV septum RV free wall margin Cardiac cavity LV trabeculation Left arterial appendage Arota Aortic root Valve Aortic valve Mitral valve

(87) TABLE-US-00002 TABLE 2 Visibility of the structures in Score relation to the reference images 1 Poor image quality 2 Lower image quality 3 Mild noise, but acceptable 4 Average 5 Good 6 Excellent

(88) All CT images including denoising CT images may be uploaded on picture archiving and communication system (PACS) for visual grading. Randomly selected 25 CT scans from mitral valve prolapse patients and 25 CT scans from coronary artery disease patients may be included for VGA. Total 1300 CT images (50 selected CT scans×13 structures×original and denoising CT) may be scored. Two radiologists may perform VGA, and all CT scans may be scored independently, without side-by-side comparison.

(89) Quantitative Analysis

(90) The image noise and signal-to-noise (SNR) of all images may be obtained at four anatomical structures, for example, ascending aorta, left ventricular cavity, left ventricular septum, and proximal right coronary artery. The size of region of interest to evaluate SNR may vary to fit each anatomic structure, but it may be confined into each structure without overlapping other structures.

(91) Statistical Analysis

(92) An embodiment of the inventive concept may compare VGA scores obtained from original CT images and denoising images using chi-square test. Image noise and SNR may be compared using paired t-test. P-values of less than 0.05 may indicate statistical significance. Herein, statistical analyses may be performed using commercial software.

(93) FIG. 6 is a drawing illustrating an example of restoration results of a low-dose cardiac CT image by a method according to an embodiment of the inventive concept. FIG. 6 illustrates restoration results from the dataset of mitral valve prolapse patients. Herein, FIG. 6 illustrates restoration results from the low-dose cardiac CT image using the generator G.sub.AB according to an embodiment of the inventive concept. The arrow indicates the distinctly different region between the input image from phase 1 and the target image from phase 8.

(94) As may be observed in the difference images (input-result) between the input image and the result image shown in FIG. 6, it may be verified that noise is only removed to reconstruct the image clearly and information of the low-dose CT image which is the real input value remains intact.

(95) FIG. 7 is a drawing illustrating another example of restoration results of a low-dose cardiac CT image by a method according to another embodiment of the inventive concept. FIG. 7 illustrates image restoration results when the noise level of the low-dose X-ray CT image is similar to that of the routine-dose X-ray CT image.

(96) For example, FIG. 7 illustrates restoration results for data with noise of the routine-dose X-ray CT image in phase 1 since the cardiac impulse and the image capture time are not exactly matched. As may be observed in FIG. 7, the neural network G.sub.AB according to an embodiment of the inventive concept satisfies predetermined identification conditions of only removing noise, shows advantages of being robust to the noise level, and does not make artificial features. This may be a considerably important portion in medical images and may be a great advantage of the neural network according to an embodiment of the inventive concept.

(97) As such, the method according to an embodiment of the inventive concept may remove noise from a low-dose X-ray CT image using a unsupervised learning based neural network learned using unmatched data to reconstruct the low-dose X-ray CT image as a high-quality image.

(98) Furthermore, the method according to an embodiment of the inventive concept may increase reconstruction performance, may reduce an amount of calculation necessary for reconstruction, and may enhance a reconstruction speed by configuring a neural network using a local basis and a non-local basis according to a convolutional framelet.

(99) Furthermore, the method according to an embodiment of the inventive concept is applicable to enhancing image quality of various medical images, such as CT and MRI images, when ensuring unmatched data, other than when it is difficult to fundamentally obtain matched data such as cardiac CT. Furthermore, the method according to an embodiment of the inventive concept is applicable to the low-dose X-ray CT image reconstruction which reduces the amount of radiation emission and is applicable to various CT techniques.

(100) FIG. 8 is a block diagram illustrating a configuration of an image processing device according to an embodiment of the inventive concept. FIG. 8 illustrates a configuration of a device which performs FIGS. 2 to 7.

(101) Referring to FIG. 8, an image processing device 800 according to an embodiment of the inventive concept may include a reception unit 810 and a reconstruction unit 820.

(102) The reception unit 810 may receive a low-dose X-ray CT image.

(103) Herein, the reception unit 810 may receive a low-dose X-ray CT image of coronary CTA.

(104) The reconstruction unit 820 may remove noise from the low-dose X-ray CT image using a unsupervised learning based neural network learned using unmatched data to reconstruct a routine-dose X-ray CT image corresponding to the low-dose X-ray CT image.

(105) Herein, the neural network may be learned based on a predefined identity loss, a predefined cyclic loss, and a predefined adversarial loss, with respect to the low-dose X-ray CT image and the routine-dose X-ray image and may be learned in a network architecture shown in FIG. 3. In detail, the neural network may learn a first generator G.sub.AB, a second generator G.sub.BA, a first discriminator D.sub.A, and a second discriminator D.sub.B based on an identity loss between a low-dose X-ray CT image X.sub.A and a routine-dose X-ray CT image X.sub.AB and identity losses L.sub.identity1 and L.sub.identity2 between a routine-dose X-ray CT image x.sub.B and a low-dose X-ray CT image X.sub.BA, a cyclic loss between the low-dose X-ray CT image X.sub.A and a routine-dose X-ray CT image X.sub.ABA, cyclic losses L.sub.cycle1 and L.sub.cycle2 between the routine-dose X-ray CT image x.sub.B and a routine-dose X-ray CT image X.sub.BAB, and an adversarial loss of the first discriminator D.sub.A and adversarial losses L.sub.adv1 and L.sub.adv2 of the second discriminator D.sub.B.

(106) Herein, the neural network may include a neural network based on a convolutional framelet.

(107) It is apparent to those skilled in the art that, although the description is omitted in the image processing device 800 of FIG. 8, the respective components configuring FIG. 8 may include all details described in FIGS. 1 to 7.

(108) The foregoing devices may be realized by hardware elements, software elements and/or combinations thereof. For example, the devices and components illustrated in the exemplary embodiments of the inventive concept may be implemented in one or more general-use computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor or any device which may execute instructions and respond. A processing unit may implement an operating system (OS) or one or software applications running on the OS. Further, the processing unit may access, store, manipulate, process and generate data in response to execution of software. It will be understood by those skilled in the art that although a single processing unit may be illustrated for convenience of understanding, the processing unit may include a plurality of processing elements and/or a plurality of types of processing elements. For example, the processing unit may include a plurality of processors or one processor and one controller. Also, the processing unit may have a different processing configuration, such as a parallel processor.

(109) Software may include computer programs, codes, instructions or one or more combinations thereof and may configure a processing unit to operate in a desired manner or may independently or collectively control the processing unit. Software and/or data may be permanently or temporarily embodied in any type of machine, components, physical equipment, virtual equipment, computer storage media or units or transmitted signal waves so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be dispersed throughout computer systems connected via networks and may be stored or executed in a dispersion manner. Software and data may be recorded in one or more computer-readable storage media.

(110) The methods according to the above-described exemplary embodiments of the inventive concept may be implemented with program instructions which may be executed through various computer means and may be recorded in computer-readable media. The media may also include, alone or in combination with the program instructions, data files, data structures, and the like. The program instructions recorded in the media may be designed and configured specially for the exemplary embodiments of the inventive concept or be known and available to those skilled in computer software. Computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as compact disc-read only memory (CD-ROM) disks and digital versatile discs (DVDs); magneto-optical media such as floptical disks; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, and the like. Program instructions include both machine codes, such as produced by a compiler, and higher level codes that may be executed by the computer using an interpreter. The described hardware devices may be configured to act as one or more software modules to perform the operations of the above-described exemplary embodiments of the inventive concept, or vice versa.

(111) According to embodiments of the inventive concept, the image processing device may remove noise from a low-dose X-ray CT image using a unsupervised learning based neural network learned using unmatched data to reconstruct the low-dose X-ray CT image as a high-quality image.

(112) According to embodiments of the inventive concept, the image processing device may increase reconstruction performance, may reduce an amount of calculation necessary for reconstruction, and may enhance a reconstruction speed by configuring a neural network using a local basis and a non-local basis according to a convolutional framelet.

(113) According to embodiments of the inventive concept, the image processing device is applicable to enhancing image quality of various medical images, such as CT and MRI images, when ensuring unmatched data, other than when it is difficult to fundamentally obtain matched data such as cardiac CT. Furthermore, the image processing device is applicable to the low-dose X-ray CT image reconstruction which reduces the amount of radiation emission and is applicable to various CT techniques.

(114) Such an embodiment of the inventive concept may be a unsupervised learning based algorithm for learning a neural network using unmatched data which departs from the limitation of the supervised learning.

(115) While a few exemplary embodiments have been shown and described with reference to the accompanying drawings, it will be apparent to those skilled in the art that various modifications and variations can be made from the foregoing descriptions. For example, adequate effects may be achieved even if the foregoing processes and methods are carried out in different order than described above, and/or the aforementioned elements, such as systems, structures, devices, or circuits, are combined or coupled in different forms and modes than as described above or be substituted or switched with other components or equivalents.

(116) Therefore, other implements, other embodiments, and equivalents to claims are within the scope of the following claims.