Method for Extracting Significant Texture Features of B-ultrasonic Image and Application Thereof

Abstract

A method for extracting significant texture features of a B-ultrasonic image and application thereof discloses a channel attention mechanism network, i.e. a context activation residual network, which is designed to effectively model the B-ultrasonic liver fibrosis texture information, and which uses the global context information to strengthen important texture features and suppress useless texture features, such that the deep residual network can capture more significant texture information in the B-ultrasonic images. The process can be mainly divided into two phases: training and testing. During the training phase, the context activation residual network may he trained by using the B-ultrasonic image blocks as input and the pathological results of liver biopsy as labels. During the testing phase, the B-ultrasonic image blocks may be input into the trained non-invasive liver fibrosis diagnosis model to obtain the liver fibrosis staging result for each ultrasonic image.

Claims

1. A method for extracting significant texture features of a B-ultrasonic image, comprising steps of: (1) establishing multi-center ultrasonic image data sets, which comprises B-ultrasonic images of different patients and corresponding liver biopsy results; (2) cropping image blocks of three different resolutions from each of the B-ultrasonic images, for data enhancement; partitioning the image block data sets into B-ultrasonic image training sets and B-ultrasonic image testing sets at a certain ratio, wherein the B-ultrasonic image block data is used as model input data, and the liver biopsy result is used as model label data; (3) performing transfer learning on a deep context activation residual network by using ImageNet 2012 data sets, to obtain a pre-trained liver fibrosis staging model; (4) fine-tuning the pre-trained liver fibrosis staging model with the B-ultrasonic image training sets obtained in step (2), to obtain a resulting B-ultrasonic liver fibrosis staging model based on the context activation residual network; (5) inputting the B-ultrasonic image testing sets obtained in step (2) into the liver fibrosis staging model obtained in step (3), to obtain liver fibrosis staging results for the B-ultrasonic images in the B-ultrasonic image testing sets; wherein, at step (3), the deep context activation residual network is formed by stacking multiple context activation residual blocks, wherein the context activation residual block consists of two parts: residual block and context activation block, wherein the residual block is used to extract texture features in the B-ultrasonic image, and each channel for the residual block is responsible for extracting texture information for different features, wherein the context activation block is used to strengthen important texture features in the residual block, while suppressing useless texture features therein, so that the residual block can be used to extract more significant texture features in the B-ultrasonic image, The functional expression of the residual block embedded into the context activation block is as follows:
y=f(x)+x=ReLU(F(BN(W.sub.3g(W.sub.2g(W.sub.1x)))))+x
g(⋅)=ReLU(BN(⋅)) where x and y are the input and output of the residual block, respectively, BN(⋅) is a batch normalization operation, ReLU(⋅) is a rectified linear unit, F(⋅) is a context activation block, W.sub.1 and W.sub.3 are both 1×1 convolutions, and W.sub.2 is 3×3 convolution.

2. The method according to claim 1, characterized in that the context activation block F(⋅) mainly comprises three operations: global context aggregation for obtaining global texture information; group normalization for eliminating inconsistent distribution of texture features caused by different samples and enhancing robustness of the model; and context activation for learning the importance weight for each of the channels with global context information, wherein the importance weight describes the importance of the texture feature learned by each of the channels, wherein the higher the value, the more important the texture feature.

3. The method according to claim 2, characterized in that to simplify, letting o=F(⋅), wherein the global context aggregation is used to obtain channel characterization vectors z=[z.sub.1, z.sub.2, . . . , z.sub.k, . . . , z.sub.c] by global average pooling operation: $Z_k=\frac{1}{H\times W}\underset{i=1}{\overset{W}{.Math.}}\underset{j=1}{\overset{H}{.Math.}}{o}_k\left(i,j\right)$ where W and H are the width and length of a feature map, C represents the number of channels in the feature map, k∈{1, 2, . . . , C}, i and j represent the spatial location points with coordinates (i, j) in the feature map, wherein the group normalization is used to group the channel characterization vectors z per channel dimension, and then normalize the feature vectors in each group, to obtain normalized channel characterization vectors v=[v.sub.1, . . . , v.sup.i, . . . , v.sup.G], where v.sup.i can be expressed as: $v^i=\frac{1}{{\sigma }^i}\left(p^i-{\mu }^i\right),{\mu }^i=\frac{1}{m}{.Math.}_{n\in S_i}{Z}_n,{\sigma }^1=\sqrt{\frac{1}{m}{.Math.}_{n\in S_i}{\left(Z_n-{\mu }^1\right)}^2+ϵ},$ where p.sup.i=[z.sub.mi+1, . . . , z.sub.m(i+1)], $m=\frac{C}{G},$ G represents the number of groups, s.sub.i represents the channel index set of the i-th group, ∈ is a small constant for ensuring stable numerical calculation, n represents the channel index, i represents the group index, μ.sup.i and σ.sup.i represent the mean and variance of the i-th group of features respectively, and v.sup.i indicates the normalized i-th group of feature vectors; wherein the context activation process comprises the step of performing a simple linear transformation on each of the channels and normalizing it to a value between 0 to 1 by a sigmoid function δ, as follows:
a=δ(β.Math.v+γ), where β and γ are learnable weights and biases, and .Math. indicates that the corresponding channels are multiplied; wherein the input is readjusted by using the learned texture importance weight, and the output of the context activation block is expressed as õ=o.Math.α; embedding the context activation block into the residual block finally, wherein the output of the residual block is re-expressed as:
y=ReLU({tilde over (o)})+x.

4. The method according to claim 1, characterized in that the process of pre-training the non-invasive diagnosis model of liver fibrosis by using ImageNet in step (3) further comprises the steps of: using the B-ultrasonic image training sets in the ImageNet data sets for training the non-invasive liver fibrosis diagnosis model, wherein the input is natural image, and the label indicates the category of each image; and using the cross-entropy between the output value of the non-invasive liver fibrosis diagnosis model and the label as objective function, wherein the weight in the model is continuously calculated and updated by a back-propagation algorithm and a gradient descent method until the value of the objective function is less than a set value or the total number of times of training is reached, thereby completing the pre-training of the non-invasive liver fibrosis diagnosis model.

5. The method according to claim 1, characterized in that the process of fine-tuning the non-invasive B-ultrasonic liver fibrosis diagnosis model in step (4) further comprises the steps of: depending on the B-ultrasonic image training sets obtained in step (2), uniformly adjusting the B-ultrasonic image blocks with different resolutions to have a resolution of 120×120, which is used as the input for the pre-trained non-invasive B-ultrasonic liver fibrosis diagnosis model; changing the size of the last output layer in the non-invasive B-ultrasonic liver fibrosis diagnosis model from 1000 to 3, wherein the pathological results of liver biopsy are used as labels; and using again the cross-entropy between the output value {tilde over (l)}=[{tilde over (l)}.sub.1, {tilde over (l)}.sub.2, {tilde over (l)}.sub.3] of the non-invasive liver fibrosis diagnosis model and the label l=[l.sub.1, l.sub.2, l.sub.3] as the objective function, wherein the weights in the model are fine-tuned by a back-propagation algorithm and a gradient descent method until the value of the objective function is less than a set threshold or a total number of times of training is reached, to obtain the non-invasive B-ultrasonic liver fibrosis diagnosis model based on the context activation residual network.

6. The method according to claim 5, characterized in that the cross-entropy objective function of the non-invasive fiver fibrosis diagnosis model is expressed as follows: $\mathrm{Loss}=\min -\frac{1}{m}\underset{i=1}{\overset{m}{.Math.}}\underset{i=1}{\overset{3}{.Math.}}{l}_i\log \left({\tilde{l}}_i\right)+\frac{1}{2}.Math.\theta .Math.2^2,$ where m is the total number of training samples, {tilde over (l)}.sub.i represents an output result of the non-invasive B-ultrasonic liver fibrosis diagnosis model, l.sub.i is 0 or 1, if and only if the i-th category is 1, and θ represents a training parameter in the model.

7. The method according to claim 1, characterized in that the process of obtaining the liver fibrosis staging results of the B-ultrasonic image testing sets in step (5) further comprises the step of: depending on the B-ultrasonic image testing sets obtained in step (2), uniformly adjusting the B-ultrasonic image blocks with different resolutions to have a resolution of 140×140, the central regions of the image blocks being cropped at a resolution of 120×120, which are input into the fine-tuned non-invasive B-ultrasonic liver fibrosis diagnosis model to obtain output vectors {tilde over (l)}=[{tilde over (l)}.sub.1, {tilde over (l)}.sub.2, {tilde over (l)}.sub.3], wherein the category corresponding to the largest value in the output vectors is taken as the final liver fibrosis staging result of the B-ultrasonic image.

8. Application of the method according to claim 1 in the diagnosis of liver fibrosis.

9. A medical device using; method according to claim 1.

10. Application of the medical device according to claim 9 in the non-invasive diagnosis of liver fibrosis.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0028] FIG. 1 is a diagram showing a context activation block;

[0029] FIG. 2 is a diagram showing a residual lock;

[0030] FIG. 3 is a diagram showing a context activation residual block; and

[0031] FIG. 4 is an overall frame view illustrating a method for extracting B-ultrasonic significant textures of liver fibrosis.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] In order to describe the present invention in more detail, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] It is provided a method for non-invasive B-ultrasonic diagnosis of liver fibrosis based on a context activation residual network, which includes the following steps.

[0034] At step S1, multi-center ultrasonic image data sets are established, which include B-ultrasonic images and pathological results of liver biopsy According to the METAVIR scoring system in combination with clinical treatment experience, liver fibrosis may be classified into three stages: normal or mild liver fibrosis (S0-S1), moderate liver fibrosis (S2-S3) and liver cirrhosis (S4). The pathological results of liver biopsy are used as labels and recorded as l=[l.sub.1, l.sub.2, l.sub.3].

[0035] At step S2, image blocks of three different resolutions are cropped from each of the B-ultrasonic images, the resolutions being 60×60, 100×100 and 120×120, respectively; and the number of the cropped blocks with each resolution being between 2 to 4. The obtained B-ultrasonic image blocks are partitioned into training sets and testing sets at a certain ratio (training sets: testing sets is 7:3).

[0036] At step S3, the context activation block (FIG. 1) is embedded into the residual block (FIG. 2) to form a context activation residual block having its structure as shown in FIG. 3, wherein the network depth is set to d. A non-invasive B-ultrasonic liver fibrosis diagnosis model is established by simply stacking those residual blocks.

[0037] At step S4, ImageNet data sets are used to pre-train the non-invasive B-ultrasonic liver fibrosis diagnosis model, and the weight parameters in the model are continuously updated by a back-propagation algorithm and a gradient descent method.

[0038] At step S5, the image blocks in the B-ultrasonic image training sets are uniformly adjusted to have a resolution of 120×120, which is used as the input for the non-invasive a-ultrasonic liver fibrosis diagnosis model, with the corresponding pathological results of liver biopsy as labels, and the weight parameters in the model are further updated by the back-propagation algorithm and the gradient descent algorithm.

[0039] At step S6, the images in the B-ultrasonic image testing sets are adjusted to have a resolution of 140×140, the central regions being cropped at a resolution of 120×120, which are input into the trained non-invasive liver fibrosis B-ultrasonic diagnosis model, to obtain the output vectors {tilde over (l)}=[{tilde over (l)}.sub.1, {tilde over (l)}.sub.2, {tilde over (l)}.sub.3]. The category-corresponding to the largest value in the vectors {tilde over (l)} is taken as the liver fibrosis staging result of the B-ultrasonic image.