Method and equipment for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features

Abstract

The present disclosure discloses a method and equipment for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features, wherein the method comprising: 1) collecting eligible clinical images of patients and preprocessing the collected images; 2) extracting computer vision features from a segmented image of a hepatic tumor region; 3) extracting the manual radiomics features from the segmented image of the hepatic tumor region; 4) by combining the computer vision features and the radiomics features, screening by univariate filtering and then by LASSO regression; 5) using the features resulted from screening and clinical features together for modeling by a multivariable logistic regression model, and using the Akaike information criterion (AIC) to search backward and select clinical features suitable for the best model, so as to implement the prediction of hepatocellular carcinoma pathological grading.

Claims

1. A method for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features, comprising: step (1), obtaining and preprocessing a hepatocellular carcinoma image, wherein the preprocessing comprises: segmenting a hepatic tumor region in the hepatocellular carcinoma image, re-sampling volume pixels of a segmented image of the hepatic tumor region into a space of a fixed size, normalizing a grey scale thereof, and labeling a corresponding pathological grading result; step (2), extracting computer vision features from the segmented image of the hepatic tumor region, wherein the computer vision features comprise Location of a Gaussian (LoG) filter features, Local Binary Patterns (LBP) features, Histogram of Oriented Gradients (HOG) features, and haar-like features; step (3), extracting radiomics features from the segmented image of the hepatic tumor region, wherein the radiomics features comprise morphological features, grey scale features, texture features, and wavelet features; step (4), jointing the computer vision features extracted in step (2) and the radiomics features extracted in step (3) together, and screening features in a merged feature set X={X.sub.1, X.sub.2, . . . , X.sub.i, . . . , X.sub.n} resulted from the joint, wherein n represents a number of feature vectors in the merged feature set, X.sub.i represents an i.sup.th feature vector in the merged feature set X.sub.i={x.sub.i1, x.sub.i2, . . . , x.sub.im}, X.sub.i represents an ith element in X.sub.i, and m represents a number of elements in X.sub.i; wherein the screening features in the merged feature set comprises: step (4.1), firstly, excluding single-valued features, features having variances less than a preset threshold, features irrelevant to the corresponding pathological grading, and redundant features by univariate filtering; wherein a relationship intensity between features and the corresponding pathological grading is measured through mutual information calculations, so that the features having mutual information values lower than the preset threshold are referred to as irrelevant to the pathological grading, and the mutual information calculations are performed according to: $I\left(X_i;Y\right)=\underset{y\in Y}{.Math.}\underset{x\in X_i}{.Math.}p\left(x,y\right)\log \left[\frac{p\left(x,y\right)}{p\left(x\right)p\left(y\right)}\right]$ wherein, X.sub.i represents an i.sup.th feature vector in the merged feature set, Y represents a label, p(x, y) is a joint probability density function of X and Y, and p(x) and p(y) are, respectively, marginal probability density functions of X and Y; the redundant features are those having correlation coefficients greater than 0.9, and a correlation coefficient is to be computed according to: $\rho =\frac{\underset{k}{.Math.}\left(x_{\mathrm{ik}}-{\stackrel{\_}{x}}_i\right)\left(x_{\mathrm{ji}}-{\stackrel{\_}{x}}_j\right)}{\sqrt{\underset{k}{.Math.}{\left(x_{\mathrm{ik}}-{\stackrel{\_}{x}}_i\right)}^2\underset{i}{.Math.}{\left(x_{\mathrm{jk}}-{\stackrel{\_}{x}}_j\right)}^2}}$ wherein, ρ is a calculated result of the correlation coefficient, x.sub.ik represents a k.sup.th element in the i.sup.th feature vector in the merged feature set, x.sub.jk represents a k.sup.th element in a j.sup.th feature vector in the merged feature set, and x.sub.i and {circumflex over (x)}.sub.j respectively represent mean values of all elements in an i.sup.th feature vector and a j.sup.th feature vector in the merged feature set; step (4.2), adopting a feature dimension reduction of a Least Absolute Shrinkage and Selection Operator (Lasso) regression model to train those unhelpful feature parameters to be 0 in order to obtain a sparse solution; wherein a cost function of the Lasso regression is: $J\left(\theta \right)=\frac{1}{2m}\underset{i=1}{\overset{m}{.Math.}}{\left(y^{\left(i\right)}-\left(w{x}^{\left(i\right)}+b\right)\right)}^2+\lambda {.Math.w.Math.}_1=\frac{1}{2}\mathrm{MSE}\left(\theta \right)+\lambda \underset{i=1}{\overset{n}{.Math.}}.Math.{\theta }_i.Math.$ Wherein, m is a number of samples, n is a number of features, x.sup.(i) represents all feature values of an i.sup.th sample, y.sup.(i) represents a label value of the j.sup.th sample, w is a parameter vector of a length n, ∥w.sub.1∥ represents an L1 norm of the parameter vector w, b represents a constant coefficient, and λ is a parameter for controlling a regression complexity, wherein a best value of λ is selected by taking a binomial deviation as a criterion and then iterated by a 10-fold cross validation method to apply a λ value of the binomial deviation in the Lasso regression model; and a predict score is obtained by multiplying Lasso-selected features by their respective coefficients to provide products and then summing the products; step (5), after clinical features are obtained from a hepatic tumor region image and all the clinical features are deleted one by one, adopting a multivariable logistic regression, by combining the image with all feature matrices resulted from screening in step (4), to get a classifier for pathological grading of hepatocellular carcinoma for different combinations; using an Akaike information criterion (AIC) as a criterion to search backward a most proper combination thereof, and then according to an influence of the binomial deviation and the number of variables during selection selecting a model of a lowest AIC score as a final classifier for pathological grading of hepatocellular carcinoma which may input a best combined feature matrix; wherein the AIC is defined as:
AIC=−2 ln(L)+2k wherein L is a maximum likelihood under this model, and k is a number of model variables; step (6), acquiring the hepatocellular carcinoma image to go through the pathological grading of hepatocellular carcinoma and processing according to step (1) to segment an image of the hepatic tumor region from which features of a same type as that of the best combined feature matrix said in step (5) are extracted as a feature matrix to be graded; and inputting the feature matrix to be graded into the classifier for pathological grading of hepatocellular carcinoma obtained in step (5) to output a pathological grading result of hepatocellular carcinoma.

2. The method for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features according to claim 1, wherein the computer vision features said in step (2) comprise: a. LoG filter features: LoG filter features are acquired by applying a Gauss-Laplace filter into an input image, generating an export image for each specified sigma value, and extracting intensity and texture features in multiple kinds of fineness from a LoG-filtered image; b. LBP features: LBP features are acquired by applying an LBP operator to the hepatic tumor region image to get an LBP image, and extracting intensity and texture features from the LBP image; c. HOG features: before performing an HOG algorithm, a hepatocellular carcinoma image is segmented into smaller sub-regions, histograms in a gradient direction are calculated based on pixels in cells, and the histograms of all cells are combined to form features; d. Haar-like features: after the hepatocellular carcinoma image is converted into an integral image, an area of interest is quickly calculated on different scales by using the integral image, a rectangular area is defined at each pixel, and haar features of different types are calculated wherein each haar-like feature descriptor contains two jointed rectangles: a black block and a white block, and the haar-like features are calculated by: $g\left(x\right)=\underset{B}{.Math.}\mathrm{B\_P}-\underset{W}{.Math.}\mathrm{W\_P}$ wherein, B represents a black rectangle having a pixel value of B_P; while W represents a white rectangle having a pixel value of W_P.

3. The method for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features according to claim 1, wherein the morphological features said in step (3) comprise: a surface area of a grid $A\text{:}A=\underset{i=1}{\overset{N_f}{.Math.}}{A}_i,A_i=\frac{1}{2}{\mathrm{Oa}}_i\times {\mathrm{Ob}}_i,$  wherein O.sub.ia.sub.i and O.sub.ib.sub.i are edges of an i.sup.th triangle in the grid, which are constituted of vertexes a.sub.i, b.sub.i and an origin O, N.sub.f represents a number of triangles in the grid; a surface area of a pixel $A_{\mathrm{pixel}}\text{:}{A}_{\mathrm{pixel}}=\stackrel{N_v}{\underset{k=1}{.Math.}}{A}_k,$  wherein A.sub.k represents a surface area of a single pixel A.sub.k, and N.sub.v represents a number of pixels; a perimeter $P:P=\underset{i=1}{\overset{N_w}{.Math.}}{P}_{i,}$  P.sub.i=√{square root over ((a.sub.i−b.sub.i).sup.2)}, wherein a.sub.i and b.sub.i are vertexes of an i.sup.th line in a peripheral grid, P.sub.i represents a perimeter of each line in a grid circumference, and N.sub.w represents a total number of lines; a perimeter to surface ratio P/A; sphericity $\frac{2\sqrt{\pi A}}{P};$ spherical disproportion $\frac{P}{2\sqrt{\pi A}};$ a major axis length 4√{square root over (λ.sub.major)}, wherein λ.sub.major represents a maximum principal component value resulting from a principal component analysis performed at a physical coordinate of a pixel center; and the major axis length feature produces a maximum axis length of an ellipsoid surrounding the region of interest, which is calculated by using the maximum principal component value λ.sub.major; a minor axis length feature 4√{square root over (λ.sub.minor)}, wherein λ.sub.minor represents a second maximum principal component value resulting from the principal component analysis performed at the physical coordinate of the pixel center; and the minor axis length feature produces a second maximum axis length of an ellipsoid surrounding the region of interest, which is calculated by using the second maximum principal component value λ.sub.minor; elongation: ${\sqrt{\frac{{\lambda }_{\min \mathrm{or}}}{{\lambda }_{\mathrm{major}}}}}_{{\left[[.]\right]}_{\underset{\_}{,}}}$  and grey statistical features computed in step (3) are features resulting on the basis of the intensity histogram, which represent a distribution of pixel intensities within a hepatic tumor region, wherein the intensity histogram is defined by: $H\left(i\right)=\frac{n_i}{N_p},$
i=1, 2, . . . N.sub.g wherein, i represents a discrete intensity level, N.sub.p represents a total number of pixels in the hepatic tumor region image, N.sub.g represents a number of discrete intensity level types, n.sub.i represents a number of pixels of the discrete intensity level i in the hepatic tumor region image, H(i) represents an occurrence frequency of pixels having the discrete intensity level i in the hepatic tumor region image; it is assumed that M represents a set of all pixels in the hepatic tumor region, and M (i) represents a pixel value of an i.sup.th pixel; the grey statistical features comprise: energy: $\mathrm{energy}=\underset{i=1}{\overset{N_p}{.Math.}}{\left[M\left(i\right)+c\right]}^2,$  wherein c is optional for representing a moving pixel intensity to prevent any negative value from occurring in M; total energy: $\mathrm{total}\mathrm{energy}=V_{voxel}\underset{i=1}{\overset{N_p}{.Math.}}{\left[M\left(i\right)+c\right]}^2;$ an entropy: $\mathrm{entropy}=-\underset{i=1}{\overset{N_g}{.Math.}}H\left(i\right){\log }_2\left[H\left(i\right)+\epsilon \right],$  wherein ε represents a random small positive number; an interquartile range: Interquartile range=n.sub.75−n.sub.25, wherein n.sub.25 and n.sub.75 represent, respectively, numbers of pixels having discrete intensity levels at 25.sup.th and 75.sup.th percentiles; a mean absolute deviation: $MAD=\frac{1}{N_P}\underset{i=1}{\overset{N_P}{.Math.}}.Math.M\left(i\right)-\stackrel{\_}{M}.Math.,$  wherein M represents a mean value of frequencies; a robust mean absolute deviation: $\mathrm{rMAD}=\frac{1}{N_{10-90}}\underset{i=1}{\overset{N_{10-90}}{.Math.}}.Math.M_{10-90}\left(i\right)-{\stackrel{\_}{M}}_{10-90}.Math.,$  wherein M.sub.10-90(i) represents a pixel value of the i.sup.th pixel with a discrete intensity level between 10.sup.th and 90.sup.th percentiles, M.sub.10-90 represents a mean value, and N.sub.10-90 represents a number of types of discrete intensity levels between 10.sup.th and 90.sup.th percentiles; a robust mean absolute deviation is an average distance between each intensity value and each mean value calculated on a subset of pixel matrices having a grey scale between the 10.sup.th and 90.sup.th percentiles (included); skewness: $\mathrm{skewness}=\frac{{\mu }^3}{{\sigma }^3}=\frac{\frac{1}{N_P}\underset{i=1}{\overset{N_P}{.Math.}}{\left(M\left(i\right)-\stackrel{\_}{M}\right)}^3}{{\left(\sqrt{\frac{1}{N_P}\underset{i=1}{\overset{N_P}{.Math.}}{\left(M\left(i\right)-\stackrel{\_}{M}\right)}^2}\right)}^3};$ kurtosis: $\mathrm{kurtosis}=\frac{{\mu }^4}{{\sigma }^4}=\frac{\frac{1}{N_P}\underset{i=1}{\overset{N_P}{.Math.}}{\left(M\left(i\right)-\stackrel{\_}{M}\right)}^4}{{\left(\frac{1}{N_P}\underset{i=1}{\overset{N_P}{.Math.}}{\left(M\left(i\right)-\stackrel{\_}{M}\right)}^2\right)}^2};$ uniformity: $\mathrm{uniformity}=\underset{i=1}{\overset{N_g}{.Math.}}{p\left(i\right)}^2;$ in addition, the computer vision features also comprise maximums, minimums, mean values, variances, and standard deviations; the texture features said in step (3) are features derived on the basis of a grey-level co-occurrence matrix (GLCM), a grey-level run-length matrix (GLRLM), a grey-level size-zone matrix (GLSZM) and a neighborhood grey-tone difference matrix (NGTDM); the wavelet features said in step (3) filter an original 2D image by non-sampling 2D wavelet transformation, wherein the original 2D image is considered as being constituted by a series of row vectors, and upon the wavelet filtering the original signal is segmented into two parts, high-frequency signals and low-frequency signals, which are then down sampled, the elements of row vectors subscripted by even numbers are reserved to generate a high-frequency matrix and a low-frequency matrix; newly generated matrices respectively go through wavelet filtering and down sampling for column vectors in order to obtain three high-frequency signals and a low-frequency signal; the above mentioned processes are then performed on the low-frequency signal for J.sub.dec times to finally obtain J.sub.dec×3 high-frequency signals and a low-frequency approximated image; and a modulus square expression of a Daubechies wavelet applied is:
|m.sub.0(ω)|.sup.2=[cos.sup.2(ω/2)].sup.NP[sin.sup.2(ω/2)] wherein, the wavelet has a discrete version as $m_0\left(\omega \right)=\frac{1}{\sqrt{2}}\underset{k=0}{\overset{2N-1}{.Math.}}{h}_k{e}^{-\mathrm{jk}\omega },$ $P\left(y\right)=\underset{k=0}{\overset{N-1}{.Math.}}{C}_k^{N-1+k}{y}^k,$  ω is an angular vector, h.sub.k is a parameter, and N represents an order of the wavelet; and grey statistical features and texture features are respectively calculated for 4 wavelet components resultant from a wavelet decomposition, so as to obtain the wavelet features.

4. A device for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features, comprising a storage and a processor, wherein: the storage is used for storing a computer program; and the processor, when performing the computer program, is used for implementing the method for classifying the hepatocellular carcinoma images by combining the computer vision features and the radiomics features according to claim 1.

5. A device for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features, comprising a storage and a processor, wherein: the storage is used for storing a computer program; and the processor, when performing the computer program, is used for implementing the method for classifying the hepatocellular carcinoma images by combining the computer vision features and the radiomics features according to claim 2.

6. A device for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features, comprising a storage and a processor, wherein: the storage is used for storing a computer program; and the processor, when performing the computer program, is used for implementing the method for classifying the hepatocellular carcinoma images by combining the computer vision features and the radiomics features according to claim 3.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is a flow chart of a method for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features;

(2) FIG. 2 is a schematic diagram of haar feature extraction;

(3) FIG. 3 is a performance testing curve of the embodiment in the present disclosure.

DESCRIPTION OF THE EMBODIMENTS

(4) The method provided in the present disclosure will be further described with reference to figures below.

(5) In Step (1), hepatocellular carcinoma image data is obtained from a patient and the image is preprocessed.

(6) Inclusion criteria of data: hepatocellular carcinoma is clearly diagnosed and graded pathologically; hepatic CT plain scan plus enhanced examination shall be performed within one month ahead of the tumor resection; the enhanced examinations all include three phases, including the arterial phase, the portal phase and the delayed phase; clinical and image data are complete and available for re-evaluation.

(7) Exclusion criteria of data: histopathological examination results show cholangiocarcinoma or mixed cell carcinoma; interventional therapy or chemotherapy against hepatocellular carcinoma have been taken before the imaging examination; imaging data or clinical data are missing; the tumor is not clearly displayed in the image enough to delineate the lesion.

(8) Image preprocessing: segmenting of the hepatic tumor region is to be implemented by an experienced radiologist and examined by another experienced radiologist; all volume pixels of the CT image are re-sampled into a space of a fixed size before the feature extraction; grey scales are normalized, and the corresponding pathological grading results are marked as labels.

(9) In Step (2), the computer vision features are extracted from the lesion region segmented in Step (1), wherein the extracted features mainly include the following types:

(10) LoG filter features: a Gauss-Laplace filter is applied to the input image, and an export image is generated for each specified value σ. The laplace transformation of the Gauss image is achieved by convolution of the image with a second order derivative (laplace operator) of the Gauss kernel. The Gauss kernel may be used to smooth the image, which is defined as:

(11) $\begin{array}{cc}G\left(x,y,\sigma \right)=\frac{1}{{\left(\sigma \sqrt{2\pi }\right)}^2}{e}^{-\frac{x^2+y^2}{2{\sigma }^2}}& \left(2‐1\right)\end{array}$

(12) The Gauss kernel is obtained by using a laplace kernel ∇.sup.2G(x, y) through convolution. A bandwidth of a filter in the Gauss kernel is determined by σ, and the value of a in the present disclosure is set as 1 mm, 2 mm, 3 mm, 4 mm, and 5 mm to obtain images of different roughnesses. The LoG filter features are obtained by extracting intensity and texture features in multiple kinds of fineness from the LoG-filtered image.

(13) LBP features: an original LBP operator is defined in a 3*3 window, a central pixel of which is taken as a threshold to compare with grey scale values of 8 neighbour pixels. If any surrounding pixel value is larger than or equal to the central pixel value, the position of the surrounding pixel is marked as 1, or otherwise 0. In this way, through comparison, 8 points within a 3*3 neighborhood can generate a 8-bit binary number (usually converted in to a decimal number, i. e. LBP code, which has a total of 256 kinds), namely obtaining an LBP value of the central pixel point in the window and using the LBP value to reflect the texture information of this region. The specific LBP expression is as follows:

(14) $\begin{array}{cc}TI\left(x\right)=\underset{p=0}{\overset{P}{.Math.}}{2}^p\times \mathrm{sgn}\left(g_c-g_p\right)& \left(2‐2\right)\end{array}$

(15) wherein, g.sub.c is a grey scale value of a central pixel point in the image, and g, is a grey scale value of a neighbour pixel point in the image; sgn(x) is a step function, and when x≥0, sgn(x)=1; if x<0, sgn(x)=0. The intensity and texture features are extracted from the LBP image to obtain the LBP features.

(16) HOG feature: synthesizes features by calculating and counting the histograms in the gradient direction in local regions of the image. Before performing an HOG algorithm, the image is segmented into smaller sub-regions (referred to as cells), and histograms in the gradient direction are calculated on the basis of pixels in cells, and the histograms of all cells are then combined to form image features. Main steps for calculating HOG include:

(17) a. The gradient calculation, wherein a 1D discrete differential template is used to process the image in both horizontal and vertical directions at the same time. The gradient value can be calculated by the following formula:

(18) $\begin{array}{cc}\theta =\arctan \left(\frac{I_y}{I_x}\right)& \left(2‐4\right)\end{array}$

(19) wherein I.sub.x and I.sub.y are gradient values in horizontal and vertical directions.

(20) b. Direction grouping, which is to create a cell histogram. The HOG cell is round in shape, and there may be two kinds of histogram channels at the same time, signed (0-360 degrees) and unsigned (0-180 degrees). In the present disclosure, 9 groups of unsigned histogram channels (in directions 0, 20, 40, 60, 80, 100, 120, 140, 160).

(21) c. Block construction and standardization, wherein respective cells are combined into a large section that is spatially connected. In this way, the HOG descriptor becomes a vector constituted by histogram components of all cell units in all sections. The generated sectional block has two types of geometries, namely an annular HOG and a rectangular HOG. As for the block standardization, an L2 norm is applied for definition as follow:

(22) 0$\begin{array}{cc}\phi =\frac{-v}{\sqrt{{.Math.v.Math.}_2^2+e^2}}& \left(2‐5\right)\end{array}$

(23) wherein v is a vector and e is a constant.

(24) D. Haar-like features

(25) Haar-like features convert an original image into an integral image, and quickly calculate the area of interest on different scales by using the integral image. The haar-like features are rectangular features which define a rectangular zone at each pixel and may calculate different types of haar features, as shown in FIG. 2:

(26) Each haar-like feature descriptor contains two jointed rectangles: a black bloc and a white block. The haar-like features can be calculated by the following formula:

(27) $\begin{array}{cc}g\left(x\right)=\underset{B}{.Math.}B\_P-\underset{W}{.Math.}W\_P& \left(2‐6\right)\end{array}$

(28) wherein, B represents the black rectangle having a pixel value of B_P; while W represents the white rectangle having a pixel value of W_P.

(29) In Step (3), common manual features in radiomics are extracted from the focal region segmented in Step (1), wherein the extracted features mainly include the following several types:

(30) A. Morphological features, each of which is calculated as:

(31) grid surface area A:

(32) $\begin{array}{cc}{A}_i=\frac{1}{2}{\mathrm{Oa}}_i\times {\mathrm{Ob}}_i,A=\underset{i=1}{\overset{N_f}{.Math.}}{A}_i& \left(3‐1\right)\end{array}$

(33) wherein O.sub.ia.sub.i and O.sub.ib.sub.i are edges of the i.sup.th triangle in the grid, which are constituted of vertexes a.sub.i, b.sub.i and an origin O, and N.sub.j represents a number of triangles in the grid.

(34) pixel surface area

(35) $\begin{array}{cc}{A}_{\mathrm{pixel}}=\underset{k=1}{\overset{N_v}{.Math.}}{A}_k& \left(3‐2\right)\end{array}$

(36) wherein A.sub.k represents a surface area of a single pixel A.sub.k, and N.sub.v represents a number of pixels;

(37) perimeter P:

(38) $\begin{array}{cc}{P}_i=\sqrt{{\left(a_i-b_i\right)}^2},P=\underset{i=1}{\overset{N_w}{.Math.}}{P}_i& \left(3‐3\right)\end{array}$

(39) wherein a.sub.i and b.sub.i are vertexes of the i.sup.th line in the peripheral grid. A perimeter P.sub.i of each line in the grid circumference is calculated at the first place. Then, a total perimeter P is obtained by summing up all calculated subregions, wherein N.sub.w represents a total number of lines.

(40) perimeter to surface ratio:

(41) $\begin{array}{cc}\mathrm{perimeter}\mathrm{to}\mathrm{surface}\mathrm{ratio}=\frac{P}{A}& \left(3‐4\right)\end{array}$

(42) sphericity:

(43) $\begin{array}{cc}\mathrm{sphericity}=\frac{2\sqrt{\pi A}}{P}& \left(3-5\right)\end{array}$

(44) spherical disproportion:

(45) $\begin{array}{cc}\mathrm{spherical}\mathrm{disproportion}=\frac{P}{2\sqrt{\pi A}}& \left(3-6\right)\end{array}$
a major axis length: major axis=4√{square root over (λ.sub.major)}  (3-7)

(46) A major axis length is calculated by using a maximum principal component λ.sub.major. The physical coordinate defining the pixel center in the region of interest is used for performing the principal component analysis. It is the same for the minor axis length.
A minor axis length: minor axis=4√{square root over (λ.sub.minor)}  (3-8)
The minor axis length feature produces the second maximum axis length of an ellipsoid surrounding the region of interest, which is calculated by using the second maximum principal component λ.sub.minor.

(47) elongation:

(48) $\begin{array}{cc}\mathrm{elongation}=\sqrt{\frac{{\lambda }_{\mathrm{minor}}}{{\lambda }_{\mathrm{major}}}}& \left(3‐9\right)\end{array}$

(49) B. First-order features (grey statistical features): it is assumed that M represents a set of all pixels in the hepatic tumor region, M (i) represents a pixel value of the i.sup.th pixel, and the following formula represents the definition of the intensity histogram as:

(50) $\begin{array}{cc}H\left(i\right)=\frac{n_i}{N_p},i=1,2,.Math.N_g& \left(3‐10\right)\end{array}$

(51) wherein, i represents discrete intensity level, N.sub.p represents a total number of pixels in a hepatic tumor region image, N.sub.g represents a number of discrete intensity level types, n.sub.i represents a number of pixels of the discrete intensity level i in the hepatic tumor region image, and H(i) represents the occurrence frequency of pixels having the discrete intensity level i in the hepatic tumor region image.

(52) The feature calculation formula contained is as follows:

(53) energy:

(54) 0$\begin{array}{cc}\mathrm{energy}=\underset{i=1}{\overset{N_p}{.Math.}}{\left[M\left(i\right)+c\right]}^2& \left(3‐11\right)\end{array}$

(55) wherein c is an optional value and represents a moving pixel intensity to prevent any negative value from occurring in M.

(56) total energy

(57) $\begin{array}{cc}\mathrm{total}\mathrm{energy}=V_{voxel}\underset{i=1}{\overset{N_p}{.Math.}}{\left[M\left(i\right)+c\right]}^2& \left(3‐12\right)\end{array}$

(58) entropy:

(59) $\begin{array}{cc}\mathrm{entrophy}=-\stackrel{N_g}{\underset{i=1}{.Math.}}H\left(i\right){\log }_2\left[H\left(i\right)+\epsilon \right]& \left(3-13\right)\end{array}$

(60) wherein ε represents a random small positive number.
an interquartile range: Interquartile range n.sub.75-n.sub.25  (3-14)

(61) A robust mean absolute deviation is an average distance between each intensity value and each mean value calculated on a subset of original pixel matrices having a grey scale between the 10.sup.th and 90.sup.th percentiles (included);

(62) a mean absolute deviation:

(63) $\begin{array}{cc}MAD=\frac{1}{N_P}\underset{i=1}{\overset{N_P}{.Math.}}.Math.M\left(i\right)-\stackrel{\_}{M}.Math.& \left(3‐15\right)\end{array}$

(64) a robust mean absolute deviation:

(65) $\begin{array}{cc}\mathrm{rMAD}=\frac{1}{N_{10-90}}\underset{i=1}{\overset{N_{10-90}}{.Math.}}.Math.M_{10-90}\left(i\right)-{\overline{M}}_{10-90}.Math.& \left(3‐16\right)\end{array}$

(66) A robust mean absolute deviation is an average distance between each intensity value and each mean value calculated on a subset of original pixel matrices having a grey scale between the 10.sup.th and 90.sup.th percentiles (included);

(67) skewness:

(68) $\begin{array}{cc}\mathrm{skewness}=\frac{{\mu }^3}{{\sigma }^3}=\frac{\frac{1}{N_P}\underset{i=1}{\overset{N_P}{.Math.}}{\left(M\left(i\right)-\stackrel{\_}{M}\right)}^3}{{\left(\sqrt{\frac{1}{N_P}\underset{i=1}{\overset{N_P}{.Math.}}{\left(M\left(i\right)-\stackrel{\_}{M}\right)}^2}\right)}^3}& \left(3‐17\right)\end{array}$

(69) kurtosis:

(70) $\begin{array}{cc}\mathrm{kurtosis}=\frac{{\mu }^4}{{\sigma }^4}=\frac{\frac{1}{N_P}\underset{i=1}{\overset{N_P}{.Math.}}{\left(M\left(i\right)-\stackrel{\_}{M}\right)}^4}{{\left(\frac{1}{N_P}\underset{i=1}{\overset{N_P}{.Math.}}{\left(M\left(i\right)-\stackrel{\_}{M}\right)}^2\right)}^2}& \left(3‐18\right)\end{array}$

(71) uniformity:

(72) $\begin{array}{cc}\mathrm{uniformity}=\underset{i=1}{\overset{N_g}{.Math.}}{H\left(i\right)}^2& \left(3‐19\right)\end{array}$

(73) In addition, there are commonly used first-order statistical features including maximums, minimums, mean values, variances, and standard deviations.

(74) C. Second-order features (texture features): features are extracted by using a grey-level co-occurrence matrix (GLCM), a grey-level run-length matrix (GLRLM), a grey-level size-zone matrix (GLSZM) and a neighborhood grey-tone difference matrix (NGTDM) of the image to describe textual distribution within the hepatic tumor.

(75) GLCM: A GLCM sized as N.sub.g×N.sub.g is described as a second-order joint probability function of an image region limited by a mask, which is defined as P(i, j|δ,θ). A (i, j).sup.th element in this matrix represents a number of times that a combination of a pixel at the i.sup.th grade and a pixel at the j.sup.th grade occurs along an angle θ in the image which are separated apart by δ pixels. δ of the central pixel is defined as a distance having infinite norms.

(76) GLRLM: In the grey-level run length matrix P(i, j|θ), the (i, j).sup.th element describes a number of run lengths that the i.sup.th grey scale and the j.sup.th length occur in the image (ROI) along an angle θ.

(77) GLSZM: In the grey-level zone matrix P(i, j), the (i, j).sup.th element equals to a number of zones occurring in the image that have the i.sup.th grey scale and the j.sup.th dimension. GLSZM only calculates one matrix in each of all directions in the region of interest.

(78) NGTDM: A neighborhood grey-tone difference matrix quantizes the difference between a neighborhood grey scale value and a mean grey scale value within the distance δ. A sum of absolute differences of the i.sup.th grey scale is stored in the matrix. Assuming that X.sub.gl is a group of segmented volume pixels, if X.sub.gl(j.sub.x, j.sub.y, j.sub.z)∈X.sub.gl is a grey scale of a volume pixel at the position (j.sub.x, j.sub.y, j.sub.z), a mean grey scale in the neighborhood will be:

(79) $\begin{array}{cc}\overline{A_i}=\overline{A}\left(j_x,j_y,j_z\right)=\frac{1}{W}\underset{k_x=-\delta }{\overset{\delta }{.Math.}}\underset{k_y=-\delta }{\overset{\delta }{.Math.}}\underset{k_z=-\delta }{\overset{\delta }{.Math.}}{x}_{\mathrm{gl}}\left(j_x+k_x,j_y+k_y,j_z+k_z\right)& \left(3‐20\right)\end{array}$

(80) wherein (j.sub.x, j.sub.y, j.sub.z)≠(0, 0, 0) and x.sub.gl(j.sub.x+k.sub.x, j.sub.y+k.sub.y, j.sub.z+k.sub.z)∈X.sub.gl. W is a number of pixels in the field.

(81) D. Wavelet features: to filter an original 2D image by non-sampling 2D (two dimensional) wavelet transformation. The 2D image is considered as being constituted by a series of row vectors, and upon the wavelet filtering the original signal is segmented into two parts, high-frequency signals and low-frequency signals, which are then down sampled. The elements of row vectors subscripted by even numbers are reserved, so that the high frequency part obtains a corresponding high frequency matrix, and the low frequency signal part obtains a new low frequency matrix. Newly generated matrices respectively go through wavelet filtering and down sampling one more time for column vectors in order to obtain three high-frequency signals and a low-frequency signal. The above mentioned processes are then performed on the low-frequency signal for J.sub.dec times to finally obtain J.sub.dec×3 high-frequency signals and a low-frequency approximated image. Dimensions in the present disclosure is set as decomposition of J.sub.dec=1 to obtain four combinations of wavelet transformation images in total, namely X.sub.L, Y.sub.L, X.sub.LY.sub.H, X.sub.HY.sub.L, X.sub.HY.sub.H, wherein X.sub.LY.sub.L represents images taking a low frequency band in the X direction and a low frequency band in the Y direction. A modulus square expression of the Daubechies wavelet applied is:
|m.sub.0(ω)|.sup.2=[cos.sup.2(ω/2)].sup.NP[sin.sup.2(ω/2)]

(82) wherein, the wavelet has a discrete version as

(83) $m_0\left(\omega \right)=\frac{1}{\sqrt{2}}\underset{k=0}{\overset{2N-1}{.Math.}}{h}_k{e}^{-jk\omega },P\left(y\right)=\underset{k=0}{\overset{N-1}{.Math.}}{C}_k^{N-1+k}{y}^k,$
ω is an angular vector, and h.sub.k is a parameter, and N represents the order of the wavelet. For 4 wavelet components resulting from the wavelet decomposition, the grey statistical features and the texture features are respectively calculated to obtain wavelet features.

(84) In Step (4), features extracted in Step (3) are screened. The feature screening mainly starts in two aspects, namely, filter and wrapper:

(85) i. Single-valued features are excluded. Likewise, features having variances that are too small are also filtered out.

(86) ii. Features apparently irrelevant to pathological grading according to the mutual information filtering. The calculation formula for mutual information is:

(87) 0$\begin{array}{cc}I\left(X;Y\right)=\underset{y\in Y}{.Math.}\underset{x\in X}{.Math.}p\left(x,y\right)\log \left[\frac{p\left(x,y\right)}{p\left(x\right)p\left(y\right)}\right]& \left(4-1\right)\end{array}$

(88) Wherein, X.sub.i represents an i.sup.th feature vector in the merged feature set, Y represents a label, p(x, y) is a joint probability density function of X and Y, and p(x) and p(y) are, respectively, marginal probability density functions of X and Y;

(89) iii. Redundant features having correlation coefficients greater than 0.9 are removed by the correlation analysis. The calculation formula for a correlation coefficient is:

(90) $\begin{array}{cc}\rho =\frac{\underset{i}{.Math.}\left(x_i-\stackrel{\_}{x}\right)\left(y_i-\stackrel{\_}{y}\right)}{\sqrt{\underset{i}{.Math.}{\left(x_i-\stackrel{\_}{x}\right)}^2\underset{i}{.Math.}{\left(y_i-\stackrel{\_}{y}\right)}^2}}& \left(4-2\right)\end{array}$

(91) wherein, ρ is a calculated result of the correlation coefficient, X.sub.ik represents a k.sup.th element in the i.sup.th feature vector in the merged feature set, x.sub.jk represents a k.sup.th element in a j.sup.th feature vector in the merged feature set, and x.sub.i and x.sub.j respectively represent mean values of all elements in an i.sup.th feature vector and a j.sup.th feature vector in the merged feature set;

(92) Then, the feature dimension reduction of a Lasso regression model is adopted to train those unhelpful feature parameters to be 0 in order to obtain a sparse solution. A cost function of the mentioned Lasso regression is:

(93) $\begin{array}{cc}J\left(\theta \right)=\frac{1}{2m}\underset{i=1}{\overset{m}{.Math.}}{\left(y^{\left(i\right)}-\left(w{x}^{\left(i\right)}+b\right)\right)}^2+\lambda {.Math.w.Math.}_1=\frac{1}{2}\mathrm{MSE}\left(\theta \right)+\lambda \underset{i=1}{\overset{n}{.Math.}}.Math.{\theta }_i.Math.& \left(4-3\right)\end{array}$

(94) Wherein w is a vector of a length n, excluding a coefficient θ.sub.0 of an intercept term, θ is a vector of a length n+1, including the coefficient θ.sub.0 of an intercept term, m is a number of samples, n is a number of features, x.sup.(i) represents all feature values of the i.sup.th sample, and y.sup.(i) represents the label value of the i.sup.th sample. ∥w.sub.1∥ represents an L1 norm of the parameter w, and b represents a constant item. The complexity of the LASSO regression is controlled by the parameter λ. The binomial deviation is taken as a criterion for selecting a best value of λ and then a value of λ having the least binomial deviation is selected by a 10-fold cross validation method to apply to the Lasso regression model. The LASSO-selected features are multiplied by their respective coefficients and then the products are summed up to create a new assembled predictive marker.

(95) In Step (5), features reserved in Step (4) without redundancy are integrated with those strongly correlated to the classification target. Considering the potential influence of clinical features, a multivariable logistic regression analysis is adopted to test a combination containing different clinical features and developed predictive markers; the Akaike information criterion (AIC) is used as a criterion to search backward a most proper combination thereof, and then the influence of binomial deviation and the number of variables during selection are put into the comprehensive consideration so as to select a model having the lowest AIC score as the final classifier for pathological grading of hepatocellular carcinoma. AIC is defined as:
AIC=−2 ln(L)+2k  (5-1)

(96) wherein L is the maximum likelihood under this model, and k is a number of model variables.

(97) The present disclosure adopts a multicenter method to train, test and verify the model. The specific scheme is as follows:

(98) In Step (5.1), data samples from an institution are randomly divided into a training set and an internal test set by a ratio of 7:3, ensuring that each pathological grade has an equal sample ratio. The models are trained in the training set and tested in the internal test set to obtain the best model.

(99) In Step (5.2), data samples from another institution are taken as an external independent validation set to verify the generalization ability of the model. The model performance is measured by a receiver operating characteristic curve (ROC) and an area under the curve (AUC).

(100) The model can be expressed as:
Grade=model(Index.sub.1,Index.sub.2, . . . ,Index.sub.n)  (5-2)

(101) wherein, Grade is a prediction result of the pathological grading of hepatocellular carcinoma.

(102) Another purpose of the present disclosure is to provide a piece of equipment for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features, wherein the equipment includes:

(103) a sampling module, used for segmenting the hepatic tumor region in the hepatocellular carcinoma image, re-sampling volume pixels of a segmented image of the hepatic tumor region into a space of a fixed size, normalizing a grey scale thereof, labeling the sample, and regarding the hepatic tumor region image containing the result label as a training set;

(104) a feature extraction module, used for extracting computer vision features, radiomics features, and clinical features from the hepatic tumor region obtained by the sampling module; jointing the computer vision feature set and the radiomics feature set together to finally obtain a merged feature set and a clinical feature set after the joint of the computer vision feature set and the radiomics feature set;

(105) a feature screening module, used for screening the merged feature set after joint, and combining and screening the feature set resultant from screening and the clinical feature set to obtain a final feature combination; and

(106) a classification module, used for classification according to the final feature combination.

(107) In the present disclosure.

(108) In the present disclosure, preferably, the sampling module includes:

(109) a segmenting sub-module: used to segment a hepatic tumor region in the hepatocellular carcinoma image; and

(110) a pre-processing sub-module: resampling the volume pixels of the segmented hepatic tumor region image into a space of a fixed size, normalizing the grey scale grad and labeling the sample.

(111) In the present disclosure, preferably, the feature extraction module includes:

(112) a sub-module of computer vision feature extraction: used for extracting computer vision features including LoG filter features, LBP features, HOG features and haar-like features;

(113) a sub-module of radiomics feature extraction: used for extracting radiomics features including morphological features, grey scale features, texture features, and wavelet features;

(114) a sub-module of clinical feature extraction: used for extracting clinical features;

(115) a sub-module of feature joint: used for jointing the computer vision feature set and the radiomics feature set to form a merged feature set.

(116) In the present disclosure, preferably, the feature screening module includes:

(117) a sub-module of the first-level feature screening: used for excluding single-valued features, features having variances less than a preset threshold, features having mutual information values less than a preset threshold, and features of correlation coefficients larger than 0.9; and

(118) a sub-module of the second level feature dimension reduction: used for adopting the feature dimension reduction in a Lasso regression model, and training unhelpful feature parameters to 0, so as to obtain a sparse solution and thus the merged feature set that has gone through screening.

(119) Another purpose of the present disclosure is to provide a piece of equipment for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features, wherein the equipment includes a storage and a processor:

(120) the storage is used for storing a computer program; and

(121) the processor, when performing the computer program, is used for implementing the method for classifying the hepatocellular carcinoma images by combining the computer vision features and the radiomics features according to any one of schemes mentioned above.

(122) Another purpose of the present disclosure is to provide a computer-readable storage medium that has a computer program is stored thereon and, when executed by a processor, implements the method for classifying the hepatocellular carcinoma images by combining the computer vision features and the radiomics features according to any scheme mentioned above.

(123) The specific effect of a classification method provided in the present disclosure will be further described below through an embodiment by using the method for classifying hepatocellular carcinoma images by combining computer vision features and radiomics features, and detailed steps thereof are discussed as above and will not be repeated herein. According to the present disclosure, after the model training is enabled in the training set and the parameter optimization is achieved in the internal test set, the performance test is finally performed on the independent validation set to obtain the ROC curve as shown in FIG. 3, and an average AUC value of 0.86 indicates the effectiveness of the model provided by the present disclosure on pathological grading of hepatocellular carcinoma.