Fully convolutional genetic neural network method for segmentation of infant brain record images

Abstract

Disclosed is a fully convolutional genetic neural network method for segmentation of infant brain record images. First, infant brain record image data is input and preprocessed, and genetic coding initialization is performed for parameters according to the length of a DMPGA-FCN network weight. Then, m individuals are randomly grouped into genetic native subpopulations and corresponding twin subpopulations are derived, where respective crossover probability and mutation probability pm of all the subpopulations are determined from disjoint intervals; and an optimal initialization value fa is searched for by using a genetic operator. Afterwards, fa is used as a forward propagation calculation parameter and a weighting operation is performed on the feature address featuremap. Finally, a pixel-by-pixel cross-entropy loss is calculated between predicted infant brain record images and standard segmented images to reversely update the weights, thus finally obtaining optimal weights of a network model for segmentation of the infant brain record images.

Claims

1. A fully convolutional genetic neural network method for a segmentation of infant brain record images, comprising the following steps: step 1. inputting standard segmented images of the infant brain record images, and performing a grayscale processing and an enhancement processing for the infant brain record images; then, labeling standard segmented infant brain record images by using an image labeling tool Image Labeler, wherein a training set is X=(x.sub.1, x.sub.2, . . . , x.sub.n), n=1, 2, 3, . . . ; and a label set is B=(b.sub.1, b.sub.2, . . . , b.sub.n), n=1, 2, 3, . . . ; and storing true values y_true of labels corresponding to the infant brain record images; step 2. performing a bilinear interpolation initialization f (i, j) for weight parameters of a fully convolutional genetic neural network used for the infant brain record images, wherein specific calculation is shown by the following formula (1):
f(i,j)=w.sub.1×p.sub.1+w.sub.2×p.sub.2+w.sub.3×p.sub.3+w.sub.4×p.sub.4  (1) p.sub.i denoting nearest four pixel points, i=1, 2, 3, 4; and w.sub.j denoting a corresponding weight of each point, j=1, 2, 3, 4; generating m individuals with a coding length of L by means of binary coding, wherein calculation of L is shown by the following formula (2):
L=F×F×D×N  (2) L being the coding length of genetic individuals, F being a size of convolution kernels, N being a number of the convolution kernels, and D being a depth of a convolutional layer; randomly assigning individuals to a native population Pop, and deriving twin populations Pop′ to generate an initial genetic evolution population, wherein values of a crossover probability p.sub.c and a mutation probability p.sub.m of the native population and its the twin population are both selected from disjoint intervals (0,1/2) and [1/2,1); searching a feasible domain for optimal individuals by using a genetic operator, and conducting calculation by using an inter-class variance function as a fitness function f, wherein the calculation is shown by the following formula (3):
f=n.sub.1n.sub.2(G.sub.1k−G.sub.2k).sup.2, k=1,2,3, . . . n,  (3) n.sub.1 and n.sub.2 respectively denoting numbers of pixel points of two foreground and background types in the infant brain record images, G.sub.1k denoting a gray level of a kth pixel point of a first type, and G.sub.2k denoting the gray level of the kth pixel point of a second type; criticizing a superiority of the individuals according to the fitness function f and constantly updating the individuals in populations, till a genetic algorithm meets convergence conditions, to obtain a globally optimal individuals fa; and using all the individuals in fa as initial weights of the fully convolutional genetic neural network for the infant brain record images; step 3. inputting a jth standard segmented infant brain record image in the training set to a forward propagation calculation model of the fully convolutional genetic neural network, and subjecting it to a convolution cony, a pooling pool, and a deconvolution deconv calculation through multiple layers; and finally, conducting a statistical calculation of a pixel-by-pixel classification probability, to output a predicted value matrix y_predict in a jth operation; step 4. conducting a softmax calculation softmax (y_predict) by using a true value y_true.sub.ij and a predicted value y_predict.sub.ij in a mth operation, and calculating a pixel-by-pixel cross-entropy loss of the infant brain record images, to obtain a loss value loss.sub.m in the mth operation, wherein the calculation is shown by the following formula (4):
loss.sub.m=−Σ.sub.classy_true.sub.ij log(y_predict.sub.ij)  (4) y_true.sub.ij denoting a predicted value in a ith row and a jth column of a pixel-by-pixel classification matrix, y_predict.sub.ij denoting a value in the ith row and the jth column of a classification matrix of the standard segmented infant brain record images, and class denoting a total number of i×j pixels; step 5. performing a callback of network weights by returning loos value and by means of a gradient descent algorithm, determining whether network reaches end conditions, and if yes, using final network weights as final trained weight parameters of the fully convolutional genetic neural network; and step 6. introducing the final trained weight parameters into the fully convolutional genetic neural network to form a fully convolutional genetic neural network model for the segmentation of the infant brain record images; then, inputting the infant brain record images, and subjecting the infant brain record images to the convolution cony, max-pooling max_pool, and the deconvolution deconv calculation through the multiple layers, to output segmented infant brain record images and further output pixel-by-pixel classification matrices y_ture.

2. The fully convolutional genetic neural network method for the segmentation of the infant brain record images according to claim 1, wherein the Step 2 comprises the following steps: step 2.1. establishing the forward propagation calculation model of the fully convolutional genetic neural network; randomly selecting a few of the infant brain record images from the training set of the standard segmented infant brain record images as reference images, and using the standard segmented infant brain record images, which are the rest of the standard segmented infant brain record images in the training set, as a validation set; and repeatedly performing a training and a validation by means of a cross validation; step 2.2. determining a number m of the genetic individuals, and performing the bilinear interpolation initialization f(i, j) for the weight parameters of the fully convolutional genetic neural network, wherein the calculation is shown by the formula (1); and generating the m individuals with the coding length of L by means of binary coding, wherein calculation of L is shown by the formula (2); step 2.3. randomly grouping all the individuals into native populations Pop, and deriving twin populations Pop′, to finally form the initial genetic evolution population; step 2.4. selecting the values of the crossover probability p.sub.c and the mutation probability p.sub.m of each pair of the native population Pop and its the twin population Pop′ from the disjoint intervals, that is, randomly generating the crossover probability p.sub.c that is respective, and the mutation probability p.sub.m of all the populations within (0,1/2) and [1/2,1) respectively; step 2.5. selecting three optimal individuals with a certain probability from an original population to a new population by using a selection operator; and then selecting any two chromosomes from the populations by using a crossover operator, and randomly selecting a gene locus according to a probability of p.sub.c in exchange with each site in the chromosomes, wherein the calculation is shown by the following formula (5):
a.sub.mi=a.sub.mi×(1−p.sub.c)+a.sub.mi×p.sub.c; a.sub.ni=a.sub.ni×(1−p.sub.c)+a.sub.ni×p.sub.c  (5) a.sub.mi denoting an ith site of a mth chromosome, a.sub.ni denoting the ith site of a nth chromosome, and p.sub.c being the crossover probability; and afterwards, randomly selecting an individual from the populations by using a mutation operator, and selecting one of the gene locus from the individual with a probability of p.sub.m for mutation, to generate a more superior individual, wherein the calculation is shown by the following formula (6):
a.sub.mn=a.sub.mn+(a.sub.mn−a.sub.max)×p.sub.m  (6) a.sub.mn denoting a nth site of the mth chromosome, a.sub.max denoting an upper genetic boundary, and p.sub.m being the mutation probability; step 2.6. replacing worst individuals in an immigrant population with the optimal individuals in an emigrant population among each of the populations by using a transfer operator, and then conducting calculation by using the inter-class variance function as the fitness function f, to criticize the superiority of the individuals, wherein the calculation is shown by the formula (3); and step 2.7. determining whether a derived multi-population genetic algorithm (DMPGA) meets the convergence conditions; and if yes, stopping optimization of the DMPGA, and using values of the optimal individuals as the initial weights of the fully convolutional genetic neural network for the infant brain record images; or otherwise, replacing poor individuals in each the populations successively with the individuals ranked according to a fitness in an elite population for re-initialization, and then the process returning to Step 2.4.

3. The fully convolutional genetic neural network method for the segmentation of the infant brain record images according to claim 1, wherein the forward propagation calculation model of the fully convolutional genetic neural network in the step 3 comprises a convolutional layer cony, a pooling layer max_pool, and a deconvolution layer deconv: step 3.1. inputting the infant brain record images, which are preprocessed, and subjecting the infant brain record images, which are preprocessed, to the convolution cony with a 3×3 convolution kernel and a max-pooling max_pool with a 2×2 pooling kernel for five times, so that the infant brain record images, which are preprocessed, are reduced by a factor of 2, 4, 8, 16, and 32 successively; step 3.2. extracting a feature address information featuremap during a third and a fourth convolution conv.sub.3 and conv.sub.4 and a max-pooling max_pool.sub.3 and max_pool.sub.4 operations, and performing a weighting operation Q on a weight of an address information, wherein the calculation is shown by the following formula (7):
q′=q(featuremap)⊕Q  (7) q(featuremap) denoting a value of a feature address information of the infant brain record images, q′ denoting a value of the feature address information featuremap after weighting, and Q denoting a weighted value after optimization of the DMPGA; step 3.3. after a five convolution and pooling operations, designing three deconvolution layers deconv with a 7×7 deconvolution kernel, and initializing a deconvolution weight by using optimal individual parameters obtained based on the DMPGA; and then performing a deconvolution for a fifth convolution and pooling result, wherein the calculation of a deconvolution size ks is shown by the following formula (8):
ks=w′+2×padding−stride×(w−1)  (8) w denoting a side length of the infant brain record images input in deconvolution, w′ denoting a side length of an output infant brain record feature image, stride denoting a stride length of a stroke of a deconvolution kernel, and padding denoting a padding value of a data matrix boundary of the infant brain record images; wherein this method is able to magnify an infant brain record feature image twice by means of upsampling, and a magnified image is integrated with the infant brain record feature image obtained after a fourth pooling and weighting operation, thus implementing superposition pixel by pixel; step 3.4. performing the deconvolution to magnify the infant brain record feature image that is combined twice, and then integrating the magnified image with a weighted feature image obtained after a third pooling operation, that is, superposing the images pixel by pixel, to obtain an infant brain record feature image reduced by a factor of 8; and then, performing a deconvolution deconv.sub.3 for a obtained image for the third time to magnify an image eight times, thus finally obtaining an image with the same size as an original infant brain record image; and step 3.5. determining a result of the pixel-by-pixel classification probability of the infant brain record images by means of a softmax function, and making judgment by using a threshold t, to generate a final result y_true.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is an overall flowchart of the present disclosure;

(2) FIG. 2 is a flowchart of a DMPGA for DMPGA-FCN weight optimization in the present disclosure; and

(3) FIG. 3 is a structural diagram of a fully convolutional genetic network of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

(4) The present disclosure is further described in detail below with reference to the accompanying drawings.

(5) As shown in FIGS. 1, 2, and 3, a fully convolutional genetic neural network method for segmentation of infant brain record images is provided, which specifically includes the following steps:

(6) The specific steps of the fully convolutional genetic neural network method for segmentation of infant brain record images are as follows:

(7) Step 1. Standard segmented images of the infant brain record images are input, and grayscale processing and enhancement processing are performed for the infant brain record images. Then, the standard segmented infant brain record images are labeled by using an image labeling tool Image Labeler, where a training set is X=(x.sub.1, x.sub.2, . . . , x.sub.n), n=1, 2, 3, . . . ; and a label set is B=(b.sub.1, b.sub.2, . . . , b.sub.n), n=1, 2, 3, . . . ; and true values y_true of the labels corresponding to the infant brain record images are stored.

(8) Step 2. Bilinear interpolation initialization f(i, j) is performed for weight parameters of the fully convolutional genetic neural network used for the infant brain record images, where specific calculation is shown by the following formula (1):
f(i,j)=w.sub.1×p.sub.1+w.sub.2×p.sub.2+w.sub.3×p.sub.3+w.sub.4×p.sub.4  (1)

(9) where p.sub.i denotes the nearest four pixel points, i=1, 2, 3, 4; and w.sub.j denotes a corresponding weight of each point, j=1, 2, 3, 4. m individuals with a coding length of L are generated by means of binary coding, where calculation of L is shown by the following formula (2):
L=F×F×D×N  (2)

(10) where L is a coding length of the genetic individual, F is the size of a convolution kernel, N is the number of the convolution kernels, and D is the depth of a convolutional layer.

(11) The individuals are randomly assigned to a native subpopulation Pop, and a twin population Pop′ is derived to generate an initial genetic evolution population.

(12) Values of a crossover probability pc and a mutation probability pm of the native population and its twin population are both selected from disjoint intervals (0,1/2) and [1/2,1). A feasible domain is searched for optimal individuals by using a genetic operator, and calculation is conducted by using the inter-class variance function as the fitness function f, where the calculation is shown by the following formula (3):
f=n.sub.1n.sub.2(G.sub.1k−G.sub.2k).sup.2, k=1,2,3, . . . n,  (3)

(13) where n.sub.1 and n.sub.2 respectively denote the numbers of the pixel points of the two foreground and background types in the infant brain record images, G.sub.1k denotes the gray level of the kth pixel point of the first type, and G.sub.2k denotes the gray level of the kth pixel point of the second type.

(14) The superiority of the individuals is criticized according to the fitness function f and the individuals in the population are constantly updated, till the genetic algorithm meets convergence conditions, to obtain globally optimal individuals fa; and all the individuals in fa are used as initial weights of the fully convolutional genetic neural network for the infant brain record images.

(15) Step 2 includes the following steps:

(16) Step 2.1. A forward propagation calculation model of the fully convolutional genetic neural network is established; a few infant brain record images are randomly selected from the training set of the standard segmented infant brain record images as reference images, and the images left in the training set are used as a validation set; and training and validation are repeatedly performed by means of cross validation.

(17) Step 2.2. The number m of the genetic individuals is determined, and bilinear interpolation initialization f(i, j) is performed for the weight parameters of the fully convolutional genetic neural network, where specific calculation is shown by the formula (1); and m individuals with a coding length of L are generated by means of binary coding, where calculation of L is shown by the formula (2).

(18) Step 2.3. All the individuals are randomly grouped into native populations Pop, and corresponding twin populations Pop′ are derived, to finally form initial genetic evolution populations.

(19) Step 2.4. Values of a crossover probability p.sub.c and a mutation probability p.sub.m of each pair of the native population Pop and its twin population Pop′ are selected from disjoint intervals, that is, the respective crossover probability p.sub.c and mutation probability p.sub.m of all the populations are randomly generated within (0,1/2) and [1/2,1) respectively.

(20) Step 2.5. Three optimal individuals are selected with a certain probability from the original population to the new population by using a selection operator; and then any two chromosomes are selected from the population by using a crossover operator, and a gene locus is randomly selected according to the probability p.sub.c in exchange with each site in the chromosomes. The calculation is shown by the following formula (4):
a.sub.mi=a.sub.mi×(1−p.sub.c)+a.sub.mi×p.sub.c; a.sub.ni=a.sub.ni×(1−p.sub.c)+a.sub.ni×p.sub.c  (4)

(21) where a.sub.mi denotes the ith site of the mth chromosome, a.sub.ni denotes the ith site of the nth chromosome, and p.sub.c is the crossover probability.

(22) Afterwards, an individual is randomly selected from the population by using a mutation operator, and a gene locus is selected from the individual with the probability p.sub.m for mutation, to generate a more superior individual. The calculation is shown by the following formula (5):
a.sub.mn=a.sub.mn+(a.sub.mn−a.sub.max)×p.sub.m  (5)

(23) where a.sub.mn denotes the nth site of the mth chromosome, a.sub.max denotes the upper genetic boundary, and p.sub.m is the mutation probability.

(24) Step 2.6. The worst individuals in the immigrant population are replaced with the optimal individuals in the emigrant population among the various populations by using a transfer operator, and then calculation is conducted by using the inter-class variance function as the fitness function f, to criticize the superiority of the individuals, where the calculation is shown by the formula (3).

(25) Step 2.7. It is determined whether a DMPGA meets convergence conditions; and if yes, optimization of the DMPGA is stopped, and values of the optimal individuals are used as initial weights of the fully convolutional genetic neural network for the infant brain record images; or otherwise, poor individuals in each population are replaced successively with individuals ranked according to the fitness in an elite population for re-initialization, and then the process returns to Step 2.4.

(26) Step 3. The jth standard segmented infant brain record image in the training set is input to the forward propagation calculation model of the fully convolutional genetic neural network, and is subjected to convolution cony, pooling pool, and deconvolution deconv calculation through multiple layers; and finally, statistical calculation of the pixel-by-pixel classification probability is conducted, to output a predicted value matrix y_predict in the jth operation.

(27) The forward propagation calculation model of the fully convolutional genetic neural network in step 3 includes a convolutional layer cony, a pooling layer max_pool, and a deconvolution layer deconv.

(28) Step 3.1. The preprocessed infant brain record images are input, and subjected to convolution cony with a 3×3 convolution kernel and max-pooling max_pool with a 2×2 pooling kernel for five times, so that the images are reduced by a factor of 2, 4, 8, 16, and 32 successively.

(29) Step 3.2. Feature address information featuremap is extracted during the third and fourth convolution conv.sub.3 and conv.sub.4 and max-pooling max_pool.sub.3 and max_pool.sub.4 operations, and a weighting operation Q is performed on the weight of the address information, where the calculation is shown by the following formula (6):
q′=q(featuremap)⊕Q  (6)

(30) where q(featuremap) denotes a value of the feature address information of the infant brain record images, q′ denotes a weighted value of the feature address information featuremap, and Q denotes a weighted value after optimization of the DMPGA.

(31) Step 3.3. After the five convolution and pooling operations, three deconvolution layers deconv with a 7×7 deconvolution kernel are designed, and the deconvolution weight is initialized by using optimal individual parameters obtained based on the DMPGA; and then deconvolution is performed for the fifth convolution and pooling result, where the calculation of the deconvolution size ks is shown by the following formula (7):
ks=w′+2×padding−stride×(w−1)  (7)

(32) where w denotes the side length of the infant brain record image input in deconvolution, w′ denotes the side length of an output infant brain record feature image, stride denotes the stride length of a stroke of the deconvolution kernel, and padding denotes the padding value of a data matrix boundary of the infant brain record images.

(33) This method can magnify the infant brain record feature image twice by means of upsampling, and the magnified image is integrated with the infant brain record feature image obtained after the fourth pooling and weighting operation, thus implementing superposition pixel by pixel.

(34) Step 3.4. Deconvolution is performed to magnify the combined infant brain record feature image twice, and then the magnified image is integrated with a weighted feature image obtained after the third pooling operation, that is, the images are superposed pixel by pixel, to obtain an infant brain record feature image reduced by a factor of 8. Then, deconvolution deconv3 is performed for the obtained imaged for the third time to magnify the image eight times, thus finally obtaining an image with the same size as the original infant brain record image.

(35) Step 3.5. A result of the pixel-by-pixel classification probability of the infant brain record images is determined by means of the softmax function, and judgment is made by using a threshold t, to generate a final result y_true.

(36) Step 4. Softmax calculation softmax (y_predict) is performed by using a true value y_true.sub.ij and a predicted value y_predict.sub.ij in the mth operation, and a pixel-by-pixel cross-entropy loss of the infant brain record images is calculated, to obtain a loss value loss.sub.m in the mth operation, where the calculation is shown by the following formula (8):
loss.sub.m=−Σ.sub.classy_true.sub.ij log(y_predict.sub.ij)  (8)

(37) where y_true.sub.ij denotes a predicted value in the ith row and jth column of a pixel-by-pixel classification matrix, y_predict.sub.ij denotes a value in the ith row and jth column of the classification matrix of the standard segmented infant brain record images, and class denotes a total number of i×j pixels.

(38) Step 5. Callback of the network weights is performed by returning the loos value and by means of a gradient descent algorithm; and it is determined whether the network reaches end conditions, and if yes, final network weights are used as the final trained weight parameters of the fully convolutional genetic neural network.

(39) Step 6. The final trained weight parameters are introduced into the network to form a fully convolutional genetic neural network model for segmentation of the infant brain record images; then, the infant brain record images are input, and are subjected to convolution cony, max-pooling max_pool, and deconvolution deconv calculation through multiple layers, to output segmented infant brain record images and further output pixel-by-pixel classification matrices y_ture.

(40) By means of the genetic algorithm DMPGA based on derivative properties of biological inheritance that is designed by the present disclosure, the twin populations are derived from the native populations so that the dimensionality of the target populations raises. The respective crossover probability and mutation probability of all the populations are determined from disjoint intervals, so as to guarantee good solution set search performance. Moreover, the algorithm can still maintain good convergence and population diversity during improvement of the dimensionality of the target populations and the solution set search performance. The distribution of solution sets obtained by means of the DMPGA is more uniform, and better convergence and diversity can be achieved, thus efficiently solving the weight optimization problem of the fully convolutional neural network.

(41) The fully convolutional genetic neural network designed by the present disclosure uses the distinguishing features of the infant brain record images optimized by means of the DMPGA to improve a network input, which overcomes the disadvantages of the conventional fully convolutional neural network only using the grayscale information as the features, thus enabling faster network convergence. Further, in the network deconvolution layer, a weighting operation is performed on the feature information recorded by the convolutional pooling layer, to enhance feature signals, so that the features are well retained, thus dealing with the problem of a low resolution of the infant brain record images to a certain extent. The method can further improve the segmentation efficiency of the infant brain record images, and is of great significance to the early and correct diagnosis of infant encephalopathy and rehabilitation of infants with the encephalopathy.

(42) The specific embodiments described above further describe the purpose, technical solutions, and beneficial effects of the present disclosure in further detail. It should be understood that the above descriptions are only specific embodiments of the present disclosure, and are not intended to limit the scope of the present disclosure. The equivalent changes and modifications made by any person skilled in the art without departing from the concept and principle of the present disclosure shall all fall within the protection scope of the present disclosure.