METHOD FOR EXTRACTING OIL STORAGE TANK BASED ON HIGH-SPATIAL-RESOLUTION REMOTE SENSING IMAGE

Abstract

A method for extracting an oil storage tank based on a high-spatial-resolution remote sensing image is provided, including: acquiring an oil storage tank sample, and randomly dividing the oil storage tank sample into a training set and a testing set; building an oil storage tank extraction model based on a Res2-Unet model structure, wherein the Res2-Unet is a deep learning network based on a UNet semantic segmentation structure, and a Res2Net convolution block is configured to change a feature interlayer learning to a granular learning and is arranged in a residual mode; and performing a precision verification on the testing set.

Claims

1. A method for extracting an oil storage tank based on a high-spatial-resolution remote sensing image, comprising: acquiring an oil storage tank sample, and randomly dividing the oil storage tank sample into a training set and a testing set; building an oil storage tank extraction model based on a Res2-Unet model structure, wherein the Res2-Unet is a deep learning network based on a UNet semantic segmentation structure, and a Res2Net convolution block is configured to change a feature interlayer learning to a granular learning and is arranged in a residual mode; and performing a precision verification on the testing set.

2. The method according to claim 1, further comprising: assisting a learning of the oil storage tank extraction model by using a boundary loss function, wherein the boundary loss function is defined by: $\begin{array}{cc}\mathrm{boundary}=I-\mathrm{Ero}\left(I\right)& \left(1\right)\end{array}$ $\begin{array}{cc}{L}_{\mathrm{boundary}}=\frac{{.Math.}_{k=1}^n{\left(B_{\mathrm{pred}}\left(k\right)-B_{\mathrm{gt}}\left(k\right)\right)}^2}{{.Math.}_{i=1}^n{B}_{\mathrm{pred}}\left(i\right)+{.Math.}_{j=1}^n{B}_{\mathrm{gt}}\left(j\right)}& \left(2\right)\end{array}$ wherein boundary represents a boundary of an extracted oil storage tank, I represents a result image of the extracted oil storage tank, Ero(I) represents an image obtained by performing an erosion calculation on the image I, B.sub.pred represents a boundary of the oil storage tank extracted by the oil storage tank extraction model, B.sub.gt represents true boundary information of the oil storage tank, and L.sub.boundary represents a boundary loss value.

3. The method according to claim 2, wherein a loss function of an entire model structure of the oil storage tank extraction model is obtained by combining a binary cross entropy BCE and the boundary loss function and is shown as:
L=μL.sub.bce+L.sub.boundary wherein a value of μ is set to 2, L represents a loss value of the entire oil storage tank extraction model, L.sub.bce represents a loss value corresponding to the binary cross entropy BCE, and L.sub.boundary represents a boundary loss value.

4. The method according to claim 1, wherein the deep learning network Res2-Unet comprises an encoding and a decoding, the encoding comprises performing a layer-wise Res2Net convolution operation and a pooling operation on an original input image, so as to obtain an image feature, and the decoding comprises performing a layer-wise deconvolution on the image feature to gradually increase a feature size and gradually reduce a dimension, so as to generate a single-dimensional binary result image with the same size as the input image.

5. The method according to claim 1, wherein the Res2Net convolution block is configured to divide a feature image obtained after performing a 1×1 convolution into four blocks according to a channel dimension, so as to obtain four feature sub-images x1, x2, x3 and x4; x1 is directly input to a feature image y1, x2 is performed by a 3×3 convolution to obtain a feature image y2, x3 and y2 are stitched and are performed by a 3×3 convolution to obtain a feature image y3, y3 and x4 are stitched and are performed by a 3×3 convolution to obtain a feature image y4, and y1, y2, y3 and y4 are stitched, are performed by a 1×1 convolution, and are stitched with an original feature image, so as to obtain an output feature image from the Res2Net convolution block.

6. The method according to claim 1, wherein the oil storage tank sample is acquired through a visual interpretation based on satellite images of GaoFen-1 (GF1) satellite, GaoFen-2 (GF2) satellite, GaoFen-6 (GF6) satellite and resource (ZY) satellite.

7. The method according to claim 1, wherein the oil storage tank sample is processed by: randomly cropping each scene of image into a 512×512 pixel patch according to a true location of the oil storage tank, and selecting green, blue and near-infrared channels as image channels

8. The method according to claim 1, wherein a process of training the oil storage tank extraction model comprises: step-wise encoding input data into a multi-channel feature, and step-wise decoding the multi-channel feature, by chaining with a feature generated in an encoding process, into a segmentation result binary image with the same size as an input image, and wherein 1 in the binary image represents the oil storage tank, and 0 in the binary image represents a background object.

9. The method according to claim 1, wherein a learning rate of the oil storage tank extraction model is set to 0.01, and a stochastic gradient descent method is used for an optimization strategy.

10. The method according to claim 1, wherein the performing a precision verification on the testing set comprises calculating an intersection over union, a F1 value, a recall rate and a precision rate, so as to evaluate a precision of the testing set.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0023] FIG. 1 shows an implementation flowchart of a method provided by the embodiments of the present disclosure.

[0024] FIG. 2 shows a structural diagram of a Res2-Unet network provided by the embodiments of the present disclosure.

[0025] FIG. 3 shows a schematic diagram of a Res2Net convolution block provided by the embodiments of the present disclosure, in which x1, x2, x3 and x4 represent channel sub-sets of a feature image, and y1, y2, y3 and y4 represent feature sub-set image formed after corresponding convolution.

[0026] FIG. 4 shows an oil storage tank extraction diagram provided by the embodiments of the present disclosure, in which (a) and (c) are original images, and (b) and (d) are oil storage tank extraction binary images.

DETAILED DESCRIPTION OF EMBODIMENTS

[0027] In order to make objectives, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below in connection with specific embodiments and with reference to the accompanying drawings.

[0028] FIG. 1 shows an implementation flowchart of a method proposed by the present disclosure. The method specifically includes the following steps.

[0029] In step 1, an oil storage tank sample is acquired, and the oil storage tank sample is randomly divided into a training set and a testing set.

[0030] According to a further embodiment of the present disclosure, the sample is acquired through a visual interpretation based on satellite images of GaoFen-1 (GF1) satellite, GaoFen-2 (GF2) satellite, GaoFen-6 (GF6) satellite and resource (ZY) satellite. Limited by a computing power of a computer, each scene of image may be randomly cropped into a 512×512 pixel patch according to a true location of the oil storage tank, and an image channel may include a green channel, a blue channel and a near-infrared channel. According to all acquired image patches, 90% of the samples are randomly selected for training, and the other 10% of the samples are used for testing.

[0031] In step 2, an oil storage tank extraction model is built based on a Res2-Unet model structure. The Res2-Unet is a deep learning network based on a UNet semantic segmentation structure, a Res2Net convolution block is used to change a feature interlayer learning to a granular learning and is arranged in a residual mode.

[0032] FIG. 2 shows a structural diagram of a Res2-Unet network. The model network structure proposed by the present disclosure mainly contains encoding and decoding. An original input image has a 3×512×512 size, that is, a three-channel 512×512 pixel size. A 128-channel feature may be generated after a first Res2Net convolution is performed, and a 2048-channel feature may be finally generated through layer-wise pooling and layer-wise Res2Net convolution operation. A process of performing the layer-wise Res2Net convolution operation and pooling operation on the original input image to obtain the 2048-dimensional feature is called encoding. Then, a layer-wise deconvolution is performed on the 2048-dimensional feature to gradually increase a feature size and gradually reduce a dimension, so as to finally generate a single-dimensional binary result image with the same size as the input image. This process is called decoding. In the generated binary result image, 0 represents a background, and 1 represents a target extract, which is the oil storage tank. In the decoding process, the features generated layer-wise in the encoding are stitched with the features of the corresponding size in the decoding, and a 3×3 convolution is performed to generate decoded features at the same time. This operation may be performed by comprehensively utilizing multi-layer information features, which is beneficial to the extraction of multi-scale oil storage tank.

[0033] FIG. 3 shows a schematic diagram of the Res2Net convolution block. The Res2net convolution block changes a 3×3 convolution structure in traditional 1-3-1 convolution structure to a residual structure having four 3×3 convolution structures. In the traditional 1-3-1 convolution structure, convolution operations with 1×1 convolution kernel, 3×3 convolution kernel and 1×1 convolution kernel are respectively performed on an original image, so as to obtain a feature image. The Res2Net may divide a feature image obtained by performing 1×1 convolution into four blocks according to a channel dimension, so as to obtain four feature sub-images x1, x2, x3 and x4. x1 may directly act as y1. x2 is performed by a 3×3 convolution to obtain a feature image y2. x3 and y2 are stitched and are performed by a 3×3 convolution to obtain a feature image y3. y3 and x4 are stitched and are performed by a 3×3 convolution to obtain a feature image y4. y1, y2, y3 and y4 are stitched, are performed by a 1×1 convolution and are stitched with an original feature image, so as to obtain a final output feature image from the Res2Net block. In this way, a receptive field range of each network layer may be increased, an ability to learn a multi-scale feature may be improved, and it is more suitable for extracting an oil storage tank with multi-scale complex characteristics.

[0034] According to a further embodiment of the present disclosure, in view of a current problem that a boundary information of the extracted oil storage tank is incomplete, the present disclosure proposes a boundary loss function to assist the learning of the oil storage tank extraction model. A specific definition of the boundary loss function is shown in Equation (1) and Equation (2), where boundary represents a boundary of an extracted oil storage tank, I represents a result image of the extracted oil storage tank, Ero(I) represents an image obtained by performing an erosion calculation on the image I, and I−Ero(I) represents a boundary information of the extracted oil storage tank. Based on the boundary image, a difference between a pixel value of the boundary image and a pixel value of a true boundary image may be calculated according to Equation (2), and the difference may be used as a measure of a defect of the boundary information of the extracted oil storage tank. In Equation (2), B.sub.pred represents a boundary of the oil storage tank extracted by the oil storage tank extraction model, B.sub.gt represents a true boundary information of the oil storage tank, and L.sub.boundary represents a boundary loss value. A loss function of an entire model structure may be obtained by combining a binary cross entropy BCE and the boundary loss function and may be shown as Equation (3), where a value of μ is set to 2, L represents a loss value of the entire oil storage tank extraction model, L.sub.bce represents a loss value corresponding to the binary cross entropy BCE, and L.sub.boundary represents the boundary loss value.

[00002]$\begin{array}{cc}\mathrm{boundary}=I-\mathrm{Ero}\left(I\right)& \left(1\right)\end{array}$$\begin{array}{cc}{L}_{\mathrm{boundary}}=\frac{{.Math.}_{k=1}^n{\left(B_{\mathrm{pred}}\left(k\right)-B_{\mathrm{gt}}\left(k\right)\right)}^2}{{.Math.}_{i=1}^n{B}_{\mathrm{pred}}\left(i\right)+{.Math.}_{j=1}^n{B}_{\mathrm{gt}}\left(j\right)}& \left(2\right)\end{array}$$\begin{array}{cc}L=\mu L_{\mathrm{bce}}+L_{\mathrm{boundary}}& \left(3\right)\end{array}$

[0035] According to a further embodiment of the present disclosure, in a process of training the model, input data may be step-wise encoded into a 2048-channel feature firstly, and then the 2048-channel feature may be step-wise decoded, by chaining with a feature generated in the encoding process, into a segmentation result binary image with the same size as the input image. The oil storage tank is denoted by 1 in the binary image, and the background object is denoted by 0 in the binary image. A learning rate of the model is set to 0.01, and a stochastic gradient descent (SGD) method is used for an optimization strategy.

[0036] In step 3, a precision verification is performed on the testing set.

[0037] According to a further embodiment of the present disclosure, the trained model may be tested in the testing set to obtain an oil storage tank extraction result for the testing set. FIG. 4 shows a test sample in which oil storage tanks with various sizes are densely distributed and a test sample with a worst extraction precision. As shown, the model proposed by the present disclosure may solve the extraction problem of multi-scale oil storage tanks well, and a variety of oil storage tanks may be extracted. Even if the oil storage tank is greatly affected by a shadow and has a spectrum not much different from the background object, the trained model may still successfully extract most oil storage tank pixels. In addition, an IOU (Intersection over union), a F1 value, a recall rate and a precision rate may be calculated for the entire test sample set evaluate the precision, and statistical results are shown in Table 1. A calculation method for the IOU is shown in Equation (4), where TP represents the true number of pixels of the extracted oil storage tank, TN represents the true number of pixels of the extracted background, and FP represents the number of pixels of the true background object incorrectly classified as the oil storage tank. Specific calculation methods for the recall rate, the precision rate and the F1 value are shown in Equation (5) to Equation (7), where FP represents the number of pixels of the true oil storage tank incorrectly classified as the background object.

[00003]$\begin{array}{cc}\mathrm{IoU}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}}& \left(4\right)\end{array}$$\begin{array}{cc}\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}& \left(5\right)\end{array}$$\begin{array}{cc}\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}& \left(6\right)\end{array}$$\begin{array}{cc}F1=\frac{2\times \mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}& \left(7\right)\end{array}$

TABLE-US-00001 TABLE 1 Statistical table for precision verification of oil storage tank IOU Recall Precision F1 77.2% 79.06% 96.72% 83.07

[0038] Based on the above technical solutions, it should note that the method for extracting an oil storage tank based on a high-spatial-resolution remote sensing image of the present disclosure has at least one of the following beneficial effects compared to the prior art:

[0039] A. the Unet network structure is improved using the Res2Net block, and a model's ability to learn a multi-scale feature of the oil storage tank is enhanced, so that an ability to extract different scales of oil storage tanks may be improved, and then a robustness of the model may be improved.

[0040] B. the oil storage tank extraction model is built using the Res2-Unet structure, then the model has a simple structure and is easy to train. In addition, a basic network architecture of Unet is used to layer-wise encode and decode the input image feature to obtain a final classification result, so that an ability to extract different types of oil storage tanks may be enhanced, and the mobility of the model may be improved.

[0041] C. a boundary loss is used as an auxiliary loss function to enhance the boundary information of the oil storage tank extracted by the model, so as to ensure an integrity of each extracted storage tank and reduce an influence of same object but different spectrum.

[0042] The specific embodiments described above further describe the objectives, technical solutions and advantages of the present disclosure in further detail. It should be understood that the above descriptions are only specific embodiments of the present disclosure and are not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and scope of the present disclosure shall be included within the scope of protection of the present disclosure.