PEDESTRIAN RE-IDENTIFICATION METHOD BASED ON SPATIO-TEMPORAL JOINT MODEL OF RESIDUAL ATTENTION MECHANISM AND DEVICE THEREOF

Abstract

The disclosure provides a pedestrian re-identification method based on a spatio-temporal joint model of a residual attention mechanism and a device thereof. The method includes: performing feature extraction for an input pedestrian with a pre-trained ResNet-50 model; constructing a residual attention mechanism network including a residual attention mechanism module, a feature sampling layer, a global average pooling layer and a local feature connection layer; calculating a feature distance by using a cosine distance and denoting the feature distance as a visual probability according to the trained residual attention mechanism network; performing modeling for a spatio-temporal probability according to camera ID and frame number information in a pedestrian tag of a training sample, and performing Laplace smoothing for a probability model; and calculating a final spatio-temporal joint probability by using the visual probability and the spatio-temporal probability to obtain a pedestrian re-identification result.

Claims

1. A method, comprising: a) performing feature extraction for an input pedestrian x with a ResNet-50 model obtained through pre-training to obtain a feature matrix denoted as f; b) constructing a residual attention mechanism network with a network structure comprising a residual attention mechanism module, a feature sampling layer, a global average pooling layer and a local feature connection layer; c) taking the feature matrix f with dimensions being H×W×C obtained in a) as an input of the residual attention mechanism network, and taking corresponding identity information y as a target output, wherein H, W, C refer to a length, a width and a channel number of a feature map, respectively; performing channel averaging for each spatial position of the feature matrix f as a spatial weight matrix according to the residual attention mechanism module; activating the spatial weight matrix by softmax to ensure that a convolution kernel learns different features, and calculating an attention mechanism map M.sub.SA to obtain a feature matrix F.sub.RSA with dimensions being H×W×C by F.sub.RSA=f*M.sub.SA+f; d) sampling the feature matrix F.sub.RSA with dimensions being H×W×C into $\frac{H}{6}\times W\times C$ local feature matrixes (F.sub.RSA.sub.1, F.sub.RSA.sub.2 . . . , F.sub.RSA.sub.6) with dimensions being by the feature sampling layer, and calculating local feature vectors (V.sub.RSA.sub.1, V.sub.RSA.sub.2 . . . V.sub.RSA.sub.6) by the global average pooling layer; e) connecting local features into a feature vector V.sub.RSA by the local feature connection layer, and calculating a cross entropy loss between the feature vector V.sub.RSA and the pedestrian identity y to obtain the trained residual attention mechanism network after training; f) obtaining feature vectors V.sub.RSA-α and V.sub.RSA-β corresponding to tested pedestrian images x.sub.α and x.sub.β respectively according to the trained residual attention mechanism network obtained in e), and calculating a feature distance based on a cosine distance and denoting the feature distance as a visual probability P.sub.V; g) performing modeling for a spatio-temporal probability according to camera ID and frame number information in a pedestrian tag of a training sample, and calculating the spatio-temporal probability P.sub.ST according to the obtained spatio-temporal model; and h) calculating a final joint spatio-temporal probability using the visual probability P.sub.V obtained in f) and the spatio-temporal probability P.sub.ST obtained in g) to obtain a pedestrian re-identification result.

2. The method of claim 1, wherein in c), the residual attention mechanism model is defined as follows: $Q\left(i,j\right)=\frac{\underset{t=0}{\overset{C}{.Math.}}.Math..Math.f_t\left(i,j\right)}{C}$ $M_{\mathrm{SA}}\left(i,j\right)=\frac{e^{Q\left(i,j\right)}}{{\Sigma \left(i,j\right)}^{e^{Q\left(i,j\right)}}}$ $F_{{\mathrm{RSA}}_t}\left(i,j\right)=f_t\left(i,j\right).Math.M_{\mathrm{SA}}\left(i,j\right)+f_t\left(i,j\right),$ wherein (i,j) refers to spatial position information, t refers to a channel serial number, f.sub.t(i,j) refers to a pixel point with the spatial position being (i,j) in a t-th channel of the feature matrix f, e refers to a base of a natural logarithm, and F.sub.RSA(i,j) refers to a pixel point with the spatial position being (i,j) in the feature matrix F.sub.RSA.

3. The method of claim 1, wherein in e), back propagation is performed by a stochastic gradient descent method to optimize residual attention mechanism network parameters until an upper limit of the number of training is reached, so that the trained residual attention mechanism network is obtained.

4. The method of claim 1, wherein in g), Laplace smoothing is performed for a probability model after modeling is performed for the spatio-temporal probability according to the camera ID and frame number information in the pedestrian tag of the training sample.

5. The method of claim 1, wherein in h), by using the visual probability P.sub.V obtained in f) and the spatio-temporal probability P.sub.ST obtained in g), the final joint probability is expressed as follows: $P=\frac{1}{1+{\lambda }_1.Math.e^{-{\gamma }_1.Math.P_V}}.Math.\frac{1}{1+{\lambda }_2.Math.e^{-{\gamma }_2.Math.P_{\mathrm{ST}}}},$ wherein λ.sub.1, γ.sub.1, λ.sub.2, γ.sub.2 refer to hyperparameters for balancing the visual probability and the spatio-temporal probability.

6. The method of claim 2, wherein in h), by using the visual probability P.sub.V obtained in f) and the spatio-temporal probability P.sub.ST obtained in g), the final joint probability is expressed as follows: $P=\frac{1}{1+{\lambda }_1.Math.e^{-{\gamma }_1.Math.P_V}}.Math.\frac{1}{1+{\lambda }_2.Math.e^{-{\gamma }_2.Math.P_{\mathrm{ST}}}},$ wherein λ.sub.1, γ.sub.1, λ.sub.2, γ.sub.2 refer to hyperparameters for balancing the visual probability and the spatio-temporal probability.

7. The method of claim 3, wherein in h), by using the visual probability P.sub.V obtained in f) and the spatio-temporal probability P.sub.ST obtained in g), the final joint probability is expressed as follows: $P=\frac{1}{1+{\lambda }_1.Math.e^{-{\gamma }_1.Math.P_V}}.Math.\frac{1}{1+{\lambda }_2.Math.e^{-{\gamma }_2.Math.P_{\mathrm{ST}}}},$ wherein λ.sub.1, γ.sub.1, λ.sub.2, γ.sub.2 refer to hyperparameters for balancing the visual probability and the spatio-temporal probability.

8. The method of claim 4, wherein in h), by using the visual probability P.sub.V obtained in f) and the spatio-temporal probability P.sub.ST obtained in g), the final joint probability is expressed as follows: $P=\frac{1}{1+{\lambda }_1.Math.e^{-{\gamma }_1.Math.P_V}}.Math.\frac{1}{1+{\lambda }_2.Math.e^{-{\gamma }_2.Math.P_{\mathrm{ST}}}},$ wherein λ.sub.1, γ.sub.1, λ.sub.2, γ.sub.2 refer to hyperparameters for balancing the visual probability and the spatio-temporal probability.

9. A device, comprising: a first module, configured to perform feature extraction for an input pedestrian x with a ResNet-50 model obtained through pre-training so as to obtain a feature matrix denoted as f; a second module, configured to construct a residual attention mechanism network with a network structure comprising a residual attention mechanism module, a feature sampling layer, a global average pooling layer and a local feature connection layer; a third module, configured to obtain the feature matrix f with dimensions being H×W×C, take the feature matrix f as an input of the residual attention mechanism network, and take corresponding identity information y as a target output, wherein H, W, C refer to a length, a width and a channel number of a feature map respectively, and further configured to perform channel averaging for each spatial position of the feature matrix f as a spatial weight matrix according to the residual attention mechanism module, activate the spatial weight matrix by softmax to ensure that a convolution kernel learns different features, and calculate an attention mechanism map M.sub.SA to obtain a feature matrix F.sub.RSA with dimensions being H×W×C by F.sub.RSA=f*M.sub.SA+f; a fourth module, configured to sample the feature matrix F.sub.RSA with dimensions being H×W×C into local feature matrixes (F.sub.RSA.sub.1, F.sub.RSA.sub.2 . . . , F.sub.RSA.sub.6) with dimensions being $\frac{H}{6}\times W\times C$ by the feature sampling layer and calculate local feature vectors (V.sub.RSA.sub.1, V.sub.RSA.sub.2 . . . , V.sub.RSA.sub.6) by the global average pooling layer; a fifth module, configured to connect the local features into a feature vector V.sub.RSA by the local feature connection layer and calculate a cross entropy loss between the feature vector V.sub.RSA and the pedestrian identity y to obtain the trained residual attention mechanism network after training; a sixth module, configured to obtain feature vectors V.sub.RSA-α and V.sub.RSA-β corresponding to tested pedestrian images x.sub.α and x.sub.β respectively according to the trained residual attention mechanism network obtained in e), and calculate a feature distance based on a cosine distance and denote the feature distance as a visual probability P.sub.V; a seventh module, configured to perform modeling for a spatio-temporal probability according to camera ID and frame number information in a pedestrian tag of a training sample and calculate the spatio-temporal probability P.sub.ST according to the obtained spatio-temporal model; and an eighth module, configured to calculate a final joint spatio-temporal probability by using the visual probability P.sub.V obtained by the sixth module and the spatio-temporal probability P.sub.ST obtained by the seventh module to obtain a pedestrian re-identification result.

10. The device of claim 9, wherein in the third module, the residual attention mechanism module is defined as follows: $Q\left(i,j\right)=\frac{\underset{t=0}{\overset{C}{.Math.}}.Math..Math.f_t\left(i,j\right)}{C}$ $M_{\mathrm{SA}}\left(i,j\right)=\frac{e^{Q\left(i,j\right)}}{{\Sigma \left(i,j\right)}^{e^{Q\left(i,j\right)}}}$ $F_{{\mathrm{RSA}}_t}\left(i,j\right)=f_t\left(i,j\right).Math.M_{\mathrm{SA}}\left(i,j\right)+f_t\left(i,j\right),$ wherein (i,j) refers to spatial position information, t refers to a channel serial number, f.sub.t(i,j) refers to a pixel point with the spatial position being (i,j) in the t-th channel of the feature matrix f, e refers to a base of a natural logarithm, and F.sub.RSA(i,j) refers to a pixel point with the spatial position being (i,j) in the feature matrix F.sub.RSA.

11. The device of claim 9, wherein in the fifth module, back propagation is performed by a stochastic gradient descent method to optimize residual attention mechanism network parameters until an upper limit of the number of training is reached, so that the trained residual attention mechanism network is obtained.

12. The device of claim 9, wherein in the seventh module, Laplace smoothing is performed for a probability model after modeling is performed for the spatio-temporal probability according to the camera ID and frame number information in the pedestrian tag of the training sample.

13. The device of claim 9, wherein in the eighth module, by using the visual probability P.sub.V obtained by the sixth module and the spatio-temporal probability P.sub.ST obtained by the seventh module, the final joint probability is expressed as follows: $P=\frac{1}{1+{\lambda }_1.Math.e^{-{\gamma }_1.Math.P_V}}.Math.\frac{1}{1+{\lambda }_2.Math.e^{-{\gamma }_2.Math.P_{\mathrm{ST}}}},$ wherein λ.sub.1, γ.sub.1, λ.sub.2, γ.sub.2 refer to hyperparameters for balancing the visual probability and the spatio-temporal probability.

14. The device of claim 10, wherein in the eighth module, by using the visual probability P.sub.V obtained by the sixth module and the spatio-temporal probability P.sub.ST obtained by the seventh module, the final joint probability is expressed as follows: $P=\frac{1}{1+{\lambda }_1.Math.e^{-{\gamma }_1.Math.P_V}}.Math.\frac{1}{1+{\lambda }_2.Math.e^{-{\gamma }_2.Math.P_{\mathrm{ST}}}},$ wherein λ.sub.1, γ.sub.1, λ.sub.2, γ.sub.2 refer to hyperparameters for balancing the visual probability and the spatio-temporal probability.

15. The device of claim 11, wherein in the eighth module, by using the visual probability P.sub.V obtained by the sixth module and the spatio-temporal probability P.sub.ST obtained by the seventh module, the final joint probability is expressed as follows: $P=\frac{1}{1+{\lambda }_1.Math.e^{-{\gamma }_1.Math.P_V}}.Math.\frac{1}{1+{\lambda }_2.Math.e^{-{\gamma }_2.Math.P_{\mathrm{ST}}}},$ wherein λ.sub.1, γ.sub.1, λ.sub.2, γ.sub.2 refer to hyperparameters for balancing the visual probability and the spatio-temporal probability.

16. The device of claim 12, wherein in the eighth module, by using the visual probability P.sub.V obtained by the sixth module and the spatio-temporal probability P.sub.ST obtained by the seventh module, the final joint probability is expressed as follows: $P=\frac{1}{1+{\lambda }_1.Math.e^{-{\gamma }_1.Math.P_V}}.Math.\frac{1}{1+{\lambda }_2.Math.e^{-{\gamma }_2.Math.P_{\mathrm{ST}}}},$ wherein λ.sub.1, γ.sub.1, λ.sub.2, γ.sub.2 refer to hyperparameters for balancing the visual probability and the spatio-temporal probability.

Description

BRIEF DESCRIPTIONS OF THE DRAWINGS

[0036] FIG. 1 is a flowchart of performing pedestrian re-identification based on a Bayesian spatio-temporal joint model according to an example of the disclosure;

[0037] FIG. 2 is a schematic diagram illustrating a network structure of a residual attention mechanism module according to an example of the disclosure;

[0038] FIG. 3 is a schematic diagram illustrating an application device according to an example of the disclosure; and

[0039] FIG. 4 is a structural diagram of pedestrian re-identification network based on spatio-temporal joint model of residual attention mechanism.

DETAILED DESCRIPTIONS OF EMBODIMENTS

[0040] A technical solution of the disclosure will be further described below in combination with drawings and examples.

[0041] An example of the disclosure provides a pedestrian re-identification method based on a spatio-temporal joint model of a residual attention mechanism, which is preferably performed in the following environment: a CPU of a server is Intel Xeon E5-2665, a GPU is NVIDIA GTX108Ti, an operating system is Ubuntu 16.04, and compiling environments are PyTorch 1.1.0, Python 3.5, CUDA9.0 and CUDNN7.1. During a specific implementation, a corresponding environment may be set according to requirements.

[0042] As shown in FIG. 1, an example of the disclosure provides a pedestrian re-identification method based on a spatio-temporal joint model of a residual attention mechanism. An implementation process of the method is described as follows.

[0043] a) Feature extraction is performed for an input pedestrian x with a ResNet-50 model trained on the ImageNet data set to obtain a feature matrix denoted as f.

[0044] The ImageNet data set is a public data set, and the ResNet-50 model is the prior art, which are not described herein.

[0045] b) A residual attention mechanism network is constructed. In an example, the structure of the residual attention mechanism network is a residual attention mechanism module (Residual Spatial Attention, RSA).fwdarw.a feature sampling layer.fwdarw.a Global Average Pooling layer (GAP).fwdarw.a local feature connection layer (Concat). That is, the residual attention mechanism network comprises the residual attention mechanism module, the feature sampling layer, the global average pooling layer and the local feature connection layer which are all connected sequentially.

[0046] c) The feature matrix f with dimensions being H×W×C obtained in a) is taken as an input of the residual attention mechanism network, and corresponding identity information y is taken as a target output, where H, W, C refer to a length, a width and a channel number of a feature map respectively.

[0047] As shown in FIG. 2, a residual attention mechanism structure provided by the residual attention mechanism module in an example is as follows: a weight matrix is obtained by processing the feature matrix f through a mean operation and then reshaped into a vector, and the vector is activated by softmax to ensure that a convolution kernel learns different features and reshaped again to obtain an attention mechanism map. The attention mechanism map is multiplied by the input feature matrix f to obtain a product which is added to f to obtain a feature matrix F.sub.RSA with dimensions being H×W×C.

[0048] The mean operation is an averaging operation that performs channel averaging for each spatial position of the feature matrix f as a spatial weight matrix Q(i,j). The process is expressed by the following formula.

[00007]$\begin{array}{cc}Q\left(i,j\right)=\frac{\underset{t=0}{\overset{C}{.Math.}}.Math..Math.f_t\left(i,j\right)}{C}& \left(1\right)\end{array}$

[0049] In the above formula, (i,j) refers to spatial position information, t refers to a channel serial number, and f.sub.t(i,j) refers to a pixel point with the spatial position being (i,j) in the t-th channel of the feature matrix f.

[0050] The reshape operation is a matrix dimension conversion operation that can convert a matrix with a size being H×W into a vector with a size being (H×W) or convert the vector with the size being (H×W) into the matrix with the size being H×W.

[0051] The softmax function activates the spatial weight vector to ensure that the convolution kernel learns different features. Then, the activated vector obtains an attention mechanism feature map M.sub.SA with dimensions being H×W×1 by means of the reshape operation. The process is expressed by the following formula.

[00008]$\begin{array}{cc}{M}_{\mathrm{SA}}\left(i,j\right)=\frac{e^{Q\left(i,j\right)}}{{\Sigma \left(i,j\right)}^{e^{Q\left(i,j\right)}}}& \left(2\right)\end{array}$

[0052] In the above formula, (i,j) refers to spatial position information, e refers to a base of a natural logarithm, and H, W, C refer to the length, the width and the channel number of the feature map respectively.

[0053] The feature matrix F.sub.RSA with the dimensions being H×W×C is obtained by F.sub.RSA=f*M.sub.SA+f. The process is expressed by the following formula.

F.sub.RSA.sub.t(i,j)=f.sub.t(i,j)M.sub.SA(i,j)+f.sub.t(i,j)  (3)

[0054] In the above formula, (i,j) refers to spatial position information, t refers to the channel number, f.sub.t(i,j) refers to a pixel point with the spatial position being (i,j) in the t-th channel of the feature matrix f, and F.sub.RSA(i,j) refers to a pixel point with the spatial position being (i,j) in the feature matrix F.sub.RSA.

[0055] d) The feature matrix F.sub.RSA with the dimensions being H×W×C is divided into local feature matrixes F.sub.RSA.sub.1, F.sub.RSA.sub.2 . . . , F.sub.RSA.sub.6 with dimensions being H/6×W×C, which corresponds to the operation of the feature sampling layer in b). Local feature vectors V.sub.RSA.sub.1, V.sub.RSA.sub.2 . . . , V.sub.RSA.sub.6 are calculated by the global average pooling layer, which corresponds to the operation of the global average pooling layer (GAP) in b).

[0056] e) the local features are connected into a feature vector V.sub.RSA, which corresponds to the operation of the local feature connection layer (Concat) in b), a cross entropy loss between the feature vector V.sub.RSA and the pedestrian identity y is calculated, back propagation is performed by a stochastic gradient descent method to optimize residual attention mechanism network parameters until an upper limit of the number of training is reached. In this way, a trained residual attention mechanism network is obtained.

[0057] In e), the local features are connected into the feature vector V.sub.RSA, and thus V.sub.RSA may be expressed as follows:

V.sub.RSA=concat(V.sub.RSA.sub.1,V.sub.RSA.sub.2,V.sub.RSA.sub.3,V.sub.RSA.sub.4,V.sub.RSA.sub.5,V.sub.RSA.sub.6)=(V.sub.RSA.sub.1,V.sub.RSA.sub.2,V.sub.RSA.sub.3,V.sub.RSA.sub.4,V.sub.RSA.sub.5,V.sub.RSA.sub.6)  (4).

[0058] In the above formula, V.sub.RSA.sub.1, V.sub.RSA.sub.2 . . . , V.sub.RSA.sub.6 all refer to local feature vectors.

[0059] f) The feature vectors of tested pedestrian images x.sub.α and x.sub.β are calculated as V.sub.RSA-α and V.sub.RSA-β according to the trained residual attention mechanism network obtained in e). A feature distance d(V.sub.RSA-α,V.sub.RSA-β) is calculated based on a cosine distance and denoted as a visual probability P.sub.V. The calculation formula is as follows:

[00009]$\begin{array}{cc}d\left(V_{\mathrm{RSA}-\alpha },V_{\mathrm{RSA}-\beta }\right)=\frac{V_{\mathrm{RSA}-\alpha }.Math.V_{\mathrm{RSA}-\beta }}{.Math.V_{\mathrm{RSA}-\alpha }.Math..Math..Math.V_{\mathrm{RSA}-\beta }.Math.}.& \left(5\right)\end{array}$

[0060] In the above formula, ∥⋅∥ refers to an l.sub.2 normal form of the feature vector.

[0061] g) A spatio-temporal probability is solved as follows: modeling is performed for the spatio-temporal probability according to camera ID and frame number information in a pedestrian tag of a training sample, and Laplace smoothing is then performed for a probability model.

[0062] According to spatio-temporal information carried in the image, the spatio-temporal probability P.sub.ST (p.sub.α=p.sub.β|k,c.sub.α,c.sub.β) thereof is a probability that α,β are a same person under the condition of k, c.sub.α, c.sub.β and may be expressed as follows:

[00010]$\begin{array}{cc}{\stackrel{\_}{P}}_{\mathrm{ST}}\left(p_{\alpha }=p_{\beta }k,c_{\alpha },c_{\beta }\right)=\frac{n_{c_{\alpha }.Math.c_{\beta }}^k}{{\Sigma }_l.Math.n_{c_{\alpha }.Math.c_{\beta }}^l}.& \left(6\right)\end{array}$

[0063] In the above formula, p.sub.α, p.sub.β refer to identify information corresponding to images α,β and c.sub.α, c.sub.β refer to ID numbers of the corresponding cameras for shooting the images α, β. k refers to a k-th time period (100 frames are one time period in an example). n.sub.c.sub.α.sub.,c.sub.β.sup.k refers to the number of pedestrians from the camera c.sub.α to the camera c.sub.β with a time difference falling in the k-th time period. n.sub.c.sub.α.sub.,c.sub.β.sup.l refers to the number of pedestrians from the camera c.sub.α to the camera c.sub.β with a time difference falling in the 1-th time period.

[0064] Since many jitters exist in a probability estimation model, smoothing is performed using a Laplace distribution function to reduce an interference caused by the jitters. The process is expressed as follows:

[00011]$\begin{array}{cc}{P}_{\mathrm{ST}}\left(p_{\alpha }=p_{\beta }k,c_{\alpha },c_{\beta }\right)=\frac{1}{Z}.Math.{\Sigma }_l.Math.{\stackrel{\_}{P}}_{\mathrm{ST}}\left(p_{\alpha }=p_{\beta }k,c_{\alpha },c_{\beta }\right).Math..Math.K\left(1-k\right)& \left(7\right)\\ .Math.K\left(1-k\right)=\frac{1}{2.Math.\lambda }.Math.e^{\frac{-.Math.1-k-\mu .Math.}{\lambda }}& \left(8\right)\end{array}$

[0065] In the above formulas, z=Σ.sub.k P.sub.ST(p.sub.α=p.sub.β|k,c.sub.α,c.sub.β) refers to a normalization factor, K(.) refers to a Laplace distribution function, μ refers to a distribution offset control parameter which generally is 0, λ refers to a distribution scaling control parameter which is recommended to be 50, and e refers to the base of the natural logarithm.

[0066] h) A final joint spatio-temporal probability is calculated using the visual probability P.sub.V obtained in f) and the spatio-temporal probability P.sub.ST obtained in g) so as to obtain a pedestrian re-identification result. The spatio-temporal probability and the visual probability are distributed independently, and a more accurate identification can be performed by constraining the visual probability using the spatio-temporal probability.

[0067] Since the spatio-temporal probability and the visual probability may differ in magnitude, it is required to balance the spatio-temporal probability and the visual probability by a sigmoid activation function. In h), by using the visual probability d obtained in f) and the spatio-temporal probability P.sub.ST obtained in g), a final joint probability P.sub.joint may be expressed as a Bayesian joint probability as follows:

[00012]$\begin{array}{cc}{P}_{\mathrm{joint}}=\frac{1}{1+\phi .Math..Math.e^{-\gamma .Math..Math.d}}.Math.\frac{1}{1+\phi .Math..Math.e^{-\gamma .Math..Math.P_{\mathrm{ST}}}}.& \left(10\right)\end{array}$

[0068] In the above formula, λ, ϕ.sub.refer to hyperparameters for balancing the visual probability and the spatio-temporal probability, where γ=5, and ϕ is recommended to be in (50, 70).

[0069] At the above descriptions, a) is a data pre-processing part, b-e) are network training parts, f) is a network testing part, and g-h) are joint parts of the spatio-temporal probability and the visual probability.

[0070] In a specific implementation, the above flow may be realized as an automatic operation process by adopting a computer software technology, or may be provided in a modularized manner. Correspondingly, an example of the disclosure further provides a pedestrian re-identification device based on a spatio-temporal joint model of a residual attention mechanism. The device comprises the following modules.

[0071] A first module is configured to perform feature extraction for an input pedestrian x with a ResNet-50 model obtained through pre-training so as to obtain a feature matrix denoted as f.

[0072] A second module is configured to construct a residual attention mechanism network with a network structure comprising a residual attention mechanism module, a feature sampling layer, a global average pooling layer and a local feature connection layer.

[0073] A third module is configured to obtain the feature matrix f with dimensions being H×W×C take the feature matrix f as an input of the residual attention mechanism network, and corresponding identity information y as a target output, where H, W, C refer to a length, a width and a channel number of a feature map respectively.

[0074] According to the residual attention mechanism module, channel averaging is performed for each spatial position of the feature matrix f as a spatial weight matrix; the spatial weight matrix is activated by softmax to ensure that a convolution kernel learns different features, and an attention mechanism map M.sub.SA is calculated to obtain a feature matrix F.sub.RSA with dimensions being H×W×C by F.sub.RSA=f*M.sub.SA+f.

[0075] A fourth module is configured to sample the feature matrix F.sub.RSA with dimensions being H×W×C into local feature matrixes (F.sub.RSA.sub.1, F.sub.RSA.sub.2 . . . , F.sub.RSA.sub.6) with dimensions being

[00013]$\frac{H}{6}\times W\times C$

by the feature sampling layer and calculate local feature vectors (V.sub.RSA.sub.1, V.sub.RSA.sub.2 . . . V.sub.RSA.sub.6) by the global average pooling layer.

[0076] A fifth module is configured to connect the local features into a feature vector V.sub.RSA by the local feature connection layer and calculates a cross entropy loss between the feature vector V.sub.RSA and the pedestrian identity y so as to obtain the trained residual attention mechanism network after training.

[0077] A sixth module is configured to obtain feature vectors V.sub.RSA-α and V.sub.RSA-p corresponding to test pedestrian images x.sub.α and x.sub.β respectively according to the trained residual attention mechanism network obtained in e), and calculates a feature distance based on a cosine distance and denote it as a visual probability P.sub.V.

[0078] A seventh module is configured to perform modeling for a spatio-temporal probability according to camera ID and frame number information in a pedestrian tag of a training sample and calculates the spatio-temporal probability P.sub.ST according to the obtained spatio-temporal model.

[0079] An eighth module is configured to calculate a final joint spatio-temporal probability using the visual probability P.sub.V obtained by the sixth module and the spatio-temporal probability P.sub.ST obtained by the seventh module so as to obtain a pedestrian re-identification result.

[0080] The implementation of each module may be referred to the above corresponding descriptions, and thus will not be described herein.

[0081] As shown in FIG. 3, in a specific implementation, the solution according to the disclosure may be applied in systems such as video monitoring. For example, a lens module may generate an optical image of a pedestrian, project the image onto a surface of a sensor and convert the image into an electrical signal; the sensor may convert the electrical signal into digital image information and input the information into a processing chip for intelligent calculation; the processing chip may extract spatio-temporal information from the image and input visual features of the image, which correspond to a-e) in the examples of the disclosure; in a cloud server, spatio-temporal pedestrian re-identification and joint probability calculation may be performed according to a pedestrian data set accessing a cloud, which corresponds to f-h) in the example of the disclosure. A client accepts the identification result and displays 10 persons most similar to the pedestrians shot by the camera as well as appearance time and appearance locations.

[0082] It will be obvious to those skilled in the art that changes and modifications may be made, and therefore, the aim in the appended claims is to cover all such changes and modifications.