METHOD AND SYSTEM FOR GENERATING A DYNAMIC ADDICTIVE NEURAL CIRCUITS BASED ON WEAKLY SUPERVISED CONTRASTIVE LEARNING

Abstract

A method and a system for generating a dynamic addictive neural circuit based on weakly supervised contrastive learning are disclosed. The method includes: based on a convolutional neural network, reducing a dimensionality of voxels of multiple groups of fMRI to attributes of brain region nodes, and generating multiple groups of dynamic brain connection maps containing time series based on the attributes of the brain region nodes; extracting spatio-temporal features of brain connections in the dynamic brain connection maps; inputting the spatio-temporal features into an abnormal connection detection network, calculating an abnormal probability of brain connections based on contrastive learning, and obtaining the brain connection with a highest abnormal probability at each time point; and generating the dynamic addictive neural circuit based on neuroscientific prior knowledge and the brain connection with the greatest probability of abnormality.

Claims

1. A method for generating a dynamic addictive neural circuit based on weakly supervised contrastive learning, comprising: based on a convolutional neural network, reducing a dimensionality of voxels of a plurality of groups of fMRI images to attributes of brain region nodes, and generating a plurality of groups of dynamic brain connection maps containing time series based on the attributes of the brain region nodes; wherein each group of fMRI images comprises a plurality of fMRI images taken at a fixed frequency within a preset time period; extracting spatio-temporal features of brain connections in each of the dynamic brain connection maps; inputting the spatio-temporal features into an abnormal connection detection network, calculating an abnormal probability of each of brain connections based on contrastive learning, and obtaining the brain connection with a highest abnormal probability at each time point; and generating the dynamic addictive neural circuit based on neuroscience prior knowledge and the brain connection with the highest abnormal probability at each time point.

2. The method as recited in claim 1, further comprising the following step prior to the step of extracting spatio-temporal features of brain connections in each of the dynamic brain connection maps: using a standard brain connection map as a real sample and using the dynamic brain connection maps as fake samples to create adversarial learning, and updating the dynamic brain connection maps.

3. The method as recited in claim 1, wherein the step of extracting spatio-temporal features of brain connections in each of the dynamic brain connection maps comprises: extracting spatial features of the brain connections in each of the dynamic brain connection maps by using a graph convolutional neural network based on a topological spatial brain connection focus mechanism; and extracting temporal features of the brain connections in each of the dynamic brain connection maps by using a gated recurrent unit based on a time-series brain connection focus mechanism.

4. The method as recited in claim 3, wherein the sub-step of extracting spatial features of the brain connections in each of the dynamic brain connection maps by using the graph convolutional neural network based on a topological space brain connection focus mechanism comprises: a query-node attribute vector: q.sub.c,i.sup.(l)=W.sub.c,q.sup.(l)h.sub.(l)i.sup.(t)+b.sub.c,q.sup.(l); wherein c represents an abnormal connection focus block; W.sub.c,q.sup.(l) represents a learnable weight matrix, b.sub.c,q.sup.(l) represents an offset; h.sub.(l)i.sup.(t) represents a multi-head abnormal connection focus block of an i-th node at al-th layer at time point t, t∈{1, 2, . . . , T}, wherein T represents a total number of time points; i∈{1, 2, . . . , N}, wherein N represented a total number of nodes; a key-node attribute vector: k.sub.c,j.sup.(l)=W.sub.c,k.sup.(l)h.sub.(l)j.sup.(t)+b.sub.c,k.sup.(l); wherein j represents a node, W.sub.c,k.sup.(l) represents a learnable weight matrix, and b.sub.c,k.sup.(l) represents an offset; an additional brain connection feature vector: e.sub.c,ji=W.sub.c,ee.sub.ij+b.sub.c,e wherein e.sub.ij represents an edge in the dynamic brain connection map, indicating a brain connection between the i-th node and the j-th node at time point t; W.sub.c,e represents a learnable weight matrix, and b.sub.c,e represents an offset; a brain connection focus coefficient: $α_{c, ij}^{(l)} = \frac{.Math. q_{c, i}^{(l)}, k_{c, j}^{(l)} + e_{c, ij} .Math.}{{.Math.}_{u \in N (i)} .Math. q_{c, i}^{(l)}, k_{c, u}^{(l)} + e_{c, iu} .Math.};$ wherein <q, k> represents an exponential dot product function, and N(i) represents a value other than i; a hidden state feature vector of a node at (l+1)-th layer: h.sub.(l+1)i.sup.(t)=∥.sub.c=1.sup.C[Σ.sub.jα.sub.c,ij(v.sub.c,j.sup.(l)+e.sub.c,ij]; v.sub.c,j.sup.(l)=W.sub.c,v.sup.(l)h.sub.(l)i.sup.(t)+b.sub.c,v.sup.(l); wherein W.sub.c,v.sup.(l) represents a learnable weight matrix, and b.sub.c,v.sup.(l) represents an offset.

5. The method as recited in claim 3, wherein the sub-step of extracting temporal features of the brain connections in each of the dynamic brain connection maps by using a gated recurrent unit based on a time-series brain connection focus mechanism comprises: a long-term hidden state feature vector: Current.sup.t+1=MACFB(H.sup.t); in a brain connection focus window, an output of a hidden state feature vector of a node: C.sub.i.sup.t+1=[h.sub.i.sup.t+w, . . . , h.sub.i.sup.t]; wherein a window size is w−1, and i represents a node; a time-series brain connection focus coefficient: e.sub.i.sup.t+1=r.sup.T tan h(Q(C.sub.i.sup.t+1).sup.T); wherein r represents a weight vector and Q represents a weight matrix;
a.sub.i.sup.t+1=softmax(e.sub.i.sup.t+1); a short-term hidden state feature vector: short.sub.i.sup.t+1=(a.sub.i.sup.t+1C.sub.i.sup.t+1).sup.T; at time point t+1, a hidden state feature vector set: H.sup.t+1=GRU(Current.sup.t+1, Short.sup.t+1); wherein Short.sup.t+1 represents the set of short-term hidden state feature vectors short.sub.i.sup.t+1 of all nodes, i∈{1, 2, . . . , N}, N represents total number of nodes.

6. The method as recited in claim 5, wherein a computation process of H.sup.t+1=GRU(Current.sup.t+1, Short.sup.t+1) is as follows: a brain connection state change gate: P.sup.t+1=σ(U.sub.pCurrent.sup.t+1+W.sub.pShort.sup.t+1+b.sub.p); wherein U.sub.p and W.sub.p are both learnable weight matrices, and b.sub.p is an offset; a brain connection state reorganization gate: R.sup.t+1=σ(U.sub.RCurrent.sup.t+1+W.sub.RShort.sup.t+1+b.sub.R); wherein U.sub.R and W.sub.R are both learnable weight matrices, and b.sub.R is an offset for calculating R.sup.t+1; {tilde over (H)}.sup.t+1=tan h(U.sub.H Current.sup.t+1+(R.sup.t+1⊙Short.sup.t+1)); wherein U.sub.H and W.sub.H are both learnable weight matrices;
H.sup.t+1=(1−p.sup.t+1)⊙Short.sup.t+1+p.sup.t+1⊙{tilde over (H)}.sup.t+1.

7. The method as recited in claim 3, wherein the step of inputting the spatio-temporal features into an abnormal connection detection network and calculating an abnormal probability of brain connections based on contrastive learning comprises: a probability of an abnormal brain connection is: F.sub.score=MLP(H.sup.t); f(i, j, ω)=ω.Math.σ(β.Math.(∥a⊙h.sub.i+b⊙h.sub.j∥.sub.2.sup.2−μ)); wherein h.sub.i represents a hidden state feature vector of ani-th node, h.sub.j represents a hidden state feature vector of a j-th node, σ(•) represents a sigmoid function, a and b represent optimization parameters of an output layer, p and μ represent a pair of hyperparameters, co represents a weight of an edge e.sub.ij in each of the dynamic brain connection maps; a contrast loss function: L.sub.S=min(max{0, f(H.sub.S)−Σ.sub.a∈A f(H.sub.a)+γ});
L.sub.Δ=min(max{0,f(H.sub.a)−f(H.sub.a′)+γ}); L.sub.TOTAL=L.sub.S+L.sub.Δ; wherein max{0,•} represents a hinge loss, H.sub.S represents a spatio-temporal feature of a saline injection group, H.sub.a represents a spatio-temporal feature of a nicotine injection group, wherein a∈A, a′∈A, a≠a′, and A represents a total number of groups in the nicotine injection groups; γ represents a hyperparameter.

8. The method as recited in claim 4, wherein the step of inputting the spatio-temporal features into an abnormal connection detection network and calculating an abnormal probability of brain connections based on contrastive learning comprises: a probability of an abnormal brain connection is: F.sub.score=MLP(H.sup.t); f(i, j, ω)=ω.Math.σ(β.Math.(∥a⊙h.sub.i+b⊙h.sub.j∥.sub.2.sup.2−μ)); wherein h.sub.i represents a hidden state feature vector of ani-th node, h.sub.j represents a hidden state feature vector of a j-th node, σ(•) represents a sigmoid function, a and b represent optimization parameters of an output layer, β and μ represent a pair of hyperparameters, co represents a weight of an edge e.sub.ij in each of the dynamic brain connection maps; a contrast loss function: L.sub.S=min(max{0, f(H.sub.S)−Σ.sub.a∈A f(H.sub.a)+γ});
L.sub.Δ=min(max{0,f(H.sub.a)−f(H.sub.a′)+γ}); L.sub.TOTAL=L.sub.S+ΣL.sub.Δ; wherein max{0,•} represents a hinge loss, H.sub.S represents a spatio-temporal feature of a saline injection group, H.sub.a represents a spatio-temporal feature of a nicotine injection group, wherein a∈A, a′∈A, a≠a′, and A represents a total number of groups in the nicotine injection groups; γ represents a hyperparameter.

9. The method as recited in claim 5, wherein the step of inputting the spatio-temporal features into an abnormal connection detection network and calculating an abnormal probability of brain connections based on contrastive learning comprises: a probability of an abnormal brain connection is: F.sub.score=MLP(H.sup.t); f(i, j, ω)=ω.Math.σ(β.Math.(∥a⊙h.sub.i+b⊙h.sub.j∥.sub.2.sup.2−μ)); wherein h.sub.i represents a hidden state feature vector of ani-th node, h.sub.j represents a hidden state feature vector of a j-th node, σ(•) represents a sigmoid function, a and b represent optimization parameters of an output layer, β and μ represent a pair of hyperparameters, co represents a weight of an edge e.sub.ij in each of the dynamic brain connection maps; a contrast loss function: L.sub.S=min(max{0, f(H.sub.S)−E.sub.a∈Af(H.sub.a)+γ});
L.sub.Δ=min(max{0,f(H.sub.a)−f(H.sub.a′)+γ}); L.sub.TOTAL=L.sub.S+L.sub.Δ; wherein max{0,•} represents a hinge loss, H.sub.S represents a spatio-temporal feature of a saline injection group, H.sub.a represents a spatio-temporal feature of a nicotine injection group, wherein a∈A, a′∈A, a≠a′, and A represents a total number of groups in the nicotine injection groups; γ represents a hyperparameter.

10. The method as recited in claim 6, wherein the step of inputting the spatio-temporal features into an abnormal connection detection network and calculating an abnormal probability of brain connections based on contrastive learning comprises: a probability of an abnormal brain connection is: F.sub.score=MLP(H.sup.t); f(i, j, ω)=ω.Math.σ(β.Math.(∥a⊙h.sub.i+b⊙h.sub.j∥.sub.2.sup.2−μ)); wherein h.sub.i represents a hidden state feature vector of ani-th node, h.sub.j represents a hidden state feature vector of a j-th node, σ(•) represents a sigmoid function, a and b represent optimization parameters of an output layer, p and μ represent a pair of hyperparameters, co represents a weight of an edge e.sub.ij in each of the dynamic brain connection maps; a contrast loss function: L.sub.S=min(max{0, f(H.sub.S)−Σ.sub.a∈Af(H.sub.a)+γ});
L.sub.Δ=min(max{0,f(H.sub.a)−f(H.sub.a′)+γ}); L.sub.TOTAL=L.sub.S+L.sub.Δ; wherein max{0,•} represents a hinge loss, H.sub.S represents a spatio-temporal feature of a saline injection group, H.sub.a represents a spatio-temporal feature of a nicotine injection group, wherein a∈A, a′∈A, a≠a′, and A represents a total number of groups in the nicotine injection groups; γ represents a hyperparameter.

11. The method as recited in claim 1, further comprising the following step prior to obtaining the brain connection with a highest abnormal probability at each time point: inputting the abnormal probability of each of the brain connections into a preset classifier for iterative training until a classification accuracy converges to a preset accuracy.

12. The method as recited in claim 2, further comprising the following step prior to obtaining the brain connection with a highest abnormal probability at each time point: inputting the abnormal probability of each of the brain connections into a preset classifier for iterative training until a classification accuracy converges to a preset accuracy.

13. The method as recited in claim 3, further comprising the following step prior to obtaining the brain connection with a highest abnormal probability at each time point: inputting the abnormal probability of each of the brain connections into a preset classifier for iterative training until a classification accuracy converges to a preset accuracy.

14. The method as recited in claim 4, further comprising the following step prior to obtaining the brain connection with a highest abnormal probability at each time point: inputting the abnormal probability of each of the brain connections into a preset classifier for iterative training until a classification accuracy converges to a preset accuracy.

15. The method as recited in claim 5, further comprising the following step prior to obtaining the brain connection with a highest abnormal probability at each time point: inputting the abnormal probability of each of the brain connections into a preset classifier for iterative training until a classification accuracy converges to a preset accuracy.

16. The method as recited in claim 6, further comprising the following step prior to obtaining the brain connection with a highest abnormal probability at each time point: inputting the abnormal probability of each of the brain connections into a preset classifier for iterative training until a classification accuracy converges to a preset accuracy.

17. The method as recited in claim 1, wherein the step of generating the dynamic addictive neural circuit based on neuroscience prior knowledge and the brain connection with the highest abnormal probability at each time point comprises: referring to the neuroscience prior knowledge, correcting and integrating the brain connection with the highest abnormal probability, obtaining an addictive neural circuit at each time point, and generating the dynamic addictive neural circuit according to the time series.

18. The method as recited in claim 2, wherein the step of generating the dynamic addictive neural circuit based on neuroscience prior knowledge and the brain connection with the highest abnormal probability at each time point comprises: referring to the neuroscience prior knowledge, correcting and integrating the brain connection with the highest abnormal probability, obtaining an addictive neural circuit at each time point, and generating the dynamic addictive neural circuit according to the time series.

19. The method as recited in claim 3, wherein the step of generating the dynamic addictive neural circuit based on neuroscience prior knowledge and the brain connection with the highest abnormal probability at each time point comprises: referring to the neuroscience prior knowledge, correcting and integrating the brain connection with the highest abnormal probability, obtaining an addictive neural circuit at each time point, and generating the dynamic addictive neural circuit according to the time series.

20. A system for generating a dynamic addictive neural circuit based on weakly supervised contrastive learning, comprising: a dynamic brain connection map generation module, used to reduce voxels of a plurality of groups of fMRI images to attributes of brain region nodes based on a convolutional neural network, and generate a plurality of groups of dynamic brain connection maps containing time series based on the attributes of the brain region nodes; wherein each group of fMRI images comprises a plurality of fMRI images taken at a fixed frequency within a preset time period; a spatial-temporal feature extraction module, used to extract spatio-temporal features of brain connections in each of the dynamic brain connection maps; a most abnormal brain connection acquisition module, used to input the spatio-temporal features into an abnormal connection detection network, calculate a abnormal probability of brain connections based on contrastive learning, and obtain the brain connection with a highest abnormal probability at each time point; and a dynamic addictive neural circuit generation module, used to generate the dynamic addictive neural circuit based on neuroscience prior knowledge and the brain connection with the highest abnormal probability at each time point.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0067] FIG. 1 is a flowchart of a method for generating a dynamic addictive neural circuits based on weakly supervised contrastive learning.

[0068] FIG. 2 is a flowchart of a method for extracting spatio-temporal features of brain connections.

[0069] FIG. 3 is a block diagram of a system for generating a dynamic addictive neural circuit based on weakly supervised contrastive learning.

[0070] FIG. 4 is a block diagram of a spatio-temporal feature extraction module.

DETAILED DESCRIPTION OF EMBODIMENTS

[0071] For a better understanding of the objectives, technical solutions, and advantages of the present disclosure, hereinafter the present disclosure will be described in further detail in connection with the accompanying drawing FIGS. 1-4 and some illustrative embodiments. It should be understood that the specific embodiments described here are merely used to explain the present disclosure, rather than limit the present disclosure.

[0072] Embodiments of the present disclosure disclose a method for generating a dynamic addictive neural circuit based on weakly supervised contrastive learning. Referring to FIG. 1, a method for generating a dynamic addictive neural circuit based on weakly supervised contrastive learning includes the following steps.

[0073] S11: based on a convolutional neural network, voxels of multiple sets of fMRI images are reduced to the attributes of brain region nodes, and multiple sets of dynamic brain connection maps containing time series are generated based on the attributes of the brain region nodes.

[0074] That is, fMRI images of rat animal models in a nicotine addiction experiment are used to generate a dynamic addictive neural circuit based on weakly supervised contrastive learning. The multiple groups of fMRI images include fMRI images of rats injected with normal saline and rats injected with nicotine. The number of groups of rats in the normal saline group is 1, and the number of groups of rats in the nicotine injection group can be set depending on actual conditions, e.g., 1-4. Each set of fMRI images includes multiple fMRI images captured at a fixed frequency within a preset time period. The preset time period is the time period during which the rats are injected with normal saline or nicotine, which can be set to two weeks, three weeks, four weeks or other time periods, which are not to be limited herein.

[0075] Specifically, rats in the normal saline injection group and rats in the nicotine injection group are used for the experiment. Rats in the nicotine group can be divided into multiple groups depending on different concentrations, or only one group may be set. Rats in each group are continuously injected for a predetermined period of time to simulate nicotine intake and addiction. During the preset time period, the fMRI images of rats in each group are collected at a fixed frequency, and the fMRI images of rats in each group reflected the changes of brain nerves in the time dimension after injection of normal saline or nicotine of corresponding concentrations.

[0076] As such, the convolutional neural network is used to reduce the dimensionality of voxels of each group of fMRIs, making it possible to divide the time segments of fMRI at a meaningful time resolution while keeping the time resolution constant, where each segment is used as a time point to generate a dynamic brain connection map, which fully extracts the time-series dynamic information of fMRI.

[0077] The dynamic brain connection map G is: {G.sub.t}.sub.t=1.sup.T={(V.sub.t, E.sub.t, X.sub.t)}.sub.t=1.sup.T. Specifically, in the dynamic brain connection map G, fMRI time segments are used as time series, where each time series is regarded as a time point, brain regions are nodes, and brain connections are edges; Brain connection refers to the functional connections between brain regions; T is the total number of time points; G.sub.t is a snapshot of the dynamic brain connection map at time point t; V.sub.t and E.sub.t represent the node set and edge set respectively; The edge e.sub.ij∈E.sub.t, representing the brain function connection between the i-th node and the j-th node at time point t, and its weight is co; X.sub.t={x.sub.i.sup.t}.sub.1.sup.N represents the attribute set of the nodes, where N represents the total number of nodes and x.sub.i.sup.t is the attribute vector of the i-th node at time point t. The fMRI images of the normal saline injection group I.sub.S and the nicotine injection group I.sub.a are separately input into the above-mentioned dynamic brain connection map G to obtain the dynamic brain connection maps G.sub.S, G.sub.a, where a∈{1, 2, . . . , A}, A is the total number of nicotine injection groups.

[0078] In this step, fMRI images are directly input into the weakly supervised adversarial learning model to obtain a dynamic brain connection map, eliminating redundant and complex preprocessing computations.

[0079] S12: using the standard brain connection map as a real sample and the dynamic brain connection maps as fake samples to generate adversarial learning, and updating the dynamic brain connection map.

[0080] The standard brain connection map is the brain function connection map preprocessed by the standard spatial brain template containing the prior information of the brain atlas. The standard brain connection map is used as the real sample G.sub.True, and the dynamic brain connection map obtained in the previous step is used as the fake sample G.sub.Fake to generate adversarial learning and update the dynamic brain connection maps.

[0081] The formula for generative adversarial learning is:

min max L(D,G)=E.sub.G.sub.True˜Pdata(GTrue)[log D(G.sub.True)]+E.sub.G.sub.Fake˜Pdata(GFake)[log(1−D(G(G.sub.Fake))]

[0082] where D is a discriminator representing the standard brain connection map, and G is a generator representing an abnormal brain connection map. In this application, G specifically refers to the dynamic brain connection map. Adversarial learning is generated for the dynamic brain connection maps of three categories, and the dynamic brain connection maps are updated, that is, G.sub.S and G.sub.a are input into the above formulas for update. The updated dynamic brain connection maps are dynamic brain connection maps that approximate the spatial information of the brain atlas, and represent the abnormal brain connection patterns of nicotine addiction under different categories.

[0083] It should be noted that step S12 is optional. If step 12 is performed, through adversarial learning, a dynamic brain connection map that not only conforms to the spatial position information of the standard brain template, but also has the original fMRI voxel spatial information is obtained, which improves the training accuracy. Furthermore, the annotated brain functional connection map is obtained by preprocessing the standard spatial brain template containing the prior information of the brain atlas. The standard spatial brain template has a small amount of data and low learning difficulty.

[0084] S13: extracting spatio-temporal features of brain connections in each of the dynamic brain connection maps.

[0085] Each group of dynamic brain connection maps is input into the neural network for learning, and the spatial and temporal features of brain connections in the dynamic brain connection maps are extracted.

[0086] Specifically, referring to FIG. 2, step S13 includes the following sub-steps.

[0087] S1311: using a graph convolutional neural network based on a topological space brain connection focus mechanism to extract spatial features of brain connections in a dynamic brain connection map.

[0088] The graph convolutional neural network based on the topological space brain connection focus mechanism is composed of multi-head anomalous connection focus block (MACFB), which has a powerful graph learning ability and can learn the geometric space features between nodes in the connection network.

[0089] Specifically, the number of abnormal connection focus blocks is C, and the value of C may be 2, 3 or 4.

[0090] Hidden state feature vector set: H.sub.(l).sup.(t)={h.sub.(l)1.sup.(t), h.sub.(l)2.sup.(t), . . . , h.sub.(l)i.sup.(t), . . . , h.sub.(l)N.sup.(t)},

[0091] where H.sub.(l).sup.(t) represents the multi-head abnormal connection focus block of layer 1 at time point t, t∈{1, 2, . . . , T}, T is the total number of time points; i represents a node, i∈{1, 2, . . . , N}, N is the total number of nodes. h.sub.(l)i.sup.(t) represents the multi-head abnormal connection focus block of the i-th node in the 1-th layer at time point t.

[0092] Query—node attribute vector: q.sub.c,i.sup.(l)=W.sub.c,q.sup.(l)h.sub.(l)i.sup.(t)+b.sub.c,q.sup.(l);

[0093] where i represents a node, c represents an abnormal connection focus block, W.sub.c,q.sup.(l) is a learnable weight matrix for computing a query-node attribute vector, b % is an offset for calculating of the query-node attribute vector;

[0094] Key-node attribute vector: k.sub.c,j.sup.(l)=W.sub.c,k.sup.(l)h.sub.(l)j.sup.(t)+b.sub.c,k.sup.(l);

[0095] where j represents a node, W.sub.c,k.sup.(l), is a learnable weight matrix for calculating the key-node attribute vector, b.sub.c,k.sup.(l) is an offset for calculating the key-node attribute vector;

[0096] Additional brain connection feature vector: e.sub.c,ij=W.sub.c,ee.sub.ij+b.sub.c,e;

[0097] where e.sub.ij represents the brain connection between the i-th node and the j-th node at time point t, and its weight is ω. W.sub.c,e is a learnable weight matrix for computing additional brain connection feature vectors, and b.sub.c,e is an offset for computing additional brain connection feature vectors.

[0098] Brain connection focus coefficient:

[00002] $α_{c, ij}^{(l)} = \frac{.Math. q_{c, i}^{(l)}, k_{c, j}^{(l)} + e_{c, ij} .Math.}{{.Math.}_{u \in N (i)} .Math. q_{c, i}^{(l)}, k_{c, u}^{(l)} + e_{c, iu} .Math.};$

where <q, k> is an exponential dot product function, and N(i) is a value other than i.

[0099] After calculating the brain connection focus coefficient, the message transmission in the graph is carried out to output the hidden state feature vector of the next layer of nodes;

h.sub.(l+1)i.sup.(t)=∥.sub.c=1.sub.C[Σ.sub.jα.sub.c,ij(v.sub.c,j.sup.(l)+e.sub.c,ij)];

v.sub.c,j.sup.(l)=W.sub.c,v.sup.(l)h.sub.(l)i.sup.(t)+b.sub.c,v.sup.(l);

[0100] where W.sub.c,v.sup.(l) is the learnable weight matrix for calculating v.sub.c,j.sup.(l), b.sub.c,v.sup.(l) is an offset of calculating v.sub.c,j.sup.(l).

[0101] The set H.sub.(l+1).sup.(t) of the hidden state feature vectors h.sub.(l+1)i.sup.(t) of the next layer of nodes is used as the input of the next layer of multi-head abnormal connection focus blocks, or as the final output of the graph convolutional neural network. If it is used as the final output of the graph convolutional neural network, it represents the extracted spatial features of the abnormal brain connection pattern in the dynamic brain connection map.

[0102] S1312: using a gated recurrent unit based on a time-series brain connection focus mechanism to extract temporal features of the dynamic brain connection map.

[0103] In conjunction with the gated recurrent unit of the brain connection focus mechanism, short-term and long-term hidden state features are learned, so that the hidden state features of each time-series brain functional connection map include other time-series contextual brain connection hidden state information, thereby learning the temporal features of the dynamic brain connection maps.

[0104] In the gated recurrent unit, it is needed to calculate the hidden state eigenvector at the next time t+1 from the hidden state eigenvector at time point t.

[0105] Long-term hidden state feature vector set: Current.sup.t+1=MACFB(H.sup.t);

[0106] In the initial state, H.sup.t represents the output of the graph convolutional neural network, that is, the extracted spatial features of the abnormal brain connection patterns in the dynamic brain connection map.

[0107] The short-term hidden state feature vectors are calculated based on brain connection focus window. In the brain function connection focus window, the output of the node hidden state feature vector: C.sub.i.sup.t+1=[h.sub.i.sup.t−w, . . . , h.sub.i.sup.t];

[0108] where the window size is w−1, and i represents a node.

[0109] Time series brain connection focus coefficient: e.sub.i.sup.t+1=r.sup.T tan h(Q(C.sub.i.sup.t+1).sup.T);

[0110] where r is the weight vector, Q is the weight matrix;

a.sub.i.sup.t+1=softmax(e.sub.i.sup.t+1);

[0111] Short-term node hidden state feature vector: short.sub.i.sup.t+1=(a.sub.i.sup.t+1C.sub.i.sup.t+1).sup.T.

[0112] The brain connection hidden state feature vector set at the next time point: H.sup.t+1=GRU(Current.sup.t+1, Short.sup.t+1);

[0113] Short.sup.t+1 represents the set of short-term hidden state feature vectors short.sub.i.sup.t+1 of all nodes, i∈{1, 2, . . . , N}, N is the total number of nodes.

[0114] Further, the specific calculation process of H.sup.t+1=GRU(Current.sup.t+1, Short.sup.t+1) is:

[0115] Brain connection state change gate: P.sup.t+1=σ(U.sub.pCurrent.sup.t+1+W.sub.pShort.sup.t+1+b.sub.p);

[0116] where U.sub.p and W.sub.p are both learnable weight matrices for calculating P.sup.t+1, and b.sub.p is the offset for calculating p.sup.t+1.

[0117] Brain connection state reorganization gate: R.sup.t+1=σ(U.sub.RCurrent.sup.t+1+W.sub.RShort.sup.t+1+b.sub.R);

[0118] where U.sub.R and W.sub.R are both learnable weight matrices for calculating R.sup.t+1, and b.sub.R is the offset for calculating R.sup.t+1.

{tilde over (H)}.sup.t+1=tan h(U.sub.HCurrent.sup.t+1+W.sub.H(R.sup.t+1⊙Short.sup.t+1));

[0119] where U.sub.H and W.sub.H are both learnable weight matrices for calculating {tilde over (H)}.sup.t+1.

H.sup.t+1=(1−p.sup.t+1)⊙Short.sup.t+1+p.sup.t+1⊙{tilde over (H)}.sup.t+1.

[0120] When calculating temporal features, the input is spatial features. Therefore, when using the gated recurrent unit based on the time-series brain connection focus mechanism to extract temporal features, the hidden state feature vector set H.sup.t output at time point t represents the extracted spatio-temporal features of the abnormal brain connection pattern from the dynamic brain connection map.

[0121] S14: inputting the spatio-temporal features into the abnormal connection detection network, calculating the abnormal probability of brain connections based on contrastive learning, and obtaining the brain connection with the highest abnormal probability at each time point.

[0122] In the anomaly connection detection network, at time point t, the output H.sup.t of the hidden state feature vector set is calculated through the fully connected layer, and the abnormal brain connection probability at time point t is obtained as:

F.sub.score=MLP(H.sup.t);

[0123] where the abnormal probability represents the abnormal probability of edge e.sub.ijεE.sub.t at time point t, that is, the abnormal probability of each edge in the dynamic brain connection maps G.sub.S and G.sub.a.

f(i,j,ω)=ω.Math.(∥a⊙h.sub.i+b⊙h.sub.j∥.sub.2.sup.2−μ));

[0124] where h.sub.i is the hidden state feature vector of the i-th node, h.sub.j is the hidden state of the j-th node, σ(•) is a sigmoid function, a and b are the optimization parameters of the output layer, β and μ are a pair of hyperparameters, co represents the weight of the edge e.sub.ij in the dynamic brain connection map; the spatio-temporal features of each group of dynamic brain connection maps are input into the above function f(•) to obtain f(H.sub.S) and f(H.sub.a).

[0125] Using the strategy of contrastive learning, the contrastive loss function of abnormal brain connection learning is defined as:

L.sub.S=min(max{0,f(H.sub.S)−Σ.sub.a=1.sup.Af(H.sub.a)+γ});

L.sub.Δ=min(max{0,f(H.sub.a)−f(H.sub.a′)+γ});

L.sub.TOTAL=L.sub.S+ΣL.sub.Δ;

[0126] where max max{0,•} is the hinge loss, H.sub.S is the spatio-temporal feature of the saline injection group, H.sub.a is the spatio-temporal feature of the nicotine injection group, where a∈A, a′∈A, a≠a′, A is the total number of the nicotine injection groups; γ is a hyperparameter.

[0127] Using the above-mentioned contrastive learning strategy, the saline injection group and all nicotine injection groups are used for contrastive learning, and the multiple nicotine injection groups are subjected to pairwise contrastive learning, which can fully analyze the individual differences in the dynamic brain connection maps between different control groups, and facilitate the discovery of addictive abnormal brain connections.

[0128] After the abnormal probability of the brain connection in each dynamic brain connection map is calculated by contrastive learning, the brain connection with the highest abnormal probability is obtained from multiple groups of dynamic brain connection maps at each time point.

[0129] As an implementation, before obtaining the brain connection with the highest abnormal probability at each time point, the abnormal probability of the brain connection is input to a preset classifier for iterative training until the classification accuracy converges to the preset accuracy.

[0130] Specifically, at time point t, the abnormal probability of each brain connection in each group of dynamic brain connection maps is input to a preset classifier for iterative training. The preset classifier can be selected according to actual needs, and the following classifiers may be able to realize the above functions, such as SVM, KNN, Naive Bayesian, decision tree, logistic regression, neural network algorithm, etc., which are not specifically limited herein. The preset accuracy is set according to actual requirements, and is not limited herein.

[0131] S15: generating a dynamic addictive neural circuit based on neuroscience prior knowledge and the brain connection with the highest abnormal probability at each time point.

[0132] In combination with the neuroscience prior knowledge, for each time point t, the brain functional connection with the highest abnormal probability is corrected and integrated to obtain the addictive neural circuit at each time point, and then according to the entire time series, a dynamic addictive neural circuit is generated. Correcting the brain connection with the highest abnormal probability at each time point refers to deleting the wrong brain connection and/or adjusting the brain connection with a large deviation based on the neuroscience prior knowledge. The integration of the most abnormal brain functional connection refers to the integration of the corrected brain function into a complete addictive neural circuit for each time point.

[0133] In this embodiment, based on the group label of each group of samples and a small amount of neuroscience prior knowledge, combined with contrastive learning, a weakly supervised contrastive learning model is formed, and the training difficulty is low. Furthermore, fMRI images are directly input into the weakly supervised contrastive learning model, eliminating redundant and complex preprocessing calculations. Furthermore, under the premise of small samples, through contrastive learning, the differences in brain connections between different groups of samples are obtained, and in combination with a small amount of neuroscience prior knowledge, the neural circuit mechanism of addiction is revealed, which is easy to train and has high training accuracy.

[0134] Embodiments of the disclosure further disclose a system for generating a dynamic addictive neural circuit based on weakly supervised contrastive learning. Referring to FIG. 3, the system for generating a dynamic addictive neural circuit based on weakly supervised contrastive learning includes:

[0135] a dynamic brain connection map generation module 10 used to reduce the voxels of multiple groups of fMRI images to the attributes of brain region nodes based on a convolutional neural network, and generate multiple groups of dynamic brain connection maps containing time series according to the attributes of the brain region nodes; where each group of fMRI includes multiple fMRI images taken at a fixed frequency within a preset time period;

[0136] a dynamic brain connection update module 20 used to use a standard brain connection map as a real sample and the dynamic brain connection maps as fake samples to generate adversary learning and update the dynamic brain connection maps;

[0137] a spatial-temporal feature extraction module 30 used to extract the spatio-temporal features of brain connections in each of the dynamic brain connection maps;

[0138] a most abnormal brain connection acquisition module 40 used to input the spatio-temporal features into the abnormal connection detection network, calculate the abnormal probability of brain connections based on contrastive learning, and obtain the brain connection with the highest abnormal probability at each time point;

[0139] a dynamic addictive neural circuit generation module 50 used to generate a dynamic addictive neural circuit based on neuroscience prior knowledge and the brain connection with the highest abnormal probability at each time point.

[0140] The dynamic brain connection update module 20 is optional.

[0141] With reference to FIG. 4, the spatio-temporal feature extraction module 30 specifically includes:

[0142] a spatial feature extraction module 3010 used to extract the spatial features of the brain connection in the dynamic brain connection map by using a graph convolutional neural network based on the topological space brain connection focus mechanism; and

[0143] a temporal feature extraction module 3020 used for extracting the temporal features of the dynamic brain connection map by using a gated recurrent unit based on the time-series brain connection focus mechanism.

[0144] The system embodiments described above can be used to implement the above method embodiments, and the principles and technical effects thereof are similar, and will not be repeated here.

[0145] All of the above are merely some illustrative embodiments of the present disclosure, and are not intended to limit the scope of protection of the present disclosure. Any feature disclosed in this specification (including abstracts and drawings), unless specifically stated, may be replaced by other equivalent or alternative features serving a similar purpose. That is, unless expressly stated otherwise, each feature is merely one example of a series of equivalent or similar features.

METHOD AND SYSTEM FOR GENERATING A DYNAMIC ADDICTIVE NEURAL CIRCUITS BASED ON WEAKLY SUPERVISED CONTRASTIVE LEARNING

Inventors

Cpc classification

Classification Explorer

G06N3/044

PHYSICS

Classification Explorer

G01R33/5608

PHYSICS

Classification Explorer

G06N3/047

PHYSICS

Classification Explorer

G06V10/454

PHYSICS

Classification Explorer

G06N3/084

PHYSICS

Classification Explorer

G06T2207/20076

PHYSICS

Classification Explorer

G06N3/08

PHYSICS

Classification Explorer

G06T7/0014

PHYSICS

Classification Explorer

G06N3/045

PHYSICS

Classification Explorer

G06N3/094

PHYSICS

Classification Explorer

G06N3/088

PHYSICS

Classification Explorer

G06V10/82

PHYSICS

Classification Explorer

G06N3/082

PHYSICS

Classification Explorer

G06T2207/10088

PHYSICS

Classification Explorer

G06N3/04

PHYSICS

Classification Explorer

G06V10/62

PHYSICS

International classification

Classification Explorer

G06T7/00

PHYSICS

Classification Explorer

G06V10/44

PHYSICS

Classification Explorer

G06V10/62

PHYSICS

Classification Explorer

G06N3/094

PHYSICS

Classification Explorer

G01R33/56

PHYSICS

Abstract

Claims

Description