This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize weakly supervised graph matching to align an ungrounded label graph and a visual graph corresponding to a digital image. Specifically, the disclosed system utilizes a label embedding model to generate label graph embeddings from the ungrounded label graph and a visual embedding network to generate visual graph embeddings from the visual graph. Additionally, the disclosed system determines similarity metrics indicating the similarity of pairs of label graph embeddings and visual graph embeddings. The disclosed system then generates a semantic scene graph by utilizing a graph matching algorithm to align the ungrounded label graph and the visual graph based on the similarity metrics. In some embodiments, the disclosed system utilizes contrastive learning to modify the embedding models. Furthermore, in additional embodiments, the disclosed system utilizes the semantic scene graph to train a scene graph generation neural network.

This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize weakly supervised graph matching to align an ungrounded label graph and a visual graph corresponding to a digital image. Specifically, the disclosed system utilizes a label embedding model to generate label graph embeddings from the ungrounded label graph and a visual embedding network to generate visual graph embeddings from the visual graph. Additionally, the disclosed system determines similarity metrics indicating the similarity of pairs of label graph embeddings and visual graph embeddings. The disclosed system then generates a semantic scene graph by utilizing a graph matching algorithm to align the ungrounded label graph and the visual graph based on the similarity metrics. In some embodiments, the disclosed system utilizes contrastive learning to modify the embedding models. Furthermore, in additional embodiments, the disclosed system utilizes the semantic scene graph to train a scene graph generation neural network.

A method for optically recognizing at least one target discrete entity from a plurality of discrete entities includes acquiring a first optical read-out of a marker of a first discrete entity that defines the target discrete entity, generating a first set of representations of the marker of the first discrete entity based on the first optical read-out, associating the first set of representations with the target discrete entity, acquiring a second optical read-out of a marker of at least one discrete entity from the plurality of discrete entities, generating a second set of representations of the marker of the at least one discrete entity based on the second optical read-out, comparing the second set of representations to the first set of representations, and recognizing the at least one discrete entity as the target discrete entity upon determining that the second set of representations matches the first set of representations.

There is described a neural network system implemented by one or more computers for determining graph similarity. The neural network system comprises one or more neural networks configured to process an input graph to generate a node state representation vector for each node of the input graph and an edge representation vector for each edge of the input graph; and process the node state representation vectors and the edge representation vectors to generate a vector representation of the input graph. The neural network system further comprises one or more processors configured to: receive a first graph; receive a second graph; generate a vector representation of the first graph; generate a vector representation of the second graph; determine a similarity score for the first graph and the second graph based upon the vector representations of the first graph and the second graph.

There is described a neural network system implemented by one or more computers for determining graph similarity. The neural network system comprises one or more neural networks configured to process an input graph to generate a node state representation vector for each node of the input graph and an edge representation vector for each edge of the input graph; and process the node state representation vectors and the edge representation vectors to generate a vector representation of the input graph. The neural network system further comprises one or more processors configured to: receive a first graph; receive a second graph; generate a vector representation of the first graph; generate a vector representation of the second graph; determine a similarity score for the first graph and the second graph based upon the vector representations of the first graph and the second graph.

A computational systems pathology spatial analysis platform includes: (i) a spatial heterogeneity quantification component configured for generating a global quantification of spatial heterogeneity among cells of varying phenotypes in multi-parameter cellular and subcellular imaging data; (ii) a microdomain identification component configured for identifying a plurality of microdomains for tissue samples based on the global quantification, each microdomain being associated with a tissue sample; and (iii) a weighted graph component configured for constructing a weighted graph for the multi-parameter cellular and subcellular imaging data, the weighted graph having a plurality of nodes and a plurality of edges each being located between a pair of the nodes, wherein in the weighted graph each node is a particular one of the microdomains and the edge between each pair of microdomains in the weighted graph is indicative of a degree of similarity between the pair of the microdomains.

A computational systems pathology spatial analysis platform includes: (i) a spatial heterogeneity quantification component configured for generating a global quantification of spatial heterogeneity among cells of varying phenotypes in multi-parameter cellular and subcellular imaging data; (ii) a microdomain identification component configured for identifying a plurality of microdomains for tissue samples based on the global quantification, each microdomain being associated with a tissue sample; and (iii) a weighted graph component configured for constructing a weighted graph for the multi-parameter cellular and subcellular imaging data, the weighted graph having a plurality of nodes and a plurality of edges each being located between a pair of the nodes, wherein in the weighted graph each node is a particular one of the microdomains and the edge between each pair of microdomains in the weighted graph is indicative of a degree of similarity between the pair of the microdomains.

Methods, systems, articles of manufacture, and apparatus to recalibrate confidences for image classification are disclosed. An example apparatus to classify an image includes an image crop detector to detect a first image crop from the image, the first image crop corresponding to a first object, a grouping controller to select a second image crop corresponding to a second object at a location of the first object, a prediction generator to, in response to executing a trained model, determine a label corresponding to the first object and a confidence level associated with the label, and a confidence recalibrator to recalibrate the confidence level based on a probability of the first object having a first attribute based on the second object having a second attribute, the confidence level recalibrated to increase an accuracy of the image classification.

Methods, systems, articles of manufacture, and apparatus to recalibrate confidences for image classification are disclosed. An example apparatus to classify an image includes an image crop detector to detect a first image crop from the image, the first image crop corresponding to a first object, a grouping controller to select a second image crop corresponding to a second object at a location of the first object, a prediction generator to, in response to executing a trained model, determine a label corresponding to the first object and a confidence level associated with the label, and a confidence recalibrator to recalibrate the confidence level based on a probability of the first object having a first attribute based on the second object having a second attribute, the confidence level recalibrated to increase an accuracy of the image classification.

The application discloses a lip movement capturing method and device and a storage medium. The method includes: acquiring a real-time image shot by a photographic device and extracting a real-time facial image from the real-time image; inputting the real-time facial image into a pretrained lip average model and recognizing t lip feature points representative of positions of lips in the real-time facial image; and calculating a movement direction and movement distance of the lips in the real-time facial image according to x and y coordinates of the t lip feature points in the real-time facial image. According to the application, movement information of the lips in the real-time facial image is calculated according to the coordinates of the lip feature points to implement real-time capturing of movements of the lips.