The inventive subject matter is directed to an artificial intelligence intra-operative surgical guidance system and method of use. The artificial intelligence intra-operative surgical guidance system is made of a computer executing one or more automated artificial intelligence models trained on data layer datasets collections to calculate surgical decision risks, and provide intra-operative surgical guidance; and a display configured to provide visual guidance to a user.

Existing cognitive robotic applications follow a practice of building specific applications for specific use cases. However, the knowledge of the world and the semantics are common for a robot for multiple tasks. In this disclosure, to enable usage of knowledge across multiple scenarios, a method and system for ontology guided indoor scene understanding for cognitive robotic tasks is described where in scenes are processed based on techniques filtered based on querying ontology with relevant objects in perceived scene to generate a semantically rich scene graph. Herein, an initially manually created ontology is updated and refined in online fashion using external knowledge-base, human robot interaction and perceived information. This knowledge helps in semantic navigation, aids in speech, and text based human robot interactions.

Existing cognitive robotic applications follow a practice of building specific applications for specific use cases. However, the knowledge of the world and the semantics are common for a robot for multiple tasks. In this disclosure, to enable usage of knowledge across multiple scenarios, a method and system for ontology guided indoor scene understanding for cognitive robotic tasks is described where in scenes are processed based on techniques filtered based on querying ontology with relevant objects in perceived scene to generate a semantically rich scene graph. Herein, an initially manually created ontology is updated and refined in online fashion using external knowledge-base, human robot interaction and perceived information. This knowledge helps in semantic navigation, aids in speech, and text based human robot interactions.

A non-transitory computer-readable storage medium storing a database construction program that causes a computer to execute a process that includes analyzing an input image or text to generate a semantic representation including a plurality of subgraphs defining relationships between a plurality of first parts of speech and a plurality of second parts of speech; extracting, from a plurality of subgraphs stored in a database, third parts of speech having relationships with the first parts of speech included in the plurality of subgraphs of the semantic representation; generating first knowledge including a plurality of subgraphs in which the first parts of speech in the plurality of subgraphs of the semantic representation are replaced with the third parts of speech; and registering, in the database, a remaining subgraph obtained by removing a contradictory subgraph from the plurality of subgraphs included in the first knowledge based on the semantic representation and the database.

A method for analyzing one or more conditions of a transportation pathway includes obtaining, using an imaging device of an inspection system, image data reproducible as a plurality of images of the transportation pathway, each of the plurality of images being reproducible as an image of a portion of the transportation pathway, each portion of the transportation pathway having an associated location along a length of the transportation pathway, analyzing, using one or more processors of the inspection system, the image data to determine a first plurality of metrics indicative of a condition of the transportation pathway at each of the associated locations, and generating a first graph, using the determined first plurality of metrics, that is indicative of the condition of the transportation pathway at each of the associated locations.

A method for analyzing one or more conditions of a transportation pathway includes obtaining, using an imaging device of an inspection system, image data reproducible as a plurality of images of the transportation pathway, each of the plurality of images being reproducible as an image of a portion of the transportation pathway, each portion of the transportation pathway having an associated location along a length of the transportation pathway, analyzing, using one or more processors of the inspection system, the image data to determine a first plurality of metrics indicative of a condition of the transportation pathway at each of the associated locations, and generating a first graph, using the determined first plurality of metrics, that is indicative of the condition of the transportation pathway at each of the associated locations.

This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize weakly supervised graph matching to align an ungrounded label graph and a visual graph corresponding to a digital image. Specifically, the disclosed system utilizes a label embedding model to generate label graph embeddings from the ungrounded label graph and a visual embedding network to generate visual graph embeddings from the visual graph. Additionally, the disclosed system determines similarity metrics indicating the similarity of pairs of label graph embeddings and visual graph embeddings. The disclosed system then generates a semantic scene graph by utilizing a graph matching algorithm to align the ungrounded label graph and the visual graph based on the similarity metrics. In some embodiments, the disclosed system utilizes contrastive learning to modify the embedding models. Furthermore, in additional embodiments, the disclosed system utilizes the semantic scene graph to train a scene graph generation neural network.

This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize weakly supervised graph matching to align an ungrounded label graph and a visual graph corresponding to a digital image. Specifically, the disclosed system utilizes a label embedding model to generate label graph embeddings from the ungrounded label graph and a visual embedding network to generate visual graph embeddings from the visual graph. Additionally, the disclosed system determines similarity metrics indicating the similarity of pairs of label graph embeddings and visual graph embeddings. The disclosed system then generates a semantic scene graph by utilizing a graph matching algorithm to align the ungrounded label graph and the visual graph based on the similarity metrics. In some embodiments, the disclosed system utilizes contrastive learning to modify the embedding models. Furthermore, in additional embodiments, the disclosed system utilizes the semantic scene graph to train a scene graph generation neural network.

There is described a neural network system implemented by one or more computers for determining graph similarity. The neural network system comprises one or more neural networks configured to process an input graph to generate a node state representation vector for each node of the input graph and an edge representation vector for each edge of the input graph; and process the node state representation vectors and the edge representation vectors to generate a vector representation of the input graph. The neural network system further comprises one or more processors configured to: receive a first graph; receive a second graph; generate a vector representation of the first graph; generate a vector representation of the second graph; determine a similarity score for the first graph and the second graph based upon the vector representations of the first graph and the second graph.

There is described a neural network system implemented by one or more computers for determining graph similarity. The neural network system comprises one or more neural networks configured to process an input graph to generate a node state representation vector for each node of the input graph and an edge representation vector for each edge of the input graph; and process the node state representation vectors and the edge representation vectors to generate a vector representation of the input graph. The neural network system further comprises one or more processors configured to: receive a first graph; receive a second graph; generate a vector representation of the first graph; generate a vector representation of the second graph; determine a similarity score for the first graph and the second graph based upon the vector representations of the first graph and the second graph.