An information processing apparatus acquires video image data that includes target objects including a person and an object, and specifies, by inputting the acquired video image data to a first machine learning model, a relationship between each of the target objects included in the acquired video image data. The information processing apparatus specifies, by using a feature value of the person included in the acquired video image data, a behavior of the person included in the video image data. The information processing apparatus predicts, by inputting the specified behavior of the person and the specified relationship to a probability model, a future behavior or a future state of the person.

Systems and methods for managing visual allocation are provided herein that use models to determine states based on visual data and, based thereon, output feedback based on the determined states. Visual data is initially obtained by a visual allocation management system. The visual data includes eye image sequences of a person in a particular state, such as engaging in a task or activity. Visual features can be identified from the visual data, such that glance information including direction and duration can be calculated. The visual data, information derived therefrom, and/or other contextual data is input into the models, which correspond to states, to calculate probabilities that the particular state that the person is engaged in is one of the modeled states. Based on the state identified as having the highest probability, an optimal feedback, such as a warning or instruction, can be output to a connected devices, systems, or objects.

Computer-implemented methods and systems are provided for interpreting a whole-slide image of a tissue specimen. Such a method includes detecting biological entities in a region of interest independently at each magnification level in a whole-slide image. An entity graph is generated for each magnification level, the entity graph comprising nodes, representing respective biological entities detected at that magnification level, interconnected by edges representing interactions between entities. The method also includes, for each region of interest, generating a hierarchical graph, defining a hierarchy of the entity graphs for that region, in which nodes of different entity graphs are interconnected by hierarchical edges representing hierarchical relations between nodes of the entity graphs. The method further comprises supplying the hierarchical graph for the region to an AI system to produce result data corresponding to a medical evaluation of the tissue specimen represented thereby, and outputting the result data for the tissue specimen.

A method of tracking objects of a scene is disclosed. The method determines two or more tracks which have merged. Each track is associated with at least one of the objects and having a corresponding graph structure. Each graph structure comprising at least one node representing the corresponding track. A new node representing the merged tracks is created. The graph structures are added as children nodes of the new node to create a merged graph structure. A split between the objects associated with one of the tracks represented by the nodes of the merged graph structure is determined. Similarity between one or more of the nodes in the merged graph structure and foreground areas corresponding to split objects is determined. One of the nodes in the merged graph structure is selected based on the determined similarity. A new graph structure for tracking the objects is created, the new graph structure having the selected node at the root of the new graph structure.

Implementations are described herein for tracking objects with changing appearances across temporally-disparate images. In various implementations, a first probability distribution over a plurality of classes may be determined for a first biological object depicted in a first image captured at a first point in time. The classes may represent stages of growth of biological objects. Additional probability distribution(s) over the plurality of classes may be determined for candidate biological object(s) depicted in a second image captured at a second point in time subsequent to the first point in time. The candidate biological object(s) may potentially match the first biological object depicted in the first image. Based on a time interval between the first and second points in time, the first probability distribution may be compared to the probability distribution(s) of the candidate biological object(s) depicted in the second image to match one of the candidate biological object(s) depicted in the second image to the first biological object depicted in the first image.

Implementations are described herein for tracking objects with changing appearances across temporally-disparate images. In various implementations, a first probability distribution over a plurality of classes may be determined for a first biological object depicted in a first image captured at a first point in time. The classes may represent stages of growth of biological objects. Additional probability distribution(s) over the plurality of classes may be determined for candidate biological object(s) depicted in a second image captured at a second point in time subsequent to the first point in time. The candidate biological object(s) may potentially match the first biological object depicted in the first image. Based on a time interval between the first and second points in time, the first probability distribution may be compared to the probability distribution(s) of the candidate biological object(s) depicted in the second image to match one of the candidate biological object(s) depicted in the second image to the first biological object depicted in the first image.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using surfels for vehicle localization. One of the methods includes obtaining surfel data comprising a plurality of surfels, wherein each surfel corresponds to a respective different location in an environment, and each surfel has associated data that comprises a stability measure, wherein the stability measure characterizes a permanence of a surface represented by the surfel; obtaining sensor data for a plurality of locations in the environment, the sensor data having been captured by one or more sensors of a first vehicle; determining a plurality of high-stability surfels from the plurality of surfels in the surfel data; and determining a location in the environment of the first vehicle using the plurality of selected high-stability surfels and the sensor data.

Approaches are presented for training and using scene graph generators for transfer learning. A scene graph generation technique can decompose a domain gap into individual types of discrepancies, such as may relate to appearance, label, and prediction discrepancies. These discrepancies can be reduced, at least in part, by aligning the corresponding latent and output distributions using one or more gradient reversal layers (GRLs). Label discrepancies can be addressed using self-pseudo-statistics collected from target data. Pseudo statistic-based self-learning and adversarial techniques can be used to manage these discrepancies without the need for costly supervision from a real-world dataset.

Approaches are presented for training and using scene graph generators for transfer learning. A scene graph generation technique can decompose a domain gap into individual types of discrepancies, such as may relate to appearance, label, and prediction discrepancies. These discrepancies can be reduced, at least in part, by aligning the corresponding latent and output distributions using one or more gradient reversal layers (GRLs). Label discrepancies can be addressed using self-pseudo-statistics collected from target data. Pseudo statistic-based self-learning and adversarial techniques can be used to manage these discrepancies without the need for costly supervision from a real-world dataset.

An automatic body movement recognition and association system that uses two dimensional (2D) and/or three dimensional (3D) skeletal joint information from at least one of a stand-alone depth-sensing image capture device, sensor, wearable sensor, video, and/or video streams that detects the body movements of a user. The automatic body movement recognition and association system can perform various processes on the body movement data, such as smoothing, segmentation, similarity, pooling, and dynamic modeling.