Systems and methods for managing visual allocation are provided herein that use models to determine states based on visual data and, based thereon, output feedback based on the determined states. Visual data is initially obtained by a visual allocation management system. The visual data includes eye image sequences of a person in a particular state, such as engaging in a task or activity. Visual features can be identified from the visual data, such that glance information including direction and duration can be calculated. The visual data, information derived therefrom, and/or other contextual data is input into the models, which correspond to states, to calculate probabilities that the particular state that the person is engaged in is one of the modeled states. Based on the state identified as having the highest probability, an optimal feedback, such as a warning or instruction, can be output to a connected devices, systems, or objects.

Systems and methods are disclosed to objectively identify sub-second behavioral modules in the three-dimensional (3D) video data that represents the motion of a subject. Defining behavioral modules based upon structure in the 3D video data itself—rather than using a priori definitions for what should constitute a measurable unit of action—identifies a previously-unexplored sub-second regularity that defines a timescale upon which behavior is organized, yields important information about the components and structure of behavior, offers insight into the nature of behavioral change in the subject, and enables objective discovery of subtle alterations in patterned action. The systems and methods of the invention can be applied to drug or gene therapy classification, drug or gene therapy screening, disease study including early detection of the onset of a disease, toxicology research, side-effect study, learning and memory process study, anxiety study, and analysis in consumer behavior.

Markov random field parameters are identified to use for covariance modeling of correlation between gradient terms of a loss function of the classifier. A subset of images are sampled, from a dataset of images, according to a normal distribution to estimate the gradient terms. Black-box gradient estimation is used to infer values of the parameters of the Markov random field according to the sampling. Fourier basis vectors are generated from the inferred values. An original image is perturbed using the Fourier basis vectors to obtain loss function values. An estimate of a gradient is obtained from the loss function values. An image perturbation is created using the estimated gradient. The image perturbation is added to an original input to generate a candidate adversarial input that maximizes loss in identifying the image by the classifier. The neural network classifier is queried to determine a classifier prediction for the candidate adversarial input.

A method of classifying an interaction captured in a sequence of video. A plurality of people in the video sequence is identified. An action of a first one of the people at a first time is determined. An action of a second one of the people at a second time is determined, the action of the second person being after the action of the first person. A role for the second person at the second time is determined, the role being independent of the determined actions of the first and second person. An interaction between the first person and the second person is classified based on the determined role of the second person and the determined actions of the first and second person.

Systems and methods for managing visual allocation are provided herein that use models to determine states based on visual data and, based thereon, output feedback based on the determined states. Visual data is initially obtained by a visual allocation management system. The visual data includes eye image sequences of a person in a particular state, such as engaging in a task or activity. Visual features can be identified from the visual data, such that glance information including direction and duration can be calculated. The visual data, information derived therefrom, and/or other contextual data is input into the models, which correspond to states, to calculate probabilities that the particular state that the person is engaged in is one of the modeled states. Based on the state identified as having the highest probability, an optimal feedback, such as a warning or instruction, can be output to a connected devices, systems, or objects.

Disclosed are systems and methods for automatic selection of canonical digital images from a large corpus of digital images, such as the corpus of digital images available on the web, for an entity, such as and without limitation a person, a point of interest, object, etc. The automated, unsupervised approach for selecting a diverse set of high quality, canonical digital images, is well suited for processing a large corpus of digital images. A set of canonical digital images identified for an entity can be retrieved in response to a digital image request for digital images depicting the entity.

The disclosure relates to systems and methods for evaluating a blood vessel. The method includes receiving image data of the blood vessel acquired by an image acquisition device, and predicting, by a processor, blood vessel condition parameters of the blood vessel by applying a deep learning model to the acquired image data of the blood vessel. The deep learning model maps a sequence of image patches on the blood vessel to blood vessel condition parameters on the blood vessel, where in the mapping the entire sequence of image patches contribute to the blood vessel condition parameters. The method further includes providing the blood vessel condition parameters of the blood vessel for evaluating the blood vessel.

A method, apparatus and computer-readable storage medium are provided to define a storyline based on path probabilities for a plurality of paths through the frames of a video. Relative to a method and for a plurality of frames of a video, regions of a first frame and regions of a second, subsequent frame that have been viewed are identified. tified. For each of at least one first-frame region of one or more regions of the first frame, the method determines a transition probability of transitioning from a respective first-frame frame region of the first frame to each of at least one second-frame region of a plurality of regions of the second frame. Based on the transition probabilities, the method determines a path probability for each of at least one of a plurality of paths through the frames of the video. The method additionally defines a storyline based on the path probabilities.

Image data of a vehicle occupant are collected from a plurality of cameras. A dimensional model of substantially an entire body of the vehicle occupant is generated based on the image data. A gesture performed by the vehicle occupant is recognized based on the dimensional model. A vehicle subsystem is adjusted based on the gesture.

A non-hierarchical and iteratively updated tracking system includes a first module for creating an initial trajectory model for multiple targets from a set of received image detections. A second module is connected to the first module to provide identification of multiple targets using a target model, and a third module is connected to the second module to solve a joint object function and maximal condition probability for the target module. A tracklet module can update the first module trajectory module, and after convergence, output a trajectory model for multiple targets.