Systems and methods can quantify the tumor microenvironment for diagnosis, prognosis and therapeutic response prediction by fusing different data types (e.g., morphological information from histology and molecular information from omics) using an algorithm that harnesses deep learning. The algorithm employs tensor fusion to provide end-to-end multimodal fusion to model the pairwise interactions of features across multiple modalities (e.g., histology and molecular features) and deep learning. The systems and methods improve upon traditional methods for quantifying the tumor microenvironment that rely on concatenation of extracted features.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for predicting gaze and awareness using a neural network model. One of the methods includes obtaining sensor data (i) that is captured by one or more sensors of an autonomous vehicle and (ii) that characterizes an agent that is in a vicinity of the autonomous vehicle in an environment at a current time point. The sensor data is processed using a gaze prediction neural network to generate a gaze prediction that predicts a gaze of the agent at the current time point. The gaze prediction neural network includes an embedding subnetwork that is configured to process the sensor data to generate an embedding characterizing the agent, and a gaze subnetwork that is configured to process the embedding to generate the gaze prediction.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for predicting gaze and awareness using a neural network model. One of the methods includes obtaining sensor data (i) that is captured by one or more sensors of an autonomous vehicle and (ii) that characterizes an agent that is in a vicinity of the autonomous vehicle in an environment at a current time point. The sensor data is processed using a gaze prediction neural network to generate a gaze prediction that predicts a gaze of the agent at the current time point. The gaze prediction neural network includes an embedding subnetwork that is configured to process the sensor data to generate an embedding characterizing the agent, and a gaze subnetwork that is configured to process the embedding to generate the gaze prediction.

A vision-LiDAR fusion method and system based on deep canonical correlation analysis are provided. The method comprises: collecting RGB images and point cloud data of a road surface synchronously; extracting features of the RGB images to obtain RGB features; performing coordinate system conversion and rasterization on the point cloud data in turn, and then extracting features to obtain point cloud features; inputting point cloud features and RGB features into a pre-established and well-trained fusion model at the same time, to output feature-enhanced fused point cloud features, wherein the fusion model fuses RGB features to point cloud features by using correlation analysis and in combination with a deep neural network; and inputting the fused point cloud features into a pre-established object detection network to achieve object detection. A similarity calculation matrix is utilized to fuse two different modal features.

An object detection device detects objects around an own vehicle by fusing a plurality of pieces of target information obtained by detecting the objects as targets using a plurality of detection sensors with different detection accuracies, and includes a crowd determination unit that determines that a target other than a reference target detected in a predetermined crowd region based on the reference target selected from the targets is a crowd target; a region setting unit that sets a search region so as to include a crowd determination region in which the reference target and the crowd target are detected; and an object determination unit that determines an object indicated by a target in the search region by fusing the target information for the search region.

An object detection device detects objects around an own vehicle by fusing a plurality of pieces of target information obtained by detecting the objects as targets using a plurality of detection sensors with different detection accuracies, and includes a crowd determination unit that determines that a target other than a reference target detected in a predetermined crowd region based on the reference target selected from the targets is a crowd target; a region setting unit that sets a search region so as to include a crowd determination region in which the reference target and the crowd target are detected; and an object determination unit that determines an object indicated by a target in the search region by fusing the target information for the search region.

This disclosure relates to the combining and interaction of multiple artificial intelligence (AI) models for medical image analysis. An example method includes obtaining AI models from model providers and organizing them to form associations. In response to a user request, base models are selected and provided. Additional models are further selected to combine with the base models, and medical image analysis results are presented based on applying a combination of the models to target medical image data.

This disclosure relates to the combining and interaction of multiple artificial intelligence (AI) models for medical image analysis. An example method includes obtaining AI models from model providers and organizing them to form associations. In response to a user request, base models are selected and provided. Additional models are further selected to combine with the base models, and medical image analysis results are presented based on applying a combination of the models to target medical image data.

A vehicle, system and method of navigating the vehicle. The system includes a sensor and a processor. The sensor is configured to obtain a first set of detection points representative of a lane of a road section at a first time step and a second set of detection points representative of the lane at a second time step. The processor is configured to determine a set of predicted points for the second time step from the first set of detection points, obtain a set of fused points from the second set of detection points and the set of predicted points, and navigate the vehicle using the set of fused points.

Various example embodiments provide a computer system of unsupervised learning with deep similarity for optical flow estimation and a method thereof. According to various example embodiments, the computer system may be configured to calculate deep similarity by using deep features extracted from a sequence of a plurality of images, and learn optical flow for the images based on the deep similarity. In other words, the computer system may learn deep learning model for estimating optical flow through unsupervised learning based on deep similarity for a sequence of a plurality of images. At this time, the computer system may learn optical flow by using a feature separation loss function obtained by dividing occlusion locations and non-occlusion locations on the deep similarity map.