Techniques for determining unified futures of objects in an environment are discussed herein. Techniques may include determining a first feature associated with an object in an environment and a second feature associated with the environment and based on a position of the object in the environment, updating a graph neural network (GNN) to encode the first feature and second feature into a graph node representing the object and encode relative positions of additional objects in the environment into one or more edges attached to the node. The GNN may be decoded to determine a distribution of predicted positions for the object in the future that meet a criterion, allowing for more efficient sampling. A predicted position of the object in the future may be determined by sampling from the distribution.

Among other things, techniques are described for predicting how an agent (e.g., a vehicle, bicycle, pedestrian, etc.) will move in an environment based on prior movement, the road network, the surrounding objects and/or other relevant environmental factors. One trajectory prediction technique involves generating a probability map for an agent's movement. Another trajectory prediction technique involves generating a trajectory lattice, for an agent's movement. In addition, a different trajectory prediction technique involves multi-modal regression where a classifier (e.g., a neural network) is trained to classify the probability of a number of (learned) modes such that each model produces a trajectory based on the current input.

System, methods, and other embodiments described herein relate to resolving a standoff by a vehicle. In one embodiment, a method includes generating a happens-before relationship that explains events between the vehicle and other vehicles before the standoff. The standoff may be a dispute for a right of way between the vehicle and the other vehicles. The method also includes identifying the standoff using a causality relationship analysis according to the happens-before relationship. The method also includes generating a mitigation plan for the standoff that forms standoff solutions in association with the standoff being similar to a prior standoff. The method also includes resolving the standoff by causing vehicle maneuvers associated with the vehicle according to the standoff solutions.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent behavior prediction using keypoint data. One of the methods includes obtaining data characterizing a scene in an environment, the data comprising: (i) context data comprising data characterizing historical trajectories of a plurality of agents up to the current time point; and (ii) keypoint data for a target agent; processing the context data using a context data encoder neural network to generate a context embedding for the target agent; processing the keypoint data using a keypoint encoder neural network to generate a keypoint embedding for the target agent; generating a combined embedding for the target agent from the context embedding and the keypoint embedding; and processing the combined embedding using a decoder neural network to generate a behavior prediction output for the target agent that characterizes predicted behavior of the target agent after the current time point.

In a target identification device, an acquisition unit is configured to acquire trajectory information including information on a movement trajectory of a moving object in the surroundings of a vehicle. A calculation unit is configured to calculate a likelihood for each type of moving object from the trajectory information by using a plurality of models predefined for each type of moving object. A target identification unit is configured to identify the type of the moving object according to the likelihood calculated by the calculation unit.

An autonomous vehicle (AV) implements a notification system that provides notifications to user devices in nearby vehicles to alert drivers of the nearby vehicles of the AV's planned behavior. The notifications are delivered wirelessly and output by the user device to the drivers. The planned behaviors of the AV may not be conveyed by existing mechanisms, such as turning signals or hazard lights. The AV or the receiving user device may determine when an AV's planned behavior may impact a particular driver, and provide relevant notifications to the driver.

Systems and methods for operating a robot. The methods comprise: performing, by a processor, operations to detect an object that is moving; identifying, by the processor, detected behavior of the object that constitutes an unrecognized behavior; predicting, by the processor, future movement of the object based on a circle having a radius that is function of a velocity of the object; and controlling operations of the robot based on the predicting.

An obstacle is detected based on sensor data obtained from a plurality of sensors of the ADV. Multiple trajectories of the obstacle are predicted with corresponding probabilities including a first predicted trajectory of the obstacle with a highest probability and a second predicted trajectory of the obstacle with a second highest probability. A cautionary trajectory of the ADV is planned based on at least one of a difference between the highest probability and the second highest probability or a consequence of the second trajectory. The ADV is to drive with a speed lower than a speed limit and prepare to stop in the cautionary trajectory. The ADV is controlled to drive according to the cautionary trajectory.

An obstacle is detected based on sensor data obtained from a plurality of sensors of the ADV. A distribution of a plurality of positions of the obstacle at a point of time may be predicted. A range of positions of the plurality of positions of the obstacle may be determined based on a confidence level of the range. A modified shape with a modified length of the obstacle may be determined based on the range of positions of the obstacle. A trajectory of the ADV based on the modified shape with the modified length of the obstacle may be planned. The ADV may be controlled to drive according to the planned trajectory to drive safely to avoid a collision with the obstacle.

Example embodiments relate to a method for cut-in identification and classification. An example embodiment includes a obtaining operational data about one or more vehicles; based on the operational data, identifying the presence of one or more cut-ins within the operational data; extracting, from the operational data, cut-in data that depicts one or more of the cut-ins identified within the operational data; and, based on the extracted cut-in data, training a model for controlling an autonomous vehicle. Identifying the presence of a given cut-in includes: determining that at least one vertex of a bounding box surrounding a vehicle was located more than a threshold distance within a lane being navigated by a given vehicle; and determining that the ability of the given vehicle to maintain its course and speed was impeded by the presence of the particular additional vehicle within the lane.