Systems and methods for navigating a host vehicle are disclosed. In one implementation at least one processor is programmed to receive two or more images captured by a camera of the host vehicle from an environment of the host vehicle; analyze the two or more images to identify a representation of at least a portion of a first object and a representation of at least a portion of a second object; determine a first region of the two or more images associated with the first object and a type of the first object; and determine a second region of the two or more images associated with the second object and type of the second object, wherein the type of the first object is different from the type of the second object.

Autonomous vehicles need to plan at the task level to compute a sequence of symbolic actions, such as merging left and turning right, to fulfill people's service requests, where efficiency is the main concern. At the same time, the vehicles must compute continuous trajectories to perform actions at the motion level, where safety is the most important. Task-motion planning in autonomous driving faces the problem of maximizing task-level efficiency while ensuring motion-level safety. To this end, we develop algorithm Task-Motion Planning for Urban Driving (TMPUD) that, for the first time, enables the task and motion planners to communicate about the safety level of driving behaviors. TMPUD has been evaluated using a realistic urban driving simulation platform. Results suggest that TMPUD performs significantly better than competitive baselines from the literature in efficiency, while ensuring the safety of driving behaviors.

A method for generating at least one non-human animal crossing indicator, the method may include receiving by a vehicle computerized system, non-human animal crossing indicators; obtaining sensed information regarding an environment of the vehicle; processing the sensed information, wherein the processing comprises searching for one or more non-human animal crossing indicators of the non-human animal crossing indicators; wherein the non-human animal crossing element is selected out of (i) a non-human animal crossing object and (ii) a non-human animal crossing situation; autonomously determining, when finding at least one of the one or more non-human animal crossing identifiers, that the vehicle is driving towards a non-human animal crossing or is within the non-human animal crossing; and generating an alert when determining that the vehicle is driving towards the non-human animal crossing or is within the non-human animal crossing.

A vehicle can be detected approaching a limited operation zone of a travel area. It can be determined that the vehicle is able to execute wireless commands from a computer. The object can be identified within the limited operation zone. Based on identifying the object within the limited operation zone, the vehicle can be controlled by providing a control command via wireless communications, the control command including whether the vehicle is permitted to enter the limited operation zone.

A plurality of thermal images forward of a vehicle are collected. Thermal data in the plurality of thermal images is normalized based on an ambient air temperature to generate a plurality of normalized thermal images. The plurality of normalized thermal images are input to a machine learning program trained to output an identification of an object based on the ambient air temperature and a risk of collision between the vehicle and the object. A vehicle component is actuated based on the identification of the object and the risk of collision with the object.

A method for detecting obstacles, the method may include receiving, from a plurality of vehicles, and by an I/O module of a computerized system, visual information acquired during executions of vehicle maneuvers that are suspected as being obstacle avoidance maneuvers; determining, based at least on the visual information, at least one visual obstacle identifier for visually identifying at least one obstacle; and transmitting to one or more of the plurality of vehicles, the at least one visual obstacle identifier.

Disclosed are devices, systems and methods for an audio assistant in an autonomous or semi-autonomous vehicle. In one aspect the informational audio assistant receives a first set of data from a vehicle sensor and identifies an object or condition using the data from the vehicle sensor. Audio is generated representative of a perceived danger of an object or condition. A second set of data from the vehicle sensor subsystem is received and the informational audio assistant determines whether an increased danger exists based on a comparison of the first set of data to the second set of data. The informational audio assistant will apply a sound profile to the generated audio based on the increased danger.

A vehicle system includes a processor and a memory. The memory stores instructions executable by the processor to identify an area of interest from a plurality of areas on a map, to determine that a detected sound is received in a vehicle audio sensor upon determining that a source of the sound is within the area of interest and not another area in the plurality of areas, and to operate the vehicle based at least in part on the detected sound.

Techniques for determining to modify a trajectory based on an object are discussed herein. A vehicle can determine a drivable area of an environment, capture sensor data representing an object in the environment, and perform a spot check to determine whether or not to modify a trajectory. Such a spot check may include processing to incorporate an actual or predicted extent of the object into the drivable area to modify the drivable area. A distance between a reference trajectory and the object can be determined at discrete points along the reference trajectory, and based on a cost, distance, or intersection associated with the trajectory and the modified area, the vehicle can modify its trajectory. One trajectory modification includes following, which may include varying a longitudinal control of the vehicle, for example, to maintain a relative distance and velocity between the vehicle and the object.

Image processing techniques are described to receive bounding box information that describes a bounding box located around a detected obj ect in an image, determine one or more positions of one or more reference points on the bounding box, determine, for each reference point, 3D world coordinates of a point of intersection of the reference point and the road surface, and assign the 3D world coordinates of the one or more reference points to a location of the detected object.