The present invention extends to methods, systems, and computer program products for detecting hazards in anticipation of opening vehicle doors. Vehicle sensors (e.g., rear viewing cameras) can be used to detect and classify traffic, for example, as pedestrians, bicyclists, skateboarders, roller skaters, wheel chair, etc., approaching on the side of a vehicle. When there is a possibility of a vehicle occupant opening a door into approaching traffic, a warning can be issued in the vehicle cabin to alert vehicle occupants of the approaching traffic. In one aspect, a vehicle prevents a door from opening if opening the door would likely cause an accident.

Disclosed herein are systems and method including a method for managing an autonomous vehicle. The method include providing as first input to a machine learning model a raster image and a vector associated with a context of a scene comprising an autonomous vehicle and a plurality of agents, providing as second input to the machine learning model a planned travel path for the autonomous vehicle, based the first input and the second input, outputting from the machine learning model a plurality of yield/assert predictions, wherein the plurality of yield/assert predictions comprises a respective yield/assert prediction related to whether to yield or to assert in relation to each respective agent of the plurality of agents and causing the autonomous vehicle to travel along the planned travel path while yielding or asserting against the plurality of agents according to the plurality of yield/assert predictions.

A driving assistance device executes processing relating to a behavior model of a vehicle. Detected information from the vehicle is input to a detected information inputter. An acquirer derives at least one of a travel difficulty level of a vehicle, a wakefulness level of a driver, and a driving proficiency level of the driver on the basis of the detected information that is input to the detected information inputter. A determiner determines whether or not to execute processing on the basis of at least one information item derived by the acquirer. If the determiner has made a determination to execute the processing, a processor executes the processing relating to the behavior model. It is assumed that the processor does not execute the processing relating to the behavior model if the determiner has made a determination to not execute the processing.

A method, system and non-transitory computer readable medium which monitor a road user in order to move the external position of the vehicle intent notification (eHMI) to another external position that can be seen by the road user based on the gaze direction of the road user. In some aspects, the eHMI notification displays the vehicle intent for a single autonomous vehicle. In another aspect, a group eHMI notification displays the trajectories for a plurality of autonomous and non-autonomous vehicles. Based on the gaze direction of the road user, the eHMI notification can be displayed on a single external position or on multiple external positions. Different eHMI notifications can be displayed at different external positions on the autonomous vehicle to provide information to more than one road user.

In various examples, a multi-sensor fusion machine learning model—such as a deep neural network (DNN)—may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.

This document describes Kurtosis based pruning for sensor-fusion systems. Kurtosis based pruning minimizes a total quantity of comparisons performed when fusing together large sets of data. Multiple candidate radar tracks may possibly align with one of multiple candidate visual tracks. For each candidate vision track, a weight or other evidence of matching is assigned to each candidate radar track. An inverse of matching errors between each candidate vision and each candidate radar track contributes to this evidence, which may be normalized to produce, for each candidate vision track, a distribution associated with all candidate radar tracks. A Kurtosis or shape of this distribution is calculated. Based on the Kurtosis values, some candidate radar tracks are selected for matching and other remaining candidate radar tracks are pruned. The Kurtosis aids in determining how many candidates to retain and how many to prune. In this way, Kurtosis based pruning can prevent combinatorial explosions due to large-scale matching.

The technology relates to autonomous vehicles suffering a breakdown along a roadway. Onboard systems may utilize various proactive operations to alert specific vehicles or other objects on or near the roadway about the breakdown. This can be done alternatively or in addition to turning on the hazard lights or calling for remote assistance. The disabled vehicle is able to detect nearby and approaching objects. The detection may be performed in combination with a determination of the type of object or predicted behavior for that object, enables the vehicle to generate a targeted alert that can be transmitted or otherwise presented to that particular object. This approach provides the other object, such as a vehicle, bicyclist or pedestrian, sufficient time and information about the breakdown to take appropriate corrective action. Different communication options are available and may be selected based on the particular object, environmental conditions and other factors.

A cross-traffic warning system for a motor vehicle includes first and second input devices transmitting associated first and second input signals for first and second detected objects positioned on the roadway. The system further includes a computer having one or more processors and a computer readable medium storing instructions. The processor is programmed to determine that the first object is a Vulnerable Road User (“VRU”) travelling on a first path based on the first input signal. The processor is further programmed to determine that the second object is a third party vehicle and further that the VRU and the third party vehicle are travelling on an associated one of first and second paths to imminently collide with one another based on the first and second input signals. The processor is further programmed to generate an actuation signal in response to the processor determining the imminent collision.

A vehicle control apparatus comprises a control unit that performs a traveling control, when a driver has fallen into an inappropriate state; and a notification device that performs a surround notification to notify a person around the vehicle of an execution of the traveling control. The control unit is configured to perform, as the traveling control, at least two of an in-lane stop control, a road shoulder stop control, and an autonomous driving control. When the driver has fallen into the inappropriate state, the control unit performs the traveling control that is selected based on a surrounding of the vehicle from among the above three controls, and causes the notification device to perform the surround notification in a mode that varies depending on the traveling control selected to be performed.

Systems and methods are provided for detecting road user frustration due to autonomous vehicle driving behavior. An autonomous vehicle fleet can leverage information about typical road user behavior to create an index measuring the frustration level of other road users and/or likeability of autonomous vehicle behavior by other road users. Additionally, perception algorithms associated with sound detection and tracking can define human characteristics associated with frustrated drivers or other road users.