Techniques for a perception system of a vehicle that can detect and track objects in an environment are described herein. The perception system may include a machine-learned model that includes one or more different portions, such as different components, subprocesses, or the like. In some instances, the techniques may include training the machine-learned model end-to-end such that outputs of a first portion of the machine-learned model are tailored for use as inputs to another portion of the machine-learned model. Additionally, or alternatively, the perception system described herein may utilize temporal data to track objects in the environment of the vehicle and associate tracking data with specific objects in the environment detected by the machine-learned model. That is, the architecture of the machine-learned model may include both a detection portion and a tracking portion in the same loop.

A long-baseline and long depth-range stereo vision system is provided that is suitable for use in non-rigid assemblies where relative motion between two or more cameras of the system does not degrade estimates of a depth map. The stereo vision system may include a processor that tracks camera parameters as a function of time to rectify images from the cameras even during fast and slow perturbations to camera positions. Factory calibration of the system is not needed, and manual calibration during regular operation is not needed, thus simplifying manufacturing of the system.

Methods, systems, and apparatuses, including computer programs encoded on computer storage media, for generating a prediction of at least a text and a particular type associated with an object are described in this specification. A first model output is generated by using a first machine learning model to process input data including one or more objects. The first model output identifies an existence of a particular object in the input data and specifies characteristics of the particular object. A type of the particular object is determined based on the specified characteristics. The type comprises a single-row type and a multi-row type. A single-row representation of the particular object is generated. A second model output is generated by processing the single-row representation. The second model output comprises a prediction of characters corresponding to the particular vehicle license plate.

The present disclosure relates to navigation and to systems and methods for using a dual sensor readout channel to allow for frequency detection. In one implementation, at least one processing device may receive a plurality of images acquired by a camera onboard a host vehicle, wherein the plurality of images are received via a first channel and via a second channel, and wherein the first channel is associated with a first frame capture rate, and the second channel is associated with a second frame capture rate different from the first frame capture rate. The processing device may use images received via the first channel to detect flickering and non-flickering light sources in an environment of the host vehicle; and provide, based on images received via the second channel, images for showing on one or more human-viewable displays.

Provided herein is a system and method for a vehicle system on a vehicle. The system comprises a server comprising sensor data of stop points, one or more processors, and a memory storing instructions that, when executed by the one or more processors, cause the system to perform: determining, from the stop points, one or more available stop points; selecting, from the one or more available stop points, a stop point based on a criteria; and stopping the vehicle at the selected stop point.

Provided herein is a system and method of a vehicle. The system comprises one or more sensors, processors, maps, and a memory storing instructions that, when executed by the one or more processors, causes the system to perform: monitoring a location of the vehicle while driving; detecting a sign while the vehicle is driving; capturing, frame-by-frame, data of the sign until the sign disappears from a field of view of the sensor; synchronizing each frame of the data with the location of the vehicle; determining a location of the sign based on the frame-by-frame data; in response to determining, at a frame immediately before the sign disappears from the field of view of the sensor, that the vehicle is driving towards the sign, uploading the detected sign and the location of the sign onto the one or more maps; and implementing a driving action based on the sign.

A traffic sign recognition device includes a storage device configured to store a camera image from a movable body and pieces of three-dimensional point group data, and a processor. The processor is configured to: estimate a relative position of the traffic sign candidate to the movable body; specify a set of three-dimensional point group data; and specify an image region of an object corresponding to a set region indicative of a region where the set is specified, the object including the traffic sign candidate. The processor is configured to calculate a percentage of a predetermined color component constituting a guide sign among color components constituting an image of the object. In a case where the percentage of the predetermined color component is equal to or more than a threshold, the processor recognizes the object including the traffic sign candidate to be the guide sign.

A device for identifying a signaling unit on a road on which a vehicle is traveling is described. The device is configured to determine an arrangement of one or more signal signs of the signaling unit on the basis of environmental data of one or more environmental sensors of the vehicle. The device is further configured to assign the one or more signal signs to one or more corresponding grid cells of a signal sign grid based on assignment logic.

A device for classifying image data includes a trainable pre-processing unit configured to retrieve, from a trained context, and based on the image data, at least one specification in terms of how a degree of granulation of the image data is to be reduced, and to reduce the degree of granulation of the image data in accordance with the at least one specification. The device further includes a trainable classifier configured to map the granulation-reduced image data onto an assignment to one or more classes of a specified classification.

The wrong-way driving determination method includes acquiring a backward image of the vehicle, recognizing a target included in the backward image, and determining the wrong-way driving of the vehicle based on the recognition information of the target. Here, determining the wrong-way driving includes determining whether or not the recognition information of the target includes the recognition information of the guide display surface of the traffic sign. When it is determined that the recognition information of the guide display surface is included in the recognition information of the target, it is determined that the vehicle is traveling in the wrong direction.