Techniques for determining predictions on a top-down representation of an environment based on object movement are discussed herein. Sensors of a first vehicle (such as an autonomous vehicle) may capture sensor data of an environment, which may include object(s) separate from the first vehicle (e.g., a vehicle, a pedestrian, a bicycle). A multi-channel image representing a top-down view of the object(s) and the environment may be generated based in part on the sensor data. Environmental data (object extents, velocities, lane positions, crosswalks, etc.) may also be encoded in the image. Multiple images may be generated representing the environment over time and input into a prediction system configured to output a trajectory template (e.g., general intent for future movement) and a predicted trajectory (e.g., more accurate predicted movement) associated with each object. The prediction system may include a machine learned model configured to output the trajectory template(s) and the predicted trajector(ies).

A road shape recognizer includes a peripheral information recognizer that recognizes at least two items of peripheral information based on an output of a periphery detector. A reliability assigner assigns a reliability level to each of the peripheral information. A point sequence generator generates and places a point sequence representing a shape of a road on which the own vehicle travels, based on at least two items of peripheral information and the reliability level. The point sequence generator generates and places a point sequence by generating and placing points one by one toward a distant place from a point located at a prescribed relative position to the own vehicle. The point sequence generator generates and places the next point corresponding to an amount of change in shape and a position of a point generated and placed at the end of the point sequence. The amount of change in shape is represented by the peripheral information and determined per section having a prescribed distance.

An information processing apparatus includes an image processor and a controller. The image processor performs recognition processing of recognizing attributes of predetermined regions that are respectively included in sequentially acquired images captured by a camera. The controller sets a frequency of performing the recognition processing for the predetermined region on the basis of the recognized attribute.

On-floor obstacle detection using an RGB-D camera is disclosed. An obstacle on a floor is detected by receiving an image including depth channel data and RGB channel data through the RGB-D camera, estimating a ground plane corresponding to the floor based on the depth channel data, obtaining a foreground of the image corresponding to the ground plane based on the depth channel data, performing a distribution modeling on the foreground of the image based on the RGB channel data to obtain a 2D location of the obstacle, and transforming the 2D location of the obstacle into a 3D location of the obstacle based on the depth channel data.

Systems, methods, tangible non-transitory computer-readable media, and devices for associating objects are provided. For example, the disclosed technology can receive sensor data associated with the detection of objects over time. An association dataset can be generated and can include information associated with object detections of the objects at a most recent time interval and object tracks of the objects at time intervals in the past. A subset of the association dataset including the object detections that satisfy some association subset criteria can be determined. Association scores for the object detections in the subset of the association dataset can be determined. Further, the object detections can be associated with the object tracks based on the association scores for each of the object detections in the subset of the association dataset that satisfy some association criteria.

Embodiments of the present disclosure relate to a method and apparatus for detecting ground point cloud points. The method may include: determining a segmentation plane and a ground based on a point cloud collected by a lidar; segmenting the point cloud into a first sub point cloud and a second sub point cloud based on the segmentation plane; and determining the point cloud points whose distances from the ground are smaller than a first distance threshold in the first sub point cloud as ground point cloud points, and determining the point cloud points whose distances from the ground are smaller than a second distance threshold in the second sub point cloud as the ground point cloud points, where the first distance threshold is smaller than the second distance threshold.

Embodiments of the present disclosure relate to a method and apparatus for outputting information. The method may include: acquiring point cloud data and image data collected by a vehicle during a driving process; determining a plurality of time thresholds based on a preset time threshold value range; executing following processing for each time threshold: identifying obstacles included in each point cloud frame and each image frame respectively; determining a similarity between the obstacles; determining, in response to the similarity being greater than a preset similarity threshold, whether a time interval between the point cloud frame and the image frame corresponding to two similar obstacles is less than the time threshold; and processing recognized obstacles based on a determining result, to determine the number of obstacles; and determining, based on a plurality of numbers, and outputting a target time threshold.

Provided are methods for prediction error scenario mining for machine learning methods, which can include determining a prediction error indicative of a difference between a planned decision of an autonomous vehicle and an ideal decision of the autonomous vehicle. The prediction error is associated with an error-prone scenario for which a machine learning model of an autonomous vehicle is to make planned movements. The method includes searching a scenario database for the error-prone scenario based on the prediction error. The scenario database includes a plurality of datasets representative of data received from an autonomous vehicle sensor system in which the plurality of datasets is marked with at least one attribute of the set of attributes. The method further includes obtaining the error-prone scenario from the scenario database for inputting into the machine learning model for training the machine learning model. Systems and computer program products are also provided.

System, methods, and other embodiments described herein relate to selecting a route for a vehicle to travel. In one embodiment, the detection system generates a driving maneuver recommendation for a vehicle having a plurality of sensors configured to acquire information about an environment around the vehicle, the sensors including at least a camera to capture one or more images of a scene within the environment, by determining that at least a portion of each image in a set of images captured by the camera indicates an external display in the environment, tracking an object within the portion of each image in the set of images to determine a state of the object, the state including at least a trajectory estimate for the object, and determining a recommended driving maneuver based at least in part on the determined state of the object.

A system and method generate feature space data that may be used for object detection. The system includes one or more processors and a memory. The memory may include one or more modules having instructions that, when executed by the one or more processors, cause the one or more processors to obtain a two-dimension image of a scene, generate an output depth map based on the two-dimension image of the scene, generate a pseudo-LIDAR point cloud based on the output depth map, generate a bird's eye view (BEV) feature space based on the pseudo-LIDAR point cloud, and modify the BEV feature space to generate an improved BEV feature space using feature space neural network that was trained by using a training LIDAR feature space as a ground truth based on a LIDAR point cloud.