Multi-object tracking in autonomous vehicles uses both camera data and LiDAR data for training, but not LiDAR data at query time. Thus, no LiDAR sensor is on a piloted autonomous vehicle. Example systems and methods rely on camera 2D object detections alone, rather than 3D annotations. Example systems/methods utilize a single network that is given a camera image as input and can learn both object detection and dense depth in a multimodal regression setting, where the ground truth LiDAR data is used only at training time to compute depth regression loss. The network uses the camera image alone as input at test time (i.e., when deployed for piloting an autonomous vehicle) and can predict both object detections and dense depth of the scene. LiDAR is only used for data acquisition and is not required for drawing 3D annotations or for piloting the vehicle.

In various examples, an ego-machine may analyze sensor data to identify and track features in the sensor data using. Geometry of the tracked features may be used to analyze motion flow to determine whether the motion flow violates one or more geometrical constraints. As such, tracked features may be identified as dynamic features when the motion flow corresponding to the tracked features violates the one or more static constraints for static features. Tracked features that are determined to be dynamic features may be clustered together according to their location and feature track. Once features have been clustered together, the system may calculate a detection bounding shape for the clustered features. The bounding shape information may then be used by the ego-machine for path planning, control decisions, obstacle avoidance, and/or other operations.

An obstacle detection device includes: an ultrasonic sensor signal processing unit detecting at least a distance to an obstacle among the distance and a shape of the obstacle based on an output of an ultrasonic sensor, the ultrasonic sensor being disposed on a door of a vehicle and detecting the distance within a predetermined detection range by an ultrasonic wave; a camera image processing unit detecting the distance and the shape based on an output of a camera of the vehicle, the camera capturing an image of an area including a trajectory range when the door is opened and a detection range of the ultrasonic sensor; and a controller performing open operation control of the door based on detection results of the ultrasonic sensor signal processing unit and the camera image processing unit.

An autonomous mobile device moves through a physical space using simultaneous localization and mapping (SLAM) techniques. SLAM processes images from cameras to determine localization and trajectory of the device based on features that are assumed to be stationary. SLAM performance is improved by removing moving features from consideration. A first position of a feature at a first time and data from an inertial sensor are used to determine a predicted position at a second time. The predicted position is compared to a second position of the feature at the second time. This comparison takes into consideration an assumed Gaussian error distribution of how the positions are determined. If the predicted position differs from the second position by less than a threshold value, the feature may be determined to be stationary. The stationary features are then processed using SLAM to determine the localization and trajectory information.

An information processing apparatus is provided. The apparatus comprises an image acquisition unit that acquires time-series images by an imaging unit; and a processing unit that performs image processing, generates a plurality of shifted images obtained by shifting a reference image acquired before a target image among the time-series images in an up-down direction by a plurality of types of different shift amounts, and specifies a correction amount by a pitch of the imaging unit on the basis of a difference between each of the plurality of shifted images and the target image.

A drive assist apparatus includes a storage unit that stores a three-dimensional model indicating a moving region, an input unit that receives, from a sensor group installed in the moving region, first height information indicating a first height which is a height of the mobile object and second height information indicating a second height which is a height of an object that satisfies a predetermined distance criterion from the mobile object, an extraction unit that extracts, from the three-dimensional model, a first plan view based on the first height information and a second plan view based on the second height information, a generation unit that generates a combined map for two-dimensionally showing the moving region and assisting the driving of the mobile object by combining the first plan view and the second plan view, and an output unit that transmits a generated combined map to the mobile object.

A method for 3D object tracking is described. The method includes inferring first 2D semantic keypoints of a 3D object within a sparsely annotated video stream. The method also includes matching the first 2D semantic keypoints of a current frame with second 2D semantic keypoints in a next frame of the sparsely annotated video stream using embedded descriptors within the current frame and the next frame. The method further includes warping the first 2D semantic keypoints to the second 2D semantic keypoints to form warped 2D semantic keypoints in the next frame. The method also includes labeling a 3D bounding box in the next frame according to the warped 2D semantic keypoints in the next frame.

A movable body that moves automatically includes a plurality of sensors configured to detect an obstacle in a periphery of the movable body, a selection information acquisition unit configured to acquire selection information indicating a sensor, of the plurality of sensors, to be used for detection of the obstacle, the sensor being set based on a relationship between a reference position of the obstacle and a position of the movable body, and a detection control unit configured to cause the sensor indicated by the selection information to detect the obstacle, and identify a position of the obstacle from a detection result of the obstacle.

A device and method for preventing blind spot collision based on vehicle-to-vehicle communication. The method includes detecting a forward vehicle, requesting vehicle-to-vehicle communication with the forward vehicle, receiving image data and ultrasound data of the forward vehicle, analyzing information about a moving object in a blind spot formed by the forward vehicle, based on the image data and the ultrasound data, calculating a possibility of collision with the moving object based on the information about the moving object, and performing one or both of warning notification and collision avoidance control based on the possibility of collision.

Systems, methods, tangible non-transitory computer-readable media, and devices for autonomous vehicle operation are provided. For example, a computing system can receive object data that includes portions of sensor data. The computing system can determine, in a first stage of a multiple stage classification using hardware components, one or more first stage characteristics of the portions of sensor data based on a first machine-learned model. In a second stage of the multiple stage classification, the computing system can determine second stage characteristics of the portions of sensor data based on a second machine-learned model. The computing system can generate an object output based on the first stage characteristics and the second stage characteristics. The object output can include indications associated with detection of objects in the portions of sensor data.