A method and apparatus are provided for optimizing one or more object detection parameters used by an autonomous vehicle to detect objects in images. The autonomous vehicle may capture the images using one or more sensors. The autonomous vehicle may then determine object labels and their corresponding object label parameters for the detected objects. The captured images and the object label parameters may be communicated to an object identification server. The object identification server may request that one or more reviewers identify objects in the captured images. The object identification server may then compare the identification of objects by reviewers with the identification of objects by the autonomous vehicle. Depending on the results of the comparison, the object identification server may recommend or perform the optimization of one or more of the object detection parameters.

An image processing unit identifies the shape of an obstacle that is identified from an area that appears in a peripheral image based on an image captured by a camera. The shape of the obstacle includes at least a tilt of a section of the obstacle in a road-surface direction. The section of the obstacle faces a vehicle. The image processing unit generates a superimposed image in which a mark image that is generated as a pattern that indicates the identified obstacle is superimposed onto a position that corresponds to the obstacle in the peripheral image. At this time, the image processing unit variably changes properties of the mark image based on the tilt of the obstacle identified by an obstacle identifying unit. The image processing unit then displays the generated superimposed image on display apparatus.

A method estimates a camera pose change estimation. The method includes capturing a first image of a scene with a first camera, obtaining a depth map with respect to the first camera based on the first image, capturing a second image of the scene with a second camera. The method also includes obtaining a pose change from the first camera pose to the second camera pose based on the first image and the second image, generating a set of additional pose changes based on the pose change, obtaining a set of reconstructed images and, matching each reconstructed image of the set of reconstructed images with the second image. The method selects a camera pose change estimation from the pose change and the set of additional pose changes that corresponds to a best matching reconstructed image.

In various examples, a multi-sensor fusion machine learning model – such as a deep neural network (DNN) – may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.

A system includes a mobile robot, the robot comprising a sensor; and a server operably connected to the robot over a network, the robot being configured to detect an object by processing sensor data using a convolutional neural network. A pipeline for robotic object detection using a convolutional neural network includes: a system comprising a mobile robot, the robot comprising a sensor, the system further comprising a server operably connected to the robot over a network, the robot being configured to detect an object by processing sensor data using a pipeline, the pipeline comprising a convolutional neural network, the pipeline configured to perform a data collection step, the pipeline further configured to perform a data transformation step, the pipeline further configured to perform a convolutional neural network step, the pipeline further configured to perform a network output transformation step, the pipeline further configured to perform a results output step.

A method of controlling a mobile work machine on a worksite includes receiving an indication of an object detected on the worksite, determining a location of the object relative to the mobile work machine, receiving an image of the worksite, correlating the determined location of the object to a portion of the image, evaluating the object by performing image processing of the portion of the image, and generating a control signal that controls the mobile work machine based on the evaluation.

A method for detecting a drivable area includes: collecting N consecutive video frames of a road when a vehicle is driving, where N is a positive integer greater than 1; determining a historical trajectory of a dynamic obstacle and position information of a static obstacle included in the N consecutive video frames by analyzing the N consecutive video frames with a 3D detection algorithm; correcting the historical trajectory and the position information based on a preset rule; determining a predicted trajectory of the dynamic obstacle based on the corrected historical trajectory; and determining the drivable area of the vehicle based on the predicted trajectory and the corrected position information.

Techniques are described for controlling an autonomous vehicle such as an unmanned aerial vehicle (UAV) using objective-based inputs. In an embodiment, the underlying functionality of an autonomous navigation system is exposed via an application programming interface (API) allowing the UAV to be controlled through specifying a behavioral objective, for example, using a call to the API to set parameters for the behavioral objective. The autonomous navigation system can then incorporate perception inputs such as sensor data from sensors mounted to the UAV and the set parameters using a multi-objective motion planning process to generate a proposed trajectory that most closely satisfies the behavioral objective in view of certain constraints. In some embodiments, developers can utilize the API to build customized applications for the UAV. Such applications, also referred to as “skills,” can be developed, shared, and executed to control behavior of an autonomous UAV and aid in overall system improvement.

A vehicle can use an image sensor to both detect objects and determine depth data associated with the environment the vehicle is traversing. The vehicle can capture image data and lidar data using the various sensors. The image data can be provided to a machine-learned model trained to output depth data of an environment. Such models may be trained, for example, by using lidar data and/or three-dimensional map data associated with a region in which training images and/or lidar data were captured as ground truth data. The autonomous vehicle can further process the depth data and generate additional data including localization data, three-dimensional bounding boxes, and relative depth data and use the depth data and/or the additional data to autonomously traverse the environment, provide calibration/validation for vehicle sensors, and the like.

Distance measurements are received from one or more distance measurement sensors, which may be coupled to a vehicle. A three-dimensional (3D) point cloud are generated based on the distance measurements. In some cases, 3D point clouds corresponding to distance measurements from different distance measurement sensors may be combined into one 3D point cloud. A voxelized model is generated based on the 3D point cloud. An object may be detected within the voxelized model, and in some cases may be classified by object type. If the distance measurement sensors are coupled to a vehicle, the vehicle may avoid the detected object.