A system includes an image sensor having a plurality of pixels that form a plurality of regions of interest (ROIs), and configured to operate at a frame rate higher than a threshold rate. The system also includes an image processing resource. The system further includes control circuitry configured to perform operations that include obtaining, from the image sensor, a full-resolution image of an environment. The full-resolution image contains each respective ROI of the plurality of ROIs. The operations also include selecting a particular ROI based on the full-resolution image, and detecting an object of interest in the particular ROI. The operations include determining a mode of operation by which subsequent image data generated by the particular ROI is to be processed. The operations further include processing, based on the mode of operation and the frame rate, the image data comprising a plurality of ROI images of the object of interest.

Provided are methods for prediction error scenario mining for machine learning methods, which can include determining a prediction error indicative of a difference between a planned decision of an autonomous vehicle and an ideal decision of the autonomous vehicle. The prediction error is associated with an error-prone scenario for which a machine learning model of an autonomous vehicle is to make planned movements. The method includes searching a scenario database for the error-prone scenario based on the prediction error. The scenario database includes a plurality of datasets representative of data received from an autonomous vehicle sensor system in which the plurality of datasets is marked with at least one attribute of the set of attributes. The method further includes obtaining the error-prone scenario from the scenario database for inputting into the machine learning model for training the machine learning model. Systems and computer program products are also provided.

Disclosed are apparatuses, systems, and techniques that may perform methods of pyramid optical flow processing with efficient identification and handling of object boundary pixels. In pyramid optical flow, motion vectors for pixels of image layers having a coarse resolution may be used as hints for identification of motion vectors for pixels of image layers having a higher resolution. Pixels that are located near apparent boundaries between foreground and background objects may receive multiple hints from lower-resolution image layers, for more accurate identification of matching pixels across different image levels of the pyramid.

A method for object detection obtains, from a set of RGB images lacking annotations, a set of regions that include potential objects, a bounding box, and an objectness score indicating a region prediction confidence. The method obtains, by a region scorer for each region in the set, a category from a fixed set of categories and a confidence for the category responsive to the objectness score. The method duplicates each region in the set to obtain a first and a second patch. The method encodes the patches to obtain an image vector. The method encodes a template sentence using the category to obtain a text vector for each category. The method compares the image vector to the text vector via a similarity function to obtain a similarity probability based on the confidence. The method defines a final set of pseudo labels based on the similarity probability being above a threshold.

Systems and techniques are described for implementing autonomous control of earth-moving construction and/or mining vehicles, including to automatically determine and control autonomous movement (e.g., of a vehicle's hydraulic arm(s), tool attachment(s), tracks/wheels, rotatable chassis, etc.) to move materials or perform other actions based at least in part on data about an environment around the vehicle(s). A perception system on a vehicle that includes at least a LiDAR component may be used to repeatedly map a surrounding environment and determine a 3D point cloud with 3D data points reflecting the surrounding ground and nearby objects, with the LiDAR component mounted on a component part of the vehicle that is moved independently of the vehicle chassis to gather additional data about the environment. GPS data from receivers on the vehicle may further be used to calculate absolute locations of the 3D data points.

An output of a first one of a plurality of layers within a neural network is identified. A bitmap is determined from the output, the bitmap including a binary matrix. A particular subset of operations for a second one of the plurality of layers is determined to be skipped based on the bitmap. Operations are performed for the second layer other than the particular subset of operations, while the particular subset of operations are skipped.

Systems, methods, tangible non-transitory computer-readable media, and devices for associating objects are provided. For example, the disclosed technology can receive sensor data associated with the detection of objects over time. An association dataset can be generated and can include information associated with object detections of the objects at a most recent time interval and object tracks of the objects at time intervals in the past. A subset of the association dataset including the object detections that satisfy some association subset criteria can be determined. Association scores for the object detections in the subset of the association dataset can be determined. Further, the object detections can be associated with the object tracks based on the association scores for each of the object detections in the subset of the association dataset that satisfy some association criteria.

A method for estimating a future fall of a cargo, the method may include receiving by a computerized system, sensed information related to driving sessions of multiple vehicles; applying a machine learning process on the sensed information to detect actual or estimated cargo falling events and generate one or more future falling cargo predictors for multiple types of cargo; estimating, from the sensed information, an impact of cargo falling events related to at least some of the types of cargo; and responding to the estimating, wherein the responding comprises at least one out of (a) storing the one or more future falling cargo predictors for the multiple types of cargo, (b) transmitting the one or more future falling cargo predictors for the multiple types of cargo; (c) storing the estimated impact of cargo falling events related to the at least some of the types of cargo, and (d) transmitting the impact of cargo falling events related to the at least some of the types of cargo.

The trajectory tracking method for a mobile electronic device may include tracking a trajectory of the electronic device by using results of pose estimation using odometry and results of pose estimation using visual localization (VL) as camera pose information.

The present disclosure relates to a data construction and learning system and method based on a method of splitting and arranging multiple images. The data construction and learning system based on a method of splitting and arranging multiple images includes an input unit configured to receive images captured by a plurality of cameras disposed in a vehicle, a memory in which a program for merging the images into a single image and estimating information on a road situation and an object has been stored, and a processor configured to execute the program. The processor merges and recognizes, as one situation, road situations and objects redundantly included in the images.