Vision systems for robotic assemblies for handling cargo, for example, unloading cargo from a trailer, can determine the position of cargo based on shading topography. Shading topography imaging can be performed by using light sources arranged at different positions relative to the image capture device(s).

Asynchronous robotic control utilizing a trained critic network. During performance of a robotic task based on a sequence of robotic actions determined utilizing the critic network, a corresponding next robotic action of the sequence is determined while a corresponding previous robotic action of the sequence is still being implemented. Optionally, the next robotic action can be fully determined and/or can begin to be implemented before implementation of the previous robotic action is completed. In determining the next robotic action, most recently selected robotic action data is processed using the critic network, where such data conveys information about the previous robotic action that is still being implemented. Some implementations additionally or alternatively relate to determining when to implement a robotic action that is determined in an asynchronous manner.

Method for controlling the operation of an industrial robot configured in particular to carry out pick-and-place or singulation tasks.

This disclosure provides systems, methods, and apparatuses, including computer programs encoded on computer storage media, that provide for training, implementing, or updated machine learning logic, such as an artificial neural network, to model a manufacturing process performed in a manufacturing robot environment. For example, the machine learning logic may be trained and implemented to learn from or make adjustments based on one or more operational characteristics associated with the manufacturing robot environment. As another example, the machine learning logic, such as a trained neural network, may be implemented in a semi-autonomous or autonomous manufacturing robot environment to model a manufacturing process and to generate a manufacturing result. As another example, the machine learning logic, such as the trained neural network, may be updated based on data that is captured and associated with a manufacturing result. Other aspects and features are also claimed and described.

Techniques are disclosed that enable the generation of predicted sequences of terminals using a generator model portion of a prediction model. Various implementations include controlling actuators of a robot based on the predicted sequences of terminals. Additional or alternative implementations include jointly training the generator model portion of the prediction model using a discriminator model portion of the prediction model using, for example, stochastic adversarial based sampling.

A computer-implemented method, executed by data processing hardware of a robot, includes receiving a three-dimensional point cloud of sensor data for a space within an environment about the robot. The method includes receiving a selection input indicating a user-selection of a target object represented in an image corresponding to the space. The target object is for grasping by an end-effector of a robotic manipulator of the robot. The method includes generating a grasp region for the end-effector of the robotic manipulator by projecting a plurality of rays from the selected target object of the image onto the three-dimensional point cloud of sensor data. The method includes determining a grasp geometry for the robotic manipulator to grasp the target object within the grasp region. The method includes instructing the end-effector of the robotic manipulator to grasp the target object within the grasp region based on the grasp geometry.

Apparatuses, method and computer program products for automatically performing sorting operations are disclosed herein. An example apparatus may comprise: an array of gripping elements, and at least one processing component configured to: obtain image data corresponding with the plurality of items; identify, from the image data, one or more characteristics of the plurality of items; determine, based at least in part on the one or more characteristics, an ordered sequence corresponding with the plurality of items; and generate a control indication to cause at least one of the gripping elements to perform the sorting operations based at least in part on the ordered sequence.

A gripping system includes a hand that grips a workpiece, a robot that supports the hand and changes at least one of a position and a posture of the hand, and an image sensor that acquires image information from a viewpoint interlocked with at least one of the position and the posture of the hand. Additionally, the gripping system includes a construction module that constructs a model by machine learning based on collection data. The model corresponds to at least a part of a process of specifying an operation command of the robot based on the image information acquired by the image sensor and hand position information representing at least one of the position and the posture of the hand. An operation module executes the operation command of the robot based on the image information, the hand position information, and the model, and a robot control module operates the robot based on the operation command of the robot operated by the operation module.

A transporter network for determining robot actions based on sensor feedback can afford robots with efficient autonomous movement. The transporter network may exploit spatial symmetries and may not need assumptions of objectness to provide accurate instructions on object manipulation. The machine-learned model of the transporter network may also allow for learning various tasks with less training examples than other machinelearned models. The machine-learned model of the transporter network may intake observation data as input and may output actions in response to the processed observation data.

Deep machine learning methods and apparatus related to the manipulation of an object by an end effector of a robot are described herein. Some implementations relate to training an action prediction network to predict a probability density which can include candidate actions of successful grasps by the end effector given an input image. Some implementations are directed to utilization of an action prediction network to visually servo a grasping end effector of a robot to achieve a successful grasp of an object by the grasping end effector.