A method and system for processing camera images is presented. The system receives a first depth map generated based on information sensed by a first type of depth-sensing camera, and receives a second depth map generated based on information sensed by a second type of depth-sensing camera. The first depth map includes a first set of pixels that indicate a first set of respective depth values. The second depth map includes a second set of pixels that indicate a second set of respective depth values. The system identifies a third set of pixels of the first depth map that correspond to the second set of pixels of the second depth map, identifies one or more empty pixels from the third set of pixels, and updates the first depth map by assigning to each empty pixel a respective depth value based on the second depth map.

Vision systems for robotic assemblies for handling cargo, for example, unloading cargo from a trailer, can determine the position of cargo based on shading topography. Shading topography imaging can be performed by using light sources arranged at different positions relative to the image capture device(s).

A robot apparatus includes: a robotic arm provided with a robotic hand capable of changing its position and its orientation by using joints; a visual sensor which measures a position or an orientation of a gripped object gripped with the robotic hand at a measurement teaching point; and a control device. The control device controls the position or the orientation of the gripped object when the gripped object is attached to an attachment target object at a corrected teaching point corrected based on a measurement result by the visual sensor. In this case, the control device determines a measurement teaching point, where the measurement with the visual sensor takes place, such that a driving direction of each of the joints from the measurement teaching point to the corrected teaching point is set to a definite driving direction.

A robot system is provided with a three-dimensional sensor which acquires three-dimensional information of an object, and a robot which includes a gripping device for gripping an object. The robot system uses first three-dimensional information which relates to a state before an object is taken out and second three-dimensional information which relates to a state after an object is taken out as the basis to acquire three-dimensional shape information of an object, and uses the three-dimensional shape information of the object as the basis to calculate a position and posture of the robot when an object is placed at a target site.

Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.

A method and system for piece-picking or piece put-away within a logistics facility. The system includes a central server and at least one mobile manipulation robot. The central server is configured to communicate with the robots to send and receive piece-picking data which includes a unique identification for each piece to be picked, a location within the logistics facility of the pieces to be picked, and a route for the robot to take within the logistics facility. The robots can then autonomously navigate and position themselves within the logistics facility by recognition of landmarks by at least one of a plurality of sensors. The sensors also provide signals related to detection, identification, and location of a piece to be picked or put-away, and processors on the robots analyze the sensor information to generate movements of a unique articulated arm and end effector on the robot to pick or put-away the piece.

An article pickup apparatus configured so as to measure surface positions of articles randomly piled on the three-dimensional space using a three-dimensional measurement instrument to acquire position information of three-dimensional points, determine a connected set made by connecting three-dimensional points present in the vicinity of each other among the three-dimensional points, and identify a position and posture of an article based on the position information of three-dimensional points belonging to the connected set. The posture of the article is identified by calculating a main component direction of the connected set by applying main component analysis to the three-dimensional points belonging to the connected set and identifying the posture of the article based on the main component direction.

The invention relates to an optimization method for improving the reliability of a collection and discharge of articles in a picking process using a robot. An article is collected from or out of a first article carrier and is placed down or thrown into or onto a second article carrier by means of a gripping unit on the robot head. In an image processing step, a gripping position for the gripping unit is calculated for collecting the article(s) by determining at least one dimension based on a captured image and by a range allocation being determined by means of a comparison with dimension ranges. Depending on a confidence value, a dimension value is determined, from stored article reference data or from a normalization value of the dimension range and allocated to the determined dimension. In a preparation step, a mathematical scattering measure function is applied for the determined dimension and for the dimension ranges, and normalization values and confidence values of the dimension ranges are determined therefrom.

Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.

Methods, apparatus, and computer readable media that are related to 3D object detection and pose determination and that may optionally increase the robustness and/or efficiency of the 3D object recognition and pose determination. Some implementations are generally directed to techniques for generating an object model of an object based on model point cloud data of the object. Some implementations of the present disclosure are additionally and/or alternatively directed to techniques for application of acquired 3D scene point cloud data to a stored object model of an object to detect the object and/or determine the pose of the object.