Embodiments determine positioning of a mannequin. One such embodiment begins by determining a frame of a grasping element of a mannequin represented by a computer-aided design (CAD) model and determining a frame of an object to be grasped, where is object is also represented by a CAD model. To continue, degrees of freedom of the mannequin are specified and limits on the specified degrees of freedom are set. In turn, using an inverse kinematic solver, positioning of the mannequin grasping the object is determined based upon: (i) the determined frame of the grasping element, (ii) the determined frame of the object, (iii) the specified degrees of freedom, and (iv) the set limits on the specified degrees of freedom.

A robot system for processing an object to be packaged as a product, a packaging method, and a method of processing the same are provided. The object has multiple surfaces, and multiple e-package information tags are provided on the surfaces of the object for storing information of the product. Each surface is provided with one of the e-package information tags. The information of the product includes information of a location, an orientation and physical features of the object. In operation, the robot system controls a sensing device to detect and capture one of the e-package information tags on the object to obtain a captured image, and processes the captured image to obtain the information of the product. Based on the information of the product, the robot system controls a robotic grasping device to perform a robotic manipulation for handling the object.

A grasping system includes a robotic arm having a gripper. A fixed sensor monitors a grasp area and an onboard sensor moves with the gripper also monitors the area. A controller receives information indicative of a position of an object to be grasped and operates the robotic arm to bring the gripper into a grasp position adjacent the object based on information provided by the fixed sensor. The controller is also programmed to operate the gripper to grasp the object in response to information provided by the first onboard sensor.

Implementations are directed to training a machine learning model that, once trained, is used in performance of robotic grasping and/or other manipulation task(s) by a robot. The model can be trained using simulated training examples that are based on simulated data that is based on simulated robot(s) attempting simulated manipulations of various simulated objects. At least portions of the model can also be trained based on real training examples that are based on data from real-world physical robots attempting manipulations of various objects. The simulated training examples can be utilized to train the model to predict an output that can be utilized in a particular taskand the real training examples used to adapt at least a portion of the model to the real-world domain can be tailored to a distinct task. In some implementations, domain-adversarial similarity losses are determined during training, and utilized to regularize at least portion(s) of the model.

Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.

A hand for transferring a transfer target can be selected even when the combination of the transfer target and the hand is not taught. A machine learning device includes: state observation means for acquiring at least a portion of image data obtained by imaging a transfer target as input data; label acquisition means for acquiring information related to grasping means attached to a robot for transferring the transfer target as a label; and learning means for performing supervised learning using a set of the input data acquired by the state observation means and the label acquired by the label acquisition means as teacher data to construct a learning model that outputs information related to the grasping means appropriate for the transferring.

A method for operating a robotic system including determining an initial pose of a target object based on imaging data; calculating a confidence measure associated with an accuracy of the initial pose; and determining that the confidence measure fails to satisfy a sufficiency condition; and deriving a motion plan accordingly for scanning an object identifier while transferring the target object from a start location to a task location.

A robot operable within a 3-D volume includes a gripper movable between an open position and a closed position to grasp any one of a plurality of objects, an articulatable portion coupled to the gripper and operable to move the gripper to a desired position within the 3-D volume, and an object detection system operable to capture information indicative of the shape of a first object of the plurality of objects positioned to be grasped by the gripper. A computer is coupled to the object detection system. The computer is operable to identify a plurality of possible grasp locations on the first object and to generate a numerical parameter indicative of the desirability of each grasp location, wherein the numerical parameter is at least partially defined by the next task to be performed by the robot.

Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.

A grasping robot includes: a grasping mechanism configured to grasp a target object; an image-pickup unit configured to shoot a surrounding environment; an extraction unit configured to extract a graspable part that can be grasped by the grasping mechanism in the surrounding environment by using a learned model that uses an image acquired by the image-pickup unit as an input image; a position detection unit configured to detect a position of the graspable part; a recognition unit configured to recognize a state of the graspable part by referring to a lookup table that associates the position of the graspable part with a movable state thereof; and a grasping control unit configured to control the grasping mechanism so as to displace the graspable part in accordance with the state of the graspable part recognized by the recognition unit.