A handling apparatus has an arm having a joint; a holding portion attached to the arm and configured to hold an object; a sensor configured to detect a plurality of the objects; and a control apparatus configured to control the arm and the holding portion, wherein the control apparatus is configured to calculate an ease of holding the object by the holding portion as a score based on information acquired by the sensor with respect to each object and each holding method, select the object to hold and the holding method according to the score, and calculate a position for holding the selected object and an orientation of the arm.

A robot control device includes an obtaining unit that obtains, from an image sensor that captures a workpiece group to be handled by a robot, a captured image, a simulation unit that simulates operation of the robot, and a control unit that performs control such that the captured image is obtained if, in the simulation, the robot is retracted from an image capture forbidden space, in which an image is potentially captured with the workpiece group and the robot overlapping each other, and which is set based on either or both a first space being the visual field range of the image sensor, and a columnar second space obtained by taking a workpiece region including the workpiece group or each of divided regions into which the workpiece region is divided, as a bottom area, and extending the bottom area to the position of the image sensor.

According to an embodiment, there is provided a handling system capable of handling a plurality of objects, the handling system including: a movable arm, a holder, a sensor, and a controller. The holder is attached to the movable arm and is capable of holding the object. The sensor is capable of detecting the object. The controller controls the movable arm and the holder. The controller determines whether or not to change an arrangement of the object before the object is held, on the basis of information acquired from the sensor. In a case where it is determined to change the arrangement of the object, the controller evaluates effectiveness of an arrangement change operation for each object, and decides on an arrangement change operation on the basis of an evaluation result.

A system and method identifying an object, such as a transparent object, to be picked up by a robot from a bin of objects. The method includes obtaining a 2D red-green-blue (RGB) color image and a 2D depth map image of the objects using a 3D camera, where pixels in the depth map image are assigned a value identifying the distance from the camera to the objects. The method generates a segmentation image of the objects using a deep learning mask R-CNN (convolutional neural network) that performs an image segmentation process that extracts features from the RGB image and assigns a label to the pixels so that objects in the segmentation image have the same label. The method then identifies a location for picking up the object using the segmentation image and the depth map image.

A system and method for identifying an object to be picked up by a robot. The method includes obtaining a 2D red-green-blue (RGB) color image and a 2D depth map image of the objects using a 3D camera, where pixels in the depth map image are assigned a value identifying the distance from the camera to the objects. The method generates a segmentation image of the objects using a deep learning convolutional neural network that performs an image segmentation process that extracts features from the RGB image, assigns a label to the pixels so that objects in the segmentation image have the same label and rotates the object using the orientation of the object in the segmented image. The method then identifies a location for picking up the object using the segmentation image and the depth map image and rotates the object when it is picked up.

A method and computing system comprising identifying one or more candidate objects for selection by a robot. A path to the one or more candidate objects may be determined based upon, at least in part, a robotic environment and at least one robotic constraint. A feasibility of grasping a first candidate object of the one or more candidate objects may be validated. If the feasibility is validated, the robot may be controlled to physically select the first candidate object. If the feasibility is not validated, at least one of a different grasping point of the first candidate object, a second path, or a second candidate object may be selected.

A method for operating a picking robot comprising an end effector assembly and a vision assembly, and related controller device is disclosed, the method comprising picking a subject with the end effector assembly from a bin comprising a plurality of subjects; moving the subject to a delivery station; and releasing the subject on the delivery station, wherein the method comprises locking a joint connection of the end effector assembly prior to and/or during the act of moving the subject to the delivery station.

An information processing apparatus to output control information for controlling a suction device configured to suction a packaged object includes an input unit and an output unit. The input unit inputs an image obtained by performing image capturing of a surface of the packaged object. The output unit outputs the control information to control timing at which suction is started by the suction device, based on a state of the surface of the packaged object specified from the image.

A system for eliminating interference of randomly stacked workpieces is disclosed. The system includes a three-dimensional sensing module, a pick-up apparatus and a control module. The control module is coupled to the three-dimensional sensing module and the pick-up apparatus. The control module is configured to control the three-dimensional sensing module to capture a three-dimensional image, analyze the three-dimensional image to obtain an image information, select a target workpiece to be picked up according to the image information, arrange an interference elimination path for the target workpiece, and control the pick-up apparatus to eliminate interference of the target workpiece according to the interference elimination path.

An operation control device for a robot comprises: an input part inputting at least one operation candidate, and a captured image including an object to be processed; a first learning device that has finished learning performed according to first learning data to output a first evaluation value indicating evaluation of each operation candidate when the robot performs a first processing operation upon input of the captured image and the operation candidate; a second learning device that has finished learning performed according to second learning data which differs from the first learning data, to output a second evaluation value indicating evaluation of each operation candidate when the robot performs a second processing operation upon input of the captured image and the operation candidate; and an evaluation part that, based on at least one of the first evaluation value and the second evaluation value, calculates a command value.