A method for identifying objects by shape in close proximity to other objects of different shapes obtains point cloud information of multiple objects. The objects are arranged in at least two trays and the trays are stacked. A depth image of the objects is obtained according to the point cloud information, and the depth image of the objects is separated and layered to obtain a layer information of all the objects. An object identification system also disclosed. Three-dimensional machine vision is utilized in identifying the objects, improving the accuracy of object identification, and enabling the mechanical arm to accurately grasp the required object.

An estimation device includes a memory and at least one processor. The at least one processor is configured to acquire information regarding a target object. The at least one processor is configured to estimate information regarding a location and a posture of a gripper relating to where the gripper is able to grasp the target object. The estimation is based on an output of a neural model having as an input the information regarding the target object. The estimated information regarding the posture includes information capable of expressing a rotation angle around a plurality of axes.

An object manipulation apparatus according to an embodiment of the present disclosure includes a memory and a hardware processor coupled to the memory. The hardware processor is configured to: calculate, based on an image in which one or more objects to be grasped are contained, an evaluation value of a first behavior manner of grasping the one or more objects; generate information representing a second behavior manner based on the image and a plurality of evaluation values of the first behavior manner; and control actuation of grasping the object to be grasped in accordance with the information being generated.

Disclosed are various embodiments of a three-dimensional perception and object manipulation robot gripper configured for connection to and operation in conjunction with a robot arm. In some embodiments, the gripper comprises a palm, a plurality of motors or actuators operably connected to the palm, a mechanical manipulation system operably connected to the palm, a plurality of fingers operably connected to the motors or actuators and configured to manipulate one or more objects located within a workspace or target volume that can be accessed by the fingers. A depth camera system is also operably connected to the palm. One or more computing devices are operably connected to the depth camera and are configured and programmed to process images provided by the depth camera system to determine the location and orientation of the one or more objects within a workspace, and in accordance therewith, provide as outputs therefrom control signals or instructions configured to be employed by the motors or actuators to control movement and operation of the plurality of fingers so as to permit the fingers to manipulate the one or more objects located within the workspace or target volume. The gripper can also be configured to vary controllably at least one of a force, a torque, a stiffness, and a compliance applied by one or more of the plurality of fingers to the one or more objects.

Deep machine learning methods and apparatus, some of which are related to determining a grasp outcome prediction for a candidate grasp pose of an end effector of a robot. Some implementations are directed to training and utilization of both a geometry network and a grasp outcome prediction network. The trained geometry network can be utilized to generate, based on two-dimensional or two-and-a-half-dimensional image(s), geometry output(s) that are: geometry-aware, and that represent (e.g., high-dimensionally) three-dimensional features captured by the image(s). In some implementations, the geometry output(s) include at least an encoding that is generated based on a trained encoding neural network trained to generate encodings that represent three-dimensional features (e.g., shape). The trained grasp outcome prediction network can be utilized to generate, based on applying the geometry output(s) and additional data as input(s) to the network, a grasp outcome prediction for a candidate grasp pose.

A picking system is provided, which is capable of picking up an object even when the object is not registered in advance. The picking system includes: a picking device holding the object; an RGB-D camera acquiring three-dimensional point cloud data of the object to be picked up by the picking device; and a control device controlling the picking device based on a detection result by the RGB-D camera. The control device generates a geometric model of the object by combining simple geometric primitives while referring to the three-dimensional point cloud data, and calculates a holding position of the object for the picking device based on the geometric model.

A method and computing system comprising identifying one or more candidate objects for selection by a robot. A path to the one or more candidate objects may be determined based upon, at least in part, a robotic environment and at least one robotic constraint. A feasibility of grasping a first candidate object of the one or more candidate objects may be validated. If the feasibility is validated, the robot may be controlled to physically select the first candidate object. If the feasibility is not validated, at least one of a different grasping point of the first candidate object, a second path, or a second candidate object may be selected.

A method for operating a picking robot comprising an end effector assembly and a vision assembly, and related controller device is disclosed, the method comprising picking a subject with the end effector assembly from a bin comprising a plurality of subjects; moving the subject to a delivery station; and releasing the subject on the delivery station, wherein the method comprises locking a joint connection of the end effector assembly prior to and/or during the act of moving the subject to the delivery station.

A workpiece holding apparatus include holding means for attracting and holding each workpiece in turn from among workpieces placed in a 3D space; first information acquisition means for acquiring 3D information of workpieces; candidate calculation means for calculating, based on the acquired 3D information of the workpieces, candidate holding points, the candidate holding points being, when the holding means holds each of the workpieces in turn, candidates for a holding point of that workpiece; second information acquisition means for acquiring information about each of other workpieces present within a predetermined range from each of the candidate holding points on the workpieces; and control means for selecting one of the candidate holding points based on the information about the workpiece acquired by the second information acquisition means, and controlling the holding means so that the holding means holds the workpiece at the selected candidate holding point.

Systems, methods, and associated components for robotic manipulation of physical objects. The physical objects include three-dimensional gripping features configured to be detected by an optics system and gripped by an end-effector of a robotic arm with sufficient gripping force to move the physical objects against the force of gravity. Sets of the physical objects can have different sizes and shapes and, in some examples, include identically constructed three-dimensional gripping features.