A picking facility is realized that can shorten the time required to transfer an article from a first support body to a second support body. Of a plurality of articles 50 supported by the first support body 51, the article 50 located at the highest position and the article 50 whose upper face T1 is present in a range of a set distance D downward from the upper face T1 of the article 50 located at the highest position are set as transfer-target articles 50A, and the control device performs a selection control to preferentially select, from the transfer-target articles 50A, a transfer-target article 50A in the normal orientation SC, and a transfer control to control the transfer device so as to transfer the transfer-target article 50A selected through the selection control from the first support body 51 to the second support body.

A robotic manipulation planning system, including at least one processor; and a non-transitory computer-readable storage medium including instructions that, when executed by the at least one processor, cause the at least one processor to: process perception data to detect known, familiar, and unknown objects to generate manipulation candidates; filter manipulation candidates against constraints to reduce the manipulation candidates; and determine quality metrics for the reduced manipulation candidates using a soft-body simulation technique.

Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.

Techniques described herein relate to generating a posteriori knowledge about where objects are typically located within environments to improve object location. In various implementations, output from vision sensor(s) of a robot may include visual frame(s) that capture at least a portion of an environment in which a robot operates/will operate. The visual frame(s) may be applied as input across a machine learning model to generate output that identifies potential location(s) of an object of interest. The robot's position/pose may be altered based on the output to relocate one or more of the vision sensors. One or more subsequent visual frames that capture at least a not-previously-captured portion of the environment may be applied as input across the machine learning model to generate subsequent output identifying the object of interest. The robot may perform task(s) that relate to the object of interest.

A remote control system includes: an imaging unit that shoots an environment in which a device to be operated including an end effector is located; a recognition unit that recognizes objects that can be grasped by the end effector based on a shot image of the environment shot by the imaging unit; an operation terminal that displays the shot image and receive handwritten input information input to the displayed shot image; and an estimation unit that, based on the objects that can be grasped and the handwritten input information input to the shot image, estimates an object to be grasped which has been requested to be grasped by the end effector from among the objects that can be grasped and estimates a way of performing a grasping motion by the end effector, the grasping motion having been requested to be performed with regard to the object to be grasped.

A method for controlling a robotic arm to grasp and place objects includes acquiring a plurality of sets of images each of which including an RGB image and a depth image. The RGB image and the depth image of each set are associated with each other. A plurality of fused images are obtained by fusing of depth information on each RGB image based on depth information of corresponding depth image. Once a three-dimensional map is constructed based on the plurality of fused images, a robotic arm is controlled to grasp and place objects based on the three-dimensional map.

A method for operating a robotic system including determining an initial pose of a target object based on imaging data; calculating a confidence measure associated with an accuracy of the initial pose; and determining that the confidence measure fails to satisfy a sufficiency condition; and deriving a motion plan accordingly for scanning an object identifier while transferring the target object from a start location to a task location.

A method includes steps of: capturing an image of a container; recognizing at least one object in the container based on the image; determining at least one first coordinate set corresponding to the at least one object; determining at least one second coordinate set that corresponds to target one (s) of the at least one first coordinate set and that relates to a fixed picking device of a robotic arm; adjusting position(s) of unfixed picking device(s) of the robotic arm if necessary; controlling the robotic arm to pick up one (s) of the at least one object that correspond(s) to the at least one second coordinate set with the fixed. picking device and/or at least one unfixed picking device.

Robots, robot systems, and methods for operating the same based on environment models including haptic data are described. An environment model which includes representations of objects in an environment is accessed, and a robot system is controlled based on the environment model. The environment model incudes haptic data, which provides more effective control of the robot. The environment model is populated based on visual profiles, haptic profiles, and/or other data profiles for objects or features retrieved from respective databases. Identification of objects or features can be based on cross-referencing between visual and haptic profiles, to populate the environment model with data not directly collected by a robot which is populating the model, or data not directly collected from the actual objects or features in the environment.

The present invention relates to a picking system for order-related picking of goods stockpiled in racks, with an automated picking cell including a gripper for gripping and picking the goods and a cell control unit for actuating the gripper, with a transport system for conveying the racks to a picking position near the picking cell, wherein the racks have shelves and/or containers on and/or in which goods are stored, wherein at least one shelf and/or container of a rack arranged in the picking position can be conveyed from a storage position inside the rack into a removal position in which the at least one shelf and/or container is at least partially arranged in front of the rack, wherein the gripper is actuated by the cell control unit to grip the goods stockpiled on the shelf and/or in the container while the shelf and/or container is in the removal position.