A controller for controlling an operation of a machine or a robot for handling an object by includes: a sensor signal acquisition unit that acquires a signal from a sensor that outputs three-dimensional point cloud data and a corresponding camera image of an imaging subject; a region acquisition unit that acquires a region having a height lower than a predetermined value; a non-object region setting unit that sets, as a non-object region, a region of the camera image corresponding to the acquired region; a complementary color setting unit that sets a color of the non-object region to a color complementary to a color of the object; an object position recognition unit that recognizes a position of the object relative to the imaging subject from the camera image; and an operation control unit that controls the operation of the machine or the robot using at least the recognized position.

A method and system are provided for controlling an industrial automation machine using non-contact (virtual) position encoding. The system and method can be used to determine the position of an object under assembly on a conveyor system without mechanical coupling to a line or drive mechanism of the line.

A grasping system includes a robotic arm having a gripper. A fixed sensor monitors a grasp area and an onboard sensor moves with the gripper also monitors the area. A controller receives information indicative of a position of an object to be grasped and operates the robotic arm to bring the gripper into a grasp position adjacent the object based on information provided by the fixed sensor. The controller is also programmed to operate the gripper to grasp the object in response to information provided by the first onboard sensor.

A gripper including: an orientation sensor configured to generate an orientation reading for a target object; a first grasping blade and a second grasping blade configured to secure the target object in conjunction with the first grasping blade and at an opposite end of the target object relative to the first grasping blade; a first position sensor, of the first grasping blade, configured to generate a first position reading of the first grasping blade relative to the target object; a second position sensor, of the second grasping blade, configured to generate a second position reading of the second grasping blade relative to the target object; and a blade actuator configured to secure the target object with the first grasping blade and the second grasping blade based on a valid orientation of the orientation reading and based on the first position reading and the second position reading indicating a stable condition.

A robot device according to various embodiments comprises a camera, a robot arm, and a control device electrically connected to the camera and the robot arm, wherein the control device can be configured to collect a robot arm control record about a random operation, acquire, from the camera, a camera image in which a working space of the robot arm is photographed, implement an augmented reality model by rendering a virtual object corresponding to an object related to objective work in the camera image, and update a control policy for the objective work by performing image-based policy learning on the basis of the augmented reality model and the control record.

A system and method for operating a robotic system to register unrecognized objects is disclosed. The robotic system may use first image data representative of an unrecognized object located at a start location to derive an initial minimum viable region (MVR) and to implement operations for initially displacing the unrecognized object. The robotic system may analyze second image data representative of the unrecognized object after the initial displacement operations to detect a condition representative of an accuracy of the initial MVR. The robotic system may register the initial MVR or an adjustment thereof based on the detected condition.

Provided is a machine-learning device which can efficiently perform machine learning. The machine-learning device comprises: a vision execution unit which captures an image of an object W by means of a visual sensor by executing a vision execution command from a robot program, and detects or determines the object W from the captured image; a result acquisition unit which acquires the detection result or the determination result for the object W by executing a result acquisition command from the robot program; an additional annotation unit which gives a label to the captured image on the basis of the detection result or the determination result for the image of the object W by executing an annotation command from the robot program, and acquires new training data; and a learning unit which performs machine learning by using the new training data by executing a learning command from the robot program.

The present invention easily generates training data needed for generating a training model for identifying a position for retrieving a bulk-loaded work piece. An information processing device, according to the present invention, for processing information for retrieving a work piece using a hand, said device comprising: a reception unit that receives retrieval conditions including information about the hand or the work piece; a pre-processing unit that at least derives the position of the center of gravity of the work piece on the basis of a 3D CAD model of the work piece; and a first processing unit that, on the basis of the derived position of the center of gravity of the work piece, derives sectional features of the 3D CAD model of the work piece according to the retrieval conditions.

A control device sets a relative relation amount between a plurality of target objects that are to be final objectives, repeatedly acquires observation data from a sensor and calculates the relative relation amount between the plurality of target objects existing in the environment from the acquired observation data. Further, the control device determines a series of the relative relation amounts in a state that is to be an objective from the relative relation amount at a time point at which control of the behavior starts until the relative relation amount as the final objectives is realized, and repeatedly determines control instructions so as to change the relative relation amount in a present state calculated from the latest observation data into the relative relation amount in a state of an objective to be transitioned to next. Then, the control device outputs the determined control instruction to a robot device.

A method for training a machine learning model to derive a movement vector for a robot from image data. The method includes acquiring images from a perspective of an end-effector of the robot, forming training image data elements from the acquired images, generating augmentations of the training image data elements, training an encoder network using contrastive loss and training a neural network to reduce a loss between movement vectors output by the neural network in response to embedding outputs provided by the encoder network and respective ground truth movement vectors.