A method to control operation of a robot includes generating at least one virtual image by an optical 3D measurement system and with respect to a 3D measurement coordinate system, the at least one virtual image capturing a surface region of a component. The method further includes converting a plurality of point coordinates of the virtual image into point coordinates with respect to a robot coordinate system by a transformation instruction and controlling a tool element of the robot using the point coordinates with respect to the robot coordinate system so as to implement the operation.

[Object] To calibrate an internal model more efficiently and more precisely. [Solution] Provided is a robot arm apparatus including: an arm unit made up of a plurality of links joined by one or a plurality of a joint unit, the arm unit being connectable to an imaging unit. An internal model including at least geometric information about the arm unit and focus position information about the imaging unit is updated using internal model information acquired in a state in which the imaging unit is pointed at a reference point in real space.

A controller is provided for performing dynamic grasping of a target object using visual sensory inputs. The controller includes a robotic interface connected to a robotic arm including links connected by joints having actuators and encoders, and a gripper of the end-effector of the robotic arm configured to grasp the target object in response to robot control signals, and a vision sensor configured to continuously provide visual observations for tracking poses of the target object in a workspace and compute grasp poses, wherein the vision sensor is mounted on a distal end of the robotic arm adjacent to the gripper. The controller trains the Eye-on-Hand reinforcement learner policy, tracks the poses of the target object, and generates robot control signals to follow the target object while keeping it in the field of view of the vision sensor and grasp the target object in the workspace.

A robot for interacting with a target object includes a robotic manipulator, a calibrating image, a camera and a processor. The robotic manipulator corresponds to a robotic manipulator coordinate. The calibrating image is disposed on the robotic manipulator. The camera corresponds to a camera coordinate and for shooting the target object and generating a picture. The processor is configured to move the robotic manipulator such that the calibrating image moves towards the target object and enters the picture. The processor records robotic manipulator coordinate datasets and camera coordinate datasets of the calibrating image as the calibrating image moving towards the target object, and uses the robotic manipulator coordinate datasets and the camera coordinate datasets to execute a hand-eye calibrating algorithm to obtain a calibrated mapping between the camera coordinate and the robotic manipulator coordinate.

A calibration device performs calibration based on detection results of an object from a sensor. The calibration device includes a determiner that determines, based on a size of a field of view of the sensor and a size of the object, a range in which a posture of a robotic arm with the object attached or with the sensor attached to detect the object is changed, an obtainer that repeatedly obtains a combination of information about the posture of the robotic arm and a detection result of the object from the sensor while the posture of the robotic arm is being changed within the range determined by the determiner to obtain a plurality of the combinations, and a calibrator that performs, based on the plurality of combinations obtained by the obtainer, calibration to determine correspondence between the postures of the robotic arm and the detection results of the object.

This invention provides a system and method for calibration of a 3D vision system using a multi-layer 3D calibration target that removes the requirement of accurate pre-calibration of the target. The system and method acquires images of the multi-layer 3D calibration target at different spatial locations and at different times, and computes the orientation difference of the 3D calibration target between the two acquisitions. The technique can be used to perform vision-based single-plane orientation repeatability inspection and monitoring. By applying this technique to an assembly working plane, vision-based assembly working plane orientation repeatability, inspection and monitoring can occur. Combined with a moving robot end effector, this technique provides vision-based robot end-effector orientation repeatability inspection and monitoring. Vision-guided adjustment of two planes to achieve parallelism can be achieved. The system and method operates to perform precise vision-guided robot setup to achieve parallelism of the robot's end-effector and the assembly working plane.

A robot system includes a robot, a controller, and a sensor. The robot 1) comprises links connected to one another by joints. The sensor comprises an optical sensor portion arranged to sense displacement of one of the links and an evaluation portion for deriving velocity information from image data output by the sensor portion. The controller calculates an expected position of a reference point on one of the links based on rotation angles associated with the joints and on a vector of intrinsic parameters of the robot, controls movement of the reference point along a predetermined path so that the expected position varies with a predetermined velocity, compares a velocity measured by the sensor during movement to the predetermined velocity, and minimizes a deviation between the measured and predetermined velocities by varying the vector of intrinsic parameters.

A system includes a machine tool 10, a robot 25 having a camera 31, and a transportation device 35 having the robot 25 mounted thereon, and an identification figure is arranged in a machining area of the machine tool 10.

A robot arrangement includes at least a movable robot arm arrangement configured to move within a workspace defining a 3D motion range; a sensor arrangement installed on the robot arm arrangement comprising at least a first sensor device providing at least a first field of view; wherein the sensor arrangement is configured to be moved in such a way that the at least first field of view of the sensor arrangement covers at least a first portion of the workspace defining a first region of movement of the robot arm arrangement.

A system includes a machine tool 10, a robot 25 having a camera 31, and a transportation device 35 having the robot 25 mounted thereon, and an identification figure is arranged in a machining area of the machine tool 10.