A method is performed by a computing system. The method includes receiving image data from an imaging device, and determining, using the image data, a plurality of image-space tools, each image-space tool associated with a tool of a plurality of tools, each tool controlled by a manipulator of a plurality of manipulators. The method further includes determining a first correspondence between a first image-space tool of the plurality of image-space tools and a first tool of the plurality of tools based on a first disambiguation setting associated with the first tool.

The present disclosure provides a method for simultaneously performing a robot's kinematic calibration and hand-eye calibration. One exemplary method comprises acquiring a plurality of point clouds of a calibration fixture and formulating this simultaneous calibration problem as an optimization problem based on the collection of point clouds of the same fixture.

An automatic robotic arm system and a coordinating method for robotic arm and computer vision thereof are disclosed. A beam-splitting mirror splits an incident light into a visible light and a ranging light and respectively guides to an image capturing device and an optical ranging device arranged in the different reference axes. In a calibration mode, a transformation relation is computed based on a plurality of the calibration postures and corresponding calibration images. In an operation mode, a mechanical space coordinate is determined based on an operation image and the transformation relation, and the robotic arm is controlled to move based on the mechanical space coordinate.

Described are machine vision systems and methods for simultaneous kinematic and hand-eye calibration. A machine vision system includes a robot and a 3D sensor in communication with a control system. The control system is configured to move the robot to poses, and for each pose: capture a 3D image of calibration target features and robot joint angles. The control system is configured to obtain initial values for robot calibration parameters, and determine initial values for hand-eye calibration parameters based on the initial values for the robot calibration parameters, the 3D image, and joint angles. The control system is configured to determine final values for the hand-eye calibration parameters and robot calibration parameters by refining the hand-eye calibration parameters and robot calibration parameters to minimize a cost function.

A robot system includes a fixed camera that obtains first measurement data by detecting a plurality of features positioned within a detection range, the detection range including at least part of a range in which a robot arm is movable, a hand camera movable with the robot arm, and a control apparatus that controls the robot arm. A calibration function that relates a value obtained as part of the first measurement data to a command value provided to the robot arm at each of a plurality of positions and orientations at which the hand camera obtains second measurement data by detecting each mark.

A system and method for robustly calibrating a vision system and a robot is provided. The system and method enables a plurality of cameras to be calibrated into a robot base coordinate system to enable a machine vision/robot control system to accurately identify the location of objects of interest within robot base coordinates.

A hand-eye calibration method, apparatus, system, and storage medium for a robot are provided. The method includes: obtaining images of a target body of a calibration target acquired by an imaging device when the robot (110) is in different poses; for any pose, determining, based on at least one image collected in the pose, a first transformation relationship between a first coordinate system of the calibration target and a second coordinate system of the imaging device in the pose; obtaining a second transformation relationship between a third coordinate system and a fourth coordinate system of the robot (110); and determining, based on the first transformation relationships and the second transformation relationships corresponding to the different poses, a third transformation relationship between the second coordinate system and one of the third coordinate system and the fourth coordinate system. Further provided is a calibration target.

A gripping system includes a hand that grips a workpiece, a robot that supports the hand and changes at least one of a position and a posture of the hand, and an image sensor that acquires image information from a viewpoint interlocked with at least one of the position and the posture of the hand. Additionally, the gripping system includes a construction module that constructs a model by machine learning based on collection data. The model corresponds to at least a part of a process of specifying an operation command of the robot based on the image information acquired by the image sensor and hand position information representing at least one of the position and the posture of the hand. An operation module executes the operation command of the robot based on the image information, the hand position information, and the model, and a robot control module operates the robot based on the operation command of the robot operated by the operation module.

A robot system includes a robot including a robot arm and a first camera, a second camera installed separately from the robot, and a control device which controls the robot and the second camera. The first camera has already been calibrated in advance, and a first calibration data which is the calibration data between the coordinate system of the robot and the coordinate system of the first camera is known. The control device (i) images a calibration pattern with the first camera to acquire a first pattern image and images a calibration pattern with the second camera to acquire a second pattern image, and (ii) executes calibration process for obtaining a second calibration data which is the calibration data between the coordinate system of the robot and the coordinate system of the second camera using the first pattern image, the second pattern image, and the first calibration data.

A system and method for robustly calibrating a vision system and a robot is provided. The system and method enables a plurality of cameras to be calibrated into a robot base coordinate system to enable a machine vision/robot control system to accurately identify the location of objects of interest within robot base coordinates.