A robot control system and a method for automatic camera calibration is presented. The robot control system includes a control circuit configured to determine all corner locations of an imaginary cube that fits within a camera field of view, and determine a plurality of locations that are distributed on or throughout the imaginary cube. The control circuit is further configured to control a robot arm to move a calibration pattern to the plurality of locations, and to receive a plurality of calibration images corresponding to the plurality of locations, and to determine respective estimates of intrinsic camera parameters based on the plurality of calibration images, and to determine an estimate of a transformation function that describes a relationship between a camera coordinate system and a world coordinate system. The control circuit is further configured to control placement of the robot arm based on the estimate of the transformation function.

A robotic cell calibration method comprising a robotic cell system having elements comprising: one or more cameras, one or more sensors, components, and a robotic arm. The method comprises localizing positions of the one or more cameras and components relative to a position of the robotic arm using a common coordinate frame, moving the robotic arm in a movement pattern, and using the cameras and sensors to determine robotic arm position at multiple times during the movement. The method includes identifying a discrepancy in robotic arm position between a predicted position and the determined position in real time, and computing, by an auto-calibrator, a compensation for the identified discrepancy, the auto-calibrator solving for the elements in the robotic cell system as a system. The method includes modifying actions of the robotic arm in real time during the movement based on the compensation.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing planning for robotic placement tasks. One of the methods includes determining an initial in-hand state for a grasped object. A show pose for the grasped object is determined, and the object is moved to the show pose. A refined in-hand state for the grasped object is determined based on the show pose, and a placement plan is determined based on the refined in-hand state for the grasped object.

A calibration method, in a robot having a robot arm, of obtaining a position relationship between a first control point set for an end effector attached to a distal end of the robot arm and a second control point set on the distal end of the robot arm, includes a sixth step of calculating a second position relationship between a second reference position obtained from a position of the second control point in a third state and a position of the second control point in a fourth state and a first feature point in the fourth state, and a seventh step of calculating coordinates of the first feature point in a robot coordinate system based on a first position relationship and the second position relationship.

A hand-eye calibration system and method are provided. The system includes a robot on which a small pattern is mounted, a camera configured to photograph the robot, a memory, and a processor configured to move the robot, acquire posture information of the moved robot, acquire an image from the camera, move the camera after performing the robot movement, the posture information acquisition, and the image acquisition a first predetermined number of times, and perform hand-eye calibration for the robot based on the posture information and the images, which are obtained by repeatedly performing of the robot movement, the posture information acquisition, the image acquisition, and the camera movement.

A supplementary metrology position coordinates determination (SMPD) system is used with a robot. “Robot accuracy” (e.g., for controlling and sensing an end tool position of an end tool that is mounted proximate to a distal end of its movable arm configuration) is based on robot position sensors included in the robot. The SMPD system includes an imaging configuration and an XY scale and an alignment sensor for sensing alignment/misalignment therebetween, and an image triggering portion and processing portion. One of the XY scale or imaging configuration is coupled to the movable arm configuration and the other is coupled to a stationary element (e.g., a frame above the robot). The imaging configuration acquires an image of the XY scale with known alignment/misalignment, which is utilized to determine metrology position coordinates that are indicative of the end tool position, with an accuracy level that is better than the robot accuracy.

A method and system for determining poses for camera calibration is presented. The system determines a range of pattern orientations for performing the camera calibration, and determines a surface region on a surface of an imaginary sphere, which represents possible pattern orientations for the calibration pattern. The system determines a plurality of poses for the calibration pattern to adopt. The plurality of poses may be defined by respective combinations of a plurality of respective locations within the camera field of view and a plurality of respective sets of pose angle values. Each set of pose angle values of the plurality of respective sets may be based on a respective surface point selected from within the surface region on the surface of the imaginary sphere. The system outputs a plurality of robot movement commands based on the plurality of poses that are determined.

This disclosure relates to a spatial calibration method and apparatus of a robot ontology coordinate system based on a visual perception device and a storage medium. The method includes: obtaining first transformation relationships; obtaining second transformation relationships; using a transformation relationship between a visual perception coordinate system and an ontology coordinate system as an unknown variable; and resolving the unknown variable based on an equivalence relationship between a transformation relationship obtained according to the first transformation relationships and the unknown variable and a transformation relationship obtained according to the second transformation relationships and the unknown variable, to obtain the transformation relationship between the visual perception coordinate system and the ontology coordinate system.

A system and method for robustly calibrating a vision system and a robot is provided. The system and method enables a plurality of cameras to be calibrated into a robot base coordinate system to enable a machine vision/robot control system to accurately identify the location of objects of interest within robot base coordinates.

A vision-based manipulations system can be configured for, and can be operated with methods including, performing hand-eye calibrations at multiple workstations of the system, performing a cross-station calibration for the system, and determining relationships between the hand-eye calibrations and the cross-station calibration. In some embodiments, the system can be used to move a work object between the workstations based on the cross-station calibration.