A hand-eye calibration system and method are provided. The system includes a robot on which a small pattern is mounted, a camera configured to photograph the robot, a memory, and a processor configured to move the robot, acquire posture information of the moved robot, acquire an image from the camera, move the camera after performing the robot movement, the posture information acquisition, and the image acquisition a first predetermined number of times, and perform hand-eye calibration for the robot based on the posture information and the images, which are obtained by repeatedly performing of the robot movement, the posture information acquisition, the image acquisition, and the camera movement.

On the basis of a measured shape, which is a three-dimensional shape of a subject measured by means of a measuring unit using an image captured when an image capturing unit is disposed in a first position and a first attitude, a movement control unit determines a second position and a second attitude for capturing an image of the subject again, and sends an instruction to a movement mechanism. The three-dimensional shape is represented by means of height information from a reference surface. The movement control unit extracts, from the measured shape, a deficient region having deficient height information, and determines the second position and the second attitude on the basis of the height information around the deficient region of the measured shape. The position and attitude of the image capturing unit can be determined in such a way as to make it easy to eliminate the effects of shadows.

Various embodiments include a method for a robot to grab a 3D object. The method may include: determining a current position and attitude of a visual sensor of the robot relative to the 3D object; acquiring a grabbing template of the 3D object, the grabbing template comprising a specified grabbing position and attitude of the visual sensor relative to the 3D object; judging whether the grabbing template further comprises at least one reference grabbing position and attitude of the visual sensor relative to the 3D object, wherein the reference grabbing position and attitude is generated on the basis of the specified grabbing position and attitude; and based on a judgment result, using the grabbing template and the current position and attitude to generate a grabbing position and attitude of the robot.

A retrieval controller is disclosed for identifying an item to be retrieved from a flat storage surface by a robot for identifying one item, amongst items stored on a common surface that can be retrieved by a robot equipped with a lateral-motion gripper. The retrieval controller includes: a depth map computing unit configured to establish a global coordinate system, and establish an orthonormal set of basis vectors u, v and w defined in the global coordinate system, where w is approximately orthogonal to the surface that the items are stored on; a vector determination unit; an item selection unit; and a robot instructing unit configured to instruct the robot to retrieve the item based on uv coordinates of one or more w-aligned edges having associated quadrilateral-based, right prisms which do not have interiors that intersect any right, enclosing prisms selected by the item selection unit.

Robot assisted multi-view 3D scanning measurement based on path planning includes firstly, establishing a virtual simulation platform to complete the setting of measurement poses and measurement paths and perform the path evaluations of measurement paths. Then, completing the preliminary hand-eye calibration based on the properties of Kronecker product, and the preliminary hand-eye calibration is optimized by establishing a reprojection error cost function as the fitness function of the particle swarm optimization algorithm. Lastly, moving the robot to the measurement poses of the planned measurement paths, obtaining a single-view point cloud of the measured object and transforming it from the camera coordinate system to the robot base coordinate system to obtain a registered single-view point cloud based on the optimized hand-eye matrix. When registered single-view point clouds of all measurement poses are obtained, the points under the robot base coordinate system form a complete point cloud of the measured object.

This disclosure relates to a spatial calibration method and apparatus of a robot ontology coordinate system based on a visual perception device and a storage medium. The method includes: obtaining first transformation relationships; obtaining second transformation relationships; using a transformation relationship between a visual perception coordinate system and an ontology coordinate system as an unknown variable; and resolving the unknown variable based on an equivalence relationship between a transformation relationship obtained according to the first transformation relationships and the unknown variable and a transformation relationship obtained according to the second transformation relationships and the unknown variable, to obtain the transformation relationship between the visual perception coordinate system and the ontology coordinate system.

When a force sensor on a robot arm detects that the force of contact between an end of a calibration device and a calibration plate reaches a threshold, the robot arm stops, and the end of the calibration device performs marking at the contact position between the end of the calibration device and the calibration plate. The robot arm moves upward and stops at a position where the end of the robot arm is at a predetermined height. At this position, a camera at the end of the robot arm photographs marks on the calibration plate, records the coordinates of the marks in the camera coordinate system, and records the coordinates of the end of the calibration device in the robot coordinate system. A calibration transformation matrix is calculated according to the recorded coordinates of at least three marks.

A robot hand-eye calibration method includes: controlling a tail end of a robot arm to sequentially move to at least three respective positions above a calibration plate; controlling a laser provided on the robot arm, at each position, to project on the calibration plate; recording coordinates, in a robot coordinate system, of an end point of the tail end of the robot arm during projection; controlling a camera on the tail end of the robot arm to photograph the projection on the calibration plate; recording the coordinates of the projection in the camera coordinate system; and calculating a calibration transformation matrix, according to the coordinates recorded, of at least three projections on the calibration plate in the camera coordinate system and respective coordinates of the end point of the tail end of the robot arm in the robot coordinate system during each respective projection.

A system and method for robustly calibrating a vision system and a robot is provided. The system and method enables a plurality of cameras to be calibrated into a robot base coordinate system to enable a machine vision/robot control system to accurately identify the location of objects of interest within robot base coordinates.

A robot system includes a robot including a robot arm and a first camera, a second camera installed separately from the robot, and a control device which controls the robot and the second camera. The first camera has already been calibrated in advance, and a first calibration data which is the calibration data between the coordinate system of the robot and the coordinate system of the first camera is known. The control device (i) images a calibration pattern with the first camera to acquire a first pattern image and images a calibration pattern with the second camera to acquire a second pattern image, and (ii) executes calibration process for obtaining a second calibration data which is the calibration data between the coordinate system of the robot and the coordinate system of the second camera using the first pattern image, the second pattern image, and the first calibration data.