A robot system includes a robot, a vision sensor, a target position generation circuit, an estimation circuit, and a control circuit. The robot includes an end effector and is configured to work via the end effector on a workpiece which is disposed at a relative position and which is relatively movable with respect to the end effector. The vision sensor is configured to take an image of the workpiece. The target position generation circuit is configured to, based on the image, generate a target position of the end effector at every generation interval. The estimation circuit is configured to, at least based on relative position information related to the relative position, estimate a change amount in the relative position at every estimation interval. The control circuit is configured to control the robot to move the end effector based on the target position and the change amount.

Embodiments of the present disclosure provide methods for managing a robot system. In one method, orientations for links in the robot system may be obtained when the links are arranged in at least one posture, here each of the orientations indicates a direction pointed by one of the links. At least one image of an object placed in the robot system may be obtained from a vision device equipped on one of the links. Based on the orientations and the at least one image, a first mapping may be determined between a vision coordinate system of the vision device and a link coordination system of the link. Further, embodiments of present disclosure provide apparatuses, systems, and computer readable media for managing a robot system. The vision device may be calibrated by the first mapping and may be used to manage operations of the robot system.

A system and method for determining performance of a robot. In one form the robot is constructed as you assembling automotive workpieces onto an automobile assembly. In one form the robot accomplishes the task of assembling an automotive workpiece onto the automotive assembly by using vision feedback and force feedback. The vision feedback can use any number of features perform its function. Such features can include an artificial feature such as but not limited to a QR code, as well as a natural feature such as a portion of the workpiece or automotive assembly. In one embodiment the robot is capable of detecting a collision event and assessing the severity of the collision event. In another embodiment the robot is capable of evaluating its performance by attracting a performance metric against a performance threshold, and comparing a sensor fusion output with a sensor fusion output reference.

According to one aspect of the present invention, disclosed is an automatic calibration method for a calibration device connected to a camera that is disposed the end effector of a robot and to a robot controller for controlling the robot. The method comprises the steps of: acquiring, from the camera and the robot controller, a robot-based coordinate system and an image of a marker marked in the work area of the robot (wherein the acquired image and robot-based coordinate system are recorded while the end effector is moved to a plurality of sample coordinates); and estimating the position of a robot coordinate system-based marker by using the acquired image and robot-based coordinate system.

The invention describes a generic framework for hand-eye calibration of camera-guided apparatuses, wherein the rigid 3D transformation between the apparatus and the camera must be determined. An example of such an apparatus is a camera-guided robot.

Disclosed herein is a device, system and method for determining error in robotic manipulator-to-camera calibration. The method includes detecting a test object by a camera coupled to a robotic manipulator. One or more test points are identified on the test object based on a CAD model and pre-defined contact points corresponding to the test object. Arm poses are determined for the robotic manipulator to reach the test points on the 3D test object by using current robotic manipulator-to-camera calibration. While driving an end effector of the robotic manipulator based on the arm poses, any contact of the end effector on the 3D test object is recorded upon receiving a feedback from the 3D test object. An error is determined in the current robotic manipulator-to-camera calibration based on current position of the end effector relative to the one or more test points on the 3D test object.

An automatic robotic arm system and a coordinating method for robotic arm and computer vision thereof are disclosed. A beam-splitting mirror splits an incident light into a visible light and a ranging light and respectively guides to an image capturing device and an optical ranging device arranged in the different reference axes. In a calibration mode, a transformation relation is computed based on a plurality of the calibration postures and corresponding calibration images. In an operation mode, a mechanical space coordinate is determined based on an operation image and the transformation relation, and the robotic arm is controlled to move based on the mechanical space coordinate.

The present invention addresses the problem of providing an image processing system that can accurately and efficiently detect a target object even when the positional relationship between an image capturing device and the target object differs between the time of teaching and the time of detection. An image processing system 1 includes: a control unit 52 that obtains, on the basis of position information of a robot 2 for specifying the position of a visual sensor 4 in a robot coordinate system and on the basis of position information showing the position of a target object W in an image coordinate system, the positional relationship between the visual sensor 4 and the target object W; and a storage unit 51 that stores, on the basis of a model pattern consisting of feature points extracted from a teaching image and on the basis of the positional relationship between the visual sensor 4 and the target object W when capturing the teaching image, the model pattern in the form of three-dimensional position information. The control unit 52 performs detection processing, in which the target object W is detected from a detection image on the basis of a result obtained by matching the model pattern with the feature points extracted from the detection image including the target object W.

A robot system is provided which can suitably perform robot movement correction. The robot system is provided with: a visual sensor which captures a first image of a target with the robot in a prescribed position and which captures a second image of the target with the robot in the position resulting from moving the robot a prescribed distance from the aforementioned prescribed position; a calibration data storage unit which stores calibration data that associates the robot coordinate system of the robot and the image coordinate system of the visual sensor; a first acquisition unit which, on the basis of the first image and the calibration data, acquires a first position of the target in the robot coordinate system; a second acquisition unit which, on the basis of the first image and the second image, acquires a second position of the target in the robot coordinate system; and a determination unit which determines whether or not the difference between the first position and the second position is within a prescribed range.

An image processing apparatus capable of simplifying operations for determining an image pickup posture of an image pickup apparatus attached to a robot. The image processing apparatus processes an image that an image pickup apparatus attached to a robot picks up. The image processing apparatus includes a memory device that stores a set of instructions, and at least one processor that executes the set of instructions to specify a working area of the robot based on teaching point information showing a plurality of designated teaching points, specify an image pickup area of the image pickup apparatus so as to include the specified working area; and determine an image pickup posture of the robot based on the specified image pickup area.