The controller communicable with a second controller include circuitry configured to: execute a processing to operate a first machine in collaboration with a second machine controlled by the second controller in a real space; modify the processing in response to determining that, instead of controlling the second machine, the second controller controls a virtual second machine that simulates operations of the second machine in a virtual space; and execute the modified processing to operate the first machine in the real space in collaboration with the virtual second machine that operates in the virtual space.

A method of controlling a robot within a volume, the method comprising: receiving a three dimensional model including a model of the robot and a model of the volume in which the robot is configured to move within; defining a plurality of positions within the model of the volume to which the robot is moveable to, the plurality of positions being identified by an operator; receiving scanned three dimensional data of the robot and at least a part of the volume; determining a transformation algorithm using the three dimensional model and the scanned three dimensional data; applying the transformation algorithm to one or more positions of the plurality of positions to provide one or more transformed positions; and controlling movement of the robot using one or more of the transformed positions.

In one aspect, there is provided a computer-implemented method that includes receiving a request to generate workcell data representing physical dimensions of a workcell having a physical robot arm, executing a calibration program that causes the physical robot arm to move within the workcell and record locations within the workcell at which the robot arm made contact with an object, generating, from the locations within the workcell at which one or more sensors of the robot arm recorded a resistance above a threshold, a representation of physical boundaries in the workcell, obtaining an initial virtual representation of the workcell, and updating the initial virtual representation of the workcell according to the representation of physical boundaries generated from executing the calibration program.

A teaching device constructs, in a virtual space, a virtual robot system in which a virtual 3D model of a robot and a virtual 3D model of a peripheral structure of the robot are arranged, and teaches a moving path of the robot. The teaching device includes an acquisition unit configured to acquire information about a geometric error between the virtual 3D models, and a correction unit configured to correct the moving path of the robot in accordance with the information acquired by the acquisition unit.

A control device estimates a position and pose of an imaging device relative to a robot based on an image of the robot captured by the imaging device. A simulation device arranges a robot model at a teaching point, and generates a simulation image of the robot model captured by a virtual camera that is arranged so that a position and pose of the virtual camera relative to the robot model in the virtual space coincide with the estimated position and pose of the imaging device. The control device determines an amount of correction of a position and pose of the robot for the teaching point so that the position and pose of the robot on the actual image captured after the robot has been driven according to a movement command to the teaching point approximate to the position and pose of the robot model on the simulation image.

Provided is an off-line simulation system that enables performance of efficient vision correction training. This vision correction training system for vision correction training is provided with a head mount display capable of displaying an image in a virtual space, and a teaching device communicably connected to the head mount display. The teaching device has: a vision correction unit that, on the basis of a captured image captured by a camera after the position of a workpiece has been moved, performs vision correction on a predetermined movement; and a correction confirmation unit that confirms that the vision correction is appropriately performed on the predetermined movement.

A teaching device constructs, in a virtual space, a virtual robot system in which a virtual 3D model of a robot and a virtual 3D model of a peripheral structure of the robot are arranged, and teaches a moving path of the robot. The teaching device includes an acquisition unit configured to acquire information about a geometric error between the virtual 3D models, and a correction unit configured to correct the moving path of the robot in accordance with the information acquired by the acquisition unit.

A system for parameter tuning for robotic manipulators is provided. The system includes an interface configured to receive a task specification, a plurality of physical parameters, and a plurality of control parameters, wherein the interface is configured to communicate with a real-world robot via a robot controller. The system further includes a memory to store computer-executable programs including a robot simulation module, a robot controller, and an auto-tuning module a processor, in connection with the memory. In this case, the processor is configured to acquire, in communication with the real-world robot, state values of the real-world robot, state values of the robot simulation module, simultaneously update, by use of a predetermined optimization algorithm with the auto-tuning module, an estimate of one or more of the physical, and said control parameters, and store the updated parameters.

A method of calibrating a system including a camera, the method including detecting a robot navigating within an environment modeled as a geo-polygon space, including a transit of the robot through a scene of the environment captured by the camera, mapping a plurality of points occupied by the robot in images of the scene to the geo-polygon space, recording data about the mapping, and configuring at least one alert using the data recorded about the mapping, the alert executed by the computing system and configured to be triggered by an object transiting the scene.

A method and system for calibration of an augmented reality (AR) device's position and orientation based on a robot's positional configuration. A conventional visual calibration target is not required for AR device calibration. Instead, the robot itself, in any pose, is used as a three dimensional (3D) calibration target. The AR system is provided with a CAD model of the entire robot to use as a reference frame, and 3D models of the individual robot arms are combined into a single object model based on joint positions known from the robot controller. The 3D surface model of the entire robot in the current pose is then used for visual calibration of the AR system by analyzing images from the AR device camera in comparison to the surface model of the robot in the current pose. The technique is applicable to initial AR device calibration and to ongoing device tracking.