A robot system includes a robot for performing predetermined processing to a treating object, a photographing device for photographing the treating object, a robot control device for performing position compensation of a moving destination of the robot so as to track the treating object, on a basis of previously-set information on positions of the robot, the photographing device and the treating object, and an image of the treating object photographed by the photographing device, and a display device for providing an AR space. The robot control device calculates a position of the photographing device on the basis of the information on the positions of the robot and the photographing device. The display device displays an image imitating the photographing device at a corresponding position in the AR space, on a basis of the calculated position of the photographing device.

Systems and methods of assisting operator engagement with input devices include an input device configured to be operated by a hand of an operator, a repositionable structure coupled to the input device, a hand detection system, and a control unit. The control unit is configured to detect a position and an orientation of the hand using the hand detection system, determine, based on the position of the hand, a target position for the input device, wherein moving the input device from a current position of the input device to the target position moves the input device closer to a grasping position for the hand, and in response to determining that an orientation difference between the orientation of the hand and a current orientation of the input device is not greater than a threshold orientation difference, cause one or more actuators to move the input device toward the target position.

The invention provides a robot system that enables easy, efficient, and precise checking through simulation. The invention includes a virtual model display unit configured to place virtual models in a virtual space on a screen and display the virtual models simultaneously with real equipment; a robot program teaching unit configured to perform teaching of a robot program in the virtual space; a real space virtual model display unit configured to display the virtual models and teaching points of the robot program in a real space, based on a positional relationship in the virtual space; and a virtual model placement position correcting unit configured to correct placement positions of the virtual models to match the real equipment in the real space.

A method comprising identifying a robotic device and a calibration fixture in a vicinity of the robotic device; referencing the calibration fixture to a base of the robotic device to determine a first pose of the robotic device; receiving a 3D image of the environment, wherein the 3D image includes the calibration fixture; determining a second pose of the calibration fixture relative to the sensor; determining a third pose of the robotic device relative to the sensor based on the first pose and the second pose; receiving a plurality of trajectory points; determining a plurality of virtual trajectory points corresponding to the plurality of trajectory points based on the 3D image and the third pose; providing for display of the plurality of virtual trajectory points; and providing an interface for manipulating the virtual trajectory points.

A method includes determining a movement of an industrial robot in a manufacturing environment from a first position to a second position. The method also includes displaying an image showing a trajectory of the movement of the robot on a wearable headset. The displaying of the image comprises at least one of: displaying an augmented reality (AR) graphical image or video of the trajectory superimposed on a real-time actual image of the robot, or displaying a virtual reality (VR) graphical image or video showing a graphical representation of the robot together with the trajectory.

A service providing system including: a mobile robot configured to act in response to an instruction from a user through wireless communication and including a sensor having a capability corresponding to a human sensing capability to perceive an external world; a target identifying unit configured to identify whether an action target of the robot is a human being or another robot; an user apparatus control portion configured to output the signal detected by the sensor in a first manner when the action target is identified to be a human being and output the signal detected by the sensor in a second manner when the action target is identified to be the other robot; and a user apparatus configured to act in a manner perceivable by the user based on a output signal.

A human augmented robotics intelligence operation system can include a plurality of robots, each robot having a plurality of sensors; a robot control unit; and one or more articulating joints; a cloud-based robotic intelligence engine having; a communication module; a historical database; and a processor; and a human augmentation platform. The processor can be configured to make a probabilistic determination regarding the likelihood of successfully completing the particular user command. When the probabilistic determination is above a pre-determined threshold, the processor sends necessary executable commands to the robot control unit. Alternatively, when the probabilistic determination is below the predetermined threshold, the processor generates an alert and flags the operation for human review.

A remote control manipulator system includes a manipulator controlled remotely by an operator; a camera to capture an image including the manipulator; a posture sensor to detect posture data; an action instruction inputter with which the operator inputs an action instruction instructing action to move or stop the manipulator; a control device including a structural data storage to store manipulator structural data representing a structure of the manipulator, a model image generator to generate a model image with referring to the structural data storage and the posture data, and a presentation image generator to generate a presentation image by superimposing a model image on the captured image; and a display to display the presentation image.

A system and method of assisting operator engagement with input devices includes a repositionable structure configured to support a physical input device, a hand detection system, and a control unit. The repositionable structure includes one or more actuators. The physical input device is configured to be operated by a hand of an operator. The control unit is configured to detect the hand of the operator using the hand detection system and in response to a trigger condition, command the one or more actuators to move the input device from a first orientation to a second orientation, wherein the second orientation is closer to a grasping orientation of the hand than the first orientation is to the grasping orientation of the hand.

A facilitation system for facilitating remote presentation of a physical world includes a first object and an operating environment of the first object. The facilitation system includes a processing system configured to obtain an image frame depicting the physical world, identify a depiction of the first object in the image frame, and obtain a first spatial registration registering an object model with the first object in the physical world. The object model is of the first object. The processing system is further configured to obtain an updated object model corresponding to the object model updated with a current state of the first object, and generate a hybrid frame using the image frame, the first spatial registration, and the updated object model. The hybrid frame includes the image frame with the depiction of the first object replaced by a depiction of the updated object model.