An indication of a task to be performed in a network data center is received. A robotic manipulator of an autonomous robot is controlled to autonomously perform at least a portion of the task. It is determined that an assistance is required in performing an identified limited portion of the task. A notification of a request for the assistance is provided. A remote assistance from an operator in performing the identified limited portion of the task is received. Autonomous performance of the task is resumed after completion of the remote assistance for the identified limited portion of the task.

A teleoperated system comprises a display, a master input device, and a control system. The control system is configured to determine an orientation of an end effector reference frame relative to a field of view reference frame, determine an orientation of a master input device reference frame relative to a display reference frame, establish an alignment relationship between the master input device reference frame and the display reference frame, and command, based on the alignment relationship, a change in a pose of the end effector in response to a change in a pose of the master input device. The alignment relationship is independent of a position relationship between the master input device reference frame and the display reference frame. In one aspect, the teleoperated system is a telemedical system such as a telesurgical system.

The present disclosure relates to a first Web server (102, 204, 60, 70) and a second Web server (108, 214, 80, 90), and methods therein for controlling of a robot device over a cloud interface. A hyper-text transfer protocol, HTTP, request for a trajectory between a start position and a goal position is sent (S120, S230, 302, 402) towards the second Web server. One or more calculated trajectories are obtained (S122, 304) based on information as received encoded in the request. A HTTP response is sent (306) towards the first WEB server, comprising one or more calculated trajectories. Executing (S126, S266; 308, 406) of a trajectory at least based said one or more of the received trajectories is performed by the first Web server (102, 204, 60, 70). A scalable robot device control method is thus proposed, which is advantageously uses stored calculated trajectories between start and goal positions, for the robot device.

Systems and methods for remotely controlling a system using video are provided. A method in accordance the present disclosure includes detecting a video signal of an auxiliary system at a video input, wherein the video signal including images encoded with control information. The method also includes determining that the images included in the video signal include the control information. The method further includes extracting the control information from the images. Additionally, the method includes modifying operations of the system based on the control information.

A system for controlling a first robotic arm relative to a second robotic arm is disclosed. The system includes a two robotic arms each including a surgical tool and a tool driver. A central control circuit is configured to communicate with the robotic arms to determine a position of the robotic arms and modify a control algorithm for one of the robotic arms based on the relative position of the other robotic arm.

A graphical user interface for a remote controlled robot system that includes a robot view field that displays information provided by a robot and an observer view field that display observer information about one or more observers that can receive the robot information. The interface has various features that allow a master user to control the observation and participation of the observers.

The present disclosure relates to a Web server (104, 60, 70) and a method therein of determining a trajectory for controlling of a robot device over a cloud interface. From a URI-encoded HTTP request (402) for the trajectory between a first pose and a second pose of the robot device, it is determined (S112, 408) the length of matching between cached trajectories and the trajectory requested. The longest length of matching is compared (S114, 410) to a minimal matching length (406), and if the longest length is longer than the minimal matching length, a HTTP response is sent (S122, 412) comprising the trajectory being determined. If the longest length is shorter than the minimal matching length, a HTTP request to calculate is sent (S116, 414). Currently available web service architecture can be reused, and easily up-scaled.

The system, devices and methods disclosed herein enable autonomous and tele-operation of tele-operated robots for maintenance of a property around known and unknown obstacles. A method may include using an unmanned aerial vehicle for obtaining additional data relating to the property and obstacles within the property and plan a path around the obstacles using data from sensors on-board the tele-operated robot and the aerial image. A method may also provide optimization of total time needed for performing the property maintenance and the labor costs in situations where manual intervention is needed for navigating the tele-operated robot around obstacles on the property or for removing obstacles on the property.

A teleoperated system comprises a display, a master input device, and a control system. The control system is configured to determine an orientation of an end effector reference frame relative to a field of view reference frame, determine an orientation of a master input device reference frame relative to a display reference frame, establish an alignment relationship between the master input device reference frame and the display reference frame, and command, based on the alignment relationship, a change in a pose of the end effector in response to a change in a pose of the master input device. The alignment relationship is independent of a position relationship between the master input device reference frame and the display reference frame. In one aspect, the teleoperated system is a telemedical system such as a telesurgical system.

Embodiments of the present disclosure provide a system, method, apparatus and computer-readable medium for teleoperation. An exemplary system includes a robot machine having a machine body, at least one sensor, at least one robot processor, and at least one user processor operable to maintain a user simulation model of the robot machine and the environment surrounding the robot machine, the at least one user processor being remote from the robot machine. The system further includes at least one user interface comprising a haptic user interface operable to receive user commands and to transmit the user commands to the user simulation model, a display operable to display a virtual representation of the user simulation model.