A method controls a remote robotic avatar based on a description of a physical object. A message transmitter transmits a message to a remote robotic avatar instructing the remote robotic avatar to identify a physical object at a second location, where an appearance of the physical object is described in the message. One or more processors determine a first position at the first location relative to the first object, also determine a second position at the second location that correlates with the first position by being a scaled distance away from the second object relative to the remote robotic avatar. The message transmitter transmits a message directing the remote robotic avatar to reposition itself to the second position at the second location, and to initiate a teleoperative session between the first person in the first location and the remote robotic avatar at the second location.

A hardware platform includes adapters to interface with medical provisioning equipment involved in a remote robotic medical procedure at a patient site and a software system component, potentially cloud-based, to process real-time incoming data streams and a database to store the data streams for later use or archival. Each data stream can itself contain different types of sensory data. The software system component provides overall control and overrideable supervision over the remote robotic operations includes a state manager that synergistically processes the incoming data streams possibly of varying transmission rates, times of receipt or transmission, resolution, and quality. The state manager forms a near- or real-time virtual representation of the operative field. The software system component includes a data broker that can parse the RTVR into subsets data tailored to the needs of the medical provisioning and monitoring equipment, which may also be deployed and co-located at the patient site.

A robot system with a robot that has a camera and a remote control station that can connect to the robot. The connection can include a plurality of privileges. The system further includes a server that controls which privileges are provided to the remote control station. The privileges may include the ability to control the robot, joint in a multi-cast session and the reception of audio/video from the robot. The privileges can be established and edited through a manager control station. The server may contain a database that defines groups of remote control station that can be connected to groups of robots. The database can be edited to vary the stations and robots within a group. The system may also allow for connectivity between a remote control station at a user programmable time window.

Described herein is an inventory management system, and corresponding methods, in which robotic units are configured to execute at least a portion of a set of instructions in an automated manner. During execution of the set of instructions, robotic units may encounter tasks that they are unable to complete in an automated manner. The robotic units, upon encountering these tasks, may submit a request for manual operation to a control unit. In some embodiments, the request may be assigned to an available operator, which may take over control of the robotic unit using a remote manipulation device, to complete the task manually. Once the task has been completed manually, automated execution of the instructions may be resumed. In embodiments of the disclosed system, an operator may work assigned requests as those requests are received, which may eliminate downtime for the operator.

Disclosed is a method of controlling of a robot including evaluating whether a requestor of control over the robot has a privilege to do so, and if so, then recognizing the authority of the requestor. Disclosed is method of controlling a robot including selecting a point and/or a route for the robot and invoking a control that enables transmitting instructions to which a robot may respond by traveling to the point and/or along the route.

Based on data indicative of an area proximate to a robotic device, a scene is generated. Based on information from a knowledge database, a task associated with the scene is identified. A risk threshold is determined based on the scene, the task, and one or more trust thresholds. Based on the risk threshold, a ratio of sub-tasks of the task to be controlled by a user is determined. In accordance with the risk threshold, a user input is received for controlling one or more of the sub-tasks when the ratio dictates that at least one of the sub-tasks requires user intervention.

A robot system with a robot that has a camera and a remote control station that can connect to the robot. The connection can include a plurality of privileges. The system further includes a server that controls which privileges are provided to the remote control station. The privileges may include the ability to control the robot, joint in a multi-cast session and the reception of audio/video from the robot. The privileges can be established and edited through a manager control station. The server may contain a database that defines groups of remote control station that can be connected to groups of robots. The database can be edited to vary the stations and robots within a group. The system may also allow for connectivity between a remote control station at a user programmable time window.

Described in detail herein is an automated fulfillment system with automatic exception handling. A computing system can transmit instructions to an autonomous robot device to retrieve physical objects disposed in facility. An autonomous robot device can navigate to the location of the physical objects and pick up a first quantity of physical objects. The autonomous robot device can pick up the physical objects and can determine a set of attributes associated with the picked up physical objects. The autonomous robot device detect an error with the physical objects picked up by the autonomous robot device based on the set of attributes. The autonomous robot device can resolve the error with the physical objects that were picked up by the autonomous robot device. For example, autonomous robot device can discard the physical objects. The autonomous robot device can pick up replacement physical objects.

A robot system with a robot that has a camera and a remote control station that can connect to the robot. The connection can include a plurality of privileges. The system further includes a server that controls which privileges are provided to the remote control station. The privileges may include the ability to control the robot, joint in a multi-cast session and the reception of audio/video from the robot. The privileges can be established and edited through a manager control station. The server may contain a database that defines groups of remote control station that can be connected to groups of robots. The database can be edited to vary the stations and robots within a group. The system may also allow for connectivity between a remote control station at a user programmable time window.

A robot system includes robot bodies, operation devices each configured to accept operation and generate operational information for causing the robot body to operate, motion controllers configured to control operation of the corresponding robot body in response to the operational information, operation target selectors configured to receive an operation for selecting any of the robot bodies and request a permission to operate the selected robot body based on the operational information from the corresponding operation device, and an operation permitting device having a determinator configured to receive the permission request from the operation target selector and determine whether a permission is to be granted for the permission request. When the permission request is received, and the operation of the robot body selected by the operation target selector based on the operational information from a different operation device is permitted, the determinator prohibits the permission to the permission request.