Implementations described herein relate to training and refining robotic control policies using imitation learning techniques. A robotic control policy can be initially trained based on human demonstrations of various robotic tasks. Further, the robotic control policy can be refined based on human interventions while a robot is performing a robotic task. In some implementations, the robotic control policy may determine whether the robot will fail in performance of the robotic task, and prompt a human to intervene in performance of the robotic task. In additional or alternative implementations, a representation of the sequence of actions can be visually rendered for presentation to the human can proactively intervene in performance of the robotic task.

A solution for performing a set of tasks using one or more robotic devices is provided. The robotic device can be configured to perform each task using one or more effector devices, one or more sensor devices, and a hybrid control architecture including a plurality of dynamically changeable levels of autonomy. The levels of autonomy can include: full autonomy of the robotic device, teleoperation of the robotic device by a human user, and at least one level of shared control between the computer system and the human user.

A robotic logistics system is disclosed. The system includes multiple robots each having an image capture unit and a server communicatively coupled to the multiple robots. The server is configured to transmit a location of a first item to a first robot and the location of a second item to a second robot; track the positions of the first robot and the second robot; transmit a first image of the first item captured by the first robot to an operator device; receive a first verification signal from the operator device in response to the first image; transmit a second image of the second item captured by the second robot to the operator device; and receive a second verification signal from the operator device in response to the second image.

Methods, systems, and computer programs are presented for executing a mission by an autonomous device to inspect an asset. One method includes an operation for obtaining a workflow. The workflow includes operations to be executed during a mission to be performed by a robot and a destination for sending data resulting from the mission. The method further includes an operation for generating a package after completion of the mission associated with the workflow. The package is self-contained and comprises information obtained during the mission that enables generation of results. The package comprises sensor information collected by one or more sensors, telemetry information obtained by the robot, information about assets associated with the mission, software version identifier for the package generation, and routing information for transmitting the package to the destination. The method further includes an operation for analyzing the information of the package to determine results for the mission.

The present disclosure describes robots, tele-operation systems, methods, and computer program products where a robot is selectively operable in a plurality of control modes. Based on identification of a fault condition (when the robot fails to act in a suitable or sufficient manner), a control mode of the robot can be changed to provide a human operator with more explicit control over the robot. In this way, the fault condition can be resolved by human operator input, and the control modes, AI, or control paradigm for the robot can be trained to perform better in the future.

An intelligent system, such as an autonomous robot agent, includes systems and methods to learn various aspects about a task in response to instructions received from a human instructor, to apply the instructed knowledge immediately during task performance following the instruction, and to instruct other intelligent systems about the knowledge for performing the task. The learning is accomplished free of training the intelligent system. The instructions from the human instructor may be provided in a natural language format and may include deictic references. The instructions may be received while the intelligent system is online, and may be provided to the intelligent system in one shot, e.g., in a single encounter or transaction with the human instructor.

Computing platforms, methods, and storage media for sensing and controlling with respect to a robot. A robot control and sensor system may include a pressure sensor configured to be mounted on a robot, and/or mounted on a robot peripheral, to measure a sensed pressure value at the robot. The pressure sensor operates with respect to first and second pressure thresholds. A controller is in communication with the pressure sensor and may be configured to: obtain, from the pressure sensor, a sensed pressure value relating to pressure applied to the robot at the pressure sensor; generate a soft reset notification to cause the robot to enter a soft reset mode when the sensed pressure value is above the first pressure threshold; and generate a hard reset notification to cause the robot to enter a hard reset mode when the sensed pressure value is above the second pressure threshold.

Computing platforms, methods, and storage media for sensing and controlling with respect to a robot. A robot control and sensor system may include a pressure sensor configured to be mounted on a robot, and/or mounted on a robot peripheral, to measure a sensed pressure value at the robot. The pressure sensor operates with respect to first and second pressure thresholds. A controller is in communication with the pressure sensor and may be configured to: obtain, from the pressure sensor, a sensed pressure value relating to pressure applied to the robot at the pressure sensor; generate a soft reset notification to cause the robot to enter a soft reset mode when the sensed pressure value is above the first pressure threshold; and generate a hard reset notification to cause the robot to enter a hard reset mode when the sensed pressure value is above the second pressure threshold.

A robot agent (102) includes an electro-mechanical subsystem (202), a sensor subsystem (204) having one or more sensors, and a computer hardware subsystem (206) to execute one or more sets of executable instructions (212, 214, 216, 218, 220). The one or more sets of executable instructions manipulate the robot agent to predict an action to be implemented by the robot agent in performing a task (112) and predict whether the robot agent will fail in performing the action. The one or more sets of executable instructions further manipulate the robot agent to, responsive to predicting the robot agent will fail in performing the action, obtain guidance input (116) for the first action from at least one guidance source, the guidance input representing guidance for performing the action by the robot agent, and manipulate the electro-mechanical subsystem to perform the action using the guidance input.

A robot agent (102) includes an electro-mechanical subsystem (202), a sensor subsystem (204) having one or more sensors, and a computer hardware subsystem (206) to execute one or more sets of executable instructions (212, 214, 216, 218, 220). The one or more sets of executable instructions manipulate the robot agent to predict an action to be implemented by the robot agent in performing a task (112) and predict whether the robot agent will fail in performing the action. The one or more sets of executable instructions further manipulate the robot agent to, responsive to predicting the robot agent will fail in performing the action, obtain guidance input (116) for the first action from at least one guidance source, the guidance input representing guidance for performing the action by the robot agent, and manipulate the electro-mechanical subsystem to perform the action using the guidance input.