Systems, devices, and methods for training and operating (semi-)autonomous robots to complete multiple different work objectives are described. A robot control system stores a library of reusable work primitives each corresponding to a respective basic sub-task or sub-action that the robot is operative to autonomously perform. A work objective is analyzed to determine a sequence (i.e., a combination and/or permutation) of reusable work primitives that, when executed by the robot, will complete the work objective. The robot executes the sequence of reusable work primitives to complete the work objective. The reusable work primitives may include one or more reusable grasp primitives that enable(s) a robot's end effector to grasp objects. Simulated instances of real physical robots may be trained in simulated environments to develop control instructions that, once uploaded to the real physical robots, enable such real physical robots to autonomously perform reusable work primitives.

Systems, devices, and methods for training and operating (semi-)autonomous robots to complete multiple different work objectives are described. A robot control system stores a library of reusable work primitives each corresponding to a respective basic sub-task or sub-action that the robot is operative to autonomously perform. A work objective is analyzed to determine a sequence (i.e., a combination and/or permutation) of reusable work primitives that, when executed by the robot, will complete the work objective. The robot executes the sequence of reusable work primitives to complete the work objective. The reusable work primitives may include one or more reusable grasp primitives that enable(s) a robot's end effector to grasp objects. Simulated instances of real physical robots may be trained in simulated environments to develop control instructions that, once uploaded to the real physical robots, enable such real physical robots to autonomously perform reusable work primitives.

In one embodiment, a method includes determining objects and actions associated with the objects for completing a task to be executed by a robotic system, wherein each action is associated with trajectory, determining a pose for each person in an environment associated with the robotic system, predicting a trajectory for each person based on the determined pose associated with the respective person and the actions and trajectories associated with the actions, and adjusting trajectories for one or more of the actions to be performed by the robotic system based on the predicted trajectories for each person.

Implementations described herein relate to training and refining robotic control policies using imitation learning techniques. A robotic control policy can be initially trained based on human demonstrations of various robotic tasks. Further, the robotic control policy can be refined based on human interventions while a robot is performing a robotic task. In some implementations, the robotic control policy may determine whether the robot will fail in performance of the robotic task, and prompt a human to intervene in performance of the robotic task. In additional or alternative implementations, a representation of the sequence of actions can be visually rendered for presentation to the human can proactively intervene in performance of the robotic task.

A robot control device includes: a trained model built by being trained on work data; a control data acquisition section which acquires control data of the robot based on data from the trained model; base trained models built for each of a plurality of simple operations by being trained on work data; an operation label storage section which stores operation labels corresponding to the base trained models; a base trained model combination information acquisition section which acquires combination information when the trained model is represented by a combination of a plurality of the base trained models, by acquiring a similarity between the trained model and the respective base trained models; and an information output section which outputs the operation label corresponding to each of the base trained models which represent the trained model.

Provided is a robot system including: a robot including a hand; a unit job storage section configured to store a unit job; a linking job generation section configured to generate a linking job being a command to move the hand from an end position at which a first unit job has ended to a start position at which a second unit job to be executed subsequently after the first unit job is started; an action command generation section configured to generate an action command for the robot by connecting the unit jobs and the linking job in series, based on arrangement of a plurality of processing symbols; and a required time calculation section configured to calculate a required time of the action command by adding required times of the unit jobs and a required time of the linking job.

A robot system includes a robot, state detection sensors to, a timekeeping unit, a learning control unit, a determination unit, an operation device, and an input unit, and an additional learning unit. The determination unit determines whether or not the work of the robot can be continued under the control of the learning control unit based on the state values detected by the state detection sensors to and outputs determination result. The additional learning unit performs additional learning of the determination result indicating that the work of the robot cannot be continued, the operator operation force, work state output by the operation device and the input unit, and timer signal output by the timekeeping unit.

Systems, devices, and methods for training and operating (semi-)autonomous robots to complete multiple different work objectives are described. A robot control system stores a library of reusable work primitives each corresponding to a respective basic sub-task or sub-action that the robot is operative to autonomously perform. A work objective is analyzed to determine a sequence (i.e., a combination and/or permutation) of reusable work primitives that, when executed by the robot, will complete the work objective. The robot executes the sequence of reusable work primitives to complete the work objective. The reusable work primitives may include one or more reusable grasp primitives that enable(s) a robot's end effector to grasp objects. Simulated instances of real physical robots may be trained in simulated environments to develop control instructions that, once uploaded to the real physical robots, enable such real physical robots to autonomously perform reusable work primitives.

Methods, apparatus, systems and articles of manufacture are disclosed for object manipulation via action sequence optimization. An example method disclosed herein includes determining an initial state of a scene, generating a first action phase sequence to transform the initial state of the scene to a solution state of the scene by selecting a plurality of action phases based on action phase probabilities, determining whether a first simulated outcome of executing the first action phase sequence satisfies an acceptability criterion and, when the first simulated outcome does not satisfy the acceptability criterion, calculating a first cost function output based on a difference between the first simulated outcome and the solution state of the scene, the first cost function output utilized to generate updated action phase probabilities.

A robot control device includes: a trained model built by being trained on work data; a control data acquisition section which acquires control data of the robot based on data from the trained model; base trained models built for each of a plurality of simple operations by being trained on work data; an operation label storage section which stores operation labels corresponding to the base trained models; a base trained model combination information acquisition section which acquires combination information when the trained model is represented by a combination of a plurality of the base trained models, by acquiring a similarity between the trained model and the respective base trained models; and an information output section which outputs the operation label corresponding to each of the base trained models which represent the trained model.