Embodiments of the present disclosure are directed to methods, computer program products, and computer systems of a robotic apparatus with robotic instructions replicating a food preparation recipe. In one embodiment, a robotic control platform, comprises one or more sensors; a mechanical robotic structure including one or more end effectors, and one or more robotic arms; an electronic library database of minimanipulations; a robotic planning module configured for real-time planning and adjustment based at least in part on the sensor data received from the one or more sensors in an electronic multi-stage process file, the electronic multi-stage process recipe file including a sequence of minimanipulations and associated timing data; a robotic interpreter module configured for reading the minimanipulation steps from the minimanipulation library and converting to a machine code; and a robotic execution module configured for executing the minimanipulation steps by the robotic platform to accomplish a functional result.

The disclosure provides a method for generating a joint command. The method includes receiving a maneuver script including a plurality of maneuvers for a legged robot to perform where each maneuver is associated with a cost. The method further includes identifying that two or more maneuvers of the plurality of maneuvers of the maneuver script occur at the same time instance. The method also includes determining a combined maneuver for the legged robot to perform at the time instance based on the two or more maneuvers and the costs associated with the two or more maneuvers. The method additionally includes generating a joint command to control motion of the legged robot at the time instance where the joint command commands a set of joints of the legged robot. Here, the set of joints correspond to the combined maneuver.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for integrating sensor streams for robotic demonstration learning. One of the methods includes selecting, by a learning system for a robot, a base update rate for combining multiple sensor streams into a task state representation. The learning system repeatedly generates the task state representation at the base update rate, including combining, during each time period defined by the update rate, the task state representation from most recently updated sensor data processed by the plurality of neural networks. The learning system repeatedly uses the task state representations to generate commands for the robot at the base update rate.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for providing user feedback for robotic demonstration learning. One of the methods includes initiating a local demonstration learning process to collect respective local demonstration data for each of one or more demonstration subtasks defined by a skill template to be executed by a robot. Local demonstration data is repeatedly collected for each of the one or more demonstration subtasks of the skill template while a user manipulates a robot to perform each of the one or more demonstration subtasks defined by the skill template. A respective progress value for each of the one or more demonstration subtasks defined by the skill template is maintained. A user interface presentation is generated that presents a suggested demonstration to be performed by the user based on a respective progress value for each demonstration subtask.

Robots, methods, and computer program products for training and operating (semi-) autonomous robots to complete multiple different work objectives are described. A robot accesses a library of reusable work primitives from a catalog of libraries of reusable work primitives, each reusable work primitive corresponding to a respective basic sub-task or sub-action that the robot is operative to autonomously perform. A work objective is analyzed to determine a sequence (i.e., a combination and/or permutation) of reusable work primitives that, when executed by the robot, will complete the work objective. The robot executes the sequence of reusable work primitives to complete the work objective. A robot can be deployed with an appropriate stored library (or access to an appropriate library) of reusable work primitives, based on what the robot is expected to do, or what service category or role the robot will operate in.

Embodiments of the present disclosure are directed to methods, computer program products, and computer systems of a robotic apparatus with robotic instructions replicating a food preparation recipe. In one embodiment, a robotic control platform, comprises one or more sensors; a mechanical robotic structure including one or more end effectors, and one or more robotic arms; an electronic library database of minimanipulations; a robotic planning module configured for real-time planning and adjustment based at least in part on the sensor data received from the one or more sensors in an electronic multi-stage process file, the electronic multi-stage process recipe file including a sequence of minimanipulations and associated timing data; a robotic interpreter module configured for reading the minimanipulation steps from the minimanipulation library and converting to a machine code; and a robotic execution module configured for executing the minimanipulation steps by the robotic platform to accomplish a functional result.

There is provided an information processing device, an information processing terminal, and an information processing method capable of efficiently developing a recipe program to be used by a cooking robot. The processing device according to an aspect of the present technology provides a recipe program generated by combining, according to a dish, program modules prepared for each of cooking steps or for each unit of operations included in each of the cooking steps. The present technology is applicable to a server which discloses a recipe program to be used for controlling a cooking robot.

A system and method for machine understanding, using program induction, includes a visual cognitive computer including a set of components designed to execute predetermined primitive functions. The method includes determining programs using a program induction engine that interfaces with the visual cognitive computer to discover programs using the predetermined primitive functions and/or executes the discovered programs based on an input.

The purpose is to enable an instructor who does not know any programming language to easily teach robot movements and operation content to a robot without using a teach pendant. This method for generating a robot operation program includes a step, using a GUI, for sequentially executing in a plurality of template element operation programs the feature of displaying a variable specification screen for specifying a variable of a certain template element operation program, and then storing the template element operation program for which the variable was specified in a storage unit as a custom element operation program. The plurality of template element operation programs is configured so that: one or more finger position coordinates specifying a robot motion required for the element operation corresponding to the program are included as variables; and the one or more finger position coordinates are all specified, thereby specifying the robot motion.

Embodiments of the present disclosure are directed to methods, computer program products, and computer systems of a robotic apparatus with robotic instructions replicating a food preparation recipe. In one embodiment, a robotic control platform, comprises one or more sensors; a mechanical robotic structure including one or more end effectors, and one or more robotic arms; an electronic library database of minimanipulations; a robotic planning module configured for real-time planning and adjustment based at least in part on the sensor data received from the one or more sensors in an electronic multi-stage process file, the electronic multi-stage process recipe file including a sequence of minimanipulations and associated timing data; a robotic interpreter module configured for reading the minimanipulation steps from the minimanipulation library and converting to a machine code; and a robotic execution module configured for executing the minimanipulation steps by the robotic platform to accomplish a functional result.