A method of deriving autonomous control information involves receiving one or more sets of associated environment sensor information and device control instructions. Each set of associated environment sensor information and device control instructions includes environment sensor information representing an environment associated with an operator controllable device and associated device control instructions configured to cause the operator controllable device to simulate at least one action taken by at least one operator experiencing a representation of the environment generated from the environment sensor information. The method also involves deriving autonomous control information from the one or more sets of associated environment sensor information and device control instructions, the autonomous control information configured to facilitate generating autonomous device control signals from autonomous environment sensor information representing an environment associated with an autonomous device, the autonomous device control signals configured to cause the autonomous device to take at least one autonomous action.

A method of deriving autonomous control information involves receiving one or more sets of associated environment sensor information and device control instructions. Each set of associated environment sensor information and device control instructions includes environment sensor information representing an environment associated with an operator controllable device and associated device control instructions configured to cause the operator controllable device to simulate at least one action taken by at least one operator experiencing a representation of the environment generated from the environment sensor information. The method also involves deriving autonomous control information from the one or more sets of associated environment sensor information and device control instructions, the autonomous control information configured to facilitate generating autonomous device control signals from autonomous environment sensor information representing an environment associated with an autonomous device, the autonomous device control signals configured to cause the autonomous device to take at least one autonomous action.

An operation information generating apparatus includes: a task data recording unit configured to record, as task data, information regarding an operation of a person from a start to an end of a task; an operation classifying unit configured to divide an overall operation from the start to the end of the task into a plurality of partial operations; and an operation combining unit configured to select a best partial operation for each partial operation from a plurality of samples of the task data for the same task, and generates data of an optimum operation of the entire task by combining the selected best partial operations.

An automated kitchen assistant system inspects a food preparation area in the kitchen environment using a novel sensor combination. The combination of sensors includes an Infrared (IR) camera that generates IR image data and at least one secondary sensor that generates secondary image data. The IR image data and secondary image data are processed to obtain combined image data. A trained convolutional neural network is employed to automatically compute an output based on the combined image data. The output includes information about the identity and the location of the food item. The output may further be utilized to command a robotic arm, kitchen worker, or otherwise assist in food preparation. Related methods are also described.

Described are techniques for storing and retrieving items using a robotic device for moving items. Any combinations of image data depicting a manipulator interacting with an item, sensor data from sensors instrumenting the manipulator or item, item data regarding characteristics of the item, and constraint data relating to characteristics of the robotic device may be used to generate one or more configurations for the robotic device. The configurations may include points of contact and force vectors for contacting the item using the robotic device.

The present disclosure is directed to methods, computer program products, and computer systems for instructing a robot to prepare a food dish by replacing the human chef's movements and actions. Monitoring a human chef is carried out in an instrumented application-specific setting, a standardized robotic kitchen in this instance, and involves using sensors and computers to watch, monitor, record and interpret the motions and actions of the human chef, in order to develop a robot-executable set of commands robust to variations and changes in the environment, capable of allowing a robotic or automated system in a robotic kitchen to prepare the same dish to the standards and quality as the dish prepared by the human chef.

The present disclosure is directed to methods, computer program products, and computer systems for instructing a robot to prepare a food dish by replacing the human chef's movements and actions. Monitoring a human chef is carried out in an instrumented application-specific setting, a standardized robotic kitchen in this instance, and involves using sensors and computers to watch, monitor, record and interpret the motions and actions of the human chef, in order to develop a robot-executable set of commands robust to variations and changes in the environment, capable of allowing a robotic or automated system in a robotic kitchen to prepare the same dish to the standards and quality as the dish prepared by the human chef.

The Invention relates to a control device (25, 28, 29) for controlling a robot system (11) with at least one robot arm (14, 18) on which a surgical instrument (15, 19) is secured that has an end effector (17, 21), wherein the control device (25, 28, 29) comprises an imaging system (25, 28) that records the control specification of at least one hand (30.sub.L, 30.sub.R), evaluates it and converts it to corresponding control commands for one or more components of the robot system (11). In order to simplify the control of the robot system (11) and in particular to make it intuitive, it is proposed that a control unit (25) be provided which determines the orientation and/or the position and/or the degree of opening of the end effector (15, 19) of a surgical instrument (15, 19) as first parameter or first parameters, and moreover determines the orientation and/or the position and/or the degree of opening of at least one hand (30.sub.L, 30.sub.R) as second parameter or second parameters, and, in the event of a deviation of one or more of the first parameters from the corresponding second parameter, suppresses a manual control of the end effector (17, 21), and, in the event of an agreement of one or more of the first parameters with the corresponding second parameter, frees a gesture control, such that the surgical instrument (15, 19) can be actuated by hand.

The present disclosure is directed to methods, computer program products, and computer systems for instructing a robot to prepare a food dish by replacing the human chef's movements and actions. Monitoring a human chef is carried out in an instrumented application-specific setting, a standardized robotic kitchen in this instance, and involves using sensors and computers to watch, monitor, record and interpret the motions and actions of the human chef, in order to develop a robot-executable set of commands robust to variations and changes in the environment, capable of allowing a robotic or automated system in a robotic kitchen to prepare the same dish to the standards and quality as the dish prepared by the human chef.

The present disclosure is directed to methods, computer program products, and computer systems for instructing a robot to prepare a food dish by replacing the human chef's movements and actions. Monitoring a human chef is carried out in an instrumented application-specific setting, a standardized robotic kitchen in this instance, and involves using sensors and computers to watch, monitor, record and interpret the motions and actions of the human chef, in order to develop a robot-executable set of commands robust to variations and changes in the environment, capable of allowing a robotic or automated system in a robotic kitchen to prepare the same dish to the standards and quality as the dish prepared by the human chef.