A human interaction system for interaction by a user comprising social systems comprising at least a social robot and one or more virtual robot systems, and a coordination system, wherein: the social robot is controlled by a robot processing system and is configured to provide interaction with a user, said interaction including output means and input means, the one or more virtual robot systems are configured to controllably present an avatar representation of the social robot, and further configured to receive inputs, the coordination system is configured to coordinate operation of the social robot and the one or more virtual robots such that, at any one time, either the social robot or one of the virtual robot systems is active, such that, in operation, a user perceives a robot personality associated with the social robot and the avatar as associated with the active social robot or virtual robot system.

A method and system are provided for programming robots by operators without expertise in specialized robot programming languages. In the method, inputs are received from the operator generically describing a desired robot movement. The system then uses the operator inputs to translate the desired robot movement into software commands that direct the robot to perform the desired robot movement. The robot may then be programmed with the software commands and operated to perform the desired robot movement.

In an embodiment, a method includes identifying a force and torque for a robot to accomplish a task and identifying context of a portion of a movement plan indicating motion of the robot to perform the task. Based on the identified force, torque, and context, a context specific torque is determined for at least one aspect of the robot while the robot executes the portion of the movement plan. In turn, a control signal is generated for the at least one aspect of the robot to operate in accordance with the determined context specific torque.

Current technologies allow a robot to acquire a tool only if the tool is in a set known location, such as in a rack. In an embodiment, a method and corresponding system, can determine the previously unknown pose of a tool freely placed in an environment. The method can then calculate a trajectory that allows for a robot to move from its current position to the tool and attach with the tool. In such a way, tools can be located and used by a robot when placed at any location in an environment.

A problem with current food service robots is making the robots safe to work around food. A solution provided by the present disclosure is a food-safe tool switcher and corresponding tool. The tool switcher can mate with a variety of tools, which can be molded or 3D printed out of food-safe materials into a single-part, instead of constructed modularly. This provides for easier cleaning.

In one embodiment, a method is provided. The method includes obtaining sensor data indicative of a set of objects detected within an environment. The method also includes determining a set of positions of the set of objects and a set of properties of the set of objects based on the sensor data. The method further includes generating a state graph based on the sensor data. The state graph represents the set of objects and the set of positions of the set of objects. The state graph includes a set of object nodes to represent the set of objects and a set of property nodes to represent the set of properties of the set of objects. The state graph is provided to a graph enhancement module that updates the state graph with additional data to generate an enhanced state graph.

A method is provided. The method includes obtaining a state graph that represents a set of objects within an environment and a set of positions of the set of objects within the environment. The state graph includes a set of object nodes and a set of property nodes. The method also includes obtaining user input data. The user input data is generated based on a natural language input. The method further includes updating the state graph based on the user input data to generate an enhanced state graph. The enhanced state graph includes additional nodes generated based on the user input data. The method further includes providing the enhanced state graph to a planning module. The planning modules generates instructions for operating a mechanical system based on the enhanced state graph.

A method is provided. The method includes obtaining sensor data indicative of a set of objects detected within an environment. The method also includes generating a state graph based on the sensor data. The state graph includes a set of object nodes and a set of property nodes. The method further includes obtaining user input data generated based on a natural language input. The method further includes updating the state graph based on the user input data to generate an enhanced state graph. The enhanced state graph includes additional nodes generated based on the user input data. The method further includes generating a set of instructions for a set of mechanical systems based on the enhanced state graph. The method further includes operating the set of mechanical systems to achieve a set of objectives based on the set of instructions.

Systems and methods are described include a robot and/or an associated computing system that can use various cues about an environment of the robot to apply a bias to increase the accuracy of speech transcription. In some implementations, audio data corresponding to a spoken instruction to a robot is received. Candidate transcriptions of the audio data are obtained. A respective action of the robot corresponding to each of the candidate transcriptions of the audio data is determined. One or more scores indicating characteristics of a potential outcome of performing the respective action corresponding to the candidate transcription of the audio data are determined for each of the candidate transcriptions of the audio data. A particular candidate transcription is selected from among the candidate transcriptions based at least on the one or more scores. The action determined for the particular candidate transcription is performed.

A robot system includes a robot including leading end, base, and multi-articular arm, and circuitry that controls the atm to move the end based on motion control program specifying transition over time of target position and posture of the end, the transition including correction target portion starting and ending in the transition; controls the arm to move the end in response to guided manipulation applying external force to the robot while the circuitry controls the arm; obtains relative command information based on the target position and posture at start of the correction portion and specifying the target position and posture at points in the correction portion including start and end in the correction portion; and controls the arm to move the end from the position and posture based on the information, beginning at time when movement of the arm controlled by the circuitry in response to the manipulation has ended.