In an embodiment, a method and system use various sensors to determine a shape of a collection of materials (e.g., foodstuffs). A controller can determine a trajectory which achieves the desired end-state, possibly chosen from a set of feasible, collision-free trajectories to execute, and a robot executes that trajectory. The robot, executing that trajectory, scoops, grabs, or otherwise acquires the desired amount of material from the collection of materials at a desired location. The robot then deposits the collected material in the desired receptacle at a specific location and orientation.

Embodiments provide functionality to prevent collisions between robots and objects. An example embodiment detects a type and a location of an object based on a camera image of the object, where the image has a reference frame. Motion of the object is then predicted based on at least one of: the detected type of the object, the detected location of the object, and a model of object motion. To continue, a motion plan for the robot is generated that avoids having the robot collide with the object based on the predicted motion of the object and a transformation between the reference frame of the image and a reference frame of the robot. The robot can be controlled to move in accordance with the motion plan or a signal can be generated that controls the robot to operate in accordance with the motion plan.

Embodiments provide methods and systems to modify motion of a robot based on sound and context. An embodiment detects a sound in an environment and processes the sound. The processing includes comparing the detected sound to a library of sound characteristics associated with sound cues and/or extracting features or characteristics from the detected sound using a model. Motion of a robot is modified based on a context of the robot and at least one of: (i) the comparison, (ii) the features extracted from the detected sound, and (iii) the characteristics extracted from the detected sound.

In an embodiment, a method during execution of a motion plan by a robotic arm includes determining a voice command from speech of a user said during the execution of the motion plan, determining a modification of the motion plan based on the voice command from the speech of the user, and executing the modification of the motion plan by the robotic arm.

Robots, including robot arms, can interface with other modules to affect the world surrounding the robot. However, designing modules from scratch when human analogues exist is not efficient. In an embodiment, a mechanical tool, converted from human use, to be used by robots includes a monolithic adaptor having two interface components. The two interface components include a first interface component cabal be of mating with an actuated mechanism on the robot side, the second interface capable of clamping to an existing utensil. In such a way, utensils that are intended for humans can be adapted for robots and robotic arms.

A problem with current food service robots is making the robots safe to work around food. A solution provided by the present disclosure is a food-safe tool switcher and corresponding tool. The tool switcher can mate with a variety of tools, which can be molded or 3D printed out of food-safe materials into a single-part, instead of constructed modularly. This provides for easier cleaning.

In an embodiment, a cover for an automated robot includes elastic sheets that are adhered to each other in a geometry. The geometry is configured to allow the elastic sheets to expand and contract while the automated robot moves within its range of motion. The elastic sheets are attached to the automated robot by elasticity of the elastic sheets. A first group of the elastic sheets forms an elastic collar configured to grip the automated robot at a distal end and a proximal end of the cover. A person of ordinary skill in the art can recognize that nonbreakably means that during operation of the robot, the elastic sheets hold their elasticity and integrity without breaking.

In an embodiment, a method for handling an order includes determining a plurality of ingredients based on an order, received from a user over a network, for a location having a plurality of robots. The method further includes planning at least one trajectory for at least one robot based on the plurality of ingredients and utensils available at the location, and proximity of each ingredient and utensil to the at least one robot. Each trajectory can be configured to move one of the plurality of ingredients into a container associated with the order. In an embodiment, the method includes executing the at least one trajectory by the at least one robot to fulfill the order. In an embodiment, the method includes moving the container to a pickup area.

In an embodiment, a method includes determining a given material to manipulate to achieve a goal state. The goal state can be one or more deformable or granular materials in a particular arrangement. The method further includes, for the given material, determining, a respective outcome for each of a plurality of candidate actions to manipulate the given material. The determining can be performed with a physics-based model, in one embodiment. The method further can include determining a given action of the candidate actions, where the outcome of the given action reaching the goal state is within at least one tolerance. The method further includes, based on a selected action of the given actions, generating a first motion plan for the selected action.

The present disclosure generally relates to a robotic gripping system and method that utilizes vacuum suction and finger grasping, wherein the suction and grasping are actuated based on a dynamic collision model. In an exemplary embodiment, the present disclosure is directed to generating collision scenes of a surrounding environment which is used to determine possible collisions in a motion path, and which is used to selectively actuate the vacuum suction and/or finger grasping.