Methods, apparatus, and computer-readable media for determining and utilizing corrections to robot actions. Some implementations are directed to updating a local features model of a robot in response to determining a human correction of an action performed by the robot. The local features model is used to determine, based on an embedding generated over a corresponding neural network model, one or more features that are most similar to the generated embedding. Updating the local features model in response to a human correction can include updating a feature embedding, of the local features model, that corresponds to the human correction. Adjustment(s) to the features model can immediately improve robot performance without necessitating retraining of the corresponding neural network model.

An information processing device includes at least one memory, and at least one processor configured to perform, based on a state of a virtual world and a predetermined environment variable, a simulation with respect to the state of the virtual world, the state of the virtual world being based on an observation result of a real world, and the simulation being differentiable, and update the predetermined environment variable so that a result of the simulation approaches a changed state of the virtual world, the changed state being based on an observation result of the real world that is observed after the real world has changed.

In modeling contact between two or more objects (such as a robotic arm placing a block on a stack of blocks) or articulations of a series of linked joints (such as modeling a backhoe), current techniques can introduce additional energy into the system or fail to resolve a constraint imposed on the system. The current techniques attempt to resolve these issues, for example, by using very small time steps. Very small time steps, however, can significantly increase computational costs of the modeling simulation. The disclosed simulation system for rigid bodies uses a time interval to reduce linearization artifacts due to the small time steps and reduce computational costs with faster solver convergence by permitting more efficient bias calculations. High mass handling can also be improved through the more efficient bias calculations.

Methods and systems for connection-driven generation of robotic user interfaces and modification of robotic properties include detecting a connection of a robotic peripheral to a robot; obtaining a peripheral property set corresponding to the robotic peripheral, wherein the peripheral property set includes one or more properties of the robotic peripheral; modifying, based on the peripheral property set, a robotic property set that includes one or more properties of the robot to provide a modified robotic property set; generating, during runtime, a robotic graphical user interface (“RGUI”) dynamically based on the peripheral property set, wherein the RGUI provides at least one user-accessible interface to control the robot and the robotic peripheral; and controlling, based on the modified robotic property set, the robot and the robotic peripheral in response to user input received via the RGUI.

Spatial regions potentially occupied by a robot (or other machinery) or portion thereof and a human operator during performance of all or a defined portion of a task or an application are computationally estimated. These “potential occupancy envelopes” (POEs) may be based on the states (e.g., the current and expected positions, velocities, accelerations, geometry and/or kinematics) of the robot and the human operator. Once the POEs of human operators in the workspace are established, they can be used to guide or revise motion planning for task execution.

Spatial regions potentially occupied by a robot (or other machinery) or portion thereof and a human operator during performance of all or a defined portion of a task or an application are computationally estimated. These “potential occupancy envelopes” (POEs) may be based on the states (e.g., the current and expected positions, velocities, accelerations, geometry and/or kinematics) of the robot and the human operator. Once the POEs of human operators in the workspace are established, they can be used to guide or revise motion planning for task execution.

A method and system for determining geometric properties of a manipulator (2). The manipulator (2) is controlled to perform constrained motions exhibiting force interaction with the environment, or between different links of the manipulator (2), such that a kinematic chain is formed mechanically. The chain may include peripherals and external axes of motion. A constraining device, enables motions that facilitate the determination of geometric properties. A unified model of joint and link compliances facilitates determination of stiffness parameters. The force interaction is controlled with awareness of friction such that non-geometric properties are possible to identify, thereby enabling separation of non-geometric effects from the geometric ones, which improves accuracy.

Methods, apparatus, and computer readable media that are related to 3D object detection and pose determination and that may optionally increase the robustness and/or efficiency of the 3D object recognition and pose determination. Some implementations are generally directed to techniques for generating an object model of an object based on model point cloud data of the object. Some implementations of the present disclosure are additionally and/or alternatively directed to techniques for application of acquired 3D scene point cloud data to a stored object model of an object to detect the object and/or determine the pose of the object.

Provided is a simulation device for a simulation of a cooperative task carried out cooperatively by a cooperative robot and a person, and the simulation device includes a head mounting-type display device to be mounted on an operator to simulatively carry out the cooperative task, a detecting section configured to detect a position of the operator in a real space, a three-dimensional model display section configured to cause an image in which a robot system model including a cooperative robot model is arranged in a three-dimensional virtual space to be displayed on the head mounting-type display device, and a simulation execution section configured to simulatively operate the cooperative robot model in the three-dimensional virtual space based on an operation program of the cooperative robot to carry out the cooperative task and the detected position of the operator.

A method for robotic assembly includes: receiving product data including product structure data and/or product geometry data of a product with a base component and at least one assembly part to be assembled; analyzing the product data to determine robot functions relating to functions of a robot for assembly of the product as determined robot functions; generating a robot program including assembly instructions dependent on the determined robot functions and the product data; and executing the generated robot program so as to identify and/or localize the at least one assembly part and assemble the product.