Methods, apparatus, systems, and computer-readable media are provided for selecting robot poses to account for cost. In various implementations, a plurality of candidate instruction sets may be determined. Each candidate instruction set may be configured to cause a robot to assume a different respective set of poses while traversing a reference point along a path. In various implementations, a cost incurred while the robot implements the candidate instruction set to traverse the reference point along the path may be calculated. A candidate instruction set associated with an incurred cost that satisfies a first criterion may be selected from the plurality of candidate instruction sets. In some implementations, the selected candidate instruction set and incurred cost may be associated with the path.

Systems and methods for online iterative re-planning are provided herein. In one embodiment, a method includes receiving, at a first time step, a first grasp and an initial object pose of an agent. The method also includes generating a first set of candidate object trajectories based on the first grasp and the initial object pose. Candidate object trajectories of the first set of candidate object trajectories provide a number object poses from the initial object pose to a goal for a number of future time steps after the first time step. The method further includes calculating contact points for grasps associated with each candidate object trajectory of the first set of candidate object trajectories. The method further includes selecting a first candidate object trajectory from the first set of candidate object trajectories. The method includes causing the agent to execute the first candidate object trajectory at a second time step.

A method for controlling movement trajectories of a robot, the method including predicting, in an offline mode, values of at least one parameter related to the execution of alternative movement trajectories between a first position of the robot and a second position of the robot; selecting, in the offline mode, a movement trajectory based on the predicted values of the at least one parameter; and executing the selected movement trajectory by the robot. A control system for controlling movement trajectories of a robot is also provided.

Systems and techniques for grasp selection may include receiving one or more candidate object trajectories and a current grasp of a robotic hand on an object, sampling random candidate grasps for the one or more candidate object trajectories based on the current grasp, generating one or more grasps to be optimized for each of the one or more candidate object trajectories based on the sampled candidate grasps, and optimizing one or more of the grasps to be optimized for each of the one or more candidate object trajectories based on a cost function.