A computing system may direct a first robotic arm to a first position based on a first set of coordinates. The computing system may cause the first robotic arm to engage with a first structure based on the first position of the first robotic arm. Further, the computing system may direct the first robotic arm to a second position based on a second set of coordinates such that the first structure is brought within a joining proximity of a second structure without a fixture retaining the first structure and without a fixture retaining the second structure, wherein the first structure is configured to be joined with the second structure when the first and second structures are within the joining proximity, the joining proximity being a proximity at which the first and second structures can be joined together.

A manipulation controller is provided for reorienting an object by a manipulator of a robotic system. The manipulation controller includes an interface controller configured to acquire measurement data from sensors arranged on the robotic system, at least one processor, and a memory configured to store a computer-implemented method. The instructions of the method include acquiring measurement data from vision sensors and force sensors arranged on the robotic system, determining an input-output relation for the object based on a nonlinear static model representing input-output relationships between contact forces and movements of the object on the workbench, representing interaction between the object and the manipulator using complementarity constraints to capture the contact state between the object and the manipulator, formulating a representation for frictional stability of the object based on the non-linear static model at the external contacts with the workbench; formulating a bilevel optimization problem so as to maximize the frictional stability over a position trajectory of the object being manipulated on the workbench, estimating uncertainty value in physical parameters to be compensated by performing the bilevel optimization problem, solving the bilevel optimization problem using the non-linear optimization solver and generating control data with respect to a sequence of the contact forces being applied to the object by using the manipulator.