A pose controller is provided for controlling a pose of an object to assemble with a mating object by a gripper of a robot arm. The pose controller includes an interface configured to receive tactile signals from the tactile sensors and transmit a control signal to the actuators, a processor, and a memory, in association with the processor, configured to store a precomputed set of tactile depth images and instructions of computer-implemented method. The instructions cause the processor to perform steps of computing measured tactile depth images from the received tactile signals, refining the pose of the object by matching between the precomputed set of tactile depth images and the measured tactile depth images by a point-to-plane iterative closest point (ICP) algorithm, generating a gripper trajectory command based on the refined pose of the object and a target pose of the object, wherein the target pose is aligned against the mating object with a nominal distance above the mating object, and controlling the actuators of the robot arm according to a gripper trajectory by transmitting the gripper trajectory command to the robot controller.