A computer-implemented method executed by data processing hardware of a robot causes the data processing hardware to perform operations. The robot includes an articulated arm having an end effector configured to engage with an object. The operations include receiving a measured task parameter set for the end effector. The measured task parameter set representing positions of the end effector while manipulating the object. The operations also include generating a task space model for the object based on the measured task parameter set. The task space model modelling the at least one constrained axis of the object. The operations further include limiting movement of the end effector along the at least one constrained axis of the object based on the task space model.