Methods, apparatus, systems, and computer-readable media are provided for using a camera of a robot to capture an image of the robot's environment, detecting edges in the image, and localizing the robot based on comparing the detected edges in the image to edges derived from a three-dimensional (3D) model of the robot's environment from the point of view of an estimated pose of the robot in the environment. In some implementations, the edges are derived based on rendering, from the 3D model of the environment, a model image of the environment from the point of view of the estimated poseand applying an edge detector to the rendered model image to detect model image edges from the model image.

A calibration system able to calibrate mechanical parameters of a wrist part by a simpler manner is provided. The calibration system utilizes a target fastened at a predetermined position with respect to the joint closest to a hand of a robot and an imaging device set around the robot so as to calibrate the parameters of a mechanical model representing the wrist part of the robot. The posture of the target is changed from a predetermined initial position to generate a plurality of preliminary positions. Using these preliminary positions as starting points, the end point position of the robot whereby the target becomes a predetermined positional relationship with respect to the imaging device on the image obtained by capturing an image of the target is calculated. The calibration system uses the calculated end point position as the basis to calibrate the mechanical parameters of the wrist part.

Methods, apparatus, systems, and computer-readable media are provided for using a camera of a robot to capture an image of the robot's environment, detecting edges in the image, and localizing the robot based on comparing the detected edges in the image to edges derived from a three-dimensional (3D) model of the robot's environment from the point of view of an estimated pose of the robot in the environment. In some implementations, the edges are derived based on rendering, from the 3D model of the environment, a model image of the environment from the point of view of the estimated poseand applying an edge detector to the rendered model image to detect model image edges from the model image.

A system is disclosed for providing extrinsic calibration of a camera to a relative working environment of a programmable motion device that includes an end-effector. The system includes a fiducial located at or near the end-effector, at least one camera system for viewing the fiducial as the programmable motion device moves in at least three degrees of freedom, and for capturing a plurality of images containing the fiducial, and a calibration system for analyzing the plurality of images to determine a fiducial location with respect to the camera to permit calibration of the camera with the programmable motion device.