A system for locating the center line of a bolt which extends through an aircraft part, including a robot which carries a nut or collar placement device and a stereo camera. A control system operates the camera to produce two images of the fastener at a specified angle. A processor then transforms the image information to control information for the robot to align the nut or collar placement device with the centerline of the fastener and then to place the nut or collar on the end of the fastener.

Systems and methods for inter-arm registration include a computer-assisted system having a control unit coupled to a repositionable arm of a computer-assisted device. The control unit is configured to: receive, from an imaging device, successive images of an instrument mounted to the repositionable arm; determine an observed velocity of a feature of the instrument; determine an expected velocity of the feature of the instrument based on kinematics of the repositionable arm; transform the observed velocity and/or the expected velocity to a common coordinate system using a registration transform; determine an error between directions of the observed and expected velocities in the common coordinate system; and update the registration transform based on the determined error. In some embodiments, the instrument is a medical instrument and the imaging device is an endoscope. In some embodiments, the control unit is further configured to control the instrument using the registration transform.

A control apparatus that is connectable to a memory that stores computer-readable instructions, comprising a processor that is configured to execute the computer-readable instructions so as to control driving of a robot and a camera, create setting details on an image processing sequence containing image processing of a captured image captured by the camera, and setting details on a calibration of correlating a coordinate system of the robot and a coordinate system of the camera, and call up the image processing sequence in creation of the setting details on the calibration.

A position detection apparatus detects the positions of movable parts 33, 43, 53 which are movable within processing space SP inside a chamber 90. An imaging device 72 shoots reference sections 61-64, which are disposed inside the chamber 90 and the positions of which are known, and the movable parts and accordingly obtains an original image. An image processor 86 executes image processing which is for detecting the movable parts and the reference sections from the original image. An image detector 86 detects the positions of the movable parts within the processing space in accordance with position information which is indicative of the positions of the movable parts and the reference sections within the original image. It is possible to accurately detect the positions of the movable parts while suppressing the influence exerted by deviation of the imaging device.

A manufacturing method for joining first and second members to create a joined piece using a robot with pre-inputted instruction data. The method includes operating the robot to hold the second member for joining to the first member and photographing the second member to obtain an image of the second member at the holding position; comparing the image to a reference image of a joining position of a reference second member joined to a reference first member; determining a deviation amount by which the holding position of the second member deviates from the joining position in the reference image; determining a correction amount for correcting the holding position of the second member is to be corrected in order to reduce the deviation amount of the holding position of the second member; correcting the holding position of the second member according to the correction amount, and then subsequently joining the first and second members.

A robot includes an arm, and the arm is controlled using an offset of a reference point of a tool, the tool being attached to the arm, the offset being set based on a first state, in which a first image, in which the reference point is located at a control point of an image, can be taken by a imaging section, and a second state, in which a second image, in which the tool is rotated around a rotational axis passing through a position of the reference point in the state in which the reference point is located at the control point, can be taken by the imaging section, and controlled based on a third image, which is taken by the imaging section, and in which the control point is shifted from the reference point, in a process of a transition from the first state to the second state.

Techniques for inter-arm registration include a computer-assisted system having a repositionable arm and a control unit coupled to the repositionable arm. The control unit is configured to receive, from an imaging device, image data of an instrument, the instrument mounted to the repositionable arm; determine an observed geometric property of a feature set of the instrument based on the image data, the feature set comprising at least one feature; determine an expected geometric property of the feature set based on at least kinematic data of the repositionable arm; determine a difference between the observed geometric property and the expected geometric property; update, based on the difference. a registration transform to produce an updated registration transform associated with the instrument; and control the instrument using the updated registration transform.

A method for calibrating an articulable end effector of a robotic arm employing a digital camera includes commanding the end effector to achieve a plurality of poses. At each commanded end effector pose, an image of the end effector with the digital camera is captured and a scene point cloud including the end effector is generated based upon the captured image of the end effector. A synthetic point cloud including the end effector is generated based upon the commanded end effector pose, and a first position of the end effector is based upon the synthetic point cloud, and a second position of the end effector associated with the scene point cloud is determined. A position of the end effector is calibrated based upon the first position of the end effector and the second position of the end effector for the plurality of commanded end effector poses.

The invention relates to a method for AR assisted adjustment of a safety plane for a robot via a portable AR device comprising a physical display and a camera. The method comprises the steps of establishing a spatial reference system defined with reference to said robot. Connecting said portable AR device to a robot controller. Recording the surroundings of said portable AR device including said robot. Displaying said recording of said surroundings on said physical display. Establishing a virtual environment according to said spatial reference system overlaying said recorded surroundings, wherein said virtual environment comprises at least on safety plane. Manually perform, via said physical display, a safety plane displacement of said safety plane in said spatial reference system in said virtual environment, and transfer said displaced location of said safety plane to said robot controller.