Various approaches to ensuring safe operation of industrial machinery in a workcell include disposing multiple image sensors proximate to the workcell and acquiring, with at least some of the image sensors, the first set of images of the workcell; registering the sensors to each other based at least in part on the first set of images and, based at least in part on the registration, converting the first set of images to a common reference frame of the sensors; determining a transformation matrix for transforming the common reference frame of the sensors to a global frame of the workcell; registering the sensors to the industrial machinery; acquiring the second set of images during operation of the industrial machinery; and monitoring the industrial machinery during operation thereof based at least in part on the acquired second plurality of images, transformation, and registration of the sensors to the industrial machinery.

Image sensors distributed about a workcell including industrial machinery are registered using a registration object and an information tag associated therewith. The tag contains information specifying the location of the object and/or the pose of the object. This information is acquired along with images of the registration object, and the sensors are registered based at least in part on the images and the acquired information.

A method of tracking a pose of an object includes determining an initial pose of the object at a first position, receiving position data and velocity data corresponding to movement of the object to a second position by a moving device, determining an expected pose of the object at the second position based on the position and velocity data and the initial pose, receiving second image data corresponding to the object at the second position from a camera, and determining a refined pose of the object at the second position based on the second image data and the expected pose.

Autonomous operations, such as robotic grasping and manipulation, in unknown or dynamic environments present various technical challenges. For example, three-dimensional (3D) reconstruction of a given object often focuses on the geometry of the object without considering how the 3D model of the object is used in solving or performing a robot operation task. As described herein, in accordance with various embodiments, models are generated of objects and/or physical environments based on tasks that autonomous machines perform with the objects or within the physical environments. Thus, in some cases, a given object or environment may be modeled differently depending on the task that is performed using the model. Further, portions of an object or environment may be modeled with varying resolutions depending on the task associated with the model.

TOOL TRACKING A system for determining the location of a toolpiece, wherein: the toolpiece is carried by a tool and the tool comprises an imaging device for capturing images of the environment around the tool; and the system comprises an image processor communicatively coupled to the imaging device for receiving images therefrom and having access to one or more reference images of an expected environment, the image processor being configured to compare an image captured by the imaging device with at least one reference image to identify a match therebetween and to determine in dependence on characteristics of that match the location of the toolpiece.

Various approaches to ensuring safe operation of industrial machinery in a workcell include disposing multiple image sensors proximate to the workcell and acquiring, with at least some of the image sensors, the first set of images of the workcell; registering the sensors to each other based at least in part on the first set of images and, based at least in part on the registration, converting the first set of images to a common reference frame of the sensors; determining a transformation matrix for transforming the common reference frame of the sensors to a global frame of the workcell; registering the sensors to the industrial machinery; acquiring the second set of images during operation of the industrial machinery; and monitoring the industrial machinery during operation thereof based at least in part on the acquired second plurality of images, transformation, and registration of the sensors to the industrial machinery.

A robot control method is executed in a control device connected to a robot that performs a work on a work object and a sensor mounted on the robot. First, environmental information indicating a peripheral environment of the robot is calculated based on a plurality of pieces of environment sensor information acquired for the peripheral environment by using the sensor at a plurality of positions and angles. Subsequently, first sensor information is acquired by the sensor, and a first step of calculating first control information causing the robot to approach a work goal based on work goal information indicating the work goal, the environmental information, and the first sensor information is executed. Second sensor information is acquired by the sensor, and a second step of calculating second control information causing the robot to approach the work goal based on the work goal information and the second sensor information is executed.

This robot system includes a sensor which obtains data for detecting at least a position of an object which is being moved, a robot at which a tool is attached, and which performs a predetermined operation for the object by means of the tool, and a controller which controls a tool moving device which moves the tool with respect to the robot, the tool, or an arm member which is located at a distal end side of an arm of the robot with a tool control cycle which is shorter than a control cycle of the robot.

Systems and methods for sensor registration for a workcell including a floor are disclosed. In some embodiments, a disclosed method comprises: disposing a plurality of three dimensional (3D) image sensors proximate to the workcell; acquiring, with the plurality of 3D image sensors, a plurality of 3D images of the workcell, wherein at least one of the plurality of 3D images includes at least a portion of the floor; determining, based on a first user selection of a region of a first image of the plurality of 3D images of the workcell, a first floor point indicating a first portion of the floor in the first image; computing a first floor plane representing the floor based on the first floor point; and registering the 3D image sensors to each other and to the workcell based at least in part on the plurality of 3D images and the first floor plane.

Various approaches to ensuring safe operation of industrial machinery in a workcell include disposing multiple image sensors proximate to the workcell and acquiring, with at least some of the image sensors, the first set of images of the workcell; registering the sensors to each other based at least in part on the first set of images and, based at least in part on the registration, converting the first set of images to a common reference frame of the sensors; determining a transformation matrix for transforming the common reference frame of the sensors to a global frame of the workcell; registering the sensors to the industrial machinery; acquiring the second set of images during operation of the industrial machinery; and monitoring the industrial machinery during operation thereof based at least in part on the acquired second plurality of images, transformation, and registration of the sensors to the industrial machinery.