Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing planning for robotic placement tasks. One of the methods includes receiving a request for a robot to perform a placement task with an object, wherein the placement task has an associated insertion plane, wherein the request specifies a location on the insertion plane and an orientation angle for the object. A plan is generated for the robot including generating a rotational motion to transition the gripped object from an initial orientation to an orientation in which a principal vector is aligned with the insertion plane and the object is aligned with the orientation angle of the request.

Methods and apparatus for controlling lighting of a mobile robot are provided. A mobile robot includes a drive system configured to enable the mobile robot to be driven, a navigation module configured to provide control instructions to the drive system, a plurality of lighting modules, wherein each of the plurality of lighting modules includes a plurality of individually-controllable light sources, and a controller configured to control an operation of the plurality of individually-controllable light sources based, at least in part, on navigation information received from the navigation module.

Pre-plating systems and related methods are disclosed. A pre-plating system includes a press, an infeed robot configured to deliver a structural member to the press, and an outfeed robot configured to remove the structural member from the press. The press is configured to secure a plate to the structural member while the structural member is held in position by at least one of the infeed robot or the outfeed robot. A pre-plating system includes a press, a transfer pedestal, a plate picking robot, and a press loading robot. The plate picking robot is configured to pick a plate from a container and position the plate on the transfer pedestal. The press loading robot is configured to transfer the plate to the press. The press is configured to press the plate into a structural member positioned within the press.

Provided is a robot system capable of appropriately removing an object that may deform depending on a placement state. A robot system according to the present disclosure is provided with: an imaging apparatus that captures an image of an object; a robot that picks up the object; an image processing apparatus that identifies the position of the object on the basis of the image captured by the imaging apparatus; and a robot controller that causes the robot to pick up the object, the position of which has been identified by the image processing apparatus. The image processing apparatus comprises: a teaching unit that sets a search range in which a feature amount for a pattern on the object in the image captured by the imaging apparatus may be obtained; a recognition unit that extracts the pattern having the feature amount within the search range in the image captured by the imaging apparatus; and a correction unit that corrects the search range on the basis of the feature amount of the pattern extracted by the recognition unit.

A system and method for operating a robotic system to scan and register unrecognized objects is disclosed. The robotic system may use an image data representative of an unrecognized object located at a start location to implement operations for transferring the unrecognized object from the start location. While implementing the operations, the robotic system may obtain additional data, including scanning results of one or more portions of the unrecognized object not included in the image data. The robotic system may use the additional data to register the unrecognized object.

A system and method for identifying a box to be picked up by a robot from a stack of boxes. The method includes obtaining a 2D red-green-blue (RGB) color image of the boxes and a 2D depth map image of the boxes using a 3D camera. The method employs an image segmentation process that uses a simplified mask R-CNN executable by a central processing unit (CPU) to predict which pixels in the RGB image are associated with each box, where the pixels associated with each box are assigned a unique label that combine to define a mask for the box. The method then identifies a location for picking up the box using the segmentation image.

A method for palletizing by a robot includes positioning an object at an initial position adjacent to a target object location, tilting the object at an angle relative to a ground plane, shifting the object in a first direction from the initial position toward a first alignment position, shifting the object in a second direction from the first alignment position toward a second alignment position, and releasing the object from the robot to pivot the object toward the target object location.

Methods and systems for remote perception assistance to facilitate robotic object manipulation are provided herein. From a model of objects in an environment of a robotic manipulator, a virtual boundary line separating two adjacent identified virtual objects may be identified. The robotic manipulator may be configured to perform a task involving a manipulation of at least one object in the environment represented by the identified virtual objects. Based on the identifying, a request for remote assistance with verifying the virtual boundary line may be sent to a remote assistor device. A response to the request, including a modification to the virtual boundary line, may then be received from the remote assistor device. The robotic manipulator may then be caused to perform the task based on the modification to the virtual boundary line.

A system and method for operating a robotic system to register unrecognized objects is disclosed. The robotic system may use first image data representative of an unrecognized object to derive an initial minimum viable region (MVR). The robotic system may analyze second image data representative of the unrecognized object to detect a condition representative of an accuracy of the initial MVR. The robotic system may register the initial MVR or an adjustment thereof based on the detected condition.

Techniques are described that enable robotic picking systems to locate and pick boxes from an unstructured pallet using computer vision and/or one or more “exploratory picks” to determine the sizes and locations of boxes on the pallet.