Methods, systems, and apparatus, including computer programs encoded on computer storage media, for automatically generating object representations. One of the methods includes grasping, by a robot, an object at a first grasp point and generating a first partial object mesh based on one or more first sensor measurements of the object when held by the robot at the first grasp point. A second grasp point is identified for the object that is located in a region captured by the one or more first sensor measurements. A second partial object mesh is generated based on one or more second sensor measurements of the object when held by the robot at the second grasp point.

In one embodiment, a method includes receiving, from a first sensor on a robot, first sensor data indicative of an environment of the robot. The method also includes identifying, based on the first sensor data, an object of an object type in the environment of the robot, where the object type is associated with a classifier that takes sensor data from a predetermined pose relative to the object as input. The method further includes causing the robot to position a second sensor on the robot at the predetermined pose relative to the object. The method additionally includes receiving, from the second sensor, second sensor data indicative of the object while the second sensor is positioned at the predetermined pose relative to the object. The method further includes determining, by inputting the second sensor data into the classifier, a property of the object.

This disclosure provides systems, methods, and apparatuses, including computer programs encoded on computer storage media, that provide for training, implementing, or updated machine learning logic, such as an artificial neural network, to model a manufacturing process performed in a manufacturing robot environment. For example, the machine learning logic may be trained and implemented to learn from or make adjustments based on one or more operational characteristics associated with the manufacturing robot environment. As another example, the machine learning logic, such as a trained neural network, may be implemented in a semi-autonomous or autonomous manufacturing robot environment to model a manufacturing process and to generate a manufacturing result. As another example, the machine learning logic, such as the trained neural network, may be updated based on data that is captured and associated with a manufacturing result. Other aspects and features are also claimed and described.

A method for performing placement informed robotic picking of objects includes acquiring a first image of a pick scene including a number of objects and acquiring a second image of a placement area that receives objects picked from the pick scene by a robot. Object masks are computed by performing instance segmentation based on the first image. Place region masks are computed by clustering locations in the second image based on a height level from a floor of the placement area. A cost is computed for respective object-region pairs, each object-region pair defining a pairing between an object mask and a place region mask. The cost is defined at least in part by a place constraint. An object is selected to be picked from the pick scene by the robot by selecting an object-region pair based on the computed cost.

A computer-implemented method of determining actions for controlling a robot, in particular an assembly robot, includes (i) receiving a first and second input, wherein the first input is a sentence describing an action which should be carried out by the robot, wherein the second input is an image of a current state of an environment of the robot, (ii) feeding the first input into a first machine learning model and feeding the second input into a second machine learning model, wherein the first and second machine learning models are configured to determine tokens for their respective inputs, and (iv) feeding the tokens into a third machine learning model, wherein the third machine learning model outputs two outputs, wherein the first output is a switch for incorporating specialized skill networks and the second output are actions.

A sample handler of a diagnostic laboratory system includes a plurality of holding locations configured to receive sample containers. An imaging device is movable within the sample handler and is configured to capture images of the holding locations and sample containers received therein. A controller is configured to generate instructions that cause the imaging device to move within the sample handler and capture images. A classification algorithm is implemented in computer code, and includes a trained model configured to classify objects in the captured images. Other sample handlers and methods of handling sample containers are disclosed.

A robotic gripping device is provided. The robotic gripping device includes a palm and a plurality of digits coupled to the palm. The robotic gripping device also includes a time-of-flight sensor arranged on the palm such that the time-of-flight sensor is configured to generate time-of-flight distance data in a direction between the plurality of digits. The robotic gripping device additionally includes an infrared camera, including an infrared illumination source, where the infrared camera is arranged on the palm such that the infrared camera is configured to generate grayscale image data in the direction between the plurality of digits.

A method inspects weld quality in-situ. The method obtains a plurality of sequenced images of an in-progress welding process and generates a multi-dimensional data input based on the plurality of sequenced images and/or one or more weld process control parameters. The parameters may include: (i) shield gas flow rate, temperature, and pressure; (ii) voltage, amperage, wire feed rate and temperature (if applicable); (iii) part preheat/inter-pass temperature; and (iv) part and weld torch relative velocity). The method generates defect probability and analytics information by applying one or more computer vision techniques on the multi-dimensional data input. The analytics information includes predictive insights on quality features of the in-progress welding process. The method then generates a 3-D visualization of one or more as-welded regions, based on the analytics information, and the plurality of sequenced images. The 3-D visualization displays the quality features for virtual inspection and/or for determining weld quality.

A robot system includes a robot, an image capture apparatus, an image processing portion, and a control portion. The image processing portion is configured to specify in an image of a plurality of objects captured by the image capture apparatus, at least one area in which a predetermined object having a predetermined posture exists, and obtain information on position and/or posture of the predetermined object in the area. The control portion is configured to control the robot, based on the information on position and/or posture of the predetermined object, for the robot to hold the predetermined object.

Systems for autonomously batching a plurality of separated laundry articles into sorted loads for washing and drying are described. For example, each one of a plurality of collection bins is configured to receive a sorted load of separated articles including at least one common one of one or more washing and drying characteristics. A plurality of conveyors are configured to receive thereon the bins and position one bin into a loading position adjacent to an exit orifice of a sorting surface. At least one sensor disposed at least one of on, adjacent to, and within the surface is configured to detect the washing and drying characteristics. A controller in operable communication with a drive of the plurality of conveyors and the at least one sensor is configured to instruct the conveyors to move the bins to batch each separated laundry article into a bin matching the washing and drying characteristics.