Training and/or use of a machine learning model for placement of an object secured by an end effector of a robot. A trained machine learning model can be used to process: (1) a current image, captured by a vision component of a robot, that captures an end effector securing an object; (2) a candidate end effector action that defines a candidate motion of the end effector; and (3) a target placement input that indicates a target placement location for the object. Based on the processing, a prediction can be generated that indicates likelihood of successful placement of the object in the target placement location with application of the motion defined by the candidate end effector action. At many iterations, the candidate end effector action with the highest probability is selected and control commands provided to cause the end effector to move in conformance with the corresponding end effector action. When at least one release criteria is satisfied, control commands can be provided to cause the end effector to release the object, thereby leading to the object being placed in the target placement location.

Utilizing at least one existing policy (e.g. a manually engineered policy) for a robotic task, in generating reinforcement learning (RL) data that can be used in training an RL policy for an instance of RL of the robotic task. The existing policy can be one that, standing alone, will not generate data that is compatible with the instance of RL for the robotic task. In contrast, the generated RL data is compatible with RL for the robotic task at least by virtue of it including state data that is in a state space of the RL for the robotic task, and including actions that are in the action space of the RL for the robotic task. The generated RL data can be used in at least some of the initial training for the RL policy using reinforcement learning.

A machine learning device for performing machine learning with respect to a numerical control device which causes a machine tool to operate, and is provided with: a state information acquisition unit that causes the machine tool to perform cutting work, in which a cutting amount and a cutting rate are set, and acquires state information including the cutting amount and cutting rate; an action information output unit that outputs action information; a reward calculation unit that acquires determination information that is information about the strength of pressure applied to a tool at least during cutting work, the shape of the waveform of the pressure applied to the tool, and time it has taken to perform work, and outputs a reward value in reinforcement learning; and a value function update unit that updates a value function on the basis of the reward value, the state information, and the action information.

A method for training a control strategy. The method includes providing training data, which demonstrate a control behavior, according to which control actions are to be generated, and training the control strategy with the aid of imitation learning by minimizing a measure of deviation between the distribution of state transitions according to the control strategy and the distribution of state transitions according to the demonstrated control behavior using the training data.

Robotics control system (10) and method for training said robotics control system are provided. Disclosed embodiments make a gracefully blended utilization of Reinforcement Learning (RL) with conventional control by way of a dynamically adaptive interaction between respective control signals (20, 24) generated by a conventional feedback controller (18) and an RL controller (22). Additionally, disclosed embodiments make use of an iterative approach for training a control policy by effective use of virtual sensor and actuator data (60) interleaved with real-world sensor and actuator data (54). This is effective to reducing a training sample size to fulfill a blended control policy for the conventional feedback controller and the reinforcement learning controller. Disclosed embodiments may be used in a variety of industrial automation applications.

Methods for image-based detection of the tops of sample tubes used in an automated diagnostic analysis system may be based on a convolutional neural network to pre-process images of the sample tube tops to intensify the tube top circle edges while suppressing the edge response from other objects that may appear in the image. Edge maps generated by the methods may be used for various image-based sample tube analyses, categorizations, and/or characterizations of the sample tubes to control a robot in relationship to the sample tubes. Image processing and control apparatus configured to carry out the methods are also described, as are other aspects.

A system for detecting an anomaly in an execution of a task in mixed human-robot processes. Receiving human worker (HW) signals and robot signals. A processor to extract from the HW signals, task information, measurements relating to a state of the HW, and input into a Human Performance (HP) model, to obtain a state of the HW based on previously learned boundaries of the state of the HW, the state of the HW is then inputted into a Human-Robot Interaction (HRI) model, to determine a classification of an anomaly or no anomaly. Update HRI model with robot operation signals, HW signals and classified anomaly, determine a control action of a robot interacting with the HW or a type of an anomaly alarm using the updated HRI model and classified anomaly. Output the control action of the robot to change a robot action or output the type of the anomaly alarm.

A method for producing a strategy for a robot. The method includes the following steps: initializing the strategy and an episode length; repeated execution of the loop including the following steps: producing a plurality of further strategies as a function of the strategy; applying the plurality of the further strategies for the length of the episode length; ascertaining respectively a cumulative reward, which is obtained in the application of the respective further strategy; updating the strategy as a function of a second plurality of the further strategies that obtained the greatest cumulative rewards. After each execution of the loop, the episode length is increased. A computer program, a device for carrying out the method, and a machine-readable memory element on which the computer program is stored, are also described.

In a robot (industrial robot) system, a robot holds a workpiece by pinching the workpiece between movable claws. A controller, which controls the robot, includes a host controller that controls the robot to perform a positioning operation for positioning the hand to a grip position and a gripping operation for displacing each of the movable claws toward each other at the grip position. In the controller, a machine learning device acquires stop reference data set for gripping of the workpiece, distance data indicating a distance between each of the movable claws of the hand positioned at the grip position and the workpiece, and comparison data indicating a deformation amount of the workpiece before and after the gripping operation. The machine learning device performs machine learning using such acquired data, resulting in constructing a model used for setting an operation mode of the gripping operation.

A method and system are herein disclosed wherein a robot handles objects that are large, unwieldy, highly-deformable, or otherwise difficult to contain and carry. The robot is operated to navigate an environment and detect and classify objects using a sensing system. The robot determines the type, size and location of objects and classifies the objects based on detected attributes. Grabber pad arms and grabber pads move other objects out of the way and move the target object onto the shovel to be carried. The robot maneuvers objects into and out of a containment area comprising the shovel and grabber pad arms following a process optimized for the type of object to be transported. Large, unwieldy, highly deformable, or otherwise difficult to maneuver objects may be managed by the method disclosed herein.