Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an industrial plant controller that controls operation of an industrial plant. In one aspect, a method comprises generating training data using an industrial plant simulation model that simulates operation of the industrial plant. The industrial plant controller is trained by a reinforcement learning technique using the training data. The industrial plant controller is configured to process an input comprising a state vector characterizing a state of the industrial plant in accordance with a plurality of industrial plant controller parameters to generate an action selection policy output that defines a control action to be performed to control the operation of the industrial plant.

A simulation application container executes a simulation of a system in a simulation environment, through which an agent representing the system uses a reinforcement learning model to operate within the simulation environment. The simulation application container obtains data indicating how the agent performed in the simulation environment and transmits this data to a robot application container. The robot application container uses the data to update the reinforcement learning model and provides the updated reinforcement learning model to perform another iteration of the simulation and continue training the reinforcement learning model.

Training and/or using a recurrent neural network model for visual servoing of an end effector of a robot. In visual servoing, the model can be utilized to generate, at each of a plurality of time steps, an action prediction that represents a prediction of how the end effector should be moved to cause the end effector to move toward a target object. The model can be viewpoint invariant in that it can be utilized across a variety of robots having vision components at a variety of viewpoints and/or can be utilized for a single robot even when a viewpoint, of a vision component of the robot, is drastically altered. Moreover, the model can be trained based on a large quantity of simulated data that is based on simulator(s) performing simulated episode(s) in view of the model. One or more portions of the model can be further trained based on a relatively smaller quantity of real training data.

A controller for a machine tool includes a machine learning apparatus configured to learn an Nth-order time-derivative component of a speed of each axis of the machine tool. The machine learning apparatus includes: a state observation section configured to observe first state data representing the Nth-order time-derivative component of the speed of each axis as a state variable representing a current state of an environment; a determination data acquisition section configured to acquire determination data representing a properness determination result of at least any one of machining accuracy, surface quality, and machining time of the machined workpiece; and a learning section configured to learn the Nth-order time-derivative component of the speed of each axis in relation to at least any one of the machining accuracy, the surface quality, and the machining time of the machined workpiece using the state variable and the determination data.

A method selectively activates a resource to accommodate an advanced emotion. A supervisor computer receives a first piece of content, and then applies an emotion classifier to the first piece of content in order to create a first concept/emotion/sentiment/time tuple. The supervisor computer creates a second concept/emotion/sentiment/time tuple for a second piece of content, and compares the first and second tuples. If the concept in the first piece of content matches the concept in the second piece of content but that at least one of the emotion, sentiment, and time of the first piece of content does not match the emotion, sentiment, and time of the second piece of content, the supervisor computer determines that the emotion of the second piece of content is an advanced emotion that is not expressed by the first or second pieces of content, and activates a resource that accommodates the advanced emotion.

Training and/or using a recurrent neural network model for visual servoing of an end effector of a robot. In visual servoing, the model can be utilized to generate, at each of a plurality of time steps, an action prediction that represents a prediction of how the end effector should be moved to cause the end effector to move toward a target object. The model can be viewpoint invariant in that it can be utilized across a variety of robots having vision components at a variety of viewpoints and/or can be utilized for a single robot even when a viewpoint, of a vision component of the robot, is drastically altered. Moreover, the model can be trained based on a large quantity of simulated data that is based on simulator(s) performing simulated episode(s) in view of the model. One or more portions of the model can be further trained based on a relatively smaller quantity of real training data.

Provided are a controller and a machine learning device that perform machine learning to optimize the servo gain of a machine inside a facility in accordance with action conditions, action environments, and a priority factor of the machine. The control system observes machine information on a machine as state, acquires information on machining by a machine as determination data, calculates a reward based on the determination data and reward conditions, performs the machine learning of the adjustment of the servo gain of the machine, determines an action of adjustment of the servo gain of the machine based on the state data and a machine learning result of the adjustment of the servo gain of the machine, and changes the servo gain of the machine, based on the action of adjustment of the determined servo gain.

Disclosed herein is an automatic control artificial intelligence device including a collection unit configured to acquire an output value according to control of a control system; and an artificial intelligence unit operably coupled to the collection unit and configured to: communicate with the collection unit; set at least one of one or more base lines and a reward based on a gap between the one or more base lines and the output value, according to a plurality of operation goals of the control system; and update a control function for providing a control value to the control system by performing reinforcement learning based on the gap between the one or more base lines and the output value.

Methods, systems, and apparatus, including medium-encoded computer program products, for computer aided design and manufacture of physical structures using toolpaths generated by reinforcement learning for use with subtractive manufacturing systems and techniques, include: obtaining, in a computer aided design or manufacturing program, a three dimensional model of a manufacturable object; generating toolpaths that are usable by a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object by providing at least a portion of the three dimensional model to a machine learning algorithm that employs reinforcement learning, wherein the machine learning algorithm includes one or more scoring functions that include rewards that correlate with desired toolpath characteristics comprising toolpath smoothness, toolpath length, and avoiding collision with the three dimensional model; and providing the toolpaths to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object.

Provided is a machine learning device configured to perform machine learning related to optimization of a compensation value of a compensation generation unit with respect to a servo control device configured to control a servo motor configured to drive an axis of a machine tool, a robot, or an industrial machine, and that includes at least one feedback loop, a compensation generation unit configured to generate a compensation value to be applied to the feedback loop, and an abnormality detection unit configured to detect an abnormal operation of the servo motor, wherein, during a machine learning operation, when the abnormality detection unit detects an abnormality, the compensation from the compensation generation unit is stopped and the machine learning device continues optimization of the compensation value generated by the compensation generation unit.