A method and an apparatus for exclusive reinforcement learning are provided, comprising: collecting information of states of an environment through the communication interface and performing a statistical analysis on the states using the collected information; determining a first state value of a first state among the states in a training phase and a second state value of a second state among the states in an inference phase based on analysis results of the statistical analysis; performing reinforcement learning by using one reinforcement learning unit of a plurality of reinforcement learning unit which performs reinforcement learnings from different perspectives according to the first state value; and selecting one of actions determined by the plurality of reinforcement learning unit based on the second state value and applying selected action to the environment.

In one embodiment, a monitoring device ingests a plurality of data records sequentially from a data stream, each having an associated timestamp, and builds a cluster pattern for a plurality of time periods by placing each data record into a corresponding cluster of a particular time period based on the associated timestamp of each data record. The monitoring device then establishes connection between clusters of different time periods by assigning each data record of each particular time period to both an adjacent preceding and succeeding time period. The monitoring device may detect cluster transitions based on the established connections between clusters of different time periods, and can compute cluster migration metrics based on the cluster transitions. The monitoring device then predicts future cluster migration metrics based on computed cluster migration metrics, detects an anomaly about the predicted future cluster migration metrics, and reacts to the anomaly, accordingly.

Provided is a machine learning device configured to perform machine learning related to optimization of a compensation value of a compensation generation unit with respect to a servo control device configured to control a servo motor configured to drive an axis of a machine tool, a robot, or an industrial machine, and that includes at least one feedback loop, a compensation generation unit configured to generate a compensation value to be applied to the feedback loop, and an abnormality detection unit configured to detect an abnormal operation of the servo motor, wherein, during a machine learning operation, when the abnormality detection unit detects an abnormality, the compensation from the compensation generation unit is stopped and the machine learning device continues optimization of the compensation value generated by the compensation generation unit.

An adjustment support device includes: a storage unit for storing, with force state data and position data in an operation when performing force control of the industrial robot as a state variable and with data indicating a result of determining whether a result of the force control is success or failure based on predetermined criteria as determination data, a learning model generated by machine learning; an analysis unit for analyzing the learning model to analyze, for a control parameter used when the force control of the industrial robot has failed, an adjustment method of the control parameter for improving a degree of success of the force control; and an adjustment determination unit for determining, based on a result of the analysis by the analysis unit, an adjustment method of the control parameter in the force control used when the force control has failed and outputting the adjustment method.

A machine learning device performs machine learning on a numerical control device which, when a first command including a corner portion, composed of two blocks in the machining program, generates a second command in which the two blocks are replaced with m or more blocks. The machine learning device comprises: a state information acquisition unit for acquiring state information including the first command, coordinate values of each block in the m or more blocks, and location information of the machining path and the machining time; an action information output unit for outputting action information; a reward output unit for outputting a reward value based on the inward turning amount in the corner portion; and a value function updating unit for updating a value function based on the value of the reward outputted from the reward output unit, the state information and the action information.

A machine learning device that performs reinforcement learning for a servo control device and optimizes a coefficient of a filter for attenuating a specific frequency component provided in the servo control device includes a state information acquisition unit which acquires state information that includes the result of calculation of at least one of an input/output gain of the servo control device and a phase delay of input and output, the coefficient of the filter and conditions, and an action information output unit which outputs, to the filter, action information including adjustment information of the coefficient. A reward output unit determines evaluation values under the conditions based on the result of the calculation to output, as a reward, the value of a sum of the evaluation values. A value function updating unit updates an action value function based on the value of the reward, the state information and the action information.

A machine learning device that learns an operation of a robot for picking up, by a hand unit, any of a plurality of workpieces placed in a random fashion, including a bulk-loaded state, includes a state variable observation unit that observes a state variable representing a state of the robot, including data output from a three-dimensional measuring device that obtains a three-dimensional map for each workpiece, an operation result obtaining unit that obtains a result of a picking operation of the robot for picking up the workpiece by the hand unit, and a learning unit that learns a manipulated variable including command data for commanding the robot to perform the picking operation of the workpiece, in association with the state variable of the robot and the result of the picking operation, upon receiving output from the state variable observation unit and output from the operation result obtaining unit.

To provide an action information learning device, robot control system and action information learning method for facilitating the performing of cooperative work by an operator with a robot. An action information learning device includes: a state information acquisition unit that acquires a state of a robot; an action information output unit for outputting an action, which is adjustment information for the state; a reward calculation section for acquiring determination information, which is information about a handover time related to handover of a workpiece, and calculating a value of reward in reinforcement learning based on the determination information thus acquired; and a value function update section for updating a value function by way of performing the reinforcement learning based on the value of reward calculated by the reward calculation section, the state and the action.

A machine learning device that learns an operation of a robot for picking up, by a hand unit, any of a plurality of workpieces placed in a random fashion, including a bulk-loaded state, includes a state variable observation unit that observes a state variable representing a state of the robot, including data output from a three-dimensional measuring device that obtains a three-dimensional map for each workpiece, an operation result obtaining unit that obtains a result of a picking operation of the robot for picking up the workpiece by the hand unit, and a learning unit that learns a manipulated variable including command data for commanding the robot to perform the picking operation of the workpiece, in association with the state variable of the robot and the result of the picking operation, upon receiving output from the state variable observation unit and output from the operation result obtaining unit.

A machine learning device performs reinforcement learning on a controller that performs multiple processes for controlling a machine tool in parallel at multiple operation units. The machine learning device comprises: behavior information output means that outputs behavior information containing allocation of arithmetic units that perform the multiple processes to the controller; state information acquisition means that acquires state information containing a machining condition as a condition for machining set at the machine tool, and determination information generated by monitoring the implementation of the multiple processes by the multiple operation units based on the allocation in the behavior information; reward calculation means that calculates the value of a reward to be given by the reinforcement learning based on the determination information in the state information; and value function update means that updates a behavior value function based on the reward value, the state information, and the behavior information.