Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using simulated local demonstration data for robotic demonstration learning. One of the methods includes receiving perceptual data of a workcell of a robot to be configured to execute a task according to a skill template, wherein the skill template specifies one or more subtasks required to perform the skill, wherein at least one of the subtasks is a demonstration subtask that relies on learning visual characteristics of the workcell. A virtual model is generated of a portion of the workcell. A training system generates simulated local demonstration data from the virtual model of the portion of the workcell and tunes a base control policy for the demonstration subtask using the simulated local demonstration data generated from the virtual model of the portion of the workcell.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for integrating sensor streams for robotic demonstration learning. One of the methods includes selecting, by a learning system for a robot, a base update rate for combining multiple sensor streams into a task state representation. The learning system repeatedly generates the task state representation at the base update rate, including combining, during each time period defined by the update rate, the task state representation from most recently updated sensor data processed by the plurality of neural networks. The learning system repeatedly uses the task state representations to generate commands for the robot at the base update rate.

A system for tuning of axis control of a multi-axis machine and a method of operating the same are provided. The system includes a knowledge base for acquiring and maintaining factual knowledge associated with the tuning of the axis control. The factual knowledge has a uniform ontology a uniform data representation, and includes known input facts associated with known output facts. The system further includes an inference unit for automatically inferring new output facts associated with given new input facts in accordance with the factual knowledge.

In one embodiment, a monitoring device ingests a plurality of data records sequentially from a data stream, each having an associated timestamp, and builds a cluster pattern for a plurality of time periods by placing each data record into a corresponding cluster of a particular time period based on the associated timestamp of each data record. The monitoring device then establishes connection between clusters of different time periods by assigning each data record of each particular time period to both an adjacent preceding and succeeding time period. The monitoring device may detect cluster transitions based on the established connections between clusters of different time periods, and can compute cluster migration metrics based on the cluster transitions. The monitoring device then predicts future cluster migration metrics based on computed cluster migration metrics, detects an anomaly about the predicted future cluster migration metrics, and reacts to the anomaly, accordingly.

A machine learning device that learns an operation of a robot for picking up, by a hand unit, any of a plurality of workpieces placed in a random fashion, including a bulk-loaded state, includes a state variable observation unit that observes a state variable representing a state of the robot, including data output from a three-dimensional measuring device that obtains a three-dimensional map for each workpiece, an operation result obtaining unit that obtains a result of a picking operation of the robot for picking up the workpiece by the hand unit, and a learning unit that learns a manipulated variable including command data for commanding the robot to perform the picking operation of the workpiece, in association with the state variable of the robot and the result of the picking operation, upon receiving output from the state variable observation unit and output from the operation result obtaining unit.

A machine learning device that learns an operation of a robot for picking up, by a hand unit, any of a plurality of objects placed in a random fashion, including a bulk-loaded state, includes a state variable observation unit that observes a state variable representing a state of the robot, including data output from a three-dimensional measuring device that obtains a three-dimensional map for each object, an operation result obtaining unit that obtains a result of a picking operation of the robot for picking up the object by the hand unit, and a learning unit that learns a manipulated variable including command data for commanding the robot to perform the picking operation of the object, in association with the state variable of the robot and the result of the picking operation, upon receiving output from the state variable observation unit and output from the operation result obtaining unit.

Methods, systems, and apparatus, including medium-encoded computer program products, for computer aided design and manufacture of physical structures using toolpaths generated by reinforcement learning for use with subtractive manufacturing systems and techniques, include: obtaining, in a computer aided design or manufacturing program, a three dimensional model of a manufacturable object; generating toolpaths that are usable by a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object by providing at least a portion of the three dimensional model to a machine learning algorithm that employs reinforcement learning, wherein the machine learning algorithm includes one or more scoring functions that include rewards that correlate with desired toolpath characteristics comprising toolpath smoothness, toolpath length, and avoiding collision with the three dimensional model; and providing the toolpaths to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object.

Robotic devices may be trained by a user guiding the robot along a target trajectory using a correction signal. A robotic device may comprise an adaptive controller configured to generate control commands based on one or more of the trainer input, sensory input, and/or performance measure. Training may comprise a plurality of trials. During an initial portion of a trial, the trainer may observe robot's operation and refrain from providing the training input to the robot. Upon observing a discrepancy between the target behavior and the actual behavior during the initial trial portion, the trainer may provide a teaching input (e.g., a correction signal) configured to affect robot's trajectory during subsequent trials. Upon completing a sufficient number of trials, the robot may be capable of navigating the trajectory in absence of the training input.

A computer-implemented method for training a machine learning system. The machine learning system is configured to determine a control signal characterizing an action to be executed by a technical system. The method includes obtaining a safe action to be executed by the technical system including: obtaining a state signal; determining, by a parametrized policy module of the machine learning system, a distribution of potentially unsafe actions that could be executed by the technical system; sampling a potentially unsafe action from the distribution; obtaining, by a safety module of the machine learning system, the safe action. The method further includes determining a loss value based on the state signal and the safe action; and training the machine learning system by updating parameters of the policy module according to a gradient of the loss value with respect to the parameters.

A machine learning device performs machine learning on a numerical control device which, when a first command including a corner portion, composed of two blocks in the machining program, generates a second command in which the two blocks are replaced with m or more blocks. The machine learning device comprises: a state information acquisition unit for acquiring state information including the first command, coordinate values of each block in the m or more blocks, and location information of the machining path and the machining time; an action information output unit for outputting action information; a reward output unit for outputting a reward value based on the inward turning amount in the corner portion; and a value function updating unit for updating a value function based on the value of the reward outputted from the reward output unit, the state information and the action information.