Implementations are provided for operably coupling multiple robot controllers to a single virtual environment, e.g., to generate training examples for training machine learning model(s). In various implementations, a virtual environment may be simulated that includes an interactive object and a plurality of robot avatars that are controlled independently and contemporaneously by a corresponding plurality of robot controllers that are external from the virtual environment. Sensor data generated from a perspective of each robot avatar of the plurality of robot avatars may be provided to a corresponding robot controller. Joint commands that cause actuation of one or more joints of each robot avatar may be received from the corresponding robot controller. Joint(s) of each robot avatar may be actuated pursuant to corresponding joint commands. The actuating may cause two or more of the robot avatars to act upon the interactive object in the virtual environment.

A structure allows more efficient simulation using preliminary obtained image data for a target such as a workpiece. A simulator includes a creating unit that virtually creates a simulation target system in a three-dimensional virtual space, a measurement unit that performs image measurement of preliminary obtained image data using a visual sensor and outputs a measurement result, a reception unit that receives a setting of an imaging area defined for the visual sensor in a manner associated with the created system, a calculation unit that calculates a transform parameter based on a relative positional relationship between the created system and the set imaging area, and an execution unit that receives an input of a result obtained by transforming the measurement result generated by the measurement unit using the transform parameter, and executes a control operation in accordance with a control program associated with the system.

A Multi-Purpose Dynamic Simulation and run-time Control platform includes a virtual process environment coupled to a physical process environment, where components/nodes of the virtual and physical process environments cooperate to dynamically perform run-time process control of an industrial process plant and/or simulations thereof. Virtual components may include virtual run-time nodes and/or simulated nodes. The MPDSC includes an I/O Switch which delivers I/O data between virtual and/or physical nodes, e.g., by using publish/subscribe mechanisms, thereby virtualizing physical I/O process data delivery. Nodes serviced by the I/O Switch may include respective component behavior modules that are unaware as to whether or not they are being utilized on a virtual or physical node. Simulations may be performed in real-time and even in conjunction with run-time operations of the plant, and/or simulations may be manipulated as desired (speed, values, administration, etc.). The platform simultaneously supports simulation and run-time operations and interactions/intersections therebetween.

A Multi-Purpose Dynamic Simulation and run-time Control platform includes a virtual process environment coupled to a physical process environment, where components/nodes of the virtual and physical process environments cooperate to dynamically perform run-time process control of an industrial process plant and/or simulations thereof. Virtual components may include virtual run-time nodes and/or simulated nodes. The MPDSC includes an I/O Switch which delivers I/O data between virtual and/or physical nodes, e.g., by using publish/subscribe mechanisms, thereby virtualizing physical I/O process data delivery. Nodes serviced by the I/O Switch may include respective component behavior modules that are unaware as to whether or not they are being utilized on a virtual or physical node. Simulations may be performed in real-time and even in conjunction with run-time operations of the plant, and/or simulations may be manipulated as desired (speed, values, administration, etc.). The platform simultaneously supports simulation and run-time operations and interactions/intersections therebetween.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for robot programming. One of the methods comprises generating an interactive user interface that includes an illustration of a virtual robot corresponding to a physical robot; receiving first user input data specifying a first target pose of the virtual robot; causing the physical robot to traverse to the first target pose while updating in real-time the illustration of the virtual robot as the physical robot transitions to the first target pose; receiving a user request to switch from operating in a synchronized mode to operating in an unsynchronized mode; receiving second user input data specifying a second target pose of the virtual robot; and generating an animation of the virtual robot transitioning from the first target pose to the second target pose but withholding causing the physical robot to traverse to the second target pose.

A Multi-Purpose Dynamic Simulation and run-time Control platform includes a virtual process environment coupled to a physical process environment, where components/nodes of the virtual and physical process environments cooperate to dynamically perform run-time process control of an industrial process plant and/or simulations thereof. Virtual components may include virtual run-time nodes and/or simulated nodes. The MPDSC includes an I/O Switch which delivers I/O data between virtual and/or physical nodes, e.g., by using publish/subscribe mechanisms, thereby virtualizing physical I/O process data delivery. Nodes serviced by the I/O Switch may include respective component behavior modules that are unaware as to whether or not they are being utilized on a virtual or physical node. Simulations may be performed in real-time and even in conjunction with run-time operations of the plant, and/or simulations may be manipulated as desired (speed, values, administration, etc.). The platform simultaneously supports simulation and run-time operations and interactions/intersections therebetween.

A method for applying machine learning to an application includes: a) generating a candidate policy by a learner; b) executing a program in at least one simulated application based on a set of candidate parameters provided based on the candidate policy and a state of the at least one simulated application, execution of the program providing interim results of tested sets of candidate parameters based on a measured performance information of the execution of the program; c) collecting a predetermined number of interim results and providing an end result based on a combination of the candidate parameters and/or the state with the measured performances information by a trainer; and d) generating a new candidate policy by the learner based on the end result.

This simulation device comprises a simulation implementation unit that simulates a movement of a robot device, and estimates a movement path of a robot. The simulation device comprises a position information generation unit that generates three-dimensional position information for a surface of a workpiece on the basis of the output of a vision sensor, which has imaged an actual workpiece. A display unit displays the three-dimensional position information for the surface of the workpiece, superimposed on an image of a robot device model, an image of a workpiece model, and the movement path of the robot.