CARRYING OUT AN APPLICATION USING AT LEAST ONE ROBOT

Abstract

A method for carrying out an application using at least one robot includes, repeatedly ascertaining a stochastic value of at least one robot parameter and/or at least one environmental model parameter; and carrying out a simulation of the application on the basis of the ascertained stochastic value, training at least one control agent and/or at least one classification agent using the simulations by machine learning, and carrying out the application using the robot. The method may further include configuring a controller of the robot, by means of which the application is carried out wholly or in part, based on the trained control agent, and/or classifying the application using the trained classification agent.

Claims

1-24. (canceled)

25. A method for carrying out an application using at least one robot, the method comprising: repeating multiple times: ascertaining a stochastic value of at least one robot parameter and/or at least one environmental model parameter [associated with the application?], and carrying out a simulation of the application on the basis of the ascertained stochastic value; then training at least one control agent and/or at least one classification agent using the simulations by machine learning; performing the application using the robot; and at least one of: configuring a controller of the robot, by which the application is carried out wholly or in part, on the basis of the trained control agent, or classifying the application using the trained classification agent.

26. The method of claim 25, wherein at least one of: ascertaining the stochastic value comprises at least one of ascertaining on the basis of a specified stochastic parameter model or ascertaining using at least one random generator; the simulation of the application is a multi-stage simulation; training using the simulations by machine learning comprises: training at least one of a first control agent or a first classification agent by first stages of the simulations, and training at least one additional control agent and/or additional classification agent by additional stages of the simulations; configuring the controller comprises configuring on the basis of the trained control agents; or classifying the application comprises using a plurality of trained classification agents.

27. The method of claim 25, wherein at least one of: the at least one control agent and/or the at least one classification agent comprises at least one of machine-learned anomaly detection, machine-learned error detection, or at least one artificial neural network; or the at least one control agent and/or the at least one classification agent is at least one of trained via reinforcement learning or trained using the robot.

28. The method of claim 27, wherein configuring the controller of the robot comprises configuring the controller on the basis of at least one of structure or weights of the trained neural network.

29. The method of claim 25, wherein at least one of: classifying the application with the at least one classification agent comprises at least one of classifying on the basis of at least one time segment, classifying while the application is being carried out, or classifying after the application has been carried out; or training the at least one control agent and/or the at least one classification agent comprises training on the basis of at least one state variable that is not measured when the application is carried out.

30. The method of claim 25, wherein at least one of: the robot parameter comprises a start pose and at least one of: at least one intermediate pose, a target pose of the application, a force parameter of at least one of a robot-internal or external force acting at least temporarily on the robot, or a kinematic robot structure parameter; the environmental model parameter comprises a kinematic environmental parameter; or at least one of the robot parameter or the environmental model parameter is ascertained using robot-assisted parameter identification.

31. The method of claim 30, further comprising checking whether at least one of the start pose, the intermediate pose, or the target pose can be reached with the robot.

32. The method of claim 26, wherein at least one of: the stochastic parameter model is at least one of: specified on the basis of the application, specified on the basis of user input, or visualized in an image of the application by a marked region; or at least one of the robot parameter or the environmental model parameter is at least one of: specified on the basis of the application, specified on the basis of user input, or visualized in an image of the application by a marked region.

33. The method of claim 25, further comprising at least one of: testing at least one of the configured controller of the robot, the machine-learned anomaly detection, or the error detection using at least one additional simulation; or further training at least one of the configured controller of the robot, the machine-learned anomaly detection, or the error detection using the robot.

34. The method of claim 33, wherein testing using at least one additional simulation is based on an automated or user specification of a value of at least one robot parameter and/or at least one environmental model parameter

35. The method of claim 26, wherein the stochastic parameter model is specified by machine learning.

36. A system for carrying out an application using at least one robot, the system comprising: means for repeating multiple times steps of: ascertaining a stochastic value of at least one robot parameter and/or at least one environmental model parameter associated with the application, and carrying out a simulation of the application on the basis of the ascertained stochastic value; means for training at least one control agent and/or at least one classification agent using the simulations by machine learning; means for performing the application using the robot; and means for at least one of: configuring a controller of the robot, by which the application is carried out wholly or in part, on the basis of the trained control agent, or classifying the application using the trained classification agent.

37. A method for configuring a controller of a robot for carrying out a specified task, the method comprising: recording at least one robot parameter and at least one environmental model parameter; training an agent using at least one simulation based on the recorded robot parameters and environmental model parameters by machine learning on the basis of a specified cost function; and configuring the controller of the robot based on the trained agent.

38. The method according to claim 37, wherein the specified task comprises at least one motion of the robot, in particular at least one scheduled environmental contact of the robot.

39. The method of claim 37, wherein at least one of: the robot parameter comprises at least one of: a kinematic, in particular dynamic, robot parameter, a load model parameter, a current robot pose, or a current robot operating time; or the environmental model parameter at least one of: comprises at least one of a CAD model parameter or a robot positioning in the environmental model, or is ascertained using at least one optical sensor.

40. The method of claim 37, wherein at least one of: the agent comprises an artificial neural network; or the agent is trained by reinforcement learning.

41. The method of claim 37, further comprising: further training the configured controller of the robot by machine learning, in particular reinforcement learning, using the robot.

42. The method of claim 37, wherein at least one of: at least one of the recording, training, or configuring steps comprises user input support by a software assistant, in particular a user interface; or at least one of the robot parameters or the environmental model parameters are stored in at least one of an asset administration shell or a data cloud.

43. A system for configuring a controller of a robot for carrying out a specified task, the system comprising means for: recording at least one robot parameter and at least one environmental model parameter; training an agent using at least one simulation based on the recorded robot parameters and environmental model parameters by machine learning on the basis of a specified cost function; and configuring the controller of the robot based on the trained agent.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0153] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and, together with a general description of the invention given above, and the detailed description given below, serve to explain the principles of the invention.

[0154] FIG. 1 schematically depicts a system according to one embodiment of the present invention;

[0155] FIG. 2 schematically illustrates parts of the system of FIG. 1;

[0156] FIG. 3 is a flowchart of a method according to one embodiment of the present invention;

[0157] FIG. 4 is a flowchart of a method according to a further embodiment of the present invention; and

[0158] FIG. 5 is a schematic illustration of a visualization of a stochastic parameter model and a robot parameter.

DETAILED DESCRIPTION

[0159] FIG. 1 shows a system according to one embodiment of the present invention with a robot 1, a (robot) controller 2 which communicates with the robot 1 and a cloud 4, as well as a data input/output and processing device, in particular a computer 3.

[0160] A wizard runs on its user interface, which wizard guides a user through one or more of the processes described below:

[0161] In a first step of a method according to one embodiment of the present invention (FIG. 3: S10), a robot parameter and a boot configuration are recorded. In order to carry out motion learning advantageously in a simulation environment, both the robot parameter and the environmental model parameter should be available as precisely as possible in the cloud simulation environment.

[0162] With the aid of a so-called asset administration shell (AAS), also called a digital twin, status and management data of the robot 1 are stored. An OPC UA information model is advantageously used for this purpose. In the asset administration shell of the robot, data such as the robot model, operating hours, current axis values (to ascertain a starting position), attached tools, etc. are available and are transferred to the cloud simulation environment. The simulation environment can configure the simulation with regard to the robot therefrom (CAD model, dynamic parameters, tools, current axis configuration, possibly changed dynamic parameters due to service life, etc.)

[0163] In a second step (FIG. 3: S20), the environmental model is recorded. In one embodiment, there are multiple options to choose from: [0164] Transferring a fully modeled CAD model including transformation to the robot coordinate system; [0165] Recording the environment by a 3D camera that is either manually-guided by a human or is mounted on the robot, which is [0166] is manually-guided or [0167] follows a defined and collision-free trajectory

[0168] In the case of manual guidance, it is also possible to record regions that are important for the task, for example a joining target, more precisely and from a short distance.

[0169] The environmental model generated in this way is now also transferred to the cloud simulation environment. A simple option here is to also store the data in the asset administration shell of the robot.

[0170] In one modification, the robot cell has an asset administration shell 10 (cf. FIG. 2), the environmental model and references to other involved asset administration shells. This means that the robot itself is interchangeable and as a whole, it has a more modular structure than if all the information is in the asset administration shell of the robot itself. The “cell manager” can then regulate the interaction with the subcomponents, the simulation environment 20 (cf. FIG. 2) and the execution of the learning process.

[0171] In a third step (FIG. 3: S30), the learning target is defined. A cost function is specified so that the reinforcement algorithm knows its target. In particular, it is possible in the guided wizard to specify the target, for example, by the user manually guiding the robot to the joining target and repeating this a few times in order to minimize errors.

[0172] A manual demonstration is also used in one embodiment, depending on the reinforcement learning algorithm, to initialize the algorithm or to carry out inverse reinforcement learning of the cost function. The trajectories of the demonstrations can also be stored in the asset administration shell.

[0173] In a fourth step (FIG. 3: S40), the task is learned in the cloud environment 4, preferably in parallel, using the deep reinforcement learning method.

[0174] The specific algorithm is advantageously [0175] guided policy search; [0176] soft Q-learning; [0177] A3C

[0178] or the like.

[0179] In order to overcome the simulation-reality gap, the dynamic parameters are randomized in one embodiment. If a vision system is involved, a flexible vision model is learned by domain randomization.

[0180] A geometric path planner can plan non-contact path elements and, in the case of guided policy search, initialize the linear quadratic Gaussian controllers.

[0181] The result of the algorithm is the structure of the neural network and the trained weights of a neural network. As a modification, progressive nets can be used for later fine-tuning. The results of the simulation are sent back to the robot/edge controller.

[0182] In a fifth step (FIG. 3: S50), the model is downloaded to the robot or an edge controller.

[0183] The trained model can now be played back. In the asset administration shell of the simulation instance, parameters of the simulation and the learning algorithm can also be provided (e.g. learning rate, number of iterations, etc., which can later be used for fine-tuning). In particular, the ONNX exchange format, for example, can be used for exchanging the computation graph and the weights.

[0184] In an optional sixth step (FIG. 3: S60), the model is fine-tuned on the real system.

[0185] Depending on the quality of the simulation, the model is immediately ready-to-use or is further fine-tuned on the real system. In other words, the reinforcement learning algorithm is further trained on the real system, wherein initialization using the weights and other parameters of the reinforcement algorithm is advantageous.

[0186] In a seventh step (FIG. 3: S70), the learned task can now be carried out.

[0187] FIG. 4 shows a method according to a further embodiment of the present invention which can be carried out with the system of FIG. 1.

[0188] In one step S100, a random generator 3a (cf. FIG. 1) provided in this embodiment and therefore indicated by dashed lines is used to generate a stochastic value of a robot parameter and/or an environmental model parameter, for example a two-dimensional target position of a connector 1a guided by a robot in this embodiment in a surface 10 (cf. FIG. 1), on the basis of a specified stochastic parameter model, in the embodiment a Gaussian distribution specified by a user using the wizard.

[0189] FIG. 5 shows by way of example how this robot parameter and this stochastic parameter model are visualized in an image of this joining application by a marked region in the form of a circle around the mean or expected value of the Gaussian distribution for the two-dimensional target position in the (image of the) surface. The edge of the circle visualizes a maximum value of a deviation from the mean or expected value and the different coloring indicated by different hatching, in one embodiment different (color) brightness, of the marked region visualizes the respective probability that the target position is at this point.

[0190] In one step S200, the application is simulated on the basis of the ascertained stochastic value, i.e. with a stochastic target position, on the computer 3 or in the cloud 4.

[0191] A control agent is trained by means of reinforcement learning.

[0192] In one step S300, it is checked whether the control agent has already been sufficiently trained. If this is not the case (S300: “N”), the random number generator 3a generates a new stochastic target position with which a further simulation is carried out.

[0193] If the control agent has been sufficiently trained (S300: “Y”), the controller 2 of the robot 1 is configured on the basis of the trained control agent (S400).

[0194] In addition, a classification agent is trained using the simulations carried out, for example machine-learned anomaly or error detection (S500), i.e. the simulations carried out when training the control agent are used.

[0195] The real application is then carried out using the robot 1 with the controller configured in step S400 (S600) and thereby or subsequently classified by means of anomaly or error detection (S700). The configured control can thereby be further trained.

[0196] Although embodiments have been explained in the preceding description, it is noted that a large number of modifications are possible. It is also noted that the embodiments are merely examples that are not intended to restrict the scope of protection, the applications and the structure in any way. Rather, the preceding description provides a person skilled in the art with guidelines for implementing at least one embodiment, with various changes, in particular with regard to the function and arrangement of the described components, being able to be made without departing from the scope of protection as it arises from the claims and from these equivalent combinations of features.

[0197] While the present invention has been illustrated by a description of various embodiments, and while these embodiments have been described in considerable detail, it is not intended to restrict or in any way limit the scope of the appended claims to such de-tail. The various features shown and described herein may be used alone or in any combination. Additional advantages and modifications will readily appear to those skilled in the art. The invention in its broader aspects is therefore not limited to the specific details, representative apparatus and method, and illustrative example shown and described. Accordingly, departures may be