DYNAMICS MODEL FOR GLOBALLY STABLE MODELING OF SYSTEM DYNAMICS

Abstract

A system and computer-implemented method are provided for training a dynamics model to learn the dynamics of a physical system. The dynamics model may be learned to be able to infer a future state of the physical system and/or its environment based on a current state of the physical system and/or its environment. The learned dynamics model is inherently globally stable. Instead of learning a dynamics model and attempting to separately verify its stability, the learnable dynamics model comprises a learnable Lyapunov function which is jointly learned together with the nominal dynamics of the physical system. The learned dynamics model is highly suitable for real-life applications in which a physical system may assume a state which was unseen during training as the learned dynamics model is inherently globally stable.

Claims

1. A machine learning system for training a dynamics model to learn dynamics of a physical system by learning to infer a future state of the physical system and/or its environment based on a current state of the physical system and/or its environment, the machine learning system comprising: an input interface configured for accessing: training data representing a time-series of states of the physical system and/or its environment; and model data defining a machine learnable dynamics model which includes a machine learnable Lyapunov function; and a processor subsystem configured to learn the dynamics model based on time-sequential pairs of states so as to learn to infer a future state of the physical system and/or its environment based on the current state, wherein the learning is constrained to provide a globally stable modelling of dynamics of the physical system by jointly learning the dynamics model and the Lyapunov function so that values of the learned Lyapunov function decrease along all trajectories of states inferred by the learned dynamics model.

2. The machine learning system according to claim 1, wherein the processor subsystem is configured to jointly learn the dynamics model and the Lyapunov function by learning, as the dynamics model, a projection of nominal dynamics of the physical system, which are learned based on the time-series of states, onto a function that fulfills a Lyapunov condition as defined by the Lyapunov function.

3. The machine learning system according to claim 2, wherein the projection is an orthogonal projection onto a halfspace.

4. The machine learning system according to claim 2, wherein the dynamics model includes a neural network for learning the nominal dynamics of the physical system based on the time-sequential pairs of states.

5. The machine learning system according to claim 1, wherein the machine learnable Lyapunov function is represented at least in part by an input-convex neural network.

6. The machine learning system according to claim 5, wherein the input-convex neural network includes a smoothed variant of a rectified linear unit for each activation function.

7. The machine learning system according to claim 5, wherein the Lyapunov function is shifted to establish a zero value of the Lyapunov function at an equilibrium point of the Lyapunov function, and wherein the learning of the Lyapunov function includes using a quadratic regularization term to ensure strict positive definitiveness of the Lyapunov function.

8. The machine learning system according to claim 5, wherein the processor subsystem is configured to, when learning the input-convex neural network on the training data, apply a continuously differentiable invertible function to the training data and use an output of the continuously differential invertible function as an input to the input-convex neural network.

9. A non-transitory computer-readable medium on which is stored model data defining a machine learnable dynamics model which includes a machine learnable Lyapunov function, wherein the dynamics model is learned to infer a future state of the physical system and/or its environment based on a current state, wherein the learning of the dynamics model has been constrained to provide a globally stable modelling of dynamics of the physical system by the dynamics model and the Lyapunov function having been jointly learned so that values of the learned Lyapunov function decrease along all trajectories of states inferred by the learned dynamics model.

10. A computer-implemented method for training a dynamics model to learn dynamics of a physical system by learning to infer a future state of the physical system and/or its environment based on a current state of the physical system and/or its environment, the method comprising the following steps: accessing training data representing a time-series of states of the physical system and/or its environment, and model data defining a machine learnable dynamics model which includes a machine learnable Lyapunov function; learning the dynamics model based on time-sequential pairs of states so as to learn to infer a future state of the physical system and/or its environment based on the current state, wherein the learning includes: constraining the learning to provide a globally stable modelling of the dynamics of the physical system by jointly learning the dynamics model and the Lyapunov function so that values of the learned Lyapunov function decrease along all trajectories of states inferred by the learned dynamics model.

11. A computer-implemented method for controlling or monitoring a physical system using a machine learned modelling of dynamics of the physical system, the method comprising the following steps: accessing: sensor data representing an observation of a current state of the physical system; and model data defining a machine learned dynamics model which includes a machine learnable Lyapunov function, wherein the dynamics model is learned to infer a future state of the physical system and/or its environment based on the current state, wherein the learning of the dynamics model has been constrained to provide a globally stable modelling of the dynamics of the physical system by the dynamics model and the Lyapunov function having been jointly learned so that values of the learned Lyapunov function decrease along all trajectories of states inferred by the learned dynamics model; inferring the future state of the physical system based on the observation of the current state of the physical system as obtained from the sensor data; and using an output interface to an output device which is used in the control or monitoring of the physical system, to provide output data to the output device to enable the control or monitoring of the physical system based on the inferred future state.

12. A non-transitory computer-readable medium on which is stored data representing instructions for training a dynamics model to learn dynamics of a physical system by learning to infer a future state of the physical system and/or its environment based on a current state of the physical system and/or its environment, the instructions, when executed by a processor system, causing the processor system to perform the following steps: accessing training data representing a time-series of states of the physical system and/or its environment, and model data defining a machine learnable dynamics model which includes a machine learnable Lyapunov function; learning the dynamics model based on time-sequential pairs of states so as to learn to infer a future state of the physical system and/or its environment based on the current state, wherein the learning includes: constraining the learning to provide a globally stable modelling of the dynamics of the physical system by jointly learning the dynamics model and the Lyapunov function so that values of the learned Lyapunov function decrease along all trajectories of states inferred by the learned dynamics model.

13. A control or monitoring system for controlling or monitoring a physical system using a machine learned modelling of dynamics of the physical system, the control or monitoring system comprising: an input interface for accessing: sensor data representing an observation of a current state of the physical system; and model data defining a machine learned dynamics model which includes a machine learnable Lyapunov function, wherein the dynamics model is learned to infer a future state of the physical system and/or its environment based on the current state, wherein the learning of the dynamics model has been constrained to provide a globally stable modelling of dynamics of the physical system by the dynamics model and the Lyapunov function having been jointly learned so that values of the learned Lyapunov function decrease along all trajectories of states inferred by the learned dynamics model; an output interface to an output device which is used in the control or monitoring of the physical system; and a processor subsystem configured to: infer the future state of the physical system based on the observation of the current state of the physical system as obtained from the sensor data; and using the output interface, provide output data to the output device to enable the control or monitoring of the physical system based on the inferred future state.

14. The control or monitoring system according to claim 13, wherein the output device is an actuator associated with the physical system, and wherein the processor subsystem is configured to control the physical system by providing control data) to the actuator which is based on the inferred future state of the physical system.

15. The control or monitoring system according to claim 14, wherein the system is one of: a vehicle control system, a robotics control system, a manufacturing control system, and a building control system.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0031] These and other aspects of the present invention will be apparent from and elucidated further with reference to the embodiments described by way of example in the following description and with reference to the figures.

[0032] FIG. 1 shows an example system for training a learnable dynamics model to learn dynamics of a physical system based on training data which comprises states of the physical system and/or its environment, wherein the dynamics model comprises a learnable Lyapunov function to establish globally stability of the learned dynamics model, in accordance with an example embodiment of the present invention.

[0033] FIG. 2 shows an example computer-implemented method for training the learnable dynamics model to learn the dynamics of the physical system based on the training data, in accordance with an example embodiment of the present invention.

[0034] FIGS. 3A-3C each show a contour plot of a Lyapunov function of a stable dynamics system, in which FIG. 3A illustrates the trajectory of the Lyapunov function, FIG. 3B illustrates {circumflex over ()}(x) having a component g(x) not in the half space and which may be subtracted to obtain (x), and FIG. 3C illustrates {circumflex over ()}(x) being in the half space and returned unchanged.

[0035] FIG. 4A illustrates a rectified linear unit activation function.

[0036] FIG. 4B illustrates a smoothed variant of the rectified linear unit activation function.

[0037] FIGS. 5A shows a contour plot of the dynamics of a simple damped pendulum as simulated from first principles.

[0038] FIG. 5B shows a contour plot of the learned stable dynamics model.

[0039] FIG. 5C shows a contour plot of the learned Lyapunov function.

[0040] FIG. 6A shows the prediction error of the angular position and angular velocity for an 8-link pendulum as a function of timesteps.

[0041] FIG. 6B shows the average prediction error over 1000 timesteps as a function of a number of links of the pendulum.

[0042] FIG. 7A shows a structure of a video texture generation network in which the encoder e and decoder d form a Variational Autoencoder, and the stable dynamics model f is trained together with the decoder d to predict a next frame in a video texture.

[0043] FIG. 7B shows trajectories projected onto a two-dimensional plane for, from left to right, three different runs of the dynamics model with the stability constraint and for a dynamics model without the stability constraint in the form of a generic neural network.

[0044] FIG. 8 shows system for control or monitoring using learned dynamics model.

[0045] FIG. 9 shows the system as part of an (semi-)autonomous vehicle. FIG. 10 shows a computer-readable medium comprising data.

[0046] It should be noted that the figures are purely diagrammatic and not drawn to scale. In the figures, elements which correspond to elements already described may have the same reference numerals.

LIST OF REFERENCE NUMBERS

[0047] The following list of reference numbers is provided for facilitating understanding of the figures and shall not be construed as limiting the present invention.

[0048] 20 sensor

[0049] 22 camera

[0050] 40 actuator

[0051] 42 electric motor

[0052] 60 environment

[0053] 80 (semi)autonomous vehicle

[0054] 100 system for training dynamics model

[0055] 160 processor subsystem

[0056] 180 data storage interface

[0057] 190 data storage

[0058] 192 training data

[0059] 194 model data

[0060] 196 model data

[0061] 200 method for training dynamics model

[0062] 210 accessing training data

[0063] 220 accessing model data defining dynamics model

[0064] 230 learning dynamics model based on training data

[0065] 240 joint learning of dynamics model with Lyapunov function

[0066] 300-304 contour plot of Lyapunov function of stable dynamics system

[0067] 400 rectified linear unit activation function

[0068] 410 smoothed variant of rectified linear unit activation function

[0069] 500 streamplot of dynamics as simulated from first principles

[0070] 510 streamplot of dynamics of the learned dynamics model

[0071] 520 contour plot of the learned Lyapunov function

[0072] 600 prediction error as a function of timestamp for 8-link pendulum

[0073] 610 timestamp

[0074] 620 prediction error

[0075] 630 average error over 1000 timestamps as function of number of links

[0076] 640 number of links

[0077] 650 average prediction error

[0078] 660 simple learned dynamics model

[0079] 670 stable learned dynamics model

[0080] 700 structure of video texture generation network

[0081] 710 stable dynamics model run 1

[0082] 720 stable dynamics model run 2

[0083] 730 stable dynamics model run 3

[0084] 740 nave dynamics model

[0085] 800 system for control or monitoring using learned dynamics model

[0086] 820 sensor data interface

[0087] 822 sensor data

[0088] 840 actuator interface

[0089] 842 control data

[0090] 860 processor subsystem

[0091] 880 data storage interface

[0092] 890 data storage

[0093] 900 method for control or monitoring using learned dynamics model

[0094] 910 accessing sensor data

[0095] 920 accessing model data defining learned dynamics model

[0096] 930 inferring future state from observed current state using model

[0097] 940 providing output data based on inferred future state

[0098] 1000 computer-readable medium

[0099] 1010 non-transitory data

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0100] The following describes, with reference to FIGS. 1 and 2, the training of a learnable dynamics model which comprises a learnable Lyapunov function to establish global stability of the learned dynamics model, then describes with reference to FIGS. 3A-5C the dynamics model and its training in more detail, then describes with reference to FIG. 6A-7B applications of the dynamics model which include a comparison to a non-stable dynamics model, and with reference to FIGS. 8-10 the use of the learned dynamics model for the control or monitoring of a physical system, such as an (semi-)autonomous vehicle.

[0101] FIG. 1 shows an example system 100 for training, in accordance with the present invention, a dynamics model to learn dynamics of a physical system by learning to infer a future state of the physical system and/or its environment based on a current state of the physical system and/or its environment.

[0102] The system 100 may comprise an input interface for accessing training data 192 representing a time-series of states of the physical system and/or its environment and for accessing model data 194 defining a machine learnable dynamics model which comprises a machine learnable Lyapunov function. For example, as also illustrated in FIG. 1, the input interface may be constituted by a data storage interface 180 which may access the training data 192 and the model data 194 from a data storage 190. For example, the data storage interface 180 may be a memory interface or a persistent storage interface, e.g., a hard disk or an SSD interface, but also a personal, local or wide area network interface such as a Bluetooth, Zigbee or Wi-Fi interface or an ethernet or fiberoptic interface. The data storage 190 may be an internal data storage of the system 100, such as a hard drive or SSD, but also an external data storage, e.g., a network-accessible data storage. In some embodiments, the training data 192 and the model data 194 may each be accessed from a different data storage, e.g., via a different subsystem of the data storage interface 180. Each subsystem may be of a type as is described above for the data storage interface 180.

[0103] The system 100 may further comprise a processor subsystem 160 which may be configured to, during operation of the system 100, to learn the dynamics model based on time-sequential pairs of states so as to learn to infer a future state of the physical system and/or its environment based on the current state, wherein said learning is constrained to provide a globally stable modelling of the dynamics of the physical system by jointly learning the dynamics model and the Lyapunov function so that values of the learned Lyapunov function decrease along all trajectories of states inferred by the learned dynamics model. This dynamics model and its training will be further described with reference to FIG. 3A-5C.

[0104] The system 100 may further comprise an output interface for outputting trained model data 196 representing the learned (or trained) dynamics model. For example, as also illustrated in FIG. 1, the output interface may be constituted by the data storage interface 180, with said interface being in these embodiments an input/output (IO) interface, via which the trained model data 196 may be stored in the data storage 190. For example, the model data 194 defining the untrained dynamics model may during or after the training be replaced, at least in part, by the model data 196 of the trained dynamics model, in that the parameters of the dynamics model, such as weights and other types of parameters of neural networks, may be adapted to reflect the training on the training data 192. This is also illustrated in FIG. 1 by the reference numerals 194, 196 referring to the same data record on the data storage 190. In other embodiments, the trained model data 196 may be stored separately from the model data 194 defining the untrained dynamics model. In some embodiments, the output interface may be separate from the data storage interface 180, but may in general be of a type as described above for the data storage interface 180.

[0105] FIG. 2 shows an example computer-implemented method 200 in accordance with the present invention for training a dynamics model to learn dynamics of a physical system by learning infer a future state of the physical system and/or its environment based on a current state of the physical system and/or its environment. The method 200 may correspond to an operation of the system 100 of FIG. 1, but does not need to, in that it may also correspond to an operation of another type of system, apparatus or device or in that it may correspond to a computer program.

[0106] The computer-implemented method 200 is shown to comprise, in a step titled ACCESSING TRAINING DATA, accessing 210 training data representing a time-series of states of the physical system and/or its environment, in a step titled ACCESSING MODEL DATA DEFINING DYNAMICS MODEL, accessing 220 model data defining a machine learnable dynamics model which comprises a machine learnable Lyapunov function, in a step titled LEARNING DYNAMICS MODEL BASED ON TRAINING DATA, learning 230 the dynamics model based on time-sequential pairs of states so as to learn to infer a future state of the physical system and/or its environment based on the current state, wherein said learning step 230 comprises, in a step or sub-step titled JOINT LEARNING OF DYNAMICS MODEL WITH LYAPUNOV FUNCTION, constraining the learning to provide a globally stable modelling of the dynamics of the physical system by jointly learning 240 the dynamics model and the Lyapunov function so that values of the learned Lyapunov function decrease along all trajectories of states inferred by the learned dynamics model.

[0107] The following examples describe the machine learnable dynamics model, including the training thereof, in more detail. However, the actual implementation of the machine learnable dynamics model and its training may be carried out in various other ways, e.g., on the basis of analogous mathematical concepts. For example, instead of using a neural network, a Gaussian process or another machine learnable model type may be used to learn the nominal dynamics of the physical system and to learn the Lyapunov function.

[0108] Briefly speaking, the following deals with learning continuous time dynamical systems, but also applies mutatis mutandis to discrete time dynamical systems. That is, given a state at time t, x(t).sup.n, the time-derivative of the state is modelled as

[00003]$\begin{array}{cc}\stackrel{.}{x}\left(t\right)\frac{d}{\mathrm{dt}}.Math.x\left(t\right)=f\left(x\left(t\right)\right)& \left(1\right)\end{array}$

for some function .sup.n.fwdarw..sup.n: Modeling the time evolution of such dynamical systems (or their counterparts with control inputs {dot over (x)}(t)=(x(t), u(t)) for u(t).sup.m) is a foundational problem, with applications in reinforcement learning, control, forecasting, and many other settings. Owing to their representational power, neural networks have long been a natural choice for modeling the above function. However, when using a generic neural network to model dynamics in this setting, very little can be guaranteed about the behavior of the learned system. For example, it is very difficult to say anything about the stability properties of a learned model (informally, the tendency of the system to remain within some invariant bounded set). While some recent work has begun to consider stability properties of neural networks, see the aforementioned [1, 2, 3], such work has typically done so by (softly) enforcing stability as an additional loss term on the training data. Consequently, such work can say little about the stability of the system in unseen states.

[0109] Herein, an approach is described to learning neural network dynamics that are provably stable over the entirety of the state space. To do so, the system dynamics and a Lyapunov function may be jointly learned. This stability may be a hard constraint imposed upon the model. Accordingly, unlike recent approaches, stability may not be enforced via an imposed loss function but built directly into the dynamics of the model (e.g., even a randomly initialized model in the described model class will be provably stable everywhere in state space). For that purpose, a Lyapunov function may be provided, which may be based on an input convex neural network, and which may ensure global exponential stability to an equilibrium point while still allowing for expressive dynamics.

[0110] Using these methods, the following demonstrates learning the dynamics of physical models such as n-link pendulums, e.g., with reference to FIGS. 5A-6B, and shows a substantial improvement over generic networks. It is also shown how such dynamics models can be integrated into larger network systems to learn dynamics over complex output spaces. In particular, it is shown with reference to FIGS. 7A-7B how to combine the model with a variational auto-encoder (VAE) to learn dynamic video textures.

[0111] The following primarily considers the setting of autonomous dynamics {dot over (x)}(t)=(x(t)) for x(t).sup.n. While the techniques described in this specification are applicable to the dynamics with control as well, the following discussion focuses on the autonomous case for ease of explanation. A system is defined to be globally asymptotically stable (for simplicity, around the equilibrium point x.sub.e=0) if we have x(0.fwdarw.0 as t.fwdarw. for any initial state x(0).sup.n; is locally asymptotically stable if the same holds but only for x(0)B where B is some bounded set containing the origin. Similarly, is globally (locally, respectively) exponentially stable (e.g., converges to the equilibrium exponentially quickly) if

x(t).sub.2mx(0.sub.2e.sup.at (2)

for some constants m, a0 for any x(0).sup.n (B, respectively).

[0112] The area of Lyapunov theory establishes the connection between the various types of stability mentioned above and descent according to a particular type of function known as a Lyapunov function. Specifically, let V:.sup.n.fwdarw. be a continuously differentiable positive definite function, i.e., V(x)>0 for x0 and V(0)=0. Lyapunov analysis says that is stable (according to the definitions above) if and only if we can find some function V as above such the value of this function is decreasing along trajectories generated by . Formally, this is the condition that the time derivative {dot over (V)}(x(t))<0, i.e.,

[00004]$\begin{array}{cc}\stackrel{.}{V}\left(x\left(t\right)\right)=\frac{d}{\mathrm{dt}}.Math.V\left(x\left(t\right)\right)={V\left(x\right)}^T.Math.\frac{d}{\mathrm{dt}}.Math.x\left(t\right)={V\left(x\left(t\right)\right)}^T.Math.f\left(x\left(t\right)\right)<0& \left(3\right)\end{array}$

[0113] This condition should hold for all x(t).sup.n or for all x(t)B to ensure global or local stability respectively. Similarly, f is globally asymptotically stable if and only if there exists positive definite V such that

{dot over (V)}(x(t))aV(x(t)), with c.sub.1x.sub.2.sup.2V(x)c.sub.2x.sub.2.sup.2 (4)

[0114] Showing that these conditions imply the various forms of stability is relatively straightforward, but showing the converse (that any stable system must obey this property for some V) is relatively more complex. However, we are largely concerned with the simpler of these two directions, as the goal is to enforce conditions that ensure stability.

[0115] Briefly speaking, for an arbitrary (non-stable) nominal dynamics f and a given positive definite function V(x)0, V(0)=0, V(x)>0 for x0, we may ensure global asymptotic stability by projecting {circumflex over ()} to

[00005]$f\left(x\right)=\mathrm{Proj}\left(\hat{f}\left(x\right),\left\{f.Math.\text{:}.Math.{V\left(x\right)}^T.Math.f.Math..Math.V\left(x\right)\right\}\right)=\hat{f}\left(x\right)-V\left(x\right).Math.\frac{\mathrm{ReLU}\left({V\left(x\right)}^T.Math.\hat{f}\left(x\right)+.Math..Math.V\left(x\right)\right)}{{.Math.V\left(x\right).Math.}_2^2}$

where Proj(.,.) denotes the orthogonal projection onto a halfspace. For example, automatic differentiation tools may be used to define/implement V such that including V in the definition of can be done straightforwardly. Overall, the neural network for 71 may be trained straightforwardly, similar to the training for any other function.

[0116] The following provides a more detailed explanation of the above while referring to FIGS. 3A-7B. The intuition of at least some of the techniques described in this specification may be summarized as follows: instead of learning a dynamics function and attempting to separately verify its stability via a Lyapunov function, a dynamics model and Lyapunov function are jointly learned, where the modeling of the dynamics may be inherently constrained to be stable (everywhere in the state space) according to the Lyapunov function. Specifically, following the principles mentioned above, let {circumflex over ()}:.sup.n.fwdarw..sup.n denote a nominal dynamics model referring to a dynamics model which is directly learned on the training data, e.g., without incorporating the Lyapunov function, and let V:.sup.n.fwdarw. be a positive definite function: V(x)0 for x0 and V(0)=0. Then in order to (provably, globally) ensure that a dynamics function is stable, {circumflex over ()} may be projected such that it satisfies the condition

V(x).sup.T{circumflex over ()}(x)aV(x) (9)

i.e., one may define the dynamics as

[00006]$\begin{array}{cc}\begin{array}{cc}f\left(x\right).Math.=& .Math.\mathrm{Proj}\left(\hat{f}\left(x\right),\left\{f.Math.\text{:}.Math.{V\left(x\right)}^T.Math.f-.Math..Math.V\left(x\right)\right\}\right)\\ =& .Math.(\begin{array}{cc}\hat{f}\left(x\right).Math.& \mathrm{if}.Math..Math.{V\left(x\right)}^T.Math.\hat{f}\left(x\right)-.Math..Math.V\left(x\right)\\ \hat{f}\left(x\right)-V\left(x\right).Math.\frac{{V\left(x\right)}^T.Math.\hat{f}\left(x\right)+.Math..Math.V\left(x\right)}{{.Math.V\left(x\right).Math.}_2^2}& \mathrm{otherwise}.Math.\end{array}\\ =& .Math.\hat{f}\left(x\right)-V\left(x\right).Math.\frac{\mathrm{ReLU}\left({V\left(x\right)}^T.Math.\hat{f}\left(x\right)+.Math..Math.V\left(x\right)\right)}{{.Math.V\left(x\right).Math.}_2^2}\end{array}& \left(10\right)\end{array}$

where Proj(x;C) may denote the orthogonal projection of x onto the point C, and where the second equation follows from the analytical projection of a point onto a halfspace. V may for example be defined using automatic differentiation tools. Accordingly, the gradient V term may be included into the definition of , obtaining a composite neural network for (e.g., comprising a first neural network for {circumflex over ()} and a second neural network for the Lyapunov function) which may be trained just like any other function.

[0117] FIGS. 3A-3C provide an illustration of the above. Each of FIGS. 3A-3C shows a contour plot 300-304 of a Lyapunov function of a stable dynamics system, in which FIG. 3A illustrates the trajectory of the Lyapunov function by way of an arrow pointing in a direction of increasing value of the Lyapunov function. Let

[00007]$g\left(x\right)=\frac{V\left(x\right)}{{.Math.V\left(x\right).Math.}_2^2}.Math.\mathrm{ReLU}\left({V\left(x\right)}^T.Math.\hat{f}\left(x\right)+.Math..Math.V\left(x\right)\right).$

FIG. 3B illustrates {circumflex over ()}(x) having a component g(x) not in the half space and which may be subtracted to obtain (x), and FIG. 3C illustrates {circumflex over ()}(x) being in the half space and returned unchanged.

[0118] As indicated previously, V may need to be positive definite, but additionally V may be preferred to have no local optima except 0. This is due to Lyapunov decrease condition: one may desire to guarantee stability to the equilibrium point x=0, yet the decrease condition imposed upon the dynamics may mean that V may be decreasing along trajectories of . If V has a local optimum away from the origin, the dynamics may in theory, but may not need to in practice, get stuck in this location; this may manifest itself by the V(x).sub.2.sup.2 term going to zero, which may result in the dynamics becoming undefined at the optima. The above conditions may be enforced by design decisions as described below.

[0119] The Lyapunov function V may be learned and represented by a neural network. To ensure the required properties of V, the following considerations may apply. More specifically, one may prefer to ensure (i) that V has only one (global) optimum (without loss of generality assumed to be V(0)=0), (ii) that V is positive definite, e.g., V(x)>0 for x0 and (iii) that V is continuously differentiable.

[0120] Firstly, property (i) may be enforced by representing V with an input-convex neural network (ICNN) g, which may be defined by:

z.sub.1=.sub.0(W.sub.ox+b.sub.0) z.sub.i+=.sub.0(U.sub.iz.sub.i+W.sub.ix+b.sub.i), i=1, . . . k1g(x)z.sub.k (11)

where W.sub.i may be real-valued weights mapping from inputs to the i+1 layer activations; U.sub.i may be positive weights mapping previously layer activations z.sub.i to the next layer; b.sub.i may be real-valued biases; and .sub.i may be convex, monotonically non-decreasing non-linear activations, such as the ReLU or smooth variants thereof. With this formulation, g is convex in x, and any convex function can be approximated by such networks.

[0121] Secondly, it may be desirable to ensure that V is positive definite. Representing V with an ICNN makes V convex, i.e., there is only one optimum, however it does not enforce V(0)=0. There are multiple ways to ensure V(0)=0. For example, one could find the global optimum of g and shift it or by removing the biases terms. However, shifting requires finding the optimum during training, which may be computationally expensive while removing the biases may put limitations on the class of variables that can be approximated. As an alternative approach, g may be shifted to ensure V(0)=0, while a little quadratic regularization term may be added to ensure positive definiteness, e.g., as:

V(x)=.sub.k+1(g(x)g(0))+x|.sub.2.sup.2 (12)

where .sub.k may be a positive, convex, non-decreasing function with .sub.k(0)=0 and is a small constant, for example selected such that

[00008]$\mathrm{.Math.}>\underset{x}{\max }.Math..Math.\stackrel{.}{g}\left(x\right).Math.\text{/}.Math.2.$

[0122] Thirdly, it may be desirable to ensure that V is continuously differentiable, for example by using a smoothed version 410 of the ReLU activations, as shown in FIG. 4B, that replaces the linear ReLU 400, as shown in FIG. 4A, by a quadratic function in [0,d].

[00009]$\begin{array}{cc}\left(x\right)=(\begin{array}{cc}0.Math.& \mathrm{if}.Math..Math.x0.Math.\\ x^2.Math.2.Math.d.Math.& \mathrm{if}.Math..Math.0<x<d.\\ x-d.Math..Math.2& \mathrm{otherwise}.Math.\end{array}& \left(13\right)\end{array}$

[0123] Additionally, warping to the input space may be used, such that the input to g is first mapped by any continuously differentiable, invertible function. Namely, although convexity may ensure that the Lyapunov function have no local optima, this may be a sufficient but not necessary condition, and indeed requiring a strongly convex Lyapunov function may impose too strict a requirement upon the learned dynamics. For this reason, the input to the ICNN function g(x) above may be optionally preceded by any continuously differentiable invertible function F:.sup.n.sup.n, e.g., using

V(x)=.sub.k+1(g(F(x))g(F(0)))+x.sub.2.sup.2. (14)

as the Lyapunov function. Invertibility may ensure that the sublevel sets of V (which are convex sets, by definition) may map to contiguous regions of the composite function gF, thus ensuring that no local optima exist in this composed function.

[0124] The above-described dynamics model may be demonstrated on several example problems described with reference to FIGS. 5A-7B. FIGS. 5A-5C relate to the modelling of a physically-based dynamical system, specifically the n-link pendulum. A damped, rigid n-link pendulum's state x may be described by the angular position .sub.i and angular velocity {dot over ()}.sub.i of each link i. In the following, {circumflex over ()} may be a 2n-100-100-2n fully connected neural network, and V a 2n-60-60-1 ICNN, with properties described previously. The models are trained with pairs of data (x, {dot over (x)}) produced by the symbolic algebra solver sympy.

[0125] In FIG. 5A, the simulated dynamics for the case of a simple damped pendulum (i.e., with n=1) are shown by a streamplot 500, while FIG. 5B shows a streamplot 510 corresponding to the dynamics learned by the dynamics model and with FIG. 5C showing a contour plot 520 of the learned Lyapunov function. As can be seen, the dynamics may be learned to accurately predict motion of the pendulum even over long time periods.

[0126] The learned dynamics may also be evaluated quantitatively while varying n and the time horizon of simulation. FIG. 6A shows a graph 600 showing the prediction error 620 of the angular position and angular velocity {dot over ()} for an 8-link pendulum as a function of timesteps 610, while FIG. 6B shows a graph 630 showing the average prediction error 650 over 1000 timesteps as a function of a number of links 640 of the pendulum. Both graphs 600, 630 show the errors for both a simple prior art model 660 and a learned dynamics model 670 of a type as described in this specification. While both the simple and the stable models show increasing mean error at the start of the trajectory, the dynamics model 670 is able to capture the contraction in the physical system (implied by conservation of energy) and in fact exhibits decreasing error towards the end of the simulation (true and simulated dynamics are both stable). In comparison, the error in the simple model increases.

[0127] FIGS. 7A-7B relate to another example in which the dynamics model as described in this specification may be used for stable video texture generation, for example using a Variational Autoencoder (VAE) to learn an encoding for images and the stable dynamics model to learn the dynamics in encoding-space. Given a sequence of frames (y.sub.0, y.sub.1, . . . ), the overall network may be fed a frame at time t and may be trained to reconstruct the frames at time t and t+1. Specifically, a VAE may be defined by the encoder e:y.fwdarw..sup.2n giving mean and variance , log.sub.t.sup.2=e(y.sub.t), latent state z.sub.t .sup.n(.sub.t,.sub.t.sup.2) and decoder d:.sup.n.fwdarw.y, y.sub.td(z.sub.t). The overall network may be trained to minimize both the standard VAE loss (reconstruction error plus a KL divergence term), but also to minimize the reconstruction loss of a next predicted state. The evolution of the latent dynamics may be modeled at z.sub.t+1(z.sub.t), or more precisely y.sub.t+1d((z.sub.t)). In other words, as also illustrated in FIG. 7A which shows a structure of the video texture generation network in which the encoder e and decoder d form a Variational Autoencoder and in which the stable dynamics model f is trained together with the decoder d to predict a next frame in a video texture, the overall network may be trained minimize:

[0128] minimize.sub.e,d,{circumflex over ()},V.sub.t=1.sup.T1(KL()(.sub.t, .sub.t.sup.2I(0, I)+E.sub.z[d(z.sub.t)y.sub.t.sub.2.sup.2+d(z.sub.t))y.sub.t+1.sub.2.sup.2])

[0129] The overall network may be trained on pairs of successive frames sampled from videos. To generate video textures, the dynamics model may be seeded with the encoding of a single frame and the dynamics model may be numerically integrated to obtain a trajectory. The VAE decoder may convert each step of the trajectory into a frame.

[0130] FIG. 7B shows trajectories projected onto a two-dimensional plane for, from left to right, three different runs of the dynamics model with the stability constraint and for a dynamics model without the stability constraint in the form of a generic neural network. In this example, the true latent space is 320-dimensional, and the trajectories may be projected onto a two-dimensional plane for display. For the non-stable model, the dynamics quickly diverge and produce a static image, whereas for the stable model, it is possible to generate different (stable) trajectories that keep generating realistic images over long time horizons.

[0131] FIG. 8 shows a system 800 for controlling or monitoring a physical system using a machine learned modelling of dynamics of the physical system. The system 800 may comprise an input interface 880 for accessing trained model data 196 representing a learned dynamics model as may be generated by the system 100 of FIG. 1 or the method 200 of FIG. 2 or as described elsewhere. For example, as also illustrated in FIG. 8, the input interface may be constituted by a data storage interface 880 which may access the trained model data 196 from a data storage 890. In general, the input interface 880 and the data storage 890 may be of a same type as described with reference to FIG. 1 for the input interface 180 and the data storage 190. FIG. 8 further shows the data storage 892 comprising sensor data 822 representing an observation of a current state of the physical system. As described elsewhere, in some embodiments, the sensor data 822 may also be received directly from a sensor 20 via a sensor interface 820 of via another type of interface instead of being accessed from the data storage 890 via the data storage interface 880.

[0132] The system 800 may further comprise a processor subsystem 860 which may be configured to, during operation of the system 800, infer the future state of the physical system based on the observation of the current state of the physical system as obtained from the sensor data 822, and using an output interface as described elsewhere, provide output data 842 to an output device to enable the control or monitoring of the physical system based on the inferred future state, as also described elsewhere in this specification.

[0133] It will be appreciated that the same considerations and implementation options apply for the processor subsystem 860 as for the processor subsystem 860 of FIG. 1. It will be further appreciated that the same considerations and implementation options may in general apply to the system 800 as for the system 100 of FIG. 1, unless otherwise noted.

[0134] FIG. 8 further shows various optional components of the system 800. For example, in some embodiments, the system 800 may comprise a sensor data interface 820 for directly accessing sensor data 822 acquired by a sensor 20 in an environment 60. The sensor 20 may but does not need to be part of the system 800. The sensor 20 may have any suitable form, such as an image sensor, a lidar sensor, a radar sensor, a pressure sensor, a contain temperature sensor, etc. In some embodiments, the sensor data 822 may sensor measurements of different physical quantities in that it may be obtained from two or more different sensors sensing different physical quantities. The sensor data interface 820 may have any suitable form corresponding in type to the type of sensor, including but not limited to a low-level communication interface, e.g., based on I2C or SPI data communication, or a data storage interface of a type as described above for the data storage interface 880.

[0135] In some embodiments, the system 800 may comprise an actuator interface 840 for providing control data 842 to an actuator 40 in the environment 60. Such control data 842 may be generated by the processor subsystem 860 to control the actuator 40 based on one or more inferred future states of the physical system, which may be generated using the learned dynamics model. For example, the actuator may be an electric, hydraulic, pneumatic, thermal, magnetic and/or mechanical actuator. Specific yet non-limiting examples include electrical motors, electroactive polymers, hydraulic cylinders, piezoelectric actuators, pneumatic actuators, servomechanisms, solenoids, stepper motors, etc. Such type of control is described with reference to FIG. 9 for an (semi-)autonomous vehicle.

[0136] In other embodiments (not shown in FIG. 8), the system 800 may comprise an output interface to a rendering device, such as a display, a light source, a loudspeaker, a vibration motor, etc., which may be used to generate a sensory perceptible output signal which may be generated based on one or more inferred future states of the physical system. The sensory perceptible output signal may be directly indicative of the inferred future states of the physical system, but may also represent a derived sensory perceptible output signal, e.g., for use in guidance, navigation or other type of control of the physical system.

[0137] In general, each system described in this specification, including but not limited to the system 100 of FIG. 1 and the system 800 of FIG. 8, may be embodied as, or in, a single device or apparatus, such as a workstation or a server. The device may be an embedded device. The device or apparatus may comprise one or more microprocessors which execute appropriate software. For example, the processor subsystem of the respective system may be embodied by a single Central Processing Unit (CPU), but also by a combination or system of such CPUs and/or other types of processing units. The software may have been downloaded and/or stored in a corresponding memory, e.g., a volatile memory such as RAM or a non-volatile memory such as Flash. Alternatively, the processor subsystem of the respective system may be implemented in the device or apparatus in the form of programmable logic, e.g., as a Field-Programmable Gate Array (FPGA). In general, each functional unit of the respective system may be implemented in the form of a circuit. The respective system may also be implemented in a distributed manner, e.g., involving different devices or apparatuses, such as distributed local or cloud-based servers. In some embodiments, the system 800 may be part of vehicle, robot or similar physical entity, and/or may be represent a control system configured to control the physical entity.

[0138] FIG. 9 shows an example of the above, in that the system 800 is shown to be a control system of an (semi-)autonomous vehicle 80 operating in an environment 60. The autonomous vehicle 80 may be autonomous in that it may comprise an autonomous driving system or a driving assistant system, with the latter also being referred to as a semiautonomous system. The autonomous vehicle 80 may for example incorporate the system 800 to control the steering and the braking of the autonomous vehicle based on sensor data obtained from a video camera 22 integrated into the vehicle 80. For example, the system 800 may control an electric motor 42 to perform (regenerative) braking in case the autonomous vehicle 80 is expected to collide with a traffic participant. The system 800 may control the steering and/or braking to avoid collision with the traffic participant. For that purpose, the system 800 may infer a future state of the vehicle with respect its environment, including the traffic participant, based on the sensor data obtained from the video camera. If the future state of the vehicle, e.g., its future position relative to the traffic participant, is expected to result in a collision, the system 800 may take corresponding action.

[0139] FIG. 10 shows a computer-implemented method 900 for controlling or monitoring a physical system using a machine learned modelling of dynamics of the physical system. The method 900 may correspond to an operation of the system 800 of FIG. 8, but may alternatively also be performed using or by any other system, apparatus or device.

[0140] The example method 900 is shown to comprise, in a step titled ACCESSING SENSOR DATA, accessing 910 sensor data representing an observation of a current state of the physical system, in a step titled ACCESSING MODEL DATA DEFINING LEARNED DYNAMICS MODEL, accessing 920 model data defining a machine learned dynamics model as described elsewhere in this specification, e.g., as obtained from the system 100 or method 200 of FIGS. 1-2, in a step titled INFERRING FUTURE STATE FROM OBSERVED CURRENT STATE USING MODEL, inferring 930 the future state of the physical system based on the observation of the current state of the physical system as obtained from the sensor data, and in a step titled PROVIDING OUTPUT DATA BASED ON INFERRED FUTURE STATE, using an output interface to an output device which is used in the control or monitoring of the physical system, providing 940 output data to the output device to enable the control or monitoring of the physical system based on said inferred future state.

[0141] It will be appreciated that, in general, the operations or steps of the computer-implemented methods 200 and 800 of respectively FIGS. 2 and 10 may be performed in any suitable order, e.g., consecutively, simultaneously, or a combination thereof, subject to, where applicable, a particular order being necessitated, e.g., by input/output relations.

[0142] Each method, algorithm or pseudo-code described in this specification may be implemented on a computer as a computer implemented method, as dedicated hardware, or as a combination of both. As also illustrated in FIG. 11, instructions for the computer, e.g., executable code, may be stored on a computer readable medium 1000, e.g., in the form of a series 1010 of machine-readable physical marks and/or as a series of elements having different electrical, e.g., magnetic, or optical properties or values. The executable code may be stored in a transitory or non-transitory manner. Examples of computer readable mediums include memory devices, optical storage devices, integrated circuits, servers, online software, etc. FIG. 11 shows an optical disc 1010. In an alternative embodiment, the computer readable medium 1000 may comprise trained model data 1010 defining a machine learned dynamics model as described elsewhere in this specification.

[0143] Examples, embodiments or optional features, whether indicated as non-limiting or not, are not to be understood as limiting the present invention.

[0144] In accordance with an abstract of the specification, a system and computer-implemented method are provided for training a dynamics model to learn the dynamics of a physical system. In particular, the dynamics model may be learned to be able to infer a future state of the physical system and/or its environment based on a current state of the physical system and/or its environment. The learned dynamics model is inherently globally stable. Namely, instead of learning a dynamics model and attempting to separately verify its stability, the learnable dynamics model comprises a learnable Lyapunov function which is jointly learned together with the nominal dynamics of the physical system. Accordingly, the learned dynamics model is highly suitable for real-life applications in which a physical system may assume a state which was unseen during training as the learned dynamics model is inherently globally stable.

[0145] It should be noted that the above-mentioned embodiments illustrate rather than limit the present invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the present invention. Herein, use of the verb comprise and its conjugations does not exclude the presence of elements or stages other than those stated. The article a or an preceding an element does not exclude the presence of a plurality of such elements. Expressions such as at least one of when preceding a list or group of elements represent a selection of all or of any subset of elements from the list or group. For example, the expression, at least one of A, B, and C should be understood as including only A, only B, only C, both A and B, both A and C, both B and C, or all of A, B, and C. The present invention may be implemented using hardware comprising several distinct elements, and using a suitably programmed computer. In the case of the device being described as several elements, several of these elements may be embodied by one and the same item of hardware. The mere fact that certain measures are described separately does not indicate that a combination of these measures cannot be used to advantage.