DECODER, ENCODER, CONTROLLER, METHOD AND COMPUTER PROGRAM FOR UPDATING NEURAL NETWORK PARAMETERS USING NODE INFORMATION

Abstract

Disclosed is a decoder for decoding parameters of a neural network, configured to obtain a plurality of neural network parameters of the neural network on the basis of an encoded bitstream, to obtain, e.g. to receive; e.g. to extract from an encoded bitstream, a node information describing a node of a parameter update tree, wherein the node information has a parent node identifier, which is, for example, a unique parent node identifier, for example an integer number, a string, and/or a cryptographic hash, and wherein the node information has a parameter update information, e.g. one or more update instructions, for example a difference signal between initial neural network parameters and a newer version thereof, e.g. corresponding to a child node of the update tree, and to derive one or more neural network parameters using parameter information of a parent node identified by the parent node identifier and using the parameter update information.

Claims

1. A decoder for decoding parameters of a neural network, wherein the decoder is configured to acquire a plurality of neural network parameters of the neural network on the basis of an encoded bitstream; wherein the decoder is configured to acquire a node information describing a node of a parameter update tree, wherein the node information comprises a parent node identifier, and wherein the node information comprises a parameter update information; wherein the decoder is configured to derive one or more neural network parameters using parameter information of a parent node identified by the parent node identifier and using the parameter update information.

2. The decoder according to claim 1, wherein the decoder is configured to modify one or more neural network parameters defined by the parent node, which is identified by the parent node identifier, using the parameter update information.

3. The decoder according to claim 1, wherein the decoder is configured to set up a parameter update tree, wherein a plurality of child nodes comprising different parameter update information are associated with a common parent node.

4. The decoder according to claim 3, wherein the decoder is configured to acquire one or more neural network parameters associated with a currently considered node using the parameter update information associated with the currently considered node, using a parameter information associated with a root node and using parameter update information associated with one or more intermediate nodes which are between the root node and the currently considered node in the update tree.

5. The decoder according to claim 3, wherein the decoder is configured to traverse the parameter update tree from a root node to a currently considered node, and wherein the decoder is configured to apply update instructions of visited nodes to one or more initial neural network parameters, in order to acquire one or more neural network parameters associated with the currently considered node.

6. The decoder according to claim 3, wherein the decoder is configured to aggregate a plurality of consecutive nodes of the parameter update tree and/or one or more consecutive nodes of the parameter update tree and the parameter update information.

7. The decoder according to claim 3, wherein the decoder is configured to update the parameter update tree based on the node information.

8. The decoder according to claim 1, wherein the decoder is configured to decide to choose neural network parameters associated with a root node or to choose neural network parameters associated with one of the descendent nodes of the root node.

9. The decoder according to claim 1, wherein the parameter update information comprises an update instruction defining an addition of one or more change values to one or more parameter values associated with a parent node of a currently considered node and/or a subtraction of one or more change values from one or more parameter values associated with a parent node of a currently considered node; and wherein the decoder is configured to apply an addition or subtraction of the change values defined by the update instruction, in order to acquire one or more neural network parameters associated with the currently considered node.

10. The decoder according to claim 1, wherein the decoder is configured to determine whether a parent node identifier is present; and wherein the decoder is configured to derive one or more neural network parameters according to claim 1 if the parent node identifier is present, and wherein the decoder is configured to make the currently considered node the root node if the parent node identifier is not present.

11. The decoder according to claim 1, wherein the decoder is configured to compare the parent node identifier with hash values associated with one or more nodes, to identify the parent node of the currently considered node.

12. The decoder according to claim 11, wherein the hash values are hash values of a payload portion of a compressed data unit NDU associated with one or more previously decoded nodes, while leaving a data size information and a header information unconsidered.

13. The decoder according to claim 11, wherein the parent node identifier is a combined value representing a device identifier and a serial number of which both are associated with the parent node.

14. The decoder according to claim 1, wherein the node information comprises a node identifier.

15. The decoder according to claim 1, wherein the decoder is configured to store the node identifier.

16. The decoder according to claim 1, wherein the decoder is configured to compare one or more stored node identifiers with a parent node identifier in a node information of a new node when adding the new node, in order to identify a parent node of the new node.

17. The decoder according to claim 1, wherein the node identifier comprises a device identifier and/or a parameter update tree depth information and/or a parameter update tree identifier.

18. The decoder according to claim 1, wherein the node information comprises a signaling indicating whether a node identifier is present or not.

19. The decoder according to claim 10, wherein the decoder is configured to acquire a signaling, comprising an information about the type of the parent node identifier, and wherein the decoder is configured to evaluate the signaling in order to consider the respective type of the parent node identifier.

20. The decoder according to claim 1, wherein the decoder is configured to selectively evaluate a syntax element which indicates a type of the parent node identifier, in dependence on a syntax element indicating the presence of the parent node identifier.

21. An encoder for encoding parameters of a neural network in order to acquire an encoded bitstream, wherein the encoder is configured to provide a node information describing a node of a parameter update tree, wherein the node information comprises a parent node identifier, and wherein the node information comprises a parameter update information; wherein the parameter update information describes differences between neural network parameters associated with a parent node defined by the parent node identifier and current neural network parameters.

22. A method for decoding parameters of a neural network, the method comprising acquiring a plurality of neural network parameters of the neural network on the basis of an encoded bitstream; acquiring a node information describing a node of a parameter update tree, wherein the node information comprises a parent node identifier, and wherein the node information comprises a parameter update information; and deriving one or more neural network parameters using parameter information of a parent node identified by the parent node identifier and using the parameter update information.

23. A method for encoding parameters of a neural network in order to acquire an encoded bitstream, the method comprising providing a node information describing a node of a parameter update tree, wherein the node information comprises a parent node identifier, and wherein the node information comprises a parameter update information; wherein the parameter update information describes differences between neural network parameters associated with a parent node defined by the parent node identifier and current neural network parameters.

24. A non-transitory digital storage medium having stored thereon a computer program for performing the method for decoding parameters of a neural network according to claim 22, when the computer program is run by a computer.

25. A non-transitory digital storage medium having stored thereon a computer program for performing the method for encoding parameters of a neural network according to claim 23, when the computer program is run by a computer.

26. An encoded representation of neural network parameters, wherein the encoded representation comprises a node information describing a node of a parameter update tree, wherein the node information comprises a parent node identifier and a parameter update information.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0173] The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which:

[0174] FIG. 1a shows a schematic view of a decoder according to an embodiment of the present invention;

[0175] FIG. 1b shows a schematic view of a decoder with a generalized node information according to an embodiment of the present invention;

[0176] FIG. 2 shows a schematic view of an encoder according to an embodiment of the present invention;

[0177] FIG. 3a shows a schematic view of another encoder according to an embodiment of the present invention;

[0178] FIG. 3b shows a schematic view of an encoder with a generalized node information according to an embodiment of the present invention;

[0179] FIG. 4 shows a schematic view of a further encoder according to an embodiment of the present invention.

[0180] FIG. 5 shows an example, of a parameter update tree, PUT, according to embodiments of the invention;

[0181] FIG. 6 shows a schematic example of a tensor shape conversion according to embodiments of the invention;

[0182] FIG. 7 shows an example for a topology change of a neural network according to embodiments of the invention;

[0183] FIG. 8 shows a schematic view of a neural network controller according to embodiments of the invention;

[0184] FIG. 9 shows a schematic view of a neural network federated learning controller according to embodiments of the invention;

[0185] FIG. 10 shows a schematic block diagram of a method for decoding parameters of a neural network according to embodiments of the invention;

[0186] FIG. 11 shows a schematic block diagram of a method for encoding parameters of a neural network in order to obtain an encoded bitstream according to embodiments of the invention;

[0187] FIG. 12 shows a schematic block diagram of a method for controlling a neural network according to embodiments of the invention;

[0188] FIG. 13 shows a schematic block diagram of a method for controlling neural network federated learning according to embodiments of the invention;

[0189] FIG. 14 shows a schematic view of an example of federated learning scenario according to embodiments of the invention; and

[0190] FIG. 15 shows a schematic view of an example of parameter update tree, for example exemplary parameter update tree, according to embodiments of the invention.

DETAILED DESCRIPTION OF THE INVENTION

[0191] Equal or equivalent elements or elements with equal or equivalent functionality are denoted in the following description by equal or equivalent reference numerals even if occurring in different figures.

[0192] In the following description, a plurality of details is set forth to provide a more throughout explanation of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring embodiments of the present invention. In addition, features of the different embodiments described herein after may be combined with each other, unless specifically noted otherwise.

[0193] FIG. 1a shows a schematic view of a decoder according to an embodiment of the present invention. FIG. 1 shows decoder 100 comprising an obtaining unit 110, a parameter update tree, PUT, information unit 120 and a deriving unit 130.

[0194] The decoder 100 may receive an encoded bitstream 102, based on which, as an example, obtaining unit 110 may determine a node information 112. The node information 112 may describe a node of a parameter update tree, PUT. The node information 112 comprises a parent node identifier information 114, optionally comprising or for example being a parent node identifier, and a parameter update information 116.

[0195] Using the parent node identifier information 114, the PUT information unit 120 may be configured to determine a parameter information 122 of a parent node, identified by the parent node identifier information 114.

[0196] Using the parameter update information 116 and the parameter information 122, the deriving unit 130 may derive one or more neural network parameters 104.

[0197] In simple words, an information about neural network parameters may be provided using node information 112, comprising parent node identifier information 114 and parameter update information 116, encoded in the encoded bitstream 102. Hence, a parent node identifier and the parameter update information 116 may be extracted by obtaining unit 110. For example, instead of encoding the neural network parameters themselves, a reference information in the form of the parent node identifier information 114 and an update information in the form of the parameter update information 116 may be provided.

[0198] The reference information may be identified and/or extracted using PUT information unit 120, e.g. based on a parameter update tree. The idea according to embodiments may now be to extract neural network parameters of the parent node in the form of the parameter information 122 and to update this information using the parameter update information 116 in deriving unit 130.

[0199] In other words, neural network parameter(s) 104 of a current node may be derived using neural network parameters of a parent node of the current node that are modified by the parameter update information 116 using deriving unit 130.

[0200] As an optional feature, the parameter update information 116 may, for example, be provided to the PUT information unit 120.

[0201] As another optional feature, the PUT information unit 120 may be configured to provide a PUT information 124 to the deriving unit 130. The PUT information 124 may optionally comprise an information about the parameter update tree.

[0202] FIG. 1b. shows decoder 100b comprising an obtaining unit 110b, a PUT information unit 120b and a deriving unit 130b. The obtaining unit 110b may, for example, be configured to obtain a generalized node information 112b. Information 112b may optionally be equal or similar to node information 112, e.g. comprising a parent node identifier information and a parameter update information. However, generalized node information 112b may optionally comprise additional information, for example, such as a node identifier and/or a signaling whether a node identifier is present or not and/or a signaling comprising an information about a type of a parent node identifier, for example, in the form of a syntax element, a topology change information and/or a topology change signaling, as will be explained in detail henceforth.

[0203] Furthermore, PUT information unit 120b may provide a PUT information 132 to deriving unit 130b. PUT information 132 may, for example, be the parameter information of the parent node 122 as shown in FIG. 1. Optionally, update tree information 132 may comprise optional PUT information, e.g. 124 (referring to FIG. 1).

[0204] Accordingly, obtaining unit 110b may be configured to obtain or provide such a generalized node information 112b from an encoded bitstream 102, and the PUT information unit 120b may be configured to provide or determine the update tree information 132, e.g. based on a PUT, using the generalized node information 112b.

[0205] The deriving unit 130b, may hence be configured to derive one or more neural network parameters 104b using the generalized node information 112b and the PUT information 132.

[0206] FIG. 2 shows a schematic view of an encoder according to an embodiment of the present invention. FIG. 2 shows encoder 200 comprising a node information unit 210 and a bitstream unit 220.

[0207] As shown in FIG. 2 the encoder 200 may optionally receive neural network parameter(s) 204 and/or a parameter update tree, PUT, information 206. Based on the neural network parameter(s) 204 and the PUT information 206, node information unit 210 may provide a node information 212 describing a node of a parameter update tree. Node information 212 comprises a parent node identifier information 214, optionally comprising a parent node identifier, and a parameter update information 216, wherein the parameter update information 216 describes differences between neural network parameters associated with a parent node defined by the parent node identifier information 214 and current neural network parameters 204.

[0208] Using the node information 212 the bitstream unit 220 may provide an encoded bitstream 202, the bitstream comprising encoded neural network parameters 204.

[0209] As an example, as shown in FIG. 2, the idea of encoder 200 may be to encode neural network parameters(s) 204 not simply with their respective values, but using a reference and a differential or difference information with regard to the reference. The reference may be a set of neural network parameters identified by a parent node of a parameter update tree, indicated by parent node identifier information 214. The differential or difference information of the neural network parameters 204 with respect to neural network parameters of the parent node may be the parameter update information 216.

[0210] Therefore, encoded bitstream 202 may allow a corresponding decoder that may have an information about the parameter update tree available to identify a corresponding parent node, and hence neural network parameters thereof, to adapt or to modify those in order to determine neural network parameter(s) 204.

[0211] FIG. 3a shows a schematic view of another encoder according to an embodiment of the present invention. FIG. 3 shows encoder 300 comprising a parameter update tree, PUT, unit 310 and a bitstream unit 320.

[0212] In this example, encoder 300 may receive neural network parameter(s) 304. In order to encode the parameter(s) 304 encoder 300 may determine node information 312 comprising parent node identifier information 314, optionally. comprising a parent node identifier, and parameter update information 316, using PUT unit 310, for example, such that parameter(s) 304 may be represented as neural network parameters of a parent node (represented as the parent node identifier information 314) of a parameter update tree and a modification or update information (represented as the parameters update information 316).

[0213] As explained in the context of FIG. 2 bitstream unit 320 may encode the node information 312 in an encoded bitstream 302. As an example, encoder 300, e.g. in contrast to encoder 200 may be configured to provide node information 312 based on the parameters 304 as, for example only, input signal. Therefore, the PUT unit 310 may comprise a parameter update tree information in order to provide an alternative representation of the parameters 304 in the form of the update information 316 and an identifier.

[0214] FIG. 3b shows a schematic view of an encoder with a generalized node information according to an embodiment of the present invention. FIG. 3b shows encoder 300b comprising a parameter update tree, PUT, unit 310b and a bitstream unit 320b. PUT unit 310b may be configured to provide the generalized node information 312b based on or using neural network parameter(s) 304, wherein information 312b may optionally be equal or similar to node information 312, e.g. comprising a parent node identifier information and a parameter update information.

[0215] However, generalized node information 312b may optionally comprise additional information, for example, such as a node identifier and/or a signaling whether a node identifier is present or not and/or a signaling comprising an information about a type of a parent node identifier, for example, in the form of a syntax element, a topology change information and/or a topology change signaling, as will be explained in detail henceforth.

[0216] Using the generalized node information 312b the bitstream unit 320b may provide an encoded bitstream 302b.

[0217] FIG. 4 shows a schematic view of a further encoder according to an embodiment of the present invention. FIG. 4 shows encoder 400 comprising a PUT unit 410 and a bitstream unit 420.

[0218] As an optional example, PUT unit 410 may comprise a parameter update tree, e.g. as explained in the context of FIG. 5. The PUT unit 410 may provide a node information 412, comprising a parent node identifier information 414, optionally comprising a parent node identifier, and a parameter update information 416, that is provided to the bitstream unit 420 to obtain an encoded bitstream 402. Current neural network parameters e.g. associated with a specific node of the parameter update tree may hence be encoded in the form of the parent node identifier information 414, providing an information about reference parameters and of the parameter update information 416 providing an information on how to modify the reference parameters, in order represent the current neural network parameters associated with the node. Accordingly, the parameter update information 416 may describe differences between neural network parameters associated with a parent node defined by the parent node identifier and the current neural network parameters.

[0219] As an optional feature, encoder 400, e.g. PUT unit 410, may be configured to provide a plurality of node information blocks 418, wherein a parent node identifier of a first node information block refers to a root node and wherein a parameter update information of the first node describes differences between neural network parameters associated with the root node defined by the parent node identifier of the first node information block, and neural network parameters of the first node, and wherein a parent node identifier of a N-th node information block refers to a N-1-th node and wherein a parameter update information of the N-th node describes differences between neural network parameters associated with the N-1-th node defined by the parent node identifier of the N-th node information block, and neural network parameters of the N-th node.

[0220] In simple words, encoder 400 may be configured to provide, e.g. via encoded bitstream 402 an information about a respective parameter update tree.

[0221] FIG. 5 shows an example, of a parameter update tree, PUT, according to embodiments of the invention. FIG. 5 shows PUT 500 comprising a root node R 510 and a plurality of child nodes, to name some as examples 520, 530, 540, 550, 560.

[0222] As a simplified example, root node R, 510, may correspond to a set of four neural network parameters a.sub.11=a.sub.12=a.sub.21=a.sub.22=0, as an example, represented by a tensor 512. a.sub.11, a.sub.12, a.sub.21, a.sub.22 may, for example, be parameter values associated with the node R.

[0223] Hence, accordingly, a child node U1, 530, may correspond to a set of four neural network parameters, e.g. parameter values, a.sub.11=a.sub.12=a.sub.22=0 and a.sub.21=1, as an example, represented by a tensor 532.

[0224] The idea according to embodiments may now be the following: Instead of transmitting all four neural network parameters a.sub.11=a.sub.12=a.sub.22=0 and a.sub.21=1, only a reference information in the form of a parent node identifier 534 and a parameter update information 536 may be transmitted. In the simplified example of FIG. 5 the parameter update information 536 may be four change values, as an example represented by an additive tensor 536. In many applications, many NN parameters may not change drastically in between training cycles, hence as shown with tensor 536 many change values may be zero or quantized to zero. Hence such an update information may be encoded efficiently (e.g. compression of zeros).

[0225] Hence, if an encoder and a corresponding decoder both comprise an information about neural network parameters associated with root node R, 510, for example initial neural network parameters or default neural network parameters, then it may not be necessary to fully encode neural network parameters 532 associated with node U1, 530, e.g. neural network parameters after a first training step of a neural network, but only a reference information 534 and a difference information 536.

[0226] Hence, an encoder may transmit a node information in an encoded bitstream comprising parameter update information 536 and parent node identifier 534, such that a corresponding decoder may determine neural network parameters 532 as a sum, e.g. an elementwise-sum, of the neural network parameters 512 associated with the root node R, 510, identified by the parent node identifier 534, and the parameter update information 536, for example, such that tensor 512+tensor 536=tensor 532.

[0227] Accordingly, node U2, 520, may be associated with neural network parameters 522, wherein a parent node of U2 is node R, 510, such that parent node identifier 524 may be a pointer towards root node R, 510.

[0228] Node 540, associated with neural network parameters 542, may as well be a child node of node R, 510, such that parent node identifier 544 may as well be a pointer towards root node R, 510.

[0229] In contrast, nodes 550, 560 may be child nodes of node U2, 520, hence their parent node identifiers 554, 564 may identify U2, 520. U3, 550, may be associated with neural network parameters 552, and comprise parameter update information 556, with respect to its parent node U2, 520. Vice versa, U4, 560 may be associated with neural network parameters 562, and may comprise parameter update information 566, with respect to its parent node U2, 520.

[0230] The further elements of FIG. 5 will be explained in more detail step by step with regard to further embodiments of the invention. In the following further embodiments according to the invention are explained in the context of FIGS. 1 to 5.

[0231] Accordingly to the above explanation, as an optional feature, an encoder, e.g. 200, 300, 300b and/or 400, may be configured determine differences, e.g. using node information unit 210 or PUT unit 310, 310b, 410, between one more neural network parameters, e.g. 512, defined by the parent node, e.g. R, 510, which is identified by the parent node identifier, e.g. 534, and one or more current neural network parameters, e.g. 532, in order to obtain the parameter update information, e.g. 536.

[0232] As another optional feature, an inventive decoder, e.g. 100, 100b, e.g. PUT information unit 120, 120b, and/or an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to set up a parameter update tree 500, wherein a plurality of child nodes (to name only some, for example 520, 530, 540, 550, 560) comprising different parameter update information (e.g. 526, 536, 546, 556, 566) are associated with a common parent node, e.g. R, 510. Hence, as shown in FIG. 5 some nodes, e.g. U3, 550 and U4, 560, may be associated with the common parent node R via different, e.g. intermediate nodes, e.g. U2, 520.

[0233] As an example, an inventive PUT information unit, e.g. 120, 120b, may comprise the parameter update tree and/or may, for example, be configured to set the parameter update tree up. Therefore, the PUT information unit may, as explained before, optionally receive the parameter update information 116, in order to set up or to update a corresponding PUT.

[0234] An inventive encoder, e.g. 200, 300, 300b and/or 400, may, for example, be configured to set up a PUT using the node information unit 210 or a PUT unit 310, 310b and/or 410 respectively. These units may be configured to set up, store and/or update a PUT.

[0235] As another optional feature, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to provide the node information 212, 312, 312b and/or 412 (for example a node information comprising a parameter update information 556 and a parent node identifier 554), such that it is possible to obtain one or more neural network parameters, e.g. 552 associated with a currently considered node, e.g. U3, 550, using the parameter update information, e.g. 556, associated with the currently considered node, using a parameter information, e.g. 512, associated with a root node, e.g. R, 510, and using parameter update information, e.g. 526, associated with one or more intermediate nodes, e.g. U2, 520, which are between the root node and the currently considered node in the update tree.

[0236] Hence, as an example, in order to obtain neural network parameter associated with node U3, 550, the neural network parameters 512 of the root node R, 510 may be modified according to the tree path R-U2-U3 using parameter update information 526 and 556, such that tensor 512+tensor 526+tensor 556=tensor 552 (as an example, with an elementwise summation) representing the neural network parameters of node U3.

[0237] Accordingly, as an optional feature, an inventive decoder, e.g. 100, 100b, e.g. deriving unit 130, 130b, may be configured to obtain one or more neural network parameters, e.g. 104, 104b, for example corresponding to 552, associated with a currently considered node, e.g. U3, 550, using the parameter update information, e.g. 556, associated with the currently considered node, using a parameter information, e.g. 512, associated with a root node, e.g. R, 510, and using parameter update information, e.g. 526, associated with one or more intermediated nodes, e.g. U2, 520, which are between the root node and the currently considered node in the update tree.

[0238] Therefore, for example when the PUT information unit 120, 120b comprises an information about the PUT, PUT information unit 120, 120b may provide the PUT information 132 (or the optional information 124) comprising parameter update information, e.g. 526, and optionally parent node identifiers, e.g. 524, of intermediate nodes, e.g. U2, 520, to the deriving unit, e.g. 130, 130b, to obtain the one or more neural network parameters, e.g. 104, 104b for example corresponding to 552. In this case, as an example, encoded bitstream 102 may only comprise a parameter update information, e.g. 556, and a parent node identifier, e.g. 554, of the currently considered node (which may be used in conjunction with a PUT to determine a path through the PUT to determine the neural network parameters, e.g. 552, of the currently considered node, e.g. U3, 550).

[0239] As another, e.g. alternative, optional feature, the encoded bitstream 102 may comprise an information about the PUT, e.g. 500, and/or for example information about a path of the PUT, e.g. R-U2-U3, such that the parameter update information, e.g. 116, provided by obtaining unit, e.g. 110, 110b, may comprise the parameter update information, e.g. 556 and 526, of the currently considered node, e.g. U3, 560, and an intermediate node, e.g. U2, 520. Accordingly, parent node identifiers, e.g. 554 and 524, may be provided.

[0240] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400, may be configured to provide such an encoded bitstream 202, 302, 302b and/or 403 comprising a parameter update information and parent node identifiers of currently considered nodes and intermediate nodes.

[0241] Furthermore, as another optional feature, e.g. PUT information unit 120, 120b, comprising a PUT, e.g. 500, may, for example, be configured to traverse the parameter update tree, e.g. 500 (for example using PUT information unit 120, 120b), from a root node, e.g. R, 510, to a currently considered node, e.g. U3, 550, and to apply update instructions, e.g. 526, of visited nodes, e.g. U2, 520, to one or more initial neural network parameters, e.g. 512, in order to obtain one or more neural network parameters, e.g. 552, associated with the currently considered node, e.g. U3, 550.

[0242] As another optional feature, an inventive decoder, e.g. 100, 100b, e.g. PUT information unit 120, 120b, comprising a PUT, e.g. 500, may, for example, be configured to aggregate a plurality of consecutive nodes, e.g. nodes U2, 520, and U3, 550, of the parameter update tree, e.g. 500 and/or one or more consecutive nodes, e.g. nodes U2, 520, and U3, 550, of the parameter update tree and the parameter update information, e.g. 526, 556.

[0243] Hence, as shown in FIG. 5, nodes U2, 520, and U3, 550 may be aggregated or merged together to a new node U23, 540. Therefore, neural network parameters associated with U3, namely parameters 552 may be equal to neural network parameters associated with U23, namely parameters 542. Consequently, parameter update information 546 of node U23 may be a combination (in the simple example of FIG. 5 an elementwise sum) of parameter update information 526 and 556. Accordingly, parent node identifier 544 may point to the same node or may be equal to parent node identifier 524.

[0244] As another optional feature, an inventive decoder, e.g. 100, 100b, e.g. PUT information unit 120, 120b, may optionally be configured to update the parameter update tree, e.g. 500, based on the node information, e.g. 112, 112b. As an example, PUT information unit 120, 120b may optionally comprise the information about the parameter update tree, e.g. 500. The parameter update tree may be adapted by, for example, adding a new node, for example a node U4, 560, to the parameter update tree. This may comprise adding a corresponding parent node identifier, e.g. 564. Optionally, parameter update information 116 may be proved to PUT information unit 120, 120b as well, such that parameter update information 116, e.g. corresponding to tensor 566, may be added to the PUT as well. Hence, information about the new node, e.g. U4, 560, namely the parameter update information, e.g. 566, and the parent node identifier, e.g. 564, may be provided in the bitstream, e.g. 102, optionally, with a signaling to indicate that a new node is to be added. Alternatively, an inventive decoder, e.g. 100, may be configured to add such a node autonomously.

[0245] As another optional feature, an inventive decoder 100, 100b may be configured to decide to choose neural network parameters associated with a root node, e.g. R, 510, or to choose neural network parameters associated with one of the descendent nodes of the root node.

[0246] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example bitstream unit 220, 320, 320b, 420, may optionally be configured to provide a signaling, e.g. the encoded bitstream 202, 302, 302b, and/or 402, or a signal encoded in the encoded bitstream, to a decoder, e.g. 100, 100b, to selectively choose neural network parameters associated with a root node, e.g. R, 510, or neural network parameters associated with one of the descendent nodes of the root node.

[0247] As mentioned before, the elementwise sums of tensors, as explained above, may only be one e.g. simple example for the handling of parameter update information according to embodiments of the invention.

[0248] Parameter update tree 500 further comprises a node U5, 570, associated with parameter values, e.g. neural network parameters, 572, optionally in the form of a tensor, as shown in FIG. 5. Node U5, 570, is a child node of node U1, 530, as indicated by parent node information 574. As another optional feature, parameter update information 576 of node U5 comprises a scaling.

[0249] Hence, as another optional feature, the parameter update information 116, 216, 316 and/or 416 may, for example, comprise an update instruction defining a scaling of one or more parameter values associated with a parent node of a currently considered node.

[0250] Furthermore, an inventive decoder, e.g. 100, 100b, e.g. deriving unit 130, 130b, may optionally be configured to apply a scaling defined by the update instruction, e.g. 576, in order to obtain one or more neural network parameters, e.g. 104, 104b, for example corresponding to tensor 572, associated with the currently considered node, e.g. U5, 570, and correspondingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 or PUT unit 310, 310b, 410, may be configured to determine the scaling on the basis of one or more parameter values associated with a parent node, e.g. U1, 530 of the currently considered node and parameter values, e.g. 572, of a currently considered node, e.g. U5.

[0251] For explanatory purposes, in simple words, scaling 576 may indicate to double the parameter values 532 of parent node U1 in order to obtain the parameter values 572 of node U5.

[0252] Parameter update tree 500 further comprises a node U0, 580, associated with parameter values, e.g. neural network parameters, 582, optionally in the form of a tensor, as shown in FIG. 5. Node U0, 580, is a child node of node R, 510, as indicated by parent node information 584. As another optional feature, parameter update information 586 of node U0 comprises an additive change value, e.g. +3.

[0253] Hence, as an optional feature, the parameter update information 116, 216, 316 and/or 416 (and accordingly generalized node information 112b, 312b) may comprise an update instruction defining an addition of one or more change values, e.g. a change value 3, to one or more parameter values, e.g. a.sub.12 of tensor 512, associated with a parent node, e.g. R, 510, of a currently considered node, e.g. U0, 580, and/or a subtraction of one or more change values from one or more parameter values associated with a parent node of a currently considered node.

[0254] Therefore, an inventive decoder, e.g. 100, 100b, e.g. deriving unit 130, 130b, may optionally be configured to apply an addition or subtraction of the change values defined by the update instruction, in order to obtain one or more neural network parameters associated with the currently considered node.

[0255] As an optional example, a tensor subtraction is shown in FIG. 5 with parameter update information 566 of node U4, 560. Hence, embodiments may comprise additions or subtractions, e.g. elementwise additions or subtractions, e.g. in tensor or matrix form.

[0256] Parameter update tree 500 further comprises a node U8, 590, associated with parameter values, e.g. neural network parameters, 592, optionally in the form of a tensor, as shown in FIG. 5. Node U8, 590, is a child node of node U23, 540, as indicated by parent node information 594. As another optional feature, parameter update information 596 of node U8 is a product tensor.

[0257] In other words, optionally, a plurality of neural network parameters, e.g. 592, associated with a currently considered node, e.g. U8, may be represented by a parameter tensor and an inventive decoder, e.g. 100, e.g. deriving unit 130, 130b may optionally, be configured to apply a product tensor, e.g. 596, to a parameter tensor, e.g. 542, in order to obtain the parameter tensor, e.g. 592, associated with the currently considered node, e.g. 590, for example, as shown as a simple variant, using or performing or based on an elementwise multiplication of tensor elements.

[0258] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 or PUT unit 310, 310b, 410, may be configured to provide a product tensor, e.g. 596, for application to a parameter tensor, e.g. 542, in order to obtain a parameter tensor, e.g. 592, associated with the currently considered node, e.g. U8, 590.

[0259] Furthermore, as another optional feature, a plurality of neural network parameters associated with a parent node may be represented by a parameter tensor, and the parameter update information 116, 216, 316 and/or 416 (and accordingly generalized node information 112b, 312b) may optionally, comprise a product tensor, e.g. 596.

[0260] Moreover, an inventive decoder, e.g. 100, 100b, e.g. deriving unit 130, 130b, may optionally be configured to apply the product tensor, e.g. 596, to the parameter tensor, e.g. 542, of the parent node, e.g. 540, in order to obtain a parameter tensor, e.g. 592, associated with the currently considered node, e.g. U8, 590.

[0261] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400, for example node information unit 210 and/or PUT unit 310, 310b, 410, may be configured to provide the product tensor, e.g. 596, in such a manner, that an application of the product tensor to the parameter tensor, e.g. 542, of the parent node, e.g. 540, results in a parameter tensor, e.g. 592, associated with the currently considered node, e.g. U8, 590.

[0262] Parameter update tree 500 further comprises a node U7, 600, associated with parameter values, e.g. neural network parameters, 602, optionally in the form of a tensor, as shown in FIG. 5. Node U7, 600, is a child node of node U0, 580, as indicated by parent node information 604. As another optional feature, parameter update information 606 of node U8 is an update instruction defining a weighted combination of one or more parameter values, e.g. a.sub.12 of 582, associated with a parent node, e.g. U0, 580, of the currently considered node, e.g. U7, 600, with one or more change values.

[0263] Hence, optionally, the parameter update information, e.g. 116, 216, 316 and/or 416 (and accordingly generalized node information 112b, 312b), may comprise an update instruction, e.g. 606, defining a weighted combination of one or more parameter values, e.g. a.sub.12 of 582, associated with a parent node, e.g. U0, 580, of the currently considered node, e.g. U7, 600, with one or more change values.

[0264] Accordingly, an inventive decoder, e.g. 100, 100b, e.g. deriving unit 130, 130b, may optionally be configured to apply a weighted combination of one or more parameter values associated with a parent node of the currently considered node with one or more change values, in order to obtain one or more neural network parameters associated with the currently considered node.

[0265] Again, it is to be noted that any additional information used for a processing, an optional PUT information 124 may be provided by PUT information unit 120 to the deriving unit 130. Accordingly PUI information 132 may comprise such an additional information.

[0266] Parameter update tree 500 further comprises a node U6, 610, associated with parameter values, e.g. neural network parameters, 612, optionally in the form of a tensor, as shown in FIG. 5. Node U6, 610, is a child node of node U1, 530, as indicated by parent node information 614. As another optional feature, parameter update information 616 of node U6 is an update instruction defining a replacement of one or more parameter values, in this case the parameter value a.sub.12 of tensor 532, associated with the parent node of U6 with one or more change values, in this case the one change value 5.

[0267] Hence, an inventive parameter update information 116, 216, 316 and/or 416 (and accordingly generalized node information 112b, 312b) may optionally comprise an update instruction, e.g. 616, defining a replacement of one or more parameter values associated with a parent node, e.g. U1, 530, of the currently considered node, e.g. U6, 610, with one or more change values.

[0268] Therefore, an inventive decoder, e.g. 100, 100b, e.g. deriving unit 130, 130b, may optionally be configured to replace one or more parameter values associated with a parent node of the currently considered node with one or more replacement values, in order to obtain one or more neural network parameters associated with the currently considered node.

[0269] Parameter update tree 500 further comprises a node U9, 620, associated with parameter values, e.g. neural network parameters, 612, optionally in the form of a tensor, as shown in FIG. 5. Node U9, 620, is a child node of node U6, 610, as indicated by parent node information 624.

[0270] As an optional feature, as shown with the example of U9, 620, a plurality of neural network parameters associated with a parent node of the currently considered node, e.g. U9, may be represented by a parameter tensor, e.g. 612, and a plurality of neural network parameters associated with a currently considered node may be represented by a parameter tensor, e.g. 622, and a plurality of change values may be represented by a sum tensor, e.g. sum tensor

[00001]$\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)$

of parameter update information 626.

[0271] Furthermore, as an example, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may be configured to multiply elements of the parameter tensor, e.g. 612, associated with the parent node, e.g. U6, 610, of the currently considered node, e.g. U9, 620, with a node tensor weight value, e.g. as shown with factor 2 of parameter update information 626, to obtain a scaled parameter tensor, to multiply elements of the sum tensor, e.g. sum tensor

[00002]$\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)$

of parameter update information 626, with a sum tensor weight value, e.g. weight value 1, as shown with parameter update information 626, to obtain a scaled sum tensor, and form an element-wise sum of the scaled parameter tensor and of the scaled sum tensor, in order to obtain the parameter tensor, e.g. 622, associated with the currently considered node, e.g. U9.

[0272] It is to be noted that embodiments according to FIG. 5 may be simple embodiments, for explanatory purposes, such that, for example significantly more complex, parameter update information may be used in order to represent a specific version of neural network parameters.

[0273] As an optional feature, e.g. as explained before, a plurality of neural network parameters associated with a parent node of a currently considered node may be represented by a parameter tensor, and the parameter update information 116, 216, 316 and/or 416 (and accordingly generalized node information 112b, 312b) may optionally comprise an update instruction in the form of an update tensor.

[0274] Furthermore, an inventive decoder, e.g. 100, 100b, e.g. deriving unit 130, 130b, may optionally be configured to convert the shape of the update tensor according to the shape of the parameter tensor of the parent node.

[0275] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to provide the update tensor such that a shape of the update tensor is different from a shape of the parameter tensor of the parent node.

[0276] For further explanation, reference is made to FIG. 6. FIG. 6 shows a schematic example of a tensor shape conversion according to embodiments of the invention. FIG. 6 shows an example of a currently considered layer 630 of neurons of a neural network and of a previous layer 640 of the neural network.

[0277] As an example, tensor 650, may comprise neural network parameters, e.g. weights, associated with a parent node of the currently considered node, wherein tensor elements, e.g. a.sub.11, a.sub.12, a.sub.13, of the parameter tensor 650 arranged along a first direction (e.g. along a row 652 of the tensor) may be associated with contributions of output signals of a plurality of neurons 632, 634, 636 of a previous layer 630 of the neural network to an input signal of a given neuron, e.g. 642, of a currently considered layer 640 of the neural network, and tensor elements, e.g. a.sub.12, a.sub.22, a.sub.32, of the parameter tensor arranged along a second direction (e.g. along a column 654 of the tensor) are associated with contributions of an output signal of a given neuron, e.g. 634, of a previous layer 630 of the neural network to input signals of a plurality of neurons 642, 644, 646 of a currently considered layer 640 of the neural network. In the example of FIG. 6 for brevity not all weights between layers are shown (placeholders *).

[0278] Optionally, an inventive decoder, e.g. 100, 100b, e.g. deriving unit 130, 130b or obtaining unit 110, 110b, may optionally be configured to extend a dimension an update tensor 660 in the first direction 652, if the extension or dimension of the update tensor in the first direction (e.g. a row direction) is smaller than a dimension of the parameter tensor 650 in the first direction. Alternatively or in addition, the decoder may be configured to extend a dimension of the update tensor 660 in the second direction 654, if the extension or dimension of the update tensor in the second direction (e.g. a column direction) is smaller than a dimension of the parameter tensor 650 in the second direction.

[0279] Hence, an extended update tensor 670 may be provided, such that extended update tensor 670 may be combined with parameter tensor 650 in order to modify neural network parameters of the parent node to determine neural network parameters of a current node.

[0280] In general, it is to be noted that nodes and corresponding tensors may represent neural network parameters of a layer of a neural network, and/or of a whole neural network and hence of multiple layers.

[0281] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to provide the update tensor 660, such that the extension or dimension of the update tensor 660 in the first direction 652 (e.g. a row direction) is smaller than a dimension or extension of the parameter tensor 650 in the first direction. Alternatively or in addition, the encoder may be configured to provide the update tensor 660 such that the extension or dimension of the update tensor in the second direction 654 (e.g. a column direction) is smaller than a dimension or extension of the parameter tensor 650 in the second direction.

[0282] As shown in FIG. 6, an update tensor 660 may, for example, comprise the change values u.sub.11, u.sub.12, u.sub.21 and u.sub.22. As shown with extended update tensor 670, optionally, an inventive decoder, e.g. 100, 100b, e.g. deriving unit 130, 130b, may optionally be configured to copy entries of a row of the update tensor 660, to obtain entries of one or more extension rows of a shape-converted update tensor 670, if a number or rows of the update tensor 660 is smaller than a number of rows of the parameter tensor 650.

[0283] Alternatively or in addition, the decoder may be configured to copy entries of a column of the update tensor 660, to obtain entries of one or more extension columns of a shape-converted update tensor 670, if a number or columns of the update tensor is smaller than a number of columns of the parameter tensor 650.

[0284] As shown in FIG. 6, e.g. as a simplified example, a first row of update tensor 660 may be duplicated to provide a third row of the extended update tensor 670 and a first column of update tensor 660 may be duplicated to provide a third column of the extended update tensor 670.

[0285] As optionally, shown an inventive decoder, e.g. 100, e.g. deriving unit 130, 130b, may optionally be configured to copy one or more entries of an update tensor in a row direction and in a column direction, to obtain entries of a shape-converted update tensor 670.

[0286] However, it is to be noted that only a row or a column may be copied or that only certain entries of a row and/or a column may be copied.

[0287] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to provide the update tensor 660 such that a number of rows of the update tensor is smaller than a number of rows of the parameter tensor 650. Alternatively, or in addition, the encoder may be configured to provide the update tensor 660 such that a number of columns of the update tensor is smaller than a number of columns of the parameter tensor 650.

[0288] As another optional feature, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to provide an information about an extension of the update tensor. This information may, for example additionally encoded in a bitstream, e.g. 202, 302 and/or 402. An inventive node information 112, 112b, 212, 312, 312 b and/or 412 may optionally comprise such an extension information.

[0289] Accordingly, an inventive decoder, e.g. 100, 100b, e.g. deriving unit 130, 130b, may optionally be configured to determine a need to convert the shape of the update tensor, and/or an extent of a conversion of the shape of the update tensor, in dependence on an information about an extension of the update tensor.

[0290] As another optional feature, an inventive decoder, e.g. 100, 100b, e.g. obtaining unit 110, 110b, may optionally be configured to determine whether a parent node identifier information, e.g. 114, is present, e.g. in an encoded bitstream 102, and the decoder may be configured to derive one or more neural network parameters, e.g. 104, 104b, according to any embodiment as disclosed herein, if the parent node identifier is present, and furthermore, the decoder may be configured to make the currently considered node the root node if the parent node identifier is not present.

[0291] Hence, referring to FIG. 1 and FIG. 1b the parent node identifier information 114 (or respectively the generalized node information 112b) may comprise the information whether a parent node identifier is present or is not present, e.g. instead of or in addition to a parent node identifier (e.g. if present), such that a parameter update tree, for example set up and stored by PUT information unit 120, 120b may be adapted accordingly. This may allow to discard portions of a PUT that may not be needed any more, e.g. because corresponding parameter sets may be outdated or inferior to (e.g. worse than) newer parameter sets. Hence, in simple words, an upper part of a PUT may be discarded, such that the new root node of PUT is the currently considered node. On the other hand, this way a new PUT may be set up, starting from the currently considered node.

[0292] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to provide a signaling, comprising an information whether a parent node identifier is present or not. Hence, referring to FIGS. 2 to 4, instead of the parent node identifier information 214, 314 and/or 414 may comprise such a signaling, e.g. instead of a parent node identifier, or in addition to a parent node identifier.

[0293] As another optional feature, the nodes of a PUT, e.g. nodes R, U0, to U9, 510 to 620, may be associated with a respective hash value. An inventive decoder, e.g. 100, 100b, e.g. PUT information unit 120, 120b, may optionally be configured to compare the parent node identifier (wherein the parent node identifier information 114 (or respectively the generalized node information 112b) may comprise the parent node identifier) with hash values associated with one or more nodes, to identify the parent node of the currently considered node.

[0294] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured provide a hash value associated with a node as the parent node identifier (e.g. within the parent node identifier information 214, 314 and/or 414 or respectively the generalized node information 112b, 312b, for example encoded in the bitstream 202, 302, 302b and/or 402), to identify the parent node of the currently considered node.

[0295] Optionally, the hash values may be hash values of a full compressed data unit NDU associated with one or more previously decoded nodes.

[0296] As another optional feature, the hash value may be a hash value of a payload portion of a compressed data unit associated with one or more previously encoded nodes, while leaving a data size information and a header information unconsidered.

[0297] As another optional feature, the parent node identifier may be a combined value representing a device identifier and a serial number of which both are associated with the parent node.

[0298] As another optional feature, parent node identifier may identify an update tree, e.g. 500, and/or a layer of the neural net. A PUT may, for example, represent a portion of neural network parameters of a neural network. Hence, for a neural network a plurality of update trees may be set up, such that a differentiation in between trees may be advantageous. As an example, a PUT may represent one layer of a neural network.

[0299] As another optional feature, the node information, e.g. 112, 112b, 212, 312, 312b and/or 412 may comprise a node identifier. An inventive decoder, e.g. 100, 100b, e.g. PUT information unit 120, 120b, may optionally be configured to store the node identifier. Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to store and/or to provide the node identifier.

[0300] As another optional feature, an inventive decoder, e.g. 100, 100b, e.g. PUT information unit 120, 120b, may optionally be configured to compare one or more stored node identifiers with a parent node identifier in a node information of a new node when adding the new node, in order to identify a parent node of the new node.

[0301] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to compare one or more stored node identifiers with a parent node identifier in a node information of a new node when adding the new node, in order to identify a parent node of the new node.

[0302] As another optional feature, the node identifier may identify an update tree, e.g. 500, to which the node information, e.g. 112, 112b, 212, 312, 312b and/or 412 is associated and/or a layer of the neural net to which the node information relates. Neural networks may comprise millions of parameters, hence only a selection of parameters may be organized in one single parameter tree. Furthermore, a searching time for an encoder or decoder may be reduced using an information about with which neural network layer, neural network parameters to be searched for, e.g. as parameters associated with a parent node, are associated.

[0303] As another optional feature, the node identifier, e.g. within node identifier information 114, 214, 314, 414 and respectively generalized node information 112b, 312b, may comprise a device identifier and/or a parameter update tree depth information and/or a parameter update tree identifier. One neural network may be trained on different devices, such that different sets of parameters may be available and for example even different iterations of such sets of parameters. Hence, an information about a device identifier, may allow to indicate a specific set of neural network parameters efficiently. A PUT depth information may reduce a time needed, for example, to find a corresponding parent node, in order to determine neural network parameters of a currently considered node, since it may not be necessary to search through all layers of the PUT.

[0304] As another optional feature, the node information, e.g. 112, 112b, 212, 312, 312b and/or 412, may comprise a signaling indicating whether a node identifier is present or not.

[0305] As another optional feature, the parent node identifier (e.g. within the parent node identifier information 114, 214, 314 and/or 414 and respectively generalized node information 112b, 312b, for example encoded in the bitstream 202, 302 and/or 402) is a combined value representing a device identifier and a serial number which both are associated with the parent node.

[0306] As another optional feature, the parent node identifier information 114, 214, 314 and/or 414 and respectively generalized node information 112b, 312b, may optionally comprise an information about the type of the parent node identifier. Hence, an inventive decoder, e.g. 100, 100b, e.g. obtaining unit 110, 110b, may optionally be configured to obtain a signaling, comprising an information about the type of the parent node identifier, and the decoder may be configured to evaluate the signaling in order to consider the respective type of the parent node identifier.

[0307] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to provide a signaling, comprising an information about the type of the parent node identifier.

[0308] As another optional feature, an inventive decoder, e.g. 100, 100b, e.g. PUT information unit 120, 120b, may optionally be configured to selectively evaluate a syntax element which indicates a type of the parent node identifier, in dependence on a syntax element indicating the presence of the parent node identifier.

[0309] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to selectively provide a syntax element which indicates a type of the parent node identifier, if a syntax element describing the parent node identifier is present.

[0310] FIG. 7 shows an example for a topology change of a neural network according to embodiments of the invention. FIG. 7 shows a first topology 710 of a neural network section with neurons 712, wherein neural network parameters a.sub.11, a.sub.12, a.sub.21 and a.sub.22, e.g. weights, may be represented by a parameter tensor 720. As an example, the topology of the neural network section may change to a second topology 730 with a new node 732 and additional parameters b.sub.1 and b.sub.2. The neural network parameters of the neural network with topology 730 may be represented by tensor 740.

[0311] As an optional feature, an inventive decoder, e.g. 100, 100b, e.g. obtaining unit 110, 110b, may optionally be configured, to obtain a topology change signaling within the node information, comprising an information about a topology change of the neural network. As an example, the parent node identifier information, e.g. 114, may optionally, comprise the topology change signaling. Furthermore, the decoder may be configured to modify the parameter information of the parent node according to the topology change in order to derive one or more neural network parameters, e.g. as represented by tensor 740, of the neural network with modified topology.

[0312] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured, to provide a topology change signaling within the node information, e.g. within the parent node identifier information 214, 314 and/or 414, comprising an information about a topology change of the neural network.

[0313] As another optional feature, an inventive decoder, e.g. 100, 100b, e.g. the PUT information unit 120, 120b and/or the e.g. deriving unit 130, 130b, may optionally be configured to change a shape of one or two tensors in response to a topology change information. As an example a tensor comprising change values may be adapted to a new shape of a parent node according to the new neural network topology.

[0314] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to signal a change a shape of one or two tensors, together with a signaling of a topology change.

[0315] As another optional feature, an inventive decoder, e.g. 100, 100b, e.g. PUT information unit 120, 120b, may optionally be configured to change a number of neurons of the given layer in response to the topology change information. As an example, a decoder on a device running the neural network may receive the topology change information and may hence adapt the structure of the neural network, e.g. in addition to adapting neural network parameters, e.g. weight values.

[0316] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to signal a change of a number of neurons of the given layer using the topology change information. Such topology change information may be included in a generalized node information, e.g. 312b.

[0317] As another optional feature, an inventive decoder, e.g. 100, 100b, e.g. PUT information unit 120, 120b and/or e.g. deriving unit 130, 130b, may optionally be configured to replace one or more tensor values of one or more tensors, a shape of which is to be changed, associated with a parent node of the currently considered node with one or more replacement values, in order to obtain one or more tensors having a modified size, or the decoder may be configured to replace one or more tensors, a shape of which is to be changed, associated with a parent node of the currently considered node with one or more replacement tensors, in order to obtain one or more tensors having a modified size.

[0318] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to signal a replacement of one or more tensor values of one or more tensors, a shape of which is to be changed, associated with a parent node of the currently considered node with one or more replacement values, or the encoder may be configured to signal a replacement of one or more tensors, a shape of which is to be changed, associated with a parent node of the currently considered node with one or more replacement tensors.

[0319] As another optional feature, an inventive decoder, e.g. 100, 100b, e.g. deriving unit 130, 130b, may optionally be configured to change shapes of two tensors in two update trees associated with neighboring layers of the neural net in a synchronized manner, in response to the topology change signaling.

[0320] Accordingly, an inventive encoder, e.g. 200, 300, 300b and/or 400 for example node information unit 210 and/or PUT unit 310, 310b, 410, may optionally be configured to signal a change of shapes of two tensors in two update trees associated with neighboring layers of the neural net in a synchronized manner, using the topology change signaling.

[0321] FIG. 8 shows a schematic view of a neural network controller according to embodiments of the invention. FIG. 8 shows a neural network controller 800 comprising a training unit 810, a reference unit 820, a parameter update information, PUI, unit 830 and a node information provision unit 840.

[0322] The training unit 810 may be configured to train a neural network, to obtain updated neural network parameters 812 on the basis of initial neural network parameters. As an example, the initial neural network parameters may be default parameters or for example a first set of neural network parameters, starting from which, the neural network may be trained. As an optional feature, the initial neural network parameters may be provided to the training unit 810 using or by the reference unit 820. The initial neural network parameters may be stored in the reference unit 820 or may, for example, initially provided to the neural network controller 800.

[0323] As an example, based on the updated neural network parameters a second training may be performed. Hence, the updated neural network parameters may, for example be reference or starting parameters for a second training. Such reference parameters may be stored in the reference unit 820. Therefore, in a first step, reference parameters, e.g. the parameters based on which a training is performed, may be equal to initial neural network parameters.

[0324] Furthermore, the PUI unit 830 may be configured to determine a parameter update information, PUI, 832 on the basis of the reference neural network parameters 822 and the updated neural network parameters 812. Therefore, reference unit 820 may provide the reference neural network parameters 822, e.g. the parameters based on which a training was performed in order to obtain the updated neural network, NN, parameters, to the PUI unit 830.

[0325] The PUI 832 may, for example, as well comprises one or more update instructions describing how to derive the updated neural network parameters, at least approximately, from the initial neural network parameters.

[0326] The reference NN parameters 882 may, for example, be the initial neural network parameters (e.g. even in a second, third, fourth or further training step), such that the parameter update information 832 may comprise an information on how to modify the initial neural network parameters in order to calculate or determine the updated NN parameters 812.

[0327] However, such an information may as well be included in the PUI 832, for a recursive reconstruction. Hence, an information on how to modify reference NN parameters 822, e.g. starting parameters for one training cycle, in order to obtain the updated NN parameters 812, that are different form the initial NN parameters, may be included in the PUI.

[0328] Referring to FIG. 5, the PUI may hence comprise an information on a whole path from root node R, 510, associated with initial neural network parameters, e.g. 512, to a currently considered node, e.g. U3, 550, associated with the updated NN parameters, e.g. 552, or just about a section of such a path, e.g. via one or more nodes, e.g. from U2, 520, associated with reference parameters, e.g. 522, to node U4, associated with updated NN parameters, e.g. 562. Here it is again to be noted thatin simple wordsin between the reference NN parameters 822 and the updated NN parameters one or more trainings and hence one or more parameter updates may be performed.

[0329] To put it in other words, the PUI information may optionally comprise an information on how to modify reference NN parameters 822, being an initial or arbitrary or intermediate starting point for a NN training, in order to obtain the updated NN parameters 812.

[0330] Furthermore, the node information provision unit 840 may be configured to provide a node information 802 comprising a parent node identifier information(e.g. as explained before) and the parameter update information PUI (e.g. as explained before), wherein the parent node identifier defines a parent note, parameter information of which serves as a starting point for the application of the parameter update information.

[0331] Hence, the parent node identifier, may be used to provide a reference information for the PUI information, in order to identify reference NN parameters 822 that are to be modified by the PUI information in order to obtain the updated NN parameters 832

[0332] As another optional feature, the neural network controller 800 may comprise an encoder according to any of the embodiments as disclosed herein and/or any functionality, or combination of functionalities, of any inventive encoder as disclosed herein.

[0333] FIG. 9 shows a schematic view of a neural network federated learning controller according to embodiments of the invention. FIG. 9 shows neural network federated learning controller 900 comprising a processing unit 910 and a distribution unit 920.

[0334] The neural network federated learning controller 900 is configured to receive a node information 902 of a plurality of neural networks, wherein the node information comprises a parent node identifier (or for example a parent node identifier information comprising the parent node identifier) and a parameter update information.

[0335] Furthermore, processing unit 910 is configured to combine parameter update information of several corresponding nodes of different neural networks, to obtain a combined parameter update information. Processed information 912 may comprise or may be the combined parameter update information.

[0336] Moreover, distributing unit 920 is configured to distribute the processed information, e.g. the combined parameter update information.

[0337] Hence, the neural network federated learning controller 900 may operate as a coordination unit in order to combine several training results (e.g. the parameter update information) of several corresponding nodes of different neural networks. Therefore robust neural network parameters may be extracted and provided in the form of the processed information.

[0338] As an optional feature, the neural network federated learning controller 900, e.g. processing unit 910, may be configured to combine parameter update information of several corresponding nodes having equal parent node identifiers of different neural networks, to obtain a combined parameter update information. Hence, processed information 912 may comprise or may be the combined parameter update information.

[0339] Therefore, as an example, NN training results based on equal starting parameters may be combined, in order to provide a robust set of NN parameters.

[0340] As an optional feature, the neural network federated learning controller 900, e.g. distribution unit 920 may be configured to distribute parameter information of a parent node, to which the parent node identifier is associated, to a plurality of decoders and the neural network federated learning controller 900, e.g. processing unit 912, may be configured to receive from the decoders node information comprising the parent node identifier. Furthermore, the neural network federated learning controller 900, e.g. processing unit 910, is configured to combine parameter update information of several corresponding nodes having the parent node identifier.

[0341] As another optional feature, the neural network federated learning controller 900, e.g. distribution unit 920, may, for example, be configured to provide a node information, e.g. within or being the processed information 912, describing a combined node information of a parameter update tree, wherein the combined node information comprises the parent node identifier, and wherein the combined node information comprises the combined parameter update information.

[0342] As another optional feature, the neural network federated learning controller 900 may optionally comprise an encoder according to any embodiments disclosed herein or the neural network federated learning controller 900 may optionally comprise any functionality, or combination of functionalities, of an inventive encoder as disclosed herein.

[0343] FIG. 10 shows a schematic block diagram of a method for decoding parameters of a neural network according to embodiments of the invention. Method 1000 comprises obtaining 1010 a plurality of neural network parameters of the neural network on the basis of an encoded bitstream, obtaining 1020 a node information describing a node of a parameter update tree, wherein the node information comprises a parent node identifier, and wherein the node information comprises a parameter update information, and deriving 1030 one or more neural network parameters using parameter information of a parent node identified by the parent node identifier and using the parameter update information.

[0344] FIG. 11 shows a schematic block diagram of a method for encoding parameters of a neural network in order to obtain an encoded bitstream according to embodiments of the invention. FIG. 11 shows method 1100 comprising providing 1110 a node information describing a node of a parameter update tree, wherein the node information comprises a parent node identifier, and wherein the node information comprises a parameter update information; wherein the parameter update information describes differences between neural network parameters associated with a parent node defined by the parent node identifier and current neural network parameters.

[0345] FIG. 12 shows a schematic block diagram of a method for controlling a neural network according to embodiments of the invention. FIG. 12 shows method 1200 comprising training 1210 a neural network, to obtain updated neural network parameters on the basis of initial neural network parameters, and determining 1220 a parameter update information on the basis of reference neural network parameters and the updated neural network parameters, wherein the parameter update information comprises one or more update instructions describing how to derive the updated neural network parameters, at least approximately, from the initial neural network parameters, and providing 1230 a node information comprising a parent node identifier and the parameter update information, wherein the parent node identifier defines a parent note, parameter information of which serves as a starting point for the application of the parameter update information.

[0346] FIG. 13 shows a schematic block diagram of a method for controlling neural network federated learning according to embodiments of the invention. FIG. 13 shows method 1300 comprising receiving 1310 node information of a plurality of neural networks, wherein the node information comprises a parent node identifier, and wherein the node information comprises a parameter update information. The method further comprises combining 1320 parameter update information of several corresponding nodes of different neural networks, to obtain a combined parameter update information, and distributing 1330 the combined parameter update information.

[0347] Further embodiments according to the invention are related to, or may be used for, or may address HLS (e.g. HTTP (e.g. Hypertext Transfer Protocol) live streaming) update signaling.

[0348] Furthermore, it is to be noted that embodiments can be applied to the compression of entire neural networks, and some of them can also be applied to the compression of differential updates of neural networks with respect to a base network. Such differential updates are for example useful when models are redistributed after fine-tuning or transfer learning, or when providing versions of a neural network with different compression ratios.

[0349] Embodiments may further address usage, e.g. manipulation or modification of base neural network, e g. neural network serving as reference for a differential update.

[0350] Embodiments may further address or comprise or provide updated neural network, e.g. neural network resulting from modifying the base neural network. Note: The updated neural network may, for example, be reconstructed by applying a differential update to the base neural network.

[0351] Further embodiments according to the invention may comprise syntax elements in the form of NNR units. A NNR unit may, for example be a data structure for carrying neural network data and/or related metadata which may be compressed or represented, e.g. according to embodiments of the invention.

[0352] NNR units may carry at least one of a compressed information about neural network metadata, uncompressed information about neural network metadata, topology information, complete or partial layer data, filters, kernels, biases, quantized weights, tensors and alike.

[0353] An NNR unit may, for example comprise, or consist of the following data elements [0354] NNR unit size (optional): This data element may signal the total byte size of the NNR Unit, including the NNR unit size. [0355] NNR unit header: This data element may comprise or contain information about the NNR unit type and/or related metadata. [0356] NNR unit payload: This data element may comprise or contain compressed or uncompressed data related to the neural network.

[0357] As an example, embodiments may comprise (or use) the following bitstream syntax:

TABLE-US-00001 Descriptor nnr_unit( numBytesInNNRUnit ) { nnr_unit_size( ) (optional) nnr_unit_header( ) nnr_unit_payload( ) }

TABLE-US-00002 Descriptor nnr_unit_header( ) { nnr_unit_type u(6) nnr_compressed_data_unit_payload_type u(5) . . . further optional configuration information . . . if( nnr_unit_type == NNR_NDU ) nnr_compressed_data_unit_header( ) . . . further optional configuration information . . . }

TABLE-US-00003 Descriptor nnr_compressed_data_unit_header( ) { nnr_compressed_data_unit_payload_type u(5) . . . Further optional configuration information . . . node_id_present_flag u(1) if( node_id_present_flag ) { device_id ue(1) parameter_id ue(5) put_node_depth ue(4) } parent_node_id_present_flag u(1) if( parent_node_id_present_flag ) { parent_node_id_type u(2) temporal_context_modeling_flag u(1) if( parent_node_id_type == ICNN_NDU_ID ) { parent_device_id ue(1) if( !node_id_present_flag ) { parameter_id ue(5) put_node_depth ue(4) } } else if( parent_node_id_type == ICNN_NDU_PL_SHA256 ) parent_node_payload_sha256 u(256) else parent_node_payload_sha512 u(512) } . . . further optional configuration information . . . }

[0358] The parent node identifier, may for example comprise one or more of the above syntax elements, to name some, e.g. device_id, parameter_id and/or put_node_depth.

TABLE-US-00004 Descriptor nnr_unit_payload( ) { . . . further optional configuration information . . . if( nnr_unit_type == NNR_NDU ) nnr_compressed_data_unit_payload( ) . . . further optional configuration information . . . }

TABLE-US-00005 Descriptor nnr_compressed_data_unit_payload( ) { if( nnr_compressed_data_unit_payload_type == NNR_PT_ RAW_FLOAT ) for( i = 0; i < Prod( TensorDimensions ); i++ ) raw_float32_parameter[ TensorIndex( TensorDimensions, flt(32) i, 0 ) ] decode_compressed_data_unit_payload( ) }

[0359] Using the decode_compressed_data_unit_payload( ) parameters of a base model of the neural network may be modified in order to obtain an updated model. Hence, using the above nnr_compressed_data_unit_payload( ) one or more neural network parameters using parameter information of a parent node identified by the parent node identifier and using the parameter update information may be derived.

node_id_present_flag equal to 1 may indicate that syntax elements device_id, parameter_id, and/or put_node_depth are present.
device_id may, for example, uniquely identify the device that generated the current NDU.
parameter_id may, for example, uniquely identify the parameter of the model to which the tensors stored in the NDU relate to. If parent_node_id_type is equal to ICNN_NDU_ID, parameter_id may, for example, or shall equal the parameter_id of the associated parent NDU.
put_node_depth may, for example, be the tree depth at which the current NDU is located. A depth of 0 may correspond to the root node. If parent_node_id_type is equal to ICNN_NDU_ID, put_node_depth1 may, for example, be or even must equal the put node depth of the associated parent NDU.
parent_node_id_present_flag equal to 1 may, for example, indicate that syntax element parent_node_id_type is present.
parent_node_id_type may, for example, specify the parent node id type. It may indicate which further syntax elements for uniquely identifying the parent node are present. Examples for the allowed values for parent_node_id_type are defined in Table 1

TABLE-US-00006 TABLE 1 Parent node id type identifiers (example). parent_node_id_type Identifier Description 0 ICNN_NDU_ID Indicates that syntax elements parent_device_id, parameter_id, and put_node_depth are present 1 ICNN_NDU_PL_SHA256 Indicates that syntax element parent_node_payload_sha256 is present 2 ICNN_NDU_PL_SHA512 Indicates that syntax element parent_node_payload_sha512 is present 3 Reserved
temporal_context_modeling_flag may, for example, specify whether temporal context modeling is enabled. A temporal_context_modeling_flag equal to 1 may indicate that temporal context modeling is enabled. If temporal_context_modeling_flag is not present, it is inferred to be 0.
parent_device_id may, for example, be equal to syntax element device id of the parent NDU.
parent_node_payload_sha256 may, for example, be a SHA256 hash of the nnr_compressed_data_unit_payload of the parent NDU.
parent_node_payload_sha512 may, for example, be a SHA512 hash of the nnr_compressed_data_unit_payload of the parent NDU.

[0360] Furthermore, embodiments according to the invention may comprise a row skipping feature. As an example, if enabled by flag row_skip_flag_enabled_flag, the row skipping technique signals one flag row_skip_list[i] for each value i along the first axis of the parameter tensor. If the flag row_skip_list[i] is 1, all elements of the parameter tensor for which the index for the first axis equals i are set to zero. If the flag row_skip_list[i] is 0, all elements of the parameter tensor for which the index for the first axis equals i are encoded individually.

[0361] Furthermore, embodiments according to the invention may comprise a context modelling. As an example, context modelling may correspond to associating the three type of flags sig_flag, sign_flag, and abs_level_greater_x/x2 with context models. In this way, flags with similar statistical behavior may be or should be associated with the same context model so that the probability estimator (inside of the context model) can, for example, adapt to the underlying statistics.

[0362] The context modelling of the presented approach may, for example, be as follows:

[0363] For example, twenty-four context models may be distinguished for the sig_flag, depending on the state value and whether the neighbouring quantized parameter level to the left is zero, smaller, or larger than zero.

[0364] If dq_flag is 0, only the first three context models may, for example, be used.

[0365] Three other context models may, for example, be distinguished for the sign_flag depending on whether the neighbouring quantized parameter level to the left is zero, smaller, or larger than zero.

[0366] For the abs_level_greater_x/x2 flags, each x may, for example, be use either one or two separate context models. If x<=maxNumNoRemMinus 1, two context models are distinguished depending on the sign_flag. If x>maxNumNoRemMinus1, only one context model may, for example, be used.

[0367] Furthermore, embodiments according to the invention may comprise temporal context modelling. As an example, if enabled by flag temporal_context_modeling_flag, additional context model sets for flags sig_flag, sign_flag and abs_level_greater_x may be available. The derivation of ctxIdx may then be also based on the value of a quantized co-located parameter level in the previously encoded parameter update tensor, which can, for example, be uniquely identified by the parameter update tree. If the co-located parameter level is not available or equal to zero, the context modeling, e.g. as explained before, may be applied. Otherwise, if the co-located parameter level is not equal to zero, the temporal context modeling of the presented approach may be as follows:

[0368] Sixteen context models may, for example, be distinguished for the sig_flag, depending on the state value and whether the absolute value of the quantized co-located parameter level is greater than one or not.

[0369] If dq_flag is 0, only the first two additional context models may be used.

[0370] Two more context models may, for example, be distinguished for the sign_flag depending on whether the quantized co-located parameter level is smaller or greater than zero.

[0371] For the abs_level_greater_x flags, each x may use two separate context models. These two context models may, for example, be distinguished depending on whether the absolute value of the quantized co-located parameter level is greater or equal to x1 or not.

[0372] Embodiments according to the invention may optionally comprise the following tensor syntax, e.g. a quantized tensor syntax.

TABLE-US-00007 Descriptor quant_tensor( dimensions, maxNumNoRemMinus1, entryPointOffset ) { tensor2DHeight = dimensions[ 0 ] tensor2DWidth = Prod( dimensions ) / tensor2DHeight if( general_profile_idc == 1 && tensor2DWidth > 1 ) { row_skip_enabled_flag uae(1) if( row_skip_enabled_flag ) for( i = 0; i < tensor2DHeight; i++ ) row_skip_list[ i ] (optional) ae(v) } stateId = 0 (optional) bitPointer = get_bit_pointer( ) (optional) lastOffset = 0 (optional) for( i = 0; i < Prod( dimensions ); i++ ) { idx = TensorIndex( dimensions, i, scan_order ) (optional) if( entryPointOffset != 1 && GetEntryPointldx( dimensions, i, scan_order ) != 1 && scan_order > 0 ) { (optional) lvlCurrRange = 256 (optional) j = entryPointOffset + GetEntryPointldx( dimensions, i, scan_order ) (optional) lvlOffset = cabac_offset_list[ j ] (optional) if( dq_flag ) (optional) stateld = dq_state_list[ j ] (optional) set_bit_pointer( bitPointer + lastOffset + BitOffsetList[ j ] ) (optional) lastOffset = BitOffsetList[ j ] (optional) init_prob_est_param( ) (optional) } QuantParam[ idx ] = 0 if( general_profile_idc != 1 || tensor2DWidth <= 1 || !row_skip_enabled_flag || !row_skip_list[ idx[ 0 ] ] ) int_param( idx, maxNumNoRemMinus1, stateId ) e.g. as explained in the (optional) following . . . further optional configuration information . . . } }

[0373] The skip information may, for example, comprise any or all of the above row skip information e.g. row_skip_enabled_flag and/or row_skip_list.

[0374] As an example, row_skip_enabled_flag may specify whether row skipping is enabled. A row_skip_enabled_flag equal to 1 may indicate that row skipping is enabled.

row_skip_list may specify a list of flags where the i-th flag row_skip_Isit[i] may indicate whether all tensor elements of QuantParam for which the index for the first dimension equals i are zero. If row_skip_list[i] is equal to 1, all tensor elements of QuantParam for which the index for the first dimension equals i may be zero.

[0375] Embodiments according to the invention may, for example, further comprise a quantized parameter syntax, as an example a syntax as defined in the following. All elements may be considered as optional.

TABLE-US-00008 Descriptor int_param( i, maxNumNoRemMinus1, stateId ) { sig_flag ae(v) if( sig_flag ) { QuantParam[ i ]++ sign_flag ae(v) j = 1 do { j++ abs_level_greater_x[ j ] ae(v) QuantParam[ i ] += abs_level_greater_x[ j ] } while( abs_level_greater_x[ j ] == 1 && j < maxNumNoRemMinus1 ) if( abs_level_greater_x[ j ] == 1 ) { RemBits = 0 j = 1 do { j++ abs_level_greater_x2[ j ] ae(v) if( abs_level_greater_x2[ j ] ) { QuantParam[i] += 1 << RemBits RemBits++ } } while( abs_level_greater_x2[ j ] && j <30) abs_remainder uae(RemBits) QuantParam[ i ] += abs_remainder } QuantParam[ i ] = sign_flag ? QuantParam[ i ] : QuantParam[ i ] } }
sig_flag may, for example, specify whether the quantized weight QuantParam[i] is nonzero. A sig_flag equal to 0 may, for example, indicate that QuantParam[i] is zero.
sign_flag may, for example, specify whether the quantized weight QuantParam[i] is positive or negative. A sign_flag equal to 1 may, for example, indicate that QuantParam[i] is negative. abs_level_greater_x[j] may, for example, indicate whether the absolute level of QuantParam[i] is greater j+1.
abs_level_greater_x2[j] may, for example, comprise the unary part of the exponential Golomb remainder.
abs_remainder may, for example, indicate a fixed length remainder.

[0376] Further embodiments according to the invention may, for example, comprise the following shift parameter indices syntax. All elements may be considered as optional.

TABLE-US-00009 Descriptor shift_parameter_ids( maxNumNoRemMinus1 ) { for( i = 0; i < (dq_flag ? 24 : 3; i++ ) { shift_idx( i, ShiftParameterIdsSigFlag ) } if(temporal_context_modeling_flag){ for( i = 24; i < (dq_flag ? 40 : 26); i++ ) { shift_idx( i, ShiftParameterIdsSignFlag ) } } for( i = 0; i < ( temporal_context_modeling_flag ? 5:3 ); i++ ) { shift_idx( i, ShiftParameterIdsSignFlag ) } for( i = 0; i < (temporal_context_modeling_flag ? 4 : 2)*(maxNumNoRemMinus1+1); i++ ) { shift_idx( i, ShiftParameterIdsAbsGrX ) } for( i = 0; i < 31; i++ ) { shift_idx( i, ShiftParameterIdsAbsGrX2 ) } }

[0377] Further embodiments according to the invention comprise entropy decoding processes, as explained in the following.

[0378] In general inputs to this process may, for example, be a request for a value of a syntax element and values of prior parsed syntax elements.

[0379] Output of this process may, for example be the value of the syntax element.

[0380] The parsing of syntax elements may, for example, proceed as follows:

[0381] For each requested value of a syntax element a binarization may, for example, bederived.

[0382] The binarization for the syntax element and the sequence of parsed bins may, for example, determine the decoding process flow.

Example for Initialization Process

[0383] In general, outputs of this process may, for example, be initialized DeepCABAC internal variables.

[0384] The context variables of the arithmetic decoding engine may, for example, be initialized as follows:

[0385] The decoding engine may, for example, register IvICurrRange and IvIOffset both in 16 bit register precision may, for example, be initialized by invoking the initialization process for the arithmetic decoding engine.

[0386] Embodiments according to the invention may comprise an initialization process for probability estimation parameters, e.g. as explained in the following.

[0387] Outputs of this process may, for example, be the initialized probability estimation parameters shift0, shift1, pStateIdx0, and pStateIdx1 for each context model of syntax elements sig_flag, sign_flag, abs_level_greater_x, and abs_level_greater_x2.

[0388] The 2D array CtxParameterList [][] may, for example, beinitialized as follows:

[0389] CtxParameterList[][]={{1, 4, 0, 0}, {1, 4, 41, 654}, {1, 4, 95, 1519}, {0, 5, 0, 0}, {2, 6, 30, 482}, {2, 6, 95, 1519}, {2, 6, 21, 337}, {3, 5, 0, 0}, {3, 5, 30, 482}}

[0390] If dq_flag is equal to 1 and temporal_context_modeling_flag is equal to 1, for each of the e.g. 40 context models of syntax element sig_flag, the associated context parameter shift0 may, for example, beset to CtxParameterList[setId][0], shift1 may, for example, be set to CtxParameterList[setId][1], pStateIdx0 may, for example, be set to CtxParameterList[setId][2], and pStateIdx1 may, for example, be set to CtxParameterList[setId][3], where i may, for example, be the index of the context model and where setId may, for example, be equal to ShiftParameterIdsSigFlag[i].

[0391] If dq_flag==is equal to 1 and temporal_context_modeling_flag is equal to 0, e.g. for each of the first e.g. 24 context models of syntax element sig_flag, the associated context parameter shift0 may, for example, be set to CtxParameterList[setId][0], shift1 may, for example, be set to CtxParameterList[setId][1], pStateIdx0 may, for example, be set to CtxParameterList[setId][2], and pStateIdx1 may, for example, be set to CtxParameterList[setId][3], where i may, for example, be the index of the context model and where setId may, for example, be equal to ShiftParameterIdsSigFlag[i].

[0392] If dq_flag is equal to 0 and temporal_context_modeling_flag is equal to 1, e.g. for each of the e.g. first 3 context models and e.g. context models 24 to 25 of syntax element sig_flag, the associated context parameter shift0 may, for example, be set to CtxParameterList[setId][0], shift1 may, for example, be set to CtxParameterList[setId][1], pStateIdx0 may, for example, be set to CtxParameterList[setId][2], and pStateIdx1 may, for example, be set to CtxParameterList[setId][3], where i may, for example, be the index of the context model and where setId may, for example, be equal to ShiftParameterIdsSigFlag [i].

[0393] If temporal_context_modeling_flag is equal to 1, e.g. for each of the for example 5 context models of syntax element sign_flag, the associated context parameter shift 0 may, for example, be set to CtxParameterList[setId][0], shift1 may, for example, be set to CtxParameterList[setId][1], pStateIdx0 may, for example, be set to CtxParameterList[setId][2], and pStateIdx1 may, for example, be set to CtxParameterList[setId][3], where i may, for example, be the index of the context model and where setId may, for example, be equal to ShiftParameterIdsSignFlag[i].

[0394] Otherwise, (temporal_context_modeling_flag==0), e.g. for each of the first e.g. 3 context models of syntax element sign_flag, the associated context parameter shift0 may, for example, be set to CtxParameterList[setId][0], shift1 may, for example, be set to CtxParameterList[setId][1], pStateIdx0 may, for example, be set to CtxParameterList[setId][2], and pStateIdx1 may, for example, be set to CtxParameterList[setId][3], where i may, for example, be the index of the context model and where setId may, for example, be equal to ShiftParameterIdsSignFlag[i].

[0395] If temporal_context_modeling_flag is equal to 1, e.g. for each of the 4*(cabac_unary_length_minus1+1) context models of syntax element abs_level_greater_x, the associated context parameter shift0 may, for example, be set to CtxParameterList[setId][0], shift1 may, for example, be set to CtxParameterList[setId][1], pStateIdx0 may, for example, be set to CtxParameterList[setId][2], and pStateIdx1 may, for example, be set to CtxParameterList[setId][3], where i may, for example, be the index of the context model and where setId may, for example, be equal to ShiftParameterIdsAbsGrX[i].

[0396] Otherwise, (temporal_context_modeling_flag==0), e.g. for each of the first e.g. 2*(cabac_unary_length_minus1+1) context models of syntax element abs_level_greater_x, the associated context parameter shift 0 may, for example, be set to CtxParameterList[setId][0], shift1 may, for example, be set to CtxParameterList[setId][1], pStateIdx0 may, for example, be set to CtxParameterList[setId][2], and pStateIdx1 may, for example, be set to CtxParameterList[setId][3], where i may, for example, be the index of the context model and where setId may, for example, be equal to ShiftParameterIdsAbsGrX[i].

[0397] Further embodiments according to the invention may comprise a decoding process flow, e.g. as explained in the following.

[0398] In general, inputs to this process may, for example, be all bin strings of the binarization of the requested syntax element.

[0399] Output of this process may, for example, be the value of the syntax element.

[0400] This process may specify how e.g. each bin of a bin string is parsed e.g. for each syntax element. After parsing e.g. each bin, the resulting bin string may, for example, be compared to e.g. all bin strings of the binarization of the syntax element and the following may apply: [0401] If the bin string is equal to one of the bin strings, the corresponding value of the syntax element may, for example, be the output. [0402] Otherwise (the bin string is not equal to one of the bin strings), the next bit may, for example, be parsed.

[0403] While parsing each bin, the variable binIdx may, for example, be incremented by 1 starting with binIdx being set equal to 0 for the first bin.

[0404] The parsing of each bin may, for example, be specified by the following two ordered steps: [0405] 1. A derivation process for ctxIdx and bypassFlag as may, for example, be invoked e.g. with binIdx as input and ctxIdx and bypassFlag as outputs. [0406] 2. An arithmetic decoding process may, for example, be invoked with ctxIdx and bypassFlag as inputs and the value of the bin as output.

[0407] Further embodiments according to the invention may comprise a derivation process of ctxInc for the syntax element sig_flag.

[0408] Inputs to this process may, for example, be the sig_flag decoded before the current sig_flag, the state value stateId, the associated sign_flag, if present, and, if present, the co-located parameter level (coLocParam) from the incremental update decoded before the current incremental update. If no sig_flag was decoded before the current sig_flag, it may, for example, be inferred to be 0. If no sign_flag associated with the previously decoded sig_flag was decoded, it may, for example, be inferred to be 0. If no co-located parameter level from an incremental update decoded before the current incremental update is available, it is inferred to be 0. A co-located parameter level means the parameter level in the same tensor at the same position in previously decoded incremental update.

[0409] Output of this process is the variable ctxInc.

[0410] The variable ctxInc is derived as follows: [0411] If coLocParam is equal to 0 the following applies: [0412] If sig_flag is equal to 0, ctxInc is set to stateId*3. [0413] Otherwise, if sign_flag is equal to 0, ctxInc is set to stateId*3+1. [0414] Otherwise, ctxInc is set to stateId*3+2. [0415] If coLocParam is not equal to 0 the following applies: [0416] If coLocParam is greater than 1 or less than 1, ctxInc is set to stateId*2+24. [0417] Otherwise, ctxInc is set to stateId*2+25.

[0418] Further embodiments according to the invention may comprise a derivation process of ctxInc for the syntax element sign_flag.

[0419] Inputs to this process may, for example, be the sig_flag decoded before the current sig_flag, the associated sign_flag, if present, and, if present, the co-located parameter level (coLocParam) from the incremental update decoded before the current incremental update. If no sig_flag was decoded before the current sig_flag, it may, for example, be inferred to be 0. If no sign_flag associated with the previously decoded sig_flag was decoded, it may, for example, be inferred to be 0. If no co-located parameter level from an incremental update decoded before the current incremental update is available, it may, for example, be inferred to be 0. A co-located parameter level means the parameter level in the same tensor at the same position in previously decoded incremental update.

[0420] Output of this process may, for example, be the variable ctxInc.

[0421] The variable ctxInc may, for example, be derived as follows: [0422] If coLocParam is equal to 0 the following may apply: [0423] If sig_flag is equal to 0, ctxInc may, for example, be set to 0. [0424] Otherwise, if sign_flag is equal to 0, ctxInc may, for example, be set to 1. [0425] Otherwise, ctxInc may, for example, be set to 2. [0426] If coLocParam is not equal to 0 the following may apply: [0427] If coLocParam is less than 0, ctxInc may, for example, be set to 3. [0428] Otherwise, ctxInc may, for example, be set to 4.

[0429] Further embodiments may comprise a derivation process of ctxInc for the syntax element abs_level_greater_x[j]

[0430] Inputs to this process may, for example, be the sign_flag decoded before the current syntax element abs_level_greater_x[j] and, if present, the co-located parameter level (coLocParam) from the incremental update decoded before the current incremental update. If no co-located parameter level from an incremental update decoded before the current incremental update is available, it may, for example, be inferred to be 0. A co-located parameter level means the parameter level in the same tensor at the same position in previously decoded incremental update.

[0431] Output of this process may, for example, be the variable ctxInc.

[0432] The variable ctxInc may, for example, be derived as follows: [0433] If coLocParam is equal to zero the following may apply: [0434] If sign_flag is equal to 0, ctxInc may, for example, be set to 2*j. [0435] Otherwise, ctxInc may, for example, be set to 2*j+1. [0436] If coLocParam is not equal to zero the following may apply: [0437] If coLocParam is greater or equal to j or is lower or equal to j, ctxInc may, for example, be set to 2*j+2*maxNumNoRemMinus1 [0438] Otherwise, ctxInc may, for example, be set to 2*j+2*macNumNoRemMinus1+1.

[0439] Further remarks:

[0440] In the following, different inventive embodiments and aspects will be described in a chapter Introduction, and in a chapter Parameter update tree (PUT) (example) and their respective subchapters.

[0441] Also, further embodiments will be defined by the enclosed claims.

[0442] It should be noted that any embodiments as defined by the claims can be supplemented by any of the details (features and functionalities) described in the above mentioned chapters and/or subchapters respectively and/or by any of the details (features and functionalities) described in the above disclosure.

[0443] Also, the embodiments described in the above mentioned chapters and/or subchapters can be used individually, and can also be supplemented by any of the features in another chapter, or by any feature included in the claims.

[0444] Also, it should be noted that individual aspects described herein can be used individually or in combination. Thus, details can be added to each of said individual aspects without adding details to another one of said aspects.

[0445] It should also be noted that the present disclosure describes, explicitly or implicitly, features usable in a neural network encoder (apparatus for providing an encoded representation of neural network parameters) and in a neural network decoder (apparatus for providing a decoded representation of neural network parameters on the basis of an encoded representation). Thus, any of the features described herein can be used in the context of a neural network encoder and in the context of a neural network decoder.

[0446] Moreover, features and functionalities disclosed herein relating to a method can also be used in an apparatus (configured to perform such functionality). Furthermore, any features and functionalities disclosed herein with respect to an apparatus can also be used in a corresponding method. In other words, the methods disclosed herein can be supplemented by any of the features and functionalities described with respect to the apparatuses.

[0447] Also, any of the features and functionalities described herein can be implemented in hardware or in software, or using a combination of hardware and software, as will be described in the section implementation alternatives.

[0448] The following section (comprising for example subsections or chapters 1 to 2) may be titled efficient signaling of neural network updates in distributed scenarios.

1. Introduction

[0449] In the following, background information of embodiments according to the invention is provided. However, features and/or functionalities and/or details and/or aspects explained in this chapter may optionally be incorporated in a plurality of embodiments of the invention, both individually and taken in combination. Embodiments according to the invention may comprise or may be used with said aspects and/or features.

[0450] As Neural Networks (NN) have led to break-throughs in many application areas, also efficient transmission mechanisms are developing. This may allow e.g. for carrying out the, for example possible complex, NN training process, for example, in central server devices and optionally may allow to transmit a trained NN e.g. to client devices. Here, neural network compression and representation has been standardized recently. A newer field of applications are federated learning (FL) and training scenarios, where NNs may be trained, for example, on many devices, for example at the same time. In FL scenarios, e.g. frequent communication between client devices and central server devices may be beneficial or even required. Initially, a first version of a pre-trained NN may be sent to all clients, for example, for further training, using e.g. neural network compression. Then, all clients may further train the NN, and may send an updated NN version to one (or more) servers, for example as shown in FIG. 14.

[0451] FIG. 14 shows a schematic view of an example of federated learning scenario according to embodiments of the invention. FIG. 14 shows a plurality of clients 1410, 1420, 1430, that may be configured to train neural networks. A server 1430 may receive training results, e.g. updates, of respective clients and may, based on the training results, e.g. neural network parameters, provide aggregated updated neural network parameters to the clients.

[0452] An update may e.g. be a difference signal between the initial NN and the newer NN version, for example, at the client. Accordingly, one or more, or for example all arrows between server and clients (e.g. in FIG. 14) may represent sending an NN update. The server may then collect some or, for example, all local client versions and may aggregate a new server version of the NN. The aggregation process can, for example, be a simple averaging of a plurality or for example all available network version, or a more advanced process, such as, for example, only averaging output labels. This latter method is known as federated distillation and may allow for, for example, much more flexibility, for example, at local clients. This can, for example, even include different networks and/or topologies, for example, at each client, for example, as long as the output labels of all client versions can still be aggregated at a server. The process up to here is also called a communication round (CR). Then, for example, the new server version may be sent again to a plurality or for example all clients, e.g. also as difference signal to a previous NN version, for example, for further training and the process may repeat. The FL scenario may continue for many CRs, e.g. until a certain precision (for inference) is reached. In general, FL scenarios may allow communication flexibility. In a synchronous communication scenario, a plurality or for example all N clients may send updates in each communication round, the server may aggregate the versions or for example the N versions into a new version and may then send this version to the plurality or for example all N clients, for example, for the next CR. In an asynchronous FL scenario, clients may only send updates after a certain number of CRs. Or may send updates for a successive number of CRs and may then pause for a certain time. This may mean that a server may only have a subset K<N client versions at a certain CR, for example, for aggregating a new server version, for example, to be sent to the clients.

[0453] Many or for example all these conditions in federated scenarios may entail specific signaling, for example, in order to handle the different FL variations, but optionally also, for example, to increase coding efficiency of NN update and/or difference data to be sent between server(s) and clients (e.g. in both directions). Accordingly, the following subsections describe related aspects of the invention.

2. Parameter Update Tree (PUT) (Example)

[0454] As an example, considering a, for example distributed, scenario according to embodiments with several devices (e.g. including server and client devices) and, for example, a so-called base model, which may be a neural network for which updates shall be or may be transmitted between devices. Embodiments according to this invention comprise and/or describe a scheme for representing such updates, for example, using a tree structure. A plurality or for example each individual parameter (or a group of parameters) of the base model may be associated with the root node of a tree, e.g. the tree having the beforementioned tree structure. An update for such a parameter may correspond to a child node attached to the root node. The child node may contain instructions, for example, on how to update the parameter associated with the parent node. Any node of the tree may further be updated, for example, by attaching child nodes in the same manner. An example is given in FIG. 15 where R is the root node representing one parameter of the base model.

[0455] FIG. 15 shows a schematic view of an example of parameter update tree, for example exemplary parameter update tree, according to embodiments of the invention.

[0456] Nodes U1 and U2 may describe updates to node R and node U3 may describe an update of node U2. Each node of the tree may represent a version of the parameter of the base model R and one could, for example, decide to execute the model using a particular updated version U1, U2, or U3 instead of R. In other words, according to embodiments a decoder may, for example, be configured to decide to execute the model using a particular updated version U1, U2, or U3 instead of R, for example corresponding to a specific version of neural network parameters.

[0457] In order to represent such a tree, a unique node identifier may be associated with a plurality of nodes or for example each node. For example, this could be or may be an integer number, a string, and/or a cryptographic hash (like e.g. SHA-512) associated with the node. However, within such a tree, each node identifier may be unique or for example must even be unique.

[0458] In a distributed scenario, a plurality of devices or for example each device (e.g. server or client) may maintain update trees, for example, for the parameters of the model. For example, in order to transmit a particular update from one device to another, for example only, the corresponding update along with the associated node identifier (e.g. a pointer to the parent node of the update) may be transmitted or may need to be transmitted.

[0459] In the following an example according to embodiments is disclosed in order to provide a better understanding of aspects of the invention. For example, consider a client and a server that both have node R (e.g. as in FIG. 15) available. Assume, the client creates update U2 and wants to make it available to the server. For this it may send U2 along with the node identifier of R to the server. The server may search for the node identifier of R in its version of the tree and may append U2 as a child node of R. Now, server and client may both have a tree available with R as root node and U2 as a child node of R.

2.1 Parameter Update Procedure (Example)

[0460] This section describes several different types of update instructions, for example, associated with nodes of a PUT according to embodiments of the invention. The parameter of the base model that may be associated with a PUT may be or shall be denoted tree parameter. For example, a so-called node parameter may be or shall be associated with each node of the PUT. However, in other example, a so-called node parameter may or shall be associated with, for example, a set of nodes, e.g. a reachable (e.g. reachable from the root node to a specific current node) set of nodes, or, for example, a set of consecutive nodes, starting from the root node, of the PUT, or, for example, associated with each node of the PUT. This node parameter may be derived by traversing the PUT, for example, from the root node to the desired node and applying the update instructions, for example, of each visited node to the tree parameter. For example (e.g. as in FIG. 15), the node parameter of R may equal the tree parameter and the node parameter of U1 may equal the tree parameter, for example, after applying update instructions of U1.

[0461] In the case where a node parameter (and, for example, consequently also the tree parameter) is a tensor (i.e., for example, a multi-dimensional arrays of values), it may be or shall be denoted node tensor.

[0462] In an embodiment, update instructions, for example, associated with a node may contain a so-called product tensor, for example, of the same shape as the node tensor. Updating the parameter may correspond to an element-wise product, for example, of the node tensor elements and the product tensor elements.

[0463] In an embodiment, update instructions, for example, associated with a node may contain at least one of a so-called sum tensor, for example, of the same shape as the node tensor, a scalar node tensor weight value, and/or a scalar sum tensor weight value. Updating the parameter may correspond to an element-wise weighted sum, for example, of the node tensor elements and the sum tensor elements. As an example, or for example more precisely, each element of the node tensor may be multiplied with the node tensor weight value, each element of the sum tensor may be multiplied with the sum tensor weight value, and then, the element-wise sum of both scaled tensors may be calculated. Note that both weights may also be set to 1, which may correspond to a non-weighted sum, for example, as a special case.

[0464] In another embodiment, update instructions, for example, associated with a node may contain a so-called replace tensor, for example, of the same shape as the node tensor. Updating the parameter may correspond to replacing the values of the node tensor, for example, with the values of the replace tensor.

[0465] In another embodiment, update instructions that employ an update tensor (as for example a replace tensor, a sum tensor, and/or a product tensor) may involve an, for example implicit, tensor shape conversion, for example, as follows.

[0466] In the following an example according to embodiments is disclosed in order to provide a better understanding of aspects of the invention.

[0467] Assume, the update tensor shape is identical to the node tensor shape except for one or more individual dimensions, which may equal 1. For example, for dimensions of the update tensor that are 1, it may be or shall be assumed that tensor elements along this axis are the same. For example, assume, the node tensor is given as 2D tensor [[a, b, c], [d, e, f]] (dimensions are [2, 3]). An update tensor given as [[x],[y]] (dimensions are [2, 1]) may or would implicitly be extended to [[x, x, x], [y, y, y]]. An update tensor given as [[z]] (dimensions are [1, 1]) may or would implicitly be extended to [[z, z, z], [z, z, z]]. An update tensor given as [[r, s, t]] (dimensions are [1, 3]) may or would implicitly be extended to [[r, s, t], [r, s, t]]. In other words, a decoder according to embodiments may, for example, be configured to update a tensor shape according to the example explained above. [0468] TBD: quantized-domain updates (check effects of different qps) (optional)

[0469] Optionally, quantized domain updates may, for example, be used (e.g. alternatively, or in combination with the above concepts).

2.2 Server-Side Update Aggregation Example

[0470] In a distributed scenario, according to embodiments of the invention, where a server may maintain a base model and, for example, may receive updates from, for example, different clients, the PUT at the server side may collect several update nodes, for example, for the same model. The server may decide to combine several update nodes and may, for example, create a new update node, for example, from this combination and may for example distribute it to the clients, for example, as a collectively updated model. A plurality or for example each node may then decide to continue federated learning, for example, based on this collectively updated model.

2.3 Parameter Update Tree Signaling (Example)

[0471] This section presents methods for representing elements of a PUT, for example, as data units (e.g. byte sequences) e.g. optimized for transmission between devices (e.g. including servers and clients) according to embodiments of the invention. It, for example the methods, may be combined with existing techniques for representing neural networks like, e.g., the emerging standard ISO/IEC 15938 Part 17: Compression of neural networks for multimedia content description and analysis, which is referred to as NNR throughout this document. NNR may represent individual parameters of a neural network as so-called compressed data units (NDU). Implementing the PUT concept, for example, by using the efficient compression techniques available in NNR can, for example, be carried out as follows. In accord with embodiments of the invention, consider the case where a node contains an update tensor (for example or like a replace tensor, a sum tensor, and/or a product tensor, for example, as described above). Such an update tensor can be, for example efficiently, represented as an NDU. In order to extend an NDU with, for example all necessary, information, for example, so that it can be used as a PUT node, further syntax elements can, for example, be added.

[0472] In an embodiment, a new syntax element parent_node_id_present_flag may be introduced, for example, into the nnr_compressed_data_unit_header of an NDU, for example indicating whether a parent node identifier is present in the NDU. Depending on the value of parent_node_id_present_flag, a further new syntax element parent_node_id may be transmitted that, for example, uniquely identifies another NDU that may contain the parent node of the current PUT node.

[0473] In another embodiment, the parent_node_id may be a cryptographic hash (like, e.g., SHA-512) of the parent NDU.

[0474] In another embodiment, the parent_node_id may be a cryptographic hash (like, e.g., SHA-512) of the nnr_compressed_data_unit_payload of the parent NDU.

[0475] In another embodiment, the parent_node_id may be a combined value representing a device identifier and/or a serial number of which both may, for example, be associated with the parent NDU.

[0476] In another embodiment, a new syntax element node_id may be encoded in the parent node, to be used as parent_node_id of child nodes of the parent node. node_id may, for example, be a unique identifier.

[0477] In another embodiment, syntax element node_id (which may, for example, uniquely identify a node) may be composed of a device identifier and/or a parameter update tree depth information (i.e., for example an, information about the number of nodes visited when walking the tree from the current node to the root node) and/or a parameter update tree identifier.

[0478] In another embodiment, a flag may be signaled for a node, indicating whether a node_id is present. Depending on the value of this flag, a syntax element node_id may be present or not.

[0479] In another embodiment, a new syntax element parent_node_id_type may indicate of which type the syntax element parent_node_id is. For example, possible different types of parent_node_id may be as described in the previous embodiments.

[0480] In another embodiment, it may depend on the value of syntax element parent_node_id_present flag whether syntax element parent_node_id_type is signaled or not.

2.3.1 Topology Change Signaling (Example)

[0481] In a distributed scenario, it may sometimes be of interest to change the topology of a neural network. For example, the number of output neurons of a particular layer may or could be increased or decreased.

[0482] In an embodiment, a syntax element shape_update may be signaled within an NDU, for example, indicating whether the shape of the tensor associated with the parent node is modified or not. In the case where the parent tensor shape is updated, new tensor dimensions may be transmitted (e.g. using syntax element tensor_dimensions).

Implementation Alternatives

[0483] Although some aspects are described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus.

[0484] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

[0485] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0486] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0487] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0488] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0489] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitionary.

[0490] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.

[0491] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0492] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0493] A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0494] In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods may be performed by any hardware apparatus.

[0495] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0496] The apparatus described herein, or any components of the apparatus described herein, may be implemented at least partially in hardware and/or in software.

[0497] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0498] The methods described herein, or any components of the apparatus described herein, may be performed at least partially by hardware and/or by software.

[0499] While this invention has been described in terms of several embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations and equivalents as fall within the true spirit and scope of the present invention.