JOINT SOURCE CHANNEL CODING FOR NOISY CHANNELS USING NEURAL NETWORKS

Abstract

A communication system for conveying information from an information source across a communications channel using a joint source channel coding variational autoencoder, comprising an encoder neural network of the joint source channel coding variational autoencoder, the encoder neural network having an input layer having input nodes corresponding to a sequence of source symbols S={S1, S2, . . . , Sm}, the Si, taking values in a finite alphabet S, received at the input layer from the information source as samples thereof, and a channel input layer coupled to the input layer through one or more neural network layers, the channel input layer having nodes corresponding to a channel input distribution vector Zk={Z1, Z2, . . . , ZK}, the Zi, taking values for parameters defining a plurality of distributions, each distribution being sampleable to provide possible values for the Xi, of a channel input vector Xn={X1, X2, . . . , Xn}, the Xi taking values from the available input signal values of the communications channel, wherein the encoder neural network is configured through training to map sequences of source symbols Sm received from the information source directly to a representation as a plurality of distributions that provide possible values for the Xi, of a channel input vector Xn, usable to drive a transmitter to transmit a corresponding signal over the communications channel; a sampler, configured to produce a channel input vector Xn={X1, X2, . . . , Xn} in use by sampling the respective distribution for each channel input X defined by the channel input distribution vector Zk={Zi, Z2, . . . , ZK} output by the channel input layer of the encoder neural network; and a decoder neural network of the joint source channel coding variational autoencoder, the decoder neural network having a channel output layer having nodes corresponding to a channel output vector Yn received from a receiver receiving the signal Xn transmitted by the transmitter and transformed by the communications channel, and an output layer coupled to the channel output layer through one or more neural network layers, having nodes matching those of the input layer of the encoder neural network, wherein the decoder neural network is configured through training to map the representation of the source symbols as the channel output vector Yn transformed by the communications channel to a reconstruction of the source symbols Sm output from the output layer of the joint source channel coding variational autoencoder, the reconstruction of the source symbols Sm being usable to reconstitute the information source.

Claims

1. A communication system for conveying information from an information source across a communications channel using a joint source channel coding variational autoencoder, comprising: an encoder neural network of the joint source channel coding variational autoencoder, the encoder neural network having: an input layer having input nodes corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in an alphabet S, received at the input layer from the information source as samples thereof, and a channel input layer coupled to the input layer through one or more neural network layers, the channel input layer having nodes corresponding to a channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K}, the Z.sub.1 taking values for parameters defining a plurality of distributions, each distribution being sampleable to provide possible values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal alphabet X of the communications channel, wherein the encoder neural network is configured through training to map sequences of source symbols S.sup.m received from the information source directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over the communications channel; a sampler, configured to produce a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n} in use by sampling the respective distribution for each channel input X.sub.i defined by the channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K} output by the channel input layer of the encoder neural network; and a decoder neural network of the joint source channel coding variational autoencoder, the decoder neural network having: a channel output layer having nodes corresponding to a channel output vector Y.sup.n received from a receiver receiving the signal corresponding to channel input vector X.sup.n transmitted by the transmitter and transformed by the communications channel, and an output layer coupled to the channel output layer through one or more neural network layers, having nodes matching those of the input layer of the encoder neural network, wherein the decoder neural network is configured through training to map the representation of the source symbols as the channel output vector Y.sup.n transformed by the communications channel to a reconstruction of the source symbols Ŝ.sup.m output from the output layer of the joint source channel coding variational autoencoder, the reconstruction of the source symbols Ŝ.sup.m being usable to reconstitute the information source.

2. The communication system of claim 1, wherein the plurality of distributions have the same distribution type.

3. The communication system of claim 1, wherein the communications channel is a noisy communications channel and wherein the distribution type of the plurality of distributions is based on the characteristics of the communication channel.

4. (canceled)

5. The communication system of claim 1, wherein the distribution type is the optimal input distribution for the channel.

6. The communication system of claim 1, wherein the communications channel is modelled as a Gaussian channel, the plurality of distributions are Gaussian distributions and the parameters defining the distributions are the mean and standard deviation.

7. The communication system of claim 1, wherein the size n of the channel input vector X.sup.n is smaller than the size m of the source symbols S.sup.m and the information source is compressed during mapping.

8. The communication system of claim 1, wherein the size n of the channel input vector X.sup.n is based on the channel capacity.

9. The communication system of claim 1, wherein the information source is an image and the sequence of source symbols S.sup.m represents pixels of the image and wherein each of the plurality of distributions represents a feature of the image.

10. (canceled)

11. The communication system of claim 1, wherein the encoder neural network and decoder neural network are trained jointly.

12. The communication system of claim 1, wherein the encoder neural network and decoder neural network are trained jointly to minimise the difference between the input and output of the joint source channel coding variational autoencoder.

13. The communication system of claim 1, wherein the encoder neural network is trained to learn the parameters of the plurality of distributions.

14. The communication system of claim 1, wherein the encoder neural network and decoder neural network are: trained jointly using the backpropagation algorithm or the stochastic gradient descent training algorithm; trained jointly based on the type of communications channel; trained jointly based on a model of the communication channel, and the decoder neural network is trained further based on the real communication channel using one way communication of known training data samples from the transmitter to the receiver; trained to meet channel input constraints; trained jointly based on the type of information source, or for a given information source; and/or trained jointly based on the measure of distance from a target distribution to the plurality of distributions, wherein the distance from the target distribution is measured using KL divergence.

15.-18. (canceled)

19. The communication system of claim 1, wherein the X.sub.i of the channel input vector X.sup.n belongs to a set of complex numbers, corresponding to the I and Q components.

20. (canceled)

21. The communication system of claim 1, wherein the channel input vector X.sup.n comprises one sample from each distribution of the plurality of distributions.

22.-23. (canceled)

24. The communication system of claim 1, wherein the channel input layer is coupled to the input layer through at least five neural network layers.

25. The communication system of claim 1, wherein the difference between the sequence of source symbols S.sup.m and the reconstruction of the source symbols Ŝ.sup.m increases as the noise of the communications channel increases.

26. The communication system of claim 1, wherein the information source has not been compressed before being input into the joint source channel coding variational autoencoder.

27. (canceled)

28. A transmitter device comprising: an encoder neural network of a joint source channel coding variational autoencoder, the encoder neural network comprising: an input layer having input nodes corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in a finite alphabet S, received at the input layer from an information source as samples thereof, and a channel input layer coupled to the input layer through one or more neural network layers, the channel input layer having nodes corresponding to a channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K}, the Z.sub.i taking values for parameters defining a plurality of distributions, each distribution being sampleable to provide possible values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal alphabet X of the communications channel, wherein the encoder neural network is configured through training to map sequences of source symbols S.sup.m received from the information source directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n usable to drive a transmitter to transmit a corresponding signal over a communications channel; a sampler, configured to produce a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n} in use by sampling the respective distribution for each channel input X.sub.i defined by the channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K} output by the channel input layer of the encoder neural network; and a transmitter to transmit the channel input vector X.sup.n over the communications channel.

29. A transmitter device as claimed in claim 28 comprising a plurality of said encoder neural networks, each trained for joint source channel coding of different given information sources or types of information source, for transmission to and decoding thereof at a receiver.

30. (canceled)

31. A training method of a communication system for conveying information from an information source across a communications channel using a joint source channel coding variational autoencoder, the training method comprising: mapping, by an encoder neural network of a joint source channel coding variational autoencoder, sequences of source symbols S.sup.m received from the information source directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over the communications channel; and mapping, by a decoder neural network of the joint source channel coding variational autoencoder, the representation of the source symbols as the channel output vector Y.sup.n transformed by the communications channel to a reconstruction of the source symbols Ŝ.sup.m output from the output layer of the joint source channel coding variational autoencoder, comparing the sequences of source symbols Ŝ.sup.m to the reconstruction of source symbols Ŝ.sup.m, amending parameters of the joint source channel coding variational autoencoder based on the comparison, and repeating the above steps until the reconstruction of source symbols Ŝ.sup.m is usable to reconstitute the information source.

32.-33. (canceled)

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0139] Embodiments of the invention are further described hereinafter with reference to the accompanying drawings, in which:

[0140] FIG. 1 provides an example illustration of a neural network.

[0141] FIG. 2 provides an example of a joint source channel coding autoencoder.

[0142] FIG. 3 provides an illustration of a communication system according to an example.

[0143] FIG. 4 provides an illustration of a transmitter device according to an example.

[0144] FIG. 5 provides an illustration of a transmitter device according to another example.

[0145] FIG. 6 provides an illustration of a receiver device according to an example.

[0146] FIG. 7 provides another example of a joint source channel coding autoencoder.

[0147] FIG. 8 provides an illustration of a communication system according to another example.

[0148] FIG. 9 provides an example flowchart of a method of an encoder neural network.

[0149] FIG. 10 provides an example flowchart of a training method of a communication system.

[0150] FIG. 11 provides an example of a joint source channel coding variational autoencoder.

[0151] FIG. 12 provides another illustration of a communication system according to an example.

[0152] FIG. 13 provides an illustration of the sampler according to an example.

[0153] FIG. 14 provides another example flowchart of a method of an encoder neural network.

[0154] FIG. 15 provides another example flowchart of a training method of a communication system.

[0155] FIG. 16 provides another illustration of a transmitter device according to an example.

[0156] FIG. 17 is a schematic diagram showing a computer according to an example.

[0157] FIG. 18 provides another illustration of a communication system according to an example.

DETAILED DESCRIPTION

[0158] The present disclosure describes a communication system for conveying information from an information source across a communications channel using a joint source channel coding autoencoder, the communication system producing a high fidelity reconstruction of the information source for different information sources and different channel noise.

[0159] The communications channel is used to convey information from one or more transmitters to one or more receivers. The channel may be a physical connection, e.g. a wire, or a wireless connection such as a radio channel. The communications channel may be an optical channel or a Bluetooth channel. There is an upper limit to the performance of a communication system which depends on the system specified. In addition, there is also a specific upper limit for all communication systems which no system can exceed. This fundamental upper limit is an upper bound to the maximum achievable rate of reliable communication over a noisy channel and is known as Shannon's capacity.

[0160] The communications channel, including the noise associated with such a channel, is modelled and defined by its characteristics and statistical properties. Channel characteristics can be identified by comparing the input and output of the channel, the output of which is likely to be a randomly distorted version of the input. The distortion indicates channel statistics such as additive noise, or other imperfections in the communication medium such as fading or synchronization errors between the transmitter and the receiver. Channel characteristics include the distribution model of the channel noise, slow fading and fast fading. Common channel models include binary symmetric channel and additive white Gaussian noise (AWGN) channel.

[0161] Neural networks are machine learning models that employ multiple layers of nonlinear units (known as artificial “neurons”) to generate an output from an input. Neural networks learn through training to produce the optimum output. They help to group unlabelled data according to similarities and can classify data when they have a labelled dataset to train on. Neural networks may be composed of several layers, each layer formed from nodes. Neural networks can have one or more hidden layers in addition to the input layer and the output layer. The output of each layer is used as the input to the next layer (the next hidden layer or the output layer) in the network. Each layer generates an output from its input using a set of parameters, which are optimized during the training stage. For example, each layer comprises a set of nodes, the nodes having learnable biases and their inputs having learnable weights. Learning algorithms can automatically tune the weights and biases of nodes of a neural network to optimise the output.

[0162] FIG. 1 provides an example illustration of a neural network 100. Neural network 100 has a plurality of nodes and connections. The nodes are arranged to be in layers. The layers are illustrated in dashed lines to show that they are simply groupings of nodes rather than being separate objects. The neural network 100 comprises an input layer 102 comprising a plurality of input nodes 104 and an output layer 122 comprising a plurality of output nodes 124. The neural network 100 further comprises a plurality of hidden layers 106, 110, 114, 118 comprising a plurality of nodes 108, 112, 116, 120. The arrows between nodes illustrate connections. The nodes 104 of the input layer 102 are connected to nodes 108 of the hidden layer 106. The nodes 108 of the hidden layer 106 are connected to nodes 112 of the hidden layer 110. The nodes 112 of the hidden layer 110 are connected to nodes 116 of the hidden layer 114. The nodes 116 of the hidden layer 114 are connected to nodes 120 of the hidden layer 118. The nodes 120 of the hidden layer 118 are connected to the nodes 124 of the output layer 122. The connection between nodes means that the output of one node is the input to the next node. The output of the node may be a numerical value. The connection may have a weight associated with it which adjusts the output value of the previous node before inputting into the next node.

[0163] Although four hidden layers are illustrated in the figure, a neural network may comprise any number of hidden layers. Moreover, although a certain number of nodes is shown for each layer, this is merely for illustration and a layer may comprise any number of nodes. The neural network layers may also be three dimensional.

[0164] An autoencoder is a neural network that is trained to copy its input to its output. Autoencoders can provide dimensionality reduction at a hidden layer arranged to have fewer nodes than the input layer. Autoencoders also provide feature learning. An autoencoder includes an encoder neural network and a decoder neural network. These two networks are trained jointly to generate a reconstruction of the input. Typically, either the output of the encoder network has a lower dimension than the input, so that a low-dimensional representation of the input is learned; or, a loss function that imposes certain properties on the encoder output is used, that can act as a regularizer to prevent the network from learning the identity function.

[0165] In an autoencoder, the goal of the system is to make sure that each input can be regenerated based on the encoder output. The encoder therefore tries to learn the most essential features of input data samples from an information source. As the autoencoder is trained to learn the input data samples from an information source, for multiple input data sources, multiple autoencoders each trained for their specific input samples provide a highly efficient system that is adaptable for multiple different input data sources. The autoencoder will be explained further in the following figures.

[0166] The autoencoder in accordance with examples of the present disclosure can be used in a communication system for conveying information from an information source across a communications channel. The encoder neural network of the autoencoder can replace the traditional source encoder and channel encoder. The encoder neural network maps the signal directly to input signal values in the channel input layer, where it is transmitted over the communications channel. As will be explained in more detail below, the autoencoder may be a variational autoencoder in which the encoder neural network maps the signal directly to parameters of distributions in the input signal values of the channel input layer, where the distributions are sampled to produce the channel input vector that is transmitted over the communications channel. The decoder neural network of the autoencoder can replace the traditional source decoder and channel decoder. The decoder neural network receives the output of the communications channel and reconstructs the signal. The encoder neural network and decoder neural network are trained to undo the effect of the channel on the signal, such as the effect of the channel noise of the signal. The encoder neural network and decoder neural network can also be trained to learn a representation of the source signal and recover the input with the highest fidelity possible. Further, as the neural networks are trained to optimise the mapping, the efficiency of the transmission of the information source over the communications channel can be greater than in separate source and channel coding, where the separate removal and subsequent addition of redundancy may be inherently inefficient for transferring the data.

[0167] Thus, the encoder and decoder neural networks can be used as a form of joint source channel coding. This provides simplicity over having two separate systems for encoding and two separate systems for decoding. However, the limitations of the joint source channel coding techniques as mentioned above are removed. In particular, an autoencoder used for joint source channel coding, herein referred to as the joint source channel coding autoencoder, can be trained for use with different channels and different input sources. Thus, using neural networks for conveying information across a channel enables a high degree of freedom.

Joint Source Channel Coding Autoencoder

[0168] FIG. 2 provides an example of a joint source channel coding autoencoder 200. The joint source channel coding autoencoder 200 comprises an encoder neural network 202 and a decoder neural network 204. The encoder neural network and decoder neural network may be deep and/or recurrent.

[0169] For example, with reference to FIG. 1, the neural network of FIG. 1 may be a joint source channel coding autoencoder 200. The encoder neural network 202 of the joint source channel coding autoencoder 200 may be formed from layers 102, 106 and 110. The decoder neural network 204 of the joint source channel coding autoencoder 200 may be formed from layers 114, 118 and 122.

[0170] With reference to FIGS. 1 and 2, the encoder neural network 202 of the joint source channel coding autoencoder 200 has an input layer 102 having input nodes 104 corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in a finite alphabet S, received at the input layer 102 from an information source as samples thereof. The encoder neural network 202 also has a channel input layer 110 coupled to the input layer through one or more neural network layers 106, the channel input layer 110 having nodes 112 usable to provide values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal values of a communications channel.

[0171] The encoder neural network 202 is configured through training to be usable to map sequences of source symbols S.sup.m received from the information source directly to a representation as a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over the communications channel.

[0172] In an example, the channel input layer 110 is coupled to the input layer 102 through at least five neural network layers 106. The encoder neural network 202 may directly map the input nodes 104 at the input layer 102, corresponding to a sequence of source symbols S.sup.m, to the nodes 112 of the channel input layer 110, corresponding to a channel input vector X.sup.n, through the neural network.

[0173] In the communication system, for a wireless communications channel in accordance with an example, each input signal X.sub.i of the channel input vector X.sup.n may belong to a set of complex numbers, corresponding to the I and Q components. In an example, the encoder neural network 202 of the joint source channel coding autoencoder 200 may perform bandwidth compression. For example, the encoder neural network 202 may compress the information source during mapping such that the size n of the channel input vector X.sup.n is smaller than the size m of the source symbols S.sup.m Alternatively, the joint source channel coding autoencoder 200 may perform a bandwidth expansion, where the size n of the channel input vector X.sup.n is larger than the size m of the source symbols S.sup.m.

[0174] When the neural network of FIG. 1 is modelled as a joint source channel coding autoencoder, channel input layer 110 and channel output layer 114 are not directly connected, as indicated by the dashed arrows connecting the nodes of these layers. The communications channel is between channel input layer 110 and channel output layer 114. Thus, the values output from the encoder neural network may not be equal to the values input into the decoder neural network (for example due to noise on the communications channel).

[0175] The decoder neural network 204 is to map the channel output signals received at the channel output layer 114 to a reconstructed version of the channel input vector S.sup.m. The decoder neural network 204 may be a standard decoder of an autoencoder. The mapping may be a deterministic mapping. (Alternatively, where the autoencoder is a variational autoencoder) the mapping may be a probabilistic mapping.

[0176] The decoder neural network 204 of the joint source channel coding autoencoder 200 has a channel output layer 114 having nodes 116 corresponding to a channel output vector Y.sup.n received from a receiver receiving the signal X.sup.n transmitted by the transmitter and transformed by the communications channel. The decoder neural network 204 also has an output layer 122 coupled to the channel output layer 114 through one or more neural network layers 118, having nodes 124 matching those of the input layer 104 of the encoder neural network.

[0177] The decoder neural network 204 is configured through training to map the representation of the source symbols as the channel output vector Y.sup.n transformed by the communications channel to a reconstruction of the source symbols Ŝ.sup.m output from the output layer of the joint source channel coding autoencoder 200, the reconstruction of the source symbols Ŝ.sup.m being usable to reconstitute the information source.

[0178] In an example, the channel output layer 114 is coupled to the output layer 122 through at least five neural network layers 118. The decoder neural network 204 may directly map the nodes 116 at the channel output layer 114, corresponding to a channel output vector Y.sup.n, to the nodes 124 of the output layer 122, corresponding to a reconstruction of the source symbols Ŝ.sup.m, through the neural network.

[0179] The encoder neural network 202 and decoder neural network 204 may have any number of layers and each layer may comprise any number of nodes. In an example, the output layer of the encoder neural network 202 has the same number of nodes as the input layer of the decoder neural network 204. In another example, the input layer of the encoder neural network 202 has the same number of nodes as the output layer of the decoder neural network 204.

[0180] In an example, the size n of the channel input vector X.sup.n is based on the channel capacity or the channel resources allocated for use by the transmitter (for example, the radio resources, or OFDM symbols, allocated to a radio bearer for a transmitter terminal for digital (e.g. QAM) modulation thereby in an LTE communication system). Alternatively or additionally, the size n of the channel input vector X.sup.n may be based on the size m of the source symbols S.sup.m or the size m of the reconstruction of the source symbols S.sup.m. In an example, the size of the source symbols S.sup.m is not equal to the size of the reconstruction of the source symbols S.sup.m. The size m of the reconstruction of the source symbols S.sup.m may be larger than the size n of the channel output vector Y.sup.n. The size n of the channel input vector X.sup.n may be equal to the size n of the channel output vector Y.sup.n In another example, the size n of the channel input vector X.sup.n may be different from the size n of the channel output vector Y.sup.n for example, for a multiple input multiple output channel.

[0181] FIG. 3 provides an illustration of a communication system 300 according to an example. The communication system comprises an information source 302. The information source 302 may be any information source. For example, the information source 302 may be a discrete or continuous information source. If the information source 302 is a continuous information source, it may be sampled before being input into the encoder neural network 202. In an example, the information source 302 is an image and the sequence of source symbols S.sup.m are pixels of the image. For example, when the image is a colour image, the sequence of source symbols S.sup.m may be a matrix having three values for each pixel, such as the red, green and blue values when the image has RGB components.

[0182] In another example, the information source is text, video or sensor measurements. The information source may comprise a structured data set. In an example, the information source is raw, i.e. the information source has not been compressed before being input into the joint source channel coding autoencoder 200.

[0183] The information source is input into input layer 102 of the encoder neural network 202 of the joint source channel coding autoencoder 200. The encoder neural network 202 is described above with reference to FIGS. 1 and 2. The encoder neural network 202 has a channel input layer 110 coupled to the input layer 102 through one or more neural network layers 106, the channel input layer 110 having nodes 112 corresponding to a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal alphabet X of the communications channel.

[0184] The encoder neural network 202 of the joint source channel coding autoencoder 200 receives, at input nodes of an input layer 102, samples of an information source corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in a finite alphabet S. Next, the encoder neural network 202 maps the sequence of source symbols S.sup.m from the input layer through one or more neural network layers 106 to a channel input layer 110, the channel input layer 110 having nodes 112 usable to provide values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal alphabet X of the communications channel 310. The encoder neural network 202 is configured through training to be usable to map sequences of source symbols S.sup.m received from the information source 302 directly to a representation as a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over a communications channel 310.

[0185] In an example, the channel input layer 110 has nodes corresponding to channel input vector X.sup.n and the sequences of source symbols S.sup.m are mapped directly to a representation as the channel input vector X.sup.n. The values of the channel input vector X.sup.n may need to be normalized before being input into the communications channel 310 to comply with average input power constraints.

[0186] The channel input vector X.sup.n may be transmitted across communications channel 310. Communications channel 310 may be any type of communications channel. The types of communication channels and associated characteristics and statistical properties have been previously explained, and the communications channel 310 may be any of those previously explained. For example, communications channel 310 may be a noisy communications channel. The noisy communications channel is specified by its input and output, and the output may be a randomly distorted version of the input. This distortion may be at least partially caused by noise and may also be caused by other imperfections in the communication medium such as fading or synchronization errors between the transmitter and the receiver.

[0187] In an example, the communication channel transforms the channel input vector X.sup.n into the channel output vector Y.sup.n in a random manner. If the channel is memoryless, then the distribution of the channel output conditioned on the channel input is given by

[00001] $P_{Y^{n} | X^{n}} (y^{n} | x^{n}) = {.Math.}_{i = 1}^{n} P_{Y | X} (y_{i} | x_{i})$

[0188] However, the channel may have memory, which means that the output symbol at time i may depend on the channel inputs before time i as well, or the channel behaviour may change over time, for example, due to bursty noise. Typical digital communication systems use channel codes that are optimized for independent identically distributed (IID) channels.

[0189] The quality of the reconstruction of the input vector Ŝ.sup.m is measured using a prespecified distortion measure. In most cases an additive distortion measure is considered, where the distortion is given by the sum of the distortions of individual source samples. For example

[00002] $D (S^{m}, {\hat{S}}^{m}) = \frac{1}{m} Σ_{i = 1}^{m} d (S_{i}, {\hat{S}}_{.Math.})$

[0190] In case of image transmission, the distortion may be measured by the average squared error distortion.

[0191] The communications channel 310 may be an additive white Gaussian noise (AWGN) channel. Channel output vector Y.sup.n may be a distorted version of the channel input vector.

[0192] As the input to the decoder neural network becomes less similar to the output of the encoder neural network, the Bit Error Rate in the reconstruction of the information source increases.

[0193] The decoder neural network receives the channel output vector. The decoder neural network 204 is described above with reference to FIGS. 1 and 2. Thus the decoder neural network of the communication system produces a reconstruction of the source symbols Ŝ.sup.m usable to reconstitute the information source 302.

[0194] In an example, communication system 300 is a practical communication system. In this example, an input cost constraint is applied on the channel input, due to the limited energy of the transmitter or interference limitations. The cost function assigns a specific cost to each possible channel input symbol. The average cost constraint of P requires that every channel input vector satisfies:

[00003] $\frac{1}{n} {.Math.}_{i = 1}^{n} c (X_{i}) \leq P$

[0195] In a wireless communication system, the input cost constraint may be the transmission power, given by c(X)=|X|.sup.2. The cost function enables error minimisation between the information source and the reconstructed information source.

[0196] The encoder neural network 202 and decoder neural network 204 of the joint source channel coding autoencoder 200 may be trained jointly to obtain a neural network capable of sufficiently reconstructing the information source 302. For example, they may be trained jointly based on the characteristics of the communication channel, based on the type of information source, for a given information source and/or to meet channel input constraints. The encoder neural network 202 and decoder neural network 204 of the joint source channel coding autoencoder 200 may be trained jointly using performance measures. One performance measure may be the loss function.

[0197] The encoder neural network 202 and decoder neural network 204 of the joint source channel coding autoencoder 200 may be trained jointly to minimise the difference between the input and output of the joint source channel coding autoencoder 200. The difference between the input and output of the joint source channel coding autoencoder 200 may be another performance measure. The difference may be measured by observing the similarity between the information source and the reconstructed information source.

[0198] The encoder neural network 202 and decoder neural network 204 may be trained jointly using the backpropagation algorithm or the stochastic gradient descent training algorithm. In an example, in addition to being trained jointly, the decoder neural network may be trained further based on the real communication channel 310 using one way communication of known training data samples from the transmitter to the receiver. The real communication channel is the physical communication channel rather than a model of the channel characteristics that may be used on a computer for simulation. Known training data samples may be chosen for their ability to train the neural network of the joint source channel coding autoencoder 200. Known training data samples may be used for all joint source channel coding autoencoders 200 or may be chosen based on the information source 302 to be input into the joint source channel coding autoencoder 200. Alternatively, the encoder neural network 202 and decoder neural network 204 may be trained jointly using a known training set, such as the MNIST data set.

[0199] FIG. 4 provides an example of a transmitter device 400. The transmitter device 400 comprises an encoder neural network 202 of a joint source channel coding autoencoder 200 and a transmitter 404. The encoder neural network 202 of the transmitter device 400 comprises an input layer 102 having input nodes 104 corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in a finite alphabet S, received at the input layer 102 from an information source 302 as samples thereof, and a channel input layer 110 coupled to the input layer 102 through one or more neural network layers 106, the channel input layer having nodes usable to provide values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal alphabet X of the communications channel, wherein the encoder neural network is configured through training to be usable to map sequences of source symbols S.sup.m received from the information source directly to a representation as a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over a communications channel. The transmitter 404 is to transmit the channel input vector X.sup.n over the communications channel.

Transmitter/Receiver Having Multiple Joint Source Channel Coding Autoencoders for Multiple Information Sources

[0200] FIG. 5 provides another example of a transmitter device 500. The transmitter device 500 comprises a plurality of encoder neural networks 212, 222, each encoder neural network 212, 222 of a joint source channel coding autoencoder 200, and each encoder neural network 212, 222 trained to receive a different information source 312, 322 or a different type of information source 312, 322. Each encoder neural network 212, 222 comprises: an input layer 102 having input nodes 104 corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in a finite alphabet S, received at the input layer from an information source as samples thereof, and a channel input layer 110 coupled to the input layer 102 through one or more neural network layers 106, the channel input layer 110 having nodes 112 usable to provide values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal alphabet X of the communications channel 310, wherein the encoder neural network 212, 222 is configured through training to be usable to map sequences of source symbols S.sup.m received from the information source 312, 322 directly to a representation as a channel input vector X.sup.n, usable to drive a transmitter 404 to transmit a corresponding signal over a communications channel 310.

[0201] The transmitter device 500 also comprises a transmitter 404 to transmit the channel input vector X.sup.n over the communications channel 310. The channel input vector X.sup.n may include signal values X.sup.i encoded by one or both encoder neural networks, such that the transmitter device 500 may transmit at the same time signal values from multiple different encoder neural networks to send over the communications channel multiple different information sources using joint source channel coding. The transmitter device 400 may also comprise a plurality of transmitters to transmit multiple channel input vectors X.sup.n for the plurality of encoder neural networks 212, 222 over the communications channel 310. The transmitter 404 may transmit the channel input vector X.sup.n of each encoder neural network simultaneously and/or at a different frequency over the communications channel.

[0202] The transmitter device 500 comprises a first encoder neural network 212 and a second encoder neural network 222. Although two encoder neural networks are illustrated, the transmitter device 500 may comprise any number of encoder neural networks. The first encoder neural network 212 and second encoder neural network 222 are examples of encoder neural network 202 of FIGS. 1 and 2. The first encoder neural network 212 is part of a joint source channel coding autoencoder. The second encoder neural network 212 is part of a joint source channel coding autoencoder. Each encoder neural network may be of a different joint source channel coding autoencoder. The joint source channel coding autoencoder comprising the first encoder neural network 212 and the joint source channel coding autoencoder comprising the second encoder neural network 222 are examples of joint source channel coding autoencoder 200 of FIG. 2. Each encoder neural network may have a different number of nodes at the channel input layer 102. In an example, the channel input layer 110 of one or more encoder neural networks 212, 222 of the transmitter device has nodes 112 corresponding to the channel input vector X.sup.n and the sequences of source symbols S.sup.m are mapped directly to a representation as the channel input vector X.sup.n.

[0203] Information source 312 is input into input layer 102 of the first encoder neural network 212. Information source 322 is input into input layer 102 of the second encoder neural network 222. Information source 312 and information source 322 are examples of information source 302 of FIG. 3. Information source 312 is not the same as information source 322. For example, information source 312 may have a different structure to information source 322. In an example, the information source 312 received at the input to the first encoder neural network 212 may be of a different size to information source 322 received at the input to the second encoder neural network 222. In another example, information source 312 may be of a different type to information source 322. For example, information source 312 may be an image and information source 322 may be a video.

[0204] The joint source channel coding autoencoder 200 comprising the first encoder neural network 212 and the joint source channel coding autoencoder 200 comprising the second encoder neural network 222 are separate joint source channel coding autoencoders 200. In an example, the joint source channel coding autoencoder 200 comprising the first encoder neural network 212 and the joint source channel coding autoencoder 200 comprising the second encoder neural network 222 are different. As described above, the information source or type of information source input into the first encoder neural network is different to the information source or type of information source input into the second encoder neural network. The joint source channel coding autoencoder comprising the first encoder neural network 212 is trained for information source 312 and the joint source channel coding autoencoder comprising the second encoder neural network 222 is trained for information source 322. Thus, the joint source channel coding autoencoder 200 comprising the first encoder neural network 212 and the joint source channel coding autoencoder 200 comprising the second encoder neural network 222 are trained for different information sources. This means the neural network of the first encoder neural network 212 may differ to the neural network of the second encoder neural network 222. For example, the joint source channel coding autoencoder 200 comprising the first encoder neural network 212 and the joint source channel coding autoencoder 200 comprising the second encoder neural network 222 may have different amounts of neural network layers, different amounts of nodes per layer, different weights and/or biases associated with the nodes etc. Moreover, the first encoder neural network 212 and the second encoder neural network 222 may have different amounts of neural network layers, different amounts of nodes per layer, different weights and/or biases associated with the nodes etc.

[0205] FIG. 6 provides an illustration of a receiver device 600 according to an example. The receiver device 600 may comprise a first decoder neural network 214 and a second decoder neural network 224. Although two decoder neural networks are illustrated, the receiver device 600 may comprise any number of decoder neural networks that correspond to one of the plurality of encoder neural networks of the transmitter device. The receiver device 600 may be in communication with the transmitter device 500 of FIG. 5. The decoder neural networks 214, 224 of the receiver device may correspond to the encoder neural networks 212, 222 of the transmitter device. For example, a joint source channel coding autoencoder 200 may comprise first encoder neural network 212 and first decoder neural network 214 and a separate joint source channel coding autoencoder 200 may comprise second encoder neural network 222 and second decoder neural network 224. Thus, the communication channel between each encoder neural networks and corresponding decoder neural network may have the same characteristics and distribution. The number of decoder neural networks may equate to the number of encoder neural networks of the transmitter device 500.

[0206] In an alternative embodiment, first decoder neural network 214 and second decoder neural network 224 may be located in separate receiver devices. Thus, the communication channel of the first encoder neural network and the second encoder neural network may have different characteristics and distributions. Each receiver device may only have one decoder neural network. Transmitter device 500 may be in communication with both receiver devices. Each receiver device may comprise at least one decoder neural network corresponding to one of the plurality of encoder neural networks 212, 222 of the transmitter device 500.

[0207] Each decoder neural network 214, 224 of receiver device 200 may have a channel output layer having nodes corresponding to a channel output vector Y.sup.n received from a receiver receiving a signal corresponding to at least the plurality of signal values X.sup.p of the channel input vector X.sup.n transmitted by the transmitter and transformed by the communications channel, and an output layer coupled to the channel output layer through one or more neural network layers, having nodes matching those of the input layer of the encoder neural network, wherein the first decoder neural network is configured through training to map the representation of the source symbols as the channel output vector Y.sup.n transformed by the communications channel to a reconstruction of the source symbols Ŝ.sup.m output from the output layer of the joint source channel coding autoencoder, the reconstruction of the source symbols Ŝ.sup.m being usable to reconstitute the information source.

[0208] The receiver device 600 also comprises a receiver 602 to receive a signal corresponding to the channel input vector X.sup.n transmitted by the transmitter of the transmitter device over the communications channel.

[0209] As encoder neural network is for a different information source or type of information source, the transmitter comprising the encoder neural networks is highly efficient for the specific information sources and is flexible in that it is capable of receiving a plurality of different information sources or types of information source

Multiple Decoder Neural Networks of a Joint Source Channel Coding Autoencoder for Adaptive Bandwidth Expansion

[0210] FIG. 7 provides another example of a joint source channel coding autoencoder. Joint source channel coding autoencoder 700 is an example of joint source channel coding autoencoder 200 of FIG. 2. As described in relation to FIG. 2, the neural network of FIG. 1 may also be a joint source channel coding autoencoder 700. Joint source channel coding autoencoder 700 comprises an encoder neural network 702, a first decoder neural network 704 and a second decoder neural network 706. Although two decoder neural networks are illustrated, the joint source channel coding autoencoder 700 may comprise any number of decoder neural networks 704, 706. Decoder neural network 704 and decoder neural network 706 of joint source channel coding autoencoder 700 are examples of decoder neural network 204.

[0211] FIG. 8 provides an illustration of a communication system 800 according to another example. The communication system 800 is for conveying information from an information source across a communications channel using a joint source channel coding autoencoder such as joint source channel coding autoencoder 700 of FIG. 7. The communication system 800 comprises an encoder neural network 702, a first decoder neural network 704 and a second decoder neural network 706 of joint source channel coding autoencoder 700. Communication system 800 may comprise any number of encoder neural networks and decoder neural networks. For example, communication system 800 may comprise a plurality of decoder neural networks and may comprise one or more receiver devices comprising one or more of the plurality of decoder neural networks 704, 706. Encoder neural network 702 is an example of encoder neural network 202 of FIGS. 1 and 2 and decoder neural networks 704, 706 are examples of decoder neural network 204 of FIGS. 1 and 2. The first decoder neural network 704 and the second decoder neural network 706 are of the same joint source channel coding autoencoder 700 as the encoder neural network 702.

[0212] The encoder neural network 702 of the communication system 800 has an input layer 102 having input nodes corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sup.m}, the S.sub.i taking values in an alphabet S, received at the input layer from the information source as samples thereof. The encoder neural network 702 also has a channel input layer 110 coupled to the input layer through one or more neural network layers, the channel input layer having nodes usable to provide values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal alphabet X of the communications channel, the channel input vector X.sup.n comprising a plurality of signal values X.sup.p usable to reconstruct an information source, wherein the number of the plurality of signal values X.sup.p is smaller than the total number of signal values X.sup.n, and wherein at least one of the remaining signal values of the channel input vector X.sup.n is usable to increase the quality of the reconstructed information source. The encoder neural network 702 is configured through training to be usable to map sequences of source symbols S.sup.m received from the information source directly to a representation as a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over the communications channel. In an example, the sequences of source symbols S.sup.m received from the information source are mapped directly to a representation as a channel input vector X.sup.n.

[0213] The communication system 800 further comprises a first decoder neural network 704 and a second decoder neural network 706 of the joint source channel coding autoencoder 700. Each decoder neural network 704, 706 has a channel output layer 114 having nodes 116 corresponding to a channel output vector Y received from a receiver receiving a signal corresponding to at least the plurality of signal values X.sup.p of the channel input vector X.sup.n transmitted by the transmitter and transformed by the communications channel 310, and an output layer 122 coupled to the channel output layer 114 through one or more neural network layers 118, having nodes matching those of the input layer 102 of the encoder neural network 702, wherein the decoder neural network is configured through training to map the representation of the source symbols as the channel output vector Y transformed by the communications channel 310 to a reconstruction of the source symbols Ŝ.sup.m output from the output layer 122 of the joint source channel coding autoencoder 700, the reconstruction of the source symbols Ŝ.sup.m being usable to reconstitute the information source 302. The number of signal values of the channel output vector Y received by the first decoder network 704 is more than the number of signal values of the channel output vector Y received by the second decoder neural network 706.

[0214] The plurality of signal values X.sup.p may be a subset of the channel input vector X.sup.n. The channel output vector Y.sup.p received from the receiver may correspond to a subset of the channel input vector X.sup.n, the subset transmitted by the transmitter and transformed by the communications channel. In an example, the sequences of source symbols S.sup.m received from the information source 302 are mapped directly to a representation as a channel input vector X.sup.n. In an example, the reconstruction of the source symbols Ŝ.sup.m of the first decoder neural network 702 is of a higher quality than the reconstruction of the source symbols Ŝ.sup.m of the second decoder neural network 704. The plurality of signal values X.sup.p usable to reconstruct an information source may be a plurality of consecutive signal values of the channel input vector X.sup.n and may be a first plurality of consecutive signal values of the channel input vector X.sup.n. The channel output layer of the first decoder neural network 702 may receive channel output vector Y.sup.p from a receiver receiving a signal corresponding to the plurality of signal values X.sup.p of the channel input vector X.sup.n. The channel output layer of the second decoder neural network may receive a channel output vector Y.sup.n from a receiver receiving a signal corresponding to the channel input vector X.sup.n.

[0215] In an example, the number of symbols of the reconstruction of the source symbols Ŝ.sup.m of the first decoder neural network is different to the number of symbols of the reconstruction of the source symbols Ŝ.sup.m of the second decoder neural network. The number p of the plurality of signal values X.sup.p may be based on the characteristics of the communications channel and/or capabilities of the receiver comprising the second decoder neural network. The number of p of the plurality of signal values X.sup.p may be such that the information source can be reconstructed at the second decoder neural network. The number n of the channel input vector X.sup.n may be based on the characteristics of the communications channel and/or capabilities of the receiver comprising the first decoder neural network. In an example, the first decoder neural network 704 may receive all signal values transmitted by the encoder neural network 702.

[0216] The encoder neural network 702 of the communication system 800 may be included in a transmitter device such as the transmitter device of FIG. 4. The transmitter device is for conveying information from an information source across a communications channel using a joint source channel coding autoencoder 700. The first decoder neural network 704 and/or second decoder neural network 706 of FIG. 7 may be included in a receiver device such as the receiver device of FIG. 6. In an example, first decoder neural network 704 and second decoder neural network 706 may be located in the same receiver device, which is in communication with transmitter device. The receiver 602 of the receiver device may receive channel input vector X.sup.n, which may be input into the first decoder neural network 704 and the second decoder neural network may receive some of the signal values of the channel input vector X.sup.n such as the plurality of signal values X.sup.p. The first decoder neural network 704 may therefore produce a higher quality reconstruction of the information source 302 than the second neural network 706 but the second decoder neural network 706 may be able to produce a reconstruction of the information source quicker than the first decoder neural network 704 due to having to receive and process fewer signal values.

[0217] Further, where only the reduced number of signal values X.sup.p are transmitted/received over the communications channel (for example, when insufficient channel capacity/radio resources are allocated to allow the transmission of the entirety of the channel input vector X.sup.n), the second decoder neural network can still be used to reconstruct the information source, albeit with a lower quality. When the transmitter/receiver are able to transmit/receive an increased number of signal values such that the entirety of the channel input vector X.sup.n can be transmitted/received (for example, when increased channel capacity/radio resources are allocated), then the first decoder neural network can be used to reconstruct a higher quality version of the information source.

[0218] Alternatively, first decoder neural network 704 and second decoder neural network 706 may be located in separate receiver devices and the transmitter device may be in communication with both receiver devices. The receiver device comprising the first decoder neural network 704 may therefore produce a higher quality reconstruction of the information source 302 than the receiver device comprising the second neural network 706. However, the receiver device comprising the second decoder neural network 706 may be able to produce a reconstruction of the information source quicker than the receiver device comprising the first decoder neural network 704 due to having to receive and process fewer signal values.

[0219] Depending on the locations of the separate receiver devices, the communication channel between the encoder neural network and the first decoder neural network and the communication channel between the encoder neural network and the second decoder neural network may have different characteristics and distributions. An advantage of the communication system is that the joint source channel coding autoencoder can be trained for one or more specific communication channels, even if they are different. The training method will be described later with reference to FIG. 10.

[0220] In an example, the second decoder neural network 706 may be located in a receiver device of a lower quality than the receiver device comprising the first decoder neural network 704. Hence, the second decoder neural network 706 may receive fewer signal values of each channel input vector X.sup.n than the first decoder neural network 704.

[0221] In an example, each of the remaining signal values of the channel input vector X.sup.n, when received in addition to the plurality of signal values X.sup.p, are usable to produce a reconstruction of the source symbols Ŝ.sup.m with a higher quality than the reconstruction of the source symbols Ŝ.sup.m using the plurality of signal values X.sup.p. For example, a receiver may receive only the plurality of signal values X.sup.p of the channel input vector X.sup.n and the decoder neural network of the receiver reconstructs the information source. If the receiver also receives one remaining signal value, then the decoder neural network of the receiver can produce a reconstruction of the information source at a higher quality. If the receiver receives another remaining signal value, then the decoder neural network of the receiver can produce a reconstruction of the information source at a higher quality.

[0222] In another example, a plurality of the remaining signal values of the channel input vector X.sup.n, when received in addition to the plurality of signal values X.sup.p, together are usable to produce a reconstruction of the source symbols Ŝ.sup.m with a higher quality than the reconstruction of the source symbols Ŝ.sup.m using the plurality of signal values X.sup.p.

[0223] All of the remaining signal values of the channel input vector X.sup.n may be usable to increase the quality of the reconstructed information source. The first decoder neural network, the second decoder neural network and the encoder neural network may be trained jointly. The training method will be described later with reference to FIG. 10.

[0224] The encoder neural network 702 may be trained to produce a number of signal values to be transmitted to enable reconstruction and to produce additional signal values to be transmitted to enable reconstruction of a higher quality. In the example communication system 800 of FIG. 8, the total number of signal values transmitted at one time from the encoder neural network 802 may be four. Two signal values, Y.sub.1,2, may be transmitted (for example when channel capacity/resource allocation only enables two signal values to be transmitted) to enable reconstruction of the information source to a reasonable level. The remaining signal values Y.sub.3,4, when combined with the two signal values Y.sub.1,2, can improve the quality of the reconstruction of the information source. Thus, if a receiver comprising a decoder neural network, such as the second decoder neural network 224, was to receive the two signal values Y.sub.1,2, the decoder neural network 224 could reconstruct the information source. If a receiver comprising a decoder neural network, such as the first decoder neural network 704, was to receive the two signal values Y.sub.1,2 and both of the remaining signal values Y.sub.3,4, receiving a total of four signal values, the first decoder neural network 704 could reconstruct the information source and the reconstruction would be of a higher quality than the reconstruction at both of the other receivers. This is an example of the communications system 800 but any number of signal values can be transmitted by the transmitter having the encoder neural network 202 and received from a receiver or receivers having the first decoder neural network 214 and/or second decoder neural network 224.

[0225] The quality of the reconstruction is determined by the similarity between the information source and the reconstruction of the information source. For example, if the reconstruction closely matches the information source, the reconstruction is of a high quality. Alternatively, if the reconstruction does not closely match the information source, the reconstruction is of a low quality. The quality of the reconstruction of the information source may be measured by the number of errors of the reconstruction or the number of symbols of the reconstruction. The measured number of errors may be a percentage of the total number of symbols of the reconstruction of the information source. The quality of the reconstruction of the information source may depend on the size of the information source and the size of the reconstruction of the information source.

[0226] FIG. 9 provides an example flowchart of a method 900 of an encoder neural network of a joint source channel coding autoencoder. The encoder neural network performing the method 900 may be encoder neural network 702 of FIGS. 7 and 8. The method 900 is for conveying information from an information source across a communications channel 310. Firstly, samples of an information source corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in a finite alphabet S, are received 902 at input nodes of an input layer. Secondly, the sequence of source symbols S.sup.m from the input layer are mapped 904 through one or more neural network layers to a channel input layer, the channel input layer having nodes usable to provide values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal values of the communications channel 310. The channel input vector X.sup.n comprises a plurality of signal values X.sup.p usable to reconstruct an information source. The number p of the plurality of signal values X.sup.p is smaller than the total number n of the signal values in the channel input vector X.sup.n, and at least one of the remaining signal values

[0227] of the channel input vector X.sup.n is usable to increase the quality of the reconstructed information source. In this method 900, the encoder neural network 702 is configured through training to be usable to map sequences of source symbols S.sup.m received from the information source directly to a representation as a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over a communications channel 310.

[0228] FIG. 10 is an example flowchart of a training method 1000 of a communication system 800. The training method 1000 is for training a joint source channel coding autoencoder 700 to convey information from an information source 302 across a communications channel 310. The training method 1000 is of a communication system 800 comprising an encoder neural network 702, a first decoder neural network 704 and a second decoder neural network 706, wherein the number of signal values of the channel output vector Y received by the first decoder network 704 is more than the number of signal values of the channel output vector Y received by the second decoder neural network 706. Firstly the encoder neural network 702 of the joint source channel coding autoencoder 700 maps 1002 sequences of source symbols S.sup.m received from the information source to a representation as a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over the communications channel 310. Next, the first decoder network maps 1004 the representation of the source symbols as the channel output vector Y transformed by the communications channel to a reconstruction of the source symbols Ŝ.sup.m output from the output layer of the joint source channel coding autoencoder 700. Following the mapping by the first decoder neural network, the sequences of source symbols S.sup.m are compared 1006 to the reconstruction of source symbols Ŝ.sup.m at the output layer of the first decoder neural network. A loss function is then formulated 1008 for the first decoder neural network 704 based on the comparison. The second decoder neural network 706 performs the same steps 1010, 1012, 1014 as the first decoder neural network 704. Once the loss functions of the first decoder neural network and second decoder network have been formulated, after both steps 1008 and 1014, parameters of the joint source channel coding autoencoder are amended 1016 based on the loss function for the first decoder neural network and the loss function for the second decoder neural network. It is then decided 1018 whether the reconstruction of source symbols Ŝ.sup.m at the first decoder neural network is usable to reconstitute the information source and the reconstruction of source symbols Ŝ.sup.m at the second decoder neural network is usable to reconstitute the information source. If both reconstructions are usable to reconstitute the information source, then the training ends 1020. If not, the steps 1002 to 1016 are repeated.

[0229] Steps 1004, 1006 and 1008 performed by the first decoder neural network 704 are independent of steps 1010, 1012 and 1014 performed by the second decoder neural network 706 and these sets of steps can therefore be performed independent of each other. For example, steps 1010, 1013 and 1014 can be performed in parallel, before or after steps 1004, 1006 and 1008. However, both steps 1004 and 1010 are performed after the mapping 1002 by the encoder neural network.

[0230] If the communication system 800 includes more than two decoder neural networks, steps 1004 to 1008 may be repeated for all of the decoder neural networks and the amending 1016 of the parameters will be based on the loss functions of all the decoder neural networks of the communication system 800. The steps 1002 to 1016 may be repeated until the reconstruction of source symbols Ŝ.sup.m at the second decoder neural network is usable to reconstitute the information source and the reconstruction of source symbols Ŝ.sup.m at the first decoder neural network is of a higher quality than the reconstruction of source symbols Ŝ.sup.m at the second decoder neural network.

[0231] It is to be noted that the parameters of the joint source channel coding autoencoder 200 are only amended if the comparison shows that the reconstructions of source symbols Ŝ.sup.m of the decoder neural networks are not usable to reconstitute the information source and/or are not at the required quality. In an example, sequences of source symbols S.sup.m received from the information source are mapped directly to a representation as a channel input vector X.sup.n.

[0232] The parameters to be amended 1016 based on the comparison may be the weights and/or biases of the nodes of the neural network layers. The parameters may be amended 1016 based on the channel characteristics, for example signal to noise ratio. The communication system is flexible and so the encoder neural network could then also be used with differing channels with the same SNR. The channel characteristics may be found by using channel sounding, i.e. sending known pulses across the channel. The parameters may also be amended 1016 based on the information source, for example, based on the data structure of the information source. Through training, the neural network may increase in dimension to more than two dimensions.

[0233] As mentioned previously, the first decoder neural network 704 and the second decoder neural network 706 may be located in different receivers and so the communication channel between the encoder neural network 702 and the first decoder neural network 704 and the communication channel between the encoder neural network 702 and the second decoder neural network 706 may have different characteristics and distributions. During training, the parameters of the joint source channel coding autoencoder are amended to ensure that the reconstruction of source symbols Ŝ.sup.m of both the decoder neural networks is usable to reconstitute the information source. Thus, the characteristics of the communication channel between the encoder neural network 702 and the first decoder neural network 704 and the characteristics of the communication channel between the encoder neural network 702 and the second decoder neural network 706 are learnt through training such that, for each decoder neural network, the channel output vector Y.sup.n transformed by the communication channel can still be mapped to a reconstruction of source symbols Ŝ.sup.m that is usable to reconstitute the information source.

[0234] As illustrated in FIG. 10, both decoder neural networks 704, 706 are trained jointly together with the encoder neural network 702. The decoder neural networks 704, 706 receive different amounts of signal values and the reconstruction of source symbols Ŝ.sup.m may therefore differ. For example, as the first decoder neural network receives a larger number of signal values from the encoder neural network, the reconstruction of source symbols Ŝ.sup.m at the first decoder neural network may be of a higher quality. The loss function indicates the quality of the reconstruction of source symbols Ŝ.sup.m at each decoder neural network 704, 706. For example, the loss function of each decoder neural network is formulated based on the comparison of the sequences of source symbols S.sup.m and the reconstruction of source symbols Ŝ.sup.m at the output of the decoder neural network. Once the loss functions have been calculated, they may be weighted based on the desired quality of reconstruction at each receiver. For example, the parameters may be amended 1016 based on a loss function of the joint source channel coding autoencoder. This loss function may be a weighted sum of the loss functions for each decoder neural network. The weighting of the loss functions may be amended to balance the reconstruction qualities of the first decoder neural network and the second decoder neural network. For example, weighting of each of the loss functions is based on the quality of the communications channel between the encoder neural network and the first decoder neural network and the quality of the communications channel between the encoder neural network and the second decoder neural network. In an example, the weighting of each of the loss functions is based on the desired reconstruction quality of the first decoder neural network and the second decoder neural network.

[0235] The quality of the reconstruction of the second decoder neural network 706 may be higher as it has received more information. However, the weighting factor will determine how much emphasis is put on the quality of each layer. For example, putting all the weight on one of the decoders will ignore the other, and hence, the ignored network will not be trained, whereas in general, the autoencoder will learn to balance the reconstruction qualities of the two decoders.

[0236] In summary, there is provided a transmitter device 500 for conveying information from an information source 302 across a communications channel 310 using a joint source channel coding autoencoder 200, comprising: a plurality of encoder neural networks 202, each encoder neural network 202 of a joint source channel coding autoencoder 200, and each encoder neural network 202 trained to receive a different information source 302 or a different type of information source 302, wherein each encoder neural network 302 comprises: an input layer 102 having input nodes 104 corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in a finite alphabet S, received at the input layer from an information source 302 as samples thereof, and a channel input layer 110 coupled to the input layer 102 through one or more neural network layers 106, the channel input layer 110 having nodes 112 usable to provide values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal alphabet X of the communications channel 310, wherein the encoder neural network 202 is configured through training to be usable to map sequences of source symbols S.sup.m received from the information source 302 directly to a representation as a channel input vector X.sup.n, usable to drive a transmitter 404 to transmit a corresponding signal over a communications channel 310; and a transmitter 404 to transmit the channel input vector X.sup.n over the communications channel 310.

[0237] Moreover, there is provided a communication system 800 for conveying information from an information source 302 across a communications channel 310 using a joint source channel coding autoencoder 700, comprising: an encoder neural network 702 of the joint source channel coding autoencoder 700, the encoder neural network 702 having: an input layer 102 having input nodes 104 corresponding to a sequence of source symbols S.sup.m={S.sub.i, S.sub.2, . . . , S.sup.m}, the S.sub.i taking values in an alphabet S, received at the input layer 102 from the information source 302 as samples thereof, and a channel input layer 110 coupled to the input layer 102 through one or more neural network layers 106, the channel input layer 110 having nodes 112 usable to provide values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal alphabet X of the communications channel 310, the channel input vector X.sup.n comprising a plurality of signal values X.sup.p usable to reconstruct an information source 302, wherein the number p of the plurality of signal values X.sup.p is smaller than the total number n of signal values of the channel input vector X.sup.n, and wherein at least one of the remaining symbols of the channel input vector X.sup.n is usable to increase the quality of the reconstructed information source. The encoder neural network 702 is configured through training to be usable to map sequences of source symbols S.sup.m received from the information source 302 directly to a representation as a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over the communications channel 310; a first decoder neural network 704 and a second decoder neural network 706 of the joint source channel coding autoencoder 700, each decoder neural network having: a channel output layer 114 having nodes corresponding to a channel output vector Y received from a receiver receiving a signal corresponding to at least the plurality of signal values X.sup.p of the channel input vector X.sup.n transmitted by the transmitter and transformed by the communications channel 310, and an output layer 122 coupled to the channel output layer through one or more neural network layers 118, having nodes matching those of the input layer 102 of the encoder neural network 702, wherein the decoder neural network is configured through training to map the representation of the source symbols as the channel output vector Y transformed by the communications channel 310 to a reconstruction of the source symbols Ŝ.sup.m output from the output layer 122 of the joint source channel coding autoencoder 700, the reconstruction of the source symbols Ŝ.sup.m being usable to reconstitute the information source 302; and wherein the number of signal values of the channel output vector Y received by the first decoder network 704 is more than the number of signal values of the channel output vector Y received by the second decoder neural network 706.

[0238] The plurality of decoder neural networks of the joint source channel coding autoencoder connected to the encoder neural network of the joint source channel coding autoencoder enable a flexible system whereby the encoder neural network can transmit a channel input vector X.sup.n that enables both decoder neural networks to reconstruct the information source even if they receive a differing ‘amount’ of the signal corresponding to the channel input vector X.sup.n. The plurality of decoder neural networks in the joint source channel coding autoencoder enable multiple levels of reconstruction at one receiver or reconstruction at all receivers, if the decoder neural networks are located in separate receivers, even if the receivers have different capabilities. Also, the joint source channel coding can be responsive to changes in channel capacity/resource allocation on the communications channel, allowing sufficient symbols to be communicated to allow reconstruction of a relatively low quality version of the information source at times of relatively low channel capacity, and sufficient symbols to be communicated to allow reconstruction of a relatively high quality version of the information source at times of relatively high channel capacity.

Joint Source Channel Coding Variational Autoencoder

[0239] A variational autoencoder is a particular type of autoencoder. Thus, the joint source channel coding variational autoencoder is an example of the joint source channel coding autoencoders detailed above, such as joint source channel coding autoencoders 200 of FIG. 2 and 700 of FIG. 7. Joint source channel coding variational autoencoders have the same advantages as joint source channel coding autoencoders. Moreover, the features of the joint source channel coding autoencoder, as previously explained, also apply to the joint source channel coding variational autoencoder. Thus, features of the joint source channel coding autoencoder may also be features of the joint source channel coding variational autoencoder and features of the joint source channel coding variational autoencoder may also be features of the joint source channel coding autoencoder.

[0240] In an autoencoder, the goal of the system is to make sure that each input can be regenerated based on the encoder output. The encoder therefore tries to learn the most essential features of each input data sample. But, for example, if the encoder output dimension is sufficiently large, each input data point could be mapped to a distinct output, and the decoder can simply learn the inverse mapping from encoder outputs to input values. In principle, the encoder can map two similar inputs to completely different output values. Therefore, in an autoencoder, if the encoder output differs from the decoder input (for example due to a noisy channel), the decoder neural network may not be able to reconstruct the information source. This is because the output layer of the encoder neural network comprises specific values. Even if the values are similar, these similar values may map to other specific values that may be very different. A variational autoencoder enables optimisation of the reconstruction of the information source. For a variational autoencoder, the goal is to learn the stochastic model that generates the underlying source signals. Therefore, in a variational autoencoder, the outputs of the encoder neural network are parameters of distributions, the distributions being based on certain features of the input data and being randomly sampleable to provide output values. The sampler may be considered to form part of the variational autoencoder.

[0241] For example, the distributions may be based on statistical features of a structured data set. In another example, the distributions may be based on features of an image, such as a face, or a smile. In another example, the distributions may be based on abstract structures that generate the image. If the (sampled) output of the encoder neural network differs from the input to the decoder neural network (for example due to a noisy channel), the variational autoencoder can still produce a reconstruction of the information source, which decreases in quality as the difference between encoder neural network output a decoder neural network input increases. The variational autoencoder will be explained further in the following figures.

[0242] The variational autoencoder in accordance with examples of the present disclosure can be used in a communication system for conveying information from an information source across a communications channel. The encoder neural network of the variational autoencoder can replace the traditional source encoder and channel encoder. The encoder neural network maps the signal directly to the channel input layer (or to distributions therein defined by the encoder neural network), where it is sampled and transmitted over the communications channel. The decoder neural network of the variational autoencoder can replace the traditional source decoder and channel decoder. The decoder neural network receives the output of the communications channel and reconstructs the signal. The encoder neural network and decoder neural network are trained to undo the effect of the channel on the signal, such as the effect of the channel noise of the signal. The encoder neural network and decoder neural network can also be trained to learn a representation of the source signal and recover the input with the highest fidelity possible. Further, as the neural networks are trained to optimise the mapping, the efficiency of the transmission of the information source over the communications channel can be greater than in separate source and channel coding, where the separate removal and subsequent addition of redundancy may be inherently inefficient for transferring the data.

[0243] Thus, the encoder and decoder neural networks can be used as a form of joint source channel coding. This provides simplicity over having two separate systems for encoding and two separate systems for decoding. However, the limitations of the joint source channel coding techniques as mentioned above are removed. In particular, a variational autoencoder used for joint source channel coding, herein referred to as the joint source channel coding variational autoencoder, can be trained for use with different channels and different input sources. Thus, using neural networks for conveying information across a channel enables a high degree of freedom.

[0244] FIG. 11 provides an example of a joint source channel coding variational autoencoder 1100. The joint source channel coding variational autoencoder 1100 comprises an encoder neural network 1102 and a decoder neural network 1104. The encoder neural network and decoder neural network may be deep and/or recurrent.

[0245] For example, with reference to FIG. 1, the neural network of FIG. 1 may be a joint source channel coding variational autoencoder 1100. The encoder neural network 1102 of the joint source channel coding variational autoencoder 1100 may be formed from layers 102, 106 and 110. The decoder neural network 1104 of the joint source channel coding variational autoencoder 1100 may be formed from layers 114, 118 and 122.

[0246] With reference to FIGS. 1 and 11, the encoder neural network 1102 of the joint source channel coding variational autoencoder 1100 has an input layer 102 having input nodes 104 corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in a finite alphabet S, received at the input layer 102 from an information source as samples thereof. The encoder neural network 1102 also has a channel input layer 110 coupled to the input layer through one or more neural network layers 106, the channel input layer 110 having nodes 112 corresponding to a channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K}, the Z.sub.i taking values for parameters defining a plurality of distributions, each distribution being sampleable to provide possible values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal values of a communications channel.

[0247] The encoder neural network 1102 is configured through training to map sequences of source symbols S.sup.m received from the information source directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over the communications channel.

[0248] In an example, the channel input layer 110 is coupled to the input layer 102 through at least five neural network layers 106. The encoder neural network 1102 may directly map the input nodes 104 at the input layer 102, corresponding to a sequence of source symbols S.sup.m, to the nodes 112 of the channel input layer 110, corresponding to a channel input distribution vector Z.sup.k, through the neural network.

[0249] In the communication system, for a wireless communications channel in accordance with an example, each input signal X.sub.i of the channel input vector X.sup.n may belong to a set of complex numbers, corresponding to the I and Q components. In an example, the encoder neural network 1102 of the joint source channel coding variational autoencoder 1100 may perform bandwidth compression. For example, the encoder neural network 1102 may compress the information source during mapping such that the size n of the channel input vector X.sup.n is smaller than the size m of the source symbols S.sup.m Alternatively, the joint source channel coding variational autoencoder 1100 may perform a bandwidth expansion, where the size n of the channel input vector X.sup.n is larger than the size m of the source symbols S.sub.m.

[0250] In an example, the plurality of distributions have the same distribution type. For example, the plurality of distributions may be Gaussian distributions defined by a mean and a standard deviation. In an example, for k/2 Gaussian distributions, the first k/2 elements of the channel input distribution vector Z.sup.k encode the mean value of the corresponding channel input distributions, while the remaining k/2 elements of the channel input distribution vector Z.sup.k encode the standard deviations of the corresponding channel input distributions. In this example, the i-th channel input X.sub.i is a sample from a Gaussian distribution with mean Z.sub.i and standard deviation Z.sub.i+k. In an example, each distribution of the plurality of distributions is sampled once. Thus, for distributions defined by two distribution parameters, the size n of the channel input vector X.sup.n is half the size k of the channel input distribution vector Z.sup.k. Moreover, in such an example, the channel input vector X.sup.n comprises one sample from each distribution of the plurality of distributions.

[0251] The distribution type of the plurality of distributions may be the optimal input distribution for the channel. Thus, by understanding channel models, the optimal input distributions can be calculated. The encoder neural network 1102 may be trained as part of the training of the joint source channel coding variational autoencoder 1100 to learn the parameters of the plurality of distributions. In an example, the type of distribution of the plurality of distributions is based on the characteristics of the communication channel. For example, the type of distribution may be based on the distribution model of the channel noise or the fading. For example if the communications channel can be modelled as a Gaussian channel, then the plurality of distributions are Gaussian distributions.

[0252] An advantage of using a variational autoencoder for conveying information over a communications channel is that, for many common channel models such as binary symmetric channels and AWGN channels, we know the optimal input distribution. Moreover, when transmitting a Gaussian source distribution over a Gaussian channel, no coding is necessary, and simple uncoded transmission with power allocation achieves the optimal Shannon bound. Thus, if the channel is AWGN, by using the joint source channel coding variational autoencoder 1100, the channel can be used at its theoretical limit. This suggests that if we can represent the underlying data set through a Gaussian distribution, as is achievable in examples of the present disclosure, that would correspond to the optimal input distribution to the channel.

[0253] When the neural network of FIG. 1 is modelled as a joint source channel coding variational autoencoder, channel input layer 110 and channel output layer 114 are not directly connected, as indicated by the dashed arrows connecting the nodes of these layers. The communications channel is between channel input layer 110 and channel output layer 114. Thus, the values output from the (sampled distributions defined by the) encoder neural network may not be equal to the values input into the decoder neural network.

[0254] The decoder neural network 1104 is to map the channel output sequence to a reconstructed version of the channel input vector Ŝ.sup.m. The decoder neural network 1104 may be a standard decoder of an autoencoder. The mapping may be a deterministic mapping. Alternatively the mapping may be a probabilistic mapping.

[0255] The decoder neural network 1104 of the joint source channel coding variational autoencoder 1100 has a channel output layer 114 having nodes 116 corresponding to a channel output vector Y.sup.n received from a receiver receiving the signal X.sup.n transmitted by the transmitter and transformed by the communications channel. The decoder neural network 1104 also has an output layer 122 coupled to the channel output layer 114 through one or more neural network layers 118, having nodes 124 matching those of the input layer 104 of the encoder neural network.

[0256] The decoder neural network 1104 is configured through training to map the representation of the source symbols as the channel output vector Y.sup.n transformed by the communications channel to a reconstruction of the source symbols Ŝ.sup.m output from the output layer of the joint source channel coding variational autoencoder 1100, the reconstruction of the source symbols Ŝ.sup.m being usable to reconstitute the information source.

[0257] In an example, the channel output layer 114 is coupled to the output layer 122 through at least five neural network layers 118. The decoder neural network 1104 may directly map the nodes 116 at the channel output layer 114, corresponding to a channel output vector Y.sup.n, to the nodes 124 of the output layer 122, corresponding to a reconstruction of the source symbols Ŝ.sup.m, through the neural network.

[0258] The encoder neural network 1102 and decoder neural network 1104 may have any number of layers and each layer may comprise any number of nodes. In an example, the output layer of the encoder neural network 1102 has the same number of nodes as the input layer of the decoder neural network 1104. In another example, the input layer of the encoder neural network 1102 has the same number of nodes as the output layer of the decoder neural network 1104.

[0259] In an example, the size n of the channel input vector X.sup.n is based on the channel capacity or the channel resources allocated for use by the transmitter (for example, the radio resources, or OFDM symbols, allocated to a radio bearer for a transmitter terminal for digital (e.g. QAM) modulation thereby in an LTE communication system). Alternatively or additionally, the size n of the channel input vector X.sup.n may be based on the size m of the source symbols S.sup.m or the size m of the reconstruction of the source symbols Ŝ.sup.m. In an example, the size of the source symbols S.sup.m is not equal to the size of the reconstruction of the source symbols Ŝ.sup.m. The size k of the channel input distribution vector Z.sup.k may be larger than the size n of the channel input vector X.sup.n. The size m of the reconstruction of the source symbols Ŝ.sup.m may be larger than the size n of the channel output vector Y.sup.n. The size n of the channel input vector X.sup.n may be equal to the size n of the channel output vector Y.sup.n In another example, the size n of the channel input vector X.sup.n may be different from the size n of the channel output vector Y.sup.n, for example, for a multiple input multiple output channel.

[0260] FIG. 12 provides an illustration of a communication system 1200 according to an example. The communication system comprises an information source 302. The information source 302 has been described previously with reference to FIG. 3. The information source is input into input layer 102 of the encoder neural network 1102 of the joint source channel coding variational autoencoder 1100. The encoder neural network 1102 is described above with reference to FIGS. 1 and 11. The encoder neural network 1102 has a channel input layer 110 coupled to the input layer 102 through one or more neural network layers 106, the channel input layer 110 having nodes 112 corresponding to a channel input distribution vector Z.sup.K={Z.sub.1, Z.sub.2, . . . , Z.sub.K}, the Z.sub.i taking values for parameters defining a plurality of distributions.

[0261] The communication system 1200 further comprises a sampler 1206. The sampler 306 receives the channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K} and samples the respective distribution for each channel input X.sub.i defined by the channel input distribution vector to produce a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}. For example, the sampler may produce outputs X.sub.1 1214 and X.sub.2 1216. The sampler will be described in more detail below. The sample values may need to be normalized before being input into the communications channel to comply with average input power constraints.

[0262] Channel input vector X.sup.n may be transmitted across communications channel 310. Communications channel has been described previously with reference to FIG. 3. Due to the use of distributions, even if the input to the decoder neural network 1104 does not equal the output of the encoder neural network 1102, the joint source channel coding variational autoencoder 1100 may still be able to reconstruct the information source 302. This is because the channel input distribution vector defines a plurality of distributions, each distribution being sampleable to provide possible values for the X.sub.i of a channel input vector usable to drive a transmitter to transmit a corresponding signal over the communications channel 310. As described above, the plurality of distributions enable graceful degradation of the performance of the communications system. This means that the quality of reconstruction of the input source slowly degrades as the channel noise increases.

[0263] As the input to the decoder neural network becomes less similar to the output of the encoder neural network, the reconstruction of the information source reduces in similarity to the information source and consequently the quality of the reconstruction is reduced. In this example, as the noise of the communications channel 310 increases, the difference between the sequence of source symbols S.sup.m and the reconstruction of the source symbols Ŝ.sup.m may increase.

[0264] The decoder neural network 1104 receives the channel output vector. The decoder neural network 1104 is described above with reference to FIGS. 1 and 11. Thus the decoder neural network of the communication system produces a reconstruction of the source symbols Ŝ.sup.m usable to reconstitute the information source 302.

[0265] FIG. 13 provides an illustration of the sampler 1206 according to an example. Sampler 1206 may receive two inputs Z.sub.2 1324 and Z.sub.4 1326. These inputs are the parameters of a distribution such as distribution 1322. Sampler 1206 may sample the distribution defined by the parameters Z.sub.2 and Z.sub.4 to produce an output X.sub.2 1328. The output X.sub.2 1328 may be any value that falls within the distribution 1322 and is illustrated by the spot labelled X.sub.2 on the scale.

[0266] In an example, the distribution 1322 is a Gaussian distribution with a mean and a standard deviation. For example, the Gaussian distribution may have a standard deviation σ.sub.2=2 and a mean μ.sub.2=−3. Z.sub.2 1324 provides the standard deviation and Z.sub.4 1326 provides the mean to the sampler. The sampler samples the Gaussian distribution to produce the output X.sub.2 1328. For example, when sampling a Gaussian distribution with a standard deviation σ.sub.2=2 and a mean μ.sub.2=−3, the sampler may produce output X.sub.2=−7.

[0267] In another example, sampler 1206 may receive two inputs Z.sub.1 1304 and Z.sub.3 1306. These inputs are the parameters of a distribution such as distribution 1302. Sampler 1206 may sample the distribution defined by the parameters Z.sub.1 and Z.sub.3 to produce an output X.sub.1 1308, 1310. The output X.sub.1 1308, 1310 may be any value that falls within the distribution 1302 and is illustrated by the spots labelled output X.sub.1 on the scale.

[0268] In an example, the distribution 1302 is a Gaussian distribution with a mean and a standard deviation. For example, the Gaussian distribution 1302 may have a standard deviation σ.sub.1=1 and a mean μ.sub.1=2.5. Z.sub.1 1304 provides the standard deviation and Z.sub.3 1306 provides the mean to the sampler. The sampler randomly samples the distribution 1302. Therefore, the output of the sampler varies even if the same input distribution is used. It is acknowledged that the distribution and, consequently, the parameters of the distribution will not be constant but will vary depending on the information source. Nevertheless, for illustrational purposes, at a first time and a second time the sampler receives the same parameters. For example, at a first time, t=1, the sampler may receive parameters defining the distribution 1302 and samples the distribution 1302 to produce the output X.sub.1=5. At a second time, the sampler may receive parameters defining the distribution 1302 and produce the output X.sub.1=3. Thus, an input of the encoder neural network does not directly map to a specific output of the encoder neural network. Even if the values at the input layer remain constant, the output values will constantly change due to the sampling of the distribution. Thus, the input layer of the decoder neural network will receive differing values but will learn to map these back to the same information source in order to reconstruct the information source. Therefore, even if the output of the encoder neural network is distorted when transmitted through the channel, the decoder neural network will receive different values but will still be able to reconstruct the information source.

[0269] FIG. 14 provides an example flowchart of a method 1400 of an encoder neural network 1102 of a joint source channel coding variational autoencoder 1100. The method 1400 is for conveying information from an information source across a communications channel 310. Firstly, samples of an information source corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in a finite alphabet S, are received 1402 at input nodes of an input layer. Secondly, the sequence of source symbols S.sup.m from the input layer are mapped 1404 through one or more neural network layers to a channel input layer, the channel input layer having nodes corresponding to a channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K}, the Z.sub.i taking values for parameters defining a plurality of distributions, each distribution being sampleable to provide possible values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal values of the communications channel 310. In this method 1400, the encoder neural network 1102 is configured through training to map sequences of source symbols S.sup.m received from the information source directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over a communications channel 310.

[0270] The encoder neural network 1102 and decoder neural network 1104 of the joint source channel coding variational autoencoder 1100 may be trained jointly to obtain a neural network capable of sufficiently reconstructing the information source 302. For example, they may be trained jointly based on the characteristics of the communication channel, based on the type of information source, for a given information source and/or to meet channel input constraints. The encoder neural network 1102 and decoder neural network 1104 of the joint source channel coding variational autoencoder 1100 may be trained jointly using performance measures. One performance measure may be the loss function.

[0271] The encoder neural network 1102 and decoder neural network 1104 of the joint source channel coding variational autoencoder 1100 may be trained jointly to minimise the difference between the input and output of the joint source channel coding variational autoencoder 1100. The difference between the input and output of the joint source channel coding variational autoencoder 1100 may be another performance measure. The difference may be measured by observing the similarity between the information source and the reconstructed information source. Alternatively or additionally, the difference may be measured by finding the “distance” between the actual channel input distribution and the target distribution. The distance between the probability distributions can be measured using KL divergence. For target distribution q(X.sup.n), the KL divergence D.sub.KL is calculated using the following equation.

D.sub.KL[P.sub.ϕ(X.sup.n|S.sup.m)∥q(X.sup.n]

[0272] Where ϕ represents the encoder network parameters. For an AWGN channel, since a Gaussian input distribution is capacity-achieving, the target distribution may be set as a zero-mean Gaussian with covariance P.Math.I, where P is the average input power constraint. Thus, the encoder neural network 1102 and decoder neural network 1104 may be trained jointly based on the measure of distance from the target distribution to the plurality of distributions.

[0273] Joint training has been further explained with reference to the encoder neural network 202 and decoder neural network 204 of the joint source channel coding autoencoder 200 of FIG. 2 and the same training can be applied to the joint source channel coding variational autoencoder 1100 of FIG. 11.

[0274] FIG. 15 provides an example flowchart of a training method 1500 of a communication system 1200. The training method 1500 is for training a joint source channel coding variational autoencoder 1100 to convey information from an information source 302 across a communications channel 310. Firstly the encoder neural network 1102 of the joint source channel coding variational autoencoder 1100 maps 1502 sequences of source symbols S.sup.m received from the information source directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over the communications channel 310. Next, the decoder neural network 1104 of the joint source channel coding variational autoencoder 1100, maps 1504 the representation of the source symbols as the channel output vector Y.sup.n transformed by the communications channel to a reconstruction of the source symbols Ŝ.sup.m output from the output layer of the joint source channel coding variational autoencoder 1100. Following the mapping by the encoder neural network and decoder neural network, the sequences of source symbols S.sup.m are compared 1506 to the reconstruction of source symbols Ŝ.sup.m. The parameters of the joint source channel coding variational autoencoder 1100 are then amended 1508 based on the comparison. It is then decided 1510 whether the reconstruction of source symbols Ŝ.sup.m is usable to reconstitute the information source. If the reconstruction of source symbols Ŝ.sup.m is usable to reconstitute the information source, then the method ends 1512. If not, the method steps 1502 to 1508 are repeated until the reconstruction of source symbols Ŝ.sup.m is usable to reconstitute the information source. It is to be noted that the parameters of the joint source channel coding variational autoencoder 1100 are only amended if the comparison shows that the reconstruction of source symbols Ŝ.sup.m is not usable to reconstitute the information source.

[0275] The parameters to be amended 1508 based on the comparison may be the weights and/or biases of the nodes of the neural network layers. The parameters of the distributions may be trained based on the characteristics of the channel. For example, the channel capacity differs based on the signal to noise ratio (SNR) and so the encoder neural network can be trained based on the SNR. The communication system is flexible and so the encoder neural network can then also be used with differing channels with the same SNR. The channel characteristics may be found by using channel sounding, i.e. sending known pulses across the channel. The parameters of the distributions may be trained based on the information source, for example, based on the data structure of the information source. Through training, the neural network may increase in dimension to more than two dimensions.

[0276] FIG. 16 provides an example of a transmitter device 1600. The transmitter device 1600 comprises an encoder neural network 1102 of a joint source channel coding variational autoencoder 1100, a transmitter 1604 and a sampler 1606. The encoder neural network 1102 of the transmitter device 1600 comprises an input layer having input nodes corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in a finite alphabet S, received at the input layer from an information source as samples thereof, and a channel input layer coupled to the input layer through one or more neural network layers, the channel input layer having nodes corresponding to a channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K}, the Z.sub.i taking values for parameters defining a plurality of distributions, each distribution being sampleable to provide possible values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal values of the communications channel, wherein the encoder neural network is configured through training to map sequences of source symbols S.sup.m received from the information source directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over a communications channel.

[0277] The sampler 1606 of the transmitter device 1600 is configured to produce a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n} in use by sampling the respective distribution for each channel input X.sub.i defined by the channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K} output by the channel input layer of the encoder neural network. The sampler may be an example of the sampler 1206 of FIG. 12. The transmitter 1604 of the transmitter device 1600 transmits the channel input vector X.sup.n over the communications channel.

[0278] FIG. 17 is a schematic diagram showing a computer 1700 according to an example. Computer 1700 comprises a processor 1702 and a storage medium 1704. The storage medium 1704 is a computer-readable storage medium comprising instructions which, when executed by the computer 1700, cause the computer 1700 to carry out the steps of method 900 of FIG. 9 or method 1000 of FIG. 10 or method 1400 of FIG. 14 or method 1500 of FIG. 15. In an example, the computer-readable storage medium comprises instructions which, when executed by the computer 1700, cause the computer 1700 to carry out the steps of a plurality of the methods 900, 1000, 1400 and 1500.

Referring back to FIG. 5, in an example, the encoder neural networks of the transmitter device 500 may be encoder neural networks of joint source channel coding variational autoencoder 1100. The transmitter device 500 comprises a first encoder neural network 212 and a second encoder neural network 222. These encoder neural networks may be examples of encoder neural network 1102 of FIG. 11 and may be part of separate joint source channel coding variational autoencoders 1100. In an example, in the transmitter device 500, each encoder neural network may be of a joint source channel coding variational autoencoder 1100 and the nodes of the channel input layer of each encoder neural network may correspond to a channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K}, the Z.sub.i taking values for parameters defining a plurality of distributions, each distribution being sampleable to provide possible values for the X.sub.i of the channel input vector X.sup.n={X.sub.i, X.sub.2, . . . , X.sub.n}. Moreover, each encoder neural network may be configured through training to map sequences of source symbols S.sup.m received from the information source directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n. The transmitter device 500 may also comprise a sampler (not shown), for example sampler 1606 of FIG. 16, which may be configured to produce a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n} in use by sampling the respective distribution for each channel input X.sub.i defined by the channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K} output by the channel input layer of each encoder neural network. The sampler has been described above in relation to FIG. 13.

[0279] FIG. 18 provides another illustration of a communication system 1800 according to an example. The communication system 1800 of FIG. 18 is an example of the communication system 800 of FIG. 8. The communication system may comprise encoder neural network 1802, first decoder neural network 1804 and second decoder neural network 1806. Encoder neural network 1802, first decoder neural network 1804 and second decoder neural network 1806 may be of a joint source channel coding variational autoencoder, such as joint source channel coding variational encoder 1100 of FIG. 11. The nodes of the channel input layer of the encoder neural network 1802 may correspond to a channel input distribution vector Z.sub.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K}, the Z.sub.i taking values for parameters defining a plurality of distributions, each distribution being sampleable to provide possible values for the X.sub.i of the channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}. The encoder neural network 1802 may be configured through training to map sequences of source symbols S.sup.m received from the information source directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n.

[0280] The communication system may further comprise a sampler 1812, for example sampler 1206 of FIG. 12, configured to produce a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n} in use by sampling the respective distribution for each channel input X.sub.i defined by the channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K} output by the channel input layer of the encoder neural network. For example, the sampler may receive inputs Z.sub.1 to Z.sub.8, sample the inputs and produce outputs X.sub.1 1808, X.sub.21810, X.sub.3 1818, X.sub.4 1816. The sampler 1812 has been described above in relation to FIG. 13.

[0281] The encoder neural network 1802 of the communication system 1800 of FIG. 18 may be included in a transmitter device such as transmitter device 1600 of FIG. 16. The transmitter device may comprise a sampler, such as sampler 1606, configured to produce a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n} in use by sampling the respective distribution for each channel input X.sub.i defined by the channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K} output by the channel input layer of the encoder neural network 1802.

[0282] The method 900 of FIG. 9 and the training method 1000 of FIG. 10 may be performed for the communication system of FIG. 18. For example, for the method 900 of FIG. 9, the joint source channel coding autoencoder may be a joint source channel coding variational autoencoder, and the channel input layer may have nodes corresponding to a channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K}, the Z.sub.i taking values for parameters defining a plurality of distributions, each distribution being sampleable to provide possible values for the X.sub.i of a channel input vector X.sup.n. The encoder neural network may be configured through training to map sequences of source symbols S.sub.m received from the information source directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n.

[0283] In another example, for the training method 1000 of FIG. 10, the joint source channel coding autoencoder may be a joint source channel coding variational autoencoder, and the sequences of source symbols S.sup.m received from the information source may be mapped directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n.

[0284] In summary, there is provided a communication system 300 for conveying information from an information source 302 across a communications channel 310 using a joint source channel coding variational autoencoder 1100, comprising an encoder neural network 1102 of the joint source channel coding variational autoencoder 1100, the encoder neural network 1102 having an input layer 102 having input nodes 104 corresponding to a sequence of source symbols S.sup.m={S.sub.1, S.sub.2, . . . , S.sub.m}, the S.sub.i taking values in a finite alphabet S, received at the input layer 102 from the information source 302 as samples thereof, and a channel input layer 110 coupled to the input layer 102 through one or more neural network layers 106, the channel input layer 110 having nodes 112 corresponding to a channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K}, the Z.sub.i taking values for parameters defining a plurality of distributions, each distribution being sampleable to provide possible values for the X.sub.i of a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n}, the X.sub.i taking values from the available input signal values of the communications channel 310, wherein the encoder neural network 1102 is configured through training to map sequences of source symbols S.sup.m received from the information source 302 directly to a representation as a plurality of distributions that provide possible values for the X.sub.i of a channel input vector X.sup.n, usable to drive a transmitter to transmit a corresponding signal over the communications channel 310; a sampler 306, configured to produce a channel input vector X.sup.n={X.sub.1, X.sub.2, . . . , X.sub.n} in use by sampling the respective distribution for each channel input X.sub.i defined by the channel input distribution vector Z.sup.k={Z.sub.1, Z.sub.2, . . . , Z.sub.K} output by the channel input layer of the encoder neural network 1102; and a decoder neural network 1104 of the joint source channel coding variational autoencoder 1100, the decoder neural network 1104 having a channel output layer 114 having nodes 116 corresponding to a channel output vector Y.sup.n received from a receiver receiving the signal X.sup.n transmitted by the transmitter and transformed by the communications channel, and an output layer 122 coupled to the channel output layer 114 through one or more neural network layers 118, having nodes 124 matching those of the input layer 102 of the encoder neural network 1102, wherein the decoder neural network 1104 is configured through training to map the representation of the source symbols as the channel output vector Y.sup.n transformed by the communications channel 310 to a reconstruction of the source symbols Ŝ.sup.m output from the output layer 122 of the joint source channel coding variational autoencoder 1100, the reconstruction of the source symbols Ŝ.sup.m being usable to reconstitute the information source 302.

[0285] Throughout the description and claims of this specification, the words “comprise” and “contain” and variations of them mean “including but not limited to”, and they are not intended to (and do not) exclude other components, integers or steps. Throughout the description and claims of this specification, the singular encompasses the plural unless the context otherwise requires. In particular, where the indefinite article is used, the specification is to be understood as contemplating plurality as well as singularity, unless the context requires otherwise.

[0286] Features, integers, characteristics or groups described in conjunction with a particular aspect, embodiment or example of the invention are to be understood to be applicable to any other aspect, embodiment or example described herein unless incompatible therewith. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive. The invention is not restricted to the details of any foregoing embodiments. The invention extends to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed. In particular, any dependent claims may be combined with any of the independent claims and any of the other dependent claims.

JOINT SOURCE CHANNEL CODING FOR NOISY CHANNELS USING NEURAL NETWORKS

Assignee

Inventors

Cpc classification

Classification Explorer

G06N3/088

PHYSICS

Classification Explorer

H04N19/94

ELECTRICITY

Classification Explorer

G06N3/047

PHYSICS

Classification Explorer

H04N19/30

ELECTRICITY

Classification Explorer

G06N3/045

PHYSICS

Classification Explorer

H03M13/6312

ELECTRICITY

International classification

Classification Explorer

G06N3/04

PHYSICS

Classification Explorer

G06N3/08

PHYSICS

Classification Explorer

H03M13/00

ELECTRICITY

Classification Explorer

H04N19/94

ELECTRICITY

Abstract

Claims

Description