COMMUNICATION SYSTEM, RECEIVER, EQUALIZATION SIGNAL PROCESSING CIRCUIT, METHOD, AND NON-TRANSITORY COMPUTER READABLE MEDIUM

Abstract

A detector coherent-receives a signal being transmitted from a transmitter. A filter group includes a plurality of filters connected in series along a signal path of a reception signal. The plurality of filters include a plurality of non-linear distortion compensation filters and one or more linear distortion compensation filters. A coefficient updating unit controls a filter coefficient of the plurality of non-linear distortion compensation filters and a filter coefficient of at least some of the linear distortion compensation filters. The coefficient updating unit adaptively controls the filter coefficient, by using an error back propagation method, based on a difference between an output signal being output from the filter group and a predetermined value of the output signal.

Claims

1. An equalization signal processing circuit comprising: a filter group configured to include a plurality of filters connected in series along a signal path of a reception signal being coherent-received, the plurality of filters including a plurality of non-linear distortion compensation filters configured to compensate for non-linear distortion included in the reception signal and one or more linear distortion compensation filters configured to compensate for linear distortion included in the reception signal; at least one memory storing instructions; and at least one processor configured to execute the instructions to: adaptively control, by using an error back propagation method, a filter coefficient of at least some of a plurality of non-linear distortion compensation filters and the linear distortion compensation filter, based on a difference between an output signal being output from the filter group and a predetermined value of the output signal.

2. The equalization signal processing circuit according to claim 1, wherein the plurality of non-linear distortion compensation filters include an intra-transmitter non-linear distortion compensation filter configured to compensate for non-linear distortion occurring in a transmitter, and an intra-receiver non-linear distortion compensation filter configured to compensate for non-linear distortion occurring in a receiver.

3. The equalization signal processing circuit according to claim 2, wherein, in the filter group, the intra-receiver non-linear distortion compensation filter is arranged before the intra-transmitter non-linear distortion compensation filter.

4. The equalization signal processing circuit according to claim 3, wherein the one or more linear distortion compensation filters include an intra-transmitter linear distortion compensation filter configured to compensate for linear distortion occurring in the transmitter, and an intra-receiver linear distortion compensation filter configured to compensate for linear distortion occurring in the receiver, and, in the filter group, the intra-receiver linear distortion compensation filter is arranged after the intra-receiver non-linear distortion compensation filter and before the intra-transmitter linear distortion compensation filter, and the intra-transmitter linear distortion compensation filter is arranged before the intra-transmitter non-linear distortion compensation filter.

5. The equalization signal processing circuit according to claim 4, wherein the one or more linear distortion compensation filters further include at least one of a wavelength dispersion compensation filter, a polarization separation filter, or a carrier phase compensation filter, and, in the filter group, at least one of the wavelength dispersion compensation filter, the polarization separation filter, or the carrier phase compensation filter are arranged between the intra-receiver linear distortion compensation filter and the intra-transmitter linear distortion compensation filter.

6. The equalization signal processing circuit according to claim 1, wherein the non-linear distortion compensation filter includes a deep neural network (DNN), a convolutional neural network (CNN), or a Volterra filter.

7. The equalization signal processing circuit according to claim 6, wherein the non-linear distortion compensation filter further includes a linear filter being connected in parallel with the DNN, the CNN, or the Volterra filter, adds an output of the DNN, the CNN, or the Volterra filter and an output of the linear filter, and outputs the added result as an output signal.

8. A receiver comprising: the equalization signal processing circuit according to claim 1; and a detector configured to coherent-receive a signal transmitted from a transmitter via a transmission path.

9. The receiver according to claim 8, wherein a plurality of non-linear distortion compensation filters include an intra-transmitter non-linear distortion compensation filter configured to compensate for non-linear distortion occurring in the transmitter, and an intra-receiver non-linear distortion compensation filter configured to compensate for non-linear distortion occurring in a receiver.

10. The receiver according to claim 9, wherein, in a filter group, the intra-receiver non-linear distortion compensation filter is arranged before the intra-transmitter non-linear distortion compensation filter.

11. The receiver according to claim 8, wherein the non-linear distortion compensation filter includes a deep neural network (DNN), a convolutional neural network (CNN), or a Volterra filter.

12. A communication system comprising: a transmitter configured to transmit a signal via a transmission path; and the receiver according to claim 8.

13. The communication system according to claim 12, wherein a plurality of non-linear distortion compensation filters include an intra-transmitter non-linear distortion compensation filter configured to compensate for non-linear distortion occurring in the transmitter, and an intra-receiver non-linear distortion compensation filter configured to compensate for non-linear distortion occurring in the receiver.

14. The communication system according to claim 13, wherein, in a filter group, the intra-receiver non-linear distortion compensation filter is arranged before the intra-transmitter non-linear distortion compensation filter.

15. The communication system according to claim 12, wherein the non-linear distortion compensation filter includes a deep neural network (DNN), a convolutional neural network (CNN), or a Volterra filter.

16. An equalization signal processing method comprising: calculating a loss function, based on a difference between an output signal being output from a filter group in which a plurality of filters connected in series along a signal path of a reception signal being coherent-received are included, the plurality of filters including a plurality of non-linear distortion compensation filters configured to compensate for non-linear distortion included in the reception signal and one or more linear distortion compensation filters configured to compensate for linear distortion included in the reception signal, and a predetermined value of the output signal; and adaptively controlling, by using an error back propagation method, a filter coefficient of at least some of a plurality of non-linear distortion compensation filters and the linear distortion compensation filter, based on the loss function.

17. A non-transitory computer readable medium storing a program for causing a processor to execute processing of: calculating a loss function, based on a difference between an output signal being output from a filter group in which a plurality of filters connected in series along a signal path of a reception signal being coherent-received are included, the plurality of filters including a plurality of non-linear distortion compensation filters configured to compensate for non-linear distortion included in the reception signal and one or more linear distortion compensation filters configured to compensate for linear distortion included in the reception signal, and a predetermined value of the output signal; and adaptively controlling, by using an error back propagation method, a filter coefficient of at least some of a plurality of non-linear distortion compensation filters and the linear distortion compensation filter, based on the loss function.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0034] The above and other aspects, features and advantages of the present disclosure will become more apparent from the following description of certain exemplary embodiments when taken in conjunction with the accompanying drawings, in which:

[0035] FIG. 1 is a block diagram schematically illustrating a communication system according to the present disclosure;

[0036] FIG. 2 is a block diagram illustrating a schematic configuration of a receiver;

[0037] FIG. 3 is a block diagram illustrating a signal transmission system according to one example embodiment of the present disclosure;

[0038] FIG. 4 is a block diagram illustrating an example of digital signal processing in an equalization unit;

[0039] FIG. 5 is a block diagram illustrating a configuration example of a non-linear filter;

[0040] FIG. 6 is a flowchart illustrating an operation procedure of the equalization unit;

[0041] FIG. 7 is a graph illustrating a simulation result;

[0042] FIG. 8 is a block diagram illustrating an example of reception side equalization digital signal processing described in Non Patent Literature 1;

[0043] FIG. 9 is a block diagram illustrating a 2×2 MIMO filter;

[0044] FIG. 10 is a block diagram illustrating an adaptive multi-layer filter for performing equalization signal processing;

[0045] FIG. 11 is a block diagram illustrating a WL 2×1 filter; and

[0046] FIG. 12 is a block diagram illustrating an example of reception side digital signal processing.

[0047] FIG. 13 is a block diagram illustrating an example of configuration of a reception side digital signal processing.

EMBODIMENTS

[0048] Prior to description of an example embodiment of the present disclosure, an outline of the present disclosure will be described. FIG. 1 schematically illustrates a communication system according to the present disclosure. A communication system 10 includes a transmitter 11 and a receiver 15. The transmitter 11 and the receiver 15 are connected to each other via a transmission path 13. The transmitter 11 transmits a signal via the transmission path 13. The receiver 15 receives a signal transmitted from the transmitter 11 via the transmission path 13.

[0049] FIG. 2 illustrates a schematic configuration of the receiver 15. The receiver 15 includes a detector 21 and an equalization signal processing circuit 22. The detector 21 coherent-receives a signal transmitted from the transmitter 11. The equalization signal processing circuit 22 performs equalization signal processing on the reception signal being coherent-received.

[0050] The equalization signal processing circuit 22 includes a filter group 23 and a coefficient updating unit 26. The filter group 23 includes a plurality of filters connected in series along a signal path of a reception signal. The plurality of filters includes a plurality of non-linear distortion compensation filters 24 and one or more linear distortion compensation filters 25. Each of the non-linear distortion compensation filters 24 compensates for non-linear distortion included in the reception signal. Each of the linear distortion compensation filters 25 compensates for linear distortion included in the reception signal.

[0051] The coefficient updating unit 26 controls filter coefficients of the plurality of non-linear distortion compensation filters 24 and at least one filter coefficient of one or more linear distortion compensation filters 25. The coefficient updating unit 26 adaptively controls the filter coefficient, by using the error back propagation method, based on a difference between an output signal being output from the filter group 23 and a predetermined value of the output signal.

[0052] In the present disclosure, the coefficient updating unit 26 calculates a gradient of each filter coefficient by the error back propagation method, for example, as a loss function to minimize magnitude of a difference from a desired state of an output of a final filter block of the filter group 23. The coefficient updating unit 26 controls the filter coefficient of the non-linear distortion compensation filter 24 and the filter coefficient of at least some of the linear distortion compensation filter 25 in response to the gradient of the filter coefficient. In the present disclosure, the coefficient updating unit 26 controls the filter coefficients of not only the final stage filter but also the non-linear distortion compensation filter 24 arranged in a preceding stage, by using the magnitude of the difference from the desired state of the output of the final filter block of the filter group 23. By adopting such a configuration, in a communication system in which various pieces of distortion are present in a reception signal, it is possible to perform non-linear distortion compensation in a transmitter and a receiver with high accuracy.

[0053] Hereinafter, an example embodiment of the present disclosure will be described in detail with reference to the drawings. FIG. 3 illustrates a signal transmission system according to one example embodiment of the present disclosure. In the present example embodiment, it is assumed that the signal transmission system is an optical fiber communication system that adopts a polarization multiplexing QAM system and performs coherent reception. An optical fiber communication system 100 includes an optical transmitter 110, a transmission path 130, and an optical receiver 150. The optical fiber communication system 100 constitutes, for example, an optical submarine cable system. The optical fiber communication system 100 corresponds to the communication system 10 illustrated in FIG. 1. The optical transmitter 110 corresponds to the transmitter 11 illustrated in FIG. 1. The transmission path 130 corresponds to the transmission path 13 illustrated in FIG. 1. The optical receiver 150 corresponds to the receiver 15 illustrated in FIG. 1.

[0054] The optical transmitter 110 converts a transmission data into a polarization multiplexed optical signal. The optical transmitter 110 includes an encoding unit 111, a pre-equalization unit 112, a digital analog converter (DAC) 113, an optical modulator 114, and a laser diode (LD) 115. The encoding unit 111 encodes a transmission data and generates a signal sequence for optical modulation. In a case of the polarization multiplexing QAM system, the encoding unit 111 generates a total of four series of signals being an in-phase (I) component and a quadrature (Q) component of each of X polarization (first polarization) and Y polarization (second polarization). Note that, in FIG. 3, for the sake of simplification of the drawing, encoded four-series signals are illustrated as one solid line. Hereinafter, one solid line illustrated in FIG. 3 collectively represents signal series having a predetermined number, as a physical entity.

[0055] The pre-equalization unit 112 performs pre-equalization for compensating for distortion or the like of a device in the optical transmitter 110 in advance for the encoded four-series signal. The DAC 113 converts each of the four-series signals being performed the pre-equalization into an analog electric signal.

[0056] The LD 115 outputs continuous wave (CW) light. The optical modulator 114 modulates the CW light output from the LD 115 in response to the four-series signals output from the DAC 113, and generates an optical signal of polarization multiplexing QAM. The optical signal (polarization multiplexed optical signal) generated by the optical modulator 114 is output to the transmission path 130.

[0057] The transmission path 130 transmits the polarization multiplexed optical signal output from the optical transmitter 110 to the optical receiver 150. The transmission path 130 includes an optical fiber 132 and an optical amplifier 133. The optical fiber 132 guides an optical signal transmitted from the optical transmitter 110. The optical amplifier 133 amplifies an optical signal, and compensates for a propagation loss in the optical fiber 132. The optical amplifier 133 is configured, for example, as an erbium doped fiber amplifier (EDFA). The transmission path 130 may include a plurality of optical amplifiers 133.

[0058] The optical receiver 150 includes an LD 151, a coherent receiver 152, an analog digital converter (ADC) 153, an equalization unit 154, a decoding unit 155, and a distortion estimation unit 156. In the optical receiver 150, circuits such as the equalization unit (equalizer) 154 and the decoding unit (decoder) 155 may be configured by using a device such as a digital signal processor (DSP), for example.

[0059] The LD 151 outputs CW light as local oscillator light. In the present example embodiment, the coherent receiver 152 is configured as a polarization diversity type coherent receiver. The coherent receiver 152 performs coherent detection on an optical signal transmitted through the optical fiber 132, by using the CW light output from the LD 151. The coherent receiver 152 outputs four-series reception signals (electric signals) being equivalent to the I component and Q component of the X polarization and Y polarization being performed coherent detection. The coherent receiver 152 corresponds to the detector 21 illustrated in FIG. 2.

[0060] The ADC 153 samples the reception signal output from the coherent receiver 152, and converts the reception signal into a signal in a digital domain. The equalization unit 154 performs reception side equalization signal processing on the four-series reception signals being sampled by the ADC 153. The equalization unit 154 performs equalization signal processing on the reception signal, and thereby compensates for various pieces of distortion in the optical fiber communication system including non-linear distortion occurring in the optical transmitter 110 and the optical receiver 150. The decoding unit 155 decodes the signal being performed the equalization signal processing by the equalization unit 154, and restores the transmitted data. The decoding unit 155 outputs the restored data to not-illustrated another circuit.

[0061] FIG. 4 illustrates an example of digital signal processing in the equalization unit 154. The equalization unit 154 includes an intra-receiver non-linear distortion compensation filter 161, an intra-receiver linear distortion compensation filter 162, a wavelength dispersion compensation filter 163, a polarization separation filter 164, a carrier phase compensation filter 165, an intra-transmitter linear distortion compensation filter 166, and an intra-transmitter non-linear distortion compensation filter 167 in this order from an input side of an optical signal. The equalization unit 154 further includes a coefficient updating unit 170, a phase-locked loop (PLL) 171, and a loss function calculation unit 172. The equalization unit 154 corresponds to the equalization signal processing circuit 22 illustrated in FIG. 2.

[0062] Considering an optical fiber communication system, distortion occurs in the following order: (1) distortion in a transmitter, (2) a phenomenon in an optical fiber (wavelength dispersion, polarization variation/polarization mode dispersion), (3) a frequency offset, and (4) distortion in a receiver. Herein, (2) and (3) are interchangeable when a non-linear effect in an optical fiber is ignored. In the present example embodiment, in consideration of an order in which distortion occurs and interchangeability, a filter which performs intra-receiver non-linear distortion compensation, intra-receiver linear distortion compensation, wavelength dispersion compensation, polarization variation compensation, carrier phase compensation, intra-transmitter linear distortion compensation, and intra-transmitter non-linear distortion compensation in this order is used.

[0063] The intra-receiver non-linear distortion compensation filter 161 is a filter for compensating for intra-receiver non-linear distortion. The intra-receiver non-linear distortion compensation filter 161 includes a non-linear filter arranged for each polarization. The non-linear filter included in the intra-receiver non-linear distortion compensation filter 161 is a non-linear filter whose output is expressed in a form differentiable with respect to an input and a coefficient. The intra-receiver non-linear distortion compensation filter 161 may be configured by using, for example, a DNN, a convolutional neural network (CNN), or a Volterra filter. The intra-transmitter non-linear distortion compensation filter 167 is a filter for compensating for intra-transmitter non-linear distortion. The intra-transmitter non-linear distortion compensation filter 167 includes a non-linear filter arranged for each polarization, similarly to the intra-receiver non-linear distortion compensation filter 161. The intra-receiver non-linear distortion compensation filter 161 and the intra-transmitter non-linear distortion compensation filter 167 each correspond to the non-linear distortion compensation filter 24 illustrated in FIG. 2.

[0064] FIG. 5 illustrates a configuration example of a non-linear filter that may be used for the intra-receiver non-linear distortion compensation filter 161 and the intra-transmitter non-linear distortion compensation filter 167. The intra-receiver non-linear distortion compensation filter 161 and the intra-transmitter non-linear distortion compensation filter 167 include a non-linear filter 180 for each polarization. The non-linear filter 180 includes a signal conversion unit 181, CNNs 182 and 184, linear filters 183 and 185, and a signal conversion unit 186.

[0065] One series complex signal is input to the non-linear filter 180 associated with each polarization. The signal conversion unit 181 converts the input complex signal into two real signals of the IQ component. In a transmitter and a receiver, usually, non-linear distortion occurs independently for each IQ component. Therefore, compensation of non-linear distortion is performed for each IQ component. A signal of the I component is branched into two and input to the CNN 182 and the linear filter 183. In addition, a signal of the Q component is branched into two and input to the CNN 184 and the linear filter 185. The linear filters 183 and 185 have the same time spread as the time spread of convolution by CNNs 182 and 184, respectively. The linear filters 183 and 185 serve to help convergence of adaptive control of coefficients of CNNs 182 and 184.

[0066] A signal acquired by adding an output of the CNN 182 and an output of the linear filter 183 with respect to the signal of the I component is input to the signal conversion unit 186. In addition, a signal acquired by adding an output of the CNN 184 and an output of the linear filter 185 with respect to the signal of the Q component is input to the signal conversion unit 186. The signal conversion unit 186 converts the signal of the I component and the signal of the Q component into complex signals, and outputs the converted complex signals.

[0067] Generally, random values are selected as initial values of the coefficients inside the CNN. Therefore, the output of the CNN becomes a random value in an initial stage of the control. On the other hand, in consideration of distortion compensation in a communication system, it is usually assumed that influence of non-linear distortion is perturbed and is not large compared to other distortion. Thus, unlike image recognition, which is another common application of the CNN, in the CNN application to distortion compensation in a communication system, linear processing is considered to function as a zero order solution.

[0068] In the non-linear filter 180 illustrated in FIG. 5, initial coefficients of the linear filters 183 and 185 are not random values, but are set to appropriately selected values. In the non-linear filter 180, the outputs of the CNNs 182 and 184, and the outputs of the linear filters 183 and 185 whose initial coefficients are appropriately selected are added. In this case, the non-linear filter 180 can output a value close to a desired state to some extent from beginning of the adaptive control. Therefore, when the non-linear filter 180 as illustrated in FIG. 5 is used, a time required for convergence of the adaptive control can be shortened.

[0069] Returning to FIG. 4, the intra-receiver linear distortion compensation filter 162 is a filter for compensating for intra-receiver linear distortion. The intra-receiver linear distortion compensation filter 162 includes a WL 2×1 filter arranged for each polarization. The wavelength dispersion compensation filter 163 is a filter for performing wavelength dispersion compensation. The wavelength dispersion compensation filter 163 includes an SL filter arranged for each polarization. The polarization separation filter 164 is a filter for performing polarization variation compensation. The polarization separation filter 164 includes a 2×2 MIMO SL filter.

[0070] The carrier phase compensation filter 165 is a filter for performing carrier phase compensation. The carrier phase compensation filter 165 includes an SL filter arranged for each polarization. The intra-transmitter linear distortion compensation filter 166 is a filter for compensating for intra-transmitter linear distortion. The intra-transmitter linear distortion compensation filter 166 includes a WL 2×1 filter arranged for each polarization. The intra-transmitter linear distortion compensation filter 166 is arranged before the intra-transmitter non-linear distortion compensation filter 167. The intra-receiver linear distortion compensation filter 162 is arranged after the intra-receiver non-linear distortion compensation filter 161 and before the intra-transmitter linear distortion compensation filter 166. The intra-receiver linear distortion compensation filter 162, the wavelength dispersion compensation filter 163, the polarization separation filter 164, the carrier phase compensation filter 165, and the intra-transmitter linear distortion compensation filter 166 each correspond to the linear distortion compensation filter 25 included in the filter group 23 illustrated in FIG. 2.

[0071] Note that, in theory, the linear filters 183 and 185 used in the non-linear filter 180 illustrated in FIG. 5 can also serve as a role of the intra-receiver linear distortion compensation filter 162 and the intra-transmitter linear distortion compensation filter 166 illustrated in FIG. 4. However, due to a relationship in which the outputs of the CNNs 182 and 184 and the outputs of the linear filters 183 and 185 are added, the time spread of the linear filters 183 and 185 is necessary to match with that of the CNNs 182 and 184. Generally, the time spread of the linear distortion and the time spread of the non-linear distortion occurring in the transmitter or in the receiver are different. Therefore, even when the intra-receiver non-linear distortion compensation filter 161 or the intra-transmitter non-linear distortion compensation filter 167 includes the linear filters 183 and 185, it is preferable in design that the intra-receiver linear distortion compensation filter 162 or the intra-transmitter linear distortion compensation filter 166 is separately arranged.

[0072] In FIG. 4, the filters handled in each of the blocks are configured according to a characteristic of the distortion to be compensated by each filter. For example, accumulation of wavelength dispersion does not change when a communication path is not changed. Therefore, the filter of the wavelength dispersion compensation filter 163 can be handled statically after once the coefficient is set in response to a wavelength dispersion amount to be compensated. In addition, the wavelength dispersion is an SL process, there is little polarization dependence, and there is no mixing between polarization due to only the wavelength dispersion. For this reason, the filter of the wavelength dispersion compensation filter 163 is an SL filter that does not have a cross term between polarization and is independent of each polarization.

[0073] In contrast, since intra-receiver linear distortion, i.e., intra-receiver IQ distortion, is a WL process, the filter of the intra-receiver linear distortion compensation filter 162 is a WL filter. Since the intra-receiver IQ distortion usually does not cause mixing between polarization, a 2× 1 WL filter being independent of each polarization is used for the intra-receiver linear distortion compensation filter 162. When it is supposed that a receiver having a configuration in which mixing between polarization is suggested to occur in the intra-receiver IQ distortion is used, a 4×2 WL filter having a cross term between polarization is used in the intra-receiver linear distortion compensation filter 162.

[0074] Note that, in the above description, an example in which the equalization unit 154 includes the intra-receiver non-linear distortion compensation filter 161, the intra-receiver linear distortion compensation filter 162, the wavelength dispersion compensation filter 163, the polarization separation filter 164, the carrier phase compensation filter 165, the intra-transmitter linear distortion compensation filter 166, and the intra-transmitter non-linear distortion compensation filter 167 has been described. However, the filter included in the equalization unit 154 is not limited to the filter described above. For example, when there is a factor occurring distortion other than the distortion described above, the equalization unit 154 may further include a filter for compensating for the distortion. In that case, the filter is inserted in an appropriate position, considering an order in which the distortion occurs. Alternatively, some of the filters described above can be omitted in the equalization unit 154. As an example, the equalization unit 154 may include a coefficient-fixed matched filter after the intra-transmitter non-linear distortion compensation filter 167. In that case, the filter block in the final stage may be a coefficient-fixed matched filter.

[0075] The loss function calculation unit 172 calculates magnitude of a difference from a desired state of an output signal of the intra-transmitter non-linear distortion compensation filter 167, which is the output of the final filter block, as a loss function. The coefficient updating unit 170 adaptively controls the filter coefficients of the filter blocks other than the wavelength dispersion compensation filter 163 and the carrier phase compensation filter 165, which can be handled statically. That is, the coefficient updating unit 170 adaptively controls the filter coefficient of each filter block of the intra-receiver non-linear distortion compensation filter 161, the intra-receiver linear distortion compensation filter 162, the polarization separation filter 164, the intra-transmitter linear distortion compensation filter 166, and the intra-transmitter non-linear distortion compensation filter 167. The coefficient updating unit 170 updates the coefficient of each filter block by calculating a gradient in each filter block, by using the error back propagation method, as a loss function to minimize a loss function calculated by the loss function calculation unit 172. The coefficient updating unit 170 corresponds to the coefficient updating unit 26 illustrated in FIG. 2.

[0076] Herein, in the intra-receiver non-linear distortion compensation filter 161, the intra-receiver linear distortion compensation filter 162, the wavelength dispersion compensation filter 163, the polarization separation filter 164, the carrier phase compensation filter 165, the intra-transmitter linear distortion compensation filter 166, and the intra-transmitter non-linear distortion compensation filter 167, the output of each filter block can be differentiated with respect to an input and a coefficient. Therefore, the coefficient updating unit 170 can sequentially calculate the gradient for the loss function of the coefficient from the final layer, by using the error back propagation method. For example, the coefficient updating unit 170 uses the magnitude of the difference from the desired state of the final output as a loss function, and updates the filter coefficient of each filter block, by using a stochastic gradient descent method, in such a way as to minimize average magnitude of the loss function.

[0077] The PLL 171 determines a phase rotation amount of the carrier phase compensation filter 165, based on the output of the intra-transmitter non-linear distortion compensation filter 167, which is the final output of the filter block. The coefficient of the carrier phase compensation filter 165 is controlled by the PLL 171.

[0078] An operation procedure will be described. FIG. 6 illustrates an operation procedure (equalization signal processing method) of the equalization unit 154. The loss function calculation unit 172 calculates a loss function, based on a difference between an output signal being output from the intra-transmitter non-linear distortion compensation filter 167, which is a final filter block of a plurality of filter blocks connected in series, and a desired value of the output signal (step S1). The coefficient updating unit 170 adaptively controls the filter coefficients of the intra-receiver non-linear distortion compensation filter 161, the intra-receiver linear distortion compensation filter 162, the polarization separation filter 164, the intra-transmitter linear distortion compensation filter 166, and the intra-transmitter non-linear distortion compensation filter 167, based on the loss function, by using the error back propagation method (step S2). Apart from the above, the PLL 171 controls the coefficient of the carrier phase compensation filter 165 by using the output signal output from the intra-transmitter non-linear distortion compensation filter 167.

[0079] Hereinafter, the operation of the equalization unit 154 according to the present example embodiment will be described in detail. In the following description, it is considered that the FIR filter is used as all of the filters of each block, including a filter convolved by the CNN. In general, it is considered that, in the filter group, L-layer filters are connected in series. In a case of the example in FIG. 4, L=7.

[0080] An output (output vector) of the filter in an l-th stage (1≤1≤L) at time k (k is an integer) is denoted by u.sub.i.sup.[l][k], and an input (input vector) is denoted by u.sub.i.sup.[l-1][k]. i and j each represent the polarization thereof. The following description can be easily extended to a case of a spatial multiplex transmission or the like by extending an allowable range of i to twice of the number of modes. In a case where it is assumed that lengths of the input vector and the output vector are M.sub.in.sup.[l] and M.sub.out.sup.[l], respectively, the output vector u.sub.i.sup.[l][k] and the input vector u.sub.i.sup.[l-1][k] are represented by equations 1 and 2 below, respectively.

u.sub.i.sup.[l][k]=(u.sub.i.sup.[l][k],u.sub.i.sup.[l][k−1], . . . ,u.sub.i.sup.[l][k−M.sub.out.sup.[l]+1]).sup.T  (1)

u.sub.i.sup.[l-1][k]=(u.sub.i.sup.[l-1][k],u.sub.i.sup.[l-1][k−1], . . . ,u.sub.i.sup.[l-1][k−M.sub.in.sup.[l]+1]).sup.T  (2)

[0081] In the above equation, [T] represents transposition. In the above equation, each component of the vector is a complex signal.

[0082] Regarding the number of taps M.sup.[l] of a filter in the l-th stage, from a characteristic of convolution, the following can be said.

M.sup.[l]=M.sub.in.sup.[l]−M.sub.out.sup.[l]+1  (3)

When the filter in the l-th stage is an SL MIMO filter, the FIR filter coefficient (coefficient vector) h.sub.ij.sup.[l] of the M.sup.[l] taps is represented by an equation 4 below.

h.sub.ij.sup.[l]=(h.sub.ij.sup.[l][0],h.sub.ij.sup.[l][1], . . . ,h.sub.ij.sup.[l][M.sup.[l]−1]).sup.T  (4)

[0083] When the input (input vector) of the filter in the l-th stage is as follows,

u.sub.i.sup.[l-1][k]=(u.sub.i.sup.[l-1][k],u.sub.i.sup.[l-1][k−1], . . . ,u.sub.i.sup.[l-1][k−M.sup.[l]+1]).sup.T  (5)

an output sample can be said as the following.

[00001]$\begin{array}{cc}{u}_i^{\left(l\right)}\left[k\right]=\underset{j=1}{\overset{2}{.Math.}}{h}_{\mathrm{ij}}^{\left(l\right)†}{\stackrel{\_}{u}}_j^{\left[l-1\right]}\left[k\right]& \left(6\right)\end{array}$

In the above equation, [.sup.+] represents Hermite conjugate. Therefore, the following can be said.

[00002]$\begin{array}{cc}{u}_i^{\left(l\right)}\left[k\right]=\underset{j=1}{\overset{2}{.Math.}}{H}_{\mathrm{ij}}^{\left(l\right)*}{u}_j^{\left[l-1\right]}\left[k\right]& \left(7\right)\end{array}$

In an equation 7, [*] represents a complex conjugate. The H.sub.ij.sup.[l] is represented by an equation 8 below.

[00003]$\begin{array}{cc}{H}_{\mathrm{ij}}^{\left(l\right)}=\left(\begin{array}{ccccccc}{h}_{\mathrm{ij}}^{\left(l\right)}\left[0\right]& h_{\mathrm{ij}}^{\left(l\right)}\left[1\right]& .Math.& h_{\mathrm{ij}}^{\left(l\right)}\left[M^{\left(l\right)}-1\right]& 0& .Math.& 0\\ 0& \ddots & \ddots & & \ddots & \ddots & .Math.\\ .Math.& & & & & & 0\\ 0& .Math.& 0& h_{\mathrm{ij}}^{\left(l\right)}\left[0\right]& h_{\mathrm{ij}}^{\left(l\right)}\left[1\right]& .Math.& h_{\mathrm{ij}}^{\left(l\right)}\left[M^{\left(l\right)}-1\right]\end{array}\right)& \left(8\right)\end{array}$

[0084] When the above equation 7 is transformed, an equation 9 below is acquired.

[00004]$\begin{array}{cc}{u}_i^{\left(l\right)}\left[k\right]=\underset{j=1}{\overset{2}{.Math.}}{U}_j^{\left(l-1\right)}\left[k\right]h_{\mathrm{ij}}^{\left(l\right)*}& \left(9\right)\end{array}$

[0085] In the above equation 9, the U.sub.j.sup.[l-1] can be said as following.

[00005]$\begin{array}{cc}{U}_j^{\left[l-1\right]}=\left(\begin{array}{cccc}{u}_j^{\left[l-1\right]}\left[k\right]& u_j^{\left[l-1\right]}\left[k-1\right]& .Math.& u_j^{\left[l-1\right]}\left[k-M^{\left[l\right]}+1\right]\\ u_j^{\left[l-1\right]}\left[k-1\right]& u_j^{\left[l-1\right]}\left[k-2\right]& .Math.& u_j^{\left[l-1\right]}\left[k-M^{\left[l\right]}\right]\\ .Math.& & & .Math.\\ u_j^{\left[l-1\right]}\left[k-M_{\mathrm{out}}^{\left[l\right]}+1\right]& u_j^{\left[l-1\right]}\left[k-M_{\mathrm{out}}^{\left[l\right]}\right]& .Math.& u_j^{\left[l-1\right]}\left[k-M_{\mathrm{in}}^{\left[l\right]}+1\right]\end{array}\right)& \left(10\right)\end{array}$

In the equation 9, by replacing a sum for j with j=i, this also includes a case of a 1×1 SL filter for each polarization.

[0086] When the filter in the 1-th stage is a WL MIMO filter, h.sub.ij.sup.[l] represented by the equation 4 and h*.sub.ei.sup.[l] represented by an equation 11 below are filter coefficients (coefficient vectors).

h*.sub.ij.sup.[l]=(h*.sub.ij.sup.[l][0],h*.sub.ij.sup.[l][1], . . . ,h*.sub.ij.sup.[l][M.sup.[l]−1]).sup.T  (11)

[0087] The output sample becomes the following.

[00006]$\begin{array}{cc}{u}_i^{\left(l\right)}\left[k\right]=\underset{j=1}{\overset{2}{.Math.}}{h}_{\mathrm{ij}}^{\left(l\right)†}{\stackrel{\_}{u}}_j^{\left[l-1\right]}\left[k\right]+\underset{j=1}{\overset{2}{.Math.}}{h}_{*\mathrm{ij}}^{\left(l\right)†}{\stackrel{\_}{u}}_j^{\left[l-1\right]*}\left[k\right]& \left(12\right)\end{array}$

Similarly to the above, when H*.sub.ij.sup.[l] is as follows,

[00007]$\begin{array}{cc}{H}_{*\mathrm{ij}}^{\left(l\right)}=\left(\begin{array}{ccccccc}{h}_{*\mathrm{ij}}^{\left(l\right)}\left[0\right]& h_{\mathrm{ij}}^{\left(l\right)}\left[1\right]& .Math.& h_{*\mathrm{ij}}^{\left(l\right)}\left[M^{\left(l\right)}-1\right]& 0& .Math.& 0\\ 0& \ddots & \ddots & & \ddots & \ddots & .Math.\\ .Math.& & & & & & 0\\ 0& .Math.& 0& h_{*\mathrm{ij}}^{\left(l\right)}\left[0\right]& h_{*\mathrm{ij}}^{\left(l\right)}\left[1\right]& .Math.& h_{*\mathrm{ij}}^{\left(l\right)}\left[M^{\left(l\right)}-1\right]\end{array}\right)& \left(13\right)\end{array}$

the following can be said.

[00008]$\begin{array}{cc}\begin{array}{c}{u}_i^{\left(l\right)}\left[k\right]=\underset{j=1}{\overset{2}{.Math.}}{H}_{\mathrm{ij}}^{\left(l\right)*}{u}_j^{\left[l-1\right]}\left[k\right]+\underset{j=1}{\overset{2}{.Math.}}{H}_{*\mathrm{ij}}^{\left(l\right)*}{u}_j^{\left[l-1\right]*}\left[k\right]\\ =\underset{j=1}{\overset{2}{.Math.}}{U}_{\mathrm{ij}}^{\left(l-1\right)}\left[k\right]h_{\mathrm{ij}}^{\left[l\right]}+\underset{j=1}{\overset{2}{.Math.}}{U}_j^{\left(l-1\right)}\left[k\right]h_{*\mathrm{ij}}^{\left[l\right]*}\end{array}& \left(14\right)\end{array}$

This also includes a case of a 2×1 WL filter for each polarization.

[0088] When the filter in the 1-th stage is a non-linear filter having the configuration illustrated in FIG. 5, an input of the non-linear filter converted into a real signal of the IQ component is as follows.

x.sub.i[k]=(x.sub.i[k],x.sub.i[k−1], . . . ,x.sub.i[k−M.sub.in.sup.[l]+1]).sup.T  (15)

Herein, i is i=1, 2, 3, and 4, and each is equivalent to the I component and the Q component of the X polarization, and the I component and the Q component of the Y polarization, respectively. For example, the following can be said.

x.sub.i[k]=Re[u.sub.i.sup.[l-1][k]]  (16)

In the above equation 16, Re represents a real part. The output y.sub.i[k] of the non-linear filter converted into a real signal of the IQ component is as follows.

.sub.i[k]=(.sub.i[k],.sub.i[k−1], . . . ,.sub.i[k−M.sub.out.sup.[l]+1]).sup.T  (17)

Similarly to the input of the non-linear filter, i is i=1, 2, 3, and 4, and each is equivalent to the I component and Q component of the X polarization, and the I component and Q component of the Y polarization, respectively.

[0089] Herein, focusing on any of the components j=1, 2, 3, and 4, and a non-linear filter to be applied to the component will be considered. The CNN is as P-layer, and an input of p-layer (1≤p≤P) of the CNN is as follows.

x.sub.i.sup.(p-1)[k]=(x.sub.i.sup.(p-1)[k],x.sub.i.sup.(p-1)[k−1], . . . ,x.sub.i.sup.(p-1)[k−M.sub.in.sup.(p)+1]).sup.T  (18)

For the input of p=first layer of the CNN, the following can be said.

x.sub.i.sup.(0)[k]=x.sub.j[k]  (19)

In the CNN, since the non-linear filter is applied independently for each polarization and each IQ component, i=1. Note that, it can be easily extended when the non-linear filter includes a cross term instead of being applied independently.

[0090] The output of the p-th layer of the CNN is as follows.

x.sub.i.sup.(p)=(x.sub.i.sup.(p)[k],x.sub.i.sup.(p)[k−1], . . . ,x.sub.i.sup.(p)[k−M.sub.out.sup.(p)+1]).sup.T  (20)

In an intermediate layer of the CNN, i=1 is not required, and i=1, 2, . . . , N.sup.(p) with the number of filters of the p-th layer as the N.sup.(p).

[0091] The output of the p=P-th layer, which is the final layer of the CNN, is represented as follows.

x.sub.i.sup.(P)[k]=.sub.i.sup.(CNN)[k]  (21)

In the final P-th layer of the CNN, i=1. An activation function of the p-th layer of the CNN is defined as g.sup.(p). For example, the g.sup.(p) is a rectified linear unit (ReLU) except for the final P-th layer, and is a linear function (Linear) in the final P-th layer. Between the input and the output of the p-th layer of the CNN, the following relationship holds.

[00009]$\begin{array}{cc}{z}_i^{\left(p\right)}\left[k\right]=\stackrel{N^{\left(p\right)}}{\underset{j=1}{.Math.}}{H}_{\mathrm{ij}}^{\left(p\right)}{x}_j^{\left(p-1\right)}\left[k\right]+b_i^{\left(p\right)}1& \left(22\right)\end{array}$$\begin{array}{cc}{x}_i^{\left(p\right)}\left[k\right]=g^{\left(p\right)}\left(z_i^{\left(p\right)}\left[k\right]\right)& \left(23\right)\end{array}$

[0092] When the g.sup.(p) is the linear function, the following can be said.

g.sup.(p)(.sub.i.sup.(p)[k−n])=.sub.i.sup.(p)[k−n]  (24)

When the g.sup.(p) is the ReLU, the following can be said.

g.sup.(p)(.sub.i.sup.(p)[k−n])=max(.sub.i.sup.(p)[k−n],0)  (25)

Herein, “1” in the equation 22 is a vector having an appropriate size and all components with 1. In addition, the equation 22 can also be represented as follows.

[00010]$\begin{array}{cc}{z}_i^{\left(p\right)}\left[k\right]=\stackrel{N^{\left(p\right)}}{\underset{j=1}{.Math.}}{X}_j^{\left(p-1\right)}\left[k\right]h_{\mathrm{ij}}^{\left(p\right)}+b_i^{\left(p\right)}1& \left(26\right)\end{array}$

The h.sub.ij.sup.(p) and b.sub.i.sup.(p) constituting the H.sub.ij.sup.(p) are the filter coefficients of the p-th layer of the CNN. The h.sub.ij.sup.(p) is the number of taps M.sup.(p), and M.sup.(p)=M.sub.in.sup.(p)−M.sub.out.sup.(p)+1. By sequentially calculating this from the first layer of the CNN, y.sub.i.sup.(CNN)[k]=x.sub.i.sup.(p) [k] is acquired.

[0093] The output of the linear filter to be added to the output of the CNN is as y.sub.i.sup.(L)[k]. The output y.sub.i.sup.(L)[k] of the linear filter is as follows.

[00011]$\begin{array}{cc}{y}_i^{\left(L\right)}\left[k\right]-\underset{j}{.Math.}{H}_{\mathrm{ij}}^{\left(L\right)}{x}_j^{\left(L\right)}\left[k\right]& \left(27\right)\end{array}$

Herein, a sum for j is not necessary when the non-linear filter is independently performed for each polarization and each IQ component, and x.sub.j.sup.(L)[k]=x.sub.j[k] for j being focused first. h.sub.ij.sup.(L) constituting H.sub.ij.sup.(L) is the coefficient of the linear filter. The number of taps in the linear filter is as follows in such a way as to be similar to the number of taps in the CNN.

[00012]$\begin{array}{cc}{M}^{\left(L\right)}=1-P+\stackrel{P}{\underset{p=1}{.Math.}}{M}^{\left(p\right)}& \left(28\right)\end{array}$

A sum y.sub.i.sup.(CNN)[k]+y.sub.i.sup.(L)[k] of the output of the CNN and the output of the linear filter corresponds to the following regarding any of j=1 to j=4 being focused first.

.sub.j[k]=.sub.i.sup.(CNN)[k]+.sub.i.sup.(L)[k]  (29)

[0094] The output of the filter of the final L-th layer can be calculated by sequentially applying the input-output relationship of the various filters described above from the input. The filter output (output vector) of the final L-th layer is u.sub.i.sup.[L][k], where M.sub.out.sup.[L]=1. Therefore, from the equation 1, the following can be said.

u.sub.i.sup.[L][k]=u.sub.i.sup.[L][k]  (30)

[0095] In the present example embodiment, a loss function φ is constructed from the final filter output. The loss function can be constructed in a manner such as a CMA or a DDLMS. For example, in a case of a normal CMA, k is a symbol timing, r is a desired value (predetermined value) of an amplitude of a filter output, and then an expected value <φ[k]> of magnitude of an error from the desired value r of the filter output is as a loss function. Herein <⋅> represents an expected value. An instantaneous value φ[k] of the loss function is as follows.

[00013]$\begin{array}{cc}\varphi \left[k\right]=\stackrel{2}{\underset{i=1}{.Math.}}{\left(r^2-{.Math.u_i^{\left[L\right]}\left[k\right].Math.}^2\right)}^2& \left(31\right)\end{array}$

[0096] The coefficient updating unit 170 (refer to FIG. 4) controls all the filter coefficients of the filter blocks being adaptively controlled from the first layer to the L-th layer in such a way as to minimize the loss function <φ[k]>. The coefficient updating unit 170 controls the filter coefficients of each filter block by using the stochastic gradient descent method. The stochastic gradient descent method is a method in which control of the coefficient is repeated in such a way as to minimize the instantaneous value of the loss function, based on the gradient of the coefficient of the instantaneous value φ[k] of the loss function.

[0097] In the stochastic gradient descent method, the loss function to be minimized is φ=φ[k]. For a certain coefficient ξ, the coefficient update by the stochastic gradient descent method is represented as follows.

[00014]$\begin{array}{cc}{\xi }^{*}.fwdarw.{\xi }^{*}-2\alpha \frac{\partial \varphi }{\partial \xi }& \left(32\right)\end{array}$

In the above equation 32, α is a step size for determining strength of the coefficient update. The gradient of the loss function φ for all filter coefficients can be calculated by the error back propagation method as described below.

[0098] When a loss function of the CMA is used for the output of the final stage of the filter, the following can be said.

[00015]$\begin{array}{cc}\frac{\partial \varphi }{\partial y_i\left[k\right]}=-2e_i{u}_i^{\left[L\right]*}\left[k\right]& \left(33\right)\end{array}$$\begin{array}{cc}\frac{\partial \varphi }{\partial y_i^{*}\left[k\right]}=-2e_i{u}_i^{\left[L\right]}\left[k\right]& \left(34\right)\end{array}$$\begin{array}{cc}{e}_i=r^2-{.Math.u_i^{\left[L\right]}\left[k\right].Math.}^2& \left(35\right)\end{array}$

This is a gradient with respect to the filter output in the L-th stage being the final stage of the loss function. From the gradient of the loss function with respect to the output of the filter in the l-th stage, the filter coefficient of the l-th stage of the loss function, and the gradient with respect to the filter input, i.e., the output of the filter in l-1st stage, can be calculated.

[0099] When the filter in the 1-th stage is an SL MIMO filter, calculation of differentiation results as follows.

[00016]$\begin{array}{cc}\frac{\partial \varphi }{\partial h_{\mathrm{ij}}^{\left[l\right]}}=U_j^{\left[l-1\right]†}\left[k\right]\frac{\partial \varphi }{\partial u_i^{\left[l\right]*}\left[k\right]}& \left(36\right)\end{array}$$\begin{array}{cc}\frac{\partial \varphi }{\partial u_j^{\left[l-1\right]}\left[k\right]}=\underset{i=1}{\overset{2}{.Math.}}{H}_{\mathrm{ij}}^{\left[l\right]†}\frac{\partial \varphi }{\partial u_i^{\left[l\right]}\left[k\right]}& \left(37\right)\end{array}$$\begin{array}{cc}\frac{\partial \varphi }{\partial u_i^{\left[l-1\right]*}\left[k\right]}=\underset{i=1}{\overset{2}{.Math.}}{H}_{\mathrm{ij}}^{\left[l\right]T}\frac{\partial \varphi }{\partial u_i^{\left[l\right]*}\left[k\right]}& \left(38\right)\end{array}$

[0100] When the filter in the l-th stage is a WL MIMO filter, calculation of the differentiation results as follows.

[00017]$\begin{array}{cc}\frac{\partial \varphi }{\partial h_{\mathrm{ij}}^{\left[l\right]}}=U_j^{\left[l-1\right]†}\left[k\right]\frac{\partial \varphi }{\partial u_i^{\left[l\right]*}\left[k\right]}& \left(39\right)\end{array}$$\begin{array}{cc}\frac{\partial \varphi }{\partial h_{*\mathrm{ij}}^{\left[l\right]}}=U_j^{\left[l-1\right]T}\left[k\right]\frac{\partial \varphi }{\partial u_i^{\left[l\right]*}\left[k\right]}& \left(40\right)\end{array}$$\begin{array}{cc}\frac{\partial \varphi }{\partial u_j^{\left[l-1\right]}\left[k\right]}=\underset{i=1}{\overset{2}{.Math.}}\left(H_{\mathrm{ij}}^{\left[l\right]†}\frac{\partial \varphi }{\partial u_i^{\left[l\right]}\left[k\right]}+H_{*\mathrm{ij}}^{\left[l\right]T}\frac{\partial \varphi }{\partial u_i^{\left[l\right]*}\left[k\right]}\right)& \left(41\right)\end{array}$$\begin{array}{cc}\frac{\partial \varphi }{\partial u_j^{\left[l-1\right]*}\left[k\right]}=\underset{i=1}{\overset{2}{.Math.}}\left(H_{*\mathrm{ij}}^{\left[l\right]†}\frac{\partial \varphi }{\partial u_i^{\left[l\right]}\left[k\right]}+H_{\mathrm{ij}}^{\left[l\right]T}\frac{\partial \varphi }{\partial u_i^{\left[l\right]*}\left[k\right]}\right)& \left(42\right)\end{array}$

[0101] When the filter in the l-th stage is the non-linear filter having the configuration illustrated in FIG. 5, the differentiation with respect to the output of the non-linear filter converted into the real signal of the IQ component is as follows, based on the differentiation of Wirtinger.

[00018]$\begin{array}{cc}\frac{\partial \varphi }{\partial y_{2i-1}\left[k\right]}=\frac{\partial \varphi }{\partial u_i^{\left[l\right]}\left[k\right]}+\frac{\partial \varphi }{\partial u_i^{\left[l\right]*}\left[k\right]}& \left(43\right)\end{array}$$\begin{array}{cc}\frac{\partial \varphi }{\partial y_{2i}\left[k\right]}=i\left(\frac{\partial \varphi }{\partial u_i^{\left[l\right]}\left[k\right]}-\frac{\partial \varphi }{\partial u_i^{\left[l\right]*}\left[k\right]}\right)& \left(44\right)\end{array}$

Herein, i=1, 2. The differentiation with respect to the input of the non-linear filter converted into the real signal of the IQ component has the following relationship with the differentiation with respect to the filter input in the l-th stage.

[00019]$\begin{array}{cc}\frac{\partial \varphi }{\partial u_i^{\left[l-1\right]}\left[k\right]}=\frac{1}{2}\left(\frac{\partial \varphi }{\partial x_{2i-1}\left[k\right]}-i\frac{\partial \varphi }{\partial x_{2i}\left[k\right]}\right)& \left(45\right)\end{array}$$\begin{array}{cc}\frac{\partial \varphi }{\partial u_i^{\left[l-1\right]*}\left[k\right]}=\frac{1}{2}\left(\frac{\partial \varphi }{\partial x_{2i-1}\left[k\right]}+i\frac{\partial \varphi }{\partial x_{2i}\left[k\right]}\right)& \left(46\right)\end{array}$

[0102] The differentiation with respect to the outputs y.sub.i.sup.(CNN)[k] and y.sub.i.sup.(L)[k] of the CNN is as follows.

[00020]$\begin{array}{cc}\frac{\partial \varphi }{\partial y_i^{\left(\mathrm{CNN}\right)}\left[k\right]}=\frac{\partial \varphi }{\partial y_i^{\left(L\right)}\left[k\right]}=\frac{\partial \varphi }{\delta y_i\left[k\right]}& \left(47\right)\end{array}$

Herein, i=1, . . . , 4.

[0103] The differentiation with respect to the output of the final P-th layer of the CNN is as follows.

[00021]$\begin{array}{cc}\frac{\partial \varphi }{\partial x_i^{\left(p\right)}\left[k\right]}=\frac{\partial \varphi }{\partial y_i^{\left(\mathrm{CNN}\right)}\left[k\right]}& \left(48\right)\end{array}$

[0104] A relationship between the differentiation with respect to the output of the p-th layer of the CNN, and the differentiation with respect to the coefficient of the p-th layer and the input of the p-th layer can be derived as follows.

[00022]$\begin{array}{cc}\frac{\partial \varphi }{\partial z_i^{\left(p\right)}\left[k-n\right]}=\frac{\partial \varphi }{\partial x_i^{\left(p\right)}\left[k-n\right]}\frac{\partial x_i^{\left(p\right)}\left[k-n\right]}{\partial z_i^{\left(p\right)}\left[k-n\right]}& \left(49\right)\end{array}$

[0105] When the g.sup.(p) is the Linear, an equation 50 below holds.

[00023]$\begin{array}{cc}\frac{\partial x_i^{\left(p\right)}\left[k-n\right]}{\partial z_i^{\left(p\right)}\left[k-n\right]}=1& \left(50\right)\end{array}$

[0106] When the g.sup.(p) is the ReLU, an equation 51 below holds.

[00024]$\begin{array}{cc}\frac{\partial x_i^{\left(p\right)}\left[k-n\right]}{\partial z_i^{\left(p\right)}\left[k-n\right]}=\{\begin{array}{c}1\left(z_i^{\left(p\right)}\left[k-n\right]\ge 0\right)\\ 0\left(z_i^{\left(p\right)}\left[k-n\right]<0\right)\end{array}=\chi \left(z_i^{\left(p\right)}\left[k-n\right]\right)& \left(51\right)\end{array}$

[0107] In the equation 51, χ represents a step function.

[0108] By repeatedly applying equations 52 to 54 below, all the coefficients of the CNN and the differentiation ∂φ/∂x.sub.j.sup.(0)[k] for the input of the CNN can be acquired.

[00025]$\begin{array}{cc}\frac{\partial \varphi }{\partial h_{\mathrm{ij}}^{\left(p\right)}}=X_j^{\left(p-1\right)T}\left[k\right]\frac{\partial \varphi }{\partial z_i^{\left(p\right)}\left[k\right]}& \left(52\right)\end{array}$$\begin{array}{cc}\frac{\partial \varphi }{\partial b_i^{\left(p\right)}}=\underset{n=1}{\overset{M^{\left(p\right)}}{.Math.}}\frac{\partial \varphi }{\partial z_i^{\left(p\right)}\left[k-n\right]}& \left(53\right)\end{array}$$\begin{array}{cc}\frac{\partial \varphi }{\partial x_j^{\left(p-1\right)}\left[k\right]}=\underset{i=1}{\overset{N^{\left(p\right)}}{.Math.}}{H}_{\mathrm{ij}}^{\left(p\right)T}\frac{\partial \varphi }{\partial z_i^{\left(p\right)}\left[k\right]}& \left(54\right)\end{array}$

[0109] Regarding the linear filter, the following can be said.

[00026]$\begin{array}{cc}\frac{\partial \varphi }{\partial h_{\mathrm{ij}}^{\left(L\right)}}=X_j^{\left(L\right)T}\left[k\right]\frac{\partial \varphi }{\partial y_i^{\left(L\right)}\left[k\right]}& \left(55\right)\end{array}$$\begin{array}{cc}\frac{\partial \varphi }{\partial x_j^{\left(L\right)}\left[k\right]}=\underset{i}{.Math.}{H}_{\mathrm{ij}}^{\left(L\right)T}\frac{\partial \varphi }{\partial y_i^{\left(L\right)}\left[k\right]}& \left(56\right)\end{array}$

From the above, the differentiation with respect to the input of the non-linear filter can be calculated by an equation 57 below.

[00027]$\begin{array}{cc}\frac{\partial \varphi }{\partial x_j\left[k\right]}=\frac{\partial \varphi }{\partial x_j^{\left(0\right)}\left[k\right]}+\frac{\partial \varphi }{\partial x_j^{\left(L\right)}\left[k\right]}& \left(57\right)\end{array}$

[0110] By applying the error back propagation for the various filters described above sequentially from the final layer, the gradients for all coefficients can be calculated up to the filter of the first layer. As a result, the filter coefficients of each filter block can be updated.

[0111] In the configuration illustrated in FIG. 4, the filter coefficient of the wavelength dispersion compensation filter 163 is determined from an accumulated wavelength dispersion amount D to be compensated, by using an equation 58 below.

[00028]$\begin{array}{cc}{H}_{\mathrm{CD}}\left(\omega \right)=\exp \left(i\frac{{\lambda }^2}{4\pi c}D{\omega }^2\right)& \left(58\right)\end{array}$

[0112] In the equation 58, λ is a wavelength of an optical signal, and c is a speed of light. For the carrier phase compensation filter 165, the coefficient is determined using an equation 59 below.

h.sub.CPEi=exp(−iθ.sub.i[k])  (59)

[0113] In the equation 59, θ.sub.i[k] is determined by the PLL 171, based on the output of the filter in the final stage, as described above.

[0114] As described above, the filter coefficient of the filter block being adaptively controlled can be updated in such a way as to bring the output of the filter block in the final stage closer to a desired state. Accordingly, in an optical fiber communication system including non-linear distortion occurring in a transmitter and a receiver, it is possible to perform reception side equalization signal processing for compensating for various pieces of distortion.

[0115] In order to verify an effect of the reception side equalization signal processing for compensating for various pieces of distortion, the present inventor performs simulation. In the simulation, a single mode fiber 100 km transmission of a 32 Gbaud polarization multiplexed 16QAM signal is simulated.

[0116] In the simulation, non-linear distortion of an a sin(bx) type is applied in the transmitter assuming a characteristic of a Mach-Zehnder optical modulator. In the receiver, non-linear distortion of an a tan h(bx) type is applied assuming clipping. A root Nyquist filter with a roll-off rate of 0.1 is applied to the transmission side and the reception side. A polarization variation and linear IQ distortion in the transmitter and the receiver are not applied. A reception optical signal-to-noise ratio (OSNR) is set to 30 dB/0.1 nm. The reception side equalization signal processing method is applied to the reception signal sampled by the double oversampling. A loss function is constructed by the DALMS.

[0117] As simulation conditions, four conditions are considered: no distortion in the transmitter/receiver (no NL), non-linear distortion in the transmitter (Tx NL), non-linear distortion in the receiver (Rx NL), and non-linear distortion in the transmitter/receiver (both NL). In the simulation, for each condition, magnitude of error vector magnitude (EVM) of the output when the following four types of reception side equalization signal processing are performed is evaluated. [0118] Reception side equalization signal processing that does not compensate for non-linear distortion in the transmitter and the receiver, being equivalent to the configuration illustrated in FIG. 10 (Linear) [0119] In the configuration in FIG. 4, the reception side equalization signal processing in a configuration in which the intra-transmitter non-linear distortion compensation filter 167 is arranged but the intra-receiver non-linear distortion compensation filter 161 is not arranged (Tx CNN) [0120] In the configuration in FIG. 4, the reception side equalization signal processing in a configuration in which the intra-receiver non-linear distortion compensation filter 161 is arranged but the intra-transmitter non-linear distortion compensation filter 167 is not arranged (Rx CNN) [0121] In the configuration in FIG. 4, the reception side equalization signal processing in a configuration in which both the intra-receiver non-linear distortion compensation filter 161 and the intra-transmitter non-linear distortion compensation filter 167 are arranged (both CNN)

[0122] FIG. 7 illustrates simulation results. In a graph illustrated in FIG. 7, a vertical axis represents EVM. The EVM represents a vector difference at a predetermined time between an ideal transmission signal and the measured reception signal, and the smaller the value, the better performance of the equalization signal processing. Referring to the graph illustrated in FIG. 7, it can be seen that the EVM is improved in cases of Tx CNN, Rx CNN, and both CNN as compared with a case of Linear. In this manner, it is confirmed from the simulation results that it is possible to compensate for the non-linear distortion occurring in the transmitter and/or the receiver.

[0123] Note that, in the above example embodiment, an example in which the equalization unit 154 includes both the intra-receiver non-linear distortion compensation filter 161 and the intra-transmitter non-linear distortion compensation filter 167 has been described. In the above example embodiment, the equalization unit 154 may have only one filter for compensating for the non-linear distortion. For example, it may be known that either non-linear distortion in a transmitter or non-linear distortion in a receiver has a small influence on a reception signal. In such a case, the equalization unit 154 may be configured to include either the intra-receiver non-linear distortion compensation filter 161 or the intra-transmitter non-linear distortion compensation filter 167, such as Tx CNN or Rx CNN described above. Even in this case, by adaptively controlling the coefficients of the filters constituting the filter group by using the output signal of the filter in the final stage of the filter group, it is possible to achieve high-accuracy distortion compensation.

[0124] In the above example embodiment, the equalization unit 154 may be configured as any digital signal processing circuit. For example, the equalization unit 154 may be configured as a circuit including at least one processor 410 and at least one memory 420 as shown in FIG. 13. In this case, the processor 410 included in the equalization unit 154 may read a program stored in the memory 420, and thereby perform the reception side equalization signal processing.

[0125] The above program includes instructions (or software codes) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored in a non-transitory computer readable medium or a tangible storage medium. By way of example, and not a limitation, non-transitory computer readable media or tangible storage media can include a random-access memory (RAM), a read-only memory (ROM), a flash memory, a solid-state drive (SSD) or other types of memory technologies, a Compact Disc (CD), a digital versatile disc (DVD), a Blu-ray disc or other types of optical disc storage, and magnetic cassettes, magnetic tape, magnetic disk storage or other types of magnetic storage devices. The program may be transmitted on a transitory computer readable medium or a communication medium. By way of example, and not a limitation, transitory computer readable media or communication media can include electrical, optical, acoustical, or other forms of propagated signals.

[0126] A communication system, a receiver, an equalization signal processing circuit, a method, and a program according to the present disclosure can perform non-linear distortion compensation in a transmitter and a receiver with high accuracy, in a communication system in which various pieces of distortion including non-linear distortion in the transmitter and the receiver are present.

[0127] The above described embodiments can be combined as desirable by one of ordinary skill in the art.

[0128] While the present disclosure has been particularly shown and described with reference to example embodiments thereof, the present disclosure is not limited to these example embodiments. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the claims.