The present invention is a computationally-efficient compensator for removing nonlinear distortion. The compensator operates in a digital post-compensation configuration for linearization of devices or systems such as analog-to-digital converters and RF receiver electronics. The compensator also operates in a digital pre-compensation configuration for linearization of devices or systems such as digital-to-analog converters, RF power amplifiers, and RF transmitter electronics. The compensator effectively removes nonlinear distortion in these systems in a computationally efficient hardware or software implementation by using one or more factored multi-rate Volterra filters. Volterra filters are efficiently factored into parallel FIR filters and only the filters with energy above a prescribed threshold are actually implemented, which significantly reduces the complexity while still providing accurate results. For extremely wideband applications, the multi-rate Volterra filters are implemented in a demultiplexed polyphase configuration which performs the filtering in parallel at a significantly reduced data rate. The compensator is calibrated with an algorithm that iteratively subtracts an error signal to converge to an effective compensation signal. The algorithm is repeated for a multiplicity of calibration signals, and the results are used with harmonic probing to accurately estimate the Volterra filter kernels. The compensator improves linearization processing performance while significantly reducing the computational complexity compared to a traditional nonlinear compensator.

The technology described in this document can be embodied in a computer-implemented method that includes receiving, at one or more processing devices, a plurality of values representing a set of coefficients of an adaptive filter over a period of time, and identifying, by the one or more processing devices based on the plurality of values, a phase error associated with a transfer function of the adaptive filter. The method also includes adjusting, based on the identified phase error, a phase associated with the transfer function of the adaptive filter such that coefficients calculated using the adjusted transfer function reduce the phase error. The method further includes determining a set of coefficients for the adaptive filter based on the adjusted transfer function, and programming the adaptive filter with the determined set of coefficients to enable operation of the adaptive filter.

The technology described in this document can be embodied in a computer-implemented method that includes receiving, at one or more processing devices, a plurality of values representing a set of coefficients of an adaptive filter over a period of time, and identifying, by the one or more processing devices based on the plurality of values, a phase error associated with a transfer function of the adaptive filter. The method also includes adjusting, based on the identified phase error, a phase associated with the transfer function of the adaptive filter such that coefficients calculated using the adjusted transfer function reduce the phase error. The method further includes determining a set of coefficients for the adaptive filter based on the adjusted transfer function, and programming the adaptive filter with the determined set of coefficients to enable operation of the adaptive filter.

A digital signal processing (DSP)-based serializer-deserializer (SERDES) includes a first filter configured to mitigate inter-symbol interference (ISI) attributed to dispersion associated with a long-reach transmission medium. The SERDES includes a second filter configured to shape the ISI. The SERDES includes also includes a third filter coupled in parallel with the second filter and configured to reduce ISI attributed to reflections associated to both near-zero delays and long delays.

A digital signal processing (DSP)-based serializer-deserializer (SERDES) includes a first filter configured to mitigate inter-symbol interference (ISI) attributed to dispersion associated with a long-reach transmission medium. The SERDES includes a second filter configured to shape the ISI. The SERDES includes also includes a third filter coupled in parallel with the second filter and configured to reduce ISI attributed to reflections associated to both near-zero delays and long delays.

A digital signal processing (DSP)-based serializer-deserializer (SERDES) includes a first filter configured to mitigate inter-symbol interference (ISI) attributed to dispersion associated with a long-reach transmission medium. The SERDES includes a second filter configured to shape the ISI. The SERDES includes also includes a third filter coupled in parallel with the second filter and configured to reduce ISI attributed to reflections associated to both near-zero delays and long delays.