Method and apparatus for adaptive signal processing

Abstract

A method for adaptive signal processing is provided. In the method, a second vector is obtained by initializing a first vector without regularization of a cost function. The cost function is regularized with the first vector and the second vector as variables. The first vector is updated based on an input signal, according to the regularized cost function. Then, an output signal is provided based on the updated first vector. The second vector is updated based on the update of the first vector. An apparatus for adaptive signal processing is provided accordingly. The method and the apparatus are well compatible with existing adaptive signal processing. The convergence coefficients of the adaptive filter system become more stable. Moreover, impact of an extra penalty added to the cost function on a bias can be minimized, and the increased complexity of the system is very limited.

Claims

1. A method for adaptive signal processing, comprising: obtaining a second vector by initializing a first vector without regularization of a cost function; regularizing the cost function with the first vector and the second vector as variables; updating the first vector based on an input signal, according to the regularized cost function; providing an output signal based on the updated first vector; and updating the second vector based on the update of the first vector.

2. The method of claim 1, wherein the obtaining of the second vector further comprises: converging the first vector based on an input signal; and configuring the second vector as a value of the first vector when convergence of the first vector fulfills a predetermined criterion.

3. The method of claim 1, wherein the updating of the first vector further comprises: taking a gradient of the regularized cost function; and configuring the first vector based on the gradient.

4. The method of claim 1, wherein the second vector is updated every predetermined number of iterations of updating the first vector.

5. The method of claim 1, wherein the second vector is updated using a soft update.

6. The method of any of claim 1, wherein the method is performed in an adaptive filter.

7. An apparatus for adaptive signal processing, comprising: a processor; and a memory communicatively coupled to the processor and adapted to store instructions which, when executed by the processor, cause the apparatus to perform operations of: obtaining a second vector by initializing a first vector without regularization of a cost function; regularizing the cost function with the first vector and the second vector as variables; updating the first vector based on an input signal, according to the regularized cost function; providing an output signal based on the updated first vector; and updating the second vector based on the update of the first vector.

8. The apparatus of claim 7, wherein the instructions, when executed by the processor, cause the apparatus to initialize the first vector by: converging the first vector based on an input signal; and configuring the second vector as a value of the first vector when convergence of the first vector fulfills a predetermined criterion.

9. The apparatus of claim 7, wherein the instructions, when executed by the processor, cause the apparatus to update the first vector by: taking a gradient of the regularized cost function; and configuring the first vector based on the gradient.

10. The apparatus of claim 7, wherein the second vector is updated every predetermined number of iterations of updating the first vector.

11. The apparatus of claim 7, wherein the second vector is updated using a soft update.

12. The apparatus of claim 7, wherein the apparatus is implemented in an adaptive filter.

13. A non-transitory computer readable medium having a computer program stored thereon which, when executed by a set of one or more processors of an apparatus, causes the apparatus to perform: obtaining a second vector by initializing a first vector without regularization of a cost function; regularizing the cost function with the first vector and the second vector as variables; updating the first vector based on an input signal, according to the regularized cost function; providing an output signal based on the updated first vector; and updating the second vector based on the update of the first vector.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The present disclosure may be best understood by way of example with reference to the following description and accompanying drawings that are used to illustrate embodiments of the present disclosure. In the drawings:

(2) FIG. 1 is a block diagram illustrating a system block of the adaptive filter;

(3) FIG. 2 is a graph illustrating a bias between the LMS approach with the custom character .sub.2 regularization and the standard LMS approach;

(4) FIG. 3 is a flow chart illustrating exemplary processes for adaptive signal processing according to some embodiments of the present disclosure;

(5) FIG. 4 is a flow chart illustrating a method for adaptive signal processing according to some embodiments of the present disclosure;

(6) FIG. 5 is a more specific flow chart illustrating a method for adaptive signal processing according to some embodiments of the present disclosure;

(7) FIG. 6 is a block diagram illustrating an apparatus for adaptive signal processing according to some embodiments of the present disclosure;

(8) FIG. 7 is another block diagram illustrating an apparatus for adaptive signal processing according to some embodiments of the present disclosure;

(9) FIG. 8 is a graph illustrating a performance comparison among the LMS approach with the regularization according to the present disclosure, the LMS approach with the custom character .sub.2 regularization and the standard LMS approach in the unstable case; and

(10) FIG. 9 is a graph illustrating a performance comparison among the LMS approach with the regularization according to the present disclosure, the LMS approach with the custom character .sub.2 regularization and the standard LMS approach in the convergence case.

DETAILED DESCRIPTION

(11) The following detailed description describes a method and apparatus for adaptive signal processing. In the following detailed description, numerous specific details such as logic implementations, types and interrelationships of system components, etc. are set forth in order to provide a more thorough understanding of the present disclosure. It should be appreciated, however, by one skilled in the art that the present disclosure may be practiced without such specific details. In other instances, control structures, circuits and instruction sequences have not been shown in detail in order not to obscure the present disclosure. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation.

(12) References in the specification to one embodiment, an embodiment, an example embodiment etc. indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

(13) Bracketed texts and blocks with dashed borders (e.g., large dashes, small dashes, dot-dash, and dots) may be used herein to illustrate optional operations that add additional features to embodiments of the present disclosure. However, such notation should not be taken to mean that these are the only options or optional operations, and/or that blocks with solid borders are not optional in certain embodiments of the present disclosure.

(14) In the following detailed description and claims, the terms coupled and connected, along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. Coupled is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, cooperate or interact with each other. Connected is used to indicate the establishment of communication between two or more elements that are coupled with each other.

(15) An electronic device stores and transmits (internally and/or with other electronic devices) code (which is composed of software instructions and which is sometimes referred to as computer program code or a computer program) and/or data using machine-readable media (also called computer-readable media), such as machine-readable storage media (e.g., magnetic disks, optical disks, read only memory (ROM), flash memory devices, phase change memory) and machine-readable transmission media (also called a carrier) (e.g., electrical, optical, radio, acoustical or other forms of propagated signalssuch as carrier waves, infrared signals). Thus, an electronic device (e.g., a computer) includes hardware and software, such as a set of one or more processors coupled to one or more machine-readable storage media to store code for execution on the set of processors and/or to store data. For instance, an electronic device may include non-volatile memory containing the code since the non-volatile memory can persist code/data even when the electronic device is turned off (when power is removed), and while the electronic device is turned on, that part of the code that is to be executed by the processor(s) of that electronic device is typically copied from the slower non-volatile memory into volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)) of that electronic device. Typical electronic devices also include a set of one or more physical interfaces to establish connections (to transmit and/or receive code and/or data using propagating signals) with other electronic devices. One or more parts of an embodiment of the present disclosure may be implemented using different combinations of software, firmware, and/or hardware.

(16) In the adaptive signal processing, the custom character .sub.1 regularization and .sub.2 regularization as described above both constrain the model parameter which lies in a region close to zero. More generally, the model parameter may be regularized to be near any specific point in space, e.g., .sub.0. If the parameter may be regularized to be closer to the true one, the result will be better. In fact, when we do not know if the correct value should be positive or negative, zero may be a default value that is practicable in many applications.

(17) In order to decrease the bias which is introduced by the regularization term, the custom character .sub.2 regularization term () in the cost function may be modified as:
()=.sub.0.sub.2.sup.2 (1).

(18) In the case that a sparse solution is preferred, the custom character .sub.1 regularization term () in the cost function may be used and modified as:
()=.sub.0.sub.1 (2).

(19) If .sub.0 is the true value of , the bias will be:
bias({tilde over (J)}(;n))=E(())=0 (3).

(20) Then, .sub.0 may be initialized and updated to keep .sub.0 not far from the true value of , which will be described in more detail with reference to FIG. 3 below.

(21) FIG. 3 is a flow chart illustrating exemplary processes of an adaptive signal processing approach 300 according to some embodiments of the present disclosure.

(22) At block 301, the parameter may be initialized in the case that the cost function is not regularized, i.e., without any regularization term, to converge the parameter . As an example, the LMS approach may be used herein to produce a least mean square of the error signal e(n) which is a difference between the desired signal d(n) and the actual signal y(n). Typical approaches include at least the standard LMS approach and the normalized least mean square (NLMS) approach.

(23) The cost function J in the standard LMS and NLMS approaches may be:
J(n)=e(n).sub.2.sup.2 (4)
where .Math..sub.2 is the custom character .sub.2 vector norm operator.

(24) At block 302, an iteration of the adaptive signal processing, e.g., the LMS approach, may be executed on , and an input signal x(k) may be received in the k.sup.th iteration, where k=0, 1, 2 . . . , e.g., received by an adaptive filter as an input. The coefficient (k) in the k.sup.th iteration may converge.

(25) At decision block 303, it is determined whether a predetermined convergence criterion is fulfilled. If not, the process continues with the iteration. If so, .sub.0 may be initialized as a value of the converged (k) and the process may proceed to block 304.

(26) At block 304, the current .sub.0 may be used in the regularization so that the cost function may be regularized as:
{tilde over (J)}(;n)=J(n)+.sub.0.sub.2.sup.2 (5).

(27) may then be updated based on the regularized cost function. At block 305, another iteration of the adaptive signal processing may be executed on , and an input signal x(m) may be received in the m.sup.th iteration, where m=0, 1, 2 . . . , e.g., received by the adaptive filter also as an input.

(28) As an example, a stochastic gradient descent approach may be used in which is utilized as a gradient operator. A gradient of the regularized cost function is taken as:
{tilde over (J)}=(.sub.0)+J(6).

(29) As an example, using the LMS approach, the parameter may be updated in a single gradient step as:
(1)+.sub.0J(7)
where is a step size coefficient.

(30) As a further example, if the NLMS approach is preferred, the update may be performed as:

(31) $\begin{matrix} (1 -) +_{0} - \frac{}{{.Math. x .Math.}_{2}^{2}} J & (8) \end{matrix}$
where x is the input signal.

(32) As a still further example, if a sparse solution is preferred, the custom character .sub.1 regularization may be used to update as:

(33) for the LMS,
sgn(.sub.0)J(9)

(34) and

(35) for the NLMS,

(36) $\begin{matrix} - sgn (-_{0}) - \frac{}{{.Math. x .Math.}_{2}^{2}} J & (10) \end{matrix}$
where
sgn(z)=[sgn(z.sub.1) sgn(z.sub.2) . . . sgn(z.sub.M)]

(37) for

(38) $z = [\begin{matrix} z_{1} & z_{2} & .Math. & z_{M} \end{matrix}] and sgn (z_{i}) = {\begin{matrix} 1, & z_{i} > 0 \\ 0, & z_{i} = 0 \\ - 1, & z_{i} < 0 \end{matrix} .$

(39) In order for the parameter not to dramatically oscillate or diverge, the parameter .sub.0 may be updated. As an example, the update of .sub.0 may include two scenarios. In the first scenario, .sub.0 may be updated every T iterations of updates of , wherein T is a predetermined number. For instance, at decision block 306, it is determined whether the iteration number m of the update of is a multiple of T. If so, a value of (m) may be assigned to .sub.0 at block 307, and then the process may proceed back to block 304 with new values of and .sub.0 being substituted into Equation (5). If not, the process may directly proceed to block 304 with only the updated . In the second scenario, a soft update may be performed on .sub.0 at block 308, e.g., .sub.0(1).sub.0+, with 0<<<1. This means that .sub.0 is constrained to change slowly, greatly improving the stability of . The process may then proceed back to block 304 with the new and .sub.0.

(40) Then, the currently updated and .sub.0 are used in the regularization again in the block 304 to go through the next iteration.

(41) At block 309, a signal y(m) for the m.sup.th iteration may be output in real time, e.g., an output signal of the adaptive filter.

(42) FIG. 4 is a flow chart illustrating a method 400 for adaptive signal processing according to some embodiments of the present disclosure. The method 400 may be performed in an adaptive filter by way of example only for convenience of description but it is not limited thereto. As an example, operations in this and other flow charts will be described with reference to the exemplary embodiments of the other figures. However, it should be appreciated that the operations of the flow charts may be performed by embodiments of the present disclosure other than those discussed with reference to the other figures, and the embodiments of the present disclosure discussed with reference to these other figures may perform operations different than those discussed with reference to the flow charts.

(43) In one embodiment, the method 400 begins with obtaining a second vector (e.g., .sub.0 as depicted in FIG. 3) by initializing a first vector (e.g., as depicted in FIG. 3) without regularizing a cost function (e.g., J as depicted in FIG. 3) (block 401). The adaptive filter may regularize the cost function using the first vector and the second vector as variables (block 402). As an example, if the convergence of the first vector meets a predetermined criterion, the cost function may be regularized based on the first vector and the second vector, e.g., according to Equation (5) described above.

(44) The adaptive filter may receive an input signal and apply the regularized cost function to the input signal to update the first vector (block 403). The adaptive filter may provide an output signal at least based on the updated first vector (block 404), and update the second vector based on the update of the first vector (block 405).

(45) FIG. 5 is a more specific flow chart illustrating a method 500 for adaptive signal processing according to some embodiments of the present disclosure. The method 500 may be performed in an adaptive filter by way of example only for convenience of description, but it is not limited thereto.

(46) In one embodiment, the adaptive filter may converge a first vector (e.g., as depicted in FIG. 3) based on an input signal (e.g., x(k) as depicted in FIG. 3) without regularization of a cost function (block 501). As an example, the k.sup.th iteration of the LMS approach may be executed on the first vector. The adaptive filter may assign a value of the first vector to a second vector (e.g., .sub.0 as depicted in FIG. 3) when the convergence of the first vector meets a predetermined convergence criterion (block 502).

(47) The adaptive filter may regularize the cost function with the first vector and the second vector as variables (block 503), e.g., in accordance with Equation (5).

(48) As an example, the adaptive filter may take a gradient of the regularized cost function (block 504) and configure the first vector at least based on the gradient (block 505) to update the first vector. As a further example, the first vector may be updated based on the gradient and the current second vector, e.g., in accordance with Equations (6)-(10) described above.

(49) The adaptive filter may provide an output signal at least based on the currently updated first vector (block 506). As an example, during the m.sup.th iteration of the adaptive filter processing on the first vector, the signal y(m) may be output by the adaptive filter.

(50) As an example, the adaptive filter may update the second vector every T iterations of the updates of the first vector (block 507), e.g., configure the second vector as a value of the currently updated first vector after the 0.sup.th, T.sup.th, 2T.sup.th, 3T.sup.th . . . iteration for the first vector. As another example, the adaptive filter may update the second vector by means of a soft update (block 508), e.g., as described in the block 308 of FIG. 3. The updated first vector and the updated second vector may then act as the variables for the regularized cost function as described above in the block 503.

(51) FIG. 6 is a block diagram illustrating an apparatus 600 for adaptive signal processing according to some embodiments of the present disclosure. As an example, the apparatus 600 may be implemented in an adaptive filter, but it is not limited thereto. It should be appreciated that the apparatus 600 may be implemented using components other than those illustrated in FIG. 6.

(52) With reference to FIG. 6, the apparatus 600 may comprise at least a processor 601, a memory 602, an interface 603 and a communication medium 604. The processor 601, the memory 602 and the interface 603 may be communicatively coupled to each other via the communication medium 604.

(53) The processor 601 may include one or more processing units. A processing unit may be a physical device or article of manufacture comprising one or more integrated circuits that read data and instructions from computer readable media, such as the memory 602, and selectively execute the instructions. In various embodiments, the processor 601 may be implemented in various ways. As an example, the processor 601 may be implemented as one or more processing cores. As another example, the processor 601 may comprise one or more separate microprocessors. In yet another example, the processor 601 may comprise an application-specific integrated circuit (ASIC) that provides specific functionality. In still another example, the processor 601 may provide specific functionality by using an ASIC and/or by executing computer-executable instructions.

(54) The memory 602 may include one or more computer-usable or computer-readable storage medium capable of storing data and/or computer-executable instructions. It should be appreciated that the storage medium is preferably a non-transitory storage medium.

(55) The interface 603 may be a device or article of manufacture that enables the apparatus 600 to send data to or receive data from external devices.

(56) The communication medium 604 may facilitate communication among the processor 601, the memory 602 and the interface 603. The communication medium 604 may be implemented in various ways. For example, the communication medium 604 may comprise a Peripheral Component Interconnect (PCI) bus, a PCI Express bus, an accelerated graphics port (AGP) bus, a serial Advanced Technology Attachment (ATA) interconnect, a parallel ATA interconnect, a Fiber Channel interconnect, a USB bus, a Small Computing System Interface (SCSI) interface, or another type of communications medium.

(57) In the example of FIG. 6, the instructions stored in the memory 602 may include those that, when executed by the processor 601, cause the apparatus 600 to implement the method described with respect to FIG. 4 or 5.

(58) FIG. 7 is another block diagram illustrating an apparatus 700 for adaptive signal processing according to some embodiments of the present disclosure. As an example, the apparatus 700 may be implemented in an adaptive filter, but it is not limited thereto. It should be appreciated that the apparatus 700 may be implemented using components other than those illustrated in FIG. 7.

(59) With reference to FIG. 7, the apparatus 700 may comprise at least an initialization unit 701, a regularization unit 702, an updating unit 703 and a signal output unit 704.

(60) The initialization unit 701 may be adapted to perform at least the operation described in the block 401 of FIG. 4. As an example, the initialization unit 701 may further comprise a convergence unit 705 and a second vector configuration unit 706. The convergence unit 705 may be adapted to perform at least the operation described in the block 501 of FIG. 5. The second vector configuration unit 706 may be adapted to perform at least the operation described in the block 502 of FIG. 5.

(61) The regularization unit 702 may be adapted to perform at least the operations described in the block 402 of FIG. 4 and in the block 503 of FIG. 5.

(62) The updating unit 703 may be adapted to perform at least the operations described in the blocks 403 and 405 of FIG. 4 and in the blocks 507 and 508 of FIG. 5. As an example, the updating unit 703 may further comprise a gradient unit 707 and a first vector configuration unit 708. The gradient unit 707 may be adapted to perform at least the operation described in the block 504 of FIG. 5. The first vector configuration unit 708 may be adapted to perform at least the operation described in the block 505 of FIG. 5.

(63) The signal output unit 704 may be adapted to perform at least the operations described in the block 404 of FIG. 4 and in the block 506 of FIG. 5.

(64) Some units are illustrated as separate units in FIG. 7. However, this is merely to indicate that the functionality is separated. The units may be provided as separate elements. However, other arrangements are possible, e.g., some of them may be combined as one unit. Any combination of the units may be implemented in any combination of software, hardware, and/or firmware in any suitable location. For example, there may be more controllers configured separately, or just one controller for all of the components.

(65) The units shown in FIG. 7 may constitute machine-executable instructions embodied within e.g. a machine readable medium, which when executed by a machine will cause the machine to perform the operations described. Besides, any of these units may be implemented as hardware, such as an application specific integrated circuit (ASIC), Digital Signal Processor (DSP), Field Programmable Gate Array (FPGA) or the like.

(66) Moreover, it should be appreciated that the arrangements described herein are set forth only as examples. Other arrangements (e.g., more controllers or more detectors, etc.) may be used in addition to or instead of those shown, and some units may be omitted altogether. Functionality and cooperation of these units are correspondingly described in more detail with reference to FIGS. 4 and 5.

(67) FIG. 8 is a graph illustrating a performance comparison among the LMS approach with the regularization according to the present disclosure, the LMS approach with the custom character .sub.2 regularization and the standard LMS approach in the unstable case in which the coefficients of the adaptive filter do not converge stably. FIG. 9 is a graph illustrating a performance comparison among the LMS approach with the regularization according to the present disclosure, the LMS approach with the custom character .sub.2 regularization and the standard LMS approach in the convergence case in which the convergence coefficients of the adaptive filter are stable. The adaptive signal processing of the DPD system uses the adaptive filter to mimic a desired filter by dynamically estimating the filter coefficients. Normalized mean square errors (NMSEs) are plotted against iteration numbers in FIGS. 8 and 9. As can be seen from FIG. 8, both the LMS approach according to the present disclosure and the LMS approach with the existing custom character .sub.2 regularization outperform the standard LMS approach since adding the regularizing term stabilizes the adaptive filter. As shown in FIG. 9, both the LMS approach according to the present disclosure and the standard LMS approach outperform the LMS approach with the existing .sub.2 regularization due to a distinct bias introduced by the prior art regularization term. Therefore, the LMS approach according to the present disclosure is robust for linearization as compared to the other approaches.

(68) Some portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of transactions on data bits within a computer memory. These algorithmic descriptions and representations are ways used by those skilled in the signal processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of transactions leading to a desired result. The transactions are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

(69) It should be appreciated, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the above discussion, it is appreciated that throughout the description, discussions utilizing terms such as processing or computing or calculating or determining or displaying or the like, refer to actions and processes of a computer system, or a similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

(70) The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method transactions. The required structure for a variety of these systems will appear from the description above. In addition, embodiments of the present disclosure are not described with reference to any particular programming language. It should be appreciated that a variety of programming languages may be used to implement the teachings of embodiments of the present disclosure as described herein.

(71) An embodiment of the present disclosure may be an article of manufacture in which a non-transitory machine-readable medium (such as microelectronic memory) has stored thereon instructions (e.g., computer code) which program one or more signal processing components (generically referred to here as a processor) to perform the operations described above. In other embodiments, some of these operations might be performed by specific hardware components that contain hardwired logic (e.g., dedicated digital filter blocks and state machines). Those operations might alternatively be performed by any combination of programmed signal processing components and fixed hardwired circuit components.

(72) In the foregoing detailed description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments thereof. It will be evident that various modifications may be made thereto without departing from the spirit and scope of the present disclosure as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.

(73) Throughout the description, some embodiments of the present disclosure have been presented through flow diagrams. It should be appreciated that the order of transactions and transactions described in these flow diagrams are only intended for illustrative purposes and not intended as a limitation of the present disclosure. One having ordinary skill in the art would recognize that variations can be made to the flow diagrams without departing from the spirit and scope of the present disclosure as set forth in the following claims.

Method and apparatus for adaptive signal processing

Assignee

Inventors

Cpc classification

Classification Explorer

G06N3/08

PHYSICS

Classification Explorer

H03H21/0067

ELECTRICITY

Classification Explorer

H03H2021/0045

ELECTRICITY

Classification Explorer

H03H2021/0056

ELECTRICITY

Classification Explorer

H03H21/0043

ELECTRICITY

Classification Explorer

H03H2021/0081

ELECTRICITY

International classification

Classification Explorer

H03H7/30

ELECTRICITY

Classification Explorer

H03H21/00

ELECTRICITY

Abstract

Claims

Description