Methods and systems for improved signal decomposition

Abstract

A method for improving decomposition of digital signals using training sequences is presented. A method for improving decomposition of digital signals using initialization is also provided. A method for sorting digital signals using frames based upon energy content in the frame is further presented. A method for utilizing user input for combining parts of a decomposed signal is also presented.

Claims

1. An apparatus capable of creating an initial set of values in a row of a weight matrix in non-negative matrix factorization to decompose digital signals, the apparatus capable of: generating the initial set of values of the row of the weight matrix from a ratio of a first function of a first signal of a plurality of digital audio signals divided by a second function of at least two other signals of the plurality of the digital audio signals, wherein the row in the weight matrix determines a decomposition of the plurality of digital audio signals into signal components; and audibly outputting a portion of one or more of the components of the decomposed plurality of digital audio signals at least based on input from an interface element.

2. The apparatus of claim 1, wherein the digital audio signals are one or more of binaural or multichannel audio signals.

3. The apparatus of claim 1, wherein the first and second functions are calculated from one or more filtered versions of said digital audio signals.

4. The apparatus of claim 1, wherein the first and second functions are calculated in one or more of the time domain, the frequency domain, theme-frequency domain.

5. The apparatus of claim 1, wherein the first and second functions are calculated using one or more of energy, power, root mean square, geometric mean, arithmetic mean, euclidean norm, taxicab norm, or Lp norm.

6. The apparatus of claim 1, wherein the digital audio signals are one or more of binaural or multichannel audio signals and the output portion of the one or more of the components are used for one or more of: source separation, signal restoration, signal enhancement, noise removal, un-mixing, up-mixing and re-mixing.

7. The apparatus of claim 1, wherein the output portion of the one or more of the components are used for one or more of: source separation, signal restoration, signal enhancement, noise removal, un-mixing, up-mixing and re-mixing.

8. The apparatus of claim 1, wherein the plurality of digital signals are a single source coming from an original mixture of multiple sources.

9. The apparatus of claim 1, further comprising automatically sorting one or more of the components.

10. A non-transitory computer readable information storage media having stored therein instructions, that when executed by one or more processors, cause to be performed a method of creating an initial set of values in a row of a weight matrix in non-negative matrix factorization to decompose digital signals, the method comprising: generating the initial set of values of the row of the weight matrix from a ratio of a first function of a first signal of a plurality of digital audio signals divided by a second function of at least two other signals of the plurality of the digital audio signals, wherein the row in the weight matrix determines a decomposition of the plurality of digital audio signals into signal components; and audibly outputting a portion of one or more of the components of the decomposed plurality of digital audio signals at least based on input from an interface element.

11. The media of claim 10, wherein the digital audio signals are one or more of binaural or multichannel audio signals.

12. The media of claim 10, wherein the first and second functions are calculated from one or more filtered versions of said digital audio signals.

13. The media of claim 10, wherein the first and second functions are calculated in one or more of the time domain, the frequency domain, theme-frequency domain.

14. The media of claim 10, wherein the first and second functions are calculated using one or more of energy, power, root mean square, geometric mean, arithmetic mean, euclidean norm, taxicab norm, or Lp norm.

15. The media of claim 10, wherein the digital audio signals are one or more of binaural or multichannel audio signals and the output portion of the one or more of the components are used for one or more of: source separation, signal restoration, signal enhancement, noise removal, un-mixing, up-mixing and re-mixing.

16. The media of claim 10, wherein the output portion of the one or more of the components are used for one or more of: source separation, signal restoration, signal enhancement, noise removal, un-mixing, up-mixing and re-mixing.

17. The media of claim 10, wherein the plurality of digital signals are a single source coming from an original mixture of multiple sources.

18. The media of claim 10, further comprising automatically sorting one or more of the components.

19. An apparatus capable of creating an initial set of values in a row of a weight matrix in non-negative matrix factorization to decompose digital signals, the apparatus comprising: means for generating the initial set of values of the row of the weight matrix from a ratio of a first function of a first signal of a plurality of digital audio signals divided by a second function of at least two other signals of the plurality of the digital audio signals, wherein the row in the weight matrix determines a decomposition of the plurality of digital audio signals into signal components; and means for audibly outputting a portion of one or more of the components of the decomposed plurality of digital audio signals at least based on input from an interface element.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) For a more complete understanding of the invention, reference is made to the following description and accompanying drawings, in which:

(2) FIG. 1 illustrates an exemplary schematic representation of a processing method based on decomposition;

(3) FIG. 2 illustrates an exemplary schematic representation of the creation of an extended spectrogram using a training sequence, in accordance with embodiments of the present invention;

(4) FIG. 3 illustrates an example of a source signal along with a function that is derived from an energy ratio, in accordance with embodiments of the present invention;

(5) FIG. 4 illustrates an exemplary schematic representation of a set of source signals and a resulting initialization matrix in accordance with embodiments of the present invention;

(6) FIG. 5 illustrates an exemplary schematic representation of a block diagram showing a NMF decomposition method, in accordance with embodiments of the present invention; and

(7) FIG. 6 illustrates an exemplary schematic representation of a user interface in accordance with embodiments of the present invention.

DETAILED DESCRIPTION

(8) Hereinafter, embodiments of the present invention will be described in detail in accordance with the references to the accompanying drawings. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present application.

(9) The exemplary systems and methods of this invention will sometimes be described in relation to audio systems. However, to avoid unnecessarily obscuring the present invention, the following description omits well-known structures and devices that may be shown in block diagram form or otherwise summarized.

(10) For purposes of explanation, numerous details are set forth in order to provide a thorough understanding of the present invention. It should be appreciated however that the present invention may be practiced in a variety of ways beyond the specific details set forth herein. The terms determine, calculate and compute, and variations thereof, as used herein are used interchangeably and include any type of methodology, process, mathematical operation or technique.

(11) FIG. 1 illustrates an exemplary case of how a decomposition method can be used to apply any type of processing. A source signal 101 is decomposed in signal parts or components 102, 103 and 104. Said components are sorted 105, either automatically or manually from a human user. Therefore the original components are rearranged 106, 107, 108 according to the sorting process. Then a combination of some or all of these components forms any desired output 109. When for example said combination of components forms a single source coming from an original mixture of multiple sources, said procedure refers to a source separation technique. When for example residual components represent a form of noise, said procedure refers to a denoise technique. All embodiments of the present application may refer to a general decomposition procedure, including but not limited to non-negative matrix factorization, independent component analysis, principal component analysis, singular value decomposition, dependent component analysis, low-complexity coding and decoding, stationary subspace analysis, common spatial pattern, empirical mode decomposition, tensor decomposition, canonical polyadic decomposition, higher-order singular value decomposition, tucker decomposition, etc.

(12) In an exemplary embodiment, a non-negative matrix factorization algorithm can be used to perform decomposition, such as the one described in FIG. 1. Consider a source signal x.sub.m(k), which can be any input signal and k is the sample index. In a particular embodiment, a source signal can be a mixture signal that consists of N simultaneously active signals s.sub.n(k). In particular embodiments, a source signal may always be considered a mixture of signals, either consisting of the intrinsic parts of the source signal or the source signal itself and random noise signals or any other combination thereof. In general, a source signal is considered herein as an instance of the source signal itself or one or more of the intrinsic parts of the source signal or a mixture of signals.

(13) In an exemplary embodiment, the intrinsic parts of an image signal representing a human face could be the images of the eyes, the nose, the mouth, the ears, the hair etc. In another exemplary embodiment, the intrinsic parts of a drum snare sound signal could be the onset, the steady state and the tail of the sound. In another embodiment, the intrinsic parts of a drum snare sound signal could be the sound coming from each one of the drum parts, i.e. the hoop/rim, the drum head, the snare strainer, the shell etc. In general, intrinsic parts of a signal are not uniquely defined and depend on the specific application and can be used to represent any signal part.

(14) Given the source signal x.sub.m(k), any available transform can be used in order to produce the non-negative matrix V.sub.m. from the source signal. When for example the source signal is non-negative and two-dimensional, V.sub.m can be the source signal itself. When for example the source signal is in the time domain, the non-negative matrix V.sub.m can be derived through transformation in the time-frequency domain using any relevant technique including but not limited to a short-time Fourier transform (STFT), a wavelet transform, a polyphase filterbank, a multi rate filterbank, a quadrature mirror filterbank, a warped filterbank, an auditory-inspired filterbank, etc.

(15) A non-negative matrix factorization algorithm typically consists of a set of update rules derived by minimizing a distance measure between V.sub.m and W.sub.mH.sub.m, which is sometimes formulated utilizing some underlying assumptions or modeling of the source signal. Such an algorithm may produce upon convergence a matrix product that approximates the original matrix V.sub.m as in equation (1).

(16) $\begin{array}{cc}{V}_m\approx {\hat{V}}_m-W_m{H}_m& \left(1\right)\end{array}$

(17) The matrix W.sub.m has size F×K and the matrix H.sub.m has size K×T, where K is the rank of the approximation (or the number of components) and typically K<<FT. Each component may correspond to any kind of signal including but not limited to a source signal, a combination of source signals, a part of a source signal, a residual signal. After estimating the matrices W.sub.m and H.sub.m, each F×1 column w.sub.j,m of the matrix W.sub.m, can be combined with a corresponding 1×T row h.sub.j,m.sup.T of matrix H.sub.m and thus a component mask A.sub.j,m can be obtained

(18) $\begin{array}{cc}{A}_{j,m}=w_{j,m}{h}_{j,m}^T& \left(2\right)\end{array}$

(19) When applied to the original matrix V.sub.m, this mask may produce a component signal z.sub.j,m(k) that corresponds to parts or combinations of signals present in the source signal. There are many ways of applying the mask A.sub.j,m and they are all in the scope of the present invention. In a particular embodiment, the real-valued mask A.sub.j,m could be directly applied to the complex-valued matrix X.sub.m, that may contain the time-frequency transformation of x.sub.m(k) as in (3).

(20) $\begin{array}{cc}{Z}_{j,m}=A_{j,\tau n}\circ X_m& \left(3\right)\end{array}$
where ∘ is the Hadamart product. In this embodiment, applying an inverse time-frequency transform on Z.sub.j,m produces the component signals z.sub.j,m(k).

(21) In many applications, multiple source signals are present (i.e. multiple signals x.sub.m(k) with m=1, 2, . . . M) and therefore multichannel information is available. In order to exploit such multichannel information, non-negative tensor factorization (NTF) methods can be also applied (see Section 1.5 in A. Cichocki, R. Zdunek, A. H. Phan, S.-I. Amari, “Nonnegative Matrix and Tensor Factorization: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation”, John Wiley & Sons, 2009). Alternatively, appropriate tensor unfolding methods (see Section 1.4.3 in A. Cichocki, R. Zdunek, A. H. Phan, S.-I. Amari, “Nonnegative Matrix and Tensor Factorization: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation”, John Wiley & Sons, 2009) will transform the multichannel tensors to a matrix and enable the use of NMF methods. All of the above decomposition methods are in the scope of the present invention. In order to ensure the convergence of NMF to a meaningful factorization that can provide useful component signals, a number of training techniques have been proposed. In the context of NMF, training typically consists of estimating the values of matrix W.sub.m, and it is sometimes referred to as supervised or semi-supervised NMF.

(22) In an exemplary embodiment of the present application, a training scheme is applied based on the concept of training sequences. A training sequence ŝ.sub.m(k) is herein defined as a signal that is related to one or more of the source signals (including their intrinsic parts). For example, a training sequence can consist of a sequence of model signals s′.sub.i,m(k). A model signal may be any signal and a training sequence may consist of one or more model signals. In some embodiments, a model signal can be an instance of one or more of the source signals (such signals may be captured in isolation), a signal that is similar to an instance of one or more of source signals, any combination of signals similar to an instance of one or more of the source signals, etc. In the preceding, a source signal is considered the source signal itself or one or more of the intrinsic parts of the source signal. In specific embodiments, a training sequence contains model signals that approximate in some way the signal that we wish to extract from the source signal under processing. In particular embodiments, a model signal may be convolved with shaping filters g.sub.i(k) which may be designed to change and control the overall amplitude, amplitude envelope and spectral shape of the model signal or any combination of mathematical or physical properties of the model signal. The model signals may have a length of L.sub.t samples and there may be R model signals in a training sequence, making the length of the total training sequence equal to L.sub.tR. In particular embodiments, the training sequence can be described as in equation (4):

(23) $\begin{array}{cc}{\hat{s}}_m\left(k\right)=\underset{r=0}{\overset{R-1}{.Math.}}\left[g_1\left(k\right)*s_{1,m}^{\prime }\left(k\right)\right]B\left(k;{\mathrm{iL}}_t,{\mathrm{iL}}_t+L_t-1\right)& \left(4\right)\end{array}$
where B(x; a, b) is the boxcar function given by:

(24) $\begin{array}{cc}B\left(x;a_1,b\right)-\{\begin{array}{cc}0& \mathrm{if}x<a\mathrm{and}x>b\\ 1& \mathrm{if}a\le x\le b\end{array}& \left(5\right)\end{array}$

(25) In an exemplary embodiment, a new non-negative matrix Ŝ.sub.m is created from the signal ŝ.sub.m(k) by applying the same time-frequency transformation as for x.sub.m(k) and is appended to V.sub.m as

(26) $\begin{array}{cc}{\stackrel{\_}{V}}_m=\left[{\hat{S}}_m.Math.V_m.Math.{\hat{S}}_m\right]& \left(6\right)\end{array}$

(27) In specific embodiments, a matrix Ŝ.sub.m can be appended only on the left side or only on the right side or on both sides of the original matrix V.sub.m, as shown in equation 6. This illustrates that the training sequence is combined with the source signal. In other embodiments, the matrix V.sub.m can be split in any number of sub-matrices and these sub-matrices can be combined with any number of matrices Ŝ.sub.m, forming an extended matrix V.sub.m. After this training step, any decomposition method of choice can be applied to the extended matrix V.sub.m. If multiple source signals are processed simultaneously in a NTF or tensor unfolded NMF scheme, the training sequences for each source signal may or may not overlap in time. In other embodiments, when for some signals a training sequence is not formulated, the matrix V.sub.m may be appended with zeros or a low amplitude noise signal with a predefined constant or any random signal or any other signal. Note that embodiments of the present application are relevant for any number of source signals and any number of desired output signals.

(28) An example illustration of a training sequence is presented in FIG. 2. In this example, a training sequence ŝ.sub.m(k) 201 is created and transformed to the time-frequency domain through a short-time Fourier transform to create a spectrogram Ŝ.sub.m 202. Then, the spectrogram of the training sequence Ŝ.sub.m is appended to the beginning of an original spectrogram V.sub.m 203, in order to create an extended spectrogram V.sub.m 204. The extended spectrogram 204 can be used in order to perform decomposition (for example NMF), instead of the original spectrogram 203.

(29) Another aspect that is typically overlooked in decomposition methods is the initialization of the weight matrix H.sub.m. Typically this matrix can be initialized to random, non-negative values. However, by taking into account that in many applications, NMF methods operate in a multichannel environment, useful information can be extracted in order to initialize H.sub.m in a more meaningful way. In a particular embodiment, an energy ratio between a source signal and other source signals is defined and used for initialization of H.sub.m.

(30) When analyzing a source signal into frames of length L.sub.f with hop size L.sub.h and an analysis window w(k) we can express the κ-th frame as a vector

(31) $\begin{array}{cc}{x_m\left(k\right)=[x_m{\mathrm{kL}}_h)w\left(0\right)x_m\left({\mathrm{kL}}_h+1\right)w\left(1\right).Math.x_m\left({\mathrm{kL}}_h+L_f-1\right)w\left(L_f-1\right)]}^T& \left(7\right)\end{array}$
and the energy of the κ-th frame of the m-th source signal is given as

(32) $\begin{array}{cc}ℰ.Math.x_m\left(\kappa \right).Math.-\frac{1}{L_f}{.Math.x_m\left(\kappa \right).Math.}^2& \left(8\right)\end{array}$

(33) The energy ratio for the m-th source signal is given by

(34) $\begin{array}{cc}{\mathrm{ER}}_m\left(\kappa \right)-\frac{ℰ\left[x_m\left(\kappa \right)\right]}{\underset{\underset{i\ne m}{i=1}}{\overset{M}{.Math.}}ℰ\left[x_m\left(\kappa \right)\right]}& \left(9\right)\end{array}$
The values of the energy ratio ER.sub.m(κ) can be arranged as a 1×T row vector and the M vectors can be arranged into an M×T matrix Ĥ.sub.m. If K=M then this matrix can be used as the initialization value of H.sub.m. If K>M, this matrix can be appended with a (K−M)×T randomly initialized matrix or with any other relevant matrix. If K<M, only some of rows of Ĥ.sub.m can be used.

(35) In general, the energy ratio can be calculated from the original source signals as described earlier or from any modified version of the source signals. In another embodiment, the energy ratios can be calculated from filtered versions of the original signals. In this case bandpass filters may be used and they may be sharp and centered around a characteristic frequency of the main signal found in each source signal. This is especially useful in cases where such frequencies differ significantly for various source signals. One way to estimate a characteristic frequency of a source signal is to find a frequency bin with the maximum magnitude from an averaged spectrogram of the sources as in:

(36) 0$\begin{array}{cc}{\omega }_m^c-\underset{\omega }{\mathrm{argmax}}\left[\frac{1}{T}\underset{n=1}{\overset{T}{.Math.}}.Math.X_m\left(\kappa ,\omega \right).Math.\right]& \left(10\right)\end{array}$
where ω is the frequency index. A bandpass filter can be designed and centered around ω.sub.m.sup.c. The filter can be IIR, FIR, or any other type of filter and it can be designed using any digital filter design method. Each source signal can be filtered with the corresponding band pass filter and then the energy ratios can be calculated.

(37) In other embodiments, the energy ratio can be calculated in any domain including but not limited to the time-domain for each frame κ, the frequency domain, the time-frequency domain, etc. In this case ER.sub.m(κ) can be given by

(38) $\begin{array}{cc}{\mathrm{ER}}_{\omega }\left(\kappa \right)-f\left({\mathrm{ER}}_m\left(\kappa ,\omega \right)\right)& \left(11\right)\end{array}$
where f(.Math.) is a suitable function that calculates a single value of the energy ratio for the κ-th frame by an appropriate combination of the values ER.sub.m(κ, ω). In specific embodiments, said function could choose the value of ER.sub.m(κ, ω.sub.m.sup.c) or the maximum value for all ω, or the mean value for all ω, etc. In other embodiments, the power ratio or other relevant metrics can be used instead of the energy ratio.

(39) FIG. 3 presents an example where a source signal 301 and an energy ratio are each plotted as functions (amplitude vs. time) 302. The energy ratio has been calculated and is shown for a multichannel environment. The energy ratio often tracks the envelope of the source signal. In specific signal parts (for example signal position 303), however, the energy ratio has correctly identified an unwanted signal part and does not follow the envelope of the signal.

(40) FIG. 4 shows an exemplary embodiment of the present application where the energy ratio is calculated from M source signals x.sub.1(k) to x.sub.M(k) that can be analyzed in T frames and used to initialize a weight matrix Ĥ.sub.m of K rows. In this specific embodiment there are 8 source signals 401, 402, 403, 404, 405, 406, 407 and 408. Using the 8 source signals the energy ratios are calculated 419 and used to initialize 8 rows of the matrix Ĥ.sub.m 411, 412, 413, 414, 415, 416, 417 and 418. In this example, since the rows of matrix Ĥ.sub.m are 10 (more than the source signals), the rows 409 and 410 are initialized with random signals.

(41) Using the initialization and training steps described above, a meaningful convergence of the decomposition can be achieved. After convergence, the component masks are extracted and applied to the original matrix in order to produce a set of K component signals z.sub.j,m(k) for each source signal x.sub.m(k). In a particular embodiment, said component signals are automatically sorted according to their similarity to a reference signal r.sub.m(k). First, an appropriate reference signal r.sub.m(k) must be chosen which can be different according to the processing application and can be any signal including but not limited to the source signal itself (which also includes one or many of its inherent parts), a filtered version of the source signal, an estimate of the source signal, etc. Then the reference signal is analyzed in frames and we define the set

(42) $\begin{array}{cc}{\Omega }_m=\left\{\kappa :ℰ\left[r_m\left(\kappa \right)\right]>E_r\right\}& \left(12\right)\end{array}$
which indicates the frames of the reference signal that have significant energy, that is their energy is above a threshold E.sub.T. We calculate the cosine similarity measure

(43) $\begin{array}{cc}{c}_{j,m}\left(\kappa \right)-\frac{r_m\left(\kappa \right).Math.z_{j,m}\left(\kappa \right)}{.Math.r_m\left(\kappa \right).Math..Math.z_{j,m}\left(\kappa \right).Math.},\kappa \in {\Omega }_m\mathrm{and}j-1,.Math.,K& \left(13\right)\end{array}$
and then calculate

(44) $\begin{array}{cc}{c}_{j,m}^{\prime }-f\left(c_{j,m}\left(\kappa \right)\right)& \left(14\right)\end{array}$

(45) In particular embodiments, f(.Math.) can be any suitable function such as max, mean, median, etc. The component signals z.sub.j,m(k) that are produced by the decomposition process can now be sorted according to a similarity measure, i.e. a function that measures the similarity between a subset of frames of r.sub.m(k) and z.sub.j,m(k). A specific similarity measure is shown in equation (13), however any function or relationship that compares the component signals to the reference signals can be used. An ordering or function applied to the similarity measure c.sub.j,m(k) then results in ć.sub.j,m. A high value indicates significant similarity between r.sub.m(k) and z.sub.j,m(k) while a low value indicates the opposite. In particular embodiments, clustering techniques can be used instead of using a similarity measure, in order to group relevant components together, in such a way that components in the same group (called cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). In particular embodiment, any clustering technique can be applied to a subset of component frames (for example those that are bigger than a threshold E.sub.T), including but not limited to connectivity based clustering (hierarchical clustering), centroid-based clustering, distribution-based clustering, density-based clustering, etc.

(46) FIG. 5 presents a block diagram where exemplary embodiments of the present application are shown. A time domain source signal 501 is transformed in the frequency 502 domain using any appropriate transform, in order to produce the non-negative matrix V.sub.m 503. Then a training sequence is created 504 and after any appropriate transform it is appended to the original non-negative matrix 505. In addition, the source signals are used to derive the energy ratios and initialize the weight matrix 506. Using the above initialized matrices, NMF is performed on V.sub.m 507. After NMF, the signal components are extracted 508 and after calculating the energy of the frames, a subset of the frames with the biggest energy is derived 509 and used for the sorting procedure 510.

(47) In particular embodiments, human input can be used in order to produce desired output signals. After automatic or manual sorting and/or categorization, signal components are typically in a meaningful order. Therefore, a human user can select which components from a predefined hierarchy will form the desired output. In a particular embodiment, K components are sorted using any sorting and/or categorization technique. A human user can define a gain μ for each one of the components. The user can define the gain explicitly or intuitively. The gain can take the value 0, therefore some components may not be selected. Any desired output y.sub.m(k) can be extracted as any combination of components z.sub.j,m(k):

(48) $\begin{array}{cc}{y}_m\left(k\right)=\underset{i=1}{\overset{K}{.Math.}}{\mu }_j\left(k\right)z_{j,m}\left(k\right)& \left(15\right)\end{array}$

(49) In FIG. 6 two exemplary user interfaces are illustrated, in accordance with embodiments of the present application, in the forms of a knob 601 and a slider 602. Such elements can be implemented either in hardware or in software.

(50) In one particular example, the total number of components is 4. When the knob/slider is in position 0, the output will be zeroed, when it is in position 1 only the first component will be selected and when it is in position 4 all four components will be selected. When the user has set the value of the knob and/or slider at 2.5 and assuming that a simple linear addition is performed, the output will be given by:

(51) $\begin{array}{cc}{y}_m\left(k\right)=z_{1,m}\left(k\right)+z_{2,m}\left(k\right)+0.5z_{3,m}\left(k\right)& \left(16\right)\end{array}$

(52) In another embodiment, a logarithmic addition can be performed or any other gain for each component can be derived from the user input.

(53) Using similar interface elements, different mapping strategies regarding the component selection and mixture can be also followed. In another embodiment, in knob/slider position 0 of FIG. 6, the output will be the sum of all components, in position 1 components the output will be the sum of components 1, 2 and 3 and in position 4 the output will be zeroed. Therefore, assuming a linear addition scheme for this example, putting the knob/slider at position 2.5 will produce an output given by:

(54) $\begin{array}{cc}{y}_m\left(k\right)=z_{1,m}\left(k\right)+0.5z_{2,m}\left(k\right)& \left(17\right)\end{array}$

(55) Again, the strategy and the gain for each component can be defined through any equation from the user-defined value of the slider/knob.

(56) In another embodiment, source signals of the present invention can be microphone signals in audio applications. Consider N simultaneously active signals s.sub.n(k) (i.e. sound sources) and M microphones set to capture those signals, producing the source signals x.sub.m(k). In particular embodiments, each sound source signal may correspond to the sound of any type of musical instrument such as a multichannel drums recording or human voice. Each source signal can be described as

(57) $\begin{array}{cc}{z}_m\left(k\right)-\underset{m=1}{\overset{N}{.Math.}}\left[{\rho }_s\left(k,{\theta }_{\mathrm{mm}}\right)*z_m\left(k\right)\right]*\left[p_x\left(k,{\theta }_{\mathrm{mn}}\right)*k_{\mathrm{mm}}\left(k\right)\right]& \left(18\right)\end{array}$
for m=1, . . . , M. ρ.sub.s(k, θ.sub.mn) is a filter that takes into account the source directivity, ρ.sub.c(k, θ.sub.mn) is a filter that describes the microphone directivity, h.sub.mn(k) is the impulse response of the acoustic environment between the n-th sound source and m-th microphone and * denotes convolution.
In most audio applications each sound source is ideally captured by one corresponding microphone. However, in practice each microphone picks up the sound of the source of interest but also the sound of all other sources and hence equation (18) can be written as

(58) $\begin{array}{cc}{x}_m\left(k\right)-\left[{\rho }_k\left(k,{\theta }_{\mathrm{mm}}\right)*s_m\left(k\right)\right]*\left[{\rho }_c\left(k,{\theta }_{\mathrm{mm}}\right)*h_{\mathrm{mm}}\left(k\right)\right]+\underset{n\ne m}{\underset{n=1}{\overset{N}{.Math.}}}\left[{\rho }_s\left(k,{\theta }_{mm}\right)*s_m\left(k\right)\right]*\left[{\rho }_c\left(k,{\theta }_{mm}\right)*h_{mm}\left(k\right)\right]& \left(19\right)\end{array}$

(59) To simplify equation (19) we define the direct source signal as

(60) 0$\begin{array}{cc}{\hat{s}}_m\left(k\right)-\left[{\rho }_s\left(k,{\theta }_{mm}\right)*s_m\left(k\right)\right]*\left[{\rho }_c\left(k,{\theta }_{mm}\right)*h_{mm}\left(k\right)\right]& \left(20\right)\end{array}$

(61) Note that here m=n and the source signal is the one that should ideally be captured by the corresponding microphone. We also define the leakage source signal as

(62) $\begin{array}{cc}{\stackrel{\_}{s}}_{\mathrm{nm}}\left(k\right)-\left[{\rho }_s\left(k,{\theta }_{mm}\right)*s_m\left(k\right)\right]*\left[{\rho }_c\left(k,{\theta }_{mm}\right)*h_{mm}\left(k\right)\right]& \left(21\right)\end{array}$

(63) In this case m≠n and the source signal is the result of a source that does not correspond to this microphone and ideally should not be captured. Using equations (20) and (21), equation (19) can be written as

(64) $\begin{array}{cc}{x}_m\left(k\right)-{\tilde{s}}_m\left(k\right)+\underset{n\ne m}{\underset{n=1}{\overset{N}{.Math.}}}{\stackrel{\_}{s}}_{n,m}\left(k\right)& \left(22\right)\end{array}$

(65) There are a number of audio applications that would greatly benefit from a signal processing method that would extract the direct source signal {tilde over (s)}.sub.m(k) the source signal x.sub.m(k) and remove the interfering leakage sources s.sub.n,m(k).

(66) One way to achieve this is to perform NMF on an appropriate representation of x.sub.m(k) according to embodiments of the present application. When the original mixture is captured in the time domain, the non-negative matrix V.sub.m can be derived through any signal transformation. For example, the signal can be transformed in the time-frequency domain using any relevant technique such as a short-time Fourier transform (STFT), a wavelet transform, a polyphase filterbank, a multi rate filterbank, a quadrature mirror filterbank, a warped filterbank, an auditory-inspired filterbank, etc. Each one of the above transforms will result in a specific time-frequency resolution that will change the processing accordingly. All embodiments of the present application can use any available time-frequency transform or any other transform that ensures a non-negative matrix V.sub.m.

(67) By appropriately transforming x.sub.m(k), the signal X.sub.m(κ, ω) can be obtained where κ=0, . . . , T−1 is the frame index and ω=0, . . . , F−1 is the discrete frequency bin index. From the complex-valued signal X.sub.m(κ, ω) we can obtain the magnitude V.sub.m(κ, ω). The values of V.sub.m(κ, ω) form the magnitude spectrogram of the time-domain signal x.sub.m(k). This spectrogram can be arranged as a matrix V.sub.m of size F×T. Note that where the term spectrogram is used, it does not only refer to the magnitude spectrogram but any version of the spectrogram that can be derived from

(68) $\begin{array}{cc}{V}_m\left(\kappa ,\omega \right)-f\left({.Math.X_m\left(\kappa ,\omega \right).Math.}^2\right)& \left(23\right)\end{array}$
where f(.Math.) can be any suitable function (for example the logarithm function). As seen from the previous analysis, all embodiments of the present application are relevant to sound processing in single or multichannel scenarios.

(69) While the above-described flowcharts have been discussed in relation to a particular sequence of events, it should be appreciated that changes to this sequence can occur without materially effecting the operation of the invention. Additionally, the exemplary techniques illustrated herein are not limited to the specifically illustrated embodiments but can also be utilized and combined with the other exemplary embodiments and each described feature is individually and separately claimable.

(70) Additionally, the systems, methods and protocols of this invention can be implemented on a special purpose computer, a programmed micro-processor or microcontroller and peripheral integrated circuit element(s), an ASIC or other integrated circuit, a digital signal processor, a hard-wired electronic or logic circuit such as discrete element circuit, a programmable logic device such as PLD, PLA, FPGA, PAL, a modem, a transmitter/receiver, any comparable means, or the like. In general, any device capable of implementing a state machine that is in turn capable of implementing the methodology illustrated herein can be used to implement the various communication methods, protocols and techniques according to this invention.

(71) Furthermore, the disclosed methods may be readily implemented in software using object or object-oriented software development environments that provide portable source code that can be used on a variety of computer or workstation platforms. Alternatively the disclosed methods may be readily implemented in software on an embedded processor, a micro-processor or a digital signal processor. The implementation may utilize either fixed-point or floating point operations or both. In the case of fixed point operations, approximations may be used for certain mathematical operations such as logarithms, exponentials, etc. Alternatively, the disclosed system may be implemented partially or fully in hardware using standard logic circuits or VLSI design. Whether software or hardware is used to implement the systems in accordance with this invention is dependent on the speed and/or efficiency requirements of the system, the particular function, and the particular software or hardware systems or microprocessor or microcomputer systems being utilized. The systems and methods illustrated herein can be readily implemented in hardware and/or software using any known or later developed systems or structures, devices and/or software by those of ordinary skill in the applicable art from the functional description provided herein and with a general basic knowledge of the audio processing arts.

(72) Moreover, the disclosed methods may be readily implemented in software that can be stored on a storage medium, executed on programmed general-purpose computer with the cooperation of a controller and memory, a special purpose computer, a microprocessor, or the like. In these instances, the systems and methods of this invention can be implemented as program embedded on personal computer such as an applet, JAVA®. or CGI script, as a resource residing on a server or computer workstation, as a routine embedded in a dedicated system or system component, or the like. The system can also be implemented by physically incorporating the system and/or method into a software and/or hardware system, such as the hardware and software systems of an electronic device.

(73) It is therefore apparent that there has been provided, in accordance with the present invention, systems and methods for improved signal decomposition in electronic devices. While this invention has been described in conjunction with a number of embodiments, it is evident that many alternatives, modifications and variations would be or are apparent to those of ordinary skill in the applicable arts. Accordingly, it is intended to embrace all such alternatives, modifications, equivalents and variations that are within the spirit and scope of this invention.