Audio Signal Processing Apparatuses and Methods

Abstract

Audio signal processing apparatuses and methods are provided, such as an audio signal downmixing apparatus for processing an input audio signal into an output audio signal, wherein the input audio signal comprises a plurality of input channels recorded at a plurality of spatial positions and the output audio signal comprises a plurality of primary output channels. The audio signal downmixing apparatus comprises a downmix matrix determiner configured to determine for each frequency bin j of a plurality of frequency bins a downmix matrix D.sub.U with j being an integer in the range from 1 to N, and a processor configured to process the input audio signal using the downmix matrix D.sub.U into the output audio signal.

Claims

1. An apparatus, comprising: a downmix matrix determiner, configured to: determine, for each frequency bin j of a plurality of frequency bins, a downmix matrix (D.sub.U), with j being an integer in a range from 1 to N, wherein an input audio signal comprises a plurality of input channels recorded at a plurality of spatial positions, an output audio signal comprises a plurality of primary output channels, wherein, for a given frequency bin j, the downmix matrix (D.sub.U) maps a plurality of Fourier coefficients associated with the plurality of input channels of the input audio signal into a plurality of Fourier coefficients of the plurality of primary output channels of the output audio signal, wherein, for frequency bins with j being smaller than or equal to a cutoff frequency bin k, the downmix matrix (D.sub.U) is determined by determining eigenvectors of a discrete Laplace-Beltrami operator (L) defined by a plurality of spatial positions where the plurality of input channels are recorded, and wherein, for frequency bins with j being larger than the cutoff frequency bin k, the downmix matrix (D.sub.U) is determined by determining a first subset of eigenvectors of a covariance matrix (COV) defined by the plurality of input channels of the input audio signal; and a processor, configured to process the input audio signal using the downmix matrix (D.sub.U) into the output audio signal.

2. The apparatus of claim 1, wherein the downmix matrix determiner is configured to determine the discrete Laplace-Beltrami operator (L) using the following equations:
L=C−W
C=diag{c}
c=[c.sub.1, . . . , c.sub.p, . . . , c.sub.Q]
c.sub.p=Σ.sub.q=1.sup.Q w.sub.pq; where L, C and W are matrices having respective dimensions Q×Q, where Q is a number of input channels, diag ( . . . ) denotes a matrix diagonalization operation placing input vector elements as a diagonal of an output matrix with the rest of matrix elements being zero, c is a vector of dimension Q and w.sub.pq are local averaging coefficients.

3. The apparatus of claim 2, wherein the downmix matrix determiner is configured to determine the local averaging coefficients w.sub.pq using the following equations: $w_{pq} = \frac{1}{{.Math. r_{q} - r_{p} .Math.}^{2}}; p \neq q$ $w_{pq} = 0; p = q,$ where r.sub.p or r.sub.q is a vector defining a spatial position of the plurality of spatial positions where the plurality of input channels of the input audio signal are recorded.

4. The apparatus of claim 1, wherein, for frequency bins with j being smaller than or equal to the cutoff frequency bin k, the downmix matrix (D.sub.U) is determined by selecting the eigenvectors of the discrete Laplace-Beltrami operator (L) that have an eigenvalue that is greater than a predefined threshold.

5. The apparatus of claim 1, wherein, for frequency bins with j being larger than the cutoff frequency bin k, the downmix matrix (D.sub.U) is determined by selecting the eigenvectors of the covariance matrix (COV) that have an eigenvalue that is greater than a predefined threshold.

6. The apparatus of claim 1, wherein the downmix matrix determiner is configured to determine the cutoff frequency bin k by determining the frequency bin of the plurality of frequency bins which has the smallest compactness measure θ.sub.C of all frequency bins having a compactness measure θ.sub.C greater than a predefined threshold T, wherein a compactness measure θ.sub.C of a frequency bin is determined using the following equation: $θ_{C} = \frac{{.Math. diag ({\hat{U}}^{H} .Math. COV .Math. .Math. \hat{U}) .Math.}_{F}}{{.Math. off ({\hat{U}}^{H} .Math. COV .Math. .Math. \hat{U}) .Math.}_{F}}$ wherein .Math. denotes a unitary matrix containing selected eigenvectors of the discrete Laplace-Beltrami operator (L), .Math..sup.H denotes the hermitian transpose of .Math., diag ( . . . ) denotes a matrix diagonalization operation zeroing all coefficients except the coefficients along the diagonal of the matrix given a matrix input, off ( . . . ) denotes a matrix operation zeroing all coefficients on the diagonal of the matrix and ∥ . . . ∥.sub.F denotes the Frobenius norm.

7. The apparatus of claim 1, wherein the apparatus further comprises a downmix matrix extension determiner, configured to determine a downmix matrix extension (D.sub.W) by determining a second subset of eigenvectors of the covariance matrix (COV) containing at least one eigenvector of the covariance matrix (COV) for providing at least one auxiliary output channel of the output audio signal, wherein the first subset of eigenvectors of the covariance matrix (COV) and the second subset of eigenvectors of the covariance matrix (COV) are disjoint sets and wherein the downmix matrix (D.sub.U) and the downmix matrix extension (D.sub.W) define an extended downmix matrix (D).

8. The apparatus of claim 7, wherein the downmix matrix extension determiner is configured to determine the second subset of eigenvectors of the covariance matrix (COV) by: determining, for each eigenvector of the covariance matrix (COV), a plurality of angles between the eigenvector and a plurality of vectors defined by columns of the downmix matrix (D.sub.U); determining, for each eigenvector, the smallest angle of the plurality of angles between the eigenvector and the plurality of vectors defined by the columns of the downmix matrix (D.sub.U); and selecting those eigenvectors of the covariance matrix (COV) for which the smallest angle between the eigenvector and the plurality of vectors defined by the columns of the downmix matrix (D.sub.U) is bigger than a threshold angle θ.sub.MIN.

9. The apparatus of claim 1, wherein the processor is configured to process the input audio signal for each of the plurality of input channels in a form of a plurality of input audio signal time frames, and wherein the plurality of Fourier coefficients associated with the plurality of input channels of the input audio signal are obtained by discrete Fourier transforms of the plurality of input audio signal time frames.

10. The apparatus of claim 9, wherein the downmix matrix determiner is configured to determine the covariance matrix (COV) defined by the plurality of input channels of the input audio signal by determining coefficients c.sub.xy of the covariance matrix (COV) for a given input audio signal time frame n of the plurality of input audio signal time frames and for a given frequency bin j of the plurality of frequency bins using the following equation:
c.sub.xy(n,j)=E{j.sub.x.Math.j*.sub.y} where E{ } denotes an expectation operator, j.sub.x denotes a Fourier coefficient at frequency bin j for input channel x of the input audio signal, * denotes the complex conjugate and x and y range from 1 to a number of input channels Q.

11. The apparatus of claim 9, wherein the downmix matrix determiner is configured to determine the covariance matrix (COV) defined by the plurality of input channels of the input audio signal by determining coefficients c.sub.xy of the covariance matrix (COV) for a given input audio signal time frame n of the plurality of input audio signal time frames and for a given frequency bin j of the plurality of frequency bins using the following equation:
c.sub.xy(n,j)=β.Math.c.sub.xy(n-1,j)+(1−β).Math.ĉ.sub.xy(n,j) where β denotes a forgetting factor with 0≦β<1, ĉ.sub.xy(n,j) denotes the real part of E{j.sub.x.Math.j*.sub.y}, j.sub.x denotes a Fourier coefficient at frequency bin j for input channel x of the input audio signal, * denotes the complex conjugate and x and y range from 1 to the number of input channels Q.

12. A method, comprising: determining, for each frequency bin j of a plurality of frequency bins, a downmix matrix (D.sub.U), wherein j is an integer in a range from 1 to N, wherein an input audio signal comprises a plurality of input channels recorded at a plurality of spatial positions, an output audio signal comprises a plurality of primary output channels, wherein, for a given frequency bin j, the downmix matrix (D.sub.U) maps a plurality of Fourier coefficients associated with the plurality of input channels of the input audio signal into a plurality of Fourier coefficients of the primary output channels of the output audio signal, wherein, for frequency bins with j being smaller than or equal to a cutoff frequency bin k, the downmix matrix (D.sub.U) is determined by determining eigenvectors of a discrete Laplace-Beltrami operator (L) defined by the plurality of spatial positions where the plurality of input channels are recorded, and wherein, for frequency bins with j being larger than the cutoff frequency bin k, the downmix matrix (D.sub.U) is determined by determining a first subset of eigenvectors of a covariance matrix (COV) defined by the plurality of input channels of the input audio signal; and processing the input audio signal using the downmix matrix (D.sub.U) into the output audio signal.

13. A computer program, comprising: a non-transitory memory; and a program code stored on the non-transitory memory, wherein the program code, when executed on a computer causes the computer to: determine, for each frequency bin j of a plurality of frequency bins, a downmix matrix (D.sub.U), wherein j is an integer in a range from 1 to N, wherein an input audio signal comprises a plurality of input channels recorded at a plurality of spatial positions, an output audio signal comprises a plurality of primary output channels, wherein, for a given frequency bin j, the downmix matrix (D.sub.U) maps a plurality of Fourier coefficients associated with the plurality of input channels of the input audio signal into a plurality of Fourier coefficients of the primary output channels of the output audio signal, wherein, for frequency bins with j being smaller than or equal to a cutoff frequency bin k, the downmix matrix (D.sub.U) is determined by determining eigenvectors of a discrete Laplace-Beltrami operator (L) defined by the plurality of spatial positions where the plurality of input channels are recorded, and wherein, for frequency bins with j being larger than the cutoff frequency bin k, the downmix matrix (D.sub.U) is determined by determining a first subset of eigenvectors of a covariance matrix (COV) defined by the plurality of input channels of the input audio signal; and processing the input audio signal using the downmix matrix (D.sub.U) into the output audio signal.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Further embodiments of the invention will be described with respect to the following figures, in which:

[0045] FIG. 1 shows a schematic diagram of an audio signal downmixing apparatus according to an embodiment and an audio signal upmixing apparatus according to an embodiment as part of an audio signal processing system; and

[0046] FIG. 2 shows a schematic diagram of an audio signal downmixing method according to an embodiment.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

[0047] In the following detailed description, reference is made to the accompanying drawings, which form a part of the disclosure, and in which are shown, by way of illustration, specific aspects in which the disclosure may be practiced. It is understood that other aspects may be utilized and structural or logical changes may be made without departing from the scope of the present disclosure. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims.

[0048] It is understood that a disclosure in connection with a described method may also hold true for a corresponding device or system configured to perform the method and vice versa. For example, if a specific method step is described, a corresponding device or apparatus may include a unit to perform the described method step, even if such unit is not explicitly described or illustrated in the figures. Further, it is understood that the features of the various exemplary aspects described herein may be combined with each other, unless specifically noted otherwise.

[0049] FIG. 1 shows a schematic diagram of an audio signal downmixing apparatus 105 according to an embodiment as part of an audio signal processing system 100.

[0050] The audio signal downmixing apparatus 105 is configured to process an input audio signal into an output audio signal, wherein the input audio signal comprises a plurality of input channels 113 recorded at a plurality of spatial positions and the output audio signal comprises a plurality of primary output channels 123. In an embodiment, the multichannel input audio signal 113 comprises Q input channels. In an embodiment, the audio signal downmixing apparatus 105 is configured to process the multichannel input audio signal 113 in a frame-wise manner, i.e. in the form of a plurality of input audio signal time frames, wherein an audio signal time frame can have a length of, for instance, about 10 to 40 ms per channel. In an embodiment, subsequent input audio signal time frames can be partially overlapping. In an embodiment, the multichannel input audio signal 113 is processed in the frequency domain. In an embodiment, an input audio signal time frame of a channel of the multichannel input audio signal 113 is transformed into the frequency domain by means of a discrete Fourier transformation, in particular a FFT, yielding a plurality of Fourier coefficients j.sub.x at frequency bin j of the input channel x of the multichannel audio input signal 113, wherein j runs from 1 to N, i.e. the total number of frequency bins, and x runs from 1 to the total number of input channels Q.

[0051] The audio signal downmixing apparatus 105 comprises a downmix matrix determiner 107 configured to determine for each frequency bin j (and in case of a frame-wise processing of the multichannel input audio signal 113 for every input audio signal time frame) a downmix matrix D.sub.U, wherein for a given frequency bin j the downmix matrix D.sub.U maps the plurality of Fourier coefficients associated with the plurality of input channels 113 of the input audio signal into a plurality of Fourier coefficients of the primary output channels 123 of the output audio signal.

[0052] Moreover, the audio signal downmixing apparatus 105 comprises a processor 109 configured to process the multichannel input audio signal 113 using the downmix matrix D.sub.U into the output audio signal.

[0053] For frequency bins with j being smaller than or equal to a cutoff frequency bin k the downmix matrix D.sub.U is determined by the downmix matrix determiner 107 by determining eigenvectors of the discrete Laplace-Beltrami operator L defined by the plurality of spatial positions where the plurality of input channels 113 are or have been recorded at. In an embodiment, the plurality of spatial positions where the plurality of input channels 113 are or have been recorded at are defined by the spatial positions of a corresponding plurality of microphones or other sound recording devices used to record the multichannel audio input signal 113. In an embodiment, information about the plurality of spatial positions where the plurality of input channels 113 have been recorded at can be provided to or stored in the downmix matrix determiner 107.

[0054] In an embodiment, the downmix matrix determiner 107 is configured to determine the discrete Laplace-Beltrami operator L using the following equations:

L=C−W,

C=diag{c},

c=[c.sub.1, . . . , c.sub.p, . . . , c.sub.Q], and

c.sub.p=Σ.sub.q=1.sup.Q w.sub.pq,

[0055] where L is a matrix representation of the Laplace-Beltrami operator and C and W are matrices having respective dimensions Q×Q, where Q is the number of input channels 113, diag ( . . . ) denotes a matrix diagonalization operation placing the input vector elements as the diagonal of the output matrix with the rest of matrix elements being zero, c is a vector of dimension Q and w.sub.pq are local averaging coefficients.

[0056] In an embodiment, the downmix matrix determiner 107 is configured to determine the local averaging coefficients w.sub.pq using the following equations:

[00003] $w_{pq} = \frac{1}{{.Math. r_{q} - r_{p} .Math.}^{2}}; p \neq q$ $w_{pq} = 0; p = q,$

[0057] where r.sub.p or r.sub.q is a 3-dimensional vector defining a spatial position of the plurality of spatial positions where the plurality of input channels of the input audio signal are recorded at, for instance, the spatial positions of Q microphones or other sound recording devices used to record the multichannel audio input signal 113.

[0058] In an embodiment, the downmix matrix determiner 107 is configured to determine the downmix matrix D.sub.U for frequency bins with j being smaller than or equal to the cutoff frequency bin k by selecting the eigenvectors of the discrete Laplace-Beltrami operator L that have an eigenvalue that is greater than a predefined threshold value λ.sub.L.

[0059] For frequency bins with j being larger than the cutoff frequency bin k the downmix matrix determiner 107 is configured to determine the downmix matrix D.sub.U by determining a first subset of eigenvectors of a covariance matrix COV defined by the plurality of input channels 113 of the input audio signal.

[0060] In an embodiment where the multichannel audio input signal 113 is processed in a frame-wise manner, the downmix matrix determiner 107 is configured to determine the covariance matrix COV defined by the plurality of input channels 113 of the input audio signal by determining coefficients c.sub.xy of the covariance matrix COV for a given input audio signal time frame n of the plurality of input audio signal time frames and for a given frequency bin j of the plurality of frequency bins using the following equation:

c.sub.xy(n,j)=E{j.sub.x.Math.j*.sub.y},

[0061] where E{ } denotes an expectation operator, * denotes the complex conjugate and x and y range from 1 to the number of input channels Q.

[0062] In an embodiment where the multichannel audio input signal 113 is processed in a frame-wise manner, the downmix matrix determiner 107 is configured to determine the covariance matrix COV defined by the plurality of input channels 113 of the input audio signal by determining the coefficients c.sub.xy of the covariance matrix COV for a given input audio signal time frame n of the plurality of input audio signal time frames and for a given frequency bin j of the plurality of frequency bins using the following equation:

c.sub.xy(n,j)=β.Math.c.sub.xy(n-1,j)+(1−β).Math.ĉ.sub.xy(n,j),

[0063] where β denotes a forgetting factor with 0≦β<1 and ĉ.sub.xy(n,j) denotes the real part of E{j.sub.x.Math.j*.sub.y}.

[0064] In an embodiment, in order to reduce the computational complexity the Fourier coefficients can be grouped into B different bands based on certain psychoacoustical scales, such as the Bark scale or the Mel scale, and the determination of the covariance matrix COV can be performed per band b, where b ranges from 1 to B. In this case, a simplified covariance matrix can be used having the following coefficients by performing, e.g., an addition:

[00004] ${\overline{c}}_{xy, b} (n, j) = \underset{j \in b}{.Math.} .Math. .Math. c_{xy} (n, j) .$

[0065] This grouping into B bands reduces the computational complexity by only taking a subset of the overall Fourier coefficients.

[0066] In an embodiment, the downmix matrix determiner 107 is configured to determine the downmix matrix D.sub.U for frequency bins with j being larger than the cutoff frequency bin k by selecting as a first subset of eigenvectors those eigenvectors of the covariance matrix COV that have an eigenvalue that is greater than a predefined threshold value λ.sub.COV.

[0067] In an embodiment, the downmix matrix determiner 107 is configured to determine eigenvectors of the covariance matrix COV for a given input audio signal time frame n of the plurality of input audio signal time frames and for a given frequency bin j of the plurality of frequency bins by means of an eigenvalue decomposition (EVD), i.e.

COV(n,j)=UAU.sup.H,

[0068] where U is a unitary matrix containing the eigenvectors, A is a diagonal matrix containing the eigenvalues and U.sup.H is the Hermitian transpose of the matrix U.

[0069] In an embodiment, the eigenvectors of the covariance matrix COV are calculated iteratively by exploiting the rank-one modification character of the covariance matrix estimate to reduce the computational complexity, because it is not necessary to perform the EVD for each frame n.

[0070] Exploiting the nature of the autocorrelation estimation in the transform domain leads to an efficient Karhunen-Loeve Transform (KLT)

Λ.sup.(i)(n)=αΛ.sup.(i)(n-1)+(1−α)Y.sup.(i)N(n)Y.sup.(i)(n),

Y.sup.(i)(n):=X.sup.(i)(n)U.sup.(i)(n-1).

[0071] where α is a forgetting factor having a value between 0 and 1 and Y and X denote the output and input Fourier coefficients arranged as row vectors of the downmix operation performed by the matrix U.

[0072] The estimation is based on a rank-one modification of a diagonal matrix. It has been shown in the literature that the eigenvalues of Λ.sup.(i)(n) are the zeros of the function

[00005] $.Math. w (λ) := 1 + (1 - α) .Math. {.Math.}_{q = 1}^{Q} .Math. .Math. \frac{y_{q}^{2}}{{αλ}_{q}^{(i)} (n - 1) - λ}, .Math. .Math. w (λ) = 0 .Math. .Math. for$ $λ \in {λ_{q}^{(i)} (n) | λ_{q}^{(i)} (n) .Math. .Math. is .Math. .Math. an .Math. .Math. eigen .Math. .Math. value .Math. .Math. of .Math. .Math. the .Math. .Math. .Math. modified .Math. .Math. matrix .Math. .Math. Λ^{(i)} (n)}$

[0073] The zeros of the function w(λ) can be found iteratively. However, the convergence of the search process is quadratic. Once the eigenvalues are computed, the eigenvectors of the modified spatio-temporal transformed autocorrelation matrix G.sub.Uq of Λ.sup.(i)(n) can be explicitly computed by means of the following equations:

[00006] $G_{U_{q}} = \frac{Y^{(i)} (n) .Math. Λ_{q}^{{(i)}^{- 1}} (n)}{.Math. Y^{(i)} (n) .Math. Λ_{q}^{{(i)}^{- 1}} (n) .Math.}, .Math. Λ_{q}^{(i)} (n) := Λ_{q}^{(i)} (n - 1) - λ_{q}^{(i)} (n) .Math. I_{M \times M}$

[0074] In an embodiment, the downmix matrix determiner 107 is configured to determine the cutoff frequency bin k by determining the frequency bin of the plurality of frequency bins which has the smallest compactness measure θ.sub.C of all frequency bins having a compactness measure θ.sub.C greater than a predefined threshold T, wherein the compactness measure θ.sub.C of a frequency bin is defined by the following equation:

[00007] $θ_{C} = \frac{{.Math. diag ({\hat{U}}^{H} .Math. COV .Math. .Math. \hat{U}) .Math.}_{F}}{{.Math. off ({\hat{U}}^{H} .Math. COV .Math. .Math. \hat{U}) .Math.}_{F}},$

[0075] wherein .Math. denotes a unitary matrix containing the selected eigenvectors of the discrete Laplace-Beltrami operator L, .Math..sup.H denotes the hermitian transpose of .Math., diag ( . . . ) denotes a matrix diagonalization operation zeroing all coefficients except the coefficients along the diagonal of the matrix given a matrix input, off ( . . . ) denotes a matrix operation zeroing all coefficients on the diagonal of the matrix and ∥ . . . ∥.sub.F denotes the Frobenius norm. For the sake of simplicity the indexes n and j have been omitted in the above equation defining the compactness measure θ.sub.C of a frequency bin. As j goes from lower to higher frequencies (j=1 to N), the compactness measure θ.sub.C gets smaller. The choice of the cutoff frequency bin k is then determined heuristically using the predefined threshold T, where listening tests can be taken into account to make sure, that perceptually lossless encoding is possible.

[0076] The embodiments of the present invention includes embodiments where the cutoff frequency bin k is equal to the frequency bin corresponding to the highest frequency. As the person in the art will appreciate, in such a case the downmix matrix D.sub.U is solely defined by the eigenvectors of the discrete Laplace-Beltrami operator L for all frequency bins.

[0077] In an embodiment, the audio signal downmixing apparatus 105 further comprises a downmix matrix extension determiner 111 configured to determine a downmix matrix extension D.sub.W by determining a second subset of eigenvectors of the covariance matrix COV containing at least one eigenvector of the covariance matrix COV for providing at least one auxiliary output channel 125 of the output audio signal. The first subset of eigenvectors of the covariance matrix COV determined by the downmix matrix determiner 107 and the second subset of eigenvectors of the covariance matrix COV determined by the downmix matrix extension determiner 111 are determined in such a way that the first and second subset of eigenvectors are disjoint sets. The downmix matrix D.sub.U and the downmix matrix extension D.sub.W together define an extended downmix matrix D.

[0078] In an embodiment, the downmix matrix extension determiner 111 is configured to determine the second subset of eigenvectors of the covariance matrix COV by means of the following steps. In a first step the downmix matrix determiner 111 determines for each eigenvector of the covariance matrix COV a plurality of angles between the eigenvector and a plurality of vectors defined by the columns of the downmix matrix D.sub.U. In a second step the downmix matrix determiner 111 determines for each eigenvector the smallest angle of the plurality of angles between the eigenvector and the plurality of vectors defined by the columns of the downmix matrix D.sub.U. In a third step the downmix matrix determiner 111 selects those eigenvectors of the covariance matrix COV for which the smallest angle between the eigenvector and the plurality of vectors defined by the columns of the downmix matrix D.sub.U is bigger than a predefined threshold angle θ.sub.MIN.

[0079] The downmix matrix D.sub.U defines a subspace U of the space defined by the extended downmix matrix D. The downmix matrix extension D.sub.W defines a subspace W of the space defined by the extended downmix matrix D. The subspace angle between the subspace U and the subspace W is defined by as the minimum angle between all vectors u spanning the subspace U and all vectors w spanning the subspace W, i.e.

[00008] $θ_{1} := \min .Math. {\arccos (\frac{.Math. 〈 u, w 〉 .Math.}{.Math. u .Math. .Math. .Math. w .Math.}) | u \in , w \in } = ∠ (u_{1}, w_{1}),$

[0080] where <u,w> denotes the dot product of the vectors u and w and ∥u∥ denotes the norm of the vector u.

[0081] An example is given below for the exemplary case M=2 and Q=4 so that the subspace U is spanned by the vectors u1 and u2, i.e. U ={u1, u2} and the subspace W is spanned by the vectors w1, w2, w3 and w4, i.e. W={w1, w2, w3, w4}. In an embodiment, the following angles are calculated:

θ.sub.1=∠(u1, w1) θ.sub.5=∠(u2, w1)

θ.sub.2=∠(u1, w2) θ.sub.6=∠(u2, w2)

θ.sub.3=∠(u1, w3) θ.sub.7=∠(u2, w3)

θ.sub.4=∠(u1, w4) θ.sub.8=∠(u2, w4).

[0082] For calculating the subspace angle between the eigenvectors of the covariance matrix COV and the space spanned by the downmix matrix D.sub.U, θ is computed between every eigenvector and the columns of the downmix matrix D.sub.U. In the above example, this leads to the following angles:

θ.sub.a=min(θ.sub.1, θ.sub.5) θ.sub.c=min(θ.sub.3, θ.sub.7)

θ.sub.b=min(θ.sub.2, θ.sub.6) θ.sub.d=min(θ.sub.4, θ.sub.8)

[0083] The eigenvectors of the covariance matrix COV are sorted by decreasing subspace angle, where those having the larger angles are preferably selected for defining the downmix matrix extension D.sub.W. For example, in the case θ.sub.c>θ.sub.a>θ.sub.b>θ.sub.d at least the eigenvector w3 associated with the angles θ.sub.3 and θ.sub.7 will be selected as part of the downmix matrix extension D.sub.W.

[0084] As already mentioned above, the above described embodiments of the audio signal downmixing apparatus 105 can be implemented as a component of an encoding apparatus 101 of the audio signal processing system 100 shown in FIG. 1. As already described above, the audio signal downmixing apparatus 105 of the encoding apparatus 101 receives as input the input audio signal comprising Q input audio signal channels 113.

[0085] As described in detail above, the audio signal downmixing apparatus 105 processes on the basis of the downmix matrix D.sub.U or, in an embodiment, the extended downmix matrix D the Q channels of the multichannel input audio signal 113 and provides M primary output channels 123 of the audio output signal and, in an embodiment, furthermore up to Q-M auxiliary output channels 125 of the audio output signal.

[0086] The encoding apparatus 101 further comprises an encoder A 119 and another encoder B 121. The encoder A 119 receives as an input the M primary output channels 123 provided by the audio signal downmixing apparatus 105. The other encoder B 121 receives as an input from zero up to Q-M auxiliary output channels 125 provided by the audio signal downmixing apparatus 105.

[0087] The encoder A 119 is configured to encode the M primary output channels 123 provided by the audio signal downmixing apparatus 105 into a first bit stream 127. The other encoder B 121 is configured to encode the up to Q-M auxiliary output channels 125 provided, in an embodiment, by the audio signal downmixing apparatus 105 into a second bit stream 129. In an embodiment, the encoder A 119 and the other encoder B 121 can be implemented as a single encoder providing as an output a single bit stream.

[0088] The first bit stream 127 and the second bit stream 129 are provided as inputs to a decoding apparatus 103 of the audio signal processing system 100 shown in FIG. 1. The decoding apparatus 103 comprises corresponding decoders, namely a decoder A 133 and another decoder B 143, for decoding the first bit stream 127 and the second bit stream 129, respectively.

[0089] The decoder A 133 is configured to decode the first bit stream 127 such that the M primary input channels 135 provided by the decoder A 133 as output correspond to the M primary output channels 123 provided by the audio signal downmixing apparatus 105, i.e. such that the M primary input channels 135 provided by the decoder A 133 as output are essentially identical to the M primary output channels 123 provided by the audio signal downmixing apparatus 105 or a degraded version thereof (in case of a lossy codec implemented in the encoder A 119 and the decoder A 133).

[0090] The other decoder B 143 is configured to decode the second bit stream 129 such that the up to Q-M auxiliary input channels 145 provided by the other decoder B 143 as output correspond to the up to Q-M auxiliary output channels 125 provided by the audio signal downmixing apparatus 105, i.e. such that the up to Q-M auxiliary input channels 145 provided by the other decoder B 143 as output are essentially identical to the up to Q-M auxiliary output channels 125 provided by the audio signal downmixing apparatus 105 or a degraded version thereof (in case of a lossy codec implemented in the other encoder B 121 and the other decoder B 143).

[0091] In the embodiment shown in FIG. 1, the decoding apparatus 103 comprises an audio signal upmixing apparatus 139. In an embodiment, the audio signal upmixing apparatus 139 and/or the componets thereof are configured to perform essentially the inverse operation of the audio signal processing apparatus 105 and or the components thereof to generate an output audio signal 149. To this end, the audio signal upmixing apparatus 139 can comprise an upmix matrix determiner 137, a processor 141 and an upmix matrix extension determiner 147. In an embodiment, the processor 141 essentially performs the inverse operations (by means of a generalized-inverse method, e.g., pseudo-inverse) of the processor 109 of the audio signal processing apparatus 105 of the encoding apparatus 101. In an embodiment, the upmix matrix determiner 137 could be configured to determine an upmix matrix on the basis of the eigenvectors of the Laplace-Beltrami operator L and, if applicable, on the basis of the eigenvectors of the covariance matrix COV. In an embodiment, any additional data that the audio signal upmixing apparatus 139 can use for generating the output audio signal, such as metadata, can be transmitted via a bit stream 131. For instance, in an embodiment the audio signal downmixing apparatus 105 can provide the eigenvectors of the Laplace-Beltrami operator and/or, if applicable, the eigenvectors of the covariance matrix COV via the bit stream 131 to the audio signal upmixing apparatus 139 of the decoding apparatus for generating the output audio signal 149. The bit stream 131 can be encoded. An additional signal processing tool, i.e., remix (e.g., panning and wave field synthesis), can be further applied to the output audio signal 149 to obtain the targeted desired output audio signal. As the person skilled in the art will appreciate, the M primary input channels 135 provided by the decoder A 133 represent the M primary input channels 135 and the up to Q-M auxiliary input channels 145 provided by the other decoder B 143 represent the up to Q-M auxiliary input channels 145 of the input audio signal processed by the audio signal upmixing apparatus 139.

[0092] FIG. 2 shows a schematic diagram of an embodiment of an audio signal processing method 200 for processing an input audio signal into an output audio signal, wherein the input audio signal comprises a plurality of input channels 113 recorded at a plurality of spatial positions and the output audio signal comprises a plurality of primary output channels 123.

[0093] The audio signal processing method 200 comprises a step 201 of determining for each frequency bin j of a plurality of frequency bins a downmix matrix D.sub.U with j being an integer in the range from 1 to N, wherein for a given frequency bin j the downmix matrix D.sub.U maps a plurality of Fourier coefficients associated with the plurality of input channels 113 of the input audio signal into a plurality of Fourier coefficients of the primary output channels 123 of the output audio signal, wherein for frequency bins with j being smaller than or equal to a cutoff frequency bin k the downmix matrix D.sub.U is determined by determining eigenvectors of the discrete Laplace-Beltrami operator L defined by the plurality of spatial positions where the plurality of input channels 113 are recorded, and wherein for frequency bins with j being larger than the cutoff frequency bin k the downmix matrix D.sub.U is determined by determining a first subset of eigenvectors of a covariance matrix COV defined by the plurality of input channels 113 of the input audio signal.

[0094] Furthermore, the audio signal processing method 200 comprises a step 203 of processing the input audio signal using the downmix matrix D.sub.U into the output audio signal.

[0095] Embodiments of the invention may be implemented in a computer program for running on a computer system, at least including code portions for performing steps of a method according to the invention when run on a programmable apparatus, such as a computer system or enabling a programmable apparatus to perform functions of a device or system according to the invention.

[0096] A computer program is a list of instructions such as a particular application program and/or an operating system. The computer program may for instance include one or more of: a subroutine, a function, a procedure, an object method, an object implementation, an executable application, an applet, a servlet, a source code, an object code, a shared library/dynamic load library and/or other sequence of instructions designed for execution on a computer system.

[0097] The computer program may be stored internally on computer readable storage medium or transmitted to the computer system via a computer readable transmission medium. All or some of the computer program may be provided on transitory or non-transitory computer readable media permanently, removably or remotely coupled to an information processing system. The computer readable media may include, for example and without limitation, any number of the following: magnetic storage media including disk and tape storage media; optical storage media such as compact disk media (e.g., CD-ROM, CD-R, etc.) and digital video disk storage media; nonvolatile memory storage media including semiconductor-based memory units such as FLASH memory, EEPROM, EPROM, ROM; ferromagnetic digital memories; MRAM; volatile storage media including registers, buffers or caches, main memory, RAM, etc.; and data transmission media including computer networks, point-to-point telecommunication equipment, and carrier wave transmission media, just to name a few.

[0098] A computer process typically includes an executing (running) program or portion of a program, current program values and state information, and the resources used by the operating system to manage the execution of the process. An operating system (OS) is the software that manages the sharing of the resources of a computer and provides programmers with an interface used to access those resources. An operating system processes system data and user input, and responds by allocating and managing tasks and internal system resources as a service to users and programs of the system.

[0099] The computer system may for instance include at least one processing unit, associated memory and a number of input/output (I/O) devices. When executing the computer program, the computer system processes information according to the computer program and produces resultant output information via I/O devices.

[0100] The connections as discussed herein may be any type of connection suitable to transfer signals from or to the respective nodes, units or devices, for example via intermediate devices. Accordingly, unless implied or stated otherwise, the connections may for example be direct connections or indirect connections. The connections may be illustrated or described in reference to being a single connection, a plurality of connections, unidirectional connections, or bidirectional connections. However, different embodiments may vary the implementation of the connections. For example, separate unidirectional connections may be used rather than bidirectional connections and vice versa. Also, plurality of connections may be replaced with a single connection that transfers multiple signals serially or in a time multiplexed manner. Likewise, single connections carrying multiple signals may be separated out into various different connections carrying subsets of these signals. Therefore, many options exist for transferring signals.

[0101] Those skilled in the art will recognize that the boundaries between logic blocks are merely illustrative and that alternative embodiments may merge logic blocks or circuit elements or impose an alternate decomposition of functionality upon various logic blocks or circuit elements. Thus, it is to be understood that the architectures depicted herein are merely exemplary, and that in fact many other architectures can be implemented which achieve the same functionality.

[0102] Thus, any arrangement of components to achieve the same functionality is effectively “associated” such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality can be seen as “associated with” each other such that the desired functionality is achieved, irrespective of architectures or intermedial components. Likewise, any two components so associated can also be viewed as being “operably connected,” or “operably coupled,” to each other to achieve the desired functionality.

[0103] Furthermore, those skilled in the art will recognize that boundaries between the above described operations merely illustrative. The multiple operations may be combined into a single operation, a single operation may be distributed in additional operations and operations may be executed at least partially overlapping in time. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be altered in various other embodiments.

[0104] Also for example, the examples, or portions thereof, may implemented as soft or code representations of physical circuitry or of logical representations convertible into physical circuitry, such as in a hardware description language of any appropriate type.

[0105] Also, the invention is not limited to physical devices or units implemented in nonprogrammable hardware but can also be applied in programmable devices or units able to perform the desired device functions by operating in accordance with suitable program code, such as mainframes, minicomputers, servers, workstations, personal computers, notepads, personal digital assistants, electronic games, automotive and other embedded systems, cell phones and various other wireless devices, commonly denoted in this application as ‘computer systems’.

[0106] However, other modifications, variations and alternatives are also possible. The specifications and drawings are, accordingly, to be regarded in an illustrative rather than in a restrictive sense.

Audio Signal Processing Apparatuses and Methods

Inventors

Cpc classification

Classification Explorer

H04S2400/03

ELECTRICITY

Classification Explorer

G10L19/008

PHYSICS

Classification Explorer

H04S3/008

ELECTRICITY

Classification Explorer

H04S2400/01

ELECTRICITY

International classification

Classification Explorer

G10L19/008

PHYSICS

Classification Explorer

H04S3/00

ELECTRICITY

Abstract

Claims

Description