Downmixer and method for downmixing at least two channels and multichannel encoder and multichannel decoder

Abstract

A downmixer for downmixing at least two channels of a multichannel signal having the two or more channels includes: a processor for calculating a partial downmix signal from the at least two channels; a complementary signal calculator for calculating a complementary signal from the multichannel signal, the complementary signal being different from the partial downmix signal; and an adder for adding the partial downmix signal and the complementary signal to obtain a downmix signal of the multichannel signal.

Claims

1. A downmixer for downmixing at least two channels of a multichannel signal comprising two or more channels, comprising: a processor for calculating a partial downmix signal from the at least two channels using adding the two or more channels; a complementary signal calculator for calculating a complementary signal from the multichannel signal, the complementary signal being different from the partial downmix signal; and an adder for adding the partial downmix signal and the complementary signal to acquire a downmix signal of the multichannel signal.

2. The downmixer of claim 1, wherein the processor is configured to calculate the partial downmix signal so that a predefined energy or amplitude relation between the at least two channels of the multichannel signal and the partial downmix signal is fulfilled, when the at least two channels are in phase and so that an energy loss is created in the partial downmix signal with respect to the at least two channels, when the at least two channels are out of phase, and wherein the complementary signal calculator is configured to calculate the complementary signal so that the energy or amplitude loss of the partial downmix signal is partly or fully compensated by the adding of the partial downmix signal and the complementary signal in the adder.

3. The downmixer of claim 1, wherein the complementary signal calculator is configured to calculate the complementary signal so that the complementary signal comprises a coherence index of less than 0.7 with respect to the partial downmix signal, wherein a coherence index of 0.0 shows a full incoherence and a coherence index of 1.0 shows a full coherence.

4. The downmixer of claim 1, wherein the complementary signal calculator is configured to use, for calculating the complementary signal, one signal of the following groups of signals comprising a first channel of the at least two channels, a second channel of the at least two channels, a difference between the first channel and the second channel, a difference between the second channel and the first channel, a further channel of the multichannel signal, when the multichannel signal comprises more channels than the at least two channels, or a decorrelated first channel, a decorrelated second channel, a decorrelated further channel, a decorrelated difference involving the first channel and the second channel or a decorrelated partial downmix signal.

5. The downmixer of claim 1, wherein the processor is configured for: calculating time or frequency-dependent weighting factors for weighting a sum of the at least two channels in accordance with a predefined energy or amplitude relation between the at least two channels and a sum signal of the at least two channels; and comparing a calculated weighting factor to a predefined threshold; and using the calculated weighting factor for calculating the partial downmix signal, when the calculated weighting factor is in a first relation to the predefined threshold, or when the calculated weighting factor is in a second relation to the predefined threshold being different from the first relation, using the predefined threshold instead of the calculated weighting factor for calculating the partial downmix signal, or when the calculated weighting factor is in a second relation to the predefined threshold being different from the first relation, deriving a modified weighting factor using a modification function, wherein the modification function is so that the modified weighting factor is closer to the predefined threshold than the calculated weighting factor.

6. The downmixer of claim 1, wherein the processor is configured for: calculating time or frequency-dependent weighting factors for weighting a sum of the at least two channels in accordance with a predefined energy or amplitude relation between the at least two channels and a sum signal of the at least two channels; and deriving a modified weighting factor using a modification function, wherein the modification function is so that the modified weighting factor results in an energy of the partial downmix signal being smaller than an energy as defined by the predefined energy relation.

7. The downmixer of claim 1, wherein the processor is configured to weight as sum signal of the at least two channels using time or frequency-dependent weighting factors, wherein the weighting factors W.sub.1 are calculated so that the weighting factors comprise values being in a range of ±20% of values determined based on the following equation for a frequency bin k and a time index n: $W_1\left[k,n\right]=\frac{\sqrt{{.Math.L\left[k,n\right].Math.}^2+{.Math.R\left[k,n\right].Math.}^2}}{A\left(.Math.L\left[k,n\right].Math.+.Math.R\left[k,n\right].Math.\right)},$ for a subband b and a time index n: $W_1\left[b,n\right]=\frac{\sqrt{{.Math.}_{k\in b}{.Math.L\left[k,n\right].Math.}^2+{.Math.}_{k\in b}{.Math.R\left[k,n\right].Math.}^2}}{A\left({.Math.}_{k\in b}.Math.L\left[k,n\right].Math.+{.Math.}_{k\in b}.Math.R\left[k,n\right].Math.\right)},$ wherein A is a real valued constant, wherein L represents a first channel of the at least two channels and R represents a second channel of the at least two channels of the multichannel signal.

8. The downmixer of claim 1, wherein the complementary signal calculator is configured to use one channel of the at least two channels and to weight the used channel using time or frequency dependent complementary weighting factors W.sub.2, wherein the complementary weighting factors W.sub.2 are calculated so that the complementary weighting factors comprise values being in a range of ±20% of values determined based on the following equation for a frequency bin k and a time index n: $W_2\left[k,n\right]=\left(1-\frac{.Math.L\left[k,n\right]+R\left[k,n\right].Math.}{.Math.L\left[k,n\right].Math.+.Math.R\left[k,n\right].Math.}\right),$ for a subband b and a time index n: $W_2\left[b,n\right]=\left(1-\frac{{.Math.}_{k\in b}.Math.L\left[k,n\right]+R\left[k,n\right].Math.}{{.Math.}_{k\in b}.Math.L\left[k,n\right].Math.+{.Math.}_{k\in b}.Math.R\left[k,n\right].Math.}\right),$ wherein L represents a first channel of the two or more channels and R represents a second channel of the two or more channels of the multichannel signal.

9. The downmixer of claim 1, wherein the complementary signal calculator is configured to use a difference between a first channel of the two or more channels and a second channel of the two or more channels of the multichannel signal and to weight the difference using time and frequency dependent complementary weighting factors, wherein the complementary weighting factors are calculated so that the complementary weighting factors comprise values being in the range of ±20% of values determined based on the following equations:
W.sub.2=−p±√{square root over (p.sub.2−q)} where $p=\frac{.Math.W_1\left(L+R\right),L-R.Math.}{{.Math.L-R.Math.}^2}=\left(\frac{W_1\left({.Math.L.Math.}^2-{.Math.R.Math.}^2\right)}{{.Math.L-R.Math.}^2}\right)q=\frac{{\left(W_1.Math.L+R.Math.\right)}^2-\frac{{.Math.L.Math.}^2+{.Math.R.Math.}^2}{2}}{{.Math.L-R.Math.}^2}$ wherein L is the first channel of the two or more channels and R is the second channel of the two or more channels of the multichannel signal.

10. The downmixer of claim 1, wherein the complementary signal calculator is configured to use a difference between a first channel of the two or more channels and a second channel of the two or more channels of the multichannel signal and to weight the difference using time and frequency dependent complementary weighting factors, wherein the complementary weighting factors are calculated so that the complementary weighting factors comprise values being in the range of ±20% of values determined based on the following equations:
W.sub.2=−|p|+√{square root over (p.sup.2−q)} where $p=\frac{.Math.W_1\left(L+R\right),L-R.Math.}{{.Math.L-R.Math.}^2}=\left(\frac{W_1\left({.Math.L.Math.}^2-{.Math.R.Math.}^2\right)}{{.Math.L-R.Math.}^2}\right)q=\frac{{\left(W_1.Math.L+R.Math.\right)}^2-\frac{{.Math.L.Math.}^2+{.Math.R.Math.}^2}{2}}{{.Math.L-R.Math.}^2}$ wherein L is the first channel of the two or more channels and R is the second channel of the two or more channels of the multichannel signal.

11. The downmixer of claim 1, wherein the processor is configured: to calculate a sum signal from the at least two channels; to calculate weighting factors for weighting the sum signal in accordance with a predetermined relation between the sum signal and the at least two channels; to modify calculated weighting factors being higher than a predefined threshold, and to apply the modified weighting factors for weighting the sum signal to acquire the partial downmix signal.

12. The downmixer of claim 1, wherein the processor is configured to modify the calculated weighting factors to be in a range of ±20% of the predefined threshold, or to modify the calculated weighting factors so that the calculated weighting factors comprise values being in a range of ±20% of values determined based on the following equations: $W_1=\{\begin{array}{c}x\mathrm{if}x\le \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}+\left(1-\frac{1}{\sqrt{2}}\right)\left(1-\exp \left(\frac{\frac{1}{\sqrt{2}}-x}{1-\frac{1}{\sqrt{2}}}\right)\right)\mathrm{if}x>\frac{1}{\sqrt{2}}\end{array}$ wherein $x=\frac{1}{A}\left(\frac{\sqrt{{.Math.L\left[k,n\right].Math.}^2+{.Math.R\left[k,n\right].Math.}^2}}{\sqrt{{.Math.L+R.Math.}^2}}\right)$ wherein A is a real valued constant, L is a first channel of the two or more channels and R is a second channel of the two or more channels of the multichannel signal.

13. A method for downmixing at least two channels of a multichannel signal comprising two or more channels, comprising: calculating a partial downmix signal from the at least two channels using adding the two or more channels; calculating a complementary signal from the multichannel signal, the complementary signal being different from the partial downmix signal; and adding the partial downmix signal and the complementary signal to acquire a downmix signal of the multichannel signal.

14. A multichannel encoder, comprising: a parameter calculator for calculating multichannel parameters from at least two channels of a multichannel signal comprising the two or more than two channels, and a downmixer of claim 1; and an output interface for outputting or storing an encoded multichannel signal comprising one or more downmix signals and/or the multichannel parameters.

15. A method for encoding a multichannel signal, comprising: calculating multichannel parameters from at least two channels of a multichannel signal comprising two or more than two channels; downmixing in accordance with the method of claim 13; and outputting or storing an encoded multichannel signal comprising the one or more downmix signals and the multichannel parameters.

16. A non-transitory digital storage medium having a computer program stored thereon to perform the method for downmixing at least two channels of a multichannel signal comprising two or more channels, comprising: calculating a partial downmix signal from the at least two channels using adding the two or more channels; calculating a complementary signal from the multichannel signal, the complementary signal being different from the partial downmix signal; and adding the partial downmix signal and the complementary signal to acquire a downmix signal of the multichannel signal, when said computer program is run by a computer.

17. A non-transitory digital storage medium having a computer program stored thereon to perform the method for encoding a multichannel signal, comprising: calculating multichannel parameters from at least two channels of a multichannel signal comprising two or more than two channels; downmixing in accordance with the method as claimed in claim 13; and outputting or storing an encoded multichannel signal comprising one or more downmix signals and the multichannel parameters, when said computer program is run by a computer.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Embodiments of the present invention will be detailed subsequently referring to the appended drawings, in which:

(2) FIG. 1 is a block diagram of a downmixer in accordance with an embodiment;

(3) FIG. 2a is a flow chart for illustrating the energy loss compensation feature;

(4) FIG. 2b is a block diagram illustrating an embodiment of the complementary signal calculator;

(5) FIG. 3 is a schematic block diagram illustrating a downmixer operating in the spectral domain and having an adder output connected to different alternatives or cumulative processing elements;

(6) FIG. 4 illustrates an advantageous procedure implemented by the processor for processing the partial downmix signal;

(7) FIG. 5 illustrates a block diagram of a multichannel encoder in an embodiment;

(8) FIG. 6 illustrates a block diagram of a multichannel decoder;

(9) FIG. 7a illustrates the singularity point of the sum component in accordance with conventional technology;

(10) FIG. 7b illustrates equations for calculating the downmix in the conventional-technology example of FIG. 7a;

(11) FIG. 8a illustrates an energy relation of a downmixing in accordance with an embodiment;

(12) FIG. 8b illustrates equations for the embodiment of FIG. 8a;

(13) FIG. 8c illustrates alternative equations with a more coarse frequency resolution of the weighting factors;

(14) FIG. 8d illustrates the downmix phase for the FIG. 8a embodiment;

(15) FIG. 9a illustrates a gain limitation chart for the sum signal in a further embodiment;

(16) FIG. 9b illustrates an equation for calculating the downmix signal M for the embodiment of FIG. 9a;

(17) FIG. 9c illustrates a manipulation function for calculating a manipulated weighting factor for the calculation of the sum signal of the embodiment of FIG. 9a;

(18) FIG. 9d illustrates the calculations of the weighting factors for the calculation of the complementary signal W.sub.2 for the embodiment of FIG. 9a-FIG. 9c;

(19) FIG. 9e illustrates an energy relation of the downmixing of FIG. 9a-9d;

(20) FIG. 9f illustrates the gain W.sub.2 for the embodiment of FIGS. 9a-9e;

(21) FIG. 10a illustrates a downmix energy for a further embodiment;

(22) FIG. 10b illustrates equations for the calculation of the downmix signal and the first weighting factor W.sub.1 for the embodiment of FIG. 10a;

(23) FIG. 10c illustrates procedures for calculating the second or complementary signal weighting factors for the embodiment of FIG. 10a-10b;

(24) FIG. 10d illustrates equations for the parameters p and q of the FIG. 10c embodiment;

(25) FIG. 10e illustrates the gain W.sub.2 as function of ILD and IPD of the downmixing with respect to the embodiment illustrated in FIG. 10a to 10d.

DETAILED DESCRIPTION OF THE INVENTION

(26) FIG. 1 illustrates a downmixer for downmixing at least two channels of a multichannel signal 12 having the two or more channels. Particularly, the multichannel signal can only be a stereo signal with a left channel L and a right channel R, or the multichannel signal can have three or even more channels. The channels can also include or consist of audio objects. The downmixer comprises a processor 10 for calculating a partial downmix signal 14 from the at least two channels from the multichannel signal 12. Furthermore, the downmixer comprises a complementary signal calculator 20 for calculating a complementary signal from the multichannel signal 12, wherein the complementary signal 22 is output by block 20 is different from the partial downmix signal 14 output by block 10. Additionally, the downmixer comprises an adder 30 for adding the partial downmix signal and the complementary signal to obtain a downmix signal 40 of the multichannel signal 12. Generally, the downmix signal 40 has only a single channel or, alternatively, has more than one channel.

(27) Generally, however, the downmix signal has fewer channels than are included in the multichannel signal 12. Thus, when the multichannel signal has, for example, five channels, the downmix signal may have four channels, three channels, two channels or a single channel.

(28) The downmix signal with one or two channels is advantageous as compared to a downmix signal having more than two channels. In the case of a two channel signal as the multichannel signal 12, the downmix signal 40 only has a single channel.

(29) In an embodiment, the processor 10 is configured to calculate the partial downmix signal 14 so that the predefined energy-related or amplitude-related relation between the at least two channels and the partial downmix signal is fulfilled, when the at least two channels are in phase and so that an energy loss is created in the partial downmix signal with respect to the at least two channels, when the at least two channels are out of phase. Embodiments and examples for the predefined relation are that the amplitudes of the downmix signal are in a certain relation to the amplitudes of the input signals or the subband-wise energies, for example, of the downmix signal are in a predefined relation to the energies of the input signals. One particularly interesting relation is that the energy of the downmix signal either over the full bandwidth or in subbands is equal to an average energy of the two downmix signals or the more than two downmix signals. Thus, the relation can be with respect to energy, or with respect to amplitude. Furthermore, the complementary signal calculator 20 of FIG. 1 is configured to calculate the complementary signal 22 so that the energy loss of the partial downmix signal as illustrated at 14 in FIG. 1 is partly or fully compensated by adding the partial downmix signal 14 and the complementary signal 22 in the adder 30 of FIG. 1 to obtain the downmix signal.

(30) Generally, embodiments are based on the controlled energy or amplitude-equalization of the sum signal mixed with the complementary signal also derived from the input channels.

(31) Embodiments are based on a controlled energy or amplitude-equalization of the sum signal mixed with a complementary signal also derived from the input channels. The energy-equalization of the sum signal is controlled for avoiding problems at the singularity point but also to minimize significantly signal impairments due to large fluctuations of the gain. The complementary signal is there to compensate the remaining energy loss or at least a part of it. The general form of the new downmix can be expressed as
M[k,n]=W.sub.1[k,n](L[k,n]+R[k,n])+W.sub.2[k,n]S[k,n]

(32) where the complementary signal S[k,n] are ideally orthogonal as much as possible to the sum signal, but can be in practice chosen as
S[k,n]=L[k,n]
or
S[k,n]=R[k,n]
or
S[k,n]=L[k,n]−R[k,n].

(33) In all cases, the downmixing generates first the sum channel L+R as it is done in conventional passive and active downmixing approaches. The gain W.sub.1 [k, n] aims at equalizing the energy of the sum channel for either matching the average energy or the average amplitude of the input channels. However, unlike conventional active downmixing approaches, W.sub.1[k, n] is limited to avoid instability problems and to avoid that the energy relations are restored based on an impaired sum signal.

(34) A second mixing is done with the complementary signal. The complementary signal is chosen such that its energy doesn't vanish when L[k, n] and R[k, n] are out-of-phase. W.sub.2[k, n] compensates the energy-equalization due to the limitation introduced in W.sub.1[k, n].

(35) As illustrated, the complementary signal calculator 20 is configured to calculate the complementary signal so that the complementary signal is different from the partial downmix signal. In quantities, it is advantageous that a coherence index of the complementary signal is less than 0.7 with respect to the partial downmix signal. In this scale, a coherence index of 0.0 shows a full incoherence and a coherence index of 1.0 shows a full coherence. Thus, a coherence index of less than 0.7 has proven to be useful so that the partial downmix signal and the complementary signal are sufficiently different from each other. However, coherence indices of less than 0.5 and even less than 0.3 are more advantageous.

(36) FIG. 2a illustrates a procedure performed by the processor. Particularly, as illustrated in item 50 of FIG. 2a, the processor calculates the partial downmix signal with an energy loss with respect the at least two channels that represent the input into the processor. Furthermore, the complementary signal calculator 52 calculates the complementary signal 22 of FIG. 1 to partly or fully compensate for the energy loss.

(37) In an embodiment illustrated in FIG. 2b, the complementary signal calculator comprises a complementary signal selector or complementary signal determiner 23, a weighting factor calculator 24 and a weighter 25 to finally obtain the complementary signal 22. Particularly, the complementary signal selector or complementary signal determiner 23 is configured to use, for calculating the complementary signal, one signal of a group of signals consisting of a first channel such as L, a second channel such as R, a difference between the first channel and the second channel as indicated L-R in FIG. 2b. Alternatively, the difference can also be R-L. A further signal used by the complementary signal selector 23 can be a further channel of the multichannel signal, i.e., a channel that is not selected to be by the processor for calculating the partial downmix signal. This channel can, for example, be a center channel, or a surround channel or any other additional channel comprising an object. In other embodiments, the signal used by the complementary signal selector is a decorrelated first channel, a decorrelated second channel, a decorrelated further channel or even the decorrelated partial downmix signal as calculated by the processor 14. In advantageous embodiments, however, either the first channel such as L or the second channel such as R or, even more advantageously, the difference between the left channel and the right channel or the difference between the right channel and the left channel are advantageous for calculating the complementary signal.

(38) The output of the complementary signal selector 23 is input into a weighting factor calculator 24. The weighting factor calculator additionally typically receives the two or more signals to be combined by the processor 10 and the weighting factor calculator calculates weights W.sub.2 illustrated at 26. Those weights together with the signal used and determined by the complementary signal selector 23 are input into the weighter 25, and the weighter then weights the corresponding signal output from block 23 using the weighting factors from block 26 to finally obtain the complementary signal 22.

(39) The weighting factors can only be time-dependent, so that for a certain block or frame in time, a single weighting factor W.sub.2 is calculated. In other embodiments, however, it is advantageous to use time and frequency dependent weighting factors W.sub.2 so that, for a certain block or frame of the complementary signal, not only a single weighting factor for this time block is available, but a set of weighting factors W.sub.2 for a set of different frequency values or spectral bins of the signal generated or selected by block 23.

(40) A corresponding embodiment for time and frequency dependent weighting factors not only for usage of the complementary signal calculator 20, but also for usage of the processor 10 is illustrated in FIG. 3.

(41) Particularly, FIG. 3 illustrates a downmixer in an advantageous embodiment that comprises a time-spectrum converted 60 for converting time domain input channels into frequency domain input channels, where each frequency domain input channel has a sequence of spectra. Each spectrum has a separate time index n and, within each spectrum, a certain frequency index k refers to a frequency component uniquely associated with the frequency index. Thus, in an example, when a block has 512 spectral values, then the frequency k runs from 0 to 511 in order to uniquely identify each one of the 512 different frequency indices.

(42) The time-spectrum converter 60 is configured for applying an FFT and, advantageously, an overlapping FFT so that the sequence of spectra obtained by block 60 are related to overlapping blocks of the input channels. However, non-overlapping spectral conversion algo-rithms and other conversions apart from an FFT such as DCT or so can be used as well.

(43) Particularly, the processor 10 of FIG. 1 comprises a first weighting factor calculator 15 for calculating weights W.sub.1 for individual spectral indices k or weighting factors W.sub.1 for sub-bands b, where a subband is broader than a spectral value with respect to frequency, and typically, comprises two or more spectral values.

(44) The complementary signal calculator 20 of FIG. 1 comprises a second weighting factor calculator that calculates the weighting factors W.sub.2. Thus, item 24 can be similarly constructed as item 24 of FIG. 2b.

(45) Furthermore, the processor 10 of FIG. 1 calculating the partial downmix signal comprises a downmix weighter 16 that receives, as an input, the weighting factors W.sub.1 and that outputs the partial downmix signal 14 that is forwarded to the adder 30. Furthermore, the embodiment illustrated in FIG. 3 additionally comprises the weighter 25 already described with respect FIG. 2b that receives, as an input, the second weighting factors W.sub.2.

(46) The adder 30 outputs the downmix signal 40. The downmix 40 can be used in several different occurrences. One way to use the downmix signal 40 is to input it into a frequency domain downmix encoder 64 illustrated in FIG. 3 that outputs an encoded downmix signal. An alternative procedure is to insert the frequency domain representation of the downmix signal 40 into a spectrum-time converter 62 in order to obtain, at the output of block 62, a time domain downmix signal. A further embodiment is to feed the downmix signal 40 into a further downmix processor 66 that generates some kind of process downmix channel such as a transmitted downmix channel, a stored downmix channel, or a downmix channel that has performed some kind of equalization, a gain variation etc.

(47) In embodiments, the processor 10 is configured for calculating time or frequency-dependent weighting factors W.sub.1 as illustrated by block 15 in FIG. 3 for a weighting a sum of the at least two channels in accordance with a predefined energy or amplitude relation between the at least two channels and a sum signal of the at least two channels. Furthermore, subsequent to this procedure that is also illustrated in item 70 of FIG. 4, the processor is configured to compare a calculated weighting factor W.sub.1 for a certain frequency index k and a certain time index n or for a certain spectral subband b and a certain time index n to a predefined threshold as indicated at block 72 of FIG. 4. This comparison is performed advantageously for each spectral index k or for each subband index b or for each time index n and advantageously for one spectrum index k or b and for each time index n. When the calculated weighting factor is in a first relation to the predefined threshold such as below the threshold as illustrated at 73, then the calculated weighting factor W.sub.1 is used as indicated at 74 in FIG. 4. When, however, the calculated weighting factor is in a second relation to the predefined threshold that is different from the first relation to the predefined threshold such as above the threshold as indicated at 75, the predefined threshold is used instead of the calculated weighting factor for calculating the partial downmix signal in block 16 of FIG. 3 for example. This is a “hard” limitation of W.sub.1. In other embodiments, a kind of a “soft limitation” is performed. In this embodiment, a modified weighting factor is derived using a modification function, wherein the modification function is so that the modified weighting factor is closer to the predefined threshold then the calculated weighting factor.

(48) The embodiment in FIG. 8a-8d uses a hard limitation, while the embodiment in FIG. 9a-9f and the embodiment in FIG. 10a-10e use a soft limitation, i.e., a modification function.

(49) In a further embodiment, the procedure in FIG. 4 is performed with respect to block 70 and block 76, but a comparison to a threshold as discussed with respect to block 72 is not performed. Subsequent to the calculation in block 70, a modified weighting factor is derived using the modification function of the above description of block 76, wherein the modification function is so that a modified weighting factor results in an energy of the partial downmix signal being smaller than an energy of the predefined energy relation. Advantageously, the modification function that is applied without a specific comparison is so that it limits, for high values of W.sub.1 the manipulated or modified weighting factor to a certain limit or only has a very small increase such as a log or In function or so that, though not being limited to a certain value only has a very slow increase anymore so that stability problems as discussed before are substantially avoided or at least reduced.

(50) In an advantageous embodiment illustrated in FIG. 8a-8d, the downmix is given by:
M[k,n]=W.sub.1[k,n](L[k,n]+R[k,n])+W.sub.2[k,n]L[k,n]

(51) where

(52) $W_1\left[k,n\right]=\frac{\sqrt{{.Math.L\left[k,n\right].Math.}^2+{.Math.R\left[k,n\right].Math.}^2}}{A\left(.Math.L\left[k,n\right].Math.+.Math.R\left[k,n\right].Math.\right)}{W}_2\left[k,n\right]=\left(1-\frac{.Math.L\left[k,n\right]+R\left[k,n\right].Math.}{.Math.L\left[k,n\right].Math.+.Math.R\left[k,n\right].Math.}\right)$

(53) In the above equation, A is a real valued constant advantageously being equal to the square root of 2, but A can have different values between 0.5 or 5 as well. Depending on the application, even values different from the above mentioned values can be used as well.

(54) Given that
|L[k,n]+R[k,n]|≤|L[k,n]|+|R[k,n]|,
W.sub.1[k,n] and W.sub.2[k,n] are positive and W.sub.1[k,n] is limited to

(55) $\frac{\sqrt{2}}{2A}$
or e.g. 0.5.

(56) The mixing gains can be computed bin-wise for each index k of the STFT as described in the previous formulas or can be computed band-wise for each non-overlapping sub-band gathering a set of indices b of the STFT. The gains are calculated based on the following equation:

(57) $W_1\left[b,n\right]=\frac{\sqrt{{.Math.}_{k\in b}{.Math.L\left[k,n\right].Math.}^2+{.Math.}_{k\in b}{.Math.R\left[k,n\right].Math.}^2}}{\sqrt{2}\left({.Math.}_{k\in b}.Math.L\left[k,n\right].Math.+{.Math.}_{k\in b}.Math.R\left[k,n\right].Math.\right)}{W}_2\left[b,n\right]=\left(1-\frac{{.Math.}_{k\in b}.Math.L\left[k,n\right]+R\left[k,n\right].Math.}{{.Math.}_{k\in b}.Math.L\left[k,n\right].Math.+{.Math.}_{k\in b}.Math.R\left[k,n\right].Math.}\right)$

(58) Since the energy preservation during the equalization is not a hard constraint, the energy of the resulting downmix signal varies compared the average energy of the input channel. The energy relation depends on the ILD and IPD as illustrated in FIG. 8a.

(59) In contrast to the simple active downmixing method, which preserves a constant relation between the output energy and the average energy of the input channels, the new downmix signal does not show any singularity as illustrated in FIG. 8d. Indeed, in FIG. 7a a jump of a magnitude Pi (180°), can be observed at IP=Pi and ILD=0 dB, while in FIG. 8d, the jump is of 2Pi (360°), which corresponds to a continuous change in the unwrapped phase domain.

(60) Listening test results confirm that the new down-mix method results in significantly less instabilities and impairments for a large range of stereo signals than conventional active downmixing.

(61) In this context, FIG. 8a illustrates, along the x-axis, the inter-channel level difference between an original left and an original right channel in dB. Furthermore, the downmix energy is indicated in a relative scale between 0 and 1.4 along the y-axis and the parameter is the inter-channel phase difference IPD. Particularly, it appears that the energy of the resulting downmix signal varies particularly dependent on the phase between the channels and, for a phase of Pi (180°), i.e., for an out of phase situation, the energy variation is, at least for positive inter-channel level differences, in good shape. FIG. 8b illustrates equations for calculating the downmix signal M and it also becomes clear that, as the complementary signal, the left channel is selected. FIG. 8c illustrates weighting factors W.sub.1 and W.sub.2 not only for individual spectral indices, but for subbands where a set of indices from the STFT, i.e., at least two spectral values k are added together to obtain a certain sub-band.

(62) Compared to the conventional technology illustrated in FIG. 7a and FIG. 7b, any singularity is not included anymore when FIG. 8d is compared to FIG. 7a.

(63) FIG. 9a-9f illustrates a further embodiment, where the downmix is calculated using the difference between left and right signals L and R as the basis for the complementary signal. Particularly, in this embodiment.
M[k,n]=W.sub.1[k,n](L[k,n]+R[k,n])+W.sub.2[k,n](L[k,n]−R[k,n])

(64) where the set of gains W.sub.1[k,n] and W.sub.2[k,n] are computed such that the energy relation between the down-mixed signal and the input channels holds in every condition.

(65) First the gain W.sub.1[k,n] is computed for equalizing the energy till a given limit, where A is again a real valued number equal to √{square root over (2)} or different from this value:

(66) $x=\frac{1}{A}\left(\frac{\sqrt{{.Math.L\left[k,n\right].Math.}^2+{.Math.R\left[k,n\right].Math.}^2}}{\sqrt{{.Math.L+R.Math.}^2}}\right)$$W_1=\{\begin{array}{cc}x& \mathrm{if}x\le \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}+\left(1-\frac{1}{\sqrt{2}}\right)\left(1-\exp \left(\frac{\frac{1}{\sqrt{2}}-x}{1-\frac{1}{\sqrt{2}}}\right)\right)& \mathrm{if}x>\frac{1}{\sqrt{2}}\end{array}$

(67) As a consequence, the gain W.sub.1 [k, n] of the sum signal is limited to the range [0, 1] as shown in FIG. 9a. In the equation for x, an alternative implementation is to use the denominator without a square root.

(68) If the two channels have an IPD greater than pi/2, W.sub.1 can no more compensate for the loss of energy, and it will be then coming from the gain W.sub.2. W.sub.2 is computed as one of the roots of the following quadratic equation:

(69) $E_M={.Math.M.Math.}^2={.Math.W_1\left(L+R\right)+W_{2}L.Math.}^2=\frac{L^2+R^2}{2}$

(70) The roots of the equation are given by:
W.sub.2=−p+√{square root over (p.sub.2−q)},
where

(71) $p=\frac{.Math.W_1\left(L+R\right),L-R.Math.}{{.Math.L-R.Math.}^2}=\left(\frac{W_1\left({.Math.L.Math.}^2-{.Math.R.Math.}^2\right)}{{.Math.L-R.Math.}^2}\right)$$q=\frac{{\left(W_1.Math.L+R.Math.\right)}^2-\frac{{.Math.L.Math.}^2+{.Math.R.Math.}^2}{2}}{{.Math.L-R.Math.}^2}$

(72) One of the two roots can be then selected. For both roots, the energy relation is preserved for all conditions as shown in FIG. 9e.

(73) If the two channels have an IPD greater than pi/2, W.sub.1 can no more compensate for the loss of energy, and it will be then coming from the gain W.sub.2. W.sub.2 is computed as one of the roots of the following quadratic equation:

(74) $E_M={.Math.M.Math.}^2={.Math.W_1\left(L+R\right)+W_{2}L.Math.}^2=\frac{L^2+R^2}{2}$

(75) The roots of the equation are given by:
W.sub.2=−p±√{square root over (p.sup.2−q)},
where

(76) $p=\frac{.Math.W_1\left(L+R\right),L-R.Math.}{{.Math.L-R.Math.}^2}=\left(\frac{W_1\left({.Math.L.Math.}^2-{.Math.R.Math.}^2\right)}{{.Math.L-R.Math.}^2}\right)$$q=\frac{{\left(W_1.Math.L+R.Math.\right)}^2-\frac{{.Math.L.Math.}^2+{.Math.R.Math.}^2}{2}}{{.Math.L-R.Math.}^2}$

(77) One of the two roots can be then selected. For both roots, the energy relation is preserved for all conditions as shown in FIG. 9f.

(78) Advantageously, the root with the minimum absolute value is adaptively selected for W.sub.2[k,n].. Such an adaptive selection will result in a switch from one root to another for ILD=0 dB, which once again can create a discontinuity.

(79) In contrast to the state-of-the art, this approach solves the comb-filtering effect of the downmix and spectral bias without introducing any singularity. It maintains the energy relations in all conditions but introduces more instabilities compared to the advantageous embodiment.

(80) Thus, FIG. 9a illustrates a comparison of the gain limitation obtained by the factors W.sub.1 of the sum signal in the calculation of the partial downmix signal of this embodiment. Particularly, the straight line is the situation before normalization or before modification of the value as discussed before with respect to block 76 of FIG. 4. And, the other line that approaches a value of 1 for the modification function as a function of the weighting factor W.sub.1. It becomes clear that an influence of the modification function occurs at values above 0.5 but the deviation only becomes really visible for values W.sub.1 of about 0.8 and greater.

(81) FIG. 9b illustrates the equation implemented by the FIG. 1 block diagram for this embodiment.

(82) Furthermore, FIG. 9c illustrates how the values W.sub.1 are calculated and, therefore, FIG. 9a illustrates the functional situation of FIG. 9c. Finally. FIG. 9d illustrates the calculation of W.sub.2, i.e., the weighting factors used by the complementary signal generator 20 of FIG. 1.

(83) FIG. 9e illustrates that the downmix energy is the same and equal to 1 for all phase differences between the first and the second channels and for all level differences ALD between the first and the second channels.

(84) However, FIG. 9f illustrates the discontinuities incurred by the calculations of the rules of the equation for Eu of FIG. 9d due to the fact there is a denominator in the equation for p and the equation for q illustrated in FIG. 9d that can become 0.

(85) FIGS. 10a-10e illustrate a further embodiment that can be seen as a compromise between the two earlier described alternatives.

(86) The downmixing is given by;
M=W.sub.1[k](L[k]+R[k])+W.sub.2[k](L[k]−R[k])

(87) Where

(88) 0$x=\frac{1}{A}\left(\frac{\sqrt{{.Math.L\left[k,n\right].Math.}^2+{.Math.R\left[k,n\right].Math.}^2}}{\sqrt{{\left(L+R\right)}^2}}\right)$$W_1=\{\begin{array}{cc}x& \mathrm{if}x\le \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}+\left(1-\frac{1}{\sqrt{2}}\right)\left(1-\exp \left(\frac{\frac{1}{\sqrt{2}}-x}{1-\frac{1}{\sqrt{2}}}\right)\right)& \mathrm{if}x>\frac{1}{\sqrt{2}}\end{array}$

(89) In the equation for x, an alternative implementation is to use the denominator without a square root.

(90) In this case the quadratic equation to solve is:

(91) $E_M={.Math.M.Math.}^2={.Math.W_1\left(L+R\right)+W_{2}L.Math.}^2={\left(\frac{.Math.L.Math.+.Math.R.Math.}{2}\right)}^2$

(92) This time the gain W.sub.2 is not exactly taken as one of the roots of the quadratic equation but rather:
W.sub.2=−|p|+√{square root over (p.sup.2−q)}
where

(93) $p=\frac{.Math.W_1\left(L+R\right),L-R.Math.}{{.Math.L-R.Math.}^2}=\left(\frac{W_1\left({.Math.L.Math.}^2-{.Math.R.Math.}^2\right)}{{.Math.L-R.Math.}^2}\right)$$q=\frac{{\left(W_1.Math.L+R.Math.\right)}^2-{\left(\frac{.Math.L.Math.+.Math.R.Math.}{2}\right)}^2}{{.Math.L-R.Math.}^2}$

(94) As a result, the energy relation is not preserved all the time as shown in FIG. 10a. On the other hand the gain W.sub.2 doesn't show any discontinuities in FIG. 10e and compared to the second embodiment instability problems are reduced.

(95) Thus, FIG. 10a illustrates the energy relation of this embodiment illustrated by FIGS. 10a-10e where, once again, the downmix energy is illustrated at the y-axis and the inter-channel level difference is illustrated at the x-axis. FIG. 10b illustrates the equations applied by FIG. 1 and the procedures performed for calculating the first weighting factors W.sub.1 as illustrated with respect to block 76. Furthermore, FIG. 10c illustrates the alternative calculation of W.sub.2 with respect to the embodiment of FIG. 9a-9f. Particularly, p is subjected to an absolute value function which appears when comparing FIG. 10c to the similar equation in FIG. 9d.

(96) FIG. 10d then once again shows the calculation of p and q and FIG. 10d roughly corresponds to the equations in FIG. 10d at the bottom.

(97) FIG. 10e illustrates the energy relation of this new downmixing in accordance with the embodiment illustrated in FIG. 10a-10d, and it appears that the gain W.sub.2 only approaches a maximum value of 0.5.

(98) Although the preceding description and certain FIGS. provide detailed equations, it is to be noted that advantages are already obtained even when the equations are not calculated exactly, but when the equations are calculated, but the results are modified. Particularly, the functionalities of the first weighting factor calculator 15 and the second weighting factor calculator 24 of FIG. 3 are performed so that the first weighting factors or the second weighting factors have values being in a range of ±20% of values determined based on the above given equations. In the advantageous embodiment, the weighting factors are determined to have values being in a range of ±10% of the values determined by the above equations. In even more advantageous embodiments, the deviation is only ±1% and in the most advantageous embodiments, the results of the equations are exactly taken. But, as stated, advantages of the present invention are even obtained, when deviations of ±20% from the above described equations are applied.

(99) FIG. 5 illustrates an embodiment of a multichannel encoder, in which the inventive downmixer as discussed before with respect to FIGS. 1-4, 8a-10e can be used. Particularly, the multichannel encoder comprises a parameter calculator 82 for calculating multichannel parameters 84 from at least two channels of the multichannel signal 12 having the two or more channels. Furthermore, the multichannel encoder comprises the downmixer 80 that can be implemented as discussed before and that provides one or more downmix channels 40. Both, the multichannel parameters 84 and the one or more downmix channels 40 are input into an output interface 86 for outputting an encoded multichannel signal comprising the one or more downmix channels and/or the multichannel parameters. Alternatively, the output interface can be configured for storing or transmitting the encoded multichannel signal to, for example, a multichannel decoder illustrated in FIG. 6. The multichannel decoder illustrated in FIG. 6 receives, as an input, the encoded multichannel signal 88. This signal is input into an input interface 90, and the input interface 90 outputs, on the first hand, the multichannel parameters 92 and, on the other hand, the one or more downmix channels 94. Both data items, i.e., the multichannel parameters 92 and downmix channels 94 are input into a multichannel reconstructor 96 that reconstructs, at its output, an approximation of the original input channels and, in general, outputs output channels that may comprise or consist of output audio objects or anything like that as indicated by reference numeral 98. Particularly, the multichannel encoder in FIG. 5 and the multichannel decoder in FIG. 6 together represent an audio processing system where the multichannel encoder is operative as discussed with respect to FIG. 5 and where the multichannel decoder is, for example, implemented as illustrated in FIG. 6 and is, in general, configured for decoding the encoded multichannel signal to obtain a reconstructed audio signal illustrated at 98 in FIG. 6. Thus, the procedures illustrated with respect to FIG. 5 and FIG. 6 additionally represent a method of processing an audio signal comprising a method of multichannel encoding and a corresponding method of multichannel decoding.

(100) An inventively encoded audio signal can be stored on a digital storage medium or a non-transitory storage medium or can be transmitted on a transmission medium such as a wire-less transmission medium or a wired transmission medium such as the Internet.

(101) Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

(102) Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed.

(103) Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

(104) Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

(105) Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier or a non-transitory storage medium.

(106) In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

(107) A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.

(108) A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.

(109) A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

(110) A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

(111) In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are advantageously performed by any hardware apparatus.

(112) While this invention has been described in terms of several embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and com-positions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations and equivalents as fall within the true spirit and scope of the present invention.

REFERENCES

(113) [1] U.S. Pat. No. 7,343,281 B2, “PROCESSING OF MULTI-CHANNEL SIGNALS”, Koninklijke Philips Electronics N. V., Eindhoven (NL) [2] Samsudin, E. Kumiawati, Ng Boon Poh, F. Sattar, and S. George, “A Stereo to Mono Downmixing Scheme for MPEG-4 Parametric Stereo Encoder,” in IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 5, 2006, pp. 529-532. [3] T. M. N. Hoang, S. Ragot, B. Kövesi, and P. Scalart, “Parametric Stereo Extension of ITU-T G. 722 Based on a New Downmixing Scheme,” IEEE International Workshop on Multimedia Signal Processing (MMSP) (2010). [4] W. Wu, L. Miao, Y. Lang, and D. Virette, “Parametric Stereo Coding Scheme with a New Downmix Method and Whole Band Inter Channel Time/Phase Differences,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2013, pp. 556-560. [5] Alexander Adami, Emanuël A. P. Habets, Jürgen Herre, “DOWN-MIXING USING COHERENCE SUPPRESSION”, 2014 IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP) [6] Vilkamo, Juha; Kuntz, Achim; Füg, Simone, “Reduction of Spectral Artifacts in Multi-channel Downmixing with Adaptive Phase Alignment”, AES Aug. 22, 2014