METHOD AND APPARATUS FOR CONTROLLING MULTICHANNEL AUDIO FRAME LOSS CONCEALMENT

Abstract

A method of approximating a lost or corrupted multichannel audio frame of a multichannel audio signal in a decoding device is provided. The device may generate a down-mix error concealment frame and transform the frame into a frequency domain to generate a transformed down-mix error concealment frame. The device may decorrelate the transformed frame to generate a decorrelated concealment frame. The device may obtain a residual signal spectrum of a stored residual signal of a previously received multichannel audio signal frame and generate an energy adjusted decorrelated residual signal concealment frame using the residual signal spectrum. The device may obtain a set of multi-channel audio substitution parameters and provide the frames and substitution parameters to an audio synthesis component to generate a synthesized multichannel audio frame. The device performs an inverse frequency domain transformation of the audio frame to generate a substitution frame for the lost or corrupted audio frame.

Claims

1. A method of approximating a lost or corrupted multichannel audio frame of a received multichannel audio signal in a decoding device comprising a processor, the method comprising the following operations performed by the processor: generating a down-mix error concealment frame; transforming the down-mix error concealment frame into a frequency domain to generate a transformed down-mix error concealment frame; decorrelating the transformed down-mix concealment frame to generate a decorrelated concealment frame; obtaining a residual signal spectrum of a stored residual signal of a previously received multichannel audio signal frame; generating an energy adjusted decorrelated residual signal concealment frame using the residual signal spectrum; obtaining a set of multi-channel audio substitution parameters; providing the transformed down-mix error concealment frame, the energy-adjusted decorrelated residual concealment frame, and multi-channel audio substitution parameters to a parametric multi-channel audio synthesis component to generate a synthesized multichannel audio frame; and performing an inverse frequency domain transformation of the synthesized multichannel audio frame to generate a substitution frame for the lost or corrupted multichannel audio frame.

2. The method of claim 1 wherein the set of multi-channel audio substitution parameters is obtained by repeating the parameters from the previously received multi-channel audio signal frame.

3. The method of claim 1 further comprising: generating multi-channel audio signals based on the substitution frame; and outputting the multi-channel audio signals towards at least one loudspeaker for playback.

4. The method of claim 1 wherein obtaining the residual signal spectrum comprises retrieving the residual signal spectrum from a storage device.

5. The method of claim 1 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises: phase-shifting peak sinusoid components of the residual signal spectrum; and adjusting an energy of a noise spectrum of non-peak sinusoid components of the residual signal spectrum of the stored residual signal.

6. The method of claim 1 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises: detecting peak frequencies of the residual signal spectrum of the stored residual signal on a fractional frequency scale; associating each peak frequency with a number of peak frequency bins representing the peak frequency; applying a phase adjustment to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and populating remaining bins of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.

7. The method of claim 1 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises: detecting whether there are peak frequencies in the residual signal spectrum of the stored residual signal on a fractional frequency scale; responsive to detecting no peak frequencies in the residual signal spectrum: populating each bin of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the bins to match an energy level of a noise spectrum of the residual signal spectrum; responsive to detecting peak frequencies in the residual signal spectrum: associating each peak frequency with a number of peak frequency bins representing the peak frequency; applying a phase adjustment to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and populating remaining bins of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.

8. The method of claim 6 wherein adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum comprises matching the energy level on a band basis.

9. The method of claim 6 wherein adjusting the energy level comprises combining a phase of bins of the decorrelated concealment frame with a magnitude of the bins of the residual signal concealment spectrum.

10. The method of claim 9 wherein combining the phase comprises applying an approximate phase adjustment by matching a sign and an order of a real component and an imaginary component of the residual signal concealment spectrum to the decorrelated concealment frame.

11. The method of claim 7 wherein matching the energy level comprises: calculating an energy matching gain factor g.sub.k as $g_k=\sqrt{\frac{{X_{\hat{R},\mathrm{mem}}\left(k\right)}^2}{{X_{D,\mathrm{ECU}}\left(m,k\right)}^2}}$ and populating the remaining bins with an energy adjusted decorrelated residual concealment frame
X.sub.R,ECU(m,k)=g.sub.kX.sub.D,ECU(m,k), k.Math.G.sub.i.

12. The method of claim 7 wherein a band b spans a range of bins k.sub.start(b) . . . k.sub.end(b) and matching the energy level comprises: calculating an energy matching gain factor g.sub.b as $g_b=\sqrt{\frac{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{\hat{R},\mathrm{mem}}\left(k\right)}^2}{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{D,ECU}\left(m,k\right)}^2}}$ and populating the remaining bins with an energy adjusted decorrelated residual concealment frame
X.sub.R,ECU(m,k)=g.sub.bX.sub.D,ECU(m,k), k.Math.G.sub.i for band b.

13-17. (canceled)

18. An apparatus configured to approximate a lost or corrupted multichannel audio frame of a received multichannel audio signal, the apparatus comprising: at least one processor; memory communicatively coupled to the processor, said memory comprising instructions executable by the processor, which cause the processor to perform operations comprising: generating a down-mix error concealment frame; transforming the down-mix error concealment frame into a frequency domain to generate a transformed down-mix error concealment frame; decorrelating the transformed down-mix concealment frame to generate a decorrelated concealment frame; obtaining a residual signal spectrum of a stored residual signal of a previously received multichannel audio signal frame; generating an energy adjusted decorrelated residual signal concealment frame using the residual signal spectrum; obtaining a set of multi-channel audio substitution parameters; providing the transformed down-mix error concealment frame, the energy-adjusted decorrelated residual concealment frame, and multi-channel audio parameters from the previously received multichannel audio signal frame to a parametric multi-channel audio synthesis component to generate a synthesized multichannel audio frame; and performing an inverse frequency domain transformation of the synthesized multichannel audio frame to generate a substitution frame for the lost or corrupted multichannel audio frame.

19. The apparatus of claim 18 wherein the set of multi-channel audio substitution parameters is obtained by repeating the parameters from the previously received multi-channel audio signal frame.

20. The apparatus of claim 18 further comprising: generating multi-channel audio signals based on the substitution frame; and outputting the multi-channel audio signals towards at least one loudspeaker for playback.

21. The apparatus of claim 18 wherein obtaining the residual signal spectrum comprises retrieving the residual signal spectrum from a storage device.

22. The apparatus of claim 18 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises: phase-shifting peak sinusoid components of the residual signal spectrum; and adjusting an energy of a noise spectrum of non-peak sinusoid components of the residual signal spectrum of the stored residual signal.

23. The apparatus of claim 18 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises: detecting peak frequencies of the residual signal spectrum of the stored residual signal on a fractional frequency scale; associating each peak frequency with a number of peak frequency bins representing the peak frequency; applying a phase adjustment to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and populating remaining bins of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.

24. The apparatus of claim 18 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises: detecting whether there are peak frequencies in the residual signal spectrum of the stored residual signal on a fractional frequency scale; responsive to detecting no peak frequencies in the residual signal spectrum: populating each bin of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the bins to match an energy level of a noise spectrum of the residual signal spectrum; responsive to detecting peak frequencies in the residual signal spectrum: associating each peak frequency with a number of peak frequency bins representing the peak frequency; applying a phase adjustment to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and populating remaining bins of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.

25. The apparatus of claim 23 wherein adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum comprises matching the energy level on a band basis.

26. The apparatus of claim 23 wherein adjusting the energy level comprises combining a phase of bins of the decorrelated concealment frame with a magnitude of the bins of the residual signal concealment spectrum.

27. The apparatus of claim 26 wherein combining the phase comprises applying an approximate phase adjustment by matching a sign and an order of a real component and an imaginary component of the residual signal concealment spectrum to the decorrelated concealment frame.

28. The apparatus of claim 25 wherein matching the energy level comprises: calculating an energy matching gain factor g.sub.k as $g_k=\sqrt{\frac{{X_{\hat{R},\mathrm{mem}}\left(k\right)}^2}{{X_{D,\mathrm{ECU}}\left(m,k\right)}^2}}$ and populating the remaining bins with an energy adjusted decorrelated residual concealment frame
X.sub.R,ECU(m,k)=g.sub.kX.sub.D,ECU(m,k), k.Math.G.sub.i.

29. The apparatus of claim 25 wherein a band b spans (1501) a range of bins k.sub.start(b) . . . k.sub.end(b) and matching the energy level comprises: calculating an energy matching gain factor g.sub.b as $g_b=\sqrt{\frac{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{\hat{R},\mathrm{mem}}\left(k\right)}^2}{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{D,ECU}\left(m,k\right)}^2}}$ and populating the remaining bins with an energy adjusted decorrelated residual concealment frame
X.sub.R,ECU(m,k)=g.sub.bX.sub.D,ECU(m,k), k.Math.G.sub.i for band b.

30. An audio decoder comprising the apparatus according to claim 18.

31-32. (canceled)

33. A computer program product comprising a non-transitory computer readable medium storing computer program code which when executed by at least one processor causes the at least one processor to: generate a down-mix error concealment frame; transform the down-mix error concealment frame into a frequency domain to generate a transformed down-mix error concealment frame; decorrelate the transformed down-mix concealment frame to generate a decorrelated concealment frame; obtain a residual signal spectrum of a stored residual signal of a previously received multichannel audio signal frame; generate an energy adjusted decorrelated residual signal concealment frame using the residual signal spectrum; obtaining a set of multi-channel audio substitution parameters; provide the transformed down-mix error concealment frame, the energy-adjusted decorrelated residual concealment frame, and multi-channel audio parameters from the previously received multichannel audio signal frame to a parametric multi-channel audio synthesis component to generate a synthesized multichannel audio frame; and perform an inverse frequency domain transformation of the synthesized multichannel audio frame to generate a substitution frame for the lost or corrupted multichannel audio frame.

34-56. (canceled)

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this application, illustrate certain non-limiting embodiments of inventive concepts. In the drawings:

[0022] FIG. 1 is a block diagram illustrating an example of an environment of a loss concealment system according to some embodiments;

[0023] FIG. 2 is a block diagram illustrating components of a parametric stereo codec according to some embodiments;

[0024] FIG. 3 is a plot illustrating a sinusoid component and a noise spectrum that are combined according to some embodiments;

[0025] FIG. 4 is a block diagram illustrating a stereo parametric encoder according to some embodiments;

[0026] FIG. 5 is a block diagram illustrating a stereo parametric decoder according to some embodiments;

[0027] FIG. 6 is a block diagram illustrating operations to generate a residual signal according to some embodiments of inventive concepts;

[0028] FIG. 7 is a block diagram illustrating operations to generate a substitution multichannel audio frame according to some embodiments of inventive concepts;

[0029] FIG. 8 is a flow chart illustrating operations of a decoder according to some embodiments of inventive concepts;

[0030] FIG. 9 is a flow chart illustrating operations of a decoder to generate a residual signal according to some embodiments of inventive concepts;

[0031] FIGS. 10A and 10B are an illustration of a generated spectrum of a generated residual signal according to some embodiments of inventive concepts;

[0032] FIG. 11 is a block diagram illustrating a decoder according to some embodiments of inventive concepts;

[0033] FIGS. 12-18 are flow charts illustrating operations of a decoder in accordance with some embodiments of inventive concepts.

[0034] FIG. 19 is a block diagram illustrating an approximate phase adjustment in accordance with some embodiments of inventive concepts.

DETAILED DESCRIPTION

[0035] Inventive concepts will now be described more fully hereinafter with reference to the accompanying drawings, in which examples of embodiments of inventive concepts are shown. Inventive concepts may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of present inventive concepts to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present/used in another embodiment.

[0036] The following description presents various embodiments of the disclosed subject matter. These embodiments are presented as teaching examples and are not to be construed as limiting the scope of the disclosed subject matter. For example, certain details of the described embodiments may be modified, omitted, or expanded upon without departing from the scope of the described subject matter.

[0037] FIG. 1 illustrates an example of an operating environment of a decoder 100 that may be used to decode multichannel bitstreams as described herein. The decoder 100 may be part of a media player, a mobile device, a set-top device, a desktop computer, and the like. The decoder 100 receives encoded bitstreams. The bitstreams may be sent from an encoder, from a storage device 104, from a device on the cloud via network 102, etc. During operation, decoder 100 receives and processes the frames of the bitstream as described herein. The decoder 100 outputs multi-channel audio signals and transmits the multi-channel audio signals to a multi-channel audio player 106 having at least one loudspeaker for playback of the multi-channel audio signals. Storage device 104 may be part of a storage depository of multi-channel audio signals such as a storage repository of a store or a streaming music service, a separate storage component, a component of a mobile device, etc. Multichannel audio player may be a Bluetooth speaker, a device having at least one loudspeaker, a mobile device, a streaming music service, etc.

[0038] FIG. 11 is a block diagram illustrating elements of decoder 100 configured to decode multi-channel audio frames and provide concealment for lost or corrupt frame according to some embodiments of inventive concepts. As shown, decoder 100 may include a network interface circuit 1105 (also referred to as a network interface) configured to provide communications with other devices/entities/functions/etc. The decoder 100 may also include a processor circuit 1101 (also referred to as a processor) coupled to the network interface circuit 1105, and a memory circuit 1103 (also referred to as memory) coupled to the processor circuit. The memory circuit 1103 may include computer readable program code that when executed by the processor circuit 1101 causes the processor circuit to perform operations according to embodiments disclosed herein.

[0039] According to other embodiments, processor circuit 1101 may be defined to include memory so that a separate memory circuit is not required. As discussed herein, operations of the decoder 100 may be performed by processor 1101 and/or network interface 1105. For example, processor 1101 may control network interface 1105 to transmit communications to multichannel audio players 106 and/or to receive communications through network interface 102 from one or more other network nodes/entities/servers such as encoder nodes, depository servers, etc. Moreover, modules may be stored in memory 1103, and these modules may provide instructions so that when instructions of a module are executed by processor 1101, processor 1101 performs respective operations.

[0040] In one embodiment, the multi-channel decoder of a multi-channel encoder and decoder system as outlined in FIG. 2 may be used. In more detail, the encoder can be described with reference to FIG. 4. In the description that follows, two channels will be used to describe the embodiments. These embodiments may be used with more than two channels. The multi-channel encoder processes the input left and right channels (designated as CH1 and CH2 in FIG. 2 and denoted L and R in FIG. 4) in segments referred to as frames. For a given frame m the two input channels may be written

[00001]$\hspace{1em}\{\begin{array}{c}l\left(m,n\right)\\ r\left(m,n\right)\end{array}$

where l denotes the left channel, r denotes the right channel, n=0, 1, 2, . . . , N denotes the sample number in frame m and N is the length of the frame. In an embodiment, the frames may be extracted with an overlap in the encoder such that the decoder may reconstruct the multi-channel audio signals using an overlap add strategy. The input channels are windowed with a suitable windowing function w(n) and transformed to the Discrete Fourier Transform (DFT) domain. Note that other frequency domain representations may be used here, such as a Quadrature Mirror Filter (QMF) filter bank, a Hybrid QMF filter bank or an odd DFT (ODFT) representation which is composed of the MDCT and MDST transform components.

[00002]$\hspace{1em}\{\begin{array}{c}{X}_L\left(m,k\right)=\underset{n=0}{\overset{N-1}{.Math.}}l\left(m,n\right)w\left(n\right)e^{-\frac{j2\pi \mathrm{kn}}{N}}\\ X_R\left(m,k\right)=\underset{n=0}{\overset{N-1}{.Math.}}r\left(m,n\right)w\left(n\right)e^{-\frac{j2\pi \mathrm{kn}}{N}}\end{array}$

[0041] The signals are then analyzed in parametric analysis block 410 to extract the ITD, IPD and ILD parameters. In addition, the channel coherence may be analyzed, and an ICC parameter may be derived. The set of multi-channel audio parameters for frame m may be denoted P(m), which contains the complete set of ITD, IPD, ILD and ICC parameters used in the parametric representation. The parameters are encoded by a parameter encoder 430 and added to the bitstream to be stored and/or transmitted to a decoder.

[0042] Before producing a down-mix channel, in one embodiment, it may be beneficial to compensate for the ITD and IPD to reduce the cancellation and maximize the energy of the down-mix. The ITD compensation may be implemented both in time domain before the frequency transform or in frequency domain, but it essentially performs a time shift on one or both channels to eliminate the ITD. The phase alignment may be implemented in different ways, but the purpose is to align the phase such that the cancellation is minimized. This ensures maximum energy in the down-mix. The ITD and IPD adjustments may be done in frequency bands or be done on the full frequency spectrum and it should preferably be done using the quantized ITD and IPD parameters to ensure that the modification can be inverted in the decoder stage.

[0043] The embodiments described below are independent of the realization of the IPD and ITD parameter analysis and compensation. In other words, the embodiments are not dependent on how the IPD and ITP are analyzed or compensated In such embodiments, the ITD and IPD adjusted channels are denoted with an asterisk:

[00003]$\hspace{1em}\{\begin{array}{c}{X}_L^{*}\left(m,k\right)\\ X_R^{*}\left(m,k\right)\end{array}$

[0044] The ITD and IPD adjusted input channels are then down-mixed by the parametric analysis and down-mix block 410 to produce a mid/side representation, also called a down-mix/side representation. One way to perform the down-mix is to use the sum and difference of the signals.

[00004]$\hspace{1em}\{\begin{array}{c}{X}_M\left(m,k\right)=\frac{\left(X_L^{*}\left(m,k\right)+X_R^{*}\left(m,k\right)\right)}{2}\\ X_S\left(m,k\right)=\frac{\left(X_L^{*}\left(m,k\right)+X_R^{*}\left(m,k\right)\right)}{2}\end{array}$

[0045] The down-mix signal X.sub.M(m, k) is encoded by down-mix encoder 420 to be stored and/or transmitted to a decoder. This encoding may be done in frequency domain, but it may also be done in time domain. In that case a DFT synthesis stage is required to produce a time domain version of the down-mix signal, which is in turn provided to the down-mix encoder 420. The transformation to time domain may, however, introduce a delay misalignment with the multi-channel audio parameters that would require additional handling. In one embodiment, this is solved by introducing additional delay or by interpolating the parameters to ensure that the decoder synthesis of the down-mix and the multi-channel audio parameters are aligned.

[0046] The complementary side signal X.sub.S(m, k) may be generated from the down-mix and the obtained multi-channel audio parameters by a local parametric synthesis block 440. A side signal prediction X.sub.{tilde over (S)}(m, k) can be derived using the down-mix signal

X.sub.{tilde over (S)}(m,k)=p(X.sub.M(m,k))

where p(⋅) is a predictor function and may be implemented as a single scaling factor α which minimizes the mean squared error (MSE) between the side signal and the predicted side signal. Further, the prediction may be applied on frequency bands and involve a prediction parameter for each frequency band b.

X.sub.{tilde over (S)}(m,k)=α.sub.bX.sub.M(m,k), k=k.sub.start(b) . . . k.sub.end(b)

[0047] If the coefficients of band b are designated as column vectors X.sub.{tilde over (S)},b(m) and X.sub.M,b(m), the minimum MSE predictor can be derived as

[00005]${\alpha }_b=\frac{{X_{M,b}\left(m\right)}^T{X}_{S,b}\left(m\right)}{{X_{M,b}\left(m\right)}^T{X}_{M,b}\left(m\right)}$

However, this expression may be simplified to produce a more stable prediction parameter. The prediction parameter α.sub.b can be used as an alternative implementation of the ILD parameter. Further details are described in the prediction mode of reference [4].

[0048] Given the predicted side signal, a prediction residual X.sub.R(m, k) can be created [4].

X.sub.R(m,k)=X.sub.S(m,k)−X{tilde over (S)}(m,k)

[0049] The prediction residual may be inputted in to a residual encoder 450. The encoding may be done directly in DFT domain or it could be done in time domain. Similarly, as for the down-mix encoder, a time domain encoder would require a DFT synthesis which may require alignment of the signals in the decoder. The residual signal represents the diffuse component which is not correlated with the down-mix signal. If a residual signal is not transmitted, a solution in one embodiment may be to substitute a signal for the residual signal in the stereo synthesis state in the decoder with the signal coming from a decorrelated version of the decoded down-mix signal. The substitute is typically used for low bitrates where the bit budget is too low to represent the residual signal with any useful resolution. For intermediate bit rates, it is common to encode a part of the residual. In this case the lower frequencies are often encoded, since they are perceptually more relevant. For the remaining part of the spectrum, the decorrelator signal is used as a substitute for the residual signal in the decoder. This approach is often referred to as a hybrid coding mode [4]. Further details are provided in the decoder description below.

[0050] The representation of the encoded down-mix, the encoded multi-channel audio parameters, and the encoded residual signal is multiplexed into a bitstream 360, which may be transmitted to a decoder or stored in a medium for future decoding.

[0051] In one embodiment, a multi-channel decoder is used in DFT domain as outlined in FIGS. 5-7. FIG. 5 illustrates an embodiment of a decoder in which the blocks of FIG. 6 that generate a residual signal in case of a lost frame. FIG. 7 illustrates an embodiment of a combination of the blocks of FIGS. 5 and 6. In the description that follows, the blocks of FIG. 7 shall be used. However, it should be noted that the demux 710 of FIG. 7 provides at least the same functions as demux 510 of FIG. 5, the down mix decoder 715 of FIG. 7 provides at least the same functions as the down mix decoder 520 of FIG. 5, the stereo parameters decoder 725 of FIG. 7 provides at least the same functions of stereo parameters 530 of FIG. 5, decorrelator 730 of FIG. 7 provides at least the same functions of decorrelator 540 of FIG. 5, residual decoder 735 of FIG. 7 provides at least the same functions as residual decoder 550 of FIG. 5, parametric synthesis block 760 of FIG. 7 provides at least the same functions of parametric synthesis block 560 of FIG. 5. Similarly, the down-mix PLC 720 of FIG. 7 provides at least the same functions of down-mix PLC 610 of FIG. 6, the decorrelator 730 of FIG. 7 provides at least the same functions of decorrelator 620 of FIG. 6, memory 740 of FIG. 7 provides at least the same functions of memory 630 of FIG. 6, spectral shaper 745 of FIG. 7 provides at least the same functions of spectral shaper 640 of FIG. 6, phase-ecu 750 of FIG. 7 provides at least the same functions as phase-ecu 650 of FIG. 6, signal combiner 755 of FIG. 7 provides at least the same functions as signal combiner 660 of FIG. 6, and parametric synthesis block 760 of FIG. 7 provides at least the same functions of parametric synthesis block 670 of FIG. 6.

[0052] Turning now to FIG. 7, a down-mix decoder 715 provides a reconstructed down-mix signal {circumflex over (M)}(m, n) which is segmented into DFT analysis frames m and n=0, 1, 2, . . . , N−1 denote the sample numbers within frame m. The analysis frames are typically extracted with an overlap which permits an overlap-add strategy in the DFT synthesis stage. The corresponding DFT spectra may be obtained through a DFT transform

[00006]$X_{\hat{M}}\left(m,k\right)=\underset{n=0}{\overset{N-1}{.Math.}}\hat{M}\left(m,n\right)w\left(n\right)e^{-\frac{j2\pi \mathrm{kn}}{N}}$

where w(n) denotes a suitable windowing function. The shape of the windowing function can be designed using a trade-off between frequency characteristics and algorithmic delay due to length of the overlapping regions. Similarly, a residual decoder 635 produces a reconstructed residual signal {circumflex over (R)}(m, n) for frame m and time instances n=0, 1, 2, . . . N.sub.R. Note that the frame length N.sub.R may be different from N since the residual signal may be produced at a different sampling rate. Since the residual coding may be targeted only for the lower frequency range, it may be beneficial to represent it with a lower sampling rate to save memory and computational complexity. A DFT representation of the residual signal X.sub.{circumflex over (R)}(m, k) is obtained. Note that if the residual signal is upsampled in DFT domain to the same sampling rate as the reconstructed down-mix, the DFT coefficients will need to be scaled with N/N.sub.R and the X.sub.{circumflex over (R)}(m, k) would be zero-padded to match the length N. To simplify the notation, and since the embodiment is not affected by the use of different sampling rates, for purposes of better understanding of the method, the sampling rates shall be equal and N.sub.R=N in the following description. Thus, no scaling or zero-padding shall be shown.

[0053] It should be noted that the frequency transform by means of a DFT is not necessary in case the down-mix and/or the residual signal is encoded in DFT domain. In this case, the decoding of the down-mix and/or residual signal provides the DFT spectrum that are necessary for further processing.

[0054] In an error free frame, often referred to as a good frame, the multi-channel audio decoder produces the multi-channel synthesis using the decoded down-mix signal together with the decoded multi-channel audio parameters in combination with the decoded residual signal. The DFT spectrum of the residual signal X.sub.{circumflex over (R)}(m, k) is stored in memory 740, such that the variable X.sub.{circumflex over (R)},mem(k) always holds the residual signal spectrum of the last received frame.

X.sub.{circumflex over (R)},mem(k):=X.sub.{circumflex over (R)}(m,k)

[0055] In some embodiments, a relevant subpart of the spectrum may be stored in order to save memory, e.g. only the lower frequency bins. In other embodiments, the residual signal may be stored in the time domain and the DFT spectrum may be obtained only when error occurs. This could reduce the peak computational complexity since the error concealment operation typically has lower complexity than the decoding of a correctly received frame. In the description that follows, the residual signal is already transformed to DFT domain during normal operation and the residual signal is stored as a DFT spectrum. In other embodiments, the residual signal is stored in the time domain. In these embodiments, the residual signal spectrum is obtained by transforming the residual signal to the DFT domain.

[0056] The decoded down-mix {circumflex over (M)}(m, n) is fed to the decorrelator 730 to synthesize a non-correlated signal component D(m, n), and the resulting signal is transformed to DFT domain X.sub.D(m, k). Note that the decorrelation may also be carried out in the frequency domain. The decoded down-mix X.sub.{circumflex over (M)}(m, k), the decorrelated component X.sub.D(m, k), and the residual signal X.sub.{circumflex over (R)}(m, k) is fed together with the multi-channel audio parameters P(m) to the parametric multi-channel synthesis block 660 to produce the reconstructed multi-channel audio signal. After the multi-channel synthesis in DFT domain has been applied, the left and right channels are transformed to time domain and output from the stereo decoder.

[0057] Turning to FIG. 12, operations the decoder 100 may perform when the decoder 100 detects a lost or corrupted multichannel audio frame (i.e., a bad frame) of an encoded multichannel audio signal. When the decoder detects a lost or corrupted frame, i.e., a bad frame (as represented by the bad frame indicator (BFI) in FIG. 7), the PLC technique is performed. In operation 1201, the PLC of the down-mix decoder 715 is activated and generates an error concealment frame for the down-mix {circumflex over (M)}.sub.ECU(m, n). The down-mix error concealment frame is frequency transformed to produce the corresponding DFT spectrum X.sub.{circumflex over (M)},ECU(m, n) in operation 1203. In operation 1205, the transformed down-mix error concealment frame may be input into the same decorrelator function 730 that is used for the down-mix to generate the decorrelated concealment frame D.sub.ECU(m, n) or input to a different decorrelator function and then frequency transformed to produce a decorrelated down-mix concealment frame X.sub.D,ECU(m, k).

[0058] The decorrelator function may be done in time domain before transformation, in the form of an all-pass filter, a delay, or a combination thereof. It may also be done in frequency domain after the frequency transform, in which case it would operate on frames, likely including past frames.

[0059] In operation 1207, a residual signal spectrum is obtained. The residual signal spectrum may be retrieved from storage when it has been previously stored. In situations where the residual signal is stored prior to DFT transformation operations, then the residual signal spectrum is obtained by performing a DFT operation on the stored residual signal. To generate a concealment frame for the residual signal, an energy adjusted decorrelated residual signal is generated in operation 1209. In operation 1209, a Phase ECU 750 performs a phase extrapolation or phase evolution strategy on a residual signal from the past synthesis which is stored in memory 740 as previously described. See also [3].

[0060] Turning to FIG. 13, the phase extrapolation or phase evolution strategy phase-shifts the peak sinusoids of the residual signal spectrum (see sinusoid component of FIG. 3) in operation 1301 and the energy of the noise spectrum of non-peak sinusoids (see noise spectrum of FIG. 3) is adjusted in operation 1303. Further details of these operations are provided in FIG. 14.

[0061] Turning to FIG. 14, in operation 1401, the residual signal spectrum X.sub.{circumflex over (R)},mem(k), which may also be referred to as a “prototype signal” is first input to a peak detector circuit that detects peak frequencies on a fractional frequency scale. A set of peaks

F={f.sub.i}, i=1,2, . . . N.sub.peaks

may be detected which are represented by their estimated fractional frequency f.sub.i and where N.sub.peaks is the number of detected peaks. Here the fractional frequency is expressed as a fractional number of DFT bins, such that e.g. the Nyquist frequency is found at f=N/2+1. In operation 1403, each detected peak is then associated with a number of frequency bins representing the detected peak. The number of frequency bins may be found by rounding the fractional frequency to the closest integer and including the neighboring bins, e.g. the N.sub.near peaks on each side:

[00007]$\{\begin{array}{c}{k}_i=\left[f_i\right]\\ G_i=\left\{k_i-N_{n\mathrm{ear}},\mathrm{.Math.},k_i,\mathrm{.Math.},k_i+N_{near}\right\}\end{array}\hspace{1em}$

[0062] where [⋅] represents the rounding operation and G.sub.i is the group of bins representing the peak at frequency f.sub.i. The number N.sub.near is a tuning constant that is determined when designing the system. A larger N.sub.near gives higher accuracy in each peak representation, but also introduces a larger distance between peaks that may be modeled. A suitable value for N.sub.near may be 1 or 2. A concealment spectrum X.sub.R,ECU(m, k) for the residual signal is formed by inserting the group of bins, including a phase adjustment operation 1405, which is based on the fractional frequency and the number of samples between the analysis frame of the previous frame and where the current frame would start.

N.sub.step=N−N.sub.overlap

[0063] The phase adjustment for each peak frequency f.sub.i is applied to each corresponding group of bins G.sub.i according to the phase adjustment

Δϕ.sub.i=2πN.sub.stepf.sub.i/N

which is applied to the corresponding bins of the concealment spectrum for the residual signal

X.sub.R,ECU(m,k)=X.sub.{circumflex over (R)},mem(k)e.sup.jΔϕi,k∈G.sub.i

[0064] In operation 1407, the remaining bins of X.sub.R,ECU(m, k), which are not occupied by the peak bins G.sub.i, which may be referred to as the noise spectrum or the noise component of the spectrum, are populated using the spectral coefficients of the decorrelated concealment frame X.sub.D,ECU(m, k). To ensure the coefficients have the appropriate energy level and overall spectral shape, the energy may be adjusted to match the energy of the noise spectrum of the residual spectrum memory X.sub.{circumflex over (R)},mem (k). This may be done by setting all peak bins G.sub.i to zero in a calculation buffer and matching the energy of the remaining noise spectrum bins. The energy matching may be done on a band basis as shown in FIG. 10a.

[0065] Turning to FIG. 15, a band b is designated in operation 1501 that spans the range of bins k.sub.start(b) . . . k.sub.end(b). In operation 1503, the energy matching gain factor g.sub.b can be calculated as:

[00008]$g_b=\sqrt{\frac{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{\hat{R},\mathrm{mem}}\left(k\right)}^2}{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{D,ECU}\left(m,k\right)}^2}}$

In operation 1505, the noise spectrum bins k are filled with the energy adjusted decorrelated residual concealment frame using the energy matching gain factor:

X.sub.R,ECU(m,k)=g.sub.bX.sub.D,ECU(m,k), k.Math.G.sub.i, for band b

Note that it may also be possible to apply the scaling on wide or narrow bands or even for each frequency bin. In the case of scaling for each bin, the magnitude spectrum of the residual memory X.sub.{circumflex over (R)},mem(k) is kept while the phase is applied from the spectrum of the decorrelated concealment frame X.sub.D,ECU(m, k). For example, the scaling may be achieved either by a magnitude adjustment of X.sub.D,ECU(m, k) to match the magnitude of X.sub.{circumflex over (R)},mem(k), or by a phase adjustment of X.sub.{circumflex over (R)},mem(k) to match the phase of X.sub.D,ECU(m, k). However, performing the scaling on a band basis retains some of the spectral fine structure which may be desirable.

[0066] In an embodiment in the case of scaling for each frequency bin, applying the phase from the spectrum of the decorrelated concealment frame X.sub.D,ECU(m, k) may use an approximation of the phase. This may reduce the complexity of the scaling. The energy matching gain factor g.sub.k can be calculated as:

[00009]$g_k=\sqrt{\frac{{X_{\hat{R},\mathrm{mem}}\left(k\right)}^2}{{X_{D,\mathrm{ECU}}\left(m,k\right)}^2}}$

The noise spectrum bins k are filled with the energy adjusted decorrelated residual concealment frame using the energy matching gain factor:

X.sub.R,ECU(m,k)=g.sub.kX.sub.D,ECU(m,k), k.Math.G.sub.i,

The computation of g.sub.k involves a square root and a division, which may be computationally complex. In an embodiment, an approximate phase adjustment is used that matches the sign and the order of the absolute values of the real and imaginary components of the phase target such that the phase is moved within π/4 of the phase target. This embodiment may skip the gain scaling with the energy matching gain factor g.sub.k. X.sub.R,ECU(m, k) may be written as

[00010]$X_{R,\mathrm{ECU}}\left(m,k\right)=a+jb$$\{\begin{array}{c}a=\mathrm{sign}\left(\mathrm{Re}\left(X_{D,\mathrm{ECU}}\left(m,k\right)\right)\right).Math.c\\ b=\mathrm{sign}\left(\mathrm{Im}\left(X_{D,ECU}\left(m,k\right)\right)\right).Math.d\end{array}$

where (c, d) is

[00011]$\{\begin{array}{c}c=.Math.\mathrm{Re}\left(X_{\hat{R},\mathrm{mem}}\left(k\right)\right).Math.\\ d=.Math.\left(X_{\hat{R},\mathrm{mem}}\left(k\right)\right).Math.\end{array}\hspace{1em}$

in the case where the order of the absolute values of the real and imaginary components is the same, i.e.

|Re(X.sub.{circumflex over (R)},mem(k))|≥|Im(X.sub.{circumflex over (R)},mem(k))|∧|Re(X.sub.D,ECU(m,k))|≥|Im(X.sub.D,ECU(m,k))|

and otherwise

[00012]$\{\begin{array}{c}c=.Math.\mathrm{Im}\left(X_{\hat{R},\mathrm{mem}}\left(k\right)\right).Math.\\ d=.Math.\mathrm{Re}\left(X_{\hat{R},\mathrm{mem}}\left(k\right)\right).Math.\end{array}\hspace{1em}$

[0067] The approximate phase adjustment is illustrated in FIG. 19. In FIG. 19, the phase target is given by X.sub.D,ECU(m, k) illustrated at 1900. The non-phase adjusted ECU synthesis X.sub.{circumflex over (R)},mem(k) is illustrated at 1904. The ECU synthesis X.sub.R,ECU(m, k) after the approximate phase adjustment has been applied is illustrated at 1902. The approximate phase adjustment can be used on a band basis and/or on a per frequency bin basis.

[0068] Note that if no tonal components are found, i.e. no peaks are detected, the entire concealment frame will be composed of the decorrelated concealment frame with spectral shaping applied, X.sub.R,ECU(m, k). This is illustrated in FIG. 17. Turning to FIG. 17, in operation 1701, the decoder 100 detects whether there are peak signals in the residual signal spectrum on a fractional frequency scale. If there are peak signals, operations 1703 to 1707 are performed. Specifically, each peak frequency is associated with a number of peak frequency bins in operation 1703. Operation 1703 is similar in operation to operation 1403. In operation 1705, a phase adjustment is applied to each of the number of peak frequency bins. Operation 1705 is similar in operation to operation 1405. In operation 1707, the remaining bins are populated using spectral coefficients of the decorrelated concealment frame and the energy level of the remaining bins is adjusted to match the energy level of the noise spectrum of the residual spectrum memory. Operation 1707 is similar in operation to operation 1407. If there are no peak signals, operation 1709 is performed, which populates all bins using spectral coefficients of the decorrelated concealment frame and the energy level of the bins is adjusted to match the energy level of the noise spectrum of the residual spectrum memory.

[0069] To complete the stereo synthesis of the error concealment frame, the multi-channel parameters needs to be estimated for the lost frame. This concealment may be done with various methods, but one way that was found to give reasonable results was to just repeat the stereo parameters from the last received frame to produce the multi-channel audio substitution parameters {circumflex over (P)}(m).

[0070] The final spectrum of the conceal residual spectrum is found by combining the spectral peaks with the energy adjusted noise spectrum in signal combiner 755. An example of the combination is illustrated in FIG. 10b.

[0071] Returning to FIG. 12, in operation 1211, the down-mix error concealment frame X.sub.{circumflex over (M)},ECU(k), the decorrelated down-mix concealment frame X.sub.D,ECU(m, k) and the energy adjusted decorrelated residual concealment frame X.sub.R,ECU(m, k) is fed together with the multichannel audio parameters P(m) to the parametric synthesis block 760 to produce the reconstructed signal. After the synthesis in DFT domain has been applied, the multichannel signal is transformed to time domain (e.g., left and right channels) in operation 1213 and output from the stereo decoder.

[0072] For example, in operation 1601 of FIG. 16, multichannel audio signals are generated based on the reconstructed signal (i.e., substitution frame). In operation 1603, the multichannel audio signals are output towards at least one loudspeaker for playback.

[0073] Turning to FIGS. 5-7, DFTs and IDFTs are illustrated. The IDFTs serve to decouple the down-mix decoding and the residual decoding from the DFT analysis stage. In other embodiments, the IDFTs are not used. In yet other embodiments where the signal processing described above is performed in the time domain, the DFTs are only used to provide the a decorrelated down-mix concealment frame X.sub.D,ECU(m, k) and a residual signal spectrum X.sub.{circumflex over (R)},mem(k) while the IDFTs are used to provide their time domain counterparts.

[0074] Turning, to FIGS. 8 and 9, flowcharts are illustrated depicting how the operations of concealment of residual signal of FIG. 12 may be performed in serial or in parallel. In case of an error-free frame, the DFT spectrum of the residual signal X.sub.{circumflex over (R)}(m, k) is stored in memory and updated in every error-free frame in operation 810. This memory is later used in the concealment of the “lost frame”. When the decoder detects or is notified of frame loss/corruption, the PLC algorithm, designed for down-mix part, is activated and generates the down-mix signal {circumflex over (M)}.sub.ECU(m, n) in operation 820. PLC algorithm for down-mix can be chosen from the techniques described above. Then, {circumflex over (M)}.sub.ECU(m, n) can be fed to the decorrelator in operation 830 to extract a non-correlated signal X.sub.D,ECU(m, k). Decorrelation can also be carried out in time domain as well. Also, the memory of down-mix, which holds the down-mix signal of the past frame, may be included in the input to the decorrelator. Then sinusoid components of residual memory, residual from last good X.sub.{circumflex over (R)},mem(k), are phase shifted in operation 840. Note that operations 830 and 840 are independent from each other and can be carried out the other way around. To keep the shape of the residual signal close to the residual of last good frame, the spectrum of decorrelator signal is reshaped in operation 850 based on the residual signal of the last good frame. The phase-shifted sinusoid components of residual signal of the last good frame and the reshaped decorrelated signal are combined in operation 860 and the concealment frame for residual signal X.sub.R,ECU(m, k) is generated. In another embodiment, the decoder may process operations 820 and 830 in parallel with operation 840. This is illustrated in FIG. 9.

[0075] FIGS. 10A and 10B show an example of how the decorrelator signal is shaped. FIG. 10A illustrates a residual signal spectrum (labeled as prototype) and a decorrelator output. FIG. 10B illustrates a concealment frame for the residual signal X.sub.R,ECU(m, k) derived as described above.

[0076] As previously indicated, the input to the parametric synthesis block 660 may alternatively be in the time domain. FIG. 18 illustrates the operation of decoder 100 when the input to the parametric synthesis block 660 is in the time domain and the parametric synthesis block synthesizes the signals in the time domain. Operations 1801 to 1811 are the same operations as operations 1201 to 1211 of FIG. 12 as described above. In operation 1813, the decoder 100 performs an inverse frequency domain (IFD) transformation on the decorrelated concealment frame and the concealment frame for the residual signal. In operation 1815, the resulting IFD transformed signals and the parametric multi-channel audio time-domain substitution parameters are provided to the multi-channel audio synthesis component 760, which generates the output channels in the time domain.

Listing of Embodiments

[0077] 1. A method of approximating a lost or corrupted multichannel audio frame of a received multichannel audio signal in a decoding device comprising a processor, the method comprising the following operations performed by the processor:

[0078] generating a down-mix error concealment frame (610, 720, 820, 1201);

[0079] transforming the down-mix error concealment frame into a frequency domain to generate a transformed down-mix error concealment frame (1203);

[0080] decorrelating the transformed down-mix concealment frame to generate a decorrelated concealment frame (620, 730, 830, 1205);

[0081] obtaining a residual signal spectrum (1207) of a stored residual signal of a previously received multichannel audio signal frame;

[0082] generating an energy adjusted decorrelated residual signal concealment frame (640-660, 745-755, 850-860, 1209) using the residual signal spectrum;

[0083] obtaining a set of multi-channel audio substitution parameters;

[0084] providing (1213) the transformed down-mix error concealment frame, the energy-adjusted decorrelated residual concealment frame, and multi-channel audio substitution parameters to a parametric multi-channel audio synthesis component to generate a synthesized multichannel audio frame; and performing (1215) an inverse frequency domain transformation of the synthesized multichannel audio frame to generate a substitution frame for the lost or corrupted multichannel audio frame.

2. The method of Embodiment 1 wherein the set of multi-channel audio substitution parameters is obtained by repeating the parameters from the previously received multi-channel audio signal frame.
3. The method of any of Embodiments 1-2 further comprising:

[0085] generating (1601) multi-channel audio signals based on the substitution frame; and

[0086] outputting (1603) the multi-channel audio signals towards at least one loudspeaker for playback.

4. The method of any of Embodiments 1-3 wherein obtaining the residual signal spectrum comprises retrieving the residual signal spectrum from a storage device.
5. The method of any of Embodiments 1-4 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0087] phase-shifting peak sinusoid components (650, 750, 840, 1301) of the residual signal spectrum; and

[0088] adjusting (640, 745, 850, 1303) an energy of a noise spectrum of non-peak sinusoid components of the residual signal spectrum of the stored residual signal.

6. The method of any of Embodiments 1-4 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0089] detecting peak frequencies of the residual signal spectrum (1401, 1701) of the stored residual signal on a fractional frequency scale;

[0090] associating (1403, 1703) each peak frequency with a number of peak frequency bins representing the peak frequency;

[0091] applying a phase adjustment (650, 750, 840, 1405, 1705) to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and

[0092] populating remaining bins (1407, 1707) of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.

7. The method of any of Embodiments 1-4 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0093] detecting whether there are peak frequencies in the residual signal spectrum (650, 750, 840, 1701) of the stored residual signal on a fractional frequency scale;

[0094] responsive to detecting no peak frequencies in the residual signal spectrum: [0095] populating (1709) each bin of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the bins to match an energy level of a noise spectrum of the residual signal spectrum;

[0096] responsive to detecting peak frequencies in the residual signal spectrum: [0097] associating (1703) each peak frequency with a number of peak frequency bins representing the peak frequency; [0098] applying a phase adjustment (650, 750, 840, 1705) to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and [0099] populating remaining bins (1707) of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.
8. The method of any of Embodiments 6-7 wherein adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum comprises matching the energy level on a band basis.
9. The method of Embodiment 7 wherein a band b spans (1501) a range of bins k.sub.start(b) . . . k.sub.end(b) and matching the energy level comprises:

[0100] calculating (1503) an energy matching gain factor g.sub.b as

[00013]$g_b=\sqrt{\frac{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{\hat{R},\mathrm{mem}}\left(k\right)}^2}{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{D,ECU}\left(m,k\right)}^2}}$

and populating (1505) the remaining bins with an energy adjusted decorrelated residual concealment frame

X.sub.R,ECU(m,k)=g.sub.bX.sub.D,ECU(m,k), k.Math.G.sub.i for band b.

10. The method of any of Embodiments 1-9 wherein the generating of the energy adjusted decorrelated residual signal concealment frame is performed in parallel with the transforming of the down-mix error concealment frame into the frequency domain and the decorrelating of the transformed down-mix concealment frame.
11. The method of any of Embodiments 1-10 wherein one of the transforming of the down-mix error concealment frame into the frequency domain and the decorrelating of the transformed down-mix concealment frame is performed before the other of the transforming of the down-mix error concealment frame into the frequency domain and the decorrelating of the transformed down-mix concealment frame.
12. A decoder (100) for a communication network, the decoder (100) comprising:

[0101] a processor (1101); and

[0102] memory (1103) coupled with the processor, wherein the memory comprises instructions that when executed by the processor cause the processor to perform operations according to any of Embodiments 1-11.

13. A computer program comprising computer-executable instructions configured to cause a device to perform the method according to any one of Embodiments 1-11, when the computer-executable instructions are executed on a processor (1101) comprised in the device.
14. A computer program product comprising a computer-readable storage medium (1103), the computer-readable storage medium having computer-executable instructions configured to cause a device to perform the method according to any one of Embodiments 1-11 when the computer-executable instructions are executed on a processor (1101) comprised in the device.
15. An apparatus configured to approximate a lost or corrupted multichannel audio frame of a received multichannel audio signal, the apparatus comprising:

[0103] at least one processor (1101);

[0104] memory (1103) communicatively coupled to the processor, said memory comprising instructions executable by the processor, which cause the processor to perform operations comprising: [0105] generating a down-mix error concealment frame (610, 720, 820, 1201); [0106] transforming the down-mix error concealment frame into a frequency domain to generate a transformed down-mix error concealment frame (1203); [0107] decorrelating the transformed down-mix concealment frame to generate a decorrelated concealment frame (620, 730, 830, 1205); [0108] obtaining a residual signal spectrum (1207) of a stored residual signal of a previously received multichannel audio signal frame; [0109] generating an energy adjusted decorrelated residual signal concealment frame (640-660, 745-755, 850-860, 1209) using the residual signal spectrum; [0110] obtaining (1211) a set of multi-channel audio substitution parameters; [0111] providing (1213) the transformed down-mix error concealment frame, the energy-adjusted decorrelated residual concealment frame, and multi-channel audio parameters from the previously received multichannel audio signal frame to a parametric multi-channel audio synthesis component to generate a synthesized multichannel audio frame; and [0112] performing (1215) an inverse frequency domain transformation of the synthesized multichannel audio frame to generate a substitution frame for the lost or corrupted multichannel audio frame.
16. The apparatus of Embodiment 15 wherein the set of multi-channel audio substitution parameters is obtained by repeating the parameters from the previously received multi-channel audio signal frame.
17. The apparatus of any of Embodiments 15-16 further comprising:

[0113] generating (1601) multi-channel audio signals based on the substitution frame; and

[0114] outputting (1603) the multi-channel audio signals towards at least one loudspeaker for playback.

18. The apparatus of any of Embodiments 15-17 wherein obtaining the residual signal spectrum comprises retrieving the residual signal spectrum from a storage device.
19. The apparatus of any of Embodiments 15-18 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0115] phase-shifting peak sinusoid components (650, 750, 840, 1301) of the residual signal spectrum; and

[0116] adjusting (640, 745, 850, 1303) an energy of a noise spectrum of non-peak sinusoid components of the residual signal spectrum of the stored residual signal.

20. The apparatus of any of Embodiments 15-18 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0117] detecting peak frequencies of the residual signal spectrum (1401, 1701) of the stored residual signal on a fractional frequency scale;

[0118] associating (1403, 1703) each peak frequency with a number of peak frequency bins representing the peak frequency;

[0119] applying a phase adjustment (650, 750, 840, 1405, 1705) to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and

[0120] populating remaining bins (1407, 1707) of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.

21. The apparatus of any of Embodiments 15-18 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0121] detecting whether there are peak frequencies in the residual signal spectrum (650, 750, 840, 1701) of the stored residual signal on a fractional frequency scale;

[0122] responsive to detecting no peak frequencies in the residual signal spectrum: [0123] populating (1709) each bin of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the bins to match an energy level of a noise spectrum of the residual signal spectrum;

[0124] responsive to detecting peak frequencies in the residual signal spectrum: [0125] associating (1703) each peak frequency with a number of peak frequency bins representing the peak frequency; [0126] applying a phase adjustment (650, 750, 840, 1705) to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and [0127] populating remaining bins (1707) of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.
22. The apparatus of any of Embodiments 20-21 wherein adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum comprises matching the energy level on a band basis.
23. The apparatus of Embodiment 22 wherein a band b spans (1501) a range of bins k.sub.start(b) . . . k.sub.end(b) and matching the energy level comprises:

[0128] calculating (1503) an energy matching gain factor g.sub.b as

[00014]$g_b=\sqrt{\frac{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{\hat{R},\mathrm{mem}}\left(k\right)}^2}{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{D,ECU}\left(m,k\right)}^2}}$

and populating (1505) the remaining bins with an energy adjusted decorrelated residual concealment frame

X.sub.R,ECU(m,k)=g.sub.bX.sub.D,ECU(m,k), k.Math.G.sub.i for band b.

24. An audio decoder comprising the apparatus according to any of Embodiments 14-21.
25. A decoder configured to perform operations comprising:

[0129] generating a down-mix error concealment frame (610, 720, 820, 1201);

[0130] transforming the down-mix error concealment frame into a frequency domain to generate a transformed down-mix error concealment frame (1203);

[0131] decorrelating the transformed down-mix concealment frame to generate a decorrelated concealment frame (620, 730, 830, 1205);

[0132] obtaining a residual signal spectrum (1207) of a stored residual signal of a previously received multichannel audio signal frame;

[0133] generating an energy adjusted decorrelated residual signal concealment frame (640-660, 745-755, 850-860, 1209) using the residual signal spectrum;

[0134] obtaining (1211) a set of multi-channel audio substitution parameters;

[0135] providing (1213) the transformed down-mix error concealment frame, the energy-adjusted decorrelated residual concealment frame, and multi-channel audio parameters from the previously received multichannel audio signal frame to a parametric multi-channel audio synthesis component to generate a synthesized multichannel audio frame; and

[0136] performing (1213) an inverse frequency domain transformation of the synthesized multichannel audio frame to generate a substitution frame for the lost or corrupted multichannel audio frame.

26. The decoder of Embodiment 25 wherein the set of multi-channel audio substitution parameters is obtained by repeating the parameters from the previously received multi-channel audio signal frame.
27. A computer program product comprising a non-transitory computer readable medium storing computer program code which when executed by at least one processor causes the at least one processor to:

[0137] generate a down-mix error concealment frame (610, 720, 820, 1201);

[0138] transform the down-mix error concealment frame into a frequency domain to generate a transformed down-mix error concealment frame (1203);

[0139] decorrelate the transformed down-mix concealment frame to generate a decorrelated concealment frame (620, 730, 830, 1205);

[0140] obtain a residual signal spectrum (1207) of a stored residual signal of a previously received multichannel audio signal frame;

[0141] generate an energy adjusted decorrelated residual signal concealment frame (640-660, 745-755, 850-860, 1209) using the residual signal spectrum;

[0142] obtaining (1211) a set of multi-channel audio substitution parameters;

[0143] provide (1213) the transformed down-mix error concealment frame, the energy-adjusted decorrelated residual concealment frame, and multi-channel audio parameters from the previously received multichannel audio signal frame to a parametric multi-channel audio synthesis component to generate a synthesized multichannel audio frame; and

[0144] perform (1215) an inverse frequency domain transformation of the synthesized multichannel audio frame to generate a substitution frame for the lost or corrupted multichannel audio frame.

28. The computer program product of Embodiment 27 wherein the set of multi-channel audio substitution parameters is obtained by repeating the parameters from the previously received multi-channel audio signal frame;
29. The computer program product of any of Embodiments 27-28 wherein the non-transitory computer readable medium stores further computer program code which when executed causes the at least on processor to:

[0145] generate (1601) multi-channel audio signals based on the substitution frame; and

[0146] output (1603) the multi-channel audio signals towards at least one loudspeaker for playback

30. The computer program product of any of Embodiments 27-29 wherein obtaining the residual signal spectrum comprises retrieving the residual signal spectrum from a storage device.
31. The computer program product of any of Embodiments 27-20 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0147] phase-shifting peak sinusoid components (650, 750, 840, 1301) of the residual signal spectrum; and

[0148] adjusting (640, 745, 850, 1303) an energy of a noise spectrum of non-peak sinusoid components of the residual signal spectrum of the stored residual signal.

32. The computer program product of any of Embodiments 27-30 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0149] detecting peak frequencies of the residual signal spectrum (1401, 1701) of the stored residual signal on a fractional frequency scale;

[0150] associating (1403, 1703) each peak frequency with a number of peak frequency bins representing the peak frequency;

[0151] applying a phase adjustment (650, 750, 840, 1405, 1705) to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and

[0152] populating remaining bins (1407, 1707) of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.

33. The computer program product of any of Embodiments 27-30 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0153] detecting whether there are peak frequencies in the residual signal spectrum (650, 750, 840, 1701) of the stored residual signal on a fractional frequency scale;

[0154] responsive to detecting no peak frequencies in the residual signal spectrum: [0155] populating (1709) each bin of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the bins to match an energy level of a noise spectrum of the residual signal spectrum

(X.sub.R,ECU(m,k)=gX.sub.D,ECU(m,k))

[0156] responsive to detecting peak frequencies in the residual signal spectrum: [0157] associating (1703) each peak frequency with a number of peak frequency bins representing the peak frequency; [0158] applying a phase adjustment (650, 750, 840, 1705) to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and [0159] populating remaining bins (1707) of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.
34. The computer program product of any of Embodiments 32-33 wherein adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum comprises matching the energy level on a band basis.
35. The computer program product of Embodiment 34 wherein a band b spans (1501) a range of bins k.sub.start(b) . . . k.sub.end(b) and matching the energy level comprises:

[0160] calculating (1503) an energy matching gain factor g.sub.b as

[00015]$g_b=\sqrt{\frac{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{\hat{R},\mathrm{mem}}\left(k\right)}^2}{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{D,ECU}\left(m,k\right)}^2}}$

and populating (1505) the remaining bins with an energy adjusted decorrelated residual concealment frame

X.sub.R,ECU(m,k)=g.sub.bX.sub.D,ECU(m,k), k.Math.G.sub.i for band b.

36. A method of approximating a lost or corrupted multichannel audio frame of a received multichannel audio signal in a decoding device comprising a processor, the method comprising the following operations performed by the processor:

[0161] generating a down-mix error concealment frame (610, 720, 820, 1801);

[0162] transforming the down-mix error concealment frame into a frequency domain to generate a transformed down-mix error concealment frame (1803);

[0163] decorrelating the transformed down-mix concealment frame to generate a decorrelated concealment frame (620, 730, 830, 1805);

[0164] obtaining a residual signal spectrum (810, 1807) of a stored residual signal of a previously received multichannel audio signal frame;

[0165] generating an energy adjusted decorrelated residual signal concealment frame (640-660, 745-755, 850-860, 1809) using the residual signal spectrum;

[0166] obtaining (1811) a set of multi-channel audio substitution parameters;

[0167] performing (1813) an inverse frequency domain transformation of the transformed down-mix error concealment frame, the energy-adjusted decorrelated residual concealment frame, and multi-channel audio parameters from the previously received multichannel audio signal frame to generate a transformed down-mix error concealment time-domain frame, an energy-adjusted decorrelated residual concealment time domain frame, and multi-channel audio time domain parameters; and

[0168] providing (1815) the transformed down-mix error concealment time-domain frame, the energy-adjusted decorrelated residual concealment time-domain frame, and the multi-channel audio time-domain parameters to a parametric multi-channel audio synthesis component to generate a synthesized multichannel audio substitute frame.

37. The method of Embodiment 36 wherein the set of multi-channel audio substitution parameters is obtained by repeating the parameters from the previously received multi-channel audio signal frame.
38. The method of any of Embodiments 36-37 further comprising:

[0169] generating (1601) multi-channel audio signals based on the synthesized multichannel audio substitute frame; and

[0170] outputting (1603) the multi-channel audio signals towards at least one loudspeaker for playback.

39. The method of any of Embodiments 36-38 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0171] phase-shifting peak sinusoid components (650, 750, 840, 1301) of the residual signal spectrum; [0172] and adjusting an energy of a noise spectrum of non-peak sinusoid components (640, 745, 850, 1303) of the residual signal spectrum of the stored residual signal.
40. The method of any of Embodiments 36-38 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0173] detecting peak frequencies of the residual signal spectrum (1401, 1701) of the stored residual signal on a fractional frequency scale;

[0174] associating (1403, 1703) each peak frequency with a number of peak frequency bins representing the peak frequency;

[0175] applying a phase adjustment (650, 750, 840, 1405, 1705) to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and

[0176] populating remaining bins (1407, 1707) of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.

41. The method of any of Embodiments 36-38 wherein generating the energy adjusted decorrelated residual signal concealment frame comprises:

[0177] detecting whether there are peak frequencies in the residual signal spectrum (650, 750, 840, 1701) of the stored residual signal on a fractional frequency scale;

[0178] responsive to detecting no peak frequencies in the residual signal spectrum: [0179] populating (1709) each bin of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the bins to match an energy level of a noise spectrum of the residual signal spectrum;

[0180] responsive to detecting peak frequencies in the residual signal spectrum: [0181] associating (1703) each peak frequency with a number of peak frequency bins representing the peak frequency; [0182] applying a phase adjustment (650, 750, 840, 1705) to each of the number of peak frequency bins according to a phase adjustment to form a residual signal concealment spectrum; and [0183] populating remaining bins (1707) of the residual signal concealment spectrum using spectral coefficients of the decorrelated concealment frame and adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum.
42. The method of any of Embodiments 40-41 wherein adjusting an energy level of the remaining bins to match an energy level of a noise spectrum of the residual signal spectrum comprises matching the energy level on a band basis by:

[0184] designating (1501) a band b to span a range of bins k.sub.start(b) . . . k.sub.end(b);

[0185] calculating (1503) an energy matching gain factor g.sub.b as

[00016]$g_b=\sqrt{\frac{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{\hat{R},\mathrm{mem}}\left(k\right)}^2}{\underset{k=k_{\mathrm{start}\left(b\right)}}{\overset{k_{end\left(b\right)}}{.Math.}}{X_{D,ECU}\left(m,k\right)}^2}}$

[0186] and populating (1507) the remaining bins with an energy adjusted decorrelated residual concealment frame

X.sub.R,ECU(m,k)=g.sub.bX.sub.D,ECU(m,k), k.Math.G.sub.i for band b.

43. A computer program product comprising a non-transitory computer readable medium storing computer program code which when executed by at least one processor causes the at least one processor to:

[0187] generate a down-mix error concealment frame (1801);

[0188] transform the down-mix error concealment frame into a frequency domain to generate a transformed down-mix error concealment frame (1803);

[0189] decorrelate the transformed down-mix concealment frame to generate a decorrelated concealment frame (1805);

[0190] obtain a residual signal spectrum (1807) of a stored residual signal of a previously received multichannel audio signal frame;

[0191] generate an energy adjusted decorrelated residual signal concealment frame (1809) using the residual signal spectrum;

[0192] obtaining a set of multi-channel audio time-domain substitution parameters;

[0193] perform (1811) an inverse frequency domain transformation of the transformed down-mix error concealment frame, the energy-adjusted decorrelated residual concealment frame to generate a transformed down-mix error concealment time-domain frame and an energy-adjusted decorrelated residual concealment time domain frame; and

[0194] provide (1813) the transformed down-mix error concealment time-domain frame, the energy-adjusted decorrelated residual concealment time-domain frame, and the multi-channel audio time-domain substitution parameters to a parametric multi-channel audio synthesis component to generate a synthesized multichannel audio substitute frame.

44. The computer program product of Embodiment 38 wherein the set of multi-channel audio time-domain substitution parameters is obtained by repeating the parameters from the previously received multi-channel audio signal frame.
45. An apparatus configured to approximate a lost or corrupted multichannel audio frame of a received multichannel audio signal, the apparatus comprising:

[0195] at least one processor (1101);

[0196] memory (1103) communicatively coupled to the processor, said memory comprising instructions executable by the processor, which cause the processor to perform operations comprising: [0197] generating a down-mix error concealment frame (1801); [0198] transforming the down-mix error concealment frame into a frequency domain to generate a transformed down-mix error concealment frame (1803); [0199] decorrelating the transformed down-mix concealment frame to generate a decorrelated concealment frame (1805); [0200] obtaining a residual signal spectrum (1807) of a stored residual signal of a previously received multichannel audio signal frame; [0201] generating an energy adjusted decorrelated residual signal concealment frame (1809) using the residual signal spectrum; [0202] obtaining (1811) a set of multi-channel audio time-domain substitution parameters; [0203] performing (1813) an inverse frequency domain transformation of the transformed down-mix error concealment frame and the energy-adjusted decorrelated residual concealment frame to generate a transformed down-mix error concealment time-domain frame and an energy-adjusted decorrelated residual concealment time domain frame; and [0204] providing (1813) the transformed down-mix error concealment time-domain frame, the energy-adjusted decorrelated residual concealment time-domain frame, and the multi-channel audio time-domain substitution parameters to a parametric multi-channel audio synthesis component to generate a synthesized multichannel audio substitute frame.
46. The apparatus of Embodiment 39 wherein the set of multi-channel audio time-domain substitution parameters is obtained by repeating the parameters from the previously received multi-channel audio signal frame.

[0205] Explanations for abbreviations from the above disclosure are provided below.

TABLE-US-00001 Abbreviation Explanation DFT Discrete Fourier Transform LP Linear Prediction PLC Packet Loss Concealment ECU Error Concealment Unit FEC Frame Error Correction/Concealment MDCT Modified Discrete Cosine Transform MDST Modified Discrete Sine Transform ODFT Odd Discrete Fourier Transform LTP Long Term Predictor ITD Inter-channel Time Difference IPD Inter-channel Phase Difference ILD Inter-channel Level Difference ICC Inter-channel Coherence FD Frequency Domain TD Time Domain FLC Frame Loss Concealment BFI Bad Frame Indicator QMF Quadrature Mirror Filter bank

[0206] Citations for references from the above disclosure are provided below. [0207] [1]. C. Faller, “Parametric multichannel audio coding: synthesis of coherence cues,” in IEEE Transactions on Audio, Speech, and Language Processing, vol. 14, no. 1, pp. 299-310, January 2006. [0208] [2]. J. Lecomte et al., “Packet-loss concealment technology advances in EVS,” 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Q L D, 2015, pp. 5708-5712. [0209] [3]. S. Bruhn, E. Norvell, J. Svedberg and S. Sverrisson, “A novel sinusoidal approach to audio signal frame loss concealment and its application in the new evs codec standard,” 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Q L D, 2015, pp. 5142-5146. [0210] [4] Breebaart, J., Hotho, G., Koppens, J., Schuijers, E., “Background, Concept, and Architecture for the Recent MPEG Surround Standard on Multichannel Audio Compression”, J. Audio Eng, Soc., Vol. 55, No. 5, May 2007.

[0211] Further definitions and embodiments are discussed below.

[0212] In the above-description of various embodiments of present inventive concepts, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of present inventive concepts. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which present inventive concepts belong. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0213] When an element is referred to as being “connected”, “coupled”, “responsive”, or variants thereof to another element, it can be directly connected, coupled, or responsive to the other element or intervening elements may be present. In contrast, when an element is referred to as being “directly connected”, “directly coupled”, “directly responsive”, or variants thereof to another element, there are no intervening elements present. Like numbers refer to like elements throughout. Furthermore, “coupled”, “connected”, “responsive”, or variants thereof as used herein may include wirelessly coupled, connected, or responsive. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Well-known functions or constructions may not be described in detail for brevity and/or clarity. The term “and/or” includes any and all combinations of one or more of the associated listed items.

[0214] It will be understood that although the terms first, second, third, etc. may be used herein to describe various elements/operations, these elements/operations should not be limited by these terms. These terms are only used to distinguish one element/operation from another element/operation. Thus a first element/operation in some embodiments could be termed a second element/operation in other embodiments without departing from the teachings of present inventive concepts. The same reference numerals or the same reference designators denote the same or similar elements throughout the specification.

[0215] As used herein, the terms “comprise”, “comprising”, “comprises”, “include”, “including”, “includes”, “have”, “has”, “having”, or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof. Furthermore, as used herein, the common abbreviation “e.g.”, which derives from the Latin phrase “exempli gratia,” may be used to introduce or specify a general example or examples of a previously mentioned item, and is not intended to be limiting of such item. The common abbreviation “i.e.”, which derives from the Latin phrase “id est,” may be used to specify a particular item from a more general recitation.

[0216] Example embodiments are described herein with reference to block diagrams and/or flowchart illustrations of computer-implemented methods, apparatus (systems and/or devices) and/or computer program products. It is understood that a block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits. These computer program instructions may be provided to a processor circuit of a general purpose computer circuit, special purpose computer circuit, and/or other programmable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and/or other programmable data processing apparatus, transform and control transistors, values stored in memory locations, and other hardware components within such circuitry to implement the functions/acts specified in the block diagrams and/or flowchart block or blocks, and thereby create means (functionality) and/or structure for implementing the functions/acts specified in the block diagrams and/or flowchart block(s).

[0217] These computer program instructions may also be stored in a tangible computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the functions/acts specified in the block diagrams and/or flowchart block or blocks. Accordingly, embodiments of present inventive concepts may be embodied in hardware and/or in software (including firmware, resident software, micro-code, etc.) that runs on a processor such as a digital signal processor, which may collectively be referred to as “circuitry,” “a module” or variants thereof.

[0218] It should also be noted that in some alternate implementations, the functions/acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Moreover, the functionality of a given block of the flowcharts and/or block diagrams may be separated into multiple blocks and/or the functionality of two or more blocks of the flowcharts and/or block diagrams may be at least partially integrated. Finally, other blocks may be added/inserted between the blocks that are illustrated, and/or blocks/operations may be omitted without departing from the scope of inventive concepts. Moreover, although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

[0219] Many variations and modifications can be made to the embodiments without substantially departing from the principles of the present inventive concepts. All such variations and modifications are intended to be included herein within the scope of present inventive concepts. Accordingly, the above disclosed subject matter is to be considered illustrative, and not restrictive, and the examples of embodiments are intended to cover all such modifications, enhancements, and other embodiments, which fall within the spirit and scope of present inventive concepts. Thus, to the maximum extent allowed by law, the scope of present inventive concepts are to be determined by the broadest permissible interpretation of the present disclosure including the examples of embodiments and their equivalents, and shall not be restricted or limited by the foregoing detailed description.

[0220] Additional explanation is provided below

[0221] Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and/or is implied from the context in which it is used. All references to a/an/the element, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and/or where it is implicit that a step must follow or precede another step. Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever appropriate. Likewise, any advantage of any of the embodiments may apply to any other embodiments, and vice versa. Other objectives, features and advantages of the enclosed embodiments will be apparent from the following description.

[0222] Any appropriate steps, methods, features, functions, or benefits disclosed herein may be performed through one or more functional units or modules of one or more virtual apparatuses. Each virtual apparatus may comprise a number of these functional units. These functional units may be implemented via processing circuitry, which may include one or more microprocessor or microcontrollers, as well as other digital hardware, which may include digital signal processors (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as read-only memory (ROM), random-access memory (RAM), cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory includes program instructions for executing one or more telecommunications and/or data communications protocols as well as instructions for carrying out one or more of the techniques described herein. In some implementations, the processing circuitry may be used to cause the respective functional unit to perform corresponding functions according one or more embodiments of the present disclosure.

[0223] The term unit may have conventional meaning in the field of electronics, electrical devices and/or electronic devices and may include, for example, electrical and/or electronic circuitry, devices, modules, processors, memories, logic solid state and/or discrete devices, computer programs or instructions for carrying out respective tasks, procedures, computations, outputs, and/or displaying functions, and so on, as such as those that are described herein.