Apparatus and method for generating an error concealment signal using individual replacement LPC representations for individual codebook information
11393479 · 2022-07-19
Assignee
Inventors
- Michael Schnabel (Geroldsgruen, DE)
- Jérémie Lecomte (Fuerth, DE)
- Ralph Sperschneider (Ebermannstadt, DE)
- Manuel Jander (Hemhofen, DE)
Cpc classification
G10L19/09
PHYSICS
G10L19/06
PHYSICS
G10L19/005
PHYSICS
International classification
G10L19/005
PHYSICS
G10L19/09
PHYSICS
G10L19/028
PHYSICS
Abstract
An apparatus for generating an error concealment signal includes an LPC (linear prediction coding) representation generator for generating a first replacement LPC representation and a different second replacement LPC representation; an LPC synthesizer for filtering a first codebook information using the first replacement representation to obtain a first replacement signal and for filtering a different second codebook information using the second replacement LPC representation to obtain a second replacement signal; and a replacement signal combiner for combining the first replacement signal and the second replacement signal to obtain the error concealment signal.
Claims
1. Apparatus for generating an error concealment signal, comprising: an LPC (linear prediction coding) representation generator for generating at least one of a first replacement LPC representation and a different second replacement LPC representation; an LPC synthesizer for filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the different second replacement LPC representation to acquire a second replacement signal, wherein the apparatus is configured to influence a spectral shape of tonal and noise like parts of the error concealment signal separately, or to play out a voiced signal part almost unchanged, while a noise part is converged to background noise, or to conceal a voiced part and fade out the voiced part with a fading speed dependent on a signal characteristics, and to maintain a background noise during concealment, or to fade to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, wherein the LPC synthesizer further comprises a single LPC synthesis filter for filtering at least a portion of a combination of the first codebook information and the second codebook information or at least a portion of a combined weighted first codebook information and a weighted second codebook information using the first replacement LPC representation or the second replacement LPC representation or a weighted or unweighted combination thereof, wherein the apparatus further comprises an LPC memory initializer for initializing the single LPC synthesis filter in case of a recovery from an erroneous or lost frame to a good frame, the LPC memory initializer being configured for: feeding at least a portion of a combined first codebook information and second codebook information or at least a portion of a combined weighted first codebook information and a weighted second codebook information into the single LPC synthesis filter, saving memory states acquired by the feeding; and initializing the single LPC synthesis filter using the saved memory states, when a subsequent frame is a good frame.
2. Apparatus of claim 1, further comprising: an adaptive codebook for providing the first codebook information; and a fixed codebook for providing the second codebook information.
3. Apparatus of claim 2, wherein the fixed codebook is configured to provide a noise signal for the error concealment, and wherein the adaptive codebook is configured for providing an adaptive codebook content or an adaptive codebook content combined with an earlier fixed codebook content.
4. Apparatus of claim 1, wherein the LPC representation generator is configured to generate the first replacement LPC representation using a mean value of at least two last good frames and a weighted summation of the mean value and the last good frame, wherein a first weighting factor of the weighted summation changes over successive erroneous or lost frames, or wherein the LPC coefficient generator is configured to generate the second replacement LPC representation only using a weighted summation of a last good frame and the noise estimate, wherein a second weighting factor of the weighted summation changes over successive erroneous or lost frames.
5. Apparatus of claim 1, wherein the apparatus further comprises a controller for controlling a feedback into a first codebook providing the first codebook information, wherein the controller is configured to feed the first codebook information back into the first codebook or to feed a combination of the first codebook information and the second codebook information back into the first codebook.
6. Apparatus of claim 1, wherein the LPC representation generator is configured to generate ISF vectors for the replacement LPC representations.
7. Apparatus for generating an error concealment signal, comprising: an LPC (linear prediction coding) representation generator for generating at least one of a first replacement LPC representation and a different second replacement LPC representation; an LPC synthesizer for filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the different second replacement LPC representation to acquire a second replacement signal, wherein the apparatus is configured: to influence a spectral shape of tonal and noise like parts of the error concealment signal separately, or to play out a voiced signal part almost unchanged, while a noise part is converged to background noise, or to conceal a voiced part and fade out the voiced part with a fading speed dependent on a signal characteristics, and to maintain a background noise during concealment, or to fade to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, wherein the LPC synthesizer comprises a first LPC synthesis filter for filtering the first codebook information using the first replacement LPC representation to obtain the first replacement signal and a second LPC synthesis filter for filtering the second codebook information using the second replacement LPC representation to obtain the second replacement signal; and wherein the LPC synthesizer further comprises a single LPC synthesis filter for filtering at least a portion of a combination of the first codebook information and the second codebook information or at least a portion of a combined weighted first codebook information and a weighted second codebook information using the first replacement LPC representation or the second replacement LPC representation or a weighted or unweighted combination thereof, wherein the apparatus further comprises an LPC memory initializer for initializing, in case of an error concealment situation, first memory states of the first LPC synthesis filter and second memory states of the second LPC synthesis filter using corresponding memory states of the single LPC synthesis filter used for a good frame preceding an erroneous or lost frame.
8. Apparatus for generating an error concealment signal, comprising: an LPC (linear prediction coding) representation generator for generating at least one of a first replacement LPC representation and a different second replacement LPC representation; an LPC synthesizer for filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the different second replacement LPC representation to acquire a second replacement signal, wherein the apparatus is configured: to influence a spectral shape of tonal and noise like parts of the error concealment signal separately, or to play out a voiced signal part almost unchanged, while a noise part is converged to background noise, or to conceal a voiced part and fade out the voiced part with a fading speed dependent on a signal characteristics, and to maintain a background noise during concealment, or to fade to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, and wherein the apparatus further comprises: a gain calculator for calculating a first gain information from the first replacement LPC representation, and a second gain information from the second replacement LPC representation; and a compensator for compensating a gain influence of the first replacement LPC information using the first gain information and for compensating a gain influence of the second replacement LPC representation using the second gain information.
9. Apparatus of claim 8, wherein the gain calculator is configured to calculate: a last good power information related to a last good LPC representation before a start of the error concealment, a first power information from the first replacement LPC representation and a second power information from the second replacement LPC representation, a first gain value using the last good power information and the first power information, and a second gain value using the last good power information and the second power information, and wherein the compensator is configured for compensating using the first gain value and using the second gain value.
10. Apparatus of claim 8, wherein the gain calculator is configured to calculate an impulse response of an LPC representation and to calculate an RMS value from the impulse response to acquire a corresponding power information.
11. A method of generating an error concealment signal, comprising: generating at least one of a first replacement LPC representation and a different second replacement LPC representation; filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the second replacement LPC representation to acquire a second replacement signal; and influencing a spectral shape of tonal and noise like parts of the error concealment signal separately, or playing out a voiced signal part almost unchanged, while a noise part is converged to background noise, or concealing a voiced part and fading out the voiced part with a fading speed dependent on a signal characteristics, and maintaining a background noise during concealment, or fading to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, wherein the generating comprises using a single LPC synthesis filter for filtering at least a portion of a combination of the first codebook information and the second codebook information or at least a portion of a combined weighted first codebook information and a weighted second codebook information using the first replacement LPC representation or the second replacement LPC representation or a weighted or unweighted combination thereof, wherein the method further comprises initializing the single LPC synthesis filter in case of a recovery from an erroneous or lost frame to a good frame, the initializing comprising: feeding at least a portion of a combined first codebook information and second codebook information or at least a portion of a combined weighted first codebook information and a weighted second codebook information into the single LPC synthesis filter, saving memory states acquired by the feeding; and initializing the single LPC synthesis filter using the saved memory states, when a subsequent frame is a good frame.
12. A non-transitory digital storage medium having a computer program stored thereon to perform, when said computer program is run by a computer, the method of generating an error concealment signal, the method comprising: generating at least one of a first replacement LPC representation and a different second replacement LPC representation; filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the second replacement LPC representation to acquire a second replacement signal; and influencing a spectral shape of tonal and noise like parts of the error concealment signal separately, or playing out a voiced signal part almost unchanged, while a noise part is converged to background noise, or concealing a voiced part and fading out the voiced part with a fading speed dependent on a signal characteristics, and maintaining a background noise during concealment, or fading to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, wherein the generating comprises using a single LPC synthesis filter for filtering at least a portion of a combination of the first codebook information and the second codebook information or at least a portion of a combined weighted first codebook information and a weighted second codebook information using the first replacement LPC representation or the second replacement LPC representation or a weighted or unweighted combination thereof, wherein the method further comprises initializing the single LPC synthesis filter in case of a recovery from an erroneous or lost frame to a good frame, the initializing comprising: feeding at least a portion of a combined first codebook information and second codebook information or at least a portion of a combined weighted first codebook information and a weighted second codebook information into the single LPC synthesis filter, saving memory states acquired by the feeding; and initializing the single LPC synthesis filter using the saved memory states, when a subsequent frame is a good frame.
13. Apparatus for generating an error concealment signal, comprising: an LPC (linear prediction coding) representation generator for generating at least one of a first replacement LPC representation and a different second replacement LPC representation; an LPC synthesizer for filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the different second replacement LPC representation to acquire a second replacement signal, wherein the apparatus is configured to influence a spectral shape of tonal and noise like parts of the error concealment signal separately, or to play out a voiced signal part almost unchanged, while a noise part is converged to background noise, or to conceal a voiced part and fade out the voiced part with a fading speed dependent on a signal characteristics, and to maintain a background noise during concealment, or to fade to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, wherein the apparatus comprises a noise estimator for estimating the noise estimate from one or more preceding good frames, wherein the noise estimate represents the background spectral envelope estimate, wherein the LPC representation generator is configured to generate the first replacement LPC representation using one or more non-erroneous preceding LPC representations and the different second replacement LPC representation using a noise estimate, wherein the LPC synthesizer is configured to filter the first codebook information and the different second codebook information; and wherein the apparatus further comprises a replacement signal combiner for combining the first replacement signal and the second replacement signal to acquire the error concealment signal.
14. A method of generating an error concealment signal, comprising: generating at least one of a first replacement LPC representation and a different second replacement LPC representation; filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the second replacement LPC representation to acquire a second replacement signal; and influencing a spectral shape of tonal and noise like parts of the error concealment signal separately, or playing out a voiced signal part almost unchanged, while a noise part is converged to background noise, or concealing a voiced part and fading out the voiced part with a fading speed dependent on a signal characteristics, and maintaining a background noise during concealment, or fading to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, wherein the filtering comprises using a first LPC synthesis filter for filtering the first codebook information using the first replacement LPC representation to obtain the first replacement signal and using a second LPC synthesis filter for filtering the second codebook information using the second replacement LPC representation to obtain the second replacement signal; and wherein the filtering further comprises using a single LPC synthesis filter for filtering at least a portion of a combination of the first codebook information and the second codebook information or at least a portion of a combined weighted first codebook information and a weighted second codebook information using the first replacement LPC representation or the second replacement LPC representation or a weighted or unweighted combination thereof, wherein the method further comprises initializing, in case of an error concealment situation, first memory states of the first LPC synthesis filter and second memory states of the second LPC synthesis filter using corresponding memory states of the single LPC synthesis filter used for a good frame preceding an erroneous or lost frame.
15. A non-transitory digital storage medium having a computer program stored thereon to perform, when said computer program is run by a computer, the method of generating an error concealment signal, the method comprising: generating at least one of a first replacement LPC representation and a different second replacement LPC representation; filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the second replacement LPC representation to acquire a second replacement signal; and influencing a spectral shape of tonal and noise like parts of the error concealment signal separately, or playing out a voiced signal part almost unchanged, while a noise part is converged to background noise, or concealing a voiced part and fading out the voiced part with a fading speed dependent on a signal characteristics, and maintaining a background noise during concealment, or fading to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, wherein the filtering comprises using a first LPC synthesis filter for filtering the first codebook information using the first replacement LPC representation to obtain the first replacement signal and using a second LPC synthesis filter for filtering the second codebook information using the second replacement LPC representation to obtain the second replacement signal; and wherein the filtering further comprises using a single LPC synthesis filter for filtering at least a portion of a combination of the first codebook information and the second codebook information or at least a portion of a combined weighted first codebook information and a weighted second codebook information using the first replacement LPC representation or the second replacement LPC representation or a weighted or unweighted combination thereof, wherein the method further comprises initializing, in case of an error concealment situation, first memory states of the first LPC synthesis filter and second memory states of the second LPC synthesis filter using corresponding memory states of the single LPC synthesis filter used for a good frame preceding an erroneous or lost frame.
16. A method of generating an error concealment signal, comprising: generating at least one of a first replacement LPC representation and a different second replacement LPC representation; filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the second replacement LPC representation to acquire a second replacement signal; and influencing a spectral shape of tonal and noise like parts of the error concealment signal separately, or playing out a voiced signal part almost unchanged, while a noise part is converged to background noise, or concealing a voiced part and fading out the voiced part with a fading speed dependent on a signal characteristics, and maintaining a background noise during concealment, or fading to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, wherein the method comprises estimating the noise estimate from one or more preceding good frames, wherein the noise estimate represents the background spectral envelope estimate, wherein the generating comprises generating the first replacement LPC representation using one or more non-erroneous preceding LPC representations and the different second replacement LPC representation using a noise estimate, wherein the filtering comprises filtering the first codebook information and the different second codebook information; and wherein the method further comprises combining the first replacement signal and the second replacement signal to acquire the error concealment signal.
17. A non-transitory digital storage medium having a computer program stored thereon to perform, when said computer program is run by a computer, the method of generating an error concealment signal, the method comprising: generating at least one of a first replacement LPC representation and a different second replacement LPC representation; filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the second replacement LPC representation to acquire a second replacement signal; and influencing a spectral shape of tonal and noise like parts of the error concealment signal separately, or playing out a voiced signal part almost unchanged, while a noise part is converged to background noise, or concealing a voiced part and fading out the voiced part with a fading speed dependent on a signal characteristics, and maintaining a background noise during concealment, or fading to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, wherein the method comprises estimating the noise estimate from one or more preceding good frames, wherein the noise estimate represents the background spectral envelope estimate, wherein the generating comprises generating the first replacement LPC representation using one or more non-erroneous preceding LPC representations and the different second replacement LPC representation using a noise estimate, wherein the filtering comprises filtering the first codebook information and the different second codebook information; and wherein the method further comprises combining the first replacement signal and the second replacement signal to acquire the error concealment signal.
18. A method of generating an error concealment signal, comprising: generating at least one of a first replacement LPC representation and a different second replacement LPC representation; filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the second replacement LPC representation to acquire a second replacement signal; and influencing a spectral shape of tonal and noise like parts of the error concealment signal separately, or playing out a voiced signal part almost unchanged, while a noise part is converged to background noise, or concealing a voiced part and fading out the voiced part with a fading speed dependent on a signal characteristics, and maintaining a background noise during concealment, or fading to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, wherein the method further comprises: calculating a first gain information from the first replacement LPC representation, and a second gain information from the second replacement LPC representation; and compensating a gain influence of the first replacement LPC information using the first gain information and compensating a gain influence of the second replacement LPC representation using the second gain information.
19. A non-transitory digital storage medium having a computer program stored thereon to perform, when said computer program is run by a computer, the method of generating an error concealment signal, the method comprising: generating at least one of a first replacement LPC representation and a different second replacement LPC representation; filtering at least one of a first codebook information using the first replacement LPC representation to acquire a first replacement signal and a different second codebook information using the second replacement LPC representation to acquire a second replacement signal; and influencing a spectral shape of tonal and noise like parts of the error concealment signal separately, or playing out a voiced signal part almost unchanged, while a noise part is converged to background noise, or concealing a voiced part and fading out the voiced part with a fading speed dependent on a signal characteristics, and maintaining a background noise during concealment, or fading to background noise during concealment by fading out a tonal part without changing a spectral property of the tonal part and by fading a noise like part to a background spectral envelope estimate, wherein the method further comprises: calculating a first gain information from the first replacement LPC representation, and a second gain information from the second replacement LPC representation; and compensating a gain influence of the first replacement LPC information using the first gain information and compensating a gain influence of the second replacement LPC representation using the second gain information.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Embodiments of the present invention will be detailed subsequently referring to the appended drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
DETAILED DESCRIPTION OF THE INVENTION
(21) Advantageous embodiments of the present invention relate to controlling the level of the output signal by means of the codebook gains independently of any gain change caused by an extrapolated LPC and to control the LPC modeled spectral shape separately for each codebook. For this purpose, separate LPCs are applied for each codebook and compensation means are applied to compensate for any change of the LPC gain during concealment.
(22) Embodiments of the present invention as defined in the different aspects or in combined aspects have the advantage of providing a high subjective quality of speech/audio in case of one or more data packets not being correctly or not being received at all at the decoder side.
(23) Furthermore, the advantageous embodiments compensate the gain differences between subsequent LPCs during concealment, which might result from the LPC coefficients being changed over time, and therefore unwanted level changes are avoided.
(24) Furthermore, embodiments are advantageous in that during concealment two or more sets of LPC coefficients are used to independently influence the spectral behavior of voiced and unvoiced speech parts and also tonal and noise-like audio parts.
(25) All aspects of the present invention provide an improved subjective audio quality.
(26) According to one aspect of this invention, the energy is precisely controlled during the interpolation. Any gain that is introduced by changing the LPC is compensated.
(27) According to another aspect of this invention, individual LPC coefficient sets are utilized for each of the codebook vectors. Each codebook vector is filtered by its corresponding LPC and the individual filtered signals are just afterwards summed up to obtain the synthesized output. In contrast, state-of-the-art technology first adds up all excitation vectors (being generated from different codebooks) and just then feeds the sum to a single LPC filter.
(28) According to another aspect, a noise estimate is not used, for example as an offline-trained vector, but is actually derived from the past decoded frames so that, after a certain amount of erroneous or missing packets/frames, a fade-out to the actual background noise rather than any predetermined noise spectrum is obtained. This particularly results in a feeling of acceptance at a user side, but to the fact that even when an error situation occurs, the signal provided by the decoder after a certain number of frames is related to the preceding signal. However, the signal provided by a decoder in the case of a certain number of lost or erroneous frames is a signal completely unrelated to the signal provided by the decoder before an error situation.
(29) Applying gain compensation for the time-varying gain of the LPC allows the following advantages:
(30) It compensates any gain that is introduced by changing the LPC.
(31) Hence, the level of the output signal can be controlled by the codebook gains of the various codebooks. This allows for a pre-determined fade-out by eliminating any unwanted influence by the interpolated LPC.
(32) Using a separate set of LPC coefficients for each codebook used during concealment allows the following advantages:
(33) It creates the possibility to influence the spectral shape of tonal and noise like parts of the signal separately.
(34) It gives the chance to play out the voiced signal part almost unchanged (e.g. desired for vowels), while the noise part may quickly be converging to background noise.
(35) It gives the chance to conceal voiced parts, and fade out the voiced part with arbitrary fading speed (e.g. fade out speed dependent from signal characteristics), while simultaneously maintaining the background noise during concealment. State-of-the-art codecs usually suffer from a very clean voiced concealment sound.
(36) It provides means to fade to background noise during concealment smoothly, by fading out the tonal parts without changing the spectral properties, and fading the noise like parts to the background spectral envelope.
(37)
(38) Typically, the LPC synthesis output signals are time domain signals and the replacement signal combiner 110 performs a synthesis output signal combination by performing a synchronized sample-by-sample addition. However, other combinations, such as a weighted sample-by-sample addition or a frequency domain addition or any other signal combination can be performed by the replacement signal combiner 110 as well.
(39) Furthermore, the first codebook 102 is indicated as comprising an adaptive codebook and the second codebook 104 is indicated as comprising a fixed codebook. However, the first codebook and the second codebook can be any codebooks such as a predictive codebook as the first codebook and a noise codebook as the second codebook. However, other codebooks can be glottal pulse codebooks, innovative codebooks, transition codebooks, hybrid codebooks consisting of predictive and transform parts, codebooks for individual voice generators such as males/females/children or codebooks for different sounds such as for animal sounds, etc.
(40)
(41) Furthermore,
(42) Furthermore, combinations of the above procedures are possible, i.e. a noise codebook entry together with a certain post-processing.
(43)
(44) For the state-of-the-art just one LPC is applied. For the newly proposed method, each excitation vector, which is generated by either the adaptive or the fixed codebook, is filtered by its own set of LPC coefficients. The derivation of the individual ISF vectors is as follows:
(45) Coefficient set A (for filtering the adaptive codebook) is determined by this formula:
(46)
where alpha.sub.A is a time varying adaptive fading factor which may depend on signal stability, signal class, etc. isf.sup.−x are the ISF coefficients, where x denotes the frame number, relative to the end of the current frame: x=−1 denotes the first lost ISF, x=−2 the last good, x=−3 second last good and so on. This leads to fading the LPC which is used for filtering the tonal part, starting from the last correctly received frame towards the average LPC (averaged over three of the last good 20 ms frames). The more frames get lost, the closer the ISF, which is used during concealment, will be to this short term average ISF vector (isf′).
(47)
isf.sub.B.sup.−1=alpha.sub.B.Math.isf.sup.−2+(1−beta).Math.isf.sup.cng (block 146)
where isf.sup.cng is the ISF coefficient set derived from a background noise estimate and alpha.sub.B is the time-varying fading speed factor which advantageously is signal dependent. The target spectral shape is derived by tracing the past decoded signal in the FFT domain (power spectrum), using a minimum statistics approach with optimal smoothing, similar to [3]. This FFT estimate is converted to the LPC representation by calculating the auto-correlation by doing inverse FFT and then using Levinson-Durbin recursion to calculate LPC coefficients using the first N samples of the inverse FFT, where N is the LPC order. This LPC is then converted into the ISF domain to retrieve isf.sup.cng. Alternatively—if such tracing of the background spectral shape is not available—the target spectral shape might also be derived based on any combination of an offline trained vector and the short-term spectral mean, as it is done in G.718 for the common target spectral shape.
(48) Advantageously, the fading factors A and α.sub.B are determined depending on the decoded audio signal, i.e., depending on the decoded audio signal before the occurrence of an error. The fading factor may depend on signal stability, signal class, etc. Thus, is the signal is determined to be a quite noisy signal, then the fading factor is determined in such a way that the fading factor decreases, from time to time, more quickly than compared to a situation where a signal is quite tonal. In this situation, the fading factor decreases from one time frame to next time frame by a reduced amount. This makes sure that the fading out from the last good frame to the mean value of the last three good frames takes place more quickly in the case of noisy signals compared to non-noisy or tonal signals, where the fading out speed is reduced. Similar procedures can be performed for signal classes. For voiced signals, a fading out can be performed slower than for unvoiced signals or for music signals a certain fading speed can be reduced compared to further signal characteristics and corresponding determinations of the fading factor can be applied.
(49) As discussed in the context of
(50)
(51)
(52) However, if an error concealment situation is detected by the error concealment controller 202 of
(53) Furthermore, depending on the signal class, a controller 409 controls the switch 405 in order to either feedback a combination of both codebook outputs (subsequent to the application of the corresponding codebook gain) or to only feedback the adaptive codebook output.
(54) In accordance with an embodiment, the data for the LPC synthesis filter A 106 and the data for the LPC synthesis filter B 108 is generated by the LPC representation generator 100 of
(55) Subsequently, the switching from the normal mode to the concealment mode on one hand and from the concealment mode back to the normal mode is discussed.
(56) The transition from one common to several separate LPCs when switching from clean channel decoding to concealment does not cause any discontinuities, as the memory state of the last good LPC may be used to initialize each AR or MA memory of the separate LPCs. When doing so, a smooth transition from the last good to the first lost frame is ensured.
(57) When switching from concealment to clean channel decoding (recovery phase), the approach of the separate LPCs introduces the challenge to correctly update the internal memory state of the single LPC filter during clean-channel decoding (usually AR (auto-regressive) models are used). Just using the AR memory of one LPC or an averaged AR memory would lead to discontinuities at the frame border between the last lost and the first good frame. In the following a method is described to overcome deal with this challenge:
(58) A small portion of all excitation vectors (suggestion: 5 ms) is added at the end of any concealed frame. This summed excitation vector may then be fed to the LPC which would be used for recovery. This is shown in
(59) It is advisable to start at frame end minus 5 ms, setting the LPC AR memory to zero, derive the LPC synthesis by using any of the individual LPC coefficient sets and save the memory state at the very end of the concealed frame. If the next frame is correctly received, this memory state may then be used for recovery (meaning: used for initializing the start-of-frame LPC memory), otherwise it is discarded. This memory has to be additionally introduced; it is to be handled separately from any of the used LPC AR memories of the concealment used during concealment.
(60) Another solution for recovery is to use the method LPCO, known from USAC [4].
(61) Subsequently,
(62) Furthermore, the additional recovery LPC synthesizer X indicated at 418 is shown which receives, as an input, a sum of at least a small portion of all excitation vectors such as 5 ms.
(63) This excitation vector is input into the LPC synthesizer X 418 memory states of the LPC synthesis filter X.
(64) Then, when a switchback from the concealment mode to the normal mode occurs, the single LPC synthesis filter is controlled by copying the internal memory states of the LPC synthesis filter X into this single normal operating filter and additionally the coefficients of the filter are set by the correctly transmitted LPC representation.
(65)
(66) Additionally, as already discussed in the context of
(67)
(68) Compensator 406, 408 partly or fully compensates a gain influence of the first replacement LPC in the first gain information and compensates a gain influence of the second replacement LPC representation using the second gain information.
(69) In an embodiment, the calculator 600 is configured to calculate a last good power information related to a last good LPC representation before a start of the error concealment. Furthermore, the gain calculator 600 calculates a first power information for the first replacement LPC representation, a second power information for the second LPC representation, the first gain value using the last good power information and the first power information, and a second gain value using the last good power information and the second power information. Then, the compensation is performed in the compensator 406, 408 using the first gain value and using the second gain value. Depending on the information, however, the calculation of the last good power information can also be performed, as illustrated in the
(70) In particular, the gain calculator 600 is configured to calculate from the last good LPC representation or the first and second LPC replacement representations an impulse response and to then calculate an rms (root mean square) value from the impulse response to obtain the correspondent power information in the gain compensation, each excitation vector is—after being gained by the corresponding codebook gain—again amplified by the gains: g.sub.A or g.sub.B. These gains are determined by calculating the impulse response of the currently used LPC and then calculating the rms:
(71)
(72) The result is then compared to the rms of the last correctly received LPC and the quotient is used as gain factor in order to compensate for energy increase/loss of LPC interpolation:
(73)
(74) This procedure can be seen as a kind of normalization. It compensates the gain, which is caused by LPC interpolation.
(75) Subsequently,
(76) To this end, several steps are performed in an advantageous embodiment as illustrated in
(77) Then, in step 718, an rms value is calculated for each impulse response and in block 720 the corresponding gain is calculated using an old rms value and a new rms value and this calculation is advantageously done by dividing the old rms value by the new rms value. Finally, the result of block 720 is used to compensate the result of step 712 in order to finally obtained the compensated results as indicated at step 714.
(78) Subsequently, a further aspect is discussed, i.e. an implementation for an apparatus for generating an error concealment signal which ha the LPC representation generator 100 generating only a single replacement LPC representation, such as for the situation illustrated in
(79) The other procedures for the LPC representation generator, the gain calculator, the compensator and the LPC synthesizer can be performed in the same way as discussed in the context of
(80) As has been outlined in the context of
(81) Then a manipulator 1004 is provided which together performs the operations of for example amplifiers 402, 406 to the codebook information of a single codebook or to the codebook information of two or more codebooks in order to finally obtain a manipulated signal such as a codebook signal or a concealment signal, depending on whether the manipulator 1004 is located before the LPC synthesizer in
(82) The noise estimator is configured to process a spectral representation of a past decoded signal to provide a noise spectral representation and to convert the noise spectral representation into a noise LPC representation, where the noise LPC representation is the same kind of an LPC representation as the replacement LPC representation. Thus, when the replacement LPC representation is in the ISF-domain representation or an ISF vector, then the noise LPC representation additionally is an ISF vector or ISF representation.
(83) Furthermore, the noise estimator 206 is configured to apply a minimum statistics approach with optimal smoothing to a past decoded signal to derive the noise estimate. For this procedure, it is advantageous to perform the procedure illustrated in [3]. However, other noise estimation procedures relying on, for example, suppression of tonal parts compared to non-tonal parts in a spectrum in order to filter out the background noise or noise in an audio signal can be applied as well for obtaining the target spectral shape or noise spectral estimate.
(84) Thus, in one embodiment, a spectral noise estimate is derived from a past decoded signal and the spectral noise estimate is then converted into an LPC representation and then into an ISF domain to obtain the final noise estimate or target spectral shape.
(85)
(86) In a advantageous embodiment illustrated in
(87) ISF factor is calculated to obtain the noise estimate ISF.sup.cng to be used by the LPC representation generator 100.
(88) Subsequently,
(89) In step 1300, a mean value of two or three last good frames is calculated. In step 1302, the last good frame LPC representation is provided. Furthermore, in step 1304, a fading factor is provided which can be controlled, for example, by a separate signal analyzer which can be, for example, included in the error concealment controller 200 of
(90) In the context of calculating a single LPC replacement representation, the outputs of blocks 1300, 1304, 1306 are provided to the calculator 1308. Then, a single replacement LPC representation is calculated in such a way that subsequent to a certain number of lost or missing or erroneous frames/packets, the fading over to the noise estimate LPC representation is obtained.
(91) However, individual LPC representations for an individual codebook, such as for the adaptive codebook and the fixed codebook, are calculated as indicated at block 1310, then the procedure as discussed before for calculating ISF.sub.A.sup.−1 (LPC A) on the hand and the calculation of ISF.sub.B.sup.−1 (LPC B) is performed.
(92) Although the present invention has been described in the context of block diagrams where the blocks represent actual or logical hardware components, the present invention can also be implemented by a computer-implemented method. In the latter case, the blocks represent corresponding method steps where these steps stand for the functionalities performed by corresponding logical or physical hardware blocks.
(93) Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.
(94) Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
(95) Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
(96) Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.
(97) Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
(98) In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
(99) A further embodiment of the inventive method is, therefore, a data carrier (or a non-transitory storage medium such as a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitory.
(100) A further embodiment of the invention method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.
(101) A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.
(102) A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
(103) A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
(104) In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are advantageously performed by any hardware apparatus.
(105) While this invention has been described in terms of several embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations and equivalents as fall within the true spirit and scope of the present invention.
REFERENCES
(106) [1] ITU-T G.718 Recommendation, 2006 [2] Kazuhiro Kondo, Kiyoshi Nakagawa, “A Packet Loss Concealment Method Using Recursive Linear Prediction” Department of Electrical Engineering, Yamagata University, Japan. [3] R. Martin, Noise Power Spectral Density Estimation Based on Optimal Smoothing and Minimum Statistics, IEEE Transactions on speech and audio processing, vol. 9, no. 5, July 2001 [4] Ralf Geiger et. al., Patent application US20110173011 A1, Audio Encoder and Decoder for Encoding and Decoding Frames of a Sampled Audio Signal [5] 3GPP TS 26.190; Transcoding functions;—3GPP technical specification