Apparatus and method for processing an audio signal using a harmonic post-filter
11694704 · 2023-07-04
Assignee
Inventors
- Emmanuel Ravelli (Erlangen, DE)
- Christian Helmrich (Berlin, DE)
- Goran Markovic (Nuremberg, DE)
- Matthias Neusinger (Rohr, DE)
- Sascha Disch (Fuerth, DE)
- Manuel Jander (Hemhofen, DE)
- Martin Dietz (Nuremberg, DE)
Cpc classification
G10L21/02
PHYSICS
International classification
Abstract
An apparatus for processing an audio signal having associated therewith a pitch lag information and a gain information, includes a domain converter for converting a first domain representation of the audio signal into a second domain representation of the audio signal; and a harmonic post-filter for filtering the second domain representation of the audio signal, wherein the post-filter is based on a transfer function including a numerator and a denominator, wherein the numerator includes a gain value indicated by the gain information, and wherein the denominator includes an integer part of a pitch lag indicated by the pitch lag information and a multi-tap filter depending on a fractional part of the pitch lag.
Claims
1. An apparatus for processing an audio signal, comprising: a domain converter for converting a first domain representation of the audio signal into a second domain representation of the audio signal, the second domain representation being a time domain representation, wherein the audio signal being an input signal into the apparatus for processing has associated therewith a pitch lag information and a gain information, wherein the pitch lag information indicates a pitch lag having an integer part and a fractional part; and a harmonic post-filter for filtering the second domain representation of the audio signal, wherein the harmonic post-filter is based on a long-term prediction filter working in the time-domain, and wherein the long-term prediction filter, on which the harmonic post-filter is based, is configured to account for the integer part of the pitch lag indicated by the pitch lag information associated with the audio signal and the fractional part of the pitch lag indicated by the pitch lag information associated with the audio signal.
2. Apparatus of claim 1, wherein the long-term prediction filter, on which the harmonic post-filter is based, comprises filter parameters comprising the pitch lag, wherein the filter parameters comprising the pitch lag are determined from parameters decoded from a bitstream comprising the audio signal and the pitch lag information and the gain information.
3. Apparatus of claim 2, wherein the bitstream further comprises a decision bit, and wherein the apparatus is configured to not decode any pitch lag or gain, or to assume the pitch lag and the gain as not written into the bitstream, or to assume the pitch lag and the gain as a zero value, when the decision bit is equal to zero.
4. Apparatus of claim 1, wherein the harmonic post-filter comprises filter parameters, the filter parameters being the pitch lag and a gain derived from the pitch lag information and the gain information, respectively, wherein the harmonic post-filter is configured to have different parameters from a frame to a next frame, and wherein the apparatus further comprises a discontinuity remover for reducing a discontinuity at a border between the frame and the next frame.
5. Apparatus of claim 4, wherein the discontinuity remover comprises at least one of a cross-fader, a low-pass filter, or an LPC filter.
6. Apparatus of claim 4, wherein the discontinuity remover is configured to fade out a post filtered audio signal of the frame and, at the same time, to fade in a post filtered audio signal of the next frame.
7. Apparatus of claim 6, wherein a cross-fading characteristic of the fading out and the fading in is so that fading factors add up to one throughout a cross-fading operation.
8. The apparatus of claim 1, wherein the long-term prediction filter, on which the harmonic post-filter is based, is based on a transfer function comprising a numerator and a denominator, wherein the numerator comprises a multi-tap FIR filter for the fractional part of the pitch lag having a value of zero.
9. The apparatus of claim 1, wherein the long-term prediction filter, on which the harmonic post-filter is based, is based on a transfer function comprising a numerator and a denominator, wherein the denominator comprises a product between a multi-tap filter and a gain value comprised by the gain information associated with the audio signal.
10. The apparatus of claim 1, wherein the long-term prediction filter, on which the harmonic post-filter is based, is based on a transfer function comprising a numerator and a denominator, wherein the numerator comprises a product of a first scalar value and a second scalar value, wherein the denominator comprises the second scalar value and not the first scalar value, wherein the first scalar value and the second scalar value are predetermined and comprise values greater than 0, and wherein the second scalar value is lower than the first scalar value.
11. The apparatus of claim 10, further comprising: a filter controller configured for setting the second scalar value depending on a bitrate, by which the frequency-time converter is operated, wherein the second scalar value is set to a first value, when the bitrate comprises a first value, wherein the second scalar value is set to a second value, when the bitrate comprises a second value, wherein the second value of the bitrate is lower than the first value of the bitrate, and wherein the second value of the second scalar value is greater than the first value of the second scalar value.
12. The apparatus of claim 10, wherein the first scalar value is set between 0.6 and 1.0 and wherein the second scalar value is set between 0.1 and 0.5.
13. The apparatus of claim 1, wherein the long-term prediction filter, on which the harmonic post-filter is based, comprises a transfer function H(z) in a pole-zero representation based on the following equation:
14. The apparatus of claim 1, wherein the harmonic post-filter is configured to comprise a negative spectral tilt for compensating a loss in energy at frequencies between harmonics.
15. The apparatus of claim 1, wherein the domain converter is a frequency-time converter, wherein the first domain is a frequency domain, or wherein the domain converter is an LPC residual-time converter, wherein the first domain is an LPC residual domain.
16. The apparatus of claim 1, wherein the harmonic post-filter is configured to suppress an amount of energy between harmonics in a frame of the audio signal, wherein the amount of energy suppressed is smaller than 20% of a total energy of the time-domain representation in the frame.
17. Apparatus of claim 1, wherein a bitstream comprising the audio signal further comprises a decision bit, and wherein the apparatus is configured, when the decision bit is equal to zero, to not decode any pitch lag information or gain information, or to assume the pitch lag information and the gain information as not written into the bitstream, or to assume the pitch lag indicated by the pitch lag information and a gain indicated by the gain information as a zero value.
18. Apparatus of claim 1, wherein the long-term prediction filter, on which the harmonic post-filter is based, is based on a transfer function comprising a numerator and a denominator, wherein the numerator comprises a gain value indicated by the gain information, and wherein the denominator comprises the integer part of the pitch lag and a multi-tap filter depending on the fractional part of the pitch lag.
19. A method of processing an audio signal comprising: converting a frequency representation of the audio signal into a time-domain representation of the audio signal; wherein the audio signal being an input signal into the apparatus for processing has associated therewith a pitch lag information and a gain information, wherein the pitch lag information indicates a pitch lap having an integer part and a fractional part; and filtering the time-domain representation of the audio signal by a harmonic post-filter, wherein the harmonic post-filter is based on a long-term prediction filter working in the time-domain, and wherein the long-term prediction filter, on which the harmonic post-filter is based, is configured to account for the integer part of the pitch lag indicated by the pitch lag information associated with the audio signal and the fractional part of the pitch lag indicated by the pitch lag information associated with the audio signal.
20. A system for processing an audio signal comprising: an encoder for encoding the audio signal to obtain an encoded signal; and a decoder for decoding the encoded signal to obtain a decoded audio signal, the decoder comprising a processor, the processor comprising: a domain converter for converting a frequency representation of the decoded audio signal into a time-domain representation of the decoded audio signal, wherein the decoded audio signal being an input signal into the processor has associated therewith a pitch lag information and a gain information, wherein the pitch lab information indicates a pitch lag having an integer part and a fractional part; and a harmonic post-filter for filtering the time-domain representation of the decoded audio signal, wherein the harmonic post-filter is based on a long-term prediction filter working in the time-domain, and wherein the long-term prediction filter, on which the harmonic post-filter is based, is configured to account for the integer part of the pitch lag indicated by the pitch lag information associated with the decoded audio signal and the fractional part of the pitch lag indicated by the pitch lag information associated with the decoded audio signal.
21. The system of claim 20, wherein the encoder comprises: a pitch lag calculator for calculating the integer part and the fractional part of the pitch lag; a gain calculator for calculating the gain value; and an encoded signal former for generating the encoded signal comprising the pitch lag information, the pitch lab information having a pitch lab comprising the integer part and the fractional part and the gain information.
22. A method of processing an audio signal comprising: a method of encoding the audio signal to obtain an encoded signal; and a method of decoding the encoded signal to obtain a decoded audio signal, the method of decoding comprising a method of processing, the method of processing comprising: converting a frequency representation of the decoded audio signal into a time-domain representation of the decoded audio signal, wherein the decoded audio signal being an input signal into the method for processing has associated therewith a pitch lag information and a gain information, wherein the pitch lab information indicates a pitch lag having an integer part and a fractional part; and filtering the time-domain representation of the decoded audio signal using a harmonic post-filter, wherein the harmonic post-filter is based on a long-term prediction filter working in the time-domain, and wherein the long-term prediction filter, on which the harmonic post-filter is based, is configured to account for the integer part of the pitch lag indicated by the pitch lag information associated with the decoded audio signal and the fractional part of the pitch lag indicated by the pitch lag information associated with the decoded audio signal.
23. A non-transitory digital storage medium having a computer program stored thereon to perform, when said computer program is run by a computer, the method of processing an audio signal having associated therewith a pitch lag information and a gain information, comprising: converting a frequency representation of the audio signal into a time-domain representation of the audio signal, wherein the audio signal being an input signal into the method for processing has associated therewith a pitch lag information and a gain information, wherein the pitch lab information indicates a pitch lag having an integer part and a fractional part; and filtering the time-domain representation of the audio signal by a harmonic post-filter, wherein the harmonic post-filter is based on a long-term prediction filter working in the time-domain, and wherein the long-term prediction filter, on which the harmonic post-filter is based, is configured to account for the integer part of the pitch lag indicated by the pitch lag information associated with the audio signal and the fractional part of the pitch lag indicated by the pitch lag information associated with the audio signal.
24. A non-transitory digital storage medium having a computer program stored thereon to perform, when said computer program is run by a computer, a method of processing an audio signal, the method of processing comprising: a method of encoding the audio signal to obtain an encoded signal; and a method of decoding the encoded signal to obtain a decoded audio signal, the method of decoding comprising a method of processing, the method of processing comprising: converting a frequency representation of the decoded audio signal into a time-domain representation of the decoded audio signal, wherein the decoded audio signal being an input signal into the method for processing has associated therewith a pitch lag information and a gain information, wherein the pitch lag information indicates a pitch lag having an integer part and a fractional part; and filtering the time-domain representation of the decoded audio signal using a harmonic post-filter, wherein the harmonic post-filter is based on a long-term prediction filter working in the time-domain, and wherein the long-term prediction filter, on which the harmonic post-filter is based, is configured to account for the integer part of the pitch lag indicated by the pitch lag information associated with the decoded audio signal and the fractional part of the pitch lag indicated by the pitch lag information associated with the decoded audio signal.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Embodiments of the present invention will be detailed subsequently referring to the appended drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
DETAILED DESCRIPTION OF THE INVENTION
(13)
(14) However, for a more robust operation, it is advantageous to calculate the pitch lag information and the pitch gain information on the encoder-side.
(15) The decoder 100 comprises e.g. a frequency-time converter for converting a frequency-time representation of the audio signal into a time-domain representation of the audio signal. Thus, the decoder is not a pure time-domain speech codec, but comprises a pure transform domain decoder or a mixed transform domain decoder or any other coder operating in a domain different from a time domain. Furthermore, it is advantageous that the second domain is the time domain.
(16) The apparatus furthermore comprises a harmonic post-filter 104 for filtering the time-domain representation of the audio signal, and this harmonic post-filter is based on a transfer function comprising a numerator and a denominator. Particularly, the numerator comprises a gain value indicated by the gain information and the denominator comprises an integer part of a pitch lag indicated by the pitch lag information and, importantly, further comprises a multi-tap filter depending on a fractional part of the pitch lag.
(17) An implementation of this harmonic post filter with a transfer function H(z) is illustrated in
(18) The apparatus for processing an audio signal illustrated in
(19)
(20)
(21) The filter can be described as follows:
(22)
with the decoded gain, T.sub.int, and T.sub.fr the integer and fractional part of the decoded pitch lag, α and β two scalars that weight the gain, and B(z, T.sub.fr) a low-pass FIR filter whose coefficients depends on the fractional part of the decoded pitch lag.
(23) Note that B(z, 0) in the numerator of H(z) is used to compensate for the tilt introduced by B(z, T.sub.fr) is used to control the strength of the post-filter. A β equals to 1 produces full effects, suppressing the maximum possible amount of energy between the harmonics. A β equals to 0 disables the post-filter. Generally, a quite low value is used to not suppress too much energy between the harmonics. The value can also depend on the bitrate with a higher value at a lower bitrate, e.g. 0.4 at low bitrate and 0.2 at a high bitrate.
(24) α is used to add a slight tilt to the frequency response of H(z), in order to compensate for the slight loss in energy in the low frequencies. The value of a is generally chosen close to 1, e.g. 0.8.
(25) An example of B(z, T.sub.fr) is given in
(26) Particularly, it has been found out that even values for a between 0.6 and lower than 1.0 are useful and that, additionally, values for β between 0.1 and 0.5 have been proved to be useful as well.
(27) Furthermore, the multi-tap filter can have a variable number of taps. It has been found that for certain implementations, four taps are sufficient, where one tap is z.sup.+1. However, smaller filters with only two taps or even larger filters with more than four taps are useful for certain implementations.
(28)
(29) Particularly, as illustrated in
(30) Subsequently, an encoder implementation having certain functional blocks and operating without any pre-filter is illustrated in
(31) Subsequently, the functionality of the pitch estimator 402 is described.
(32) One pitch lag (integer part+fractional part) per frame is estimated (frame size e.g. 20 ms).
(33) This is done in 3 steps to reduce complexity and improves estimation accuracy.
(34) A pitch analysis algorithm that produces a smooth pitch evolution contour is used (e.g. Open-loop pitch analysis described in Rec. ITU-T G.718, sec. 6.6). This analysis is generally done on a subframe basis (subframe size e.g. 10 ms), and produces one pitch lag estimate per subframe. Note that these pitch lag estimates do not have any fractional part and are generally estimated on a downsampled signal (sampling rate e.g. 6400 Hz).
(35) The signal used can be any audio signal, e.g. a LPC weighted audio signal as described in Rec. ITU-T G.718, sec. 6.5.
(36) The pitch refiner operates as follows:
(37) The final integer part of the pitch lag is estimated on an audio signal x[n] running at the core encoder sampling rate, which is generally higher than the sampling rate of the downsampled signal used in a. (e.g. 12.8 kHz, 16 kHz, 32 kHz . . . ). The signal x[n] can be any audio signal e.g. an LPC weighted audio signal.
(38) The integer part of the pitch lag is then the lag that maximizes the autocorrelation function
(39)
with d around a pitch lag T estimated in step 1.a.
(40) The fractional part estimator 406 operates as follows:
T−δ.sub.1≤d≤T+δ.sub.2
(41) The fractional part is found by interpolating the autocorrelation function C(d) computed in step 2.b. and selecting the fractional pitch lag which maximizes the interpolated autocorrelation function. The interpolation can be performed using a low-pass FIR filter as described in e.g. Rec. ITU-T G.718, sec. 6.6.7.
(42) The transient detector 408 illustrated in
(43) If the input audio signal does not contain any harmonic content, then no parameters are encoded in the bitstream. Only 1 bit is sent such that the decoder knows whether he has to decode the post-filter parameters or not. The decision is made based on several parameters:
(44) a. Normalized correlation at the integer pitch lag estimated in step 1.b.
(45)
(46) The normalized correlation is 1 if the input signal is perfectly predictable by the integer pitch lag, and 0 if it is not predictable at all. A high value (close to 1) would then indicate a harmonic signal. For a more robust decision, the normalized correlation of the past frame can also be used in the decision, e.g.:
(47) If (norm.corr(curr.)*norm.corr.(prev.))>0.25, then the current frame contains some harmonic content (bit=1)
(48) b. Features computed by a transient detector (e.g. Temporal flatness measure, Maximal energy change), to avoid activating the post-filter on a signal containing a transient. e.g. If (tempFlatness>3.5 or maxEnergychange>3.5) then set bit=0 and do not send any parameters
(49) Furthermore, the gain estimator 410 calculates a gain to be input into the gain quantizer 412
(50) The gain is generally estimated on the input audio signal at the core encoder sampling rate, but it can also be any audio signal like the LPC weighted audio signal. This signal is noted y[n] and can be the same or different than x[n].
(51) The prediction y.sub.P[n] of y[n] is first found by filtering y[n] with the following filter
P(z)=B(z,T.sub.fr).sub.z.sup.−T.sup.
with T.sub.int the integer part of the pitch lag (estimated in 1.b.) and B(z, T.sub.fr) a low-pass FIR filter whose coefficients depend on the fractional part of the pitch lag T.sub.fr (estimated in 1.c.).
(52) One example of B(z) when the pitch lag resolution is ¼:
(53)
(54) The gain g is then computed as follows:
(55)
and limited between 0 and 1.
(56) Finally, the gain is quantized e.g. on 2 bits, using e.g. uniform quantization. If the gain is quantized to 0, then no parameters are encoded in the bitstream, only the one decision bit (bit=0).
(57) As outlined before, the post-filter is applied on the output audio signal after the transform decoder. It processes the signal on the frame-by-frame basis, with the same frame size as used it the encoder-side such as 20 ms. As illustrated, it is based on a long-term prediction filter H(z) whose parameters are determined from the parameters estimated at the encoder-side and decoded from the bitstream. This information comprises the decision bit, the pitch lag and the gain. If the decision bit is 0, then the pitch lag and the gain are not decoded and are assumed to be 0 not written at all into the bitstream.
(58) As discussed, if the filter parameters are different from one frame to the next frame, a discontinuity can be introduced at the border between the two frames. To avoid discontinuity, a discontinuity remover is applied such as a cross-fader or any other implementation for that purpose.
(59) Furthermore, several different ways to set the harmonic post-filter are illustrated in
(60) Particularly,
(61)
(62) Furthermore,
(63) On the other hand, β equal 0.2 has a less strong effect for suppressing energy between the harmonics and, therefore, this β-value is advantageous for high bitrates due to the fact that at such higher bitrates, not so much inter-harmonic noise exists.
(64) Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.
(65) The inventive transmitted or encoded signal can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.
(66) Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
(67) Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
(68) Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.
(69) Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
(70) In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
(71) A further embodiment of the inventive method is, therefore, a data carrier (or a non-transitory storage medium such as a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitory.
(72) A further embodiment of the invention method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.
(73) A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.
(74) A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
(75) A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
(76) In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods may be performed by any hardware apparatus.
(77) While this invention has been described in terms of several embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations and equivalents as fall within the true spirit and scope of the present invention.