LOW-COMPLEXITY TONALITY-ADAPTIVE AUDIO SIGNAL QUANTIZATION
20210366499 · 2021-11-25
Inventors
- Martin DIETZ (Nuernberg, DE)
- Guillaume Fuchs (Bubenrath, DE)
- Christian Helmrich (Erlangen, DE)
- Goran MARKOVIC (Nuernberg, DE)
Cpc classification
G10L25/18
PHYSICS
G10H1/06
PHYSICS
International classification
G10H1/06
PHYSICS
G10L19/02
PHYSICS
G10L25/18
PHYSICS
Abstract
The invention provides an audio encoder for encoding an audio signal so as to produce therefrom an encoded signal, the audio encoder including: a framing device configured to extract frames from the audio signal; a quantizer configured to map spectral lines of a spectrum signal derived from the frame of the audio signal to quantization indices, wherein the quantizer has a dead-zone, in which the input spectral lines are mapped to quantization index zero; and a control device configured to modify the dead-zone; wherein the control device includes a tonality calculating device configured to calculate at least one tonality indicating value for at least one spectrum line or for at least one group of spectral lines, wherein the control device is configured to modify the dead-zone for the at least one spectrum line or the at least one group of spectrum lines depending on the respective tonality indicating value.
Claims
1. Audio encoder for encoding an audio signal so as to produce therefrom an encoded signal, the audio encoder comprising: a framing device configured to extract frames from the audio signal; a quantizer configured to map spectral lines of a spectrum signal derived from the frame of the audio signal to quantization indices, wherein the quantizer comprises a dead-zone, in which the spectral lines are mapped to quantization index zero; and a control device configured to modify the dead-zone; wherein the control device comprises a tonality calculating device configured to calculate at least one tonality indicating value for at least one spectrum line or for at least one group of spectral lines, wherein the control device is configured to modify the dead-zone for the at least one spectrum line or the at least one group of spectrum lines depending on the respective tonality indicating value.
2. Audio encoder according to claim 1, wherein the control device is configured to modify the dead-zone in such way that the dead-zone at one of the spectral lines is larger than the dead-zone is at one of the spectral lines comprising a larger tonality or in such way that the dead-zone at one of the groups of spectral lines is larger than the dead-zone is at one of the groups of spectral lines comprising a larger tonality.
3. Audio encoder according to claim 1, wherein the control device comprises a power spectrum calculating device configured to calculate a power spectrum of the frame of the audio signal, wherein the power spectrum comprises power values for spectral lines or groups of spectral lines, wherein the tonality calculating device is configured to calculate the at least one tonality indicating value depending on the power spectrum.
4. Audio encoder according to claim 3, wherein the tonality indicating value for one of the spectral lines is based on a comparison of the power value for the respective spectral line and the sum of a predefined number of its surrounding power values of the power spectrum, or wherein the tonality indicating value for one of the groups of the spectral lines is based on a comparison of the power value for the respective group of spectral lines and the sum of a predefined number of its surrounding power values of the power spectrum.
5. Audio encoder according to claim 1, wherein the tonality indicating value for one of the spectral lines is based on the tonality indicating value of the spectral line of a preceding frame of the audio signal, or wherein the tonality indicating value for one of the groups of the spectral lines is based on the tonality indicating value of the group of spectral lines for a preceding frame of the audio signal.
6. Audio encoder according to claim 3, wherein the tonality indicating value is calculated by a formula
7. Audio encoder according to claim 1, wherein the audio encoder comprises a start frequency calculating device configured to calculate a start frequency for modifying the dead-zone, wherein the dead-zone is only modified for spectral lines representing a frequency higher than or equal to the start frequency.
8. Audio encoder according to claim 7, wherein start frequency calculating device is configured to calculate the start frequency based on a sample rate of the audio signal and/or based on a maximum bit-rate foreseen for a bitstream produced from the encoded signal.
9. Audio encoder according to claim 3, wherein the audio encoder comprises a modified discrete cosine transform calculating device configured to calculate a modified discrete cosine transform from the frame of the audio signal and a modified discrete sine transform calculating device configured to calculate a modified discrete sine transform from the frame of the audio signal, wherein the power spectrum calculating device is configured to calculate the power spectrum based on the modified discrete cosine transform and on the modified discrete sine transform.
10. Audio encoder according to claim 3, wherein power spectrum calculating device is configured to calculate the power values according to a formula P.sub.k,i=(MDCT.sub.k,i).sup.2+(MDST.sub.k,i).sup.2, wherein i is an index indicating a specific frame of the audio signal, k is an index indicating a specific spectral line, MDCT.sub.k,i is the value of the modified discrete cosine transform at the k-th spectral line of the i-th frame, MDST.sub.k,i is the value of the modified discrete sine transform at the k-th spectral line of the i-th frame, and P.sub.k,i is the power value of the k-th spectral line of the i-th frame.
11. Audio encoder according to claim 1, wherein the audio encoder comprises a spectrum signal calculating device configured to produce the spectrum signal, wherein the spectrum signal calculating device comprises an amplitude setting device configured to set amplitudes of the spectral lines of the spectrum signal in such way that an energy loss due to a modification of the dead-zone is compensated.
12. Audio encoder according to claim 11, wherein the amplitude setting device is configured to set the amplitudes of the spectrum signal depending on a modification of the dead-zone at the respective spectral line.
13. Audio encoder according to claim 11, wherein the spectrum signal calculating device comprises a normalizing device.
14. Audio encoder according to claim 11, wherein the modified discrete cosine transform from the frame of the audio signal calculated by the modified discrete cosine transform calculating device is fed to the spectrum signal calculating device
15. A system comprising an encoder and a decoder, wherein the encoder is designed according to claim 1.
16. Method for encoding an audio signal so as to produce therefrom an encoded signal, the method comprising: extracting frames from the audio signal; mapping spectral lines of a spectrum signal derived from the frame of the audio signal to quantization indices, wherein a dead-zone is used, in which the input spectral lines are mapped to quantization index zero; and modifying the dead-zone; wherein at least one tonality indicating value for at least one spectrum line or for at least one group of spectral lines is calculated, wherein the dead-zone for the at least one spectrum line or the at least one group of spectrum lines is modified depending on the respective tonality indicating value.
17. Computer program for performing, when running on a computer or a processor, the method of claim 16.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Embodiments of the present invention will be detailed subsequently referring to the appended drawings, in which:
[0049]
[0050]
DETAILED DESCRIPTION OF THE INVENTION
[0051]
[0052] a framing device 2 configured to extract frames F from the audio signal AS;
[0053] a quantizer 3 configured to map spectral lines SL.sub.1-32 (see
[0054] a control device 4 configured to modify the dead-zone DZ;
[0055] wherein the control device 4 comprises a tonality calculating device 5 configured to calculate at least one tonality indicating value TI.sub.5-32 for at least one spectrum line SL.sub.1-32 or for at least one group of spectral lines SL.sub.1-32,
[0056] wherein the control device 4 is configured to modify the dead-zone DZ for the at least one spectrum line SL.sub.1-32 or the at least one group of spectrum lines SL.sub.1-32 depending on the respective tonality indicating value TI.sub.5-32.
[0057] The framing device 2 may be configured to extract frames F from the audio signal AS by the application of a window function to the audio signal AS. In signal processing, a window function (also known as an apodization function or tapering function) is a mathematical function that is zero-valued outside of some chosen interval. By the application of the window function to the signal AS, the signal AS can be broken into short segments, which are usually called frames F.
[0058] Quantization, in digital audio signal processing, is the process of mapping a large set of input values to a (countable) smaller set—such as rounding values to some unit of precision. A device or algorithmic function that performs quantization is called a quantizer.
[0059] According to the invention a spectrum signal SPS is calculated for the frames F of the audio signal AS. The spectrum signal SPS may contain a spectrum of each of the frames F of the audio signal AS, which is a time-domain signal, wherein each spectrum is a representation of one of the frames F in the frequency domain. The frequency spectrum can be generated via a mathematical transform of the signal AS, and the resulting values are usually presented as amplitude versus frequency.
[0060] The dead-zone DZ is a zone used during quantization, wherein spectral lines SL.sub.1-32 (frequency bins) or groups of spectral lines SL.sub.1-32 (frequency bands) are mapped to quantization index zero. The dead-zone DZ has a lower limit, which is usually at an amplitude of zero, and an upper limit, which may vary for different spectral lines SL.sub.1-32 or groups of spectral lines SL.sub.1-32.
[0061] According to the invention the dead-zone DZ is may be modified by a control device 4. The control device 4 comprises a tonality calculating device 5 which is configured to calculate at least one tonality indicating value TI.sub.5-32 for at least one spectrum line SL.sub.1-32 or for at least one group spectrum lines. SL.sub.1-32
[0062] The term “tonality” refers to the tonal character of the spectrum signal SPS. In general it may be said that the tonality is high in case that the spectrum or a part thereof comprises predominantly periodic components, which means that the spectrum or the part thereof of a frame F comprises dominant peaks. The opposite of a tonal character is a noisy character. In the latter case the spectrum or the part thereof of a frame F is more flat.
[0063] Furthermore, the control device 4 is configured to modify the dead-zone DZ for the at least one spectrum line SL.sub.1-32 or the at least one group of spectrum lines SL.sub.1-32 depending on the respective tonality indicating value TI.sub.5-32.
[0064] The present invention reveals a quantization scheme with a signal-adaptive dead-zone DZ which [0065] does not necessitate any side-information, allowing its usage in existing media codecs, [0066] decides prior to quantization which dead-zone DZ to use per bin or band, saving complexity, [0067] may determine the per-bin or per-band dead-zone DZ based on band frequency and/or signal tonality.
[0068] The invention can be applied in existing coding infrastructure since only the signal quantizer 3 in the encoder 1 is changed; the corresponding decoder will still be able to read the (unaltered) bitstream produced from the encoded signal and decode the output. Unlike in [6] and references therein, the dead-zone DZ for each group of spectral lines SL.sub.1-32 or for each spectral line SL.sub.1-32 is selected before quantization, so only one quantization operation per group or spectral line SL.sub.1-32 is necessitated. Finally, the quantizer decision is not limited to choose between two possible dead-zone values, but an entire range of values. The tonality-adaptive quantization scheme outlined above may be implemented in the transform coded excitation (TCX) path of the LD-USAC encoder, a low-delay variant of xHE-AAC [4].
[0069] According to an embodiment of the invention the control device 4 is configured to modify the dead-zone DZ in such way that the dead-zone DZ at one of the spectral lines SL.sub.1-32 is larger than the dead-zone DZ is at one of the spectral lines SL.sub.1-32 having a larger tonality or in such way that the dead-zone DZ at one of the groups of spectral lines SL.sub.1-32 is larger than the dead-zone DZ is at one of the groups of spectral lines SL.sub.1-32 having a larger tonality. By this features non-tonal spectral regions will tend to be quantized to zero, which means that the quantity of the data may be reduced.
[0070] According to an embodiment of the invention the control device 4 comprises a power spectrum calculating device 6 configured to calculate a power spectrum PS (see also
[0071] According to an embodiment of the invention the tonality indicating value TI.sub.5-32 for one of the spectral lines SL.sub.1-32 is based on a comparison of the power value PS.sub.5-32 for the respective spectral line SL.sub.1-32 and the sum of a predefined number of its surrounding power values PS.sub.5-32 of the power spectrum PS, or wherein the tonality indicating value for one of the groups of the spectral lines SL.sub.1-32 is based on a comparison of the power value PS.sub.5-32 for the respective group of spectral lines and the sum of a predefined number of its surrounding power values PS.sub.5-32 of the power spectrum. By comparing a power value PS.sub.5-32 with its neighboring power values PS.sub.5-32 peak areas or flat areas of the power spectrum SP may be easily identified so that the tonality indicating value TI.sub.5-32 may be calculated in an easy way.
[0072] According to an embodiment of the invention the tonality indicating value TI.sub.5-32 for one of the spectral lines SL.sub.1-32 is based on the tonality indicating value TI.sub.5-32 of the spectral line SL.sub.1-32 of a preceding frame F of the audio signal AS, or wherein the tonality indicating value TI.sub.5-32 for one of the groups of the spectral lines SL.sub.1-32 is based on the tonality indicating value TI.sub.5-32 of the group of spectral lines SL.sub.1-32 for a preceding frame F of the audio signal AS. By these features the dead-zone DZ will be modified over time in a smooth manner.
[0073] According to an embodiment of the invention the tonality indicating value TI.sub.5-32 is calculated by a formula
wherein i is an index indicating a specific frame F of the audio signal AS, k is an index indicating a specific spectral line SL.sub.1-32, P.sub.k,i is the power value PS.sub.5-32 of the k-th spectral line SL.sub.1-32 of the i-th frame, or wherein the tonality indicating value TI.sub.5-32 is calculated by a formula
wherein i is an index indicating a specific frame F of the audio signal AS, m is an index indicating a specific group of spectral lines SL.sub.1-32, P.sub.m,i is the power value PS.sub.5-32 of the m-th group of spectral lines SL.sub.1-32 of the i-th frame. As one will note from the formula the tonality indicating value TI.sub.5-32 is calculated from power value PS.sub.5-32 of the i-th frame, which is the current frame F, and from the i-1-th frame F, which is the preceding frame F. The formula may be changed by omitting the dependency from the i-1-th frame F. Here the sum of the 7 left and 7 right neighboring power values PS.sub.5-32 of the k-th power value PS.sub.5-32 of a certain spectral line SL.sub.1-32 or the m-th power value of group of spectral lines SL.sub.1-32 is calculated and divided by the respective power value PS.sub.5-32. Using this formula a low tonality indicating value TI.sub.5-32 indicates a high tonality.
[0074] According to an embodiment of the invention the audio encoder 1 comprises a start frequency calculating device 7 configured to calculate a start frequency SF for modifying the dead-zone DZ, wherein the dead-zone DZ is only modified for spectral lines SL.sub.5-32 representing a frequency higher than or equal to the start frequency SF. This means that the dead-zone DZ is fixed for low frequencies and variable for higher frequencies. These features lead to better audio quality as the human auditory system is more sensitive at low frequencies.
[0075] According to an embodiment of the invention the start frequency calculating device 7 is configured to calculate the start frequency SF based on a sample rate of the audio signal AS and/or based on a maximum bit-rate foreseen for a bitstream produced from the encoded signal ES. By these features will audio quality may be optimized.
[0076] According to an embodiment of the invention the audio encoder 1 comprises a modified discrete cosine transform calculating device 8 configured to calculate a modified discrete cosine transform CT from the frame F of the audio signal AS and a modified discrete sine transform calculating device 9 configured to calculate a modified discrete sine transform ST from the frame F of the audio signal AS, wherein the power spectrum calculating device 6 is configured to calculate the power spectrum PS based on the modified discrete cosine transform CT and on the modified discrete sine transform ST. The modified discrete cosine transform CT has to be calculated anyway in many cases for the purpose of encoding the audio signal AS. Hence, only the modified discrete sine transform ST has to be calculated additionally for the purpose of tonality-adaptive quantization. Therefore, complexity may be reduced. However, other transforms may be used such as discrete Fourier transform or odd discrete Fourier transform.
[0077] According to an embodiment of the invention the power spectrum calculating device 6 is configured to calculate the power values according to the formula P.sub.k,i=(MDCT.sub.k,i).sup.2+(MDST.sub.k,i).sup.2, wherein i is an index indicating a specific frame F of the audio signal, k is an index indicating a specific spectral line SL.sub.1-32, MDCT.sub.k,i is the value of the modified discrete cosine transform CT at the k-th spectral line of the i-th frame, MDST.sub.k,i is the value of the modified discrete sine transform ST at the k-th spectral line of the i-th frame, and P.sub.k,i is the power value PS.sub.5-32 of the k-th spectral line of the i-th frame. The formula above allows to calculate the power values PS.sub.5-32 in an easy way.
[0078] According to an embodiment of the invention the audio encoder 1 comprises a spectrum signal calculating device 10 configured to produce the spectrum signal SPS, wherein the spectrum signal calculating device 10 comprises an amplitude setting device 11 configured to set amplitudes of the spectral lines SL.sub.1-32 of the spectrum signal SPS in such way that an energy loss due to a modification of the dead-zone DZ is compensated. By these features the quantization may be done in an energy preserving way
[0079] According to an embodiment of the invention the amplitude setting device 11 is configured to set the amplitudes of the spectrum signal SPS depending on a modification of the dead-zone DZ at the respective spectral line SL.sub.1-32. For example spectral lines SL.sub.1-32, for which the dead-zone DZ is enlarged, may be slightly amplified for this purpose.
[0080] According to an embodiment of the invention the spectrum signal calculating device 10 comprises a normalizing device 12. By this feature the subsequent quantization step may be done in an easy way.
[0081] According to an embodiment of the invention the modified discrete cosine transform CT from the frame F of the audio signal AS calculated by the modified discrete cosine transform calculating device 8 is fed to the spectrum signal calculating device 10. By these feature the modified discrete cosine transform CT is used for the purpose of quantization adaption and for the purpose of calculating the encoded signal ES.
[0082]
[0083]
[0084] Below a start frequency SF, which has been calculated by the start frequency calculating device 7, the dead-zone has a fixed size. In the example the spectral line SL.sub.1 ends outside of the dead-zone so that it will be mapped to the index one I.sub.1, whereas the spectral line SL.sub.7 ends within the dead-zone DZ so that it can be mapped to index 0 I.sub.0. However, beginning with the start frequency SF and going to higher frequencies, the size of the dead-zone DZ may be modified by the control device 4. For that purpose, the power values PS.sub.5-32 are calculated as described above. Furthermore, the tonality indicating values TI.sub.5-32 are calculated from the power values PS.sub.5-32.
[0085] In the area from k=20 to k=23 the power spectrum PS has a peak which results in low tonality indicating values TI.sub.20-23 which indicate a high tonality. In the other areas above the start frequency SF for power spectrum PS is more flat so that the tonality indicating values TI.sub.12-19 and TI.sub.24-32 are comparably higher, which indicates a lower tonality in their respective areas. As a result the dead-zone DZ is enlarged in the area from k=12 to k=19 and in the area from k=24 to k=32. This enlargement of the dead-zone DZ results in that, for example, that spectral line SL.sub.12 and spectral line SL.sub.25, which without tonality adaptive quantization would have been mapped to index one are now mapped to index zero. This zero-quantization reduces the quantity of the data to be transmitted to the decoder.
[0086] In an implementation of the invention, the encoder operation is summarized as follows: [0087] 1. During the time-to-frequency transformation step, both an MDCT (cosine part) and an MDST (sine part) are computed from the windowed input signal for the given frame. [0088] 2. The MDCT of the input frame is used for quantization, coding, and transmission. The MDST is further utilized to compute a per-bin power spectrum Pk=MDCTk.sup.2+MDSTk.sup.2. [0089] 3. With Pk a per-coding-band, or advantageously per-bin, tonality or spectral flatness measure is calculated. Several methods to achieve this are documented in the literature [1,2,3]. Advantageously, a low-complexity version with only few operations per bin is employed. In the present case, a comparison between Pk and the sum of its surrounding Pk−7 . . . k+7 is made and enhanced with a hysteresis similar to the birth/death tracker described in [3]. Moreover, bins below a certain bit-rate-dependent frequency are regarded tonal. [0090] 4. As an optional step, the tonality or flatness measure can be utilized to perform a slight amplification of the spectrum prior to quantization in order to compensate for energy loss due to a large quantizer dead-zone. More precisely, bins for which a large quantizer dead-zone is applied are amplified a bit, whereas bins for which a normal or close to-normal dead-zone (i.e. one that tends to preserve energy) is used are not modified. [0091] 5. The tonality or flatness measure of step 3 now controls the choice of dead-zone used for quantizing each frequency bin. Bins determined as having a high tonality, meaning low values of Pk−7 . . . k+7/Pk, are quantized with a default (i.e. roughly energy preserving) dead-zone, and bins with low tonality are quantized with a new enlarged dead-zone. A low-tonality bin thus tends to be quantized to zero more often than a high-tonality bin. Optionally, the size of a bin's dead-zone can be defined as a continuous function of bin tonality, with a range between the default (smallest) and a maximum dead-zone size.
[0092] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.
[0093] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a non-transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
[0094] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
[0095] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.
[0096] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
[0097] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
[0098] A further embodiment of the inventive method is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitionary.
[0099] A further embodiment of the invention method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.
[0100] A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.
[0101] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
[0102] A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
[0103] In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are performed by any hardware apparatus.
[0104] While this invention has been described in terms of several advantageous embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations, and equivalents as fall within the true spirit and scope of the present invention.
REFERENCES
[0105] [1] L. Daudet, “Sparse and Structured Decomposition of Signals with the Molecular Matching Pursuit,” IEEE Trans. on Audio, Speech, and Lang. Processing, Vol. 14, No. 5, September 2006. [0106] [2] F. Keiler, “Survey on Extraction of Sinusoids in Stationary Sounds,” in Proc. DAFX, 2002. [0107] [3] R.J. McAulay and T. F. Quatieri, “Speech Analysis/Synthesis Based on a Sinusoidal Representation,” IEEE Trans. Acoustics, Speech, and Sig. Processing, Vol. 34, No. 4, August 1986. [0108] [4] M. Neuendorf et al., “MPEG Unified Speech and Audio Coding—The ISO/MPEG Standard for High-Efficiency Audio Coding of All Content Types,” in Proc. 132nd Convention of the AES, Budapest, Hungary, April 2012. Also to appear in the Journal of the AES, 2013. [0109] [5] M. Oger et al., “Model-Based Deadzone Optimization for Stack-Run Audio Coding with Uniform Scalar Quantization,” in Proc. ICASSP 2008, Las Vegas, USA, April 2008. [0110] [6] M. Schug, EP2122615, “Apparatus and method for encoding an information signal”, 2007.