Transform encoding/decoding of harmonic audio signals
11264041 · 2022-03-01
Assignee
Inventors
- Volodya Grancharov (Solna, SE)
- Tomas Jansson Toftgård (Uppsala, SE)
- Sebastian Näslund (Solna, SE)
- Harald Pobloth (Täby, SE)
Cpc classification
International classification
G10L19/028
PHYSICS
G10L19/02
PHYSICS
Abstract
An encoder for encoding frequency transform coefficients of a harmonic audio signal include the following elements: A peak locator configured to locate spectral peaks having magnitudes exceeding a predetermined frequency dependent threshold. A peak region encoder configured to encode peak regions including and surrounding the located peaks. A low-frequency set encoder configured to encode at least one low-frequency set of coefficients outside the peak regions and below a crossover frequency that depends on the number of bits used to encode the peak regions. A noise-floor gain encoder configured to encode a noise-floor gain of at least one high-frequency set of not yet encoded coefficients outside the peak regions.
Claims
1. A method of encoding Modified Discrete Cosine Transform (MDCT) coefficients Y(k) of a harmonic audio signal, said method including the steps of: locating spectral peaks having magnitudes exceeding a predetermined threshold, wherein the spectral peaks are located by comparing coefficients to said threshold to form a vector of peak candidates, and extracting elements from the peak candidates vector in decreasing order; encoding peak regions including and surrounding the located peaks, wherein the spectral peaks are quantized together with neighboring MDCT bins; encoding, using a number of reserved bits, a first low-frequency (LF) set of coefficients outside the peak regions and below a crossover frequency that depends on the number of bits used to encode the peak regions, wherein encoding comprises encoding one or more further low-frequency sets of coefficients outside the peak regions if there are non-reserved bits available after encoding the peak regions; encoding, using a number of reserved bits, a noise-floor gain of at least one high-frequency set of not yet encoded coefficients outside the peak regions.
2. The encoding method of claim 1, wherein said threshold is calculated as
3. The encoding method of claim 1, where a weighting factor α is defined as
4. The encoding method of claim 1, wherein the step of encoding peak regions comprises: encoding spectrum position and sign of a peak; quantizing peak gain; encoding the quantized peak gain; scaling predetermined frequency bins surrounding the peak by the inverse of the quantized peak gain; and shape encoding the scaled frequency bins.
5. The encoding method of claim 1, wherein the peak region comprises the peak and four MDCT bins surrounding said peak.
6. The encoding method of claim 1, wherein the step of encoding low-frequency set of coefficients comprises grouping remaining un-quantized MDCT coefficients into 24-dimensional bands.
7. The encoding method of claim 1, wherein encoding of a low-frequency set is based on a gain-shape encoding scheme, said gain-shape encoding scheme being based on scalar gain quantization and factorial pulse shape encoding.
8. The encoding method of claim 1, including the step of encoding a noise-floor gain for each of two high-frequency sets.
9. An encoder for encoding Modified Discrete Cosine Transform (MDCT) coefficients Y(k) of a harmonic audio signal, said encoder comprising: a peak locator configured to locate spectral peaks having magnitudes exceeding a predetermined threshold, wherein the spectral peaks are located by comparing coefficients to said threshold to form a vector of peak candidates, and extracting elements from the peak candidates vector in decreasing order; a peak region encoder configured to encode peak regions including and surrounding the located peaks, wherein the spectral peaks are quantized together with neighboring MDCT bins; a low-frequency set encoder configured to encode, using a number of reserved bits, a first low-frequency set of coefficients outside the peak regions and below a crossover frequency that depends on the number of bits used to encode the peak regions, and to encode one or more further low-frequency set of coefficients outside the peak regions if there are non-reserved bits available after encoding the peak regions; and a noise-floor gain encoder configured to encode, using a number of reserved bits, a noise-floor gain of at least one high-frequency set of not yet encoded coefficients outside the peak regions.
10. The encoder of claim 9, wherein said threshold is calculated as
11. The encoder of claim 9, wherein the peak region encoder comprises: a position and sign encoder configured to encode spectrum position and sign of a peak; a peak gain encoder configured to quantize peak gain and to encode the quantized peak gain; a scaling unit configured to scale predetermined frequency bins surrounding the peak by the inverse of the quantized peak gain; a shape encoder configured to shape encode the scaled frequency bins.
12. A user equipment (UE) comprising: radio communication circuitry; and processing circuitry operatively associated with the radio communication circuitry and operative to encode Modified Discrete Cosine Transform (MDCT) coefficients Y(k) of a harmonic audio signal, based on said processing circuitry being configured to: locate spectral peaks having magnitudes exceeding a predetermined threshold, wherein the spectral peaks are located by comparing coefficients to said threshold to form a vector of peak candidates, and extracting elements from the peak candidates vector in decreasing order; encode peak regions including and surrounding the located peaks, wherein the spectral peaks are quantized together with neighboring MDCT bins; encode, using a number of reserved bits, a first low-frequency set of coefficients outside the peak regions and below a crossover frequency that depends on the number of bits used to encode the peak regions, and to encode one or more further low-frequency set of coefficients outside the peak regions if there are non-reserved bits available after encoding the peak regions; and encode, using a number of reserved bits, a noise-floor gain of at least one high-frequency set of not yet encoded coefficients outside the peak regions.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The present technology, together with further objects and advantages thereof, may best be understood by making reference to the following description taken together with the accompanying drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
DETAILED DESCRIPTION
(20)
(21) The proposed technology provides an alternative audio encoding model that handles harmonic audio signals better. The main concept is that the frequency transform vector, for example an MDCT vector, is not split into envelope and residual part, but instead spectral peaks are directly extracted and quantized, together with neighboring MDCT bins. At high frequencies, low energy coefficients outside the peaks neighborhoods are not coded, but noise-filled at the decoder. Here the signal model used in the conventional encoding, {spectrum envelope+residual} is replaced with a new model {spectral peaks+noise-floor}. At low frequencies, coefficients outside the peak neighborhoods are still coded, since they have an important perceptual role.
(22) Encoder
(23) Major steps on the encoder side are: Locate and code spectral peak regions; Code low-frequency (LF) spectral coefficients—the size of coded region depends on the number of bits remaining after peak region coding; and Code noise-floor gains for spectral coefficients outside the peak regions.
(24) First the noise-floor is estimated, then the spectral peaks are extracted by a peak picking algorithm (the corresponding algorithms are described in more detail in APPENDIX I-II). Each peak and its surrounding 4 neighbors are normalized to unit energy at the peak position, see
(25) In the above example each peak region includes 4 neighbors that symmetrically surround the peak. However it is also feasible to have both fewer and more neighbors surrounding the peak in either symmetrical or asymmetrical fashion.
(26) After the peak regions have been quantized, all available remaining bits (except reserved bits for noise-floor coding, see below) are used to quantize the low frequency MDCT coefficients. This is done by grouping the remaining un-quantized MDCT coefficients into, for example, 24-dimensional bands starting from the first bin. Thus, these bands will cover the lowest frequencies up to a certain crossover frequency. Coefficients that have already been quantized in the peak coding are not included, so the bands are not necessarily made up from 24 consecutive coefficients. For this reason the bands will also be referred to as “sets” below.
(27) The total number of LF bands or sets depends on the number of available bits, but there are always enough bits reserved to create at least one set. When more bits are available the first set gets more bits assigned until a threshold for the maximum number of bits per set is reached. If there are more bits available another set is created and bits are assigned to this set until the threshold is reached. This procedure is repeated until all available bits have been spent. This means that the crossover frequency at which this process is stopped will be frame dependent, since the number of peaks will vary from frame to frame. The crossover frequency will be determined by the number of bits that are available for LF encoding once the peak regions have been encoded.
(28) Quantization of the LF sets can be done with any suitable vector quantization scheme, but typically some type of gain-shape encoding is used. For example, factorial pulse coding may be used for the shape vector, and scalar quantizer may be used for the gain.
(29) A certain number of bits are always reserved for encoding a noise-floor gain of at least one high-frequency band of coefficients outside the peak regions, and above the upper frequency of the LF bands. Preferably two gains are used for this purpose. These gains may be obtained from the noise-floor algorithm described in APPENDIX I. If factorial pulse coding is used for the encoding the low-frequency bands some LF coefficients may not be encoded. These coefficients can instead be included in the high-frequency band encoding. As in the case of the LF bands, the HF bands are not necessarily made up from consecutive coefficients. For this reason the bands will also be referred to as “sets” below.
(30) If applicable, the spectrum envelope for a bandwidth extension (BWE) region is also encoded and transmitted. The number of bands (and the transition frequency where the BWE starts) is bitrate dependent, e.g. 5.6 kHz at 24 kbps and 6.4 kHz at 32 kbps.
(31)
(32)
(33)
(34) Decoder
(35) Major steps on the decoder are: Reconstruct spectral peak regions; Reconstruct LF spectral coefficients; and Noise-fill non-coded regions with noise, scaled with the received noise-floor gains.
(36) The audio decoder extracts, from the bit-stream, the number of peak regions and the quantization indices {I.sub.position I.sub.gain I.sub.sign I.sub.shape} in order to reconstruct the coded peak regions. These quantization indices contain information about the spectral peak position, gain and sign of the peak, as well as the index for the codebook vector that provides the best match for the peak neighborhood.
(37) The MDCT low-frequency coefficients outside the peak regions are reconstructed from the encoded LF coefficients.
(38) The MDCT high-frequency coefficients outside the peak regions are noise-filled at the decoder. The noise-floor level is received by the decoder, preferably in the form of two coded noise-floor gains (one for the lower and one for the upper half or part of the vector).
(39) If applicable, the audio decoder performs a BWE from a pre-defined transition frequency with the received envelope gains for HF MDCT coefficients.
(40)
(41) In an example embodiment the decoding of a low-frequency set is based on a gain-shape decoding scheme.
(42) In an example embodiment the gain-shape decoding scheme is based on scalar gain decoding and factorial pulse shape decoding.
(43) An example embodiment includes the step of decoding a noise-floor gain for each of two high-frequency sets.
(44)
(45)
(46) The steps, functions, procedures and/or blocks described herein may be implemented in hardware using any conventional technology, such as discrete circuit or integrated circuit technology, including both general-purpose electronic circuitry and application-specific circuitry.
(47) Alternatively, at least some of the steps, functions, procedures and/or blocks described herein may be implemented in software for execution by suitable processing equipment. This equipment may include, for example, one or several microprocessors, one or several Digital Signal Processors (DSP), one or several Application Specific Integrated Circuits (ASIC), video accelerated hardware or one or several suitable programmable logic devices, such as Field Programmable Gate Arrays (FPGA). Combinations of such processing elements are also feasible.
(48) It should also be understood that it may be possible to reuse the general processing capabilities already present in the encoder/decoder. This may, for example, be done by reprogramming of the existing software or by adding new software components.
(49)
(50)
(51) The technology described above is intended to be used in an audio encoder/decoder, which can be used in a mobile device (e.g. mobile phone, laptop) or a stationary device, such as a personal computer. Here the term User Equipment (UE) will be used as a generic name for such devices.
(52)
(53) The decision of the harmonic signal detector 78 is based on the noise-floor energy Ē.sub.nf and peak energy Ē.sub.p in APPENDIX I and II. The logic is as follows: IF Ē.sub.p/Ē.sub.nf is above a threshold AND the number of detected peaks is in a predefined range THEN the signal is classified as harmonic. Otherwise the signal is classified as non-harmonic. The classification and thus the encoding mode is explicitly signaled to the decoder.
(54)
(55)
(56)
(57)
(58)
(59) Specific implementation details for a 24 kbps mode are given below. The codec operates on 20 ms frames, which at a bit rate of 24 kbps gives 480 bits per-frame. The processed audio signal is sampled at 32 kHz, and has an audio bandwidth of 16 kHz. The transition frequency is set to 5.6 kHz (all frequency components above 5.6 kHz are bandwidth-extended). Reserved bits for signaling and bandwidth extension of frequencies above the transition frequency: ˜30-40. Bits for coding two noise-floor gains: 10. The number of coded spectral peak regions is 7-17. The number of bits used per peak region is ˜20-22, which gives a total number of ˜140-340 for coding all peaks positions, gains, signs, and shapes. Bits for coding low frequency bands: ˜100-300. Coded low frequency bands: 1-4 (each band contains 8 MDCT bins). Since each MDCT bin corresponds to 25 Hz, coded low-frequency region corresponds to 200-800 Hz. The gains used for bandwidth extension and the peak gains are Huffman coded so the number of bits used by these might vary between frames even for a constant number of peaks. The peak position and sign coding makes use of an optimization which makes it more efficient as the number of peaks increase. For 7 peaks, position and sign requires about 6.9 bits per peak and for 17 peaks the number is about 5.7 bits per peak.
(60) This variability in how many bits are used in different stages of the coding is no problem since the low frequency band coding comes last and just uses up whatever bits remain. However the system is designed so that enough bits always remain to encode one low frequency band.
(61) The table below presents results from a listening test performed in accordance with the procedure described in ITU-R BS.1534-1 MUSHRA (Multiple Stimuli with Hidden Reference and Anchor). The scale in a MUSHRA test is 0 to 100, where low values correspond to low perceived quality, and high values correspond to high quality. Both codecs operated at 24 kbps. Test results are averaged over 24 music items and votes from 8 listeners.
(62) TABLE-US-00001 System Under Test MUSHRA Score Low-pass anchor signal (bandwidth 7 kHz) 48.89 Conventional coding scheme 49.94 Proposed harmonic coding scheme 55.87 Reference signal (bandwidth 16 kHz) 100.00
(63) It will be understood by those skilled in the art that various modifications and changes may be made to the proposed technology without departure from the scope thereof, which is defined by the appended claims.
APPENDIX I
(64) The noise-floor estimation algorithm operates on the absolute values of transform coefficients |Y(k)|. Instantaneous noise-floor energies E.sub.nf(k) are estimated according to the recursion:
(65)
(66) The particular form of the weighting factor α minimizes the effect of high-energy transform coefficients and emphasizes the contribution of low-energy coefficients. Finally, the noise-floor level Ē.sub.nf is estimated by simply averaging the instantaneous energies E.sub.nf(k).
APPENDIX II
(67) The peak-picking algorithm requires knowledge of noise-floor level and average level of spectral peaks. The peak energy estimation algorithm is similar to the noise-floor estimation algorithm, but instead of low-energy, it tracks high-spectral energies:
(68)
(69) In this case the weighting factor β minimizes the effect of low-energy transform coefficients and emphasizes the contribution of high-energy coefficients. The overall peak energy Ē.sub.p is estimated by simply averaging the instantaneous energies.
(70) When the peak and noise-floor levels are calculated, a threshold level θ is formed as:
(71)
with γ=0.88579. Transform coefficients are compared to the threshold, and the ones with amplitude above it, form a vector of peak candidates. Since the natural sources do not typically produce peaks that are very close, e.g., 80 Hz, the vector with peak candidates is further refined. Vector elements are extracted in decreasing order, and the neighborhood of each element is set to zero. In this way only the largest element in certain spectral region remain, and the set of these elements form the spectral peaks for the current frame.
ABBREVIATIONS
(72) ASIC Application Specific Integrated Circuit BWE BandWidth Extension DSP Digital Signal Processors FPGA Field Programmable Gate Arrays HF High-Frequency LF Low-Frequency MDCT Modified Discrete Cosine Transform RMS Root Mean Square VQ Vector Quantizer