Generation of comfort noise
10891964 · 2021-01-12
Assignee
Inventors
Cpc classification
International classification
Abstract
A User Equipment (UE) is operative to generate CN (Comfort Noise) control parameters, e.g., as part of audio-decoding processing by the UE. A buffer of a predetermined size implemented in the UE is configured to store CN parameters for SID (Silence Insertion Descriptor) frames and active hangover frames. Processing circuitry of the UE is configured to determine a CN parameter subset relevant for SID frames based on the age of the stored CN parameters and on residual energies, and use the determined CN parameter subset to determine CN control parameters for a first SID frame following an active signal frame.
Claims
1. A method of generating Comfort Noise (CN) control parameters, the method performed by a User Equipment (UE) configured for operation in a network and comprising: storing CN parameter sets in a buffer of a predetermined size (M) for Silence Insertion Descriptor (SID) frames and active hangover frames of an encoded audio signal, where the CN parameter set stored for each SID frame or active hangover frame includes a residual energy value; determining representative CN parameters for a first SID frame following an active non-hangover frame of the encoded audio signal, based on a relevant subset of the CN parameter sets stored in the buffer, and determining the relevant subset based on an age of the stored CN parameter sets and the residual energy values; and using the representative CN parameters to determine the CN control parameters for the first SID frame.
2. The method of claim 1, wherein storing the CN parameter sets comprises updating the buffer with a new CN parameter set for newly occurring SID frames or active hangover frames; wherein determining the relevant subset of the CN parameter sets stored in the buffer comprises updating, for active non-hangover frames, a size K of an age restricted subset of the CN parameter sets stored in the buffer, based on a number p.sub.A of consecutive active non-hangover frames of the encoded audio signal and selecting the relevant subset from the age restricted subset, based on the residual energy values included in the CN parameter sets contained in the age restricted subset; and wherein using the representative CN parameters to determine the CN control parameters for the first SID frame comprises interpolating the representative CN parameters with decoded CN parameters of the first SID frame.
3. The method of claim 2, wherein updating the size K comprises updating, for the active non-hangover frames, the size K of the age restricted subset in accordance with:
K=K.sub.0 for .Math.p.sub.A<(+1).Math. where K.sub.0 is the number of CN parameter sets stored in the buffer, and the size K is the number of stored CN parameter sets included in the age restricted subset, is a predetermined constant, and is a non-negative integer.
4. The method of claim 2, wherein selecting the relevant subset from the age restricted subset comprises selecting only the CN parameter subsets in the age restricted subset for which:
E.sub.k.sub.
5. The method of claim 2, wherein each stored CN parameter set comprises a vector of Auto Regressive coefficients and the residual energy value for a corresponding one of the SID or active hangover frames represented in the buffer, Q.sup.S represents the set of AR vectors for the CN parameter sets contained in the relevant subset, and E.sup.S represents the set of residual energy values for the CN parameter sets contained in the relevant subset; and wherein determining the representative CN parameters comprises determining the representative CN parameters as {tilde over (q)} and , where {tilde over (q)} is determined as a median vector of the set Q.sup.S, is determined as a weighted mean residual energy of E.sup.S.
6. The method of claim 5, wherein the median vector
7. A non-transitory computer readable medium storing a computer program for generating Comfort Noise (CN) control parameters, said computer program comprising computer readable code units that when executed by a processing circuit of a User Equipment (UE) configured for operation in a network, causes the UE to: store CN parameter sets in a buffer in the UE of a predetermined size (M) for Silence Insertion Descriptor (SID) frames and active hangover frames of an encoded audio signal, wherein the CN parameter set stored for each SID frame or active hangover frame includes a residual energy value; determine representative CN parameters for a first SID frame following an active non-hangover frame of the encoded audio signal, based on a relevant subset of the CN parameter sets stored in the buffer, and determining the relevant subset based on an age of the stored CN parameter sets and the residual energy values; use the representative CN parameters to determine the CN control parameters for the first SID frame.
8. A User Equipment (UE) configured for operation in a network, the UE comprising: a buffer of a predetermined size (M) configured to store Comfort Noise (CN) parameter sets for Silence Insertion Descriptor (SID) frames and active hangover frames of an encoded audio signal, where the CN parameter set stored for each SID frame or active hangover frame includes a residual energy value; and processing circuitry configured to: determine representative CN parameters for a first SID frame following an active non-hangover frame of the encoded audio signal, based on a relevant subset of the CN parameter sets stored in the buffer, and determine the relevant subset based on an age of the stored CN parameter subsets and the residual energy values; and use the representative CN parameters to determine CN control parameters for the first SID frame.
9. The UE of claim 8, wherein the processing circuitry comprises: a SID and hangover frame buffer updater circuit configured to update the buffer with a new CN parameter set for each newly occurring SID frame or active hangover frame; a non-hangover frame buffer updater circuit configured to update, for active non-hangover frames, a size K of an age restricted subset of the CN parameter sets stored in the buffer, based on a number p.sub.A of consecutive active non-hangover frames of the encoded audio signal; a buffer element selector circuit configured to select the relevant subset from the age restricted subset, based on the residual energy values included in the CN parameter sets contained in the age restricted subset; a comfort noise parameter estimator circuit configured to determine the representative CN parameters from the relevant subset; and a comfort noise parameter interpolator circuit configured to determine the CN control parameters for the first SID frame by interpolating the representative CN parameters with decoded CN parameters of the first SID frame.
10. The UE of claim 9, wherein the buffer element selector circuit is configured to update, for the active non-hangover frames, the size K of the age restricted subset in accordance with:
K=K.sub.0 for .Math.p.sub.A<(+1).Math. where K.sub.0 is the number of CN parameter sets stored in the buffer, and the size K is the number of stored CN parameter sets included in the age restricted subset, is a predetermined constant, and is a non-negative integer.
11. The UE of claim 9, wherein the buffer element selector circuit is configured to select the relevant subset from the age restricted subset by selecting only the CN parameter subsets in the age restricted subset for which:
E.sub.k.sub.
12. The UE of claim 9, wherein each stored CN parameter set comprises a vector of Auto Regressive coefficients and the residual energy value for a corresponding one of the SID or active hangover frames represented in the buffer, Q.sup.S represents the set of AR vectors for the CN parameter sets contained in the relevant subset, and E.sup.S represents the set of residual energy values for the CN parameter sets contained in the relevant subset; and wherein the comfort noise parameter estimator circuit is configured to determine the representative CN parameters as {tilde over (q)} and , where {tilde over (q)} is determined as a median vector of the set Q.sup.S, and is determined as a weighted mean residual energy of E.sup.S.
13. The UE of claim 8, wherein the processing circuitry is operative as an audio decoder of the UE.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The proposed technology, together with further objects and advantages thereof, may best be understood by making reference to the following description taken together with the accompanying drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
DETAILED DESCRIPTION
(14) The embodiments described below relate to a system of audio encoder and decoder mainly intended for speech communication applications using DTX with comfort noise for inactive signal representation. The system that is considered utilizes LP for coding of both active and inactive signal frames, where a VAD is used for activity decisions.
(15) In the encoder illustrated in
(16) The disclosed embodiments are part of an audio decoder. Such a decoder 100 is schematically illustrated in
(17) The decoder 100 also includes a buffer 200 of a predetermined size M and configured to receive and store CN parameters for SID and active mode hangover frames, a unit 300 configured to determine which of the stored CN parameters that are relevant for SID based on the age of stored CN parameters, a unit 400 configured to determine which of the determined CN parameters that are relevant for SID based on residual energy measurements, and a unit 500 configured to use the determined CN parameters that are relevant for SID for the first SID frame following active signal frame(s).
(18) The parameters in the buffers are constrained to be recent in order to be relevant. Thereby the sizes of the buffers used for selection of relevant buffer subsets are reduced during longer periods of active coding. Additionally, the stored parameters are replaced by newer values during SID and actively coded hangover frames.
(19) By using circular buffers, the complexity and memory requirement for the buffer handling can be reduced. In such implementations, the already stored elements do not have to be moved when a new element is added. The position of the last added parameter, or parameter set, is used together with the size of the buffer to place new elements. When new elements are added, old elements might be overwritten.
(20) Since the buffers hold parameters from earlier SID and hangover frames they describe signal characteristics of previous audio frames that probably, but not necessarily, contain background noise. The number of parameters that are considered relevant is defined by the size of the buffer and the time, or corresponding number of frames, elapsed since the information was stored.
(21) The technology disclosed herein can be described in a number of algorithmic steps, e.g. performed at the decoder side illustrated in
(22) 1a. Step 1a (Performed by the Unit Denoted Step 1a in
(23)
j=0 if j>M1 (8)
As will be described below, subsets Q.sup.K and E.sup.K of the K.sub.0 latest stored elements in Q.sup.M and E.sup.M, respectively, define the sets of stored parameters.
(24) 1b. Step 1b (Performed by the Unit Denoted Step 1b in
(25) .sup.+ and p.sub.A is the number of consecutive active non-hangover frames. The rate of decrement relates to time, where =25 is feasible for 20 ms frames. This corresponds to a decrease by one element every half second while decoding active frames. The decrement rate constant can potentially be defined as any value
.sup.+, but it should be chosen such that old noise characteristics that are likely not to represent the current background noise are excluded from the subsets Q.sup.K and E.sup.K. The value might for example be chosen based on the expected dynamics of the background noise. In addition, the natural length of speech bursts and the behavior of the VAD may be considered, as long sequences of consecutive active frames are unlikely. Typically, the constant would be in the range 500 for 20 ms frames, which corresponds to less than 10 seconds. As an alternative equation (9) may be written in a more compact form as:
K=K.sub.0 for .Math.p.sub.A<(+1).Math.(10) where K.sub.0 is the number of CN parameters for SID frames and active hangover frames stored in the buffer 200, is a predetermined constant, and is a non-negative integer.
(26) 2. Step 2 (Performed by the Unit Denoted Step 2 in
E.sup.S={E.sub.k.sup.KE.sup.K|E.sub.k.sub.
where E.sub.k.sub.
(27) Typically, .sub.2 is selected from the range .sub.2 [0,100] as larger values would include high residual energies compared to the latest stored residual energy E.sub.k.sub.
(28) It should be noted that the energies E.sub.k.sup.K can as well as in linear domain be represented in a logarithmic domain, e.g. dB. With energies in logarithmic domain the selection of relevant buffer elements, as specified in equation (11), is described equivalently with energies E.sub.k.sup.K in linear domain as:
E.sup.S={E.sub.k.sup.KE.sup.K|E.sub.k.sub.
where log({tilde over ()}.sub.1)=.sub.1 and log({tilde over ()}.sub.2)=.sub.2. Suitable boundaries specifying the subset of the buffer E.sup.K are for example given by {tilde over ()}.sub.1=0.7 and {tilde over ()}.sub.2=1.03 or {tilde over ()}.sub.1[0.5,0.9] and {tilde over ()}.sub.2[1.0,1.25]. The corresponding vectors in the LSP buffer Q.sup.K define the subset Q.sup.S={q.sub.0.sup.S, . . . , q.sub.L1.sup.S}.
(29) 3. Step 3 (Performed by the Unit Denoted Step 3 in
(30)
w.sup.S={w.sub.j.sup.Mw.sup.M} for j|E.sub.j.sup.ME.sup.S For a maximum buffer size M=8 a suitable set of weights is:
w.sup.M={0.2, 0.16, 0.128, 0.1024, 0.08192, 0.065536, 0.0524288, 0.01048576} This means that recent energies get more weight in the residual energy mean , which makes the energy transition between active and inactive frames smoother. Among LSP vectors in the subset Q.sup.S, the median LSP vector is selected by computing the distances between all the LSP vectors in the subset buffer E.sup.S according to:
(31)
(32)
{tilde over (q)}={q.sub.lQ.sup.S|S.sub.lS.sub.m, lm} for l,m=0, . . . , L1 (16) If several vectors have equal total distance, the median can be arbitrarily chosen among those vectors. As an alternative representative LSP vector may be determined as the mean vector of the subset Q.sup.S.
(33) 4. Step 4 (performed by the unit denoted step 4 in
(34)
(35) If the subsets Q.sup.S and E.sup.S are empty, the latest extracted SID parameters may be used directly without interpolation from older noise parameters.
(36) The transmitted LSP vector {tilde over (q)}.sub.SID used in the interpolation is in the encoder usually obtained directly from the LP analysis of the current frame, i.e. no previous frames are considered. The transmitted residual energy .sub.SID is preferably obtained using LP parameters corresponding to the LSP parameters used for the signal synthesis in the decoder. These LSP parameters can be obtained in the encoder by performing steps 1-4 with a corresponding encoder side buffer. Operating the encoder in this way implies that the energy of the decoder output can be matched to the input signal energy by control of the encoded and transmitted residual energy since the decoder synthesis LP parameters are known in the encoder.
(37)
(38) Although it is true that there will be only one first SID frame following an active signal frame, it will indirectly affect the CN parameters in following SID frames due to the smoothing/interpolation.
(39)
(40)
(41)
(42)
(43) The steps, functions, procedures and/or blocks described herein may be implemented in hardware using any conventional technology, such as discrete circuit or integrated circuit technology, including both general-purpose electronic circuitry and application-specific circuitry.
(44) Alternatively, at least some of the steps, functions, procedures and/or blocks described herein may be implemented in software for execution by suitable processing equipment. This equipment may include, for example, one or several microprocessors, one or several Digital Signal Processors (DSP), one or several Application Specific Integrated Circuits (ASIC), video accelerated hardware or one or several suitable programmable logic devices, such as Field Programmable Gate Arrays (FPGA). Combinations of such processing elements are also feasible.
(45) It should also be understood that it may be possible to reuse the general processing capabilities already present in a network node, such as a mobile terminal or pc. This may, for example, be done by reprogramming of the existing software or by adding new software components.
(46)
(47) According to an aspect of the embodiments, a decoder for generating comfort noise representing an inactive signal is provided. The decoder can operate in DTX mode and can be implemented in a mobile terminal and by a computer program product which can be implemented in the mobile terminal or pc. The computer program product can be downloaded from a server to the mobile terminal.
(48)
(49)
(50) In the embodiments of the proposed technology described above the LP coefficients .sub.k are transformed to an LSP domain. However, the same principles may also be applied to LP coefficients that are transformed to an LSF, ISP or ISF domain.
(51) For codecs with attenuation of the comfort noise it can be beneficial to gradually attenuate the actively coded signal during VAD hangover frames. The energy for the comfort noise would then better match the latest actively coded frame, which further improves the perceived audio quality. An attenuation factor can be computed and applied to the LP residual for each hangover frame by:
s[n]=.Math.s[n](18)
with
(52)
where p.sub.HO is the number of consecutive VAD hangover frames. As an alternative may be computed as:
(53)
where L=0.6 and L.sub.0=6 control the maximum attenuation and rate of attenuation. The maximum attenuation can typically be selected in the range L=[0.5,1) and the rate control parameter L.sub.0 for example be selected such that
(54)
where p.sub.HO.sup.FULL is the number of frames needed for maximum attenuation. p.sub.HO.sup.FULL could for example be set to the average or maximum number of consecutive VAD hangover frames that is possible (due to the hangover addition in the VAD). Typically, this would be in the range of p.sub.HO.sup.FULL={1, . . . , 15} frames.
(55) It should be understood that the technology described herein can co-operate with other solutions handling the first CN frames following active signal segments. For example, it can complement an algorithm where a large change in CN parameters is allowed for high energy frames (relative to background noise level). For these frames, the previous noise characteristics might not much affect the update in the current SID frame. The described technology may then be used for frames that are not detected as high energy frames.
(56) It will be understood by those skilled in the art that various modifications and changes may be made to the proposed technology without departure from the scope thereof, which is defined by the appended claims.
ABBREVIATIONS
(57) ACELP Algebraic Code-Excited Linear Prediction
(58) AMR Adaptive Multi-Rate
(59) AMR NB AMR Narrowband
(60) AR Auto Regressive
(61) ASIC Application Specific Integrated Circuits
(62) CN Comfort Noise
(63) DFT Discrete Fourier Transform
(64) DSP Digital Signal Processors
(65) DTX Discontinuous Transmission
(66) EEPROM Electrically Erasable Programmable Read-only Memory
(67) FPGA Field Programmable Gate Arrays
(68) ISF Immitance Spectrum Frequencies
(69) ISP Immitance Spectrum Pairs
(70) LP Linear Prediction-.
(71) LSF Line Spectral Frequencies
(72) LSP Line Spectral Pairs
(73) MDCT Modified Discrete Cosine Transform
(74) RAM Random-access memory
(75) SAD Sound Activity Detector
(76) SID Silence Insertion Descriptor
(77) UE User Equipment
(78) VAD Voice Activity Detector