Time-Alignment of QMF Based Processing Data

Abstract

The present document relates to time-alignment of encoded data of an audio encoder with associated metadata, such as spectral band replication (SBR) metadata. An audio decoder configured to determine a reconstructed frame of an audio signal from an access unit of a received data stream is described. The access unit comprises waveform data and metadata, wherein the waveform data and the metadata are associated with the same reconstructed frame of the audio signal. The audio decoder comprises a waveform processing path configured to generate a plurality of waveform subband signals from the waveform data, and a metadata processing path configured to generate decoded metadata from the metadata.

Claims

1. (canceled)

2. An audio decoder apparatus for decoding an audio signal, the apparatus comprising: a processor for processing a waveform processing path, wherein the processor is configured to generate at least a waveform signal from waveform data obtained from an access unit of the audio signal; a metadata processor for processing a metadata processing path configured to generate decoded metadata from metadata obtained from the access unit, wherein the metadata processing path comprises a metadata delay unit configured to delay the decoded metadata by a delay, wherein the delay has a value greater than zero, wherein the value of the delay is a first integer, and wherein, the first integer multiplied by a second integer is equal to a frame length; and a metadata application and synthesis unit configured to generate a reconstructed frame of the audio signal from the at least a waveform signal and from the decoded metadata, wherein at least one of the waveform processing path or the metadata processing path comprises at least one delay unit configured to time-align the at least a waveform signal and the decoded metadata.

3. The apparatus of claim 2, wherein the value of the delay is 256.

4. The apparatus of claim 2, wherein the frame length is greater than 960.

5. The apparatus of claim 2, wherein the at least a waveform signal and the decoded metadata are time-aligned, such that an overall delay of the waveform processing path corresponds to an overall delay of metadata processing path.

6. A method for decoding an audio signal, comprising: generating, using a waveform processing path, at least a waveform signal from waveform data from the waveform data obtained from an access unit of the audio signal; generating, using a metadata processing path, decoded data from metadata obtained from the access unit, wherein the metadata processing path comprises a metadata delay unit configured to delay the decoded metadata by a delay, wherein the delay has a value greater than zero, wherein the value of the delay is a first integer, and wherein, the first integer multiplied by a second integer is equal to a frame length; and generating using a metadata application and synthesis unit, a reconstructed frame of the audio signal from the at least a waveform signal and from the decoded metadata, a reconstructed frame of the audio signal from the at least a waveform signal and from the decoded metadata, wherein at least one of the waveform processing path or the metadata processing path comprises at least one delay unit configured to time-align the at least a waveform signal and the decoded metadata

7. The method of claim 6, wherein the value of the delay is 256.

8. The method of claim 6, wherein the frame length is greater than 960.

9. The method of claim 6, wherein the at least a waveform signal and the decoded metadata are time-aligned, such that an overall delay of the waveform processing path corresponds to an overall delay of metadata processing path.

10. A non-transitory storage medium adapted for execution on a processor and for performing the method of claim 6 when carried out on the processor.

Description

SHORT DESCRIPTION OF THE FIGURES

[0043] The invention is explained below in an illustrative manner with reference to the accompanying drawings, wherein

[0044] FIG. 1 shows a block diagram of an example audio decoder;

[0045] FIG. 2a shows a block diagram of another example audio decoder;

[0046] FIG. 2b shows a block diagram of an example audio encoder; and

[0047] FIG. 3a shows a block diagram of an example audio decoder which is configured to perform audio expansion;

[0048] FIG. 3b shows a block diagram of an example audio encoder which is configured to perform audio compression; and

[0049] FIG. 4 illustrates an example framing of a sequence of frames of an audio signal.

DETAILED DESCRIPTION OF THE INVENTION

[0050] As indicated above, the present document relates to metadata alignment. In the following the alignment of metadata is outlined in the context of an MPEG HE (High Efficiency) AAC (Advanced Audio Coding) scheme. It should be noted, however, that the principles of metadata alignment which are described in the present document are also applicable to other audio encoding/decoding systems. In particular, the metadata alignment schemes which are described in the present document are applicable to audio encoding/decoding systems which make use of HFR (High Frequency Reconstruction) and/or SBR (Spectral Bandwidth Replication) and which transmit HFR/SBR metadata from an audio encoder to a corresponding audio decoder. Furthermore, the metadata alignment schemes which are described in the present document are applicable to audio encoding/decoding systems which make use of applications in a subband (notable a QMF) domain. An example for such an application is SBR. Other examples are A-coupling, post-processing, etc. In the following, the metadata alignment schemes are described in the context of the alignment of SBR metadata. It should be noted, however, that the metadata alignment schemes are also applicable to other types of metadata, notably to other types of metadata in the subband domain.

[0051] An MPEG HE-AAC data stream comprises SBR metadata (also referred to as A-SPX metadata). The SBR metadata in a particular encoded frame of the data stream (also referred to as an AU (access unit) of the data stream) typically relates to waveform (W) data in the past. In other words, The SBR metadata and the waveform data comprised within an AU of the data stream typically do not correspond to the same frame of the original audio signal. This is due to the fact that after decoding of the waveform data, the waveform data is submitted to several processing steps (such as an IMDCT (inverse Modified Discrete Cosine Transform and a QMF (Quadrature Mirror Filter) Analysis) which introduce a signal delay. At the point where the SBR metadata is applied to the waveform data, the SBR metadata is in synchronicity with the processed waveform data. As such, the SBR metadata and the waveform data are inserted into the MPEG HE-AAC data stream such that the SBR metadata reaches the audio decoder, when the SBR metadata is needed for SBR processing at the audio decoder. This form of metadata delivery may be referred to as “Just-In-Time” (JIT) metadata delivery, as the SBR metadata is inserted into the data stream such that the SBR metadata can be directly applied within the signal or processing chain of the audio decoder.

[0052] JIT metadata delivery may be beneficial for a conventional encode—transmit—decode processing chain, in order to reduce the overall coding delay and in order to reduce memory requirements at the audio decoder. However, a splice of the data stream along the transmission path may lead to a mismatch between the waveform data and the corresponding SBR metadata. Such a mismatch may lead to audible artifacts at the splicing point because wrong SBR metadata is used for spectral band replication at the audio decoder.

[0053] In view of the above, it is desirable to provide an audio encoding/decoding system which allows for the splicing of data streams, while at the same time maintaining a low overall coding delay.

[0054] FIG. 1 shows a block diagram of an example audio decoder 100 which addresses the above mentioned technical problem. In particular, the audio decoder 100 of FIG. 1 allows for the decoding of data streams with AUs 110 which comprise the waveform data 111 of a particular segment (e.g. frame) of an audio signal and which comprise the corresponding metadata 112 of the particular segment of the audio signal. By providing audio decoders 100 that decode data streams comprising AUs 110 with time-aligned waveform data 111 and corresponding metadata 112, consistent splicing of the data stream is enabled. In particular, it is ensured that the data stream can be spliced in such a manner that corresponding pairs of waveform data 111 and metadata 112 are maintained.

[0055] The audio decoder 100 comprises a delay unit 105 within the processing chain of the waveform data 111. The delay unit 105 may be placed post or downstream of the MDCT synthesis unit 102 and prior or upstream of the QMF synthesis unit 107 within the audio decoder 100. In particular, the delay unit 105 may be placed prior or upstream of the metadata application unit 106 (e.g. the SBR unit 106) which is configured to apply the decoded metadata 128 to the processed waveform data. The delay unit 105 (also referred to as the waveform delay unit 105) is configured to apply a delay (referred to as the waveform delay) to the processed waveform data. The waveform delay is preferably chosen so that the overall processing delay of the waveform processing chain or the waveform processing path (e.g. from the MDCT synthesis unit 102 to the application of metadata in the metadata application unit 106) sums up to exactly one frame (or to an integer multiple thereof). By doing so, the parametric control data can be delayed by a frame (or a multiple thereof) and alignment within the AU 110 is achieved.

[0056] FIG. 1 shows components of an example audio decoder 100. The waveform data 111 taken from an AU 110 is decoded and de-quantized within a waveform decoding and de-quantization unit 101 to provide a plurality of frequency coefficients 121 (in the frequency domain). The plurality of frequency coefficients 121 are synthesized into a (time domain) lowband signal 122 using a frequency domain to time domain transform (e.g. an inverse MDCT, Modified Discrete Cosine Transform) applied within the lowband synthesis unit 102 (e.g. the MDCT synthesis unit). Subsequently, the lowband signal 122 is transformed into a plurality of lowband subband signals 123 using an analysis unit 103. The analysis unit 103 may be configured to apply a quadrature mirror filter (QMF) bank to the lowband signal 122 to provide the plurality of lowband subband signals 123. The metadata 112 is typically applied to the plurality of lowband subband signals 123 (or to transposed versions thereof).

[0057] The metadata 112 from the AU 110 is decoded and de-quantized within a metadata decoding and de-quantization unit 108 to provide the decoded metadata 128. Furthermore, the audio decoder 100 may comprise a further delay unit 109 (referred to as the metadata delay unit 109) which is configured to apply a delay (referred to as the metadata delay) to the decoded metadata 128. The metadata delay may correspond to an integer multiple of the frame length N, e.g. D.sub.1=N, wherein D.sub.1 is the metadata delay. As such, the overall delay of the metadata processing chain corresponds to D.sub.1, e.g. D.sub.1=N.

[0058] In order to ensure that the processed waveform data (i.e. the delayed plurality of lowband subband signals 123) and the processed metadata (i.e. the delayed decoded metadata 128) arrive at the metadata application unit 106 at the same time, the overall delay of the waveform processing chain (or path) should correspond to the overall delay of the metadata processing chain (or path) (i.e. to D.sub.1). Within the waveform processing chain, the lowband synthesis unit 102 typically inserts a delay of N/2 (i.e. of half the frame length). The analysis unit 103 typically inserts a fixed delay (e.g. of 320 samples). Furthermore, a lookahead (i.e. a fixed offset between metadata and waveform data) may need to be taken into account. In the case of MPEG HE-AAC such an SBR lookahead may correspond to 384 samples (represented by the lookahead unit 104). The lookahead unit 104 (which may also be referred to as the lookahead delay unit 104) may be configured to delay the waveform data 111 (e.g. delay the plurality of lowband subband signals 123) by a fixed SBR lookahead delay. The lookahead delay enables a corresponding audio encoder to determine the SBR metadata based on a succeeding frame of the audio signal.

[0059] In order to provide an overall delay of the metadata processing chain which corresponds to an overall delay of the waveform processing chain, the waveform delay D.sub.2 should be such that:

D.sub.1=320+384+D.sub.2+N/2,

i.e. D.sub.2=N/2−320−384 (in case of D.sub.1=N).

[0060] Table 1 shows the waveform delays D.sub.2 for a plurality of different frame lengths N. It can be seen that the maximum waveform delay D.sub.2 for the different frame lengths N of HE-AAC is 928 samples with an overall maximum decoder latency of 2177 samples. In other words, the alignment of the waveform data 111 and the corresponding metadata 112 within a single AU 110 results in a maximum of 928 samples additional PCM delay. For the block of frame sizes N=1920/1536, the metadata is delayed by 1 frame, and for frame sizes N=960/768/512/384 the metadata is delayed by 2 frames. This means that the play out delay at the audio decoder 100 is increased in dependence of the block size N, and the overall coding delay is increased by 1 or 2 full frames. The maximum PCM delay at the corresponding audio encoder is 1664 samples (corresponding to the inherent latency of the audio decoder 100).

TABLE-US-00001 TABLE 1 Inverse Inherent Overall MDCT QMF SBR latency Nb. of QMF Decoder N (N/2) analysis Lookahead (Σ) D.sub.2 frames D.sub.1 synthesis latency 1920 960 320 384 1664 256 1 1920 257 2177 1536 768 320 384 1472  64 1 1536 257 1793  960 480 320 192  992 928 2 1920 257 2177  768 384 320 192  896 640 2 1536 257 1793  512 256 320 192  768 256 2 1024 257 1281  384 192 320 192  704  64 2  768 257 1025

[0061] As such, it is proposed in the present document to address the drawback of JIT metadata, by the use of signal-aligned-metadata 112 (SAM) which is aligned with the corresponding waveform data 111 into a single AU 110. In particular, it is proposed to introduce one or more additional delay units into an audio decoder 100 and/or into a corresponding audio encoder such that every encoded frame (or AU) carries the (e.g. A-SPX) metadata it uses at a later processing stage, e.g. at the processing stage when the metadata is applied to the underlying waveform data.

[0062] It should be noted that—in principle—it could be considered to apply a metadata delay D.sub.1 which corresponds to a fraction of the frame length N. By doing this, the overall coding delay could possibly be reduced. However, as shown e.g. in FIG. 1, the metadata delay D.sub.1 is applied in the QMF domain (i.e. in the subband domain). In view of this and in view of the fact that the metadata 112 is typically only defined once per frame, i.e. in view of the fact that the metadata 112 typically comprises one dedicated parameter set per frame, the insertion of a metadata delay D.sub.1 which corresponds to a fraction of a frame length N may lead to synchronization issues with respect to the waveform data 111. On the other hand, the waveform delay D.sub.2 is applied in the time-domain (as shown in FIG. 1), where delays which correspond to a fraction of a frame can be implemented in a precise manner (e.g. by delaying the time domain signal by a number of samples which corresponds to the waveform delay D.sub.2). Hence, it is beneficial to delay the metadata 112 by integer multiples of a frame (wherein the frame corresponds to the lowest time resolution for which the metadata 112 is defined) and to delay the waveform data 111 by a waveform delay D.sub.2 which may take on arbitrary values. A metadata delay D.sub.1 which corresponds to an integer multiple of the frame length N can be implemented in the subband domain in a precise manner, and a waveform delay D.sub.2 which corresponds to an arbitrary multiple of a sample can be implemented in the time domain in a precise manner. Consequently, the combination of a metadata delay D.sub.1 and a waveform delay D.sub.2 allows for an exact synchronization of the metadata 112 and the waveform data 111.

[0063] The application of a metadata delay D.sub.1 which corresponds to a fraction of the frame length N could be implemented by re-sampling the metadata 112, in accordance to the metadata delay D.sub.1. However, the re-sampling of the metadata 112 typically involves substantial computational costs. Furthermore, the re-sampling of the metadata 112 may lead to a distortion of the metadata 112, thereby affecting the quality of the reconstructed frame of the audio signal. In view of this, it is beneficial, in view of computational efficiency and in view of audio quality, to limit the metadata delay D.sub.1 to integer multiples of the frame length N.

[0064] FIG. 1 also shows the further processing of the delayed metadata 128 and the delayed plurality of lowband subband signals 123. The metadata application unit 106 is configured to generate a plurality of (e.g. scaled) highband subband signals 126 based on the plurality of lowband subband signals 123 and based on the metadata 128. For this purpose, the metadata application unit 106 may be configured to transpose one or more of the plurality of lowband subband signals 123 to generate a plurality of highband subband signals. The transposition may comprise a copy-up process of the one or more of the plurality of lowband subband signals 123. Furthermore, the metadata application unit 106 may be configured to apply the metadata 128 (e.g. scale factors comprised within the metadata 128) to the plurality of highband subband signals, in order to generate the plurality of scaled highband subband signals 126. The plurality of scaled highband subband signals 126 is typically scaled using the scale factors, such that the spectral envelope of the plurality of scaled highband subband signals 126 mimics the spectral envelope of the highband signal of an original frame of the audio signal (which corresponds to a reconstructed frame of the audio signal 127 that is generated based on the plurality of lowband subband signals 123 and from the plurality of scaled highband subband signals 126).

[0065] Furthermore, the audio decoder 100 comprises a synthesis unit 107 configured to generate the reconstructed frame of an audio signal 127 from the plurality of lowband subband signals 123 and from the plurality of scaled highband subband signals 126 (e.g. using an inverse QMF bank).

[0066] FIG. 2a shows a block diagram of another example audio decoder 100. The audio decoder 100 of FIG. 2a comprises the same components as the audio decoder 100 of FIG. 1. Furthermore, example components 210 for multi-channel audio processing are illustrated. It can be seen that in the example of FIG. 2a, the waveform delay unit 105 is positioned directly subsequent to the inverse MDCT unit 102. The determination of a reconstructed frame of an audio signal 127 may be performed for each channel of a multi-channel audio signal (e.g. of a 5.1 or a 7.1 multi-channel audio signal).

[0067] FIG. 2b shows a block diagram of an example audio encoder 250 corresponding to the audio decoder 100 of FIG. 2a. The audio encoder 250 is configured to generate a data stream comprising AUs 110 which carries pairs of corresponding waveform data 111 and metadata 112. The audio encoder 250 comprises a metadata processing chain 256, 257, 258, 259, 260 for determining the metadata. The metadata processing chain may comprise a metadata delay unit 256 for aligning the metadata with the corresponding waveform data. In the illustrated example, the metadata delay unit 256 of the audio encoder 250 does not introduce any additional delay (because the delay introduced by the metadata processing chain is greater than the delay introduced by the waveform processing chain).

[0068] Furthermore, the audio encoder 250 comprises a waveform processing chain 251, 252, 253, 254, 255 configured to determine the waveform data from an original audio signal at the input of the audio encoder 250. The waveform processing chain comprises a waveform delay unit 252 configured to introduce an additional delay into the waveform processing chain, in order to align the waveform data with the corresponding metadata. The delay which is introduced by the waveform delay unit 252 may be such that the overall delay of the metadata processing chain (including the waveform delay inserted by the waveform delay unit 252) corresponds to the overall delay of the waveform processing chain. In case of a frame length N=2048, the delay of the waveform delay unit 252 may be 2048-320=1728 samples.

[0069] FIG. 3a shows an excerpt of an audio decoder 300 which comprises an expanding unit 301. The audio decoder 300 of FIG. 3a may correspond to the audio decoder 100 of FIGS. 1 and/or 2a and further comprises the expanding unit 301 which is configured to determine a plurality of expanded lowband signals from the plurality of lowband signals 123, using one or more expanding parameters 310 taken from the decoded metadata 128 of an access unit 110. Typically, the one or more expanding parameters 310 are coupled with SBR (e.g. A-SPX) metadata comprised within an access unit 110. In other words, the one or more expanding parameters 310 are typically applicable to the same excerpt or portion of an audio signal as the SBR metadata.

[0070] As outlined above, the metadata 112 of an access unit 110 is typically associated with the waveform data 111 of a frame of an audio signal, wherein the frame comprises a pre-determined number N of samples. The SBR metadata is typically determined based on a plurality of lowband signals (also referred to as a plurality of waveform subband signals), wherein the plurality of lowband signals may be determined using a QMF analysis. The QMF analysis yields a time-frequency representation of a frame of an audio signal. In particular, the N samples of a frame of an audio signal may be represented by Q (e.g. Q=64) lowband signals, each comprising N/Q time slots or slots. For a frame with N=2048 samples and for Q=64, each lowband signal comprises N/Q=32 slots.

[0071] In case of a transient within a particular frame, it may be beneficial to determine the SBR metadata based on samples of a directly succeeding frame. This feature is referred to as the SBR lookahead. In particular, the SBR metadata may be determined based on a pre-determined number of slots from the succeeding frame. By way of example, up to 6 slots of the succeeding frame may be taken into consideration (i.e. Q*6=384 samples).

[0072] The use of the SBR lookahead is illustrated in FIG. 4 which shows a sequence of frames 401, 402, 403 of an audio signal, using different framings 400, 430 for the SBR or HFR scheme. In the case of framing 400, the SBR/HFR scheme does not make use of the flexibility provided by the SBR lookahead. Nevertheless, a fixed offset, i.e. a fixed SBR lookahead delay, 480 is used to enable the use of the SBR lookahead. In the illustrated example, the fixed offset corresponds to 6 time slots. As a result of this fixed offset 480, the metadata 112 of a particular access unit 110 of a particular frame 402 is partially applicable to time slots of waveform data 111 comprised within the access unit 110 which precedes the particular access unit 110 (and which is associated with the directly preceding frame 401). This is illustrated by the offset between the SBR metadata 411, 412, 413 and the frames 401, 402, 403. Hence, the SBR metadata 411, 412, 413 comprised within an access unit 110 may be applicable to waveform data 111 which is offset by the SBR lookahead delay 480. The SBR metadata 411, 412, 413 is applied to the waveform data 111 to provide the reconstructed frames 421, 422, 423.

[0073] The framing 430 makes use of the SBR lookahead. It can be seen that the SBR metadata 431 is applicable to more than 32 time slots of waveform data 111, e.g. due to the occurrence of a transient within frame 401. On the other hand, the succeeding SBR metadata 432 is applicable to less than 32 time slots of waveform data 111. The SBR metadata 433 is again applicable to 32 time slots. Hence, the SBR lookahead allows for flexibility with regards to the temporal resolution of the SBR metadata. It should be noted that, regardless the use of the SBR lookahead and regardless the applicability of the SBR metadata 431, 432, 433, the reconstructed frames 421, 422, 423 are generated using a fixed offset 480 with respect to the frames 401, 402, 403.

[0074] An audio encoder may be configured to determine the SBR metadata and the one or more expanding parameters using the same excerpt or portion of the audio signal. Hence, if the SBR metadata is determined using an SBR lookahead, the one or more expanding parameters may be determined and may be applicable for the same SBR lookahead. In particular, the one or more expanding parameters may be applicable for the same number of time slots as the corresponding SBR metadata 431, 432, 433.

[0075] The expanding unit 301 may be configured to apply one or more expanding gains to the plurality of lowband signals 123, wherein the one or more expanding gains typically depend on the one or more expanding parameters 310. In particular, the one or more expanding parameters 310 may have an impact on one or more compression/expanding rules which are used to determine the one or more expanding gains. In other words, the one or more expanding parameters 310 may be indicative of the compression function which has been used by a compression unit of the corresponding audio encoder. The one or more expanding parameters 310 may enable the audio decoder to determine the inverse of this compression function.

[0076] The one or more expanding parameters 310 may comprise a first expanding parameter indicative of whether or not the corresponding audio encoder has compressed the plurality of lowband signals. If no compression has been applied, then no expansion will be applied by the audio decoder. As such, the first expanding parameter may be used to turn on or off the companding feature.

[0077] Alternatively or in addition, the one or more expanding parameters 310 may comprise a second expanding parameter indicative of whether or not the same one or more expansion gains are to be applied to all of the channels of a multi-channel audio signal. As such, the second expanding parameter may switch between a per-channel or a per-multi-channel application of the companding feature.

[0078] Alternatively or in addition, the one or more expanding parameters 310 may comprise a third expanding parameter indicative of whether or not to apply the same one or more expanding gains for all the time slots of a frame. As such, the third expanding parameter may be used to control the temporal resolution of the companding feature.

[0079] Using the one or more expanding parameters 310, the expanding unit 301 may determine the plurality of expanded lowband signals, by applying the inverse of a compression function applied at the corresponding audio encoder. The compression function which has been applied at the corresponding audio encoder is signaled to the audio decoder 300 using the one or more expanding parameters 310.

[0080] The expanding unit 301 may be positioned downstream of the lookahead delay unit 104. This ensures that the one or more expanding parameters 310 are applied to the correct portion of the plurality of lowband signals 123. In particular, this ensures that the one or more expanding parameters 310 are applied to the same portion of the plurality of lowband signals 123 as the SBR parameters (within the SBR application unit 106). As such, it is ensured that the expanding operates on the same time framing 400, 430 as the SBR scheme. Due to the SBR lookahead, the framing 400, 430 may comprise a variable number of time slots, and by consequence, the expanding may operate on a variable number of time slots (as outlined in the context of FIG. 4). By placing the expanding unit 301 downstream of the lookahead delay unit 104, it is ensured that the correct framing 400, 430 is applied to the one or more expanding parameters. As a result of this, a high quality audio signal can be ensured, even subsequent to a splicing point.

[0081] FIG. 3b shows an excerpt of an audio encoder 350 comprising a compression unit 351. The audio encoder 350 may comprise the components of the audio encoder 250 of FIG. 2b. The compression unit 351 may be configured to compress (e.g. reduce the dynamic range) of the plurality of lowband signals, using a compression function. Furthermore, the compression unit 351 may be configured to determine one or more expanding parameters 310 which are indicative of the compression function that has been used by the compression unit 351, to enable a corresponding expanding unit 301 of an audio decoder 300 to apply an inverse of the compression function.

[0082] The compression of the plurality of lowband signals may be performed downstream of an SBR lookahead 258. Furthermore, the audio encoder 350 may comprise an SBR framing unit 353 which is configured to ensure that the SBR metadata is determined for the same portion of the audio signal as the one or more expanding parameters 310. In other words, the SBR framing unit 353 may ensure that the SBR scheme operates on the same framing 400, 430 as the companding scheme. In view of the fact that the SBR scheme may operate on extended frames (e.g. in case of transients), the companding scheme may also operate on extended frames (comprising additional time slots).

[0083] In the present document, an audio encoder and a corresponding audio decoder have been described which allow for the encoding of an audio signal into a sequence of time-aligned AUs comprising waveform data and metadata associated with a sequence of segments of the audio signal, respectively. The use of time-aligned AUs enables the splicing of data streams with reduced artifacts at the splicing points. Furthermore, the audio encoder and audio decoder are designed such that the splicable data streams are processed in a computationally efficient manner and such that the overall coding delay remains low.

[0084] The methods and systems described in the present document may be implemented as software, firmware and/or hardware. Certain components may e.g. be implemented as software running on a digital signal processor or microprocessor. Other components may e.g. be implemented as hardware and or as application specific integrated circuits. The signals encountered in the described methods and systems may be stored on media such as random access memory or optical storage media. They may be transferred via networks, such as radio networks, satellite networks, wireless networks or wireline networks, e.g. the Internet. Typical devices making use of the methods and systems described in the present document are portable electronic devices or other consumer equipment which are used to store and/or render audio signals.

[0085] Various aspects of the present invention may be appreciated from the following enumerated example embodiments (EEEs):

EEE1. An audio decoder configured to determine a reconstructed frame of an audio signal from an access unit of a received data stream; wherein the access unit comprises waveform data and metadata; wherein the waveform data and the metadata are associated with the same reconstructed frame of the audio signal; wherein the audio decoder comprises [0086] a waveform processing path configured to generate a plurality of waveform subband signals from the waveform data; [0087] a metadata processing path configured to generate decoded metadata from the metadata; and [0088] a metadata application and synthesis unit configured to generate the reconstructed frame of the audio signal from the plurality of waveform subband signals and from the decoded metadata; wherein the waveform processing path and/or the metadata processing path comprise at least one delay unit configured to time-align the plurality of waveform subband signals and the decoded metadata.
EEE2. The audio decoder of EEE 1, wherein the at least one delay unit is configured to time-align the plurality of waveform subband signals and the decoded metadata such that an overall delay of the waveform processing path corresponds to an overall delay of metadata processing path.
EEE3. The audio decoder of any previous EEE, wherein the at least one delay unit is configured to time-align the plurality of waveform subband signals and the decoded metadata such that the plurality of waveform subband signals and the decoded metadata are provided to the metadata application and synthesis unit just-in-time for the processing performed by the metadata application and synthesis unit.
EEE4. The audio of any previous EEE, wherein the metadata processing path comprises a metadata delay unit configured to delay the decoded metadata by an integer multiple greater zero of a frame length N of the reconstructed frame of the audio signal.
EEE5. The audio decoder of EEE 4, wherein the integer multiple is such that the delay introduced by the metadata delay unit is greater than a delay introduced by the processing of the waveform processing path.
EEE6. The audio decoder of any of EEEs 4 to 5, wherein the integer multiple is one for frame lengths N greater than 960 and wherein the integer multiple is two for frame lengths N smaller than or equal to 960.
EEE7. The audio decoder of any previous EEE, wherein the waveform processing path comprises a waveform delay unit configured to delay the plurality of waveform subband signals such that an overall delay of the waveform processing path corresponds to an integer multiple greater than zero of a frame length N of the reconstructed frame of the audio signal.
EEE8. The audio decoder of any previous EEE, wherein the waveform processing path comprises [0089] a decoding and de-quantization unit configured to decode and de-quantize the waveform data to provide a plurality of frequency coefficients indicative of the waveform signal; [0090] a waveform synthesis unit configured to generate the waveform signal from the plurality of frequency coefficients; and [0091] an analysis unit configured to generate the plurality of waveform subband signals from the waveform signal.
EEE9. The audio decoder of EEE 8, wherein [0092] the waveform synthesis unit is configured to perform a frequency domain to time domain transform; [0093] the analysis unit is configured to perform a time domain to subband domain transform; and [0094] a frequency resolution of the transform performed by the waveform synthesis unit is higher than a frequency resolution of the transform performed by the analysis unit.
EEE10. The audio decoder of EEE 9, wherein [0095] the waveform synthesis unit is configured to perform an inverse modified discrete cosine transform; and [0096] the analysis unit is configured to apply a quadrature mirror filter bank.
EEE11. The audio decoder of any of EEEs 8 to 10, wherein [0097] the waveform synthesis unit introduces a delay which depends on a frame length N of the reconstructed frame of the audio signal; and/or [0098] the analysis unit introduces a fixed delay which is independent of the frame length N of the reconstructed frame of the audio signal.
EEE12. The audio decoder of EEE 11, wherein [0099] the delay introduced by the waveform synthesis unit corresponds to half the frame length N; and/or [0100] the fixed delay introduced by the analysis unit corresponds to 320 samples of the audio signal.
EEE13. The audio decoder of any of EEEs 8 to 12, wherein the overall delay of the waveform processing path depends on a pre-determined lookahead between metadata and waveform data.
EEE14. The audio decoder of EEE 13, wherein the pre-determined lookahead corresponds to 192 or 384 samples of the audio sample.
EEE15. The audio decoder of any previous EEE, wherein [0101] the decoded metadata comprises one or more expanding parameters; [0102] the audio decoder comprises an expanding unit configured to generate a plurality of expanded waveform subband signals based on the plurality of waveform subband signals, using the one or more expanding parameters; and [0103] the reconstructed frame of the audio signal is determined from the plurality of expanded waveform subband signals.
EEE16. The audio decoder of EEE 15, wherein [0104] the audio decoder comprises a lookahead delay unit configured to delay the plurality of waveform subband signals in accordance to a pre-determined lookahead, to yield a plurality of delayed waveform subband signals; and [0105] the expanding unit is configured to generate the plurality of expanded waveform subband signals by expanding the plurality of delayed waveform subband signals.
EEE17. The audio decoder of any of EEEs 15 to 16, wherein [0106] the expanding unit is configured to generate the plurality of expanded waveform subband signals using an inverse of a pre-determined compression function; and [0107] the one or more expanding parameters are indicative of the inverse of the pre-determined compression function.
EEE18. The audio decoder of any of EEEs 15 to 17, wherein [0108] the metadata application and synthesis unit is configured to generate the reconstructed frame of the audio signal by using the decoded metadata for a temporal portion of the plurality of waveform subband signals; and [0109] the expanding unit is configured to generate the plurality of expanded waveform subband signals by using the one or more expanding parameters for the same temporal portion of the plurality of waveform subband signals.
EEE19. The audio decoder of EEE 18, wherein a temporal length of the temporal portion of the plurality of waveform subband signals is variable.
EEE20. The audio decoder (of any of EEEs 8 to 19, wherein the waveform delay unit (105) is configured to delay the waveform signal; wherein the waveform signal is represented in the time domain.
EEE21. The audio decoder of any previous EEE, wherein the metadata application and synthesis unit is configured to process the decoded metadata and the plurality of waveform subband signals (in the subband domain.
EEE22. The audio decoder of any previous EEE, wherein [0110] the reconstructed frame of the audio signal comprises a lowband signal and a highband signal; [0111] the plurality of waveform subband signals are indicative of the lowband signal; [0112] the metadata is indicative of a spectral envelope of the highband signal; and [0113] the metadata application and synthesis unit comprises a metadata application unit which is configured to perform high frequency reconstruction using the plurality of waveform subband signals and the decoded metadata.
EEE23. The audio decoder of EEE 22, wherein the metadata application unit is configured to [0114] transpose one or more of the plurality of waveform subband signals to generate a plurality of highband subband signals; and [0115] apply the decoded metadata to the plurality of highband subband signals to provide a plurality of scaled highband subband signals; wherein the plurality of scaled highband subband signals is indicative of the highband signal of the reconstructed frame of the audio signal.
EEE24. The audio decoder of EEE 23, wherein the metadata application and synthesis unit, further comprises a synthesis unit configured to generate the reconstructed frame of the audio signal from the plurality of waveform subband signals and from the plurality of scaled highband subband signals.
EEE25. The audio decoder of EEE 24 referring back to EEE 9, wherein the synthesis unit is configured to perform an inverse transform with respect to the transform performed by the analysis unit.
EEE26. An audio encoder configured to encode a frame of an audio signal into an access unit of a data stream; wherein the access unit comprises waveform data and metadata; wherein the waveform data and the metadata are indicative of a reconstructed frame of the frame of the audio signal; wherein the audio encoder comprises [0116] a waveform processing path configured to generate the waveform data from the frame of the audio signal; and [0117] a metadata processing path configured to generate the metadata from the frame of the audio signal; wherein the waveform processing path and/or the metadata processing path comprise at least one delay unit configured to time-align the waveform data and the metadata such that the access unit for the frame of the audio signal comprises the waveform data and the metadata for the same frame of the audio signal.
EEE27. The audio encoder of EEE 26, wherein the at least one delay unit is configured to time-align the waveform data and the metadata such that an overall delay of the waveform processing path corresponds to an overall delay of the metadata processing path.
EEE28. The audio encoder of any of EEEs 26 to 27, wherein the at least one delay unit is configured to time-align the waveform data and the metadata such that the waveform data and the metadata are provided to an access unit generation unit of the audio encoder just-in-time for generating a single access unit from the waveform data and from the metadata.
EEE29. The audio encoder of any of EEEs 26 to 28, wherein the waveform processing path comprises a waveform delay unit configured to insert a delay into the waveform processing path.
EEE30. The audio encoder of any of EEEs 26 to 29, wherein [0118] the frame of the audio signal comprises a lowband signal and a highband signal; [0119] the waveform data is indicative of the lowband signal; [0120] the metadata is indicative of a spectral envelope of the highband signal; [0121] the waveform processing path is configured to generate the waveform data from the lowband signal; and [0122] the metadata processing path is configured to generate the metadata from the lowband signal and from the highband signal.
EEE31. The audio encoder of EEE 30, wherein [0123] the audio encoder comprises an analysis unit configured to generate a plurality of subband signals from the frame of the audio signal; [0124] the plurality of subband signals comprises a plurality of lowband signals indicative of the lowband signal; [0125] the audio encoder comprises a compression unit configured to compress the plurality of lowband signals using a compression function, to provide a plurality of compressed lowband signals; [0126] the waveform data is indicative of the plurality of compressed lowband signals; and [0127] the metadata is indicative of the compression function used by the compression unit.
EEE32. The audio encoder of EEE 31, wherein the metadata indicative of the spectral envelope of the highband signal is applicable to the same portion of the audio signal as the metadata indicative of the compression function.
EEE33. A data stream comprising a sequence of access units for a sequence of frames of an audio signal, respectively; wherein an access unit from the sequence of access units comprises waveform data and metadata; wherein the waveform data and the metadata are associated with the same particular frame of the sequence of frames of the audio signal; wherein the waveform data and the metadata are indicative of a reconstructed version of the particular frame.
EEE34. The data stream of EEE 33, wherein the particular frame of the audio signal comprises a lowband signal and a highband signal; wherein the waveform data is indicative of the lowband signal; and wherein the metadata is indicative of a spectral envelope of the highband signal.
EEE35. The data stream of any of EEEs 33 to 34, wherein the metadata is indicative of a compression function applied to the lowband signal.
EEE36. A method for determining a reconstructed frame of an audio signal from an access unit of a received data stream; wherein the access unit comprises waveform data and metadata; wherein the waveform data and the metadata are associated with the same reconstructed frame of the audio signal; wherein the method comprises [0128] generating a plurality of waveform subband signals from the waveform data; [0129] generating decoded metadata from the metadata; [0130] time-aligning the plurality of waveform subband signals and the decoded metadata; and [0131] generating the reconstructed frame of the audio signal from the time-aligned plurality of waveform subband signals and decoded metadata.
EEE37. A method for encoding a frame of an audio signal into an access unit of a data stream; wherein the access unit comprises waveform data and metadata; wherein the waveform data and the metadata are indicative of a reconstructed frame of the frame of the audio signal; wherein the method comprises [0132] generating the waveform data from the frame of the audio signal; [0133] generating the metadata from the frame of the audio signal; and [0134] time-aligning the waveform data and the metadata such that the access unit for the frame of the audio signal comprises the waveform data and the metadata for the same frame of the audio signal.