AUDIO DATA BUFFERING FOR LOW LATENCY WIRELESS COMMUNICATION

Abstract

A method for wireless RF transmission of short audio data blocks, e.g. 0.5 ms to 2 ms blocks, with low latency. The method involves a fixed part (FP) serving as synchronization master, and one or more portable parts (PP) being synchronization slaves in a time division scheme with fixed transmission intervals, and with a fixed and limited payload capacity of the RF transmission channel, such as 1.5-3 times the capacity required to transmit the audio data blocks in real- time. Short length transmission and receiving queues (TQ, RQ), e.g. having each 2-8 spaces for audio data blocks, for the audio data blocks are used to allow retransmission of blocks in—response to a missing acknowledge response from the portable part (PP). The queuing is operated so as to result in a fixed latency C determined e.g. by the transmission and receiving queue (TQ, RQ) lengths. A two-way audio scheme can be implemented following the same principle and utilizing the same RF transmission principles. The method provides a robust and low latency wireless audio interface suitable for dedicated audio devices and/or combined audio and Human Interface Devices (HIDs), e.g. for gaming equipment.

Claims

1. A method for transmission of a digital audio signal divided into digital audio data blocks of a fixed length, between a fixed part being a timing master and one or more portable parts being timing slaves on a wireless RF transmission channel according to a time division multiplexing or time division duplexing scheme with a fixed transmission time interval, wherein a raw payload bandwidth of the wireless RF transmission channel is a fixed factor larger, than a bandwidth required to transmit the audio data blocks, wherein the wireless RF transmission comprises a fixed wireless frame length, each wireless frame containing one or more time slots, the method comprising: 1) receiving an audio data block at the fixed part, 2) queuing the audio data block at an empty space in a fixed length FIFO type transmitter queue having a plurality of spaces for audio data blocks, 3) transmitting a wireless data packet containing an audio data block from an output end of the transmitter queue from the fixed part to the portable part on the wireless RF transmission channel, 4) receiving the wireless data packet by the portable part, 5) queuing the audio data block received in the wireless data packet at an empty space in an input end of a fixed length FIFO type receiver queue having a plurality of spaces for audio data blocks, 6) transmitting an acknowledgement message from the portable part to the fixed part on the RF transmission channel in response to the received audio data block, and 7) outputting an audio data block from an output end of the receiver queue, wherein the steps 1)-7) are performed in time synchronization so as to provide a fixed latency between step 1) and step 7), wherein the fixed part is arranged to retransmit an audio data block from the transmitter queue in a subsequent wireless data packet in case no acknowledgement message has been received for transmission of that audio data block, and wherein an audio data block is deleted from the transmitter queue in response to the fixed part receiving the acknowledgement message for successful transmission of that audio data block.

2. The method according to claim 1, wherein the fixed length of the transmitter queue is equal to the fixed length of the receiver queue, and wherein this fixed length is 2-8.

3. The method according to claim 1, wherein the fixed length of the audio data blocks is selected to be within 0.2 ms to 8 ms.

4. The method according to claim 1, wherein the fixed length of the wireless frame is selected to be within 0.2 ms to 20 ms.

5. The method according to any of the preceding claim 1, wherein the fixed length of the transmitter queue is 2-8, wherein the fixed length of the receiver queue (RQ) is equal to the fixed length of the transmitter queue, wherein the fixed length of the audio data blocks is selected to be within 0.2 ms to 8 ms, wherein the fixed length of the wireless frame is selected to be within 0.2 ms to 20 ms, and wherein the fixed transmission time interval is selected to be within 0.2 ms to 10 ms.

6. The method according to claim 1, comprising re-transmitting one queued audio data block from the fixed part a plurality of times.

7. The method according to claim 1, comprising receiving Human Interface Device data at the portable part, transmitting a Human Interface Device data block in a wireless frame also containing an audio data block on the wireless RF transmission channel from the portable part (PP) to the fixed part, and outputting a Human Interface Device data block received at the fixed part.

8. The method according to claim 7, wherein said Human Interface Device data block is transmitted un-queued at the portable part and output un-queued at the fixed part.

9. The method according to claim 1, comprising duplex audio data block transmission between the fixed part the portable part.

10. The method according to claim 9, wherein the portable part is arranged to transmit audio data blocks to the fixed part on said RF transmission channel.

11. The method according to any of the preceding claim l, wherein the wireless frame comprises at least one time slot for downlink data, and one or more time slots for uplink data.

12. The method according to claim 1, comprising transmitting, in one wireless frame, one audio data block on two different RF bearer frequencies from the fixed part to the one or more portable parts.

13. A wireless RF transmitter comprising at least one RF transmitter circuit connected to an antenna, and being arranged to operate as the fixed part according to the method of claim 1.

14. A wireless RF receiver comprising at least one RF receiver circuit connected to an antenna, and being arranged to operate as the portable part according to the method of claim 1.

15. A system comprising at least one of: (i) comprising at least one RF transmitter circuit connected to an antenna and being arranged to operate as the fixed part according to the method of claim 1, and (ii) one or more wireless RF receivers comprising at least one RF receiver circuit connected to an antenna and being arranged to operate as the portable part according to the method of claim 1, wherein the system is one of: a wireless headset, a wireless mouse, a wireless gaming controller, a wireless keyboard, a wireless microphone, a wireless loudspeaker, a wireless intercom system, a video system, and a Virtual Reality system.

Description

BRIEF DESCRIPTION OF THE FIGURES

[0066] The invention will now be described in more detail with regard to the accompanying figures of which

[0067] FIG. 1 illustrates a simple block diagram of a system embodiment with a fixed part (FP) and a portable part (PP) transmitting a wireless data packet including an audio data block and with acknowledgement over a wireless RF channel,

[0068] FIG. 2 illustrates steps of a method embodiment,

[0069] FIG. 3 illustrates a block diagram of a system embodiment with an FP and a PP arranged for full duplex audio transmission,

[0070] FIGS. 4-7 illustrate various strategies for handling the TQ and RQ at different situations of retransmissions in case of packet loss, and

[0071] FIGS. 8a and 8b illustrate examples of wireless data packet formats in case of audio data only, and in case of audio data combined with HID data.

[0072] The figures illustrate specific ways of implementing the present invention and are not to be construed as being limiting to other possible embodiments falling within the scope of the attached claim set.

DETAILED DESCRIPTION OF THE INVENTION

[0073] FIG. 1 shows a block diagram illustrating the basic setup behind the invention, namely a fixed part FP and a portable part PP communicating, here shown as duplex communication, by means of a wireless electromagnetic RF signal on RF bearer, optionally two RF bearers with different frequencies. Thus, basically the FP and PP both have RF antennas and RF circuits for transmitting and receiving RF signals. The FP may be implemented as a dongle for a computer, or being an integrated part of e.g. a gaming console or other dedicated device.

[0074] The FP receives an input audio signal A_I, and transmits wireless data packets WP each with an audio data block on the RF bearer, and the wireless data packets WP are received by the PP which unpacks the audio data blocks to arrive at the original audio signal A_O. The PP transmit in response to a successfully received wireless data packet an acknowledgement message ACK back to the FP, which can then retransmit a lost audio data block in the next wireless data packet.

[0075] In some embodiments, the FP performs a scanning and preferably performs own measurement as well as collects Received Signal Strength Indication data from the PP for the supported channels, so as to dynamically select the two frequencies F1, F2 according to a predetermined algorithm for finding the frequencies F1, F2 with the minimal possible RF interference from other RF traffic in the RF band covering the supported channels, e.g. an ISM band.

[0076] By using short audio data blocks, and short buffers or queues at the FP and the PP side, a low latency can be obtained, and with a small extra capacity of the RF transmission channel, a high reliability can be obtained, since it is possible to retransmit lost wireless data packets WP. Using an RF channel with a fixed capacity of a factor of 2 of the payload capacity required for transmission of the audio data blocks, and with audio data block lengths of such as 2 ms, and with audio data block buffer or queue length at the FP and the PP side with a length of 4, a good compromise between low latency and high immunity against packet losses has been found. The low latency and high reliability may e.g. be appreciated for products such as gaming devices with audio capabilities.

[0077] In FIG. 1 only one PP is shown, however, it is to be understood that embodiments of the invention support broadcasting to a plurality of PPs, i.e. a point to multi-point transmission.

[0078] In FIG. 1 only one way audio is shown, but in case e.g. of a headset, it is to be understood that the method of the invention supports embodiments where audio data can be transmitted both uplink and downlink.

[0079] FIG. 2 illustrates steps of a preferred method embodiment. The method for wireless RF transmission of a digital signal in data packets between a FP being a timing master and one or more PP being timing slaves. The FP and PP are setup in a time division multiplexing or duplexing scheme, i.e. setup to transmit a wireless frame with a fixed time interval of e.g. 1 ms, on an RF channel such as in the 2.4 GHz band with a capacity of e.g. 2 times the payload capacity required for transmission of the audio data blocks. The structure of the wireless frame for wireless transmission is fixed, and the fixed length can be such as 2 ms.

[0080] The following method steps are understood to be continuously performed in each time interval in order to stream of audio. First step is receiving R_AB a digital audio data block at the FP, e.g. directly form of an output of an analog-to-digital converter from an audio device, e.g. a gaming headset, or from an output from a processor having performed a pre-processing, e.g. involving a compression, encryption or the like, on the audio signal. The audio data blocks have a fixed predetermined length, such as 2 ms.

[0081] Next step is queuing TQ_AB the audio data block at an empty space in a fixed length FIFO type TQ having 4 spaces for audio data blocks.

[0082] Next step is transmitting T_WP a wireless data packet containing an audio data block from an output end of the TQ from the fixed part FP to the portable part PP on the wireless RF transmission channel.

[0083] At the PP end, the wireless data packet is received R_WP, and the audio data block therein is queued RQ_AB at an empty space in an input end of a fixed length FIFO type RQ having 4 spaces for audio data blocks. In response to the received audio data block, the PP transmits T_ACK an acknowledgement message to the FP on the RF transmission channel. Further, the PP outputs O_AB an audio data block from an output end of the receiver queue RQ, and thus the resulting audio data block has been successfully transmitted, and e.g. the audio data therein can be applied to a digital-to-analog converter, e.g. after being decoded or decrypted, if required, e.g. to produce an audio signal to be played by a loudspeaker or a headphone or the like.

[0084] Some embodiment involve duplex transmission between the FP and the PP. This may be in the form of 2-way audio data transfer, and in addition it is possible as well to provide 1- or 2-way HID data transfer.

[0085] In a special embodiment, dualslot diversity is provided in the form of transmission of two data packet in one wireless frame, on two different RF transmission bearer frequencies. Such embodiment may especially involve sequentially scanning such as 10-100 supported channels within a limited frequency band, such as a limited band around 2.4 GHz, wherein the channels closely spaced in frequency, e.g. space by such as spaced by 1-2 MHz. The scanning serves to determine two bearer frequencies for the RF transmission, where the scanning reveals the least RF interference level. A control algorithm can be provided to determine which two data packets to transmit in a time window. Such control algorithm may have inputs regarding number of audio data blocks in the TQ and RQ, as well as other parameters.

[0086] FIG. 3 shows an embodiment similar to FIG. 1, except that the method includes duplex audio transmission between the FP and PP. As in FIG. 1, the FP can transmit an audio input A1_I to the PP in a wireless data packet, and the PP can accordingly output a representation of the audio signal A1_O. In this embodiment, the PP preferably utilizes the same method of the first aspect of the invention to transmit an audio input A2_I in a wireless data packet via the RF transmission channel to the FP which can then output audio signal A2_O accordingly. Such duplex audio connection can be used e.g. if the PP is a wireless headset comprising a microphone and a loudspeaker to capture audio signal A2_I, and to generate audio in response to audio signal A1_O, respectively.

[0087] In the following, a further explanation of preferred embodiments will be described, i.e. related to the queue or buffer of audio data blocks on the transmitter side, the transmitter queue TQ, and the queue or buffer of audio data blocks on the receiver side, the receiver queue RQ. The TQ and RQ are both FIFO types.

[0088] On the transmitter side, audio data blocks are entered in to the TQ at a fixed rate. On the receiver side the audio data blocks are taken out of the RQ at a fixed rate. In a transmitting device, such as for example a wireless microphone, the analog signal from a microphone transducer is analog-to-digital converted. The digital audio data, e.g. with a resolution of 8 to 32 bit, is compressed by an audio compression algorithm, typically implemented in an Digital Signal Processor. (DSP). The output data of the audio compression are collected to fixed size audio data blocks. The audio data blocks are entered to the TQ. At next occurrence of timeslot for wireless transmission, the oldest audio data block from the TQ is transmitted. If the TQ is empty when wireless transmission timeslot occurs, the wireless transmitter can either omit transmission or it can transmit an empty data packet. Usually, the wireless transmitter also transmits a wireless protocol control field, which may indicate empty data packet. When the transmitter side receives acknowledgement from the receiver side, the oldest audio data block is deleted from the TQ. If no acknowledgement is received, the audio data block will stay on the TQ and be retransmitted at next occurrence of timeslot for wireless transmission.

[0089] At the receiving device, such as for example a loudspeaker, when the wireless receiver receives an audio data block, it is entered to RQ. The receiver side replies with an acknowledgement to the transmitter side. If the RQ is already full, no acknowledgement is sent. At fixed time interval the oldest audio data block is removed from the RQ, and each data element in the block is feed to an audio de-compression algorithm, typically implemented in an DSP. The digital output data, which may have a resolution of 8 to 32 bit, may be digital-to-analog converted and outputted to a loudspeaker transducer.

[0090] The state of the TQ and RQ when no wireless transmission errors occur is a full RQ and an empty TQ. When wireless transmission errors occur, the RQ is not full, and the TQ is not empty. As long as the TQ does not run full and RQ does not run empty, there is a fixed latency from audio data input in the transmitter side to audio data output in the receiver side, even when retransmissions of wireless packets occur. Usually, the time for entry of an audio data block to TQ and the time for removal of audio data block from the RQ are aligned to the wireless transmission frames for achieving minimum overall system latency.

[0091] As the preferred state of the queues are a full RQ and an empty TQ, special handling is required during initialization of an audio transfer session. During such initialization phase, no audio data blocks are removed from the RQ until the RQ is completely full.

[0092] In the scenarios of server transmission error rate, which result in the empty RQ and full TQ, specific error handling procedures may be performed. When the TQ is full and new audio data blocks are ready to be entered to the TQ, the oldest audio data block is removed from the TQ, even though it has not been acknowledged from the receiver side. At the receiver side, when the RQ is empty, and an audio block is requested, the de-compression algorithm is informed about missing audio data and can perform packet loss concealment (PLC) to eliminate or at least reduce audible artifacts.

[0093] Following the above description of the preferred function of the TQ and RQ, FIGS. 4-7 illustrate various scenarios regarding decisions to be administered related to which audio data block(s) to transmit in a time window. This depends on the TQ on the FP side, and RQ status on the PP side, but still various solutions can be applied, depending on the given circumstances and performance demands. In FIGS. 4-7, audio data blocks are indicated with hatched boxes, while white boxes indicate empty spaces. The illustrations indicate a sequence of time frames starting from the top. To the left the FP is indicated with audio input A_I and TQ. In the middle, the wireless interface WI is indicated, and to the right the PP is indicated, including RQ and audio output A_O. The example in FIGS. 4-7 specifically shows the scenario for a TQ and an RQ with a length of 4, and with two transmissions per wireless frame (N=2). FIGS. 4-6 indicate acknowledgement messages ACK from the PP, and they indicate the wireless RF transmission on two different bearer frequencies F1, F2 in one wireless frame.

[0094] FIG. 4 illustrates the normal sequence of audio queue flow during a wireless frame when there are no transmission errors in the wireless transmission. The preferred status is an empty transmit queue and full receive queue.

[0095] FIG. 5 illustrates audio que flow when there is one transmission error in the wireless transmission WI and the audio block is retransmitted once.

[0096] FIG. 6 illustrates audio queue flow for the case when initial status is a non-empty TQ and a partly full receive queue RQ, and when the second wireless transmission WI brings the queues back to the preferred status, i.e. an empty TQ and a full RQ.

[0097] FIG. 7 illustrates queue status and packet transmissions for the case with the maximum number of sequential wireless transmission errors that can be handled without effecting the audio transfer. The shaded elements of TQ and RQ represent content at the queues at the end of a wireless frame. In the first two frames, Fr1 and Fr2, the TQ and RQ have the preferred status, i.e. the TQ is empty and the RQ is full.

[0098] The solid arrows of the wireless interface WI represent successful transmission of wireless data packets, and white arrows represent empty wireless data packet transmissions. An empty wireless packet transmission means one without audio data block or one with redundant transmission of same audio data block, but with unchanged transmission sequence number. In frames Fr3, Fr4 and Fr5, the shaded arrows represent wireless data packet transmission errors. In frame Fr5 the TQ is nearly full and the RQ is nearly empty. In the frames Fr6, Fr7 and Fr8, both wireless transmissions are successful, and the preferred status of TQ and RQ are regained. In frames Fr9, Fr10 and Fr11 the wireless transmissions are error free and the TQ and RQ have their preferred status.

[0099] FIGS. 8a and 8b illustrate examples of the wireless data to be transmitted. FIG. 8a shows an example of a layout of wireless packets for use to transmit audio data AD only. It contains a field WPC with management of wireless protocol (MAC-layer protocol control). Especially, this field WPC may carry: synchronization information, addressing, sequence numbering, acknowledgements, interference and frequency management information, antenna control, and binding and connection control commands. The MAC layer signaling does not affect the application payload bandwidth. Further, a receiver sequence number RSN is included for acknowledgement in opposite direction. A transmit sequence number TSN indicates the number on the audio data block currently being sent. A field contains the audio data AD, and a CRC field is included to allow checking the integrity of a received packet. FIG. 8b is identical with the example of FIG. 8a except that a field with un-buffered low latency HID data is included.

[0100] In a specific embodiment, the MAC/PHY is designed based on the following features: [0101] Frequency range: 2.402-2.480 GHz [0102] Modulation: 2 Mbit/s, GFSK [0103] Data integrity check: CRC-24 [0104] Encryption of payload data by AES128, CCM with addition of 32 bit MIC (configurable). The same (shared) security key is used for all FP, PP devices in a network. [0105] Antenna diversity, 2 antennas in both FP and PP. [0106] Interference scanning in both FP and PP, and interference avoidance procedures using automatic frequency (re)selection for dual slot transmission on two RF bearer frequencies. RF bearer frequency handover is preferably implemented to avoid interference, and this is preferably done seamless and does not interrupt the flow of application data. [0107] Reliable application data transfer by configurable retransmission, both downlink and uplink. The retransmission procedure uses both active carriers and can be combined with redundant transmission. [0108] MAC layer signaling field carries: synchronization information, addressing, sequence numbering, acknowledgements, interference and frequency management information, antenna control, and binding and connection control commands. The MAC layer signaling does not affect the application payload bandwidth. [0109] The FP always transmits at least a synchronization beacon when no portable devices are active.

[0110] The following are preferred features: [0111] Automatic retransmission request. [0112] 1-bit transmit and receive sequence number are used for acknowledgement transmission (and retransmission). [0113] Receiver (e.g. PP) can respond with either acknowledgement ACK or non-acknowledgement NACK—or the ACK/NACK can get lost. [0114] Acknowledgement ACK is signaled with receiver sequence number different from the transmitted sequence number. [0115] Non-acknowledgement NACK is signaled with receiver sequence number equal to the transmitted sequence number. [0116] For the retransmission method, the normal and desired state is that the RQ is full, and the TQ is empty (for minimum latency). [0117] Typical configuration: audio data block size of 2 ms (e.g. 96 samples), a wireless frame length of 2 ms, and wireless transmission every 1 ms, TQ and RQ each with space for 4 blocks in each, allowing up to 7 retransmission before data loss, resulting in total system latency of 8 ms. [0118] Retransmission scheme can also be employed for duplex audio connections. [0119] In the special situation where acknowledgements are all lost and RQ runs empty, the transmitter (e.g. FP) continues to output packets using the dualslot method (unacknowledged redundant transmission)—improved handling of asymmetric link budget. [0120] Wireless packets may additionally contain both retransmitted audio data and most recent user state data (un-retransmitted HID data). [0121] Point-to-multipoint, where some device are using retransmitted (audio) payload and others are not retransmitting (HID) payload.

[0122] To sum up, the invention provides a method for wireless RF transmission of short audio data blocks, e.g. 0.5 ms to 2 ms blocks, with low latency. The method involves a fixed part (FP) serving as synchronization master, and one or more portable parts (PP) being synchronization slaves in a time division scheme with fixed transmission intervals, and with a fixed and limited payload capacity of the RF transmission channel, such as 1.5-3 times the capacity required to transmit the audio data blocks in real-time. Short length transmission and receiving queues (TQ, RQ), e.g. having each 2-8 spaces for audio data blocks, for the audio data blocks are used to allow retransmission of blocks in response to a missing acknowledge response from the portable part (PP). The queuing is operated so as to result in a fixed latency determined e.g. by the transmission and receiving queue (TQ, RQ) lengths. A two-way audio scheme can be implemented following the same principle and utilizing the same RF transmission principles. The method provides a roboust and low latency wireless audio interface suitable for dedicated audio devices and/or combined audio and Human Interface Devices (HIDs), e.g. for gaming equipment.

[0123] Although the present invention has been described in connection with the specified embodiments, it should not be construed as being in any way limited to the presented examples. The scope of the present invention is to be interpreted in the light of the accompanying claim set. In the context of the claims, the terms “including” or “includes” do not exclude other possible elements or steps. Also, the mentioning of references such as “a” or “an” etc. should not be construed as excluding a plurality. The use of reference signs in the claims with respect to elements indicated in the figures shall also not be construed as limiting the scope of the invention. Furthermore, individual features mentioned in different claims, may possibly be advantageously combined, and the mentioning of these features in different claims does not exclude that a combination of features is not possible and advantageous.