Wireless audio synchronization
11678118 · 2023-06-13
Assignee
Inventors
Cpc classification
H04R2227/005
ELECTRICITY
H04J3/0641
ELECTRICITY
H04R5/04
ELECTRICITY
H04J3/0632
ELECTRICITY
H04J3/0667
ELECTRICITY
H04R2420/07
ELECTRICITY
International classification
H04R5/04
ELECTRICITY
Abstract
An audio distribution system includes an audio source; and a plurality of audio playback devices in communication with each other and with the audio source. A group of the audio playback devices are arranged to render audio content provided by the audio source in synchrony. One of the audio playback devices within the group is configured as an audio master which distributes audio content from the audio source to the other audio playback devices within the group, and one of the plurality of audio playback devices, other than the audio master, is configured as a clock master, which distributes clock information that the group of audio playback devices synchronizes to.
Claims
1. A system including a plurality of audio playback devices in communication with each other, the system comprising: a first audio playback device of the plurality of audio playback devices, wherein the first audio playback device is configured to distribute audio content to at least one other audio playback device of the plurality of audio playback devices; a second audio playback device of the plurality of audio playback devices, the second audio playback device different from the first audio playback device, wherein the second audio playback device is configured to distribute clock information to at least one other audio playback device of the plurality of audio playback devices, and wherein the clock information is used to help determine playback timing for the audio content; and a third audio playback device of the plurality of audio playback devices, the third audio playback device different from the first and second audio playback devices, wherein in response to the second audio playback device losing power or being configured to distribute audio content to at least one other audio playback device of the plurality of audio playback devices, the second audio playback device is no longer configured to distribute the clock information and the third audio playback device is configured to distribute the clock information, wherein the at least one other audio playback device that is configured to receive clock information from the second audio playback device includes the first audio playback device.
2. The system of claim 1, wherein the at least one other audio playback device that is configured to receive audio content from the first audio playback device includes the second audio playback device.
3. The system of claim 1, wherein the at least one other audio playback device that is configured to receive audio content from the first audio playback device is in a group with the first audio playback device, and wherein the group is configured to render the audio content in synchrony based at least in part on the clock information.
4. The system of claim 3, wherein the at least one other audio playback device that is configured to receive audio content from the first audio playback device includes the second audio playback device.
5. The system of claim 3, wherein the at least one other audio playback device that is configured to receive clock information from the second audio playback device includes the first audio playback device, and wherein the second audio playback device is outside of the group.
6. The system of claim 1, wherein the second audio playback device is selected according to an algorithm that excludes the first audio playback device from consideration.
7. The system of claim 1, wherein the second audio playback device is configured to distribute clock information to the at least one other audio playback device before the first audio playback device is configured to distribute audio content to the at least one other audio playback device.
8. The system of claim 1, wherein the first audio playback device is further configured to, in addition to distributing audio content to at least one other audio playback device, distribute an identification of when the audio content should be played.
9. The system of claim 1, wherein the at least one other audio playback device that the first audio playback device is configured to distribute audio content to includes the third audio playback device.
10. A method of distributing audio content to and synchronizing audio data among a plurality of audio playback devices that are in communication with each other, the method comprising: distributing, by a first audio playback device of the plurality of audio playback devices, audio content to at least one other audio playback device of the plurality of audio playback devices; distributing, by a second audio playback device of the plurality of audio playback devices, the second audio playback device different from the first audio playback device, clock information to at least one other audio playback device of the plurality of audio playback devices, wherein the clock information is used to help determine playback timing for the audio content; and selecting, in response to the second audio playback device losing power or being configured to distribute audio content to at least one other audio playback device of the plurality of audio playback devices, a third audio playback device of the plurality of audio playback devices to distribute the clock information to the at least one other audio playback device, the third audio playback device different from the first and second audio playback devices, wherein the at least one other audio playback device that receives clock information from the second audio playback device includes the first audio playback device.
11. The method of claim 10, wherein the at least one other audio playback device that receives audio content from the first audio playback device includes the second audio playback device.
12. The method of claim 10, wherein the at least one other audio playback device that receives audio content from the first audio playback device is in a group with the first audio playback device, and wherein the group is configured to render the audio content in synchrony based at least in part on the clock information.
13. The method of claim 12, wherein the at least one other audio playback device that receives audio content from the first audio playback device includes the second audio playback device.
14. The method of claim 12, wherein the at least one other audio playback device that receives clock information from the second audio playback device includes the first audio playback device, and wherein the second audio playback device is outside of the group.
15. The method of claim 10, further comprising selecting the second audio playback device according to an algorithm that excludes the first audio playback device from consideration.
16. The method of claim 10, wherein the second audio playback device distributes clock information to the at least one other audio playback device before the first audio playback device distributes audio content to the at least one other audio playback device.
17. The method of claim 10, further comprising distributing, by the first audio playback device, in addition to distributing audio content to at least one other audio playback device, an identification of when the audio content should be played.
18. The method of claim 10, wherein the at least one other audio playback device that the first audio playback device is configured to distribute audio content to includes the third audio playback device.
19. A method of distributing audio content to and synchronizing audio data among a plurality of audio playback devices that are in communication with each other, the method comprising: distributing, by a first audio playback device of the plurality of audio playback devices, audio content to at least one other audio playback device of the plurality of audio playback devices and an identification of when the audio content should be played; distributing, by a second audio playback device of the plurality of audio playback devices, the second audio playback device different from the first audio playback device, clock information to at least one other audio playback device of the plurality of audio playback devices, wherein the clock information is used to help determine playback timing for the audio content; and selecting, in response to the second audio playback device losing power or being configured to distribute audio content to at least one other audio playback device of the plurality of audio playback devices, a third audio playback device of the plurality of audio playback devices to distribute the clock information to the at least one other audio playback device, the third audio playback device different from the first and second audio playback devices.
20. The method of claim 19, wherein the at least one other audio playback device that the first audio playback device is configured to distribute audio content to includes the third audio playback device.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
DETAILED DESCRIPTION
(7) Some known whole home audio systems consist of a plurality of audio player units, which may be arranged in various rooms throughout a home, and which communicate with each other over a local area network (LAN). Generally, one or more of the player units has access to a source of audio content, which may be a source available over a wide area network (WAN), such as an Internet radio station. Typically, two or more of the individual units can be grouped together, at a user's discretion, into what is often referred to as a zone, in which the grouped player units within the zone playback (render) audio content in synchrony.
(8) To facilitate synchronized playback, one of the units (a master unit) will distribute audio content along with timing (clock) information that one or more of the other units (slave units) use to render the content in lock step with the master. Each of the slave devices can adjust its respective clock, or a rendering time prescribed to the audio content, based on the clock time provided by the master device (i.e., the master's clock time). This can allow the plurality of player units, each with its own clock, to synchronize the rendering of the audio content. Thus, these existing systems rely on a single master unit to serve as both an audio master as well as a clock master.
(9) This disclosure is based on the realization that it can be beneficial to separate the roles of the audio master and the clock master.
(10) Audio distribution system 100,
(11) One use of the system 100 is to play digital audio data, including but not limited to an audio stream, over one or more of the audio playback devices 110. The sources of digital audio provide access to content such as audio streams that move over network 116 to the audio playback devices. The sources of such audio streams can include, for example, Internet radio stations and user defined playlists. Each of such digital audio sources maintains a repository of audio content which can be chosen by the user to be played over one or more of the audio playback devices. Such digital audio sources can include Internet-based music services such as Pandora®, Spotify® and vTuner®, for example. Network attached storage devices such as digital audio source 130, and media server applications such as may be found on a mobile computing device, can also be sources of audio data. In a non-limiting example, the user selects the audio source and the playback devices via a user interface provided by a PC 126 and/or a mobile device 128.
(12)
(13) The audio playback device 110 also includes a network interface 220, a processor 222, audio hardware 224, power supplies 226 for powering the various components, and memory 228. Each of the processor 222, the graphical interface 212, the network interface 220, the audio hardware 224, the power supplies 226, and the memory 228 are interconnected using various buses, and several of the components may be mounted on a common motherboard or in other manners as appropriate.
(14) The network interface 220 provides for communication between the wireless speaker package 200 and audio sources and other networked wireless speaker packages and other audio playback devices via one or more communications protocols. The network interface 220 may provide either or both of a wireless interface 230 and a wired interface 232. The wireless interface 230 allows the wireless speaker package 200 to communicate wirelessly with other devices in accordance with a communication protocol such as IEEE 802.11b/g/n/ac. The wired interface 232 provides network interface functions via a wired (e.g., Ethernet) connection.
(15) In some cases, the network interface 220 may also include a network media processor 234 for supporting Apple AirPlay® (a proprietary protocol stack/suite developed by Apple Inc., with headquarters in Cupertino, Calif., that allows wireless streaming of audio, video, and photos, together with related metadata between devices). For example, if a user connects an AirPlay® enabled device, such as an iPhone or iPad device, to the network, the user can then stream music to the network connected audio playback devices via Apple AirPlay®. Notably, the audio playback device can support audio-streaming via AirPlay® and/or DLNA's UPnP protocols, and all integrated within one device.
(16) All other digital audio coming from network packets comes straight from the network media processor 234 through a USB bridge 236 to the processor 222 and runs into the decoders, DSP, and eventually is played back (rendered) via the electro-acoustic transducer(s) 215. The network interface 220 can also include a Bluetooth circuitry 238 for Bluetooth applications (e.g., for wireless communication with a Bluetooth enabled audio source such as a smartphone or tablet).
(17) Streamed data passes from the network interface 220 to the processor 222. The processor 222 can execute instructions within the wireless speaker package (e.g., for performing, among other things, digital signal processing, decoding, and equalization functions), including instructions stored in the memory 228. The processor 222 may be implemented as a chipset of chips that include separate and multiple analog and digital processors. The processor 222 may provide, for example, for coordination of other components of the audio playback device 110, such as control of user interfaces, applications run by the audio playback device 110.
(18) The processor 222 provides a processed digital audio signal to the audio hardware 224 which includes one or more digital-to-analog (D/A) converters for converting the digital audio signal to an analog audio signal. The audio hardware 224 also includes one or more amplifiers which provide amplified analog audio signals to the electroacoustic transducer(s) 215 for playback. In addition, the audio hardware 224 may include circuitry for processing analog input signals to provide digital audio signals for sharing with other devices.
(19) The memory 228 may include, for example, flash memory and/or non-volatile random access memory (NVRAM). In some implementations, instructions (e.g., software) are stored in an information carrier. The instructions, when executed by one or more processing devices (e.g., the processor 222), perform one or more processes, such as those described elsewhere herein. The instructions can also be stored by one or more storage devices, such as one or more computer- or machine-readable mediums (for example, the memory 228, or memory on the processor). The instructions may include instructions for performing decoding (i.e., the software modules include the audio codecs for decoding the digital audio streams), as well as digital signal processing and equalization.
(20)
(21) The master device 110-1 receives audio data 301 from an audio source 302 (i.e., one of 130 or 142 of
(22) To help ensure that the playback of the audio content is and remains synchronized, the respective internal clocks (a/k/a “local clocks”) of the individual audio playback devices within the group 300 are synchronized. In principle, such clocks comprise an oscillator and a counter. During synchronized (a/k/a “multi-room”) playback, a protocol, such as Network Time protocol (NTP), or the like, is utilized to distribute timing information in a clock master-slave relationship to keep the current clock time on all the devices in the group 300 synchronized.
(23) The time synchronization protocol is separate and aside from the audio stream. In that regard, one of the audio playback devices (audio playback device 110-4 in this case) is designated as a clock master. Notably, the clock master is intentionally a different device than the audio master, and, in the illustrated example, is not even a member of the playback group 300 (i.e., in the illustrated implementation, the audio playback device 110-4 does not render audio in synchrony with the devices in the group 300).
(24) The clock master provides clock data 303 (i.e., the clock master acts as a time server) to the audio playback devices 110-1, 110-2, 110-3 in the group 300 (a/k/a “clock slaves”), which then use that clock data to update their respective clocks to synchronize with that of the clock master. The clock data may be provided periodically, e.g., every 1 to 6 seconds, to keep the grouped devices updated and in sync with the clock master. Separately, the audio master may also provide a “play at” time 304; i.e., an identification of when all the devices in the group should begin playing the distributed audio content. The “play at” time 304 may represent a clock time at which the units are to render a first sample in a series of ordered audio data samples, with the remainder of the audio data samples to be played in order at a defined sample rate. The “play at” time 304 is communicated in control data that is separate from the audio stream and, in some cases, is only sent once for each track (i.e., it is not included with every frame). Every new track or stream will get a new “play at” time.
(25) The audio slaves receive the first sample in an audio stream 301 and begin playback designated “play at” time 304. Since all the grouped devices are synced to the clock master, and, thus, all have the same current clock time, they all begin playback at the same time. From there, the devices can all provide playback at a constant sample rate, and, consequently, stay in sync.
(26) With reference to
(27) For synchronized playback, the encoded data is immediately pulled out of the audio master's ring buffer 402 and is provided to the slave playback device(s) (a/k/a audio slave(s)) ring buffer 402. This distribution of encoded audio data may take place via unicast communications between the audio master and each of the individual audio slave devices. From there, the audio slaves follow the same process as outlined above with respect to the audio master. Each audio slave will decode the encoded audio pulled from the audio master, assign an offset to the frame header, and store the decoded audio frames in their respective sample buffers 406. The audio slaves each apply their own offsets to the audio frames, but these offsets will be the same as those applied by the audio master since each device is receiving the same stream and is using the same decoder software.
(28) Since the clocks on the audio master and audio slaves are in sync and the offsets are all the same, each device in the group (e.g., group 300,
(29)
(30) The process 500 also includes the selection of an audio master device (504). In some instances, the audio master is automatically set to the audio playback device that was selected to serve as the basis for the group. Alternatively, the device within the group that has the strongest WiFi signal strength may be selected as the audio master. For example, each of the audio playback devices in the group may communicate its signal strength to the other group members. Once each of the devices in the group has received the signal strength information from each of the other group members, each device will identify the group member with the highest/strongest signal strength as the audio master for the group. Alternatively or additionally, the group members may all report their respective signal strengths to one network device (e.g., the controller device) and that network device may identify the audio master to the group members.
(31) To ensure that the audio output from the audio playback devices is truly synchronized, the respective clocks of the audio playback devices are all synchronized. To achieve this clock synchronization, one of the audio playback devices is selected to serve as a clock master (506). The clock master distributes a reference time that all the audio playback devices in the group synchronize their respective clocks to.
(32) The clock master may be one of the audio playback devices in the group, or may be an audio playback device outside of the group; however, the clock master is selected such that no audio playback device can concurrently serve as both the audio master and the clock master. In that regard, the clock master can be selected according to a leader selection algorithm (a/k/a “leader election algorithm” or “leader election”) that excludes the audio master from consideration. In distributed computing, leader election is the process of designating a single node as the organizer of some task distributed among several computers (nodes), which in this case are the audio playback devices 110. Before the task is begun, all network nodes are either unaware which node will serve as the “leader” (or coordinator) of the task, or unable to communicate with the current coordinator. After a leader election algorithm has been run, however, each node throughout the network recognizes a particular node as the task leader.
(33) The network nodes communicate among themselves to decide which of them will get into the “leader” state. For that, they need some method to break the symmetry among them. For example, if each node has unique and comparable identities, then the nodes can compare their identities, and decide that the node with the highest identity is the leader. In some cases, the leader selection algorithm can select the clock master to be the playback unit that reports the lowest ping time.
(34) For example, each of the audio playback devices in the group may ping the access point (item 114,
(35) In some implementations, the audio playback devices may be arranged in a peer-to-peer network that includes the access point. In such configurations, the clock master may be selected to be the audio playback device that records the shorted ping time with the access point, as described above, or, alternatively, the clock master may be determined based on ping times among the nodes of the peer-to-peer networks. For example, the clock master may be selected to be the audio playback device (e.g., excluding the audio master) that records the smallest mean ping time with the other network nodes.
(36) With a clock master selected, the members of the group, including the audio master, can begin to synchronize (508) the clock master's clock. In some cases, the clocks of the audio playback devices can be synchronized to the master clock time according to a linear time model m.sub.nx+b.sub.n, where m.sub.n is the rate difference between the master's clock and the clock of slave.sub.n, x represents the current clock time on slave.sub.n, and b.sub.n is the offset between the master clock and the clock of slave.sub.n since booting up. In some cases, m.sub.n and b.sub.n are determined by performing a linear fit of collected timestamps from clock master and local clock over a given window of time. With this model in place, the devices can maintain some degree of synchronization even in the event that the clock master needs to be changed.
(37) In that regard, there are two situations in which a new clock master may be required: 1.) if the current clock master loses power; or 2.) if the clock master becomes the audio master—such as may occur when the current clock master is selected as the head of a new grouping. In either case, the leader selection algorithm restarts. At this point, the system is in an “open loop” state in which the audio playback devices initially maintain clock synchronization using their existing, respective m.sub.n, and b.sub.n values. Then, once a new clock master is selected, the other audio playback devices start slowly adjusting their clocks to that of the new clock master using a heuristic approach which causes the audio playback devices to adjust their respective m and b values based on the new master clock, while the new clock master slowly reduces its rate to zero, such that m.sub.master=1, and b.sub.master=0.
(38) Referring to
(39) The master selection algorithm may select (608) the audio master based on the audio playback device within a playback group that reports the strongest WiFi signal connection, excluding any device that is serving as the clock master. Alternatively, the audio master may be selected (608) based on the audio playback device within a playback group that reports the strongest WiFi signal connection, regardless of whether that device is also serving as the clock master. In which case, if the clock master is subsequently selected (608) as the audio master, then a new clock master can be selected while or after playback of the selected audio begins—that is, the audio playback devices within the group can still take advantage of the fact that their clocks were pre-synched while a new clock master is selected.
(40) While implementations have been described in which an audio playback device may not contemporaneously serve as both an audio master and a clock master for a playback group, it is contemplated that an audio playback device may serve as a clock master for one playback group, while also serving as an audio master for another, different playback group within the same audio distribution system.
(41) In some implementations, the audio source is connected to one of the audio playback devices of a playback group via a personal area network (PAN) (e.g., via a Bluetooth connection). In such cases, the device that is coupled to the audio source can be selected as the audio master—that selection can then be communicated from the audio master to the other devices in the group via the LAN.
(42) In certain implementations, the audio source is connected to one of the audio playback devices of a playback group via a hardwire connection through an auxiliary input port on the audio playback device. In such cases, the device that is directly coupled to the audio source can be selected as the audio master—that selection can then be communicated from the audio master to the other devices in the group via the LAN.
(43) Implementations of the systems and methods described above comprise computer components and computer-implemented steps that will be apparent to those skilled in the art. For example, it should be understood by one of skill in the art that the computer-implemented steps may be stored as computer-executable instructions on a computer-readable medium such as, for example, floppy disks, hard disks, optical disks, Flash ROMS, nonvolatile ROM, and RAM. Furthermore, it should be understood by one of skill in the art that the computer-executable instructions may be executed on a variety of processors such as, for example, microprocessors, digital signal processors, gate arrays, etc. For ease of exposition, not every step or element of the systems and methods described above is described herein as part of a computer system, but those skilled in the art will recognize that each step or element may have a corresponding computer system or software component. Such computer system and/or software components are therefore enabled by describing their corresponding steps or elements (that is, their functionality), and are within the scope of the disclosure.]
(44) A number of implementations have been described. Nevertheless, it will be understood that additional modifications may be made without departing from the scope of the inventive concepts described herein, and, accordingly, other implementations are within the scope of the following claims.