METHOD FOR WIRELESSLY SYNCHRONIZING ELECTRONIC DEVICES

Abstract

A method of wirelessly synchronizing a plurality of discrete electronic devices each having an audio input channel and audio output channel. The method comprises detecting an output injection parameter of the audio output channel of a first electronic device at the time when an audio signal is output by the audio output channel of the first electronic device; detecting an input injection parameter of the audio input channel of a first electronic device at the time when the audio signal is injected to the audio input channel of the first device; receiving an input injection parameter of the audio input channel of a second device; receiving an output injection parameter of the audio output channel of a second device; and determine a synchronization parameter for synchronizing the first electronic device and the second electronic device on the basis of the detected parameters and the receiving parameters of audio channels.

Claims

1. A method of wirelessly synchronizing a plurality of discrete electronic devices each having an audio input channel and audio output channel, the method comprising: providing a wireless communication channel among the plurality of electronic devices; detecting an output injection parameter of the audio output channel of a first electronic device at the time when an audio signal is output by the audio output channel of the first electronic device; detecting an input injection parameter of the audio input channel of a first electronic device at the time when the audio signal is injected to the audio input channel of the first device; receiving an input injection parameter of the audio input channel of a second device; receiving an output injection parameter of the audio output channel of a second device; and determine a synchronization parameter for synchronizing the first electronic device and the second electronic device on the basis of the detected parameters and the receiving parameters of audio channels.

2. The method of claim 1, wherein the input injection parameter includes a sample number.

3. The method of claim 1, further comprising: detecting sample frequencies of the audio channels of the first and second electronic devices; and determining the synchronization parameter on the basis of the sample frequencies.

4. The method of claim 1, further comprising: generating a 3D audio signal based on the synchronization parameter.

5. The method of claim 1, further comprising: recording a 3D audio signal based on the synchronization parameter.

6. A method of determining the clock drift between a first electronic device and a second electronic device, the method comprising: injecting a plurality of audio signals to the audio output channel of a first device and detecting injection parameters of the plurality of audio signals at the time when the plurality of audio signals are injected to the audio output channel of the first device; detecting injection parameters of the plurality of audio signals at the time when the plurality of audio signals are injected to the audio input channel of the second device; injecting a plurality of audio signals generated by the second device into the audio input channel of the first device and detecting injection parameters at the time when the plurality of audio signals generated by the second device are injected into the audio input channel of the first device, injecting the plurality of audio signals generated by the second device into the audio input channel of the second electronic device and detecting injection parameters at the time when the plurality of audio signals generated by the second electronic device are injected into the audio input channel of the second electronic device; and determining a clock drift between the first electronic device and the second electronic device based on the detected injection parameters.

7. The method of claim 5, wherein the injection parameter includes a sample number.

8. The method of claim 5, wherein the plurality of audio signals include four audio signals.

9. The method of claim 5, wherein the first electronic device and the second electronic device generate the plurality of audio signals alternately.

10. The method of claim 5, wherein the two electronic devices are relatively stationary to each other.

11. The method of claim 5, wherein the two electronic devices are subject to movement relative to each other.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Various exemplary embodiments are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements and in which:

[0012] FIG. 1 is a diagram showing an exemplary setup of two electronic devices situated nearby, according to one embodiment;

[0013] FIG. 2 is a diagram showing the process of determining the relative timing of audio subsystem of electronic devices, according to one embodiment;

[0014] FIG. 3 is a diagram showing an exemplary setup of a plurality of (>2) electronic devices situated nearby, according to one embodiment;

[0015] FIG. 4 is a diagram showing the process of determining the relative clock drift of audio subsystem of stationary electronic devices, according to one embodiment;

[0016] FIG. 5 is a diagram showing the process of determining the relative clock drift of audio subsystem of electronic devices, without restriction on device movement, according to one embodiment;

[0017] FIG. 6 is a diagram showing an alternative process of determining the relative clock drift of audio subsystem of electronic devices, without restriction on device movement, according to another embodiment;

DETAILED DESCRIPTION

[0018] In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant teachings. However, it should be apparent that the present teachings can be practiced without such details or with an equivalent arrangement. In other instances, well known methods, procedures, components, and/or circuitry have been described at a relatively high-level, without detail, in order to avoid unnecessarily obscuring aspects of the present teachings.

[0019] FIG. 1 is a diagram showing an exemplary synchronization system that has two electronic devices situated nearby, according to one embodiment. Electronic device A 101 is equipped with at least one speaker 102 and at least one microphone 103. Similarly electronic device B 105 is equipped with at least one speaker 106 and at least one microphone 107.

[0020] According to one embodiment, steps to obtain precise synchronization between electronic device A 101 and electronic device B 105 are comprised of: [0021] establishing a direct communication link 109 between devices A 101 and B 105; [0022] devices A 101 and B 105 starting audio recording through microphones 103 and 107, respectively and separately; [0023] electronic device A 101 sending an acoustic signal 104 through its speaker 102; [0024] electronic devices A 101 and B 105 detecting start time of said acoustic signal 104 received by microphones 103 and 107, respectively and separately; [0025] device B 105 sending an acoustic signal through its speaker 106 upon detecting acoustic signal 104 from device A 101; [0026] devices A 101 and B 105 detecting start time of said acoustic signal 106 through microphones 103 and 107, respectively and separately; [0027] devices A 101 and B 105 exchanging certain information about detected acoustic signals 104 and 106 over the communication link 109, and [0028] determining a synchronization parameter by one electronic device according to the information provided by the other electronic device.
Information being exchanged and the processing of such information will be described in detail in the following sections of the present application.

[0029] Referring now to FIG. 2, the audio channels of system 100 inlcude digitized samples of audio output channel (speaker) 201, audio input channel (microphone) 202 of device A 101, and digitized samples of audio input channel (microphone) 203, audio output channel (speaker) 204 of device B 102. Each of the electronic device is capable of detecting an input sample number and an output sample number of its own audio channels. For example, the acoustic signal 104 is injected into audio output channel (speaker) 201 of device A 101 starting at sample number A1. It is detected by device A 101 on its audio input channel (microphone) 202 as starting at sample number A1_prime; It is also detected by device B 105 on its audio input channel (microphone) 203 as starting at sample number B1_prime. After device B 105 detects the acoustic signal 104, the acoustic signal 108 is injected into audio output channel (speaker) 204 of device B 105 starting at sample number B2. It is detected by device B 105 on its audio input channel (microphone) 203 as starting at sample number B2_prime; It is also detected by device A 101 on its audio input channel (microphone) 202 as starting at sample number A2_prime.

[0030] It is noted that the sample numbers on the audio input/output channels 201, 202, 203, and 204 are independent of one another, and are incremented relative to their respective and in general, different starting points. In one embodiment, the synchronization method according to the present application determines a sample number T, on device A 101's audio output channel (speaker) 201, that corresponds in time to sample number B2 on device B 105's audio output channel (speaker) 204. The method also determines a sample number T_prime, on device A 101's audio input channel (microphone) 202, that corresponds in time to sample number B2_prime on device B 105's audio input channel (microphone) 203.

[0031] According to one embodiment, sample number T on device A 101's audio output channel (speaker) 201 is calculated according to the following formula:

T=[(A2_prime−A1_prime)+(B2_prime−B1_prime)*Sa/Sb]/2+A1,

wherein Sa is audio channel sampling frequency used by device A 101, and Sb is audio channel sampling frequency used by device B 105.
According to one embodiment, sample number T_prime on device A 101's audio input channel (microphone) 202 is calculated according to the following formula:

T_prime=T−A1+A1_prime.

[0032] According to one embodiment, device B 105 supplies Sb, B2_prime and B1_prime to device A 101 over the communication channel 109, in order for device A 101 to apply the aforementioned formula. According to another embodiment, device B 105 supplies the Sb, and difference between B2_prime and B1_prime, in order for device A 101 to apply the aforementioned formula.

[0033] Once device A 101 obtains T, devices A 101 and B 105 are considered to be in synchronization with each other in terms of audio output. According to one embodiment, in response to a trigger event, e.g. a user pressing a “play” button on the User Interface of a music application, device A 101 chooses a sample number Tp, and injects an audio stream to its audio output channel (speaker) 201 starting at Tp. Device A 101 also calculates the difference between Tp and T, i.e., D1=Tp−T, and send D1 along with Sa to device B 105 over the communications channel 109. Device B 105 then injects an audio stream to its auto output channel (speaker) 204 starting at the following sample number:

B2+D1*Sb/Sa.

In this way, the audio streams output by devices A 101 and B 105 are considered to be in synchronization with each other. The synchronization method is capable of reducing signal difference produced by these devices down to one sample interval. For example, for a commonly used audio channel sampling frequency of 44.1 kHz in electronic devices, the synchronization resolution is the duration of one digitized sample, or approximately 23 microseconds.

[0034] The method disclosed in the present application may also be used for synchronization when a plurality of electronic devices are recording audio signals. Once device A 101 obtains T_prime, devices A 101 and B 105 are considered to be in synchronization with each other in terms of audio input. According to one embodiment, in response to a trigger event, e.g. a user pressing a “record” button on the User Interface of a recording application, device A 101 chooses a sample number Tr, and starts to record an audio stream through its audio input channel (microphone) 202 starting at Tr. Device A 101 also calculates the difference between Tr and T_prime, i.e., D2=Tr−T_prime, and sends D2 along with Sa to device B 105 over the communications channel 109. Device B 105 then starts recording an audio stream through its auto input channel (microphone) 203 starting at sample number B2_prime+D2*Sb/Sa. The audio streams captured by devices A 101 and B 105 will be in synchronization with each other. For a commonly used audio channel sampling frequency of 44.1 kHz in electronic devices, the synchronization resolution is the duration of one digitized sample, or approximately 23 microseconds.

[0035] While the information exchanged between electronic devices and method employed to process the information have been described in accordance with the depicted embodiment of FIG. 2, it is contemplated that many equivalent arrangements may be used. For example, a correspondence between sample numbers other than B2 and T can be derived, if they are equal in distance from B2 and T, respectively. For another example, instead of sending D1 and Sa to device B 105, device A 101 can convert D1 into absolute time based on its sampling frequency, before sending it to device B 105, which will convert it back to a duration in samples based on its own sampling frequency. For yet another example, device B 105 may initiate recording or playback of sound after synchronization, instead of device A 101, by following similar steps as described above.

[0036] While system 100 and synchronization steps have been described in accordance with the depicted embodiment of FIG. 1, it is contemplated that system 100 may embody many forms and include alternative components. According to one embodiment, instead of a direct communication link 109, an indirect communication link can be established between device A 101 and device B 105, for example, through a server. According to another embodiment, a communication link (either direct or indirect) is established any time during the synchronization procedure before the electronic devices exchange information. According to another embodiment, any other component that is capable of producing an acoustic signal on either or both of the electronic devices is used in lieu of speakers 102 or 106. By way of example and not by way of limitation, this can be a haptics actuator vibrating at any frequency. According to yet another embodiment, one or both of the electronic devices is(are) connected physically or wirelessly to device(s) that is(are) capable of producing an acoustic signal, and the said device(s) is(are) used in lieu of a speaker. By way of example and not by way of limitation, this can be an external speaker connected to one or both of the electronic devices through a headphone jack.

[0037] In the foregoing sections, we described methods to wireless synchronize two electronic devices in close proximity to each other. We now turn to the case wherein multiple (>2) devices nearby need to be synchronized.

[0038] Refer now to FIG. 3, wherein an exemplary setup involving 3 devices is depicted. In one embodiment, the three devices may be synchronized pairwisely, e.g. synchronize device A 101 and device B 105 first, then synchronize device B 105 and device C 110.

In another embodiment, the three devices may be synchronized collectively.

[0039] The synchronization steps include similar processing as those in the two devices case, except for the following. When each device sends an acoustic signal through its audio output channel (104, 108, 113), itself and all other devices are recording and detecting said signal in their respective audio input channel (microphone) (103, 107, 112). Each device reports detected sample numbers or the difference between sample numbers to relevant devices. Here a “relevant device” refers to the device that generated the corresponding acoustic signals in the report. According to one embodiment, each device employs different acoustic signal characteristics so that another device can distinguish from which device an acoustic signal is originated. In addition, it is preferable to avoid two or more devices sending acoustic signals that overlap in time which may cause interference. According to one embodiment, a time-division approach can be taken, wherein each device has an assigned time slot to send its acoustic signal. According to another embodiment, a carrier sensing and random backoff mechanism can be employed.

[0040] While we used 3 devices as an example, it is apparent that the same method described in in the previous sections can be extended to apply to a greater number (>3) of devices.

[0041] Referring to FIG. 4, a diagram showing the process of determining the relative clock drift of audio subsystem of electronic devices that are stationary relative to each other, in which digitized samples of audio output channel (speaker) 201 of electronic device A 101, as well as digitized samples of audio input channel (microphone) 203 of electronic device B 105, are illustrated. An acoustic signal is injected into audio output channel (speaker) 201 of device A 101 starting at sample number A1. It is detected by device B 105 on its audio input channel (microphone) 203 as starting at sample number B1′. After a predetermined and fixed time delay T.sub.d1, a second acoustic signal is injected into audio output channel (speaker) 201 of device A 101 starting at sample number A2=T.sub.d1*S.sub.A+A1 where S.sub.A is the sampling frequency employed by device A 101. The said second acoustic signal is detected by device B 105 on its audio input channel (microphone) 203 as starting at sample number B2′. As devices 101 and 105 remain stationary relative to each other, it may be understood that the time difference between B1′ and B2′, T′.sub.d1=(B2′−B1′)/S.sub.B, where S.sub.B is the sampling frequency employed by device B 105, can be attributed entirely to the sum of time delay between A1 and A2, T.sub.d1, and drift that has occurred between the clocks. Therefore, the drift rate of device B 105's audio clock relative to device A 101's audio clock may be calculated as CD.sub.AB=(T′.sub.d1−T.sub.d1)/T.sub.d1. Similarly the drift rate of device A 101's audio clock relative to device B 105's may be calculated as CD.sub.BA=(T.sub.d1−T′.sub.d1)/T′.sub.d1. Parts per Million (ppm) is the standard unit for clock drift rate, and to convert CD.sub.AB and CD.sub.BA to ppm we use the following formulae:

PPM.sub.AB=CD.sub.AB*1000000 , PPM.sub.BA=CD.sub.BA*1000000.

[0042] It is desirable to obtain an accurate estimate of the clock drift as quickly as possible, which entails choosing T.sub.d1 to be as small as possible. Depending on clock crystals and circuits used in each device, the relative clock drift can range from less than 1 ppm for TCXO-driven clocks to more than 100 ppm for typical crystal-driven clocks. As it is generally not known a priori the approximate range of the clock drift between any given pair of devices, T.sub.d1 may be set too small to obtain an accurate estimate in some cases. By way of example, assuming a sampling frequency of 48 KHz used by both devices (i.e. 20.8 μs for each digitized audio sample), and T.sub.d1 set to 10 seconds. Further assuming measured time difference between T.sub.d1 and T′.sub.d1 to be 104 μs (i.e. 5 samples), in which case the clock drift rate is calculated to be 10.4 ppm. Because the time difference measurement has the resolution or granularity of 1 sample, this result can be off by 20%, or in other words, the true clock drift rate can fall in the range between 8.3 ppm to 12.5 ppm. A coarse-to-fine approach is therefore contemplated to solve this issue, with steps comprised of: [0043] choosing a T.sub.d1 and following the procedure described in previous sections to obtain absolute value of the drift, AD=abs(T.sub.d1−T′.sub.d1); [0044] calculating the ratio of the duration of one sample to AD; [0045] if it is less than a preset threshold (for example, 10%), declaring that an accurate estimate of clock drift has been obtained; [0046] otherwise, choosing a T.sub.d2>T.sub.d1, such that the ratio of the duration of one sample to AD, divided by the ratio of T.sub.d2 to T.sub.d1, is less than the preset threshold; [0047] injecting a third acoustic signal into audio output channel (speaker) 201 of device A 101 starting at sample number A3=T.sub.d2*SA+A1; the said third acoustic signal being detected by device B 105 on its audio input channel (microphone) 203 as starting at sample number B3′; [0048] finally T′.sub.d2=(B3′−B1′)/SB, and T.sub.d2 and T′.sub.d2 are now used in lieu of T.sub.d1 and T′.sub.d1 for the clock drift calculation.

[0049] Referring now to FIG. 5, a diagram shows the process of determining the relative clock drift of audio subsystem of electronic devices, without restriction on device movement, according to one embodiment. Digitized samples of audio output channel (speaker) 201 and audio input channel (microphone) 202 of electronic device A 101, as well as digitized samples of audio output channel (speaker) 204 and audio input channel (microphone) 203 of electronic device B 105, are illustrated. Four acoustic signals are shown, they are: [0050] 1) a first acoustic signal injected into audio output channel (speaker) 201 of device A 101 starting at sample number A1, which is detected by the same device on the audio input channel (microphone) 202 as starting at sample number A1′ and by device B 105 on its audio input channel (microphone) 203 as starting at sample number B1′; [0051] 2) a second acoustic signal injected into audio output channel (speaker) 204 of device B 105 starting at sample number B2, which is detected by the same device on the audio input channel (microphone) 203 as starting at sample number B2′ and by device A 101 on its audio input channel (microphone) 202 as starting at sample number A2′; [0052] 3) a third acoustic signal injected into audio output channel (speaker) 204 of device B 105 starting at sample number B3, which is detected by the same device on the audio input channel (microphone) 203 as starting at sample number B3′ and by device A 101 on its audio input channel (microphone) 202 as starting at sample number A3′; and [0053] 4) a fourth acoustic signal injected into audio output channel (speaker) 201 of device A 101 starting at sample number A4, which is detected by the same device on the audio input channel (microphone) 202 as starting at sample number A4′ and by device B 105 on its audio input channel (microphone) 203 as starting at sample number B4′. Note that the sample numbers on the audio input/output channels 201, 202, 203, and 204 are independent of one another, and are incremented relative to their respective and in general, different starting points.

[0054] The time elapsed between the first and second acoustic signals and between the third and fourth acoustic signals are controlled to be sufficiently small such that the two devices 101 and 105 can be considered stationary relative to each other in between the issuance of said signals. It is noted that the time elapsed between the second and third acoustic signals can be arbitrary, and the relative position of devices 101 and 105 can change during this time period.

[0055] According to one embodiment, sample number A2 on device A 101's audio output channel (speaker) 201 is calculated according to the following formula:

A2=[(A2′−A1′)+(B2′−B1′)]/2+A1;

sample number B4 on device B 105's audio output channel (speaker) 204 is calculated according to the following formula:

B4=[(A4′−A3′)+(B4′−B3′)]/2+B3.

Time difference between A2 and A4, is calculated as

TA.sub.d=(A4−A2)/S.sub.A,

and time difference between B2 and B4 is calculated as

TB.sub.d=(B4−B2)/S.sub.B.

The drift rate of device B 105's audio clock relative to device A 101's is calculated as

CD.sub.AB=(TB.sub.d−TA.sub.d)/TA.sub.d.

Similarly the drift rate of device A 101's audio clock relative to device B 105's is calculated as

CD.sub.BA=(TA.sub.d−TB.sub.d)/TB.sub.d.

[0056] Referring now to FIG. 6, a diagram shows an alternative process of determining the relative clock drift of audio subsystem of electronic devices, without restriction on device movement, according to one embodiment. Digitized samples of audio output channel (speaker) 201 and audio input channel (microphone) 202 of electronic device A 101, as well as digitized samples of audio output channel (speaker) 204 and audio input channel (microphone) 203 of electronic device B 105, are illustrated. Four acoustic signals are shown, they are [0057] 1) a first acoustic signal injected into audio output channel (speaker) 201 of device A 101 starting at sample number A1, which is detected by the same device on the audio input channel (microphone) 202 as starting at sample number A1′ and by device B 105 on its audio input channel (microphone) 203 as starting at sample number B1′; [0058] 2) a second acoustic signal injected into audio output channel (speaker) 204 of device B 105 starting at sample number B2, which is detected by the same device on the audio input channel (microphone) 203 as starting at sample number B2′ and by device A 101 on its audio input channel (microphone) 202 as starting at sample number A2′; [0059] 3) a third acoustic signal injected into audio output channel (speaker) 201 of device A 101 starting at sample number A3, which is detected by the same device on the audio input channel (microphone) 202 as starting at sample number A3′ and by device B 105 on its audio input channel (microphone) 203 as starting at sample number B3′; and [0060] 4) a fourth acoustic signal injected into audio output channel (speaker) 204 of device B 105 starting at sample number B4, which is detected by the same device on the audio input channel (microphone) 203 as starting at sample number B4′ and by device A 101 on its audio input channel (microphone) 202 as starting at sample number A4′. Note that the sample numbers on the audio input/output channels 201, 202, 203, and 204 are independent of one another, and are incremented relative to their respective and in general, different starting points.

[0061] The time elapsed between the first and second acoustic signals and between the third and fourth acoustic signals are controlled to be sufficiently small such that the two devices 101 and 105 can be considered stationary relative to each other in between the issuance of said signals. It is noted that the time elapsed between the second and third acoustic signals can be arbitrary, and the relative position of devices 101 and 105 can change during this time period.

[0062] According to one embodiment, sample number A2 on device A 101's audio output channel (speaker) 201 is calculated according to the following formula:

A2=[(A2′−A1′)+(B2′−B1′)]/2+A1;

sample number A4 is calculated according to the following formula:

A4=[(A4′−A3′)+(B4′−B3′)]/2+A3.

Time difference between A2 and A4, is calculated as

TA.sub.d=(A4−A2)/S.sub.A, and

time difference between B2 and B4 is calculated as

TB.sub.d=(B4−B2)/S.sub.B.

The drift rate of device B 105's audio clock relative to device A 101's is calculated as

CD.sub.AB=(TB.sub.d−TA.sub.d)/TA.sub.d.

Similarly the drift rate of device A 101's audio clock relative to device B 105's is calculated as

CD.sub.BA=(TA.sub.d−TB.sub.d)/TB.sub.d.

[0063] In a manner similar to that described in previous sections of the present application, if the time delay between the second and third acoustic signals is not adequate to obtain an accurate estimate of the clock drift, due to the measurement resolution which is the duration of one audio sample, a larger time delay can be chosen and two more acoustic signals can be issued, and the procedures as described in previous sections can be carried out using the timing of the fifth signal in place of the third signal and the sixth signal in place of the fourth.

[0064] A typical electronic device contains multiple clocks, driven by separate crystal oscillators. By way of example, a different clock than the one for the audio subsystem is used by the Operating System of an electronic device. Although the aforementioned methods apply to measuring the relative clock drift between the audio clocks of different devices, it is contemplated that they can be extended to measure the relative clock drift between other clocks, e.g. between the OS clocks. This is achieved, by way of example and not by way of limitation, by first measuring on each device the clock drift between the audio clock and the other clock that is of interest, then combine the results in a straightforward manner.

[0065] While the present invention has been described in detail with respect to specific embodiments thereof, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing, may readily conceive of alterations to, variations of and equivalents to these embodiments. Accordingly, the scope of the present invention should be assessed as that of the appended claims and any equivalents thereto.