System for real-time synchronization

Abstract

The invention relates to a system and a corresponding method for sharing multi-media content among a plurality of users. The invention further relates the corresponding devices and a corresponding computer program for conducting the sharing of multi-media content. The system comprises a first device comprising means for generating an invitation and a play time stamp and the system further comprises means for transmitting the play time stamp and the invitation to at least one second device. The system also comprises a third device that may generate a system time and distribute the system time. The at least one second device may play multi-media content synchronous to playback of the same multi-media content at the first device.

Claims

1. A system for real-time playback synchronization of multi-media content comprising a host device (H), at least one client device (C.sub.1, C.sub.2), and a server device (S), wherein the real-time playback synchronization is performed by: the host device (H) being configured to: generate an invitation (I) comprising information identifying the multi-media content for playback and a play time stamp (PT) in accordance with the playback of the multi-media content by the host device (H), the play time stamp (PT) defining a position of the multi-media content with respect to a reference time, the reference time being a system time (ST) serving as a reference for playback in the system, the play time stamp (PT) specifying a data package of the multi-media content, and the information identifying the multi-media content specifying a content specific pattern of sizes of data packages in predefined successive time intervals of a file of the content, and a first data package defining a starting point of the multi-media content; and transmit the invitation (I) and the play time stamp (PT) to the at least one client device (C.sub.1, C.sub.2); the server device (S) being configured to: generate the system time (ST); and distribute the system time (ST) to the at least one client device (C.sub.1, C.sub.2) and the host device (H); and the at least one client device (C.sub.1, C.sub.2) being configured to: receive the invitation (I) and the play time stamp (PT) from the host device (H); receive the system time (ST) from the server device (S); match the content specific pattern specified in the information identifying the multi-media content, which is included in the invitation (I), with patterns of multi-media content stored in a database to identify the multi-media content to be played back; and play the identified multi-media content synchronously with the host device (H) from the position defined by the play time stamp when the play time stamp (PT) matches to the system time (ST).

2. The system according to claim 1, wherein the host device (H) is further configured to transmit the play time stamp (PT) to the at least one client device (C.sub.1, C.sub.2) directly and/or via the server device (S).

3. The system according to claim 1, wherein the host device (H) is further configured to transmit the invitation (I) to the at least one client device (C.sub.1, C.sub.2) directly and/or via the server device (S).

4. The system according to claim 1, wherein the host device (H) is further configured to transmit a time of the host device (H) to the server device (S); and wherein the server device (S) is configured to generate the system time (ST) on the basis of the time of the host device (H).

5. The system according to claim 1, wherein the server device (S) is configured to transmit the system time (ST) to the host device (H); and wherein the host device (H) is configured to, upon receipt of the system time (ST), generate the play time stamp (PT) on the basis of the system time (ST).

6. The system according to claim 1, wherein the information identifying the multi-media content comprises predetermined information on the data packages of the multi-media content in least one consecutive time interval and/or meta-data of the multi-media content.

7. The system according to claim 6, wherein the predetermined information on the data packages comprises a package bitsize and/or a package interval.

8. The system according to claim 6, wherein the at least one client device (C.sub.1, C.sub.2) is configured to gain access to the multi-media content in accordance with the predetermined information and/or the meta-data of the multi-media content.

9. The system according to claim 1, wherein the play time stamp (PT) contains information on a start or a change of playback and/or a time delay with respect to a time at the host device (H) and/or relating to the system time (ST) shared by participating devices.

10. The system according to claim 1, wherein the at least one client device (C.sub.1, C.sub.2) is further configured to continuously synchronize the play time stamp (PT) of the multi-media content with the host device (H) and/or via the server device, in accordance with predetermined information and/or in accordance with a specific time stamp (PT).

11. The system according to claim 1, wherein the at least one client device (C.sub.1, C.sub.2) is configured to adjust a time latency with respect to the system time (ST) of the server device (S) and/or the host device, via continuous synchronization and/or via sending a ping signal.

12. The system according to claim 1, wherein the server device (S) is configured to send a ping signal to the at least one client device (C.sub.1, C.sub.2) and/or the host device; and wherein the server device (S) is further configured to adjust the system time (ST) in accordance with a time latency between the at least one client device (C.sub.1, C.sub.2), the host device (H), and the server device.

13. The system according to claim 1, wherein the at least one client device (C.sub.1, C.sub.2) is configured to affect the playback of the multi-media content by fastening the playback, pausing the playback, stopping the playback, or moving to any place in a playback timeline.

14. A method for real-time playback synchronization of multi-media content, the method comprising the steps of: at a host device (H): generating an invitation (I), the invitation comprising information identifying the multi-media content for playback and a play time stamp (PT) in accordance with the playback of the multi-media content by the host device (H), the play time stamp (PT) defining a position of the multi-media content with respect to a reference time, the reference time being a system time (ST) serving as a reference for playback in the system, the play time stamp (PT) specifying a data package of the multi-media content, and the information identifying the multi-media content specifying a content specific pattern of sizes of data packages in predefined successive time intervals of a file of the content, and a first data package defining a starting point of the multi-media content; and transmitting the invitation (I) and the play time stamp (PT) to at least one client device (C.sub.1, C.sub.2); at a server device (S): generating the system time (ST); and distributing the system time (ST) to the at least one client device (C.sub.1, C.sub.2) and/or the host device (H); at the at least one client device (C.sub.1, C.sub.2): receiving the invitation (I) and the play time stamp (PT) from the host device (H); receiving the system time (ST) from the server device (S); matching the content specific pattern specified in the information identifying the multi-media content, which is included in the invitation (I), with patterns of multi-media content stored in a database to identify the multi-media content to be played back; and playing the identified multi-media content synchronously with the host device (H) from the position defined by the play time stamp when the play time stamp (PT) matches the system time (ST).

15. A computer program for real time synchronization of multi-media content, when executed on a host device (H), at least one client device (C.sub.1, C.sub.2), and a server device (S), performing the steps of: at the host device (H): generating an invitation (I), the invitation comprising information identifying the multi-media content for playback and a play time stamp (PT) in accordance with the playback of the multi-media content by the host device (H), the play time stamp (PT) defining a position of the multi-media content with respect to a reference time, the reference time being a system time (ST) serving as a reference for playback in the system, the play time stamp (PT) specifying a data package of the multi-media content, and the information identifying the multi-media content specifying a content specific pattern of sizes of data packages in predefined successive time intervals of a file of the content, and a first data package defining a starting point of the multi-media content; and transmitting the invitation (I) and the play time stamp (PT) to the at least one client device (C.sub.1, C.sub.2); at the server device (S): generating the system time (ST); and distributing the system time (ST) to the at least one client device (C.sub.1, C.sub.2) and/or the host device (H); and at the at least one client device (C.sub.1, C.sub.2): receiving the invitation (I) and the play time stamp (PT) from the host device (H); receiving the system time (ST) from the server device (S); matching the content specific pattern specified in the information identifying the multi-media content, which is included in the invitation (I), with patterns of multi-media content stored in a database to identify the multi-media content to be played back; and playing the identified multi-media content synchronously with the host device (H) from the position defined by the play time stamp when the play time stamp (PT) matches the system time (ST).

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) In the following drawings:

(2) FIG. 1 schematically illustrates a system for real time synchronization according to one embodiment.

(3) FIG. 2 schematically illustrates synchronous playback at the host and the client devices.

(4) FIG. 3 schematically illustrates file identification via file matching.

(5) FIG. 4 schematically illustrates a transformation of the file matching system into a hash pattern.

(6) FIG. 5 schematically illustrates a system for real time synchronization according to a second embodiment.

(7) FIG. 6 schematically illustrates a system for real time synchronization according to a third embodiment.

DETAILED DESCRIPTION OF EMBODIMENTS

(8) The illustration in the drawings is schematical. In different drawings similar or identical elements are provided with the same reference numerals.

Embodiment 1

(9) FIG. 1 illustrates an exemplary system for real time synchronization according to one embodiment. The system comprises at least three devices. A first one of the devices may be a host device (H), a second one or more of the devices may be client devices (C.sub.1, C.sub.2 . . . ) and a third one of the devices may be a server device (S). All devices in the meaning of this application may be equal. A device may means for transmission and reception and storage of multi-media content. Devices in the meaning of this application may for example be mobile phones, personal computers or tablets. Means for playback is favourable but not an essentiality as for example device that may act as a server device does not necessarily need to play the multi-media content.

(10) The devices may communicate and signals may be transmitted by any kind of wireless or cable bound telecommunication such as internet, cellular or satellite networks, Bluetooth, NFC, Radio. Also, it is to be understood that when several signals are sent from one device, those signals may be send combined or individually.

(11) A user who wants enjoy playback of multi-media content e.g. a song or a music video, may also want to share emotions with other users such that they experience emotions simultaneously. While playing the multi-media content using a device that may act as a host device (H), the user may invite other users with client devices (C.sub.1, C.sub.2 . . . ) to enjoy the multi-media content simultaneously. Thus, the host device (H) of the user may generate an invitation (I) to invite other users.

(12) The invitation (I) may comprise information specifying the multi-media content. Those might for example be meta-data such as a title, an artist or a predetermined indication number. Furthermore, the invitation may contain information on the file structure of the multi-media content that allow file matching procedure as will be described below.

(13) The host device (H) may further generate a play time stamp in accordance with the playback of multi-media content by the host device. The play time stamp may be any kind of time stamp that is assigned to a time interval or data packages of the multi-media content as for example a presentation time stamp (PTS) and may refer to a reference time (e.g. a system time (ST) or a host time (HT)).

(14) Hence, the play time stamp (PT) defines the playback time of the multi-media content with respect to the reference time.

(15) The host device (H) may send the invitation (I) along with the play time stamp (PT) to the one or more client devices (C.sub.1, C.sub.2 . . . ) and along with the host time (HT) to the server device (S). Upon receipt of the host time (HT), the server device (S) may generate a system time (ST) on the basis of the host time and distribute the system time (ST) to the at least one client device (C.sub.1, C.sub.2 . . . ) and optionally to the host device (H).

(16) Having received the invitation (I) and the play time stamp (PL), the one or more client devices (C.sub.1, C.sub.2 . . . ) gain access (e.g. download, copy, stream) to the multi-media content on the basis of the information sent via the invitation (I). This may be done via matching the meta-data with those of multi-media content stored in a data base or via matching the content specific file structure of the multi-media content via file matching (see below). Gaining access to the multi-media content may be independent from the host device (H), i.e., the multi-media content itself does not need to be transferred to the one or more client devices (C.sub.1, C.sub.2 . . . ), which reduces the transmitted data volume and response waiting time.

(17) Additionally, but not necessarily, the one or more client devices (C.sub.1, C.sub.2 . . . ), upon acceptance of the invitation (I) and the play time stamp (PL), may also send a timing request (RQ) to the server device that may distribute the system time (ST) to the one ore more client devices (C.sub.1, C.sub.2 . . . ) upon receipt of the timing request (RQ).

(18) Optionally, the one or more client devices (C.sub.1, C.sub.2 . . . ) or the server device (H) may calculate a individual time latency into the system time (ST). The time latency may be measured via a ping signal measuring a time delay of the signal transmission. Furthermore, also the availability of the one or more client devices (C.sub.1, C.sub.2 . . . ) and/or the server device (H) may be verified via the ping signal. The ping signal is sent or received from each of the one or more client devices (C.sub.1, C.sub.2 . . . ) and/or from the server device (S) and/or the host device (H).

(19) Having received the system time (ST) and the play time stamp (PL), the one or more client devices (C.sub.1, C.sub.2 . . . ) may start playing the multi-media content staring from the respective data time interval or data packages that is specified by the play time stamp, when the play time stamp matches corresponding reference time. This is schematically illustrated in FIG. 2.

(20) The play time stamp (PT) may be continuously synchronized among the host (H) and the one ore more client devices (C.sub.1, C.sub.2 . . . ). The host device (H) may serve as a controller of the playback by the client devices (C.sub.1, C.sub.2 . . . ). If, for example, at the host device (H) playback is paused, fastened or repeated the play time stamp (PT) will be modified with respect to the system time. As the play-time stamps (PT) at the one or more client devices (C.sub.1, C.sub.2 . . . ) may likewise be synchronized the playback of the multi-media content follows the playback at the host device.

(21) In this way, the same multi-media content is played synchronously at the host (HT) and the client device (C.sub.1, C.sub.2 . . . ). Thus, a plurality of users may enjoy the same content at the same time and share the same emotions.

(22) It is to be understood that there may be no difference between the client devices (C) and the host device (H). Thus, each client device (C) may serve as a host device (H) and each host device (H) may serve as a client device (C). Thus, also a client device (C) may affect the playback at the host device (H) and/or others of the one or more client devices (C.sub.1, C.sub.2 . . . ), e.g. fasten, pause or stop the playback.

(23) File Matching

(24) In order to reliably identify multi-media content files, an approach is provided that compares data packages in a defined time and size scope. This approach may be called in this context the file matching approach and is illustrated in FIGS. 3 and 4.

(25) The position of single data packages (1) within a time and size grid generates a specific pattern, exemplary shown in FIG. 3. The pattern may be divided into several time windows which are designated as fingerprints. A first fingerprint (A) is schematically indicated. A fingerprint may comprise a time resolution with a widow (T1) of about 2 ms and comprise several data-packages (1) such as e.g. a frame. The order of the data packages and, hence, the fingerprints on a time scale is characteristic to the multi-media content.

(26) The first fingerprint (A) on the timeline of the multi-media content defines a starting point. By scanning the multi-media content files for such fingerprint, the same file may be found in the data base. A matching o the data structure to the first fingerprint (A) is an indication for the identical file. Wherein several fingerprints and a related countercheck leads to a more accurate result. Hereby, there the interval (T2) between the individual fingerprints is another identifier for identical files. Means not only the fingerprints may match but also the order and arrangement of the fingerprints in time.

(27) In practice as shown in FIG. 4, a first file may be scanned for its first significant pattern and the scanned fingerprints may be transformed into a hash pattern. Scanning a first fingerprint may be done by scanning a time window of e.g. 2 ms for the size of the data packages in this time window. The data packages at a specific time may then be transformed into a hash pattern. In the following a second fingerprint in a preceding interval may be scanned in the same way and my form a second hash pattern. The same may be done equally with a second or more files.

(28) If the hash patterns for the first fingerprint, the second fingerprint or even more fingerprints as well as their spacing matches with the fingerprints and the spacing of a second file a match is found.

(29) The time interval between the fingerprints can be set in diverse ways, e.g. in equal intervals, randomly over the timeline of the data structure or related to somehow significant patterns.

(30) This could lead to a subordinate hash code (fingerprints and their related interval) and a subordinate hash code describing the specific pattern of a fingerprint. A merging of both hash codes may deliver a final hash code.

(31) Those hash codes may be further be stored in a data base.

Embodiment 2

(32) FIG. 5 illustrates an exemplary system for real time synchronization according to a second embodiment. The second embodiment is in substances equal to the first embodiment, the distinguishing features will be described herein below.

(33) The system of the second embodiment comprises at least three devices. A first one of the devices may be a host device (H), a second one or more of the devices may be client devices (C.sub.1, C.sub.2 . . . ) and a third one of the devices may be a host device (H).

(34) The host device may generate the invitation (I) and the play time stamp (PT). The parameters are same to the first embodiment. However, in the second embodiment, the host device (H) sends the invitation (I) along with the play time stamp (PT) and the host time (H) to the server device (S).

(35) The server device generates a system time (ST) on the basis of the host time (HT). The server device (S) further sends the invitation (I) along with play time stamp (PT) and the system time (ST) to the at least one client device (C.sub.1, C.sub.2 . . . ). The server device (S) may optionally send the system time (ST) to the host device (HT).

(36) Upon receipt of the invitation (I), the system time (ST) and the play time stamp (PT), the at least one client device (C.sub.1, C.sub.2 . . . ) may play the multi-media content synchronous to the playback at the host device (H).

(37) Furthermore, the server device (H) may comprise means to store the play time stamp (PT), the system time (ST), the host time (HT) and the invitation (I). The play time stamp (PT), the system time (ST), the host time (HT) and the invitation (I) may be continuously synchronized between the host device (H), the server device (S) and the at least one client device (C.sub.1, C.sub.2 . . . ).

Embodiment 3

(38) FIG. 6 illustrates an exemplary system for real time synchronization according to a second embodiment. The second embodiment is in substances equal to the first embodiment, the distinguishing features will be described herein below:

(39) The server device (S) generates the system time (ST) and distributes the system time (ST) to host device (H). Upon receipt of the system time (ST), the host device (H) generates the play time stamp (PT) on the basis of the system time (ST). The host device (H) further generates the invitation (I) and sends the invitation (I) along with the play time stamp (PT) to the server device (S).

(40) Upon receipt of the invitation (I) and the play time stamp (PT), the server device (S) may distribute the invitation (I) along with the play time stamp (PT) and the system time (ST) to the at least one client devices (C.sub.1, C.sub.2 . . . ). The at least one client device (C.sub.1, C.sub.2 . . . ) may play the multi-media content synchronous to the playback host device as described in the first embodiment.

(41) In further embodiments it may be understood that all kinds of constellations of transmitting and sharing the play time stamp (PT), the invitation (I) and the system time (St) to and among the devices may be possible, such as transmitting the invitation (I) to the one ore more client devices (C.sub.1, C.sub.2 . . . ) directly and sending the play time stamp (PT) and the system time (ST) via the server device (S).