Synchronization of multimedia contents on second screen

Abstract

The present arrangement relates to a display a screen of a second device of an enriched content previously pushed by a computer server, in synchronism with the display of the content of a broadcast program reproduced by a first device. The method includes, the server pushing an item of information representative of the start (T.sub.REF) of the broadcasting of the program, then charged to the mobile device, via the execution of a specific software application, to begin listening to the audio band reproduced by the first device, to identify the program, by comparison with the audio signature received previously, to determine the instant (T.sub.C) of actual reproduction of the program by the first device, and to thus deduce therefrom the elapsed lag which must be taken into account so as to display the enriched content in a synchronized manner.

Claims

1. A method of automatic synchronization between the reproduction, by a first device, of a multimedia content included in a broadcast program, and the display of an enriched content, associated with the multimedia content, on a screen of a second device able to be connected to a communication network, said method comprising the steps of: automatically transmitting said enriched content to the second device by at least one computer server of said communication network, the method further comprising the steps of: prior to the reproduction of the multimedia content by the first device, during which at least one audio signature representative of the program or of the multimedia content broadcast is generated and stored in a database linked to said computer server, said audio signature is transmitted automatically for storage to the second device by said computer server, wherein in the course of the broadcasting of the program by a remote source, the broadcast program is received and reproduced by the first device: detecting, by the second device, an audio signal generated by the first device during the reproduction of the broadcast program, Identifying the instant of reproduction (t.sub.C) by comparing the audio signal detected with the audio signature stored; determining a synchronization lag at the second device after which the enriched content must be displayed on the screen of the second device, wherein said determination of said synchronization lag includes automatic transmission, by said at least one computer server destined for the second device via the communication network, an item of information representative of a reference instant (t.sub.REF) corresponding to the instant at which the program or the multimedia content is actually broadcast by the remote source; where the reception of said item of information representative of a reference instant (t.sub.REF) by the second device triggers said detection step, such that the synchronization lag is determined as a function only of the reference instant (t.sub.REF) and of the instant of reproduction (t.sub.C) identified, and wherein the method further includes displaying the enriched content on the screen of the second device in synchronization with the playing of the multimedia content.

2. The method according to claim 1, wherein said item of information representative of the reference instant (t.sub.REF) is generated by a content-manager module and transmitted to the computer server via the communication network.

3. The method according to claim 1, wherein the enriched content is generated by an enriched contents-generator module, and stored in the database before transmission to the second device.

4. The method according to claim 1, wherein the broadcast program is a radio program.

5. The method according to claim 1, wherein the broadcast program is a televised program.

6. The method according to claim 1, wherein the first device is selected from the group consisting of a television, a computer, an intelligent telephone, a games console, and any electronic device able to reproduce at least the sound of the broadcast program received.

7. The method according to claim 1, wherein the second device is selected from the group consisting of a digital tablet, an intelligent telephone, a fixed or portable computer, and any device comprising a microphone, a display screen, and means of connection to the Internet network.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The invention and the advantages that it affords will be better understood in view of the following description given with reference to the appended figures, in which:

(2) FIG. 1 schematically illustrates an exemplary communication infrastructure making it possible to implement the invention;

(3) FIG. 2 represents a series of steps that may be implemented in a synchronization method in accordance with the invention.

DETAILED DESCRIPTION

(4) In the description which follows, it is considered, by way of nonlimiting example, that the broadcast program comprising at least one multimedia content, with which it is desired to associate at least one enriched content to be displayed on a mobile device, is a televised program, consequently comprising images and sounds. As will be more apparent subsequently, the invention is also applicable in respect of audio-only programs, such as a transmission on radio waves.

(5) A first device 1 able to receive and reproduce the broadcast program has been represented with reference to FIG. 1. In the case of a televised program, the first device 1 is therefore typically a television, but can consist of any known device able to reproduce the sound and the image of the televised program respectively on loudspeakers (not represented) and a screen 10 (fixed or portable computer, intelligent telephone, games console etc).

(6) The way in which the program is actually broadcast by a remote source and then received by the first device 1 has not been represented in the figure since it does not present any particular interest, except that already indicated in the introduction of inducing a transmission lag between the instant at which the broadcasting of the program starts (reference instant t.sub.REF) and the instant at which the reproduction of this program starts on the first device 1 (instant of reproduction t.sub.C), this transmission lag being variable according to the broadcasting network used (digital terrestrial television, satellite, IP etc.).

(7) The system architecture which will allow the viewer to live a “second screen” experience essentially comprises: a second device 2 held by the viewer and integrating the second screen 20; at least one platform for push by computer servers 3; a database 4 linked to the platform; a specific software application 5 able to be loaded and executed on the second mobile device 2; and, at the level of the “back-office” part: a content-manager module 6a making it possible in particular to manage and push on the platform the item of information indicating the start of the broadcasting of the program to be listened to with the reference of the program; an enriched-contents generator module 7 connected to the database 4; an audio-signatures generator module 8 connected to the database 4.

(8) These various hardware components are able to communicate with one another via a communication network 9, typically the Internet network. The specific software application 5 comprises various parts of program code instructions executed by the second device 2 with the aim of allowing the second-screen experience.

(9) The second device 2 is for example a digital tablet, a laptop computer, an intelligent telephone, or more generally any portable or fixed device comprising a microphone, a display screen and means of connection to the communication network.

(10) A synchronization method 100 in accordance with the invention and using the system architecture defined hereinabove will now be described with reference to FIG. 2. To facilitate understanding, the hardware components of the architecture of FIG. 1 concerned in each step have been represented on the left of FIG. 2, opposite the various steps. The steps have been shown in a certain order. Nonetheless, it will be understood during the reading which follows that the order of certain steps can be modified without departing from the scope of the present invention.

(11) The synchronization method 100 comprises a series of steps (steps 110 to 140) which can occur at any moment anterior or posterior to the reference instant t.sub.REF at which the program is broadcast, but prior to the instant t.sub.C at which the multimedia content that it is desired. to enrich will actually be reproduced on the first device 1. A first step 110 comprises the generation of one or more audio signatures representative of the broadcast program concerned. This step 110 consists in producing a signature characteristic of the sound, sufficiently compressed to be transportable easily and rapidly (weight of the signature), but sufficiently detailed to define in a one-to-one manner the sensed sound. On a sound lasting several hours, it is samples rather than the entirety of the sound that are processed. The generation of the signature (or ADN) of the sound uses Fourier transform processing as its basic principle.

(12) These signatures are generated by the audio-signatures generator module 8.

(13) Moreover, the enriched content that it is desired to associate with the program. for display on the second screen 20 is generated by the contents-manager module 7.

(14) The enriched content and the audio signatures are thereafter transmitted to the database 4 for storage (step 120). it should be noted that, although the generation of the enriched content and of the audio signature has been represented as carried out in a single step 110, these operations are independent. and can be carried out in a non-concomitant manner. The same goes for the transmission step 120.

(15) The audio signature is thereafter transmitted by the computer server 3 to the second device 2 during step 130. This transmission is preferably effected according to the “push” technique. FIG. 1 illustrates by slender and continuous lines the route of an audio signature from the audio-signature generator module 8 to the second device 2.

(16) Moreover, the computer server 3 transmits the enriched content to the second device 2. This step, reference 140 in FIG. 2, is here considered. to be posterior to the transmission of the audio signature, but may in fact take place at any moment before the instant t.sub.C. FIG. 1 illustrates by thick and continuous lines the route of an enriched content from the enriched-contents generator module 7 to the second device 2.

(17) Subsequently, in the course of the broadcasting of the program by the remote source, the broadcast program will be received and reproduced by the first device 1. In particular, the sound band associated with this program will be played by the loudspeakers of this device 1. During a step 160, the server 3 will automatically transmit an item of information representative of the reference instant t.sub.REF to the second device 2 Accordingly, the module 6a is able to listen to all the sources that are broadcasting programs, and so recognize the program by identifying its previously stored audio signature (step 150). As a variant, a human operator 6b (see FIG. 1) can originate the transmission of this item of information. FIG. 1 illustrates by broken lines the route of an item of information representative of the reference instant t.sub.REF from the content-manager module 6 to the second device 2. In both cases, the item of information transmitted furthermore advantageously comprises an identifier associated with the audio signature, a table of program start and end times, as well as the enriched content or by default, a link to this enriched content that it will be appropriate to display on the second screen.

(18) On receipt of this item of information, the software application 5 executed by the second device 2 will control the latter in such a way that it listens, via its integrated microphone, to the audio signal generated by the first device 1 during the reproduction of the broadcast program. The second device 2 will thus, by comparison with what it listens to and the audio signature previously received (step 130), identify the program broadcast and the instant of reproduction t.sub.C on the first device (step 170).

(19) On the basis of the knowledge of these two values t.sub.REF and t.sub.C, the second device 2 is then able to determine the duration which has elapsed between the instant of start of broadcasting of the program and the instant of actual reception by the first device, which corresponds to the lag Δ that it must apply before commanding the display on its screen 20 of the enriched content which is pushed to it so as to be synchronous with the first screen (steps 180 and 190).

(20) Although the method compliant with the invention has been described only with reference to a “second screen” experience with respect to a televised program, it is understood that the invention finds applications provided that the broadcast program is audio. Consequently, enriched content can be provided as a supplement to any audiovisual program (television, cinema, Internet etc.), or to any audio program (radio waves, telephone, etc.).

(21) Moreover, it should be noted that the platform 3 can be composed of several servers hosted in a computing cloud. Likewise, several databases 4 can be used to store the enriched contents and/or the audio signatures.

(22) The solution proposed by the invention is at least as advantageous as the known technique of “Fingerprinting” with respect to “Watermarking”, in particular because no signal need be injected into the broadcast signal, and because the solution offers a good bandwidth.

(23) The solution of the invention is nonetheless optimized with respect to “Fingerprinting” in terms of processing time: it does not require any large quantities of servers, bandwidth, or connections to the television channel broadcasting networks (in this precise case) which must be monitored to effect all-azimuth recognition of sounds. Moreover, the consumption of the batteries of all the second devices is reduced since it is the information representative of T.sub.REF which informs the second screen to begin and to finish listening (use of the microphone and of the audio circuits) instead of listening continuously.

(24) In the foregoing, it has been assumed that the reference instant t.sub.REF pushed forward by the computer server corresponded to the instant at which the program was actually broadcast by the remote source. Of course, without departing from the scope of the present invention, this reference instant can also correspond to the instant at which the multimedia content to be enriched is actually broadcast by the remote source, thereby making it possible to reduce still more the duration for which the microphone is listening and right from the first occurrence of expected audio signature to establish the synchronization between the first and the second screen. In particular, if it is desired to enrich three multimedia contents, typically three commercial breaks, included in one and the same program, the computer server will be able to push forward three reference instants making it possible each time to calculate the synchronization lag on the basis of which the enriched content must be displayed. As a supplement to the reference instant associated with the broadcasting of each multimedia content to be enriched, provision may be made to also dispatch an end instant so as to define a time interval during which the microphone will be active.

(25) Finally, as the enriched content has been transmitted to the second device even before the instant T.sub.C at which the multimedia content that it is desired to enrich is actually reproduced by the first device, this enriched content is displayed instantaneously perfectly synchronized with the first screen, on the second screen.