DATA TRANSMISSION METHOD

Abstract

Method for data transmission, in which an audiovisual media stream of a live (e.g., sporting) event, is transmitted by a first provider via the Internet and/or via at least one mobile radio network to a plurality of user terminals, and a data stream generated on the basis of the live event is made available by a second provider. The audiovisual media stream and the data stream are fed to a data preparation device, which generates a sequence of temporally spaced data packets and provides each data packet with identification information identifying the respective data packet and a time stamp. The generated data packets are synchronized with the media stream at controllable time intervals by the time stamp and sent via a bidirectional communication link by real-time transfer protocols and real-time streaming protocols to a group of user terminals for simultaneous playback synchronized with the audiovisual media stream.

Claims

1. A method for data transmission, in which an audiovisual media stream of a live event, for example a sports event, is sent by a first provider via the Internet and/or via at least one mobile radio network to a plurality of user terminals, and a data stream generated on the basis of the live event is made available by a second provider for further processing or display via the Internet and/or via at least one mobile radio network, wherein the audiovisual media stream and the data stream are fed to a data preparation device, which generates from the data stream a sequence of temporally spaced data packets and provides each data packet with identification information identifying the respective data packet and a time stamp, wherein the generated data packets are synchronized with the media stream at controllable time intervals using the time stamp and sent via a bidirectional communication link using real-time transfer protocols to a group of user terminals for simultaneous playback synchronized with the audiovisual media stream.

2. The method according to claim 1, wherein the real-time transfer protocol is RTP (Real Time Protocol).

3. The method according to claim 1, wherein the generated data packets each contain an input prompt displayed on the user terminals, and a user-generated data packet is created from a user input and the identification information of the relevant data packet and is sent to the data preparation device via the bidirectional communication link for evaluation or forwarding to other users.

4. The method according to claim 1, wherein the data packets synchronized with the media stream with the aid of the time stamp are transmitted by the data preparation device at different bit rates via the bidirectional communication link.

5. The method according to claim 1, wherein the controllable time interval between two successive data packets of the sequence of time-spaced data packets is controlled in such a way that it satisfies the criterion ΔT>(BR/FPS)t.sub.ED, wherein BR is the bit rate in kb/s of the audiovisual media stream transmitted to the group of user terminals, FPS is the number of frames per second of the audiovisual media stream transmitted to the group of user terminals , and t.sub.ED is the latency time in seconds of the transmission of the audiovisual media stream transmitted to the group of user terminals between the data preparation device and the user terminals.

6. The method according to claim 3, wherein the evaluation of the user-generated data packet involves a check of the criterion $\frac{\left(\mathrm{qet}-\mathrm{rt}\right)⨯100}{\frac{1}{\mathrm{FPS}}⨯1000+\left(\mathrm{tBE}+\mathrm{tE}+\mathrm{tED}+\mathrm{tD}\right)}>60\%,$ wherein “qet” is a predetermined maximum time in seconds, measured from the time a particular data packet is sent by the data preparation device to the user terminals for the preparation of the user input, “rt” is the actual time in seconds until a user-generated data packet arrives at the data preparation device, “FPS” is the number of frames per second of the audiovisual media stream transmitted to the group of user terminals, “t.sub.BE” is the latency time in seconds of the transmission between the first provider and the data preparation device, “t.sub.E” is the latency time in seconds of the data processing at the data preparation device, “t.sub.ED” is the latency time in seconds of the transmission of the audiovisual media stream transmitted to the group of user terminals between the data preparation device and the user terminals, and “t.sub.D” is the latency time in seconds of data processing and display at the user terminal, and if the criterion is not met, the user-generated data packet in question is discarded.

Description

[0029] The invention will be described in more detail in the following on the basis of embodiment examples using the enclosed figures, wherein:

[0030] FIG. 1 shows a schematic representation of an embodiment of a method according to the invention, and the

[0031] FIG. 2 shows a schematic diagram of an embodiment of the data preparation device.

[0032] Firstly, reference is made to FIG. 1, which shows the basic functioning of the method according to the invention. A first provider 1 is located on site of a live event, such as a sports event, with appropriate equipment to obtain images and sound recordings and makes these images and sound recordings available as an audiovisual media stream MS via the Internet 6 and/or via at least one mobile network to a large number of user terminals 4. A second provider 2 generates a data stream DS on the basis of the live event and makes this data stream DS available for further processing or display also via the Internet 6 and/or via at least one mobile phone network. In the traditional way, the data streams DS are intended for professional users and not for the private consumer. In the case of sports events, they contain a wealth of sports data which can be used, for example, to detect any attempts at manipulation. The second provider 2 can either be on site with its own equipment for image and sound acquisition or generate the data stream DS using audiovisual data already recorded. As mentioned above, these data streams DS are highly dependent on the live event, such as the sport in question, in terms of the amount of data and the timing structure, and are subject to wide variations in terms of data size and timing structure.

[0033] In the context of the present invention, the method according to the invention makes use of the audiovisual media stream MS and the data stream DS by feeding them to a data preparation device 3 operated by an operator of the method according to the invention. The data preparation device 3 essentially carries out two steps, as explained in FIG. 2. In a first step, a sequence of time-spaced data packets DP is generated from the data stream DS at controllable intervals ΔT. The generation of the data packets DP is preferably carried out with an algorithm 8, which recognizes typical events on the basis of the data stream DS and generates information or prompts tailored to these events as interaction offers. As already mentioned, such algorithms 8 are basically known and are also used for generating the data streams DS itself. For example, they are able to automatically recognize typical events from the audio and video material, such as scoring a goal or a penalty kick in soccer. Similar algorithms 8 are used according to the invention to generate the data packets DP, for example by adding the prompt “Will team X equalize?” or “Will a goal be scored from the penalty kick?” as a ready-to-send data packet DP. The content of the data packets DP is in principle freely selectable. Therefore, it may concern additional information or comments on the transmitted contents of the media stream MS. Each data packet DP is subsequently provided with identification information identifying the respective data packet and a time stamp in a data header, which in a conventional manner contains the control information for the transmission of the data packet DP via the Internet.

[0034] In a second step, the generated data packets DP are synchronized with the media stream MS at controllable intervals ΔT. The controllable time interval ΔT between two consecutive data packets DP of the sequence of time-spaced. data packets DP is selected approximately 80 that it meets the criterion

ΔT>(BR/FPS)t.sub.ed,

wherein BR is the bit rate in kb/s (kilobits per second) of the audiovisual media stream MS transmitted to the group of user terminals 4, FPS is the number of frames per second of the audiovisual media stream MS transmitted to the group of user terminals 4, and t.sub.ed is the latency time in seconds of the transmission of the audiovisual media stream MS transmitted to the group of user terminals 4 between the data processing device 3 and the user terminals 4. This criterion provides a lower limit for the selectable time intervals ΔT, which is determined by the connection quality of the transmission between the data processing unit 3 and the user terminals 4. A poorer connection quality generally requires longer intervals ΔT than a relatively good connection quality. Of course, above these technical limits, the time interval ΔT can be chosen at will.

[0035] The synchronization is carried out using the time, stamps of the data packets DP and the time stamps of the audiovisual media stream MS, usually already provided by the first provider 1, usually in the data header of the so-called “frames” used to transmit a media stream. The result is an audiovisual media stream MS, which was synchronized with data packets DP with controllable time intervals ΔT and is henceforth and in FIGS. 1 and 2 also referred to as synchronized media data stream MD. It is advantageous if the synchronized media data stream MD is provided and transmitted by the data preparation unit 3 with different hit rates. The synchronized media data stream MD is thus available to user terminal 4 at different bit rates, so that user terminal 4 can select a suitable hit rate depending on the performance of the terminal and reception quality to optimize playback quality.

[0036] Reference is now made again to FIG. 1. The synchronized media data stream MD is transmitted via a bidirectional communication link 7 using real-time transfer protocols to a group of user terminals 4. An example of a real-time transfer protocol is currently RTP (Real Tine Protocol). RTP can be used with UDP and IP as IP/UDP/RTP. The practical implementation can be carried out with the help of WebRTC (Web Real-Time Communication), for example. The bidirectional communication link 7 can be implemented, for example, via so-called WebSockets, wherein the transmission of audiovisual data is usually carried out using a content delivery network 5 (CDN or Content Distribution Network). A content delivery network 5 is a network of regionally distributed servers connected via the Internet, with which content, usually large media files, are delivered. Individual locations are also known as PoP (Point of Presence) and consist of server clusters.

[0037] The group of user terminals 4 is preferably defined by prior registration of the users with the operator of the method according to the invention. In this process, an interested user logs on to the operator of the method according to the invention by disclosing his connection data and optionally also by disclosing a specific group as a member of which he wishes to participate, for example by means of a web-based application, or by prior installation of a corresponding software application (“app”) on his terminal device. For example, using a WebSocket protocol, the user initiates a request to transfer data, wherein after the initial data for establishing the connection has been transferred, the underlying TCP connection remains in place and allows asynchronous transfers in both directions. The respective user then receives the data packets DP synchronized with the media stream MS as a synchronized media data stream MD. Since the synchronized media data stream MD is available to user terminal 4 at different bit rates, user terminal 4 can subsequently select a suitable bit rate to optimize playback quality depending on the performance of the terminal and reception quality.

[0038] At user terminal 4 it is then ensured that during the reception of the synchronized media data stream MD, a data packet. DP and a frame of the media stream MS with the same time stamp are displayed simultaneously at user terminal 4. For example, the playback can be done by using a split screen, as indicated by the dotted lines in FIG. 1, by displaying in a first half of the screen the content of the audiovisual media stream MS, and in a second half of the screen the content of the data packet DP, for example a comment, additional information, or a prompt for the question “Will team X equalize?”.

[0039] Using a traditional input device on the user terminal 4, the user can make an input at the prompt. From the user input and the identification information of the relevant data packet DP, a user-generated data packet D is subsequently created and sent via the bidirectional communication link 7 to the data preparation unit 3 for evaluation or forwarding to other users, wherein the aforementioned content delivery network 5 can be used. The data preparation device 3 can use the relevant input, for example for competitions, or compere it with the input of other users.

[0040] The evaluation and possible forwarding of the data in data preparation device 3 is, of course, preferably automated, since in practical operation a large number of data packets DP generated by the operator and data packets D generated by the user are to be expected. It would be advantageous to have criteria that check the correct sequence of this process of sending and receiving data packets DP, i.e. plausibility criteria that make it possible to detect manipulations, for example. A simple criterion has proved to be particularly effective for this purpose, and it is therefore proposed that the evaluation of the user-generated data package D should include a review of the criterion

[00002]$\frac{\left(\mathrm{qet}-\mathrm{rt}\right)⨯100}{\frac{1}{\mathrm{FPS}}⨯1000+\left(\mathrm{tBE}{+}_{>60\%}D+\mathrm{tD}\right)}$

wherein [0041] “qet” is a predetermined maximum time in seconds, measured from the time a particular data packet DP is sent by the data preparation device 3 to the user terminals 4, for the preparation of the user input, [0042] “rt” is the actual time taken for a user-generated data packet D to arrive at the data preparation device 3 in seconds, [0043] “FPS” is the number of frames per second of the audiovisual media stream MS transmitted to the group of user terminals 4, [0044] “t.sub.BE” is the latency time in seconds of the transmission between the first provider and the data preparation device 3, [0045] “t.sub.E” is the data processing latency time in seconds for the data preparation device 3, [0046] “t.sub.ED” is the latency time in seconds of the transmission of the audiovisual media stream MS transmitted to the group of user terminals 4 between the data processing equipment 3 and the user terminals 4, and [0047] “t.sub.D” is the latency time of data processing and display at the user terminal 4 in seconds,
and if the criterion is not met, the user-generated data packet D in question is discarded.

[0048] If the ratio described above is below 60%, the user-generated data packet D in question is thus rejected, since technically hardly explainable anomalies exist and manipulations cannot be ruled out.

[0049] If no anomalies can be detected and manipulation can be ruled out, a notification of any kind is usually sent from the data preparation device 3 to the user concerned in response to the input of a user, in order to promote the community experience of the live event. This can be a notification, for example, about which or how many users in the same group have correctly answered questions that have been asked so far and include a corresponding ranking.

[0050] The method according to the invention enables the transmission of audiovisual media streams MS as well as synchronized data packets DP with a latency time of less than 300 ms. This short latency ensures that the transmitted data is reproduced on all user terminals 4 in a way that is perceptible to all users at the same time. During this simultaneous playback, synchronized playback with the data packets DP generated by the operator also takes place. This also allows users to interact based on the live event they have just experienced, both with the operator of the method according to the invention and with other users, thus intensifying the community experience of the live event being broadcast.