Method for streaming and reproducing applications via a particular telecommunications system, telecommunications network for carrying out the method, and use of a telecommunications network of this type

Abstract

The invention relates to a method for streaming and reproducing applications (apps) via a particular telecommunications system. The invention also relates to a telecommunications network. The invention further relates to the use of a telecommunications network of this type for streaming and reproducing applications (apps) via a particular telecommunications system. The invention furthermore relates to an application which makes it possible to configure the download, installation and updating of data on a terminal such that the whole application is fully executable on the terminal while downloading, installing and updating takes place on the terminal.

Claims

1. A method for streaming and reproducing an application via a telecommunications system, the method comprising: executing the application by at least one streaming server on the at least one streaming server, wherein the at least one streaming server being communicatively connected to a terminal via a connection, the connection being characterized by at least one parameter selected from the group consisting of a number of hops, latency, and bandwidth; wherein the terminal is configured to retrieve the application from the at least one streaming server, wherein the at least one streaming server is configured to provide processing power for setting up a video stream on the terminal and for encoding of the application for reproducing the application on a non-native environment for the application on the terminal; performing, by the at least one streaming server, a transmission of the application from the at least one server to the terminal; decoding, by the terminal, packet-based data representative of the application; recording an input to the application at the terminal in response to the packet-based data being decoded by the terminal; transmitting, by the terminal, the input to the at least one streaming server; registering the input at the at least one streaming server, the input being configured to alter an execution of the application at the at least one streaming server; invoking, at the terminal, a recovery process in response to packet loss having occurred during the transmission to the terminal, the recovery process including: selecting a recovery strategy; sending a request to the at least one streaming server to initialize the recovery strategy; and wherein a source code associated with the method being executed by the telecommunications system includes the following modules: a port asynchronization module including a submodule for sending a broadcast associated with an execution of the port asynchronization module; and a universal port mapper module including a plurality of submodules, the plurality of submodules including: a search submodule configured to search for external addresses and return one or more identifiers associated with any found external address; a router port submodule configured to open a port associated with devices having the one or more identifiers.

2. The method of claim 1, further including streaming the application on the terminal in real time while the application is being downloaded, installed, or updated and closing the stream when files for the application are made available in full on the terminal.

3. The method of claim 1, further including monitoring a streaming of the application via an additional telecommunication test device.

4. The method of claim 3, further including evaluating a quality of the streaming based on a test parameter selected from the group consisting of bandwidth, latency, hops, and packet loss.

5. The method of claim 1, wherein the at least one streaming server is connected to the terminal via a wireless area network (WAN).

6. The method of claim 1, further including effecting the transmission from the at least one streaming server to the terminal via a modified h.264 protocol.

7. The method of claim 1, wherein the method further includes playing back the application via streaming to the terminal independently of a downloading of the application on the terminal.

Description

(1) FIG. 1 a block diagram with a schematic representation of the relationships between the different areas and the streaming server;

(2) FIG. 2 a block diagram of the game package module;

(3) FIG. 3 a block diagram of the session management server;

(4) FIG. 4 a block diagram of the Mobile—Interactive Layer for the client;

(5) FIG. 5 a block diagram with a flow chart of the recovery module of the client;

(6) FIG. 6 Mobile Interactive Layer—exemplary illustration of the interface of a mobile terminal;

(7) FIG. 7 Recovery Strategy Process in the event of loss of a data packet and

(8) FIG. 8 an overview of the telecommunication element.

(9) FIG. 1 shows the individual elements that are necessary in the communication. Thus the streaming server 120 handles the initialization of the application and launches it in a virtual environment. For this purpose, the streaming server 120 has a Game Isolation Module 140. In this an application-friendly environment is launched which, on the one hand, guarantees the ability to execute the application, but is also responsible for the reproduction of the control signals from the client 110A. The streaming server can launch any number of instances of the same or different applications. The limiting factor in this relationship is the computing power of the GPU in graphics applications. Each application that is launched is assigned a game DB 180. This Game DB 180 is responsible for storing relevant data for the application. In order to launch an application, however, this must first be available to the game package manager 180 as a game package 170. The network module 150 of the streaming server 120 then handles the encoding and packaging of the frames. A further task of the network module 150 is the handling of recovery requests of the client 110A. To perform administrative interventions and evaluations, the Evaluator Module 190 has been developed. This module is responsible for the generation of statistics.

(10) The client, as a thin client, is used for the transmission of audio/video signals and can typically be deployed on any platform. A streaming server 120 can be in a relationship of 1:n, however, a client can only receive the communication to a particular streaming server 120. Typically, the number of clients per streaming server is not limited by the software but by the corresponding hardware capacities of the GPU of the streaming server 120.

(11) Communication between the streaming server 120 and the client 110A is always initially established via the session management server 130. This performs the initial request of the client 110A to connect to the streaming server and searches for the optimal streaming server 120 for the client 110A. One system can have multiple streaming servers operating in parallel. They also do not always need to be in the same data center or country. After the assignment of a streaming server 120 by the session management server 130 for the client 110A, the streaming server 120 handles the direct communication with the client 110A.

(12) A further element is the content server 195. This server is responsible for the delivery of certain parts in the interactive layer of the client 110A. Among other things, this controls the display of advertising according to the application that is displayed on the thin client. The necessary information is provided to the content server 195 via the session management server 30.

(13) The communication takes place primarily via the Wide Area Network (WAN) 115. This includes various modes of transmission and is not limited to certain sections.

(14) FIG. 2 shows the game package Module 160, which is part of the streaming server 120. The game package module 160 is started initially for each new application and handles six sub-areas for the application. Capture Encode Audio 210 is divided into the sections Capture 210A and Encode 210B, and responsible for sampling the audio signal. The Capture Encode Video section 220 is divided into the same sections as the audio module 210.

(15) The Port Authentication Module 230 handles the port authentication and corresponds to the provision of the connection between game stream server 120 and the client 110A. The Control Relay 240 is responsible for XXX. The task of the network relay 250 is to send off the corresponding packages and to manage incoming packets. The recovery module 260 is responsible for answering the corresponding recovery requests of the client 110A.

(16) FIG. 3 deals with the session management server 130. This has the task of authentication 310 and using a downstream DB module 315, its task is to save or store the data used for authentication. This DB module 315 is only optional, however. The option of an external authentication remains unaffected. The network section 320 is responsible for the communication between the WAN 115, streaming server 120, content server 195 and the corresponding clients. The session manager 330 is then authoritatively responsible for managing the individual sessions and handles the assignment of the clients to a corresponding streaming server. The Evaluator module has a direct connection to the individual clients and gathers relevant data for a subsequent central evaluation.

(17) FIG. 4 shows the individual elements of the client. The entire client 110 has been specially developed for the application and does not require separate software. It consists of eight sections to be described as follows.

(18) Client Session Manager 410, communicates with the streaming server 120 and the session management server and is initially responsible for the authentication and management of the client.

(19) Network module 420 is responsible for configuring and maintaining the connection. This module also handles the sending and receiving of various packets.

(20) The Controller 430 handles the dispatch of the delivered frames and audio packets as a visual image in the client.

(21) Decode Render Video 440 and Decode Render Audio 450 receive the packets that were previously received from the network module 420 and were forwarded by the controller 430.

(22) The elevator module 460 is responsible for collecting statistical data and transmits this to the session management server. Accordingly, the session management server can also optimize the connection. This results in a feedback, which makes this module very important.

(23) The recovery module 470 evaluates incoming data packets. If a data packet turns out to have errors, the module selects a recovery strategy and, if necessary, requests a new packet from the streaming server or takes other measures to compensate for the loss, without suffering a loss in terms of latency or quality.

(24) The client UI includes the interactive layer and content of the content server 195. This is where the input of the user is captured and sent off to the streaming server 120.

(25) FIG. 5 shows the structure of the content server. This is responsible for the content administration 510 and content streaming 520.

(26) The Content Administration is used for presetting e.g. the advertising to be displayed within the interactive layer in the client 110. The Content Administration 510 is designed to specify both the frequency as well as the content.

(27) The Content Streaming module 520 handles the playback of the content and is used as a central interface for all clients.

(28) FIG. 6 shows the interactive layer 600, which is part of the client UI 480. In principle, between three different sections can be differentiated.

(29) The application layer 610 reproduces the received frames and is responsible for the visual representation of the application.

(30) On top of the application layer 610 is located the UI layer 620. This layer can be configured individually, but in principle, it is definitively responsible for the input by the user in the client.

(31) In addition to the above-mentioned layers, there is the option to import content from the content server 195. This takes place in the area of the Content Layer 630.

(32) FIG. 7 shows the sequence of the recovery strategy of the client 110 in the module 470. As soon as a “package loss” has been detected 710 on the part of the client, the recovery module will select an corresponding solution on the basis of fixed criteria 720.

(33) Once the decision is made, whether Blocking 730, Not Blocking 740, Intrarefresh 750 or Frame Validation 760 has been selected, the recovery request 770 is sent to the streaming server 120. The streaming server accordingly sends a new packet and the task of the recovery module 470 has been completed.

(34) FIG. 8 shows the telecommunications element 800 and its functions schematically. Each client can be assigned such a connection element, which for reasons of clarity is not shown in the drawing. Each telecommunications element 800 is therefore in a position to perform a streaming process and a gaming install process.

(35) During the streaming process, a connecting device 900 designed as a connection tester first tests the software to be loaded of the application for the relevant streaming client, for example 110A, while at the same time, in parallel with this, in our case a game is in progress via a gaming install-process. It can also, of course, be a different application. A download device designed as a downloader 1000 downloads the software onto the terminal device, while an installation device 2000 designed as an installer performs the installation at the same time, and an updating device designed as an updater 3000 always ensures that the latest data is available, thus performs a constant update process.

(36) The device apparent from FIG. 8 which is designed as a telecommunications element 800 therefore enables the display of applications before the actual download of data, its installation and updating takes place. The telecommunications element 800 therefore allows, so to speak, a small application (Thin Client) on the terminal. This launches the appropriate processes and enables the streaming of the application while the application is installed.

(37) The telecommunications element 800 thus allows two separate processes, namely the streaming process and the user install process, for example a gaming install process.

(38) The streaming process in the telecommunications element 800 consists of two modules:

(39) Connection tester: responsible for the testing of the connection on the basis of three criteria: Latency, bandwidth and network. In case the connection does not meet the requirements, the streaming client is not started. The threshold values for the three criteria can be flexibly adjusted and tailored to the application.

(40) Streaming client: If the connection tester proves successful, then the streaming client will be started.

(41) By using the streaming technology this streams the application directly into a local window of the terminal, for example into the PC window of a player.

(42) During installation this process handles a plurality of tasks and processes them sequentially. This process is responsible for putting the application locally into an executable state, while it is being streamed. All steps are performed in the background of the system and do not require any input by the player. The streaming process performs the following steps:

(43) Download: The application is downloaded onto the local computer in parallel with the stream.

(44) Installation: The process installs the application in the background. Any required third-party software is also installed.

(45) Updating: The application is updated.

(46) Once these steps are completed, the user can run the application locally after restarting the telecommunications element 800.

(47) The features of the invention disclosed in the description, the claims and those which are apparent from the drawings can be essential to the implementation of the invention both individually and in any desired combinations.

Explanation of Terms

(48) TABLE-US-00002 Application Layer Anwendungsebene Applications-Code Anwendungscode buffering Pufferung Client Ein Client (deutsch “Kunde”, auch clientseitige Anwendung oder Clientanwendung) bezeichnet A client (German “Kunde”, also client-side application or client application) refers to a computer program, which is executed on a terminal of a network and communicates with a central server. Client UI Module Client Benutzeroberfläche Modul Client-Application Client-Anwendung Client-Session Client-Sitzung Cloud Complex of multiple servers in the Internet codec coder-decoder Content Layers Inhaltsebene Content Servers Inhalt-Server Content Streaming Inhalt Streaming Controller Regler Controller Module Regler-Modul decode thread decoding application Decoder-Render Dekodierungs-Ausfuhrer Evaluator evaluation unit Evaluator Modul Evaluierungsmodul Frame validation Frame Überprüfer interactive layer Interaktiv-Ebene Intrarefresh internal refresh/updating Loadbalancing Lastenverteilung Network Module Netzwerk-Modul Not Blocking Nicht-Blockieren Overlay Überlagerung packaging Paketierung Recovery Module Wiederherstellungs-Modul Recovery Strategies Wiederherstellungs-Strategie Render Thread is responsible for the rendering [display] of the application. Scripting script language session handler Sitzungs-Manager shared library Geteilte Bibliotheken Streaming Streaming Streaming-Server Streaming-Server Streaming-Technology Streaming-Technologie Timestamping Describes the allocation of a piece of data to a data packet UDP User Datagram Protocol UI Layer Benutzeroberfläche-Ebene WAN Wide Area Network Web-Apps Web-Applikationen Webcontent Management Web-Inhalt Management

LIST OF CITATIONS

(49) WO 2009/073830 A1 WO 2010/141522 A1 WO 2012/037170 A1 US 2014/0073428 A1 WO 2009/073830 A1 WO 2010/141522 A1 US 2015/018094 A1 WO 2014/186858 A1 Anonymous: “Cloudgaming—Wikipedia, the free encyclopedia”, 24. Jun. 2015 (2015-06-24), XP055268633, found on internet: URL:https://en.wikipedia.orq/w/index.php?title=Cloudqam inq&oldid=668477716 [accessed on 2016-04-26] Section “Types”, 2nd paragraph; page 1