SYSTEM FOR INGESTING ALTERNATIVE VIDEO AND/OR AUDIO MEDIA SYNCHRONIZED WITH THE DISPLAY OF VIDEO AND AUDIO CONTENT OF LINEAR ORIGIN, AND RELATED IMPLEMENTATION METHOD

Abstract

The present invention relates to a system installed in the digital TV receiver that provides the broadcaster with a secure method for ingesting alternative video and/or audio media, in place of passages of main video and/or audio, with a view to customizing the experience of the consumer identified during the digital video and/or audio consumer experience. The system is intended to provide the consumer with the best possible experience without any interruptions, loss of fluidity or loss of consistency in the playback of the alternative video and/or audio. The system provides a method for continuously verifying suitable conditions for playing back alternative video and audio.

Claims

1. A system for ingesting alternative video or audio media, synchronized with the display of video or audio content of linear origin and characteristic, the system comprising: a smart receiver connected to a conventional television broadcast network and a broadband network enabling access to the Internet, said smart receiver supporting the reproduction of conventional broadcast television content through its connection to the television broadcast network and, through its connection to the broadband network, also supporting the delivery of on-demand online content such as streaming services, Over-The-Top (OTT) content or video on-demand (VOD); an Over-The-Top Application (OTT Application), configured to receive the alternative video and audio (AVA) contents, through the connection to the broadband network; and a Content Replacement and Injection Module configured to receive the content through the connection to the television broadcast network, all MVA linearly (or live) broadcasted, said Module comprising: i. a Digital TV receiver tuner, which receives the content coming from the connection to the television broadcast network; ii. a Decision-making Center for deciding and performing a replacement of said MVA content with AVA content based on a continuous evaluation of relevant parameters for reproduction of said content; iii. a Synchronizer configured to send an alternative content injection command, and to command selection of said alternative content; iv. a streaming module, configured to receive the alternative video and audio content (AVA) from the OTT Application and to inject said content into a Content Selector; v. the Content Selector, which receives a Selection Command from the Synchronizer and performs replacement of the main content (MVA) with the alternative video and audio content (AVA); and vi. a Decoder, which allows the smart receiver to access main (MVA) or alternative video and audio content (AVA) downloaded or broadcasted to said smart receiver and, after its decoding, to display the alternative video and audio content (AVA) on a receiver's screen.

2. The system according to claim 1, wherein the said smart receiver is configured as an IBB digital TV receiver supporting the integration of broadcast transmission and broadband distribution to an end consumer.

3. The system according to claim 2, wherein the said IBB smart receiver can is configured to operate with different digital terrestrial, cable or satellite broadcasting technologies.

4. The system according to claim 1, wherein said broadband network is a network based on the private Internet Protocol (IP) owned by a telecommunication operator or a cable operator or a publicly available broadband provider.

5. The system according to claim 1, wherein the smart receiver uses said digital broadband connection to access the alternative audiovisual content (AVA) that can be downloaded or broadcasted to the smart receiver.

6. The system according to claim 1, wherein the said digital broadcasting connection is an Integrated Services Digital Broadcasting-Terrestrial (ISDB-T-based transmission standard used in the Brazilian Digital TV System provided by the broadcast network based on terrestrial, cable or satellite access.

7. The system according to claim 1, wherein said digital broadcasting connection is a transmission based on digital TV standard Digital Video Broadcasting—Terrestrial (DVB-T), or Advanced Television Systems Committee (ATSC), provided by the broadcast network based on terrestrial, cable or satellite access.

8. A method for implementing a system capable of ingesting alternative video or audio media based on continuous verification of relevant parameters for display, the method comprising: interpreting requests/signals from the broadcaster, informing a digital TV receiver, a structure and nature of main video and audio (MVA) content, receiving, by the digital TV receiver, a MVA content descriptor sent by the broadcaster for displaying at any time in the future, analyzing the descriptor by a decision-making center to detect if any specific content can be replaced by alternative audio and video (AVA) content, through a specific marking in the descriptor of that content, assessing, by the decision-making center, whether the digital TV receiver has sufficient conditions to perform the replacement of the MVA content with the AVA content without impairing the viewer's experience, by performing an evaluation of the relevant parameters of the receptor and its broadband connection, assessing, by the decision-making center, whether an OTT (“Over-The-Top”) application associated with the tuned channel is installed on the digital TV receiver and whether the consumer/viewer is exclusively identified, selecting by the digital TV receiver, and requesting the display of an AVA content, at the correct time and in accordance with the broadcaster's criteria, for that viewer/consumer, estimating, through the data collection module, the time required to perform the replacement of MVA content with AVA content, the maximum quality (Qmax) and the maximum duration (Dmax), of an AVA content that can be reproduced without interruptions, verifying, by the data collection module, whether the device is registered in an exclusively identified receiver database restricted to the replacement of the MVA content, analyzing, by the decision-making center, the MVA content descriptor searching for a MVA content compatible with the CET, Qmax and Dmax estimated by the data collection module, providing information, by the decision-making center, to the OTT application about the nature of the MVA content that will be replaced, and about the estimated maximum quality Qmax for the AVA content, providing, by the decision-making center, the resources for the reproduction of the AVA, checking that the conditions for its reproduction are met, requesting, by the decision-making center, to the OTT application that the AVA content be loaded into a streaming module, sending, by the decision-making center, a start command to the synchronizer, and monitoring the synchronization events to ensure synchronism in the transition between MVA->AVA->MVA reproduction, detecting, by the synchronizer the exact moment of AVA content start by monitoring the synchronization events this control being performed by the broadcaster, sending, by the synchronizer of an injection command the AVA content to the streaming module, and a content switching command to the content selector, monitoring, by the synchronizer the synchronization events sent by the broadcaster, determining the end time of the AVA content, naturally detecting the end of the AVA content by the end of the AVA content or by an end signal, and collecting, by the OTT application, the information relevant to the TV broadcaster.

9. The method according to claim 8, wherein the method further comprises the interaction of the said OTT application with an AVA content provider, in order to obtain from said source the own AVA content that will be reproduced by the system.

10. The method according to claim 9, wherein the data collection module performs the steps of: a) verifying if the digital TV receiver is registered in a database, characterized as having an active restriction for the reproduction of the AVA content and, therefore, restricted to performing the replacement of the MVA content, b) collecting relevant parameters from the digital TV receiver and its broadband connection, including: Internet connection speed; Internet connection latency; Internet connection packet loss rate; Percentage of CPU usage; Percentage of system memory usage; Percentage of video memory usage; Average time to replace a MVA content with an AVA content in this receiver; Accumulated number of failures when replacing a MVA content with an AVA content in this receiver; and Restriction to reproduce the AVA content; c) estimating the time required to perform the exchange of the MVA content for the AVA content, which is called content exchange time (CET) the maximum quality (Amax) and the maximum duration (Dmax) of an AVA content that can be reproduced without interruption; and d) collecting the relevant parameters and data estimated in the previous steps for the decision-making center.

11. The method according to claim 10, wherein the decision-making center further performs the steps of: (a) determining whether the AVA content can be reproduced in its entirety without interruption; and (b) determining whether the replacement of the MVA content with the AVA content can be carried out within an acceptable time, without impairing the viewer's experience, by performing a continuous evaluation of the relevant parameters for such replacement.

Description

BRIEF DESCRIPTION OF THE FIGURES

[0041] The following figures illustrate:

[0042] FIG. 1—representation of a system according to the present invention.

[0043] FIG. 2—representation of a module of injection and replacement of linear content with non-linear content.

[0044] FIGS. 3a and 3b—flow chart of a method of replacing MVA content with AVA content.

[0045] FIG. 4—representation of a decision-making center.

[0046] FIGS. 5a and 5b—representation of a MVA content block descriptor and example of choosing a MVA content to be replaced.

[0047] FIG. 1 illustrates an embodiment of the system according to the present invention showing a smart receiver (001) connected to a conventional television broadcast network (002) and to a broadband network (003) enabling access to the Internet. For the purposes of the present invention, each of the networks (002-003) is considered a fully digital network. The smart receiver (001) can support the reproduction of conventional broadcast television content through its connection to the television broadcast network (002), such as a digital terrestrial TV broadcast network, a digital cable TV network, or a digital satellite TV network. In addition to traditional television set functions, the smart receiver (001), and, through its connection to the broadband network (003), can also support the delivery of on-demand online content such as streaming services, Over-The-Top (OTT) content or video on-demand (VOD) through an Over-The-Top Application (OTT Application) (004).

[0048] The broadband network (003) is a standard Internet protocol (IP)-based network, such as a private IP network owned by a telecommunications operator, or a cable operator or a publicly available broadband provider.

[0049] Thus, the smart receiver (001) supports a model in which the linearly (or live) broadcasted TV content, hereinafter called MVA content (005), is consumed over a digital broadcasting connection (002) supplemented with a digital broadband internet connection (003). The digital broadcasting connection can be, for example, a transmission in the ISDB-T-based transmission standard used in the Brazilian Digital TV System, or a transmission based on another type of digital TV standard, for example, DVB-T, ATSC, etc., provided by the broadcast network based on terrestrial, cable or satellite access.

[0050] The digital broadband connection (003) allows the smart receiver (001) to access alternative audiovisual content (AVA) (006) that can be downloaded or broadcasted to the smart receiver (001).

[0051] The smart receiver (001) can be an IBB digital TV receiver that supports the integration of broadcast transmission (002) and broadband distribution (003) to the end consumer. The IBB smart receiver can operate with different digital transmission technologies, such as terrestrial, cable or satellite transmissions.

[0052] FIG. 2 illustrates a content replacement and injection module (011) of the smart receiver (001), responsible for consuming the MVA content (005), broadcasted by the broadcast transmission (002), and the AVA content (006), broadcasted over the broadband network (003). This module performs the switching of MVA content (005) to AVA content (006) through data analysis—content descriptors (007) and synchronization events (008)—sent by the broadcaster (009).

[0053] FIGS. 3a and 3b illustrate the method used by the content replacement and injection module (011) to perform the replacement of the MVA content (005) with the AVA content (006), ensuring the best possible experience for the viewer.

[0054] As observed, the system according to the present invention interprets the requests/signals from the broadcaster (009) using a technique such as, but not limited to, the so-called “System Events”. This signaling is interpreted by the system as a request to replace the MVA content (005) with the AVA content (006), informing the digital TV receiver of the structure and nature of the MVA content (005).

[0055] According to the method for implementation of said system according to the present invention in said TV receiver, in the initial step (021), the digital TV receiver receives from the broadcaster (009) a MVA content descriptor (007), as illustrated by FIGS. 5a and 5b, which will be shown at any time in the future. A decision-making center (012) then analyzes the MVA content descriptor (007) to detect if any specific content is liable to be replaced by AVA content (006), by means of a specific marking/tag in the descriptor of that content. The decision-making center (012) is responsible for assessing whether the digital TV receiver has sufficient conditions to perform the replacement of the MVA content (005) with the AVA content (006) without impairing the viewer's experience.

[0056] In the next step (022), the decision-making center (012) then evaluates whether an OTT-type application (“Over-The-Top”) (004) associated with the tuned channel is installed on the digital TV receiver and if the consumer/viewer is exclusively identified (013). In this way, the digital TV receiver will be able to select and request to display an AVA (006) at the correct time and in accordance with the broadcaster's criteria (009) for that viewer/consumer. The OTT application (004) is also responsible for interacting with an AVA content provider (010) and obtaining from this source the AVA content (005) that will be reproduced by the system. Access to the AVA content provider (010) can be done, for example, through an HTTP connection via the Internet, accessed over a broadband network (003).

[0057] In the next step (023), the decision-making center (012) seeks to determine whether the replacement of the MVA content (005) with the AVA content (006) can be performed without impairing the viewer's experience. For this purpose, the decision-making center (012) collects relevant parameters from the digital TV receiver through a data collection module (039), in order to determine: (a) whether the AVA content (006) can be reproduced in its entirety without interruptions; (b) whether the replacement of the MVA content (005) with the AVA content (006) can be performed within an acceptable time, without impairing the viewer's experience.

[0058] Examples of relevant parameters include, but are not limited to, those below: [0059] Internet connection speed (003); [0060] Internet connection latency (003); [0061] Internet connection packet loss rate (003); [0062] Percentage of CPU usage (040); [0063] Percentage of system memory usage (041); [0064] Percentage of video memory usage (042); [0065] Average time to replace a MVA content (005) with an AVA content (006) in this receiver; [0066] Accumulated number of failures when replacing a MVA content (005) with an AVA content (006) in this receiver; [0067] Restriction to reproduce the AVA content (005).

[0068] Provided with all relevant receiver parameters, according to the present invention, the data collection module (039), seeks to estimate the time required to exchange the MVA content (005) for the AVA content (006), which is called content exchanging time (CET) (044), the maximum quality (Qmax) (045), and the maximum duration (Dmax) (046), of an AVA content (006) that can be reproduced without interruption.

[0069] The data collection module (039) then checks whether the device is registered in an exclusively identified receiver database (043). If the device is registered, it will be prevented from replacing MVA content (005) with AVA content (006), and, therefore, the replacement process is concluded. The purpose of this database (043) is to register all devices that have failed in replacing MVA content (005) with AVA content (006). This database (043) can be implemented locally in the receiver (001), or remotely in a broadcaster (009) center. The broadcaster (009) can freely decide when a device is registered or removed from this database (043).

[0070] Once performed, in step (024), the decision-making center (012) analyzes the MVA content descriptor (007) to search for a MVA content (005) compatible with the CET, Qmax and Dmax estimated by the data collection module (039).

[0071] FIGS. 5a and 5b illustrate a possible MVA content descriptor (007) and compatibility with the CET (057), Qmax (058) and Dmax (059) parameters. It is observed that among the present descriptors, the MVA content (047) is not able to be replaced as its duration (051) is greater than Dmax (059), and the content (048) is not compatible, since the start time (054) is lower than the estimated CET (057). The only content eligible for replacement is content (049), which has a duration (055) of less than Dmax (059) and a sufficient transition time (056) that is greater than CET (057), so that the transition can be carried out without loss of content.

[0072] Then, in step (025), the decision-making center (012) informs the OTT application (004) about the nature of the MVA content (005) that will be replaced, as well as the estimated maximum quality Qmax (058) for AVA content (006). The OTT application (004) sends this information, together with the viewer's identification to an AVA content provider (010). It is expected that the provider will choose the AVA content (006) compatible with the MVA content (005) selected for replacement, according to techniques and products commonly used in the market for segmenting VoD (video-on-demand) content. It is important that the content meets the requirements defined by the system for maximum quality and maximum duration, in order to provide the best possible experience for the viewer.

[0073] In the next step (026), the decision-making center (012) provides resources for the reproduction of the AVA (006), by checking if the conditions for its reproduction are met. Typical conditions, for example, can be, among others, the percentage of CPU usage (040), amount of system memory (041) available, amount of storage (050) available, bandwidth speed and latency (003). The decision-making center (012) requests (037) to the OTT application (004) that the AVA content (006) be loaded into a streaming module (014), using commonly known techniques for reproducing a remote content.

[0074] A fundamental difference, however, is that the streaming module (014) must perform all the required steps to reproduce AVA content (006), with the exception of the last step, which is to inject the content into the decoder (015). We call this process of preloading the AVA content (006).

[0075] In the next step (027), if all the conditions for reproduction are met, and the AVA content (006) is properly preloaded, the decision-making center sends a start command (038) to the synchronizer (016), which monitors the synchronization events (008) to ensure the perfect synchronization in the transition between the reproduction of the MVA (005)->AVA (006)->MVA (005).

[0076] Through the synchronization events (008), the broadcaster (009) can control the exact time when the AVA content (006) should start, the exact time when the AVA content (006) should end and whether the content AVA (006) should end early.

[0077] The final condition is to ensure the return to MVA (005) reproduction status without unwanted failures, and maintaining the correct timing for resuming its display, with no loss of MVA (005) or AVA (006) content to the viewer.

[0078] In some cases, the broadcaster (009) may decide to end the reproduction of AVA content (006) early, for example, in the case of an urgent live event that must be shown to the viewer immediately.

[0079] In the next step (028), the synchronizer (016) detects the exact moment when AVA content (006) starts by monitoring the synchronization events (008). It is important that this control is performed by the broadcaster (009). Those skilled in the art will know that a broadcast transmission (002) can hardly be planned far in advance, due to the occurrence, for example, of extraordinary programs, such as live sporting events, in which it is not previously and precisely known when a MVA content (005) will start or end. Thus, unlike the MVA content descriptor (007), which must be sent as early as possible to allow preloading, the AVA content (006) start time must be notified to the digital TV receiver as close as possible to the instant that the AVA content (006) is to start. The difference in temporal requirement of these two signals shows that the synchronization event (008) is fundamental to guarantee, with maximum accuracy, the start time of reproduction of the AVA content (006). The synchronization events (008) are therefore important for the system of the present invention (011) to be able to balance these different requirements, by performing all the validation and preloading processes of an AVA content (006) as early as possible, while starting the reproduction of this AVA content (006) as accurately as possible.

[0080] Thus, once the AVA content (006) start time is detected, in step (029), the synchronizer sends an injection command (019) of AVA content (006) to the streaming module (014), and a content switching command (020) to the content selector (017). The content selector (017) switches the source of audio and video content that will be injected into the decoder (015), which previously received this content directly from the tuner (018), and now starts to receive this content from an AVA content server (010), generally through, for example, an HTTP connection via the Internet (003), using a broadband network.

[0081] In the next step (030), while the AVA content (006) is being reproduced, the synchronizer (016) monitors the synchronization events (008) sent by the broadcaster in order to determine the end time of the AVA content. This time may represent the natural end of the AVA content (006), or a voluntary and abrupt end of the AVA content (006) by the broadcaster (009).

[0082] In step (031) the system naturally detects the end of the AVA content (006) by the end of the AVA content (006) or by an end signal in this step (031). The content selector (017) promptly replaces the AVA content (006) with the MVA content (005) from the digital TV signal broadcast network (002), in step (032).

[0083] Finally, after the display of the AVA (006), the OTT application (004) can collect information relevant to the TV broadcaster (009), such as, but not necessarily restricted to, user identification, date, time, IP address, bandwidth, latency time, TV receiver manufacturer/model, AVA display status, etc., for statistical purposes.

[0084] In any situation (033), the decision-making center (012) checks, at different times, the conditions of the digital TV receiver by regularly evaluating various parameters, such as the CPU usage, amount of general-purpose memory (RAM) available, amount of storage memory for temporary use, speed and latency of the Internet connection, the characteristics of the AVA content (006) to be reproduced and the user's identification status (identified/unidentified), in such a way that the factors critical for the synchronization are mitigated. In case of failure (034), in step (035), the decision-making center (012) notifies a broadcaster center (060), by sending failure data (061), the parameters collected from the digital TV receiver at the time of the failure, the characteristics of the AVA content (006) and the identification (013) of the viewer. The broadcaster center (060) decides whether this receiver should be characterized from this moment on as having an active restriction on the reproduction of the AVA (006), registering it in the receiver database (043) as being restricted to the MVA content replacement, in step (036).

[0085] Failures in the reproduction of the AVA content (006) can be characterized by, but not limited to, the following: interruptions of the reproduction of the AVA content (006), failure to synchronize the audio and the video of the AVA content (006), spontaneous abortion of the reproduction of AVA content (006), and inability to access the AVA content provider (010).

[0086] Failures in the replacement of MVA content (005) with AVA content (006) can be characterized, for example, as delays in the switching of the MVA content (005) with the AVA content (006) and vice versa, and incompatibility of the duration of the MVA content (005) and the AVA content (006).

Embodiment Example

[0087] FIGS. 1, 2 and 3 described above represent a non-limiting schematic example of the scope of the invention. Such examples represent some of the possible ways of operating the system according to the present invention, in a digital TV receiver, where the broadcaster wishes to create a customized Video/Audio reproduction experience for a given group of consumers or for a single consumer.