METHOD FOR BUFFERING MEDIA TRANSPORT STREAM IN HETEROGENEOUS NETWORK ENVIRONMENT AND IMAGE RECEIVING APPARATUS USING THE SAME

Abstract

Provided are a method of buffering a media transport stream in a heterogeneous network environment and an image receiving apparatus using the same. A first transport stream corresponding to a first image through a first network is received and a second transport stream corresponding to a second image through a second network is received. The first image corresponding to the first transport stream is buffered based on a preset minimum transmission delay difference, and then the buffered first image and the second image corresponding to the second transport stream are processed.

Claims

1. A method of buffering transport streams received through different network paths, comprising: receiving a first transport stream corresponding to a first image through a first network; receiving a second transport stream corresponding to a second image through a second network; buffering the first image corresponding to the first transport stream based on a preset minimum transmission delay difference; and processing the buffered first image and the second image corresponding to the second transport stream, wherein the minimum transmission delay difference corresponds to a delay excluding a delay occurring in an encoding process for transmission of the first image and the second image among delays generated in a transmission process of the first image and the second image.

2. The method of claim 1, wherein the first image is a reference image serving as a reference for reproduction, and the second image is supplementary information or data that is reproduced in conjunction with the reference image.

3. The method of claim 2, wherein the minimum transmission delay difference is determined based on a segment duration of the additional image processed and transmitted as a segment, and a minimum segment buffering time predetermined in accordance with a communication processing standard.

4. The method of claim 3, wherein the minimum transmission delay difference is determined by further considering a segment window size, an additional image segment transmission time, and a reference image transport stream transmission time.

5. The method of claim 4, wherein the minimum transmission delay difference satisfies a condition of ΔT=(iS.sub.d+I.sub.R+MinbufferTime)−(B.sub.R), where ΔT represents the minimum transmission delay difference, S.sub.d represents the segment duration of the additional image, i represents the segment window size, I.sub.R represents the additional image segment transmission time, MinbufferTime represents the minimum segment buffering time, and B.sub.R represents the reference image transport stream transmission time.

6. The method of claim 1, wherein the first network is a broadcast network and the second network is an Internet network.

7. An image receiving apparatus for buffering transport streams received through different network paths, comprising: a first receiver for receiving a first transport stream corresponding to a first image through a first network; a second receiver for receiving a second transport stream corresponding to a second image through a second network; and a processer for processing and reproducing the first image and the second image, wherein the first receiver comprises a buffer for buffering a first video corresponding to the first transport stream based on a preset minimum transmission delay difference, and the minimum transmission delay difference corresponds to a delay excluding a delay occurring in an encoding process for transmission of the first image and the second image among delays generated in a transmission process of the first image and the second image.

8. The image receiving apparatus of claim 7, wherein the first image is a reference image serving as a reference for reproduction, and the second image is supplementary information or data that is reproduced in conjunction with the reference image.

9. The image receiving apparatus of claim 8, wherein the minimum transmission delay difference is determined based on a segment duration of the additional image processed and transmitted as a segment, and a minimum segment buffering time predetermined in accordance with a communication processing standard.

10. The image receiving apparatus of claim 9, wherein the minimum transmission delay difference is determined by further considering a segment window size, an additional image segment transmission time, and a reference image transport stream transmission time.

11. The image receiving apparatus of claim 10, wherein the minimum transmission delay difference satisfies a condition of ΔT=(iS.sub.d+I.sub.R+MinbufferTime)−(B.sub.R), where ΔT represents the minimum transmission delay difference, S.sub.d represents the segment duration of the additional image, i represents the segment window size, I.sub.R represents the additional image segment transmission time, MinbufferTime represents the minimum segment buffering time, and B.sub.R represents the reference image transport stream transmission time.

12. The image receiving apparatus of claim 7, wherein the first receiver further comprises a decoder for decoding the first image, which is buffered in the buffer and then is output, and outputting the decoded first image to the processer, and the second receiver comprises a decoder for decoding the second image corresponding to the received second transport stream and outputting the decoded second image to the processer.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG. 1 shows an example of an environment in which a stream is transmitted through a heterogeneous network.

[0023] FIG. 2 shows example of delays required by a transmitting/receiving process for images and each sub-process.

[0024] FIG. 3 shows an example of delays required by a transmitting/receiving process for additional images and each sub-process.

[0025] FIG. 4 a shows a schematic structural diagram of an image receiving apparatus according to an exemplary embodiment of the present invention.

[0026] FIG. 5 snows a flowchart of a method for buffering a transport stream according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] In the following detailed description, only certain exemplary embodiments of the present invention have been shown and described, simply by way of illustration. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive. Like reference numerals designate like elements throughout the specification.

[0028] Through the specification, in addition, unless explicitly described to the contrary, the word “comprise” and variations such as “comprises” or “comprising” will be understood to imply the inclusion of stated elements but not the exclusion of any other elements.

[0029] Hereinafter, referring to drawings, a method for buffering a media transport stream in a heterogeneous network environment and an image receiving apparatus using the method will be described.

[0030] FIG. 1 shows an example of an environment in which a stream is transmitted through a heterogeneous network.

[0031] As exemplary shown in FIG. 1, a broadcasting image is transmitted through a broadcasting network, a sign language image is transmitted through the Internet, and then a receiver receives the broadcasting image and the sign language image and synchronizes and displays them. For this display process, a stream corresponding to the broadcasting image transmitted through the broadcasting network is buffered for a predetermined time, for example, about 10 seconds. The streams transmitted through the heterogeneous networks have different transmission delays due to the constructions of transmitting/receiving systems and the network characteristics.

[0032] Exemplary embodiments of the present invention provide a specific method for performing buffering based on transmission delays of streams transmitted through the broadcasting network and the Internet.

[0033] In an exemplary embodiment of the present invention, a stream applied to the broadcasting network is a Moving Pictures Experts Group-2 (MPEG-2) Transport Stream (TS), a Real-time Object Delivery over Unidirectional Transport (ROTUE) TS, or an MPEG Media Transport Protocol (MMTP) TS. The ROUTE and the MMTP represent a multiplexing standard for transport streams that is currently under a standardization process in Advanced Television Systems Committee (ATSC) 3.0. Also, the transport streams transmitted to the Internet follow the MPEG Dynamic Adaptive Streaming over Hypertext Transfer Protocol (MPEG-DASH) standard. However, the present invention may not be limited thereto.

[0034] FIG. 2 shows example of delays required by a transmitting/receiving process for images and each sub-process.

[0035] An image used for reference in displaying will be referred to as a reference image, and the broadcasting image may be the reference image.

[0036] As shown in FIG. 2, the reference image, for example, a broadcasting image, is processed with image encoding (or video encoding), multiplexed, and then transmitted through a broadcasting network. The stream of the broadcasting image transmitted through the broadcasting network is received, de-multiplexed, image-decoded (or video decoded), and then reproduced as a broadcasting image by a receiver.

[0037] It is assumed that each delay generated in the encoding and multiplexing, the transmitting through the broadcasting network, and the de-multiplexing and decoding is B.sub.E, B.sub.R, and B.sub.D. The total transmission delay B.sub.Total that is required from the encoding of the reference image to the decoding may be defined as follows.

B.sub.Total=B.sub.E+B.sub.R+B.sub.D  (Equation 1)

[0038] Here, the B.sub.E represents a delay generated when encoding and decoding a reference image (in a process of signaling and stream multiplexing). B.sub.R represents a transmission delay through the broadcasting network. B.sub.D represents a delay generated when decoding the reference image and system decoding (in a process of de-multiplexing a multiplexed stream).

[0039] FIG. 3 shows an example of delays required by a transmitting/receiving process for additional images and each sub-process.

[0040] An additional image represents supplementary information or data which is reproduced in conjunction with the reference image. For example, in smart sign language broadcasting, a stream transmitted to a broadcasting network is used as the reference image and a stream transmitted to the Internet is used as the additional image.

[0041] As shown in FIG. 3, the additional image is processed by image encoding and multiplexing, divided into a plurality of segments according to Dynamic Adaptive Streaming over Hypertext Transfer Protocol (DASH) encoding, and then transmitted through, for example, the Internet. The segments of the additional image are received by a receiver. After that, they are buffered, de-multiplexed, decoded, and then reproduced as the additional image.

[0042] As above, for transmitting the additional image, compared to a process of transmitting/receiving a reference image, a DASH encoding process for generating segments according to an MPEG-DASH standard and a buffering process required for segments which are transmitted through the Internet are added. Accordingly, compared to transmitting/receiving a reference image, additional system delays occur.

[0043] If a delay in the encoding and multiplexing, a delay in the DASH encoding, a delay in the transmitting through a broadcasting network, and a delay in the buffering, de-multiplexing, and decoding are I.sub.E, I.sub.S, I.sub.R, and I.sub.MIN, respectively, the total transmission delay (I.sub.Total) required from the encoding of the additional image to the decoding may be defined as follows.

I.sub.Total=I.sub.E+I.sub.S+I.sub.R+I.sub.MIN+I.sub.D  (Equation 2)

[0044] Here, more specifically, I.sub.E represents a delay in encoding and system encoding for an additional image. I.sub.S represents a segmentation delay when generating an MPEG-2 DASH-based additional image and a media presentation description (MPD). I.sub.R represents a delay in transmitting additional image segments through the Internet. I.sub.MIN represents a minimum delay required in buffering additional image segments in a receiver. I.sub.D represents a delay in decoding and system decoding an additional image.

[0045] If it is assumed that the delays in encoding, system encoding, system decoding, and decoding a reference image are equal to the delays in encoding, system encoding, system decoding, and decoding an additional image, a transmission delay difference (ΔT) through a heterogeneous network may be represented as follows.

ΔT=(I.sub.S+I.sub.R+I.sub.MIN)−(B.sub.R)  (Equation 3)

[0046] Here, I.sub.S is a time to generate MPEG-DASH-based additional image segments and the MPD, and varies according to a minimum segment duration required for transmitting segments stably. Also, I.sub.R corresponds to a delay in transmitting an additional image segment through the Internet, and represents a time to transmit an additional image segment requested by a receiver.

[0047] If a time to request and receive an MPD by a receiver for additional image segment transmission is I.sub.MPD, Equation 3 may be represented as follows.

ΔT=(iS.sub.d+α+I.sub.MPD+I.sub.R+I.sub.MIN)−(B.sub.R)  (Equation 4)

[0048] Here, S.sub.d represents an actual segment duration of an additional image, and i represents a size of a segment window that is a duration for generating and transmitting a segment. α represents a time to generate segments and an MPD, and I.sub.MPD represents a time to request and receive an MPD by a receiver.

[0049] If it is assumed that α and I.sub.MPD are ‘0’, a time ΔT (may be called a minimum transmission delay difference) for buffering a transport stream in a broadcasting network may be defined as follows.

ΔT=(iS.sub.d+I.sub.R+MinbufferTime)−(B.sub.R)  (Equation 5)

[0050] Here, MinbufferTime represents I.sub.MIN.

[0051] If it is possible to generate a segment and an MPD and transmit the segment stably according to a request of a receiver based on the actual segment duration S.sub.d, i (the size of the segment window) is set as ‘1’. However, i may increase in order to transmit a segment stably according to an experimental environment and a service type.

[0052] The actual segment duration S.sub.d is described in an MPD of an MPEG-DASH standard and represents actual segment duration information. For example, S.sub.d=1 (s), which means that a segment consists of a one second stream.

[0053] Further, MinbufferTime is also described in an MPD of an MPEG-DASH standard and represents a minimum segment buffering time in a receiver. The receive needs to perform buffering for the MinbufferTime described in the MPD. Generally, a required minimum segment buffering time is lower than the MinbufferTime described in the MPD. That is, the required minimum segment buffering time varies according to a transmission bandwidth and a rate for additional image encoding. If a 10M encoded stream (S.sub.d=1 second) is transmission of a 10M bandwidth, a required minimum MinbufferTime is 1 second. If a 3M encoded stream (S.sub.d=1 second) is transmission of a 10M bandwidth, a required minimum MinbufferTime is 330 ms.

[0054] Accordingly, a transmission delay difference generated through a hybrid network varies according to a bandwidth situation of the Internet. However, the transmission delay difference is determined based on the additional image segment duration S.sub.d and the minimum segment buffering time (MinbufferTime, I.sub.MIN). Also, as in Equation 5, the transmission delay difference is determined based on the additional image segment duration S.sub.d, the size of the segment window for stable transmission/reception (i), the additional image segment transmission time I.sub.R, the minimum segment buffering time (MinbufferTime, I.sub.MIN), and the time B.sub.R to transmit a transport stream of a reference image.

[0055] Therefore, the related media can be stably reproduced by having the buffering time equal to the minimum transmission delay difference, that is, the minimum ΔT, except for the delay occurring in the encoding process of the reference image and the additional image.

[0056] FIG. 4 shows a configuration diagram of an image receiving apparatus according to an exemplary embodiment of the present invention.

[0057] The image receiving apparatus 100 according to an exemplary embodiment of the present invention, as shown in FIG. 4, includes a first receiver 110, a second receiver 120, and a processor 130. The image receiving apparatus 100 may receive a hybrid transport stream through different paths, for example, a broadcasting network and an Internet network. Here, the hybrid transport stream may include a program having audio and video coded based on MPEG-2.

[0058] The first receiver 110 receives and processes a reference image through a first network (e.g., a broadcasting network), and the second receiver 120 receives an additional image through a second network (e.g., the Internet). The reference image is image encoded and multiplexed, and then transmitted through the first network, that is, the broadcasting network, by a first transmitting apparatus 210. The additional image is image encoded, multiplexed, DASH-encoded, and then transmitted through the first network, that is, the Internet network, by a second transmitting apparatus 220. The reference image and the additional image may correspond to the same 3D TV contents.

[0059] The first receiver 110 specifically includes a first buffer 111 and a decoder 112 for decoding the reference image output from the first buffer 111. The first buffer 111 buffers the reference image for a minimum ΔT to compensate for the transmission delay difference generated through the hybrid network and then outputs the buffered reference image. The minimum ΔT is determined based on the additional image segment duration S.sub.d and the minimum segment buffering time (MinbufferTime, I.sub.MIN). In addition, the minimum ΔT may be determined by further considering the size of the segment window for stable transmission/reception (i), the additional image segment transmission time I.sub.R, and the reference image segment transmission time B.sub.R.

[0060] The decoder 112 decodes the reference image from the first buffer 111. For this decoding, a buffer for storing a reference image, for example, an elementary stream corresponding to the reference image, an audio stream corresponding to the reference image, synchronization information for synchronization, and the like may be further included.

[0061] The second receiver 120 includes a decoder 121 for decoding the additional image. The decoder 121 decodes the additional image and may include a buffer for storing the additional image, for example, an elementary stream corresponding to the additional image, synchronization information for synchronization, and the like, for the decoding.

[0062] FIG. 5 shows a flowchart of a method for buffering a transport stream according to an exemplary embodiment of the present invention.

[0063] The first transmitting apparatus 210 encodes and multiplexes the reference image and then transmits the encoded and multiplexed reference image through a first network (e.g., a broadcasting network). The second transmitting apparatus 220 encodes, multiplexes, DASH encodes, and transmits the additional image to a second network (e.g., the Internet).

[0064] The image receiving apparatus 100 receives a transport stream corresponding to the reference image and the additional image, respectively (S100 and S110).

[0065] After that, the image receiving apparatus 100 buffers the received reference image by a predetermined minimum ΔT to compensate for the transmission delay difference generated through the hybrid network (S120).

[0066] The image receiving apparatus 100 decodes the reference image output after being buffered, and decodes the received additional image (S130).

[0067] Thereafter, the decoded reference image and the additional image are reproduced in synchronization with each other (S140).

[0068] The above-described embodiments can be realized through a program for realizing functions corresponding to the configuration of the embodiments or a recording medium for recording the program in addition to through the above-described device and/or method, which is easily realized by a person skilled in the art. While this invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.