Apparatus and method for transceiving scene composition information in multimedia communication system

Abstract

A method for transmitting scene composition information from an apparatus therefor in a multimedia communication system is provided. The method includes generating scene composition information comprising media attributes information and temporal information, and transmitting the scene composition information, wherein the media attributes information and temporal information is separated into different formats.

Claims

1. A method of transmitting scene composition information in a multimedia system, the method comprising: generating scene composition information comprising a list of asset (LoA) which includes media data type information for a plurality of scenes and spatial and temporal information of asset (STIA) which includes spatial and temporal information related to each of a plurality of scenes and spatial and temporal information related to each of a plurality of areas composing a scene of the plurality of scenes; and transmitting the scene composition information, wherein the LoA indicates asset information of all assets composing the plurality of scenes, and wherein the spatial and temporal information related to the plurality of scenes represents that the plurality of scenes is activated in series based on a time axis, and the plurality of areas included in the scene is activated in parallel based on the time axis.

2. The method of claim 1, wherein types of an asset included in each of the plurality of areas are different from each other.

3. The method of claim 1, wherein media data type information for initialization is acquired by analyzing the LoA.

4. The method of claim 1, wherein the media data type information comprises an address of a media source, and wherein the spatial and temporal information related to each of the plurality of areas comprises at least one of a type or a format of media.

5. A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one processor to perform the method of claim 1.

6. The method of claim 1, wherein the LoA is separately described from the STIA in the scene composition information.

7. A method of receiving scene composition information in a multimedia system, the method comprising: receiving scene composition information comprising a list of asset (LoA) which includes media data type information for a plurality of scenes and spatial and temporal information of asset (STIA) which includes spatial and temporal information related to each of a plurality of scenes and spatial and temporal information related to each of a plurality of areas composing a scene of the plurality of scenes; and displaying a scene using the scene composition information, wherein the LoA indicates asset information of all assets composing the plurality of scenes, and wherein the spatial and temporal information related to the plurality of scenes represents that the plurality of scenes is activated in series based on a time axis, and the plurality of areas included in the scene is activated in parallel based on the time axis.

8. The method of claim 7, wherein types of an asset included in each of the plurality of areas are different from each other.

9. The method of claim 7, wherein media data type information for initialization is acquired by analyzing the LoA.

10. The method of claim 7, wherein the media data type information comprises an address of a media source, and wherein the spatial and temporal information related to each of the plurality of areas comprises at least one of a type or a format of media.

11. A device for transmitting scene composition information in a multimedia system, the device comprising: at least one processor configured to generate scene composition information comprising a list of asset (LoA) which includes media data type information for a plurality of scenes and spatial and temporal information of asset (STIA) which includes spatial and temporal information related to each of a plurality of scenes and spatial and temporal information related to each of a plurality of areas composing a scene of the plurality of scenes; and a transmitter configured to transmit the scene composition information, wherein the LoA indicates asset information of all assets composing the plurality of scenes, and wherein the spatial and temporal information related to the plurality of scenes represents that the plurality of scenes is activated in series based on a time axis, and the plurality of areas included in the scene is activated in parallel based on the time axis.

12. The device of claim 11, wherein types of an asset included in each of the plurality of areas are different from each other.

13. The device of claim 11, wherein media data type information for initialization is acquired by analyzing the LoA.

14. The device of claim 11, wherein the media data type information comprises an address of a media source, and wherein the spatial and temporal information related to each of the plurality of areas comprises at least one of a type or a format of media.

15. A device for receiving scene composition information in a multimedia communication system, the device comprising: a receiver configured to receive scene composition information comprising a list of asset (LoA) which includes media data type information for a plurality of scenes and spatial and temporal information of asset (STIA) which includes spatial and temporal information related to each of a plurality of scenes and spatial and temporal information related to each of a plurality of areas composing a scene of the plurality of scenes; and at least one processor configured to control a display of a scene using the scene composition information, wherein the LoA indicates asset information of all assets composing the plurality of scenes, and wherein the spatial and temporal information related to the plurality of scenes represents that the plurality of scenes is activated in series based on a time axis, and the plurality of areas included in the scene is activated in parallel based on the time axis.

16. The device of claim 15, wherein types of an asset included in the plurality of areas are different from each other.

17. The device of claim 15, wherein media data type information for initialization is acquired by analyzing the LoA.

18. The device of claim 15, wherein the media data type information comprises an address of a media source, and wherein the spatial and temporal information related to each of the plurality of areas comprises at least one of a type or a format of media.

19. The device of claim 15, wherein the LoA is separately described from the STIA in the scene composition information.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:

(2) FIG. 1 is a view schematically illustrating a process of presenting an asset using composition information (CI) in an moving picture experts group (MPEG) media transport (MMT) system according to an embodiment of the present disclosure;

(3) FIG. 2 is a view schematically illustrating a process of displaying a scene in an MMT system according to an embodiment of the present disclosure;

(4) FIG. 3 is a view schematically illustrating a process of presenting a scene, such as the scene of FIG. 2, using CI according to an embodiment of the present disclosure;

(5) FIG. 4 is a view schematically illustrating a structure of spatial and temporal information of asset (STIA) in an MMT system according to an embodiment of the present disclosure;

(6) FIG. 5 is a view schematically illustrating a structure of STIA and scenes in a time domain in an MMT system according to an embodiment of the present disclosure;

(7) FIG. 6 is a view schematically illustrating a structure of a scene and areas in a time domain in an MMT system according to an embodiment of the present disclosure;

(8) FIG. 7 is a view schematically illustrating a CI structure in which an STIA structure, such as the STIA structure as described in FIGS. 4 to 6, is considered according to an embodiment of the present disclosure;

(9) FIG. 8 is a view schematically illustrating the CI structure of FIG. 7 in a time domain according to an embodiment of the present disclosure;

(10) FIG. 9 is a view schematically illustrating a CI structure in which media attribute information of an asset may be separated into separate formats using a list of asset (LoA), continuity of an asset may be maintained even when a scene or an area is changed, and a part of a scene may be independently composed and managed as an area in an MMT system according to an embodiment of the present disclosure;

(11) FIG. 10 is a block diagram schematically illustrating an internal structure of a CI transmitting device in an MMT system according to an embodiment of the present disclosure; and

(12) FIG. 11 is a block diagram schematically illustrating an internal structure of a CI receiving device in an MMT system according to an embodiment of the present disclosure.

(13) Throughout the drawings, it should be noted that like reference numbers are used to depict the same or similar elements, features, and structures.

DETAILED DESCRIPTION

(14) The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of various embodiments of the present disclosure as defined by the claims and their equivalents. It includes various specific details to assist in that understanding but these are to be regarded as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the various embodiments described herein can be made without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and constructions may be omitted for clarity and conciseness.

(15) The terms and words used in the following description and claims are not limited to the bibliographical meanings, but, are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Accordingly, it should be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustration purpose only and not for the purpose of limiting the present disclosure as defined by the appended claims and their equivalents.

(16) It is to be understood that the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to “a component surface” includes reference to one or more of such surfaces.

(17) Embodiments of the present disclosure provide a device and a method for transmitting/receiving scene composition information in a multimedia communication system.

(18) Furthermore, embodiments of the present disclosure provide a device and a method for transmitting/receiving scene composition information, created in a form in which media attribute information and spatial-temporal information are separated from each other, in a multimedia communication system.

(19) Hereinafter, in describing embodiments of the present disclosure, the multimedia communication system is assumed to be, for example, a moving picture experts group (MPEG) media transport (MMT) system, and it is apparent that the multimedia communication system may be an evolved packet system (EPS), a long-term evolution (LTE) mobile communication system, an institute of electrical and electronics engineers (IEEE) 802.16m communication system, or the like, as well as the MMT system.

(20) In the present disclosure, a device and a method will be described for transmitting and receiving scene composition information, which is presented using, for example, extensible markup language (XML), in a multimedia communication system. However, the device and the method for transmitting and receiving scene composition information, proposed by embodiments of the present disclosure, may be applied as they are even when scene composition information presented using other formats as well as XML is used.

(21) Scene composition information newly proposed by embodiments of the present disclosure is defined as composition information (CI). In displaying various pieces of media data on one terminal, the CI represents information expressing a time and a space on a screen on which the media data is displayed and information explaining an associative relationship between the media data displayed using the information expressing the time and the space on the screen on which the media data is displayed.

(22) Media data to which the CI may be applied, such as video data, audio data, image data, text data, and the like, is defined as an asset. A list of assets used to compose one scene in a multimedia service is defined as a list of asset (LoA).

(23) Information required for accessing an asset and information required for accurately analyzing a received asset and displaying it are defined as asset information (AI). Here, the information required for accessing the asset represents an address of a media source, and may be, for example, a uniform resource identifier (URI). The information required for accurately analyzing the received asset and displaying it may include a type, a format, and the like of media, and, for example, when media data is video data, may include a profile, a level, and the like corresponding to a media format.

(24) A set of spatial information and temporal information for each asset included in CI is defined as spatial and temporal information of asset (STIA).

(25) An entire area in which media data is displayed on a screen of a terminal is defined as a scene, and one scene includes one or more areas. Here, each area may be a partial area of the scene. Spatial information for a scene, an area, and an asset is defined as spatial information (SI), and temporal information for the scene, the area, and the asset is defined as temporal information (TI).

(26) Furthermore, in using CI in a multimedia service, embodiments of the present disclosure propose a method of separating AI from STIA and providing it, a method of dividing a scene into areas and composing and managing the areas independently of the scene, and a method of maintaining continuity of an asset even when a scene or an area is changed.

(27) A process of presenting an asset using CI in an MMT system according to an embodiment of the present disclosure will be described with reference to FIG. 1.

(28) FIG. 1 is a view schematically illustrating a process of presenting an asset using CI in an MMT system according to an embodiment of the present disclosure.

(29) Referring to FIG. 1, the CI includes media attribute information and temporal information. The media attribute information is presented through AI of each asset included in LoA, and the temporal information and spatial information is presented through STIA of each asset included in the LoA.

(30) When the CI is presented as described above with reference to FIG. 1, media attribute information of an asset required for initialization may be simply acquired by analyzing only the LoA but not the entire CI. Here, as the CI is divided into the LoA and the STIA in an embodiment of the present disclosure, a connection relation between the LoA and the STIA is defined as asset connection information, and the asset connection information is referred to as ref Asset.

(31) Furthermore, when the CI is presented as described above with reference to FIG. 1, if one asset is repetitively displayed in the same scene several times, the CI may additionally include only connection information ref Asset in the CI to reduce the amount of data required for presenting the CI, without having to repetitively present media attribute information for the corresponding asset.

(32) With reference to FIG. 1, the process of presenting the asset using the CI in the MMT system according to an embodiment of the present disclosure has been described above, and a process of displaying a scene in an MMT system according to an embodiment of the present disclosure will next be described with reference to FIG. 2.

(33) FIG. 2 is a view schematically illustrating a process of displaying a scene in an MMT system according to an embodiment of the present disclosure.

(34) Referring to FIG. 2, the scene includes a plurality of areas, for example, Area 1, Area 2, and Area 3, and each of the plurality of areas includes an asset. Area 1 includes three assets, namely, a video asset, an audio asset, and a caption asset, Area 2 includes an image asset, and Area 3 includes a text asset.

(35) Accordingly, as illustrated in FIG. 2, one scene is divided into the plurality of areas, and the assets included in each of the plurality of areas are subordinated to the corresponding area, so that the areas may be composed and changed independently of the scene. Consequently, when the scene is composed as illustrated in FIG. 2, each area may be independently used.

(36) Referring to FIG. 2, the process of displaying the scene in the MMT system according to an embodiment of the present disclosure has been described above, and a process of presenting a scene using CI will next be described with reference to FIG. 3.

(37) FIG. 3 is a view schematically illustrating a process of presenting a scene, such as the scene of FIG. 2, using CI according to an embodiment of the present disclosure.

(38) Referring to FIG. 3, the areas which the scene includes are presented, and the assets included in the corresponding area are presented within the area.

(39) Which area each of the areas represents within the corresponding scene should be presented, and therefore, SI of the corresponding area is included in the CI. Furthermore, the CI includes the SI of the scene in order to provide a criterion required for presenting the SI of each of the areas.

(40) Furthermore, in order to reduce duplicate presentation of the SI, SI of the asset identical to the SI of the corresponding area in which the asset is included may be omitted.

(41) As described above, the CI structure illustrated in FIG. 3 makes it possible to intuitively identify the structure of the scene, the areas, and the assets displayed on a screen, thereby achieving easy creation and edition of the CI.

(42) With reference to FIG. 3, the process of presenting the scene of FIG. 2 using the CI has been described above, and a structure of STIA in an MMT system according to an embodiment of the present disclosure will next be described with reference to FIG. 4.

(43) FIG. 4 is a view schematically illustrating a structure of STIA in an MMT system according to an embodiment of the present disclosure.

(44) Referring to FIG. 4, there are a plurality of scenes illustrated in the STIA structure. A plurality of areas are included in each of the plurality of scenes, and a plurality of assets are included in each of the plurality of areas. In particular, the STIA structure illustrated in FIG. 4, when there are a plurality of convertible scenes included therein, does not need separate additional CI for conversion of the plurality of scenes, and therefore, even though a specific scene of the plurality of scenes is converted, playback related information of an asset may be maintained.

(45) Likewise to this, in the STIA structure illustrated in FIG. 4, when there are a plurality of convertible areas in one scene, even though a specific area of the plurality of areas is converted, playback related information of an asset may be maintained.

(46) With reference to FIG. 4, the STIA structure in the MMT system according to an embodiment of the present disclosure has been described above, and a structure of STIA and scenes in a time domain in an MMT system according to an embodiment of the present disclosure will next be described with reference to FIG. 5.

(47) FIG. 5 is a view schematically illustrating a structure of STIA and scenes in a time domain in an MMT system according to an embodiment of the present disclosure.

(48) Referring to FIG. 5, in the time domain, the STIA represents that the scenes are activated in series based on a time axis. Although playback related information of an asset may be maintained irrespective of a conversion of the scenes as described in FIG. 4, a time axis on which all of the scenes may be considered is additionally required to maintain continuity of the asset. Accordingly, as illustrated in FIG. 5, the STIA is implemented in view of the time axis.

(49) With reference to FIG. 5, the structure of the STIA and the scenes in the time domain in the MMT system according to an embodiment of the present disclosure has been described above, and a structure of a scene and areas in a time domain in an MMT system according to an embodiment of the present disclosure will next be described with reference to FIG. 6.

(50) FIG. 6 is a view schematically illustrating a structure of a scene and areas in a time domain in an MMT system according to an embodiment of the present disclosure.

(51) Referring to FIG. 6, in the time domain, the STIA represents that the areas are activated in parallel based on a time axis. Although playback related information of an asset may be maintained irrespective of a conversion of the areas as described in FIG. 4, a time axis on which all of the areas may be considered is additionally required to maintain continuity of the asset. Accordingly, as illustrated in FIG. 6, the STIA is implemented in view of the time axis, and thus, the scenes are also implemented in view of the time axis.

(52) With reference to FIG. 6, the structure of the scene and the areas in the time domain in the MMT system according to an embodiment of the present disclosure has been described above, and with reference to FIG. 7, a CI structure will next be described in which such an STIA structure as described in FIGS. 4 to 6 is considered in an MMT system according to an embodiment of the present disclosure.

(53) FIG. 7 is a view schematically illustrating a CI structure in which an STIA structure, such as the STIA structure as described in FIGS. 4 to 6, is considered according to an embodiment of the present disclosure.

(54) Referring to FIG. 7, a CI structure in which an STIA structure as described in FIGS. 4 to 6 that reflects the structure of the scenes, the areas, and the assets is considered. First, scenes in the STIA have to be activated in series. Accordingly, each scene includes TI thereof to represent an activation time point and a deactivation time point, and when a previously described scene is deactivated, a next scene is activated.

(55) Furthermore, areas included in one scene have to be activated in parallel. Accordingly, each area includes TI thereof to represent an activation time point and a deactivation time point. Here, when SI and TI existing within each of the scenes, the areas, and the assets are defined, an external reference may also exist to provide flexibility of presentation for the SI and the TI. In this way, in order to reduce duplicate presentation, when an asset has the same activation time as that of an area including itself, TI of the area may be omitted, and when an area has the same activation time as that of a scene including itself, TI of the area may be omitted. With reference to FIG. 7, the CI structure in which such an STIA structure as described in FIGS. 4 to 6 is considered has been described above, and the CI structure of FIG. 7 in a time domain will next be described with reference to FIG. 8.

(56) FIG. 8 is a view schematically illustrating the CI structure of FIG. 7 in a time domain according to an embodiment of the present disclosure.

(57) Referring to FIG. 8, although Scene 1 is changed to Scene 2 in 10 seconds after activation of STIA, Asset 1 and Asset 2 identically appear in Area 1 and Area 4. In this case, playback time information, which Asset 1 and Asset 2 included in Scene 1 have, is used for Asset 1 and Asset 2 included in Scene 2 as it is, so that continuity of the assets may be maintained.

(58) With reference to FIG. 8, the CI structure of FIG. 7 in the time domain has been described above, and with reference to FIG. 9, a CI structure will next be described in which media attribute information of an asset may be separated into separate formats using an LoA, continuity of an asset may be maintained even when a scene or an area is changed, and a part of a scene may be independently composed and managed as an area in an MMT system according to an embodiment of the present disclosure.

(59) FIG. 9 is a view schematically illustrating a CI structure in which media attribute information of an asset may be separated into separate formats using an LoA, continuity of an asset may be maintained even when a scene or an area is changed, and a part of a scene may be independently composed and managed as an area in an MMT system according to an embodiment of the present disclosure.

(60) Referring to FIG. 9, media attribute information of an asset is separated into separate formats using an LoA. Furthermore, STIA is implemented to be separate from the LoA such that continuity of an asset may be maintained even when a scene or an area is changed, and a part of a scene may be independently composed and managed as an area.

(61) With reference to FIG. 9, the CI structure has been described above in which the media attribute information of the asset may be separated into the separate formats by using the LoA, the continuity of the asset may be maintained even when the scene or the area is changed, and the part of the scene may be independently composed and managed as the area in the MMT system according to an embodiment of the present disclosure, and an internal structure of a CI transmitting device in an MMT system according to an embodiment of the present disclosure will next be described with reference to FIG. 10.

(62) FIG. 10 is a block diagram schematically illustrating an internal structure of a CI transmitting device in an MMT system according to an embodiment of the present disclosure.

(63) Referring to FIG. 10, a CI transmitting device 1000 includes a receiving unit 1011, a control unit 1013, a transmitting unit 1015, and a storage unit 1017.

(64) The control unit 1013 controls overall operations of the CI transmitting device. The control unit 1013 controls to perform overall operations related to an operation of transmitting CI for implementing, particularly, an LoA and STIA according to an embodiment of the present disclosure as separate formats. Here, the overall operations related to the operation of transmitting the CI are the same as those described with reference to FIGS. 1 to 9, and therefore, detailed descriptions thereof will be omitted.

(65) The receiving unit 1011 receives various types of signals from a CI receiving device, etc. under the control of the control unit 1013. Here, the various types of signals received by the receiving unit 1011 are the same as those described with reference to FIGS. 1 to 9, and therefore, detailed descriptions thereof will be omitted.

(66) The transmitting unit 1015 transmits various types of signals to the CI receiving device, etc. under the control of the control unit 1013. Here, the various types of signals transmitted by the transmitting unit 1015 are the same as those described with reference to FIGS. 1 to 9, and therefore, detailed descriptions thereof will be omitted.

(67) The storage unit 1017 stores the various types of signals received by the receiving unit 1011, and various types of data required for an operation of the CI transmitting device, particularly, information related to the operation of transmitting the CI.

(68) Meanwhile, although the receiving unit 1011, the control unit 1013, the transmitting unit 1015, the storage unit 1017, and the output unit 1019 are implemented as separate units in FIG. 10, it is apparent that the receiving unit 1011, the control unit 1013, the transmitting unit 1015, the storage unit 1017, and the output unit 1019 can be implemented as one integrated unit.

(69) With reference to FIG. 10, the internal structure of the CI transmitting device in the MMT system according to an embodiment of the present disclosure has been described above, and an internal structure of a CI receiving device in an MMT system according to an embodiment of the present disclosure will next be described with reference to FIG. 11.

(70) FIG. 11 is a block diagram schematically illustrating an internal structure of a CI receiving device in an MMT system according to an embodiment of the present disclosure.

(71) Referring to FIG. 11, a CI receiving device 1100 includes a receiving unit 1111, a control unit 1113, a transmitting unit 1115, and a storage unit 1117.

(72) The control unit 1113 controls overall operations of the CI receiving device. The control unit 1013 controls to perform overall operations related to an operation of receiving CI for implementing, particularly, an LoA and STIA according to an embodiment of the present disclosure as separate formats. Here, the overall operations related to the operation of receiving the CI are the same as those described with reference to FIGS. 1 to 9, and therefore, detailed descriptions thereof will be omitted.

(73) The receiving unit 1111 receives various types of signals from a CI transmitting device under the control of the control unit 1113. Here, the various types of signals received by the receiving unit 1011 are the same as those described with reference to FIGS. 1 to 9, and therefore, detailed descriptions thereof will be omitted.

(74) The transmitting unit 1115 transmits various types of signals to the CI transmitting device under the control of the control unit 1113. Here, the various types of signals transmitted by the transmitting unit 1115 are the same as those described with reference to FIGS. 1 to 9, and therefore, detailed descriptions thereof will be omitted.

(75) The storage unit 1117 stores the various types of signals received by the receiving unit 1111, information related to operations of the CI receiving device, and the like.

(76) Meanwhile, although the receiving unit 1111, the control unit 1113, the transmitting unit 1115, and the storage unit 1117 are implemented as separate units in FIG. 11, it is apparent that the receiving unit 1111, the control unit 1113, the transmitting unit 1115, and the storage unit 1117 can be implemented as one integrated unit.

(77) It will be appreciated that various embodiments of the present disclosure according to the claims and description in the specification can be realized in the form of hardware, software or a combination of hardware and software.

(78) Any such software may be stored in a non-transitory computer readable storage medium. The non-transitory computer readable storage medium stores one or more programs (software modules), the one or more programs comprising instructions, which when executed by one or more processors in an electronic device, cause the electronic device to perform a method of the present disclosure.

(79) Any such software may be stored in the form of volatile or non-volatile storage such as, for example, a storage device like a Read Only Memory (ROM), whether erasable or rewritable or not, or in the form of memory such as, for example, Random Access Memory (RAM), memory chips, device or integrated circuits or on an optically or magnetically readable medium such as, for example, a Compact Disk (CD), Digital Versatile Disc (DVD), magnetic disk or magnetic tape or the like. It will be appreciated that the storage devices and storage media are various embodiments of non-transitory machine-readable storage that are suitable for storing a program or programs comprising instructions that, when executed, implement various embodiments of the present disclosure. Accordingly, various embodiments provide a program comprising code for implementing apparatus or a method as claimed in any one of the claims of this specification and a non-transitory machine-readable storage storing such a program.

(80) While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.