Multiple clients (viewers) are allowed to share their VR spaces for communication with one another. A server-distributed stream including a video stream obtained by encoding a background image is received from a server. A client-transmitted stream including representative image meta information for displaying a representative image of another client is received from another client apparatus. The video stream is decoded to obtain the background image. The image data of the representative image is generated on the basis of the representative image meta information. Display image data is obtained by synthesizing the representative image on the background image.

Enriching video-based content with additional content is enabled by defining metadata for a video comprising a place holder and matching attribute. When the video is viewed, a matching server is utilized to select an object to be inserted in the place holder according to the matching attribute. The matching server selects the object according to video content, represented by matching attribute, according to user related information, and according to type of place holder. The place holder may be utilized to dynamically insert an interactive object to the video. The inserted object may be seamlessly inserted to the video.

The invention relates to a server for providing a graphical user interface to a client over a communication network. The graphical user interface comprises a graphical user interface element, the graphical user interface element being formed by an element shape and an element text, the element shape being represented by element shape data, the element text being represented by element text data. The server comprises an encoder configured to encode the element shape data into video data, a detector configured to detect a change associated with the graphical user interface element within the graphical user interface, and a communication interface configured to separately transmit the video data and the element text data over the communication network, the element text data being transmitted upon detection of the change associated with the graphical user interface element for providing the graphical user interface to the client.

A media playback system for presenting to a user a composition of a plurality of media streams. It has a media selection component configured to receive a scenario dataset, to receive user input for selecting viewing times defining segments of media and composition selections, and to output a list of segments of media from the scenario dataset that are authorized to be viewed by the user. The system has a playback control component configured to retrieve from media storage at least the segments of media from the output list of segments, to decode the segments of media, and to compile composition instructions. The system has a media playback component configured to receive the rendered media and the composition instructions.

Aspects of the subject disclosure may include, for example, a method that includes obtaining, by a processing system including a processor, video frames over a network; the processing system uses a machine learning algorithm to identify in each frame a first region comprising a natural image and a second region comprising a synthetic graphic image. The processing system separates the natural image from the synthetic graphic image to generate a natural video and a graphics video, encodes the natural video, and processes the graphics video to generate instructions for rendering graphic images at a client system. The client system performs a decoding procedure for the encoded video, a rendering procedure for client-side graphics in accordance with the instructions, and a compositing procedure to obtain a presentable video stream including the natural image and a client-side graphic corresponding to the synthetic graphic image. Other embodiments are disclosed.

Techniques are provided for processing media content. For example, a process can include using a scene description for to obtain media information and buffer information for a media object. The media information can identify an expected format for storing the media object in one or more output buffers of a media pipeline. The buffer information can include output buffer attributes. The process can include using the media information and the buffer information to initialize the media pipeline for the media object. The process can include obtaining the media object in a delivery format and generating one or more processed media objects. Each processed media object can be converted from the delivery format to the expected format for storing the processed media object in an output buffer. The process can include outputting the processed media object(s) to the output buffer(s) using the initialized media pipeline.

A method, computer system, and computer readable medium are provided for displaying video content. An interface to a presentation engine to enable real-time rendering of a plurality of objects is created whereby the plurality of objects are logically organized together through a plurality of scene graphs, and whereby the plurality of scene graphs is organized a media interchange format. Geometry information of the plurality of scene graphs is updated through the interface to a representation engine. The plurality of objects is streamed based on multi-resolution or multi-tessellation of heterogenous visual geometric objects in each of the scene graphs from among the plurality of scene graphs using the media interchange format.

One or more methods and systems are provided for encoding, decoding and playback of a video content in a client-server architecture. The invention proposes a video encoding and decoding method that includes identification of activities in the video content, identification of corresponding API's with related parameters corresponding to activity and storing those API's along with base frame and object frame in the database. In this invention, animation API functions are created for unknown/random activities. The playback involves decoding the data, which is a set of instructions to play the animation with given objects and base frames, and animating object frame over base frame using said API functions.

According to an embodiment of the present disclosure, there is provided a method of receiving a multimedia service, in which, by using user feedback information in a large-volume 6 degrees of freedom (6DoF) media transmission service using network-based media processing (NBMP), it is possible to reduce network traffic through efficient media transmission and a burden of rendering media content on a terminal. According to an embodiment of the present disclosure, a method of processing content includes receiving content regarding an object in a three-dimensional (3D) space, the content including media data and metadata, performing pre-rendering on the media data based on user information, and outputting pre-rendered data.

To provide an information processing apparatus, an information processing method, a reproduction processing apparatus, and a reproduction processing method that provide a user with a viewing experience with a high degree of freedom. A data input unit acquires data of a three-dimensional space configured as a single spatial three-dimensional object including a plurality of three-dimensional objects. A preprocessing unit generates three-dimensional object-specifying information for identifying a specific three-dimensional object included in a three-dimensional space. A file generation unit generates a file including data of a three-dimensional space and three-dimensional object-specifying information generated by the preprocessing unit.