Data is encoded into one or more optically encoded images. The optically encoded images are then inserted as image data into a video sequence - i.e., in video frames. Data are transmitted in-band within the video, via any conceivable video distribution channel or format. The video may be trans-coded as required - because the data are optically encoded, any video processing that even crudely preserves the frame images will preserve the optically encoded data. This scheme of in-band data transfer in video is very robust. A video receiving apparatus receives the video, inspects the image data from video frames in memory, detects optically encoded images in the image data, and decodes the optically encoded images to recover the data. The frames carrying optically encoded images are typically discarded and not rendered to a display. The receiver controls connected equipment, other than a display (e.g., a musical instrument), based on the extracted data.

One aspect of the invention relates to a video distribution system for live distributing a video containing a virtual space and an animation of a character object generated based on a motion of a distributor user. The video distribution system determines, when receiving from a viewer user watching the video a participation request to request participation in the video, which one of first and second groups the viewer user belongs to. If the viewer user is determined to belong to the first group, the video distribution system arranges a specific avatar of the viewer user in a first region within the virtual space. If the viewer user is determined to belong to the second group, the video distribution system arranges the specific avatar in a second region within the virtual space as long as a condition for participation is satisfied.

Aspects of the subject disclosure may include, for example, a method that includes obtaining, by a processing system including a processor, video frames over a network; the processing system uses a machine learning algorithm to identify in each frame a first region comprising a natural image and a second region comprising a synthetic graphic image. The processing system separates the natural image from the synthetic graphic image to generate a natural video and a graphics video, encodes the natural video, and processes the graphics video to generate instructions for rendering graphic images at a client system. The client system performs a decoding procedure for the encoded video, a rendering procedure for client-side graphics in accordance with the instructions, and a compositing procedure to obtain a presentable video stream including the natural image and a client-side graphic corresponding to the synthetic graphic image. Other embodiments are disclosed.

The embodiments relate to a method, including writing a first and a second media entity in a container file; creating a media presentation description (MPD) with a first and a second Representation; the Representations belonging to Adaptation Sets; the Representations being associated with the media entities of the container file; when one of the Representations belongs to a media entity which is a thumbnail to a viewpoint or a thumbnail to an overlay, the method includes writing in the MPD file an association/correspondence/grouping information of the said one Representation with another Representation belonging to a media entity which is a viewpoint or an overlay, correspondingly. The embodiments also relate to a method for parsing, and technical equipment for implementing the methods.

Aspects of the subject disclosure may include, for example, obtaining a first group of volumetric content, generating first metadata for the first group of volumetric content, and storing the first group of volumetric content. Further embodiments include obtaining a second group of volumetric content, generating second metadata for the second group of volumetric content, and storing the second group of volumetric content. Additional embodiments include receiving a request for user volumetric content, and obtaining first volumetric content according to the request, the first group of volumetric content comprises the first volumetric content, and obtaining second volumetric content according to the request, the second group of volumetric content comprises the second volumetric content, generating the user volumetric content according to the request, the user volumetric content comprises the first volumetric content and the second volumetric content, and providing the user volumetric content to a communication device. Other embodiments are disclosed.

A method performed by a video controller. The method receives a first video stream captured by an endoscope of a surgical system, and receives a second video stream that comprises surgical data. The method displays the second video stream superimposed above an area of the first video stream, and determines that the second video stream is to cease being superimposed. Responsive to determining that the second video stream is to cease being superimposed, the method continues to display the first video stream.

Information may be provided for review by an augmented reality (AR) user. A system may store a plurality of scenes viewed by the AR user, and for each scene information identifying (i) any pairs of real-life objects and AR augmentations presented in the scene and (ii) any pairs in the scene with which the user interacted. The system may also select from the stored plurality of scenes a first subset of scenes such that each pair with which the user interacted is presented in at least one scene in the first subset of scenes. Responsive to an instruction from the user to review historical AR information, the system may present to the user the selected first subset of scenes. The first subset may be selected such that each pair with which the user interacted is presented in at least one scene in which the user interacted with that pair.

Graphical descriptor data associated with a particular video asset may include data for one or more graphical descriptor images and graphical descriptor metadata related to each graphical descriptor image. The graphical descriptor images can be simultaneously presented on a display screen with the video asset content and used to navigate to different content in that video asset. The graphical descriptors can also be used to navigate to content of one or more different assets.

Provided are a method and device for providing interactive virtual reality content capable of increasing user immersion by naturally connecting an idle image to a branched image. The method includes providing an idle image including options, wherein an actor in the idle image performs a standby operation, while the actor performs the standby operation, receiving a user selection for an option, providing a connection image, and providing a corresponding branched image according to the selection of the user, wherein a portion of the actor in the connection image is processed by computer graphics, and the actor performs a connection operation so that a first posture of the actor at a time point at which the selection is received is smoothly connected to a second posture of the actor at a start time point of the branched image.

Disclosed herein are a system and related operating method of providing video game functionality that is contextually linked with video program content. In certain implementations, the method involves the steps of: causing presentation of a video program at a presentation device; identifying a portion of the video program as a candidate for gamification; processing the identified portion into a video game segment that represents a gamified version of the identified portion; and causing presentation of the video game segment at a playback device.