The present invention relates to a method for dynamic image content replacement in a video stream comprising generating a set of key image data (K) comprising a sequence of at least two different key images (K1, K2), periodically displaying said set of key image data (K) on a physical display, generating at least a first original video stream (O1) of a scene which includes said physical display by recording said scene with a camera, wherein said at least one video stream (O1) comprises key video frames (FK1, FK2), captures synchronously with displaying each of said at least two different key images (K1, K2) of said set of key image data (K) on said physical display, generating a mask area (MA) corresponding to an active area of said physical display visible in said key video frames from differential images (AFK) obtained from consecutive key video frames (FK1, FK2), generating at least one alternative video stream (V) by inserting of alternative image content (I) into the mask area (MA) of an original video stream, and broadcasting at least said at least one alternative video stream.

A system is provided for capturing a key signal within video frames that includes a camera that captures a sequence of media content of a live scene that includes an electronic display having a higher frame rate than an output frame rate of the camera, and a key signal processor that convert all frames in the sequence of media content to the output frame rate of the camera, analyzes a sequence of frames to determine the key signal based on the electronic display outputting a sequence of frames including media content and at least one key frame included in the sequence, and combine remaining frames of the sequence of frames to create a live output signal. Moreover, the key signal processor determines, for each pixel in the frames, whether the pixel has a set chromaticity, and generates a key mask for each pixel in each frame.

A system is provided for capturing a key signal within video frames that includes a camera that captures a sequence of media content of a live scene that includes an electronic display having a higher frame rate than an output frame rate of the camera, and a key signal processor that convert all frames in the sequence of media content to the output frame rate of the camera, analyzes a sequence of frames to determine the key signal based on the electronic display outputting a sequence of frames including media content and at least one key frame included in the sequence, and combine remaining frames of the sequence of frames to create a live output signal. Moreover, the key signal processor determines, for each pixel in the frames, whether the pixel has a set chromaticity, and generates a key mask for each pixel in each frame.

Disclosed is a broadcast providing method implemented at an electronic device including processing circuitry. The broadcast providing method includes receiving, by the processing circuitry, a broadcast image from a broadcast server, and generating, by the processing circuitry, a final image by synthesizing the broadcast image with a personalized background image using a chroma key, the personalized background image being personalized for a user of the electronic device.

A method of generating a training data set for training an image matting machine learning model includes receiving a plurality of foreground images, generating a plurality of composited foreground images by compositing randomly selected foreground images from the plurality of foreground images, and generating a plurality of training images by compositing each composited foreground image with a randomly selected background image. The training data set includes the plurality of training images.

A computer system may identify a first position of a physical camera corresponding to a first time period. The computer system may render a first virtual scene for the first time period. The system may project the first scene onto a display surface to determine a first rendered image for the first time period. The computer system may receive a first camera image of the display surface from the camera during the first time period. The system may determine a first corrected position of the camera by comparing the first rendered image to the first camera image. The system may predict a second position of the camera corresponding to a second time period. The computer system may render a second virtual scene for the second time period. The system may project the second virtual scene onto the display surface to determine a second rendered image for the second time period.

A chroma key screen (SL; SR) for arrangement along a side of a studio set comprises a plurality of vertically arranged louvres or slats (L), each angled in a direction (D) towards the front of the studio set. The screen (SL, SR) may appear to the studio camera to be a substantially continuous chroma key, but reflections (R) of the chroma key screen (SL; SR) from a window (W) or from objects (O) or people (P) within the studio set are reduced. This allows an improved augmented reality (AR) studio set to be created.

A chroma key screen (SL; SR) for arrangement along a side of a studio set comprises a plurality of vertically arranged louvres or slats (L), each angled in a direction (D) towards the front of the studio set. The screen (SL, SR) may appear to the studio camera to be a substantially continuous chroma key, but reflections (R) of the chroma key screen (SL; SR) from a window (W) or from objects (O) or people (P) within the studio set are reduced. This allows an improved augmented reality (AR) studio set to be created.

Disclosed herein are methods and systems for combining foreground video and background video using chromatic matching. In an embodiment, a system obtains foreground video data. The system obtains background video data. The system determines a color-distribution dimensionality of the background video data to be either high-dimensional chromatic or low-dimensional chromatic. The system selects a chromatic-adjustment technique from a set of chromatic-adjustment techniques based on the determined color-distribution dimensionality of the background video data. The system adjusts the foreground video data using the selected chromatic-adjustment technique. The system generates combined video data at least in part by combining the background video data with the adjusted foreground video data. The system outputs the combined video for display.

The disclosure provides an approach for image segmentation from an uncalibrated camera array. In one aspect, a segmentation application computes a pseudo depth map for each frame of a video sequence recorded with a camera array based on dense correspondences between cameras in the array. The segmentation application then fuses such pseudo depth maps computed for satellite cameras of the camera array to obtain a pseudo depth map at a central camera. Further, the segmentation application interpolates virtual green screen positions for an entire frame based on user input which provides control points and pseudo depth thresholds at the control points. The segmentation application then computes an initial segmentation based on a thresholding using the virtual green screen positions, and refines the initial segmentation by solving a binary labeling problem in a Markov random field to better align the segmentation with image edges and provide temporal coherency for the segmentation.