A generation device of a three-dimensional model including: an acquisition unit configured to acquire a first mask image indicating a structure area, which is an object still within each image captured from a plurality of viewpoints, and a second mask image indicating a foreground area, which is a moving object within each image captured from the plurality of viewpoints; a combination unit configured to generate a third mask image that integrates the structure area and the foreground area within the image captured from the plurality of viewpoints by combining the first mask image and the second mask image both acquired; and a generation unit configured to generate a three-dimensional model including the structure and the foreground by a visual volume intersection method using the third mask image.

A system and method of calibrating a broadcast video feed are disclosed herein. A computing system retrieves a plurality of broadcast video feeds that include a plurality of video frames. The computing system generates a trained neural network, by generating a plurality of training data sets based on the broadcast video feed and learning, by the neural network, to generate a homography matrix for each frame of the plurality of frames. The computing system receives a target broadcast video feed for a target sporting event. The computing system partitions the target broadcast video feed into a plurality of target frames. The computing system generates for each target frame in the plurality of target frames, via the neural network, a target homography matrix. The computing system calibrates the target broadcast video feed by warping each target frame by a respective target homography matrix.

There is provided with an image processing apparatus. A target image including an object, for a predetermined imaging range, is obtained. A background image, which is obtained from a selected image selected from a captured image for the predetermined imaging range based on brightness of the target image, is obtained. A region of the object in the target image based on the background image is obtained.

The present invention relates to a device, a system and a method for tracking at least one object and player on a playing field/court and/or for monitoring the course of the game.

Systems and methods for analyzing scenes from cameras imaging an event, such as a sporting event broadcast, are provided. Systems and methods include detecting and tracking patterns and trails. This may be performed with intra-frame processing and without knowledge of camera parameters. A system for analyzing a scene may include an object characterizer, a foreground detector, an object tracker, a trail updater, and a video annotator. Systems and methods may provide information regarding centers and spans of activity based on object locations and trails, which may be used to control camera field of views such as a camera pose and zoom level. A magnification may be determined for images in a video sequence based on the size of an object in the images. Measurements may be determined from object trails in a video sequence based on an effective magnification of images in the video sequence.

A three-dimensional reconstruction method includes: reconstructing a first three-dimensional model from a first multi-viewpoint image obtained by shooting a real space, and reconstructing a second three-dimensional model from a second multi-viewpoint image obtained by shooting the real space at a different time; and matching world coordinate systems of the first three-dimensional model and the second three-dimensional model.

An image processing apparatus 200 includes: a first saturation information obtaining unit 306 configured to obtain saturation of a frame image; a second saturation information obtaining unit 309 configured to obtain saturation of a background image; and a threshold value determining unit 311 configured to determine a threshold value, which is used for checking a difference in color information, for each pixel, based on the saturation obtained by each of the first saturation information obtaining unit 306 and the second saturation information obtaining unit 309. A hue foreground area estimating unit 312 is configured to estimate a foreground area by use of the threshold value determined by the threshold value determining unit 311 as well as color information of the frame image and color information of the background image.

A system that generates a virtual viewpoint image from multiple viewpoint images captured from different camera viewpoints uses background 3D data representing a three-dimensional shape of a background in an image capturing scene. This background 3D data is generated by using a simulation image representing a view from the camera viewpoint. Then, a difference between the simulation image and an image acquired by performing test image capturing from the same camera viewpoint as the camera viewpoint is detected and the background 3D data is updated based on detection results.

Methods and apparatus to generate photo-realistic three-dimensional models of a photographed environment are disclosed. An apparatus includes an object position calculator to determine a three-dimensional (3D) position of an object detected within a first image of an environment and within a second image of the environment. The apparatus further includes a 3D model generator to generate a 3D model of the environment based on the first image and the second image. The apparatus also includes a model integrity analyzer to detect a difference between the 3D position of the object and the 3D model. The 3D model generator automatically modifies the 3D model based on the difference in response to the difference satisfying a confidence threshold.

An information processing apparatus comprising: a correction unit configured to correct a shift between a first three-dimensional position in an image capturing region that is identified based on a position of a first feature point on a first image among a plurality of images acquired by a first acquisition unit and image capturing information about an image capturing apparatus that corresponds to the first image and a second three-dimensional position in an image capturing region that is identified based on a position of a second feature point corresponding to the first feature point on a second image among the plurality of images acquired by the first acquisition unit and the image capturing information about the image capturing apparatus that corresponds to the second image.