Systems and methods in accordance with embodiments of the invention enable feature based high resolution motion estimation from low resolution images captured using an array camera. One embodiment includes performing feature detection with respect to a sequence of low resolution images to identify initial locations for a plurality of detected features in the sequence of low resolution images, where the at least one sequence of low resolution images is part of a set of sequences of low resolution images captured from different perspectives. The method also includes synthesizing high resolution image portions, where the synthesized high resolution image portions contain the identified plurality of detected features from the sequence of low resolution images. The method further including performing feature detection within the high resolution image portions to identify high precision locations for the detected features, and estimating camera motion using the high precision locations for said plurality of detected features.

Systems and methods in accordance with embodiments of the invention enable feature based high resolution motion estimation from low resolution images captured using an array camera. One embodiment includes performing feature detection with respect to a sequence of low resolution images to identify initial locations for a plurality of detected features in the sequence of low resolution images, where the at least one sequence of low resolution images is part of a set of sequences of low resolution images captured from different perspectives. The method also includes synthesizing high resolution image portions, where the synthesized high resolution image portions contain the identified plurality of detected features from the sequence of low resolution images. The method further including performing feature detection within the high resolution image portions to identify high precision locations for the detected features, and estimating camera motion using the high precision locations for said plurality of detected features.

This invention describes a method for communicating crude motion information using tracking metadata and recovering more accurate motion information from the received tracking metadata and partial video frame data; in particular, we use metadata to convey crude boundaries of objects in the scene and signal motion information for these objects. The proposed method leaves the task of identifying the exact boundaries of an object to the decoder/client. The proposed method is particularly appealing when metadata itself carries semantics that the client is interested in, such as tracking information in surveillance applications, because, in this case, metadata does not constitute an overhead.

The proposed method involves motion descriptions that can be used to predict the appearance of an object in any one frame from its appearance in any other frame that contains the object. That is, the motion information itself allows locations within an object to be invertibly mapped to locations within the same object in any other relevant frame. This is a departure from conventional motion coding schemes, which tightly-couple motion information to the prediction strategy. This property makes the proposed method particularly suitable for applications which require flexible access to the content.

This invention describes a method for communicating crude motion information using tracking metadata and recovering more accurate motion information from the received tracking metadata and partial video frame data; in particular, we use metadata to convey crude boundaries of objects in the scene and signal motion information for these objects. The proposed method leaves the task of identifying the exact boundaries of an object to the decoder/client. The proposed method is particularly appealing when metadata itself carries semantics that the client is interested in, such as tracking information in surveillance applications, because, in this case, metadata does not constitute an overhead.

The proposed method involves motion descriptions that can be used to predict the appearance of an object in any one frame from its appearance in any other frame that contains the object. That is, the motion information itself allows locations within an object to be invertibly mapped to locations within the same object in any other relevant frame. This is a departure from conventional motion coding schemes, which tightly-couple motion information to the prediction strategy. This property makes the proposed method particularly suitable for applications which require flexible access to the content.

An image decoding method supporting a plurality of layers according to the present invention may comprise the steps of: generating an inter-layer reference picture set with respect to one or more reference layers to which a current picture can refer; generating an initial reference picture list including the inter-layer reference picture set; and predicting the current picture on the basis of the initial reference picture list. Accordingly, the present invention provides a method for generating a reference picture list including a picture of a layer, which is different from a layer to be currently encoded and decoded, and an apparatus using the same.

An image decoding method supporting a plurality of layers according to the present invention may comprise the steps of: generating an inter-layer reference picture set with respect to one or more reference layers to which a current picture can refer; generating an initial reference picture list including the inter-layer reference picture set; and predicting the current picture on the basis of the initial reference picture list. Accordingly, the present invention provides a method for generating a reference picture list including a picture of a layer, which is different from a layer to be currently encoded and decoded, and an apparatus using the same.

Techniques are described for sub-prediction unit (PU) based motion prediction for video coding in HEVC and 3D-HEVC. In one example, the techniques include an advanced temporal motion vector prediction (TMVP) mode to predict sub-PUs of a PU in single layer coding for which motion vector refinement may be allowed. The advanced TMVP mode includes determining motion vectors for the PU in at least two stages to derive motion information for the PU that includes different motion vectors and reference indices for each of the sub-PUs of the PU. In another example, the techniques include storing separate motion information derived for each sub-PU of a current PU predicted using a sub-PU backward view synthesis prediction (BVSP) mode even after motion compensation is performed. The additional motion information stored for the current PU may be used to predict subsequent PUs for which the current PU is a neighboring block.

Techniques are described for sub-prediction unit (PU) based motion prediction for video coding in HEVC and 3D-HEVC. In one example, the techniques include an advanced temporal motion vector prediction (TMVP) mode to predict sub-PUs of a PU in single layer coding for which motion vector refinement may be allowed. The advanced TMVP mode includes determining motion vectors for the PU in at least two stages to derive motion information for the PU that includes different motion vectors and reference indices for each of the sub-PUs of the PU. In another example, the techniques include storing separate motion information derived for each sub-PU of a current PU predicted using a sub-PU backward view synthesis prediction (BVSP) mode even after motion compensation is performed. The additional motion information stored for the current PU may be used to predict subsequent PUs for which the current PU is a neighboring block.

Techniques for efficient coding of digital media data are presented. A resource controller component dynamically allocates computing resources between an estimator component and a coder component. The estimator component generates an initial motion estimation of a raw video frame of a sequence of raw video frames based on a previous raw video frame. The coder component encodes the previous raw video frame to generate a reconstructed video frame in parallel with the generation of the initial motion estimation. When the previous raw video frame is reconstructed, the coder component dynamically determines whether and/or how to refine the motion estimation to encode the raw video frame based on a predefined encoding criterion. In making that determination, the coder component selects and uses a mode(s) of motion estimation refinement from a plurality of available motion estimation refinement modes to encode the raw video frame.

The present invention reduces the amount of computation that is needed when determining the block size, which is the unit in which motion information is to be held, for a video coding format in which motion estimation is performed. A program causes a computer to execute: a vector search process for determining motion information for each of the smallest blocks, which are the blocks in a predetermined deepest layer; a block size determination process for determining the block size, which is the unit in which the motion information is to be held; and a block integration process for determining whether or not to integrate one or a plurality of mutually adjacent sub-blocks with a single block having a single piece of motion information, on the basis of integration conditions including at least the motion information of the sub-blocks which are the blocks in a layer that is one layer deeper than the layer currently being considered; wherein the block size determination process determines the block size by carrying out the block integration process in layers, in a sequence from the blocks in a layer one level shallower than the smallest blocks, to the largest blocks, which are the blocks in a predetermined shallowest layer.