Systems, methods, and instrumentalities are disclosed for motion vector clipping when affine motion mode is enabled for a video block. A video coding device may determine that an affine mode for a video block is enabled. The video coding device may determine a plurality of control point affine motion vectors associated with the video block. The video coding device may store the plurality of clipped control point affine motion vectors for motion vector prediction of a neighboring control point affine motion vector. The video coding device may derive a sub-block motion vector associated with a sub-block of the video block, clip the derived sub-block motion vector, and store it for spatial motion vector prediction or temporal motion vector prediction. For example, the video coding device may clip the derived sub-block motion vector based on a motion field range that may be based on a bit depth value.

Disclosed is an inter prediction method which includes deriving control points (CPs) for the current block, wherein the CPs include a first CP and a second CP, deriving a first motion vector predictor (MVP) for the first CP and a second MVP for the second CP based on neighboring blocks of the current block, decoding a first motion vector difference (MVD) for the first CP, decoding a difference of two MVDs (DMVD) for the second CP, deriving a first motion vector (MV) for the first CP based on the first MVP and the first MVD, deriving a second MV for the second CP based on the second MVP and the DMVD for the second CP, and generating a predicted block for the current block based on the first MV and the second MV.

Computer implemented methods and systems are provided for training a machine learning architecture for surgical performance tracking and measurement based on surgical procedure video data set. The methods and systems include, in a first aspect, a sequential relation architecture and a dimensionality reduction architecture. In a second aspect, the methods and systems include a surgical instrument instance segmentation architecture, a decomposition model, and a sequential relation architecture. The video data is processed on a frame level to generate compressed or reduced representations of the video data.

Computer implemented methods and systems are provided for training a machine learning architecture for surgical performance tracking and measurement based on surgical procedure video data set. The methods and systems include, in a first aspect, a sequential relation architecture and a dimensionality reduction architecture. In a second aspect, the methods and systems include a surgical instrument instance segmentation architecture, a decomposition model, and a sequential relation architecture. The video data is processed on a frame level to generate compressed or reduced representations of the video data.

Devices, systems and methods for sample refinement and filtering method for video coding are described. In an exemplary aspect, a method for video processing includes modifying for a conversion between a block of a video and a bitstream representation of the video, a refinement value for a prediction sample in the block by applying a clipping operation to refinement value. The refinement value is derived based on a gradient value of an optical flow coding process. An output of the clipping operation is within a range. The method also includes refining the prediction sample based on the refinement value and performing the conversion based on the refined prediction sample.

A method for processing a video signal using an affine motion prediction is disclosed. The method includes checking that a current block is encoded by the affine motion prediction, obtaining motion vectors for a plurality of control points of the current block, determining a motion vector for each of a plurality of subblocks included in the current block based on the motion vectors for the plurality of control points, and performing a prediction for the current block from the motion vector for each of the plurality of subblocks, wherein each of the plurality of subblocks is configured to have a pre-defined width and a pre-defined height.

Devices, systems and methods for digital video coding, which include sub-block based inter prediction methods, are described. An exemplary method for video processing includes determining, for a conversion between a current block of video and a bitstream representation of the video, a maximum number of candidates in a sub-block based merge candidate list and/or whether to add sub-block based temporal motion vector prediction (SbTMVP) candidates to the sub-block based merge candidate list based on whether temporal motion vector prediction (TMVP) is enabled for use during the conversion or whether a current picture referencing (CPR) coding mode is used for the conversion, and performing, based on the determining, the conversion.

Provided are a coding prediction method and apparatus and a computer storage medium. The method includes that; for a coding block of an intra prediction type, Motion Vector (MV) Predictors (MVPs) of at least two control points of the coding block are determined; affine motion model-based motion estimation is performed on the coding block based on the MVPs of the at least two control points to obtain a first coding parameter of the coding block, the first coding parameter indicating a group of coding parameters corresponding to a minimum Rate-Distortion Cost (RDcost) obtained by performing motion estimation on the coding block in a non-translation motion manner; and prediction coding is performed on the coding block based on the first coding parameter.

Techniques related to derivation of motion vectors of a first color component (e.g., chroma component) from motion vectors of a second color component (e.g., luma component) are described. A video coder (e.g., video encoder or video decoder), for a CU coded in affine mode with 4:4:4 color format, may determine a motion vector for each sub-block of the luma block, and determine a motion vector for each sub-block of the chroma block based only on the motion vector for each co-located (also called collocated) sub-block of the luma block. However, for another CU coded in affine mode but with a color format other than 4:4:4 (e.g., 4:2:2 or 4:2:0), the video coder may determine a motion vector for each sub-block of the chroma block based on an average of two or more motion vectors of sub-blocks of the luma block.

Techniques related to derivation of motion vectors of a first color component (e.g., chroma component) from motion vectors of a second color component (e.g., luma component) are described. A video coder (e.g., video encoder or video decoder), for a CU coded in affine mode with 4:4:4 color format, may determine a motion vector for each sub-block of the luma block, and determine a motion vector for each sub-block of the chroma block based only on the motion vector for each co-located (also called collocated) sub-block of the luma block. However, for another CU coded in affine mode but with a color format other than 4:4:4 (e.g., 4:2:2 or 4:2:0), the video coder may determine a motion vector for each sub-block of the chroma block based on an average of two or more motion vectors of sub-blocks of the luma block.