Motion estimator apparatus and methods are presented in which a fully constrained nonlinear system of equations combining forward and backward displaced frame difference equations with a plurality of displacement vector invariant equations is solved using the input data from two image frames without approximation and without any additional constraints or assumptions to obtain an estimated displacement field. Also presented is an adaptive framework for solving a system of motion estimation equations with an integer valued block size defining a number of node points within an image, the number of node points being less than or equal to a number of pixels within the image, and a cost function based on a nonlinear least-squares principle. A system of iteration equations for the motion field on node points is solved using an iterative technique, and a degree of over-constraint can be progressively relaxed by selectively reducing the block size during the iteration.

Embodiments of the present disclosure disclose a method and a device for dynamically controlling quality of a video displaying on a display associated to an electronic device is provided. The method comprises detecting current eye position of a user and identifying at least one region of interest (ROI) on a display screen of the display device based on the current eye position of the user. Then, the method comprises predicting next position of the eye based on at least one of the current eye position of the user or the at least one ROI. Also, the method comprises converting the SD video in to a high definition (HD) video displayed on the ROI on the display screen associated with the current and next position of the eye. Further, the method comprises displaying the HD video on the ROI of the display screen.

A method of determining an output frame rate includes receiving an input sequence of frames of image data at an input frame rate, performing motion vector calculations on the frames of image data to produce motion vectors and motion statistics, determining a motion level in the frames using the motion statistics, and interpolating frames of image data at the output frame rate, wherein the output frame rate is based upon the motion level.

A method and apparatus for processing an image sequence described herein that provides a symmetric displaced frame difference equation. Next, an input image sequence can be received that includes a pair of image frames individually including multidimensional image data corresponding to a plurality of pixel locations at different times. Finally, using at least one processor, the symmetric displaced frame difference equation can be solved using an iteration equation and the image data of the pair of image frames to determine a displacement field describing displacement vectors at half of the total displacement vector.

This application provides a data processing method and apparatus, a device, and a storage medium. The method includes: processing, when a frame interpolation condition is satisfied between an obtained first real image frame and an obtained second real image frame, the first real image frame to obtain a first image and a mask image used for marking to-be-filled pixels and effective pixels in the first image; sampling mask data of the mask image pixel by pixel to determine whether pixels in the first image are to-be-filled pixels; and using color data of the first real image frame read from an internal memory to perform color filling on the to-be-filled pixel in the first image, to generate a predicted image frame displayed between the first real image frame and the second real image frame, where color data of the effective pixels in the first image is retained.

An electronic apparatus may include memory storing first and second learning models that have the same network structure and estimate a motion between two frames, a processor configured to obtain a first frame included in an input image and a second frame which is a previous frame of the first frame, and generate an interpolation frame using the obtained first frame and second frame, and the first learning model is a model trained with image data having a first characteristic, the second learning model is a model trained with image data having a second characteristic which is opposite to the first characteristic, and the processor is configured to generate a third learning model using a first control parameter and the first and second learning models, estimate a motion between the first frame and the second frame using the generated third learning model, and generate the interpolation frame based on the estimated motion.

A method includes: obtaining a first main image from a first real image and obtaining a second main image from a second real image, where the first real image and the second real image are images obtained through rendering by the electronic device; performing motion vector calculation on the first main image and the second main image, to obtain a motion vector; warping an environment in a target main image and a moving object in the target main image based on the motion vector, to generate an environment in a predicted frame and a moving object in the predicted frame; and multiplexing image data of a shadow of the moving object in the target main image into the predicted frame, to generate a shadow of the moving object in the predicted frame.

A method for interpolating video frames, includes: obtaining at least two key frames of a video, for which a motion estimation is to be performed, detecting repetitive pattern regions on the at least one key frame of the at least two key frames, estimating motion between the at least one key frame of the at least two key frames and the interpolated frame being interpolated by feeding the at least two key frames and the repetitive pattern regions to a trained motion estimation neural network.

Methods and apparatus for post-processing in-camera stabilized video. Embodiments of the present disclosure reconstruct and re-stabilize an in-camera stabilized video to provide for improved stabilization (e.g., a wider crop, etc.). In-camera sensor data may be stored and used to re-calculate orientation metadata in post-production. In-camera stabilization provides several benefits (e.g., the ability to share stabilized videos from the camera without additional post-processing as well as reduced file sizes of the shared videos). Camera-aware post-processing can reuse portions of the in-camera stabilized videos while providing additional benefits (e.g., the ability to regenerate the original captured videos in post-production and re-stabilize the videos). Camera-aware post-processing can also improve orientation metadata and remove sensor error. The disclosed techniques also enable assisted optical flow-based stabilization using the refined metadata.

In various examples, systems and methods are disclosed that perform motion detection across image frames, such as optical flow determination, to synchronize an asynchronous frame with respect to a target time for the asynchronous frame. For example, image frames from a sensor can be processed by an optical flow accelerator to detect displacement across the image frames, and the displacement can be used to interpolate a modified frame at the target time. This can be used to perform data collection and combining operations such as stitching and/or reconstruction. The synchronization can be performed from sensor data from sensors such as cameras, LIDAR sensors, and/or RADAR sensors.