A navigation system useful for providing speed and heading and other navigational data to a drive system of a moving body, e.g., a vehicle body or a mobile robot, to navigate through a space. The navigation system integrates an inertial navigation system, e.g., a unit or system based on an inertial measurement unit (IMU). with a vision-based navigation system unit or system such that the inertial navigation system can provide real time navigation data and the vision-based navigation can provide periodic, but more accurate, navigation data that is used to correct the inertial navigation system's output. The navigation system was designed with the goal in mind of providing low effort integration of inertial and video data. The methods and devices used in the new navigation system address problems associated with high accuracy dead reckoning systems (such as a typical vision-based navigation system) and enhance performance with low cost IMUs.

An apparatus configured to be head-worn by a user, includes: a screen configured to present graphics for the user; a camera system configured to view an environment in which the user is located; and a processing unit coupled to the camera system, the processing unit configured to: obtain locations of features for an image of the environment, wherein the locations of the features are identified by a neural network; determine a region of interest for one of the features in the image, the region of interest having a size that is less than a size of the image; and perform a corner detection using a corner detection algorithm to identify a corner in the region of interest.

Video image stabilization provides better performance on a generic platform for computing devices by evaluating available multimedia digital signal processing components, and selecting the available components to utilize according to a hierarchy structure for video stabilization performance for processing parts of the video stabilization. The video stabilization has improved motion vector estimation that employs refinement motion vector searching according to a pyramid block structure relationship starting from a downsampled resolution version of the video frames. The video stabilization also improves global motion transform estimation by performing a random sample consensus approach for processing the local motion vectors, and selection criteria for motion vector reliability. The video stabilization achieves the removal of hand shakiness smoothly by real-time one-pass or off-line two-pass temporal smoothing with error detection and correction.

According to an aspect, a method of position-based adjustment to display content is provided. The method includes determining a position of an observer relative to a display device. The method also includes determining a distortion correction to apply to a plurality of display content based on the position of the observer relative to the display device to correct the display content with respect to the observer. The distortion correction of the display content is output to the display device.

The present disclosure relates to methods and devices for facilitating enhancing the quality of video. An example method disclosed herein includes estimating an optical flow between a first noisy frame and a second noisy frame, the second noisy frame following the first noisy frame. The example method also includes warping a first enhanced frame to align with the second noisy frame, the warping being based on the estimation of the optical flow between the first noisy frame and the second noisy frame, the first enhanced frame being an enhanced frame of the first noisy frame. The example method also includes generating a second enhanced frame based on the warped first enhanced frame and the second noisy frame, and outputting the second enhanced frame.

A computer implemented method for warping multi-field color virtual content for sequential projection includes obtaining first and second color fields having different first and second colors. The method also includes determining a first time for projection of a warped first color field. The method further includes determining a second time for projection of a warped second color field. Moreover, the method includes predicting a first pose at the first time and predicting a second pose at the second time. In addition, the method includes generating the warped first color field by warping the first color field based on the first pose. The method also includes generating the warped second color field by warping the second color field based on the second pose.

A computer implemented method for warping virtual content from two sources includes a first source generating first virtual content based on a first pose. The method also includes a second source generating second virtual content based on a second pose. The method further includes a compositor processing the first and second virtual content in a single pass. Processing the first and second virtual content includes generating warped first virtual content by warping the first virtual content based on a third pose, generating warped second virtual content by warping the second virtual content based on the third pose, and generating output content by compositing the warped first and second virtual content.

A system of convolutional neural networks (CNNs) that synthesize middle non-existing frames from pairs of input frames includes a coarse CNN that receives a pair of images acquired at consecutive points of time, a registration module, a refinement CNN, an adder, and a motion-compensated frame interpolation (MC-FI) module. The coarse CNN outputs from the pair of images a previous feature map, a next feature map, a coarse interpolated motion vector field (IMVF) and an occlusion map, the registration module uses the coarse IMVF to warp the previous and next feature maps to be aligned with pixel locations of the IMVF frame, and outputs registered previous and next feature maps, the refinement CNN uses the registered previous and next feature maps to correct the coarse IMVF, and the adder sums the coarse IMVF with the correction and outputs a final IMVF.

Methods, systems, and non-transitory computer readable storage media are disclosed for applying piecewise deformations to digital content using a plurality of parametric patches. For example, the disclosed system generates a plurality of parametric patches (e.g., Bezier patches) within a parametric quilt for digital content (e.g., a digital image or digital text). The disclosed system also provides interface controls for user-defined split/patch locations for the parametric quilt. In one or more embodiments, the disclosed system divides digital content into a plurality of portions. The disclosed system modifies one or more parametric patches and deforms a corresponding portion(s) of the digital content based on the modified parametric patch(es). The disclosed system then recombines the portions of the digital content to generate modified digital content that includes any deformations based on the modified parametric patch(es).

A computer-implemented method of forecasting the semantic output of at least one frame, the method comprising the steps of receiving the input frames from a camera up to a predetermined time, processing via a down-sampling module of a neural network the plurality of input frames to receive a plurality of feature tensors, determining spatio-temporal correlations between the plurality of feature tensors, processing the plurality of feature tensors and the spatio-temporal correlations to receive at least one forecasted feature tensor, and processing via an up-sampling module of the neural network the at least one forecasted feature to receive at least one forecasted semantic output for a time larger than the predetermined time.