A method and apparatus of tracking poses of a rolling-shutter camera in an augmented reality (AR) system is provided. The method and apparatus use camera information and inertial sensor readings from Inertial Measurement Unit (IMU) to estimate the pose of the camera at a reference line. Thereafter, relative pose changes at scanlines may be calculated using the inertial sensor data. The estimated reference pose of the camera is then further refined based on the visual information from the camera, the relative pose changes and the optimized reference line pose of a previous image. Thereafter, the estimate of the scanline poses may be updated using the relative pose changes obtained in the earlier steps.

A system calibrates one or more sensors mounted to an autonomous vehicle. From the one or more sensors, the system identifies a primary sensor and a secondary sensor. The system determines a reference angle for the primary sensor, and based on that reference angle for the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan. The system receives, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time. The system receives, from the secondary sensor, a secondary set of sensor data recorded from the scan-start time to the scan-end time. The system calibrates the primary and secondary sensors by determining a relative transform for transforming points between the first set of scan data and the second set of scan data.

A viewing direction may define an angle/visual portion of a spherical video at which a viewing window is directed. A trajectory of viewing direction may include changes in viewing directions for playback of spherical video. Abrupt changes in the viewing directions may result in jerky or shaky views of the spherical video. Changes in the viewing directions may be stabilized to provide stabilized views of the spherical video. Amount of stabilization may be limited by a margin constraint.

A time centering module cooperates with an image sensor that is configured to capture two or more image captures at least one of 1) several different lengths of exposures of a same subject matter, 2) several different integration times for that same subject matter, and 3) any combination of these, that are to be merged into a High Dynamic Range (HDR) image capture. The time centering module is also configured to cooperate with data storage components. The time centering module correlates image data of a moving object from the two or more image captures stored in the data storage component. The image captures each have different integration times in a rolling shutter or different lengths of exposures in a global shutter in order to correlate the image data of the moving object in image captures by a midpoint in time of their respective image capture.

Techniques are disclosed for adding time data to scan lines of an image frame. In some examples, an image sensor may perform a rolling shutter image capture to produce the scan lines. Data captured by another sensor may be associated with at least a portion of a scan line based at least in part on the time data added to the scan line in some examples. Furthermore, techniques are disclosed for synchronizing data capture by multiple sensors. For example, a rolling shutter image capture performed by an image sensor may be synchronized with a data capture performed by another sensor.

A blur correction device includes a processor and a memory that is built into or coupled to the processor. The processor is configured to acquire an amount of blur correction used to correct blurring of an image obtained by imaging of an imaging element during exposure for one frame in the imaging element, and correct the blurring by performing image processing based on a most recently acquired amount of blur correction, on an unfinished image that is the image less than one frame that is being read from the imaging element. In a case in which a first reading period does not overlap with a second reading period, the processor corrects the blurring by performing the image processing based on the amount of blur correction acquired during exposure between the first reading period and the second reading period, on the unfinished image of the subsequent frame.

There is provided a notifying apparatus. A detecting unit detects a motion amount of an object from an image obtained through first shooting, the first shooting being carried out repeatedly at predetermined intervals of time. A converting unit converts the motion amount into a motion blur amount that will arise in second shooting, on the basis of the predetermined intervals of time and an exposure time used in the second shooting. A notifying unit makes a notification of motion blur on the basis of the motion blur amount. The notifying unit changes a form of the notification in accordance with a magnitude of the motion blur amount.

Systems, apparatus, and methods for stabilization and blending of exposures. So-called Electronic Image Stabilization (EIS) techniques use image manipulation software to compensate for camera motion. Unfortunately, existing EIS techniques cannot compensate for artifacts introduced by low shutter speed (e.g., blurs). Various embodiments of the present disclosure generate stabilized images from multiple exposures. In one exemplary embodiment, the stabilized exposures are blended using a linear sum of the color data for each pixel of the image. By stabilizing each exposure and linearly summing the light information, the camera shake can be removed, and the subject motion blur can be controlled. The stabilization and blending techniques enable a mathematical emulation of a selected shutter angle from many high-speed exposures.

An error estimation device includes an error prediction unit and a determination unit. The error prediction unit is configured to predict a bias error occurring in an inertial sensor mounted in a vehicle. The determination unit is configured to determine whether the bias error needs to be repredicted by the error prediction unit based on a reflection distance image acquired by an optical sensor in a rolling shutter mode and an outside light image acquired by an external camera in a global shutter mode.

A system for generating high-resolution video from low-resolution images is configured to access a first video stream and a second video stream capturing an environment. The first video stream is captured by a first video capture device. The second video stream is captured by a second video capture device. Image frames of the first video stream are temporally synchronized with corresponding image frames of the second video stream. The system is also configured to generate a composite video stream with a higher resolution than the first or second video streams. Each composite image frame of the composite video stream is generated using a respective image frame of the first video stream and a temporally synchronized corresponding image frame of the second video stream as input.