A system calibrates one or more sensors mounted to an autonomous vehicle. From the one or more sensors, the system identifies a primary sensor and a secondary sensor. The system determines a reference angle for the primary sensor, and based on that reference angle for the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan. The system receives, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time. The system receives, from the secondary sensor, a secondary set of sensor data recorded from the scan-start time to the scan-end time. The system calibrates the primary and secondary sensors by determining a relative transform for transforming points between the first set of scan data and the second set of scan data.

A computer-implemented method of determining relative velocity between a vehicle and an object. The method includes receiving sensor data generated by one or more sensors of the vehicle. The one or more sensors are configured to sense an environment through which the vehicle is moving by following a scan pattern comprising component scan lines. The method includes obtaining, by one or more processors, a point cloud frame based on the sensor data and representative of the environment and identifying, by the one or more processors, a point cloud object within the point cloud frame. The method further includes determining, by the one or more processors, that the point cloud object is skewed relative to an expected configuration of the point cloud object, and determining, by the one or more processors, a relative velocity of the point cloud object by analyzing the skew of the object.

There is provided a notifying apparatus. A detecting unit detects a motion amount of an object from an image obtained through first shooting, the first shooting being carried out repeatedly at predetermined intervals of time. A converting unit converts the motion amount into a motion blur amount that will arise in second shooting, on the basis of the predetermined intervals of time and an exposure time used in the second shooting. A notifying unit makes a notification of motion blur on the basis of the motion blur amount. The notifying unit changes a form of the notification in accordance with a magnitude of the motion blur amount.

Techniques are disclosed for adding time data to scan lines of an image frame. In some examples, an image sensor may perform a rolling shutter image capture to produce the scan lines. Data captured by another sensor may be associated with at least a portion of a scan line based at least in part on the time data added to the scan line in some examples. Furthermore, techniques are disclosed for synchronizing data capture by multiple sensors. For example, a rolling shutter image capture performed by an image sensor may be synchronized with a data capture performed by another sensor.

Synchronization of a frame rate to a detected cadence includes receiving a sequence of image frames. Motion data recorded contemporaneously with a capture of the sequence of image frames are also received. At least some of the motion data are converted from time domain data to frequency domain data. A dominant frequency in the frequency domain data is determined. Frames from the sequence of image frames are sampled at a sampling frequency related to the dominant frequency. A new image sequence is created using the sampled frames.

The present invention provides an anti-shake method for a panoramic video, which comprises: obtaining, in real time, an output video frame, a fisheye image corresponding thereto, a pixel timestamp in the video frame, and a corresponding camera gyroscope timestamp; synchronizing the pixel timestamp in the video frame with the corresponding camera gyroscope timestamp, and calculating a rotation matrix of the camera movement in the camera gyroscope timestamp; smoothing the camera movement and establishing a coordinate system of a smooth trajectory; correcting the fisheye image distortion; and rendering the fisheye image by means of forward rendering to generate a stable video. According to the present invention, the rolling shutter distortion of a panoramic video sequence can be corrected, the image distortion caused by the rolling shutter of a CMOS can be corrected, and the rolling shutter is eliminated, thereby achieving a better anti-shake effect of a video image.

A method for rolling shutter compensation for a camera sensor mounted on a moving vehicle includes estimating, based on a plurality of images of an object, a speed and a direction of movement of the vehicle; acquiring an additional image of the object having four corners; estimating a location of each of the four corners of the object in an image plane defined by the additional image; determining a corrected location in 3D space for each of the four corners of the object; determining a first compensated location for each of the four corners of the object in the image plane; determining a second compensated location for each of the four corners of the object in the image plane; and determining a difference between the first compensated locations of the four corners of the object and the second compensated locations of the four corners of the object.

A method to reduce a motion blur and a rolling shutter effect, comprising, receiving a main image frame, a main timing and a main exposure from a main camera, receiving a secondary image frame, a secondary timing and a secondary exposure from a secondary camera, correcting the secondary image frame to the main image frame, determining a delta timing based on the main timing and the secondary timing, determining a delta exposure based on the main exposure and the secondary exposure, determining a discrete motion of offset sequences based on the secondary image frame, determining a row-wise motion blur kernel based on the main image frame and the discrete motion of offset sequences and determining a spatially varying kernel deconvolution based on the main image frame and the row-wise motion blur kernel.

A frame rate is synchronized to a detected cadence in order to generate an output image sequence that is substantially stabilized. In an in-camera process, a camera receives motion data of the camera while the camera captures the sequence of image frames. A dominant frequency of motion is determined and the capture frame rate is dynamically adjusted to match the frequency of detected motion so that each image frame is captured when the camera is at approximately the same position along the axis of motion. Alternatively, in a post-processing process, frames of a captured image sequence are selectively sampled at a sampling rate corresponding to the dominant frequency of motion so that each sampled frame corresponds to an image capture that occurred when the camera is at approximately the same position along the axis of motion.

An image generation method is provided. The image generation method includes the steps of acquiring a planimetric taken input image, and segmenting the planimetric taken input image into a plurality of planimetric image regions; acquiring a region exposure time point of each planimetric image region and a lens attitude of a corresponding taking lens at the region exposure time point; correcting a planimetric position coordinate of each planimetric image region according to the lens attitude of the taking lens at the region exposure time point and a lens attitude of the taking lens at an intermediate time point of image exposure; and generating a planimetric taken output image based on the corrected planimetric position coordinates of the planimetric image regions.