In the advanced driver-assistance systems (ADAS) field, RAW sensor image processing for machine vision (MV) applications can be of critical importance. Due to red/green/blue (RGB) image components being focused by the lens at different locations in image plane, the lateral chromatic aberration (LCA) phenomenon may sometimes be observed, which causes false color around edges in the final image output, especially for high contrast edges, which can impede MV applications. Disclosed herein are low-latency, efficient, optimized designs for chromatic aberration correction (CAC) modules. In some embodiments, an in-pipeline CAC design is used that: is configured to perform on-the-fly CAC without any out-of-pipeline memory traffic; enables use of wide dynamic range (WDR) sensors; uses bicubic interpolation; supports vertical and horizontal chromatic aberration red/blue color channel offsets, reduces CAC line memory requirements, and supports flexible look-up table (LUT) down-sampling factors to improve the spatial precision of correction and accommodate popular image sensor resolutions.

In the advanced driver-assistance systems (ADAS) field, RAW sensor image processing for machine vision (MV) applications can be of critical importance. Due to red/green/blue (RGB) image components being focused by the lens at different locations in image plane, the lateral chromatic aberration (LCA) phenomenon may sometimes be observed, which causes false color around edges in the final image output, especially for high contrast edges, which can impede MV applications. Disclosed herein are low-latency, efficient, optimized designs for chromatic aberration correction (CAC) modules. In some embodiments, an in-pipeline CAC design is used that: is configured to perform on-the-fly CAC without any out-of-pipeline memory traffic; enables use of wide dynamic range (WDR) sensors; uses bicubic interpolation; supports vertical and horizontal chromatic aberration red/blue color channel offsets, reduces CAC line memory requirements, and supports flexible look-up table (LUT) down-sampling factors to improve the spatial precision of correction and accommodate popular image sensor resolutions.

Provided are a color separation element and an image sensor including the same. The color separation element includes a spacer layer; and a color separation lens array, which includes at least one nano-post arranged in the spacer layer and is configured to form a phase distribution for splitting and focusing incident light according to wavelengths, wherein periodic regions in which color separation lens arrays are repeatedly arranged are provided, and the color separation lens array is configured to interrupt phase distribution at the boundary of the periodic regions.

An eye tracking system comprising a controller configured to receive a reference image of an eye of a user and a current image of the eye of the user. The controller is also configured to determine a difference between the reference image and the current image to define a differential image. The differential image has a two dimensional pixel array of pixel locations that are arranged in a plurality of rows and columns. Each pixel location has a differential intensity value. The controller is further configured to calculate a plurality of row values by combining the differential intensity values in corresponding rows of the differential image and to determine eyelid data based on the plurality of row values.

An imaging device includes a pixel array of 1×3 pixel circuits that include 3 photodiodes in a column. Bitlines are coupled to the 1×3 pixel circuits. The bitlines are divided into groupings of 3 bitlines per column of the 1×3 pixel circuits. Each column of the 1×3 pixel circuits includes a plurality of first banks coupled to a first bitline, a plurality of second banks coupled to a second bitline, and a plurality of third banks coupled to a third bitline of a respective grouping of the 3 bitlines. The 1×3 pixel circuits are arranged into groupings of 3 1×3 pixel circuits per nine cell pixel structures that form a plurality of 3×3 pixel structures of the pixel array.

An image sensor includes a pixel array, wherein the pixel array includes a first unit pixel including first sensing pixels adjacent along a column direction and second sensing pixels adjacent along the column direction, the first sensing pixels and the second sensing pixels being adjacent along a row direction, and a same color filter overlapping first and second sensing pixels. The first sensing pixels share a first floating diffusion node. The second sensing pixels share a second floating diffusion node.

Systems and methods for performing direct conversion of image sensor data to image analytics are provided. One such system for directly processing sensor image data includes a sensor configured to capture an image and generate corresponding image data in a raw Bayer format, and a convolution neural network (CNN) coupled to the sensor and configured to generate image analytics directly from the image data in the raw Bayer format. Systems and methods for training the CNN are provided, and may include a generative model that is configured to convert RGB images into estimated images in the raw Bayer format.

Systems and methods for performing direct conversion of image sensor data to image analytics are provided. One such system for directly processing sensor image data includes a sensor configured to capture an image and generate corresponding image data in a raw Bayer format, and a convolution neural network (CNN) coupled to the sensor and configured to generate image analytics directly from the image data in the raw Bayer format. Systems and methods for training the CNN are provided, and may include a generative model that is configured to convert RGB images into estimated images in the raw Bayer format.

Described herein are embodiments of a deep residual network dedicated to color filter array mosaic patterns. A mosaic stride convolution layer is introduced to match the mosaic pattern of a multispectral filter arrays (MSFA) or a color filter array raw image. Embodiments of a data augmentation using MSFA shifting and dynamic noise are applied to make the model robust to different noise levels. Embodiments of network optimization criteria may be created by using the noise standard deviation to normalize the L.sup.1 loss function. Comprehensive experiments demonstrate that embodiments of the disclosed deep residual network outperform the state-of-the-art denoising algorithms in MSFA field.

A backside illumination image sensor that includes a semiconductor substrate with a plurality of photoelectric conversion elements and a read circuit formed on a front surface side of the semiconductor substrate, and captures an image by outputting, via the read circuit, electrical signals generated as incident light having reached a back surface side of the semiconductor substrate is received at the photoelectric conversion elements includes: a light shielding film formed on a side where incident light enters the photoelectric conversion elements, with an opening formed therein in correspondence to each photoelectric conversion element, and an on-chip lens formed at a position set apart from the light shielding film by a predetermined distance in correspondence to each photoelectric conversion element. The light shielding film and an exit pupil plane of the image forming optical system achieve a conjugate relation to each other with regard to the on-chip lens.