An image sensor has a plurality of pixels arranged in a row direction and in a column direction. Each pixel comprises a color filter that has a portion with a low transmissivity and a portion with a high transmissivity, and a photoelectric conversion element that includes a first photoelectric conversion cell which receives light transmitting through the portion with the low transmissivity of the color filter, and a second photoelectric conversion cell which receives light transmitting through the portion with the high transmissivity of the color filter. The plurality of pixels are arranged such that positions of the portions with the low transmissivity for pixels of one color are identical among the plurality of pixels, and the portions with the low transmissivity are positioned adjacent to each other between adjacent pixels of different colors in the row direction only.

There is provided in one embodiment an apparatus having an image sensor array. In one embodiment, the image sensor array can include monochrome pixels and color sensitive pixels. The monochrome pixels can be pixels without wavelength selective color filter elements. The color sensitive pixels can include wavelength selective color filter elements.

An image acquisition apparatus includes: a multispectral image sensor configured to acquire images in at least four channels based on a second wavelength band of about 10 nm to about 1,000 nm; and a processor configured to estimate illumination information of the images by inputting the images of at least four channels to a deep learning network trained in advance, and convert colors of the acquired images using the estimated illumination information.

Techniques for using motion data to generate a high resolution output color image from multiple images having sparse color information are disclosed. A camera generates multiple images. The camera's sensor is configured to have a sparse Bayer pattern. While the camera is generating the images, IMU data for each image is acquired. The IMU data indicates a corresponding pose the camera was in while the camera generated each image. The images and the IMU data are fed as input into a motion model. The motion model performs temporal filtering on the images and uses the IMU data to generate a red-only image, a green-only image, and a blue-only image. A high resolution output color image is generated by combining the red-only image, the green-only image, and the blue-only image.

An image sensor includes a two-dimensional pixel array and a lens array. The two-dimensional pixel array comprises a plurality of pixels. Some of the pixels includes two sub-pixels. A rectangular coordinate is established by taking the pixel as an origin, a length direction of the two-dimensional pixel array as an x-axis, and a width direction of the two-dimensional pixel array as a y-axis. The two sub-pixels lie in both a positive half axis and a negative half axis of the x-axis and lies in both a positive half axis and a negative half axis of the y-axis. The lens array comprises a plurality of lenses, each covering one of the pixels.

Techniques are described for efficient high-resolution output of an image captured using a high-pixel-count image sensor based on pixel binning followed by luminance-guided umsampling. For example, an image sensor array is configured according to a red-green-blue-luminance (RGBL) CFA pattern, such that at least 50-percent of the imaging pixels of the array are luminance (L) pixels. Pixel binning is used during readout of the array to concurrently generate a downsampled RGB capture frame and a downsampled L capture frame. Following the readout, the L capture frame is upsampled (e.g., by upscaling and interpolation) to generate an L guide frame with 100-percent luminance density. An upsampled RGB frame can then be generated by interpolating the RGB capture frame based both on known neighboring RGB information (e.g., from the RGB capture frame and previously interpolated information), as adjusted based on local luminance information from the L guide frame.

An imaging apparatus includes a storage portion that stores captured image data obtained by imaging a subject by an imaging element and is incorporated in the imaging element, an output portion that is incorporated in the imaging element, and a plurality of signal processing portions that are disposed outside the imaging element, in which the output portion includes a plurality of output lines each disposed in correspondence with each of the plurality of signal processing portions and outputs each of a plurality of pieces of image data into which the captured image data stored in the storage portion is divided, to a corresponding signal processing portion among the plurality of signal processing portions from the plurality of output lines, and any of the plurality of signal processing portions combines the plurality of pieces of image data.

A camera device according to an embodiment may include: an image sensor which generates first Bayer data having a first resolution; and a processor which performs deep learning on the basis of the first Bayer data to output second Bayer data having a second resolution higher than the first resolution.

The disclosure extends to methods, systems, and computer program products for producing an image in light deficient environments and associated structures, methods and features. The features of the systems and methods described herein may include providing improved resolution and color reproduction.

Provided are a device and a method for executing gain calculation processing and gain adjustment processing for matching an output of an imaging element of a multispectral camera with an output of a reference machine. As the gain calculation processing for matching an output of an adjustment camera with an output of a reference camera at the time of manufacturing the multispectral camera, a band-corresponding gain is calculated that matches the output of the adjustment camera with the output of the reference camera, on the basis of: a reference machine band-corresponding pixel value that is a pixel value within a specific band acquired on the basis of an output value of an imaging element of the reference camera; and an adjustment machine band-corresponding pixel value that is a pixel value within a specific band acquired on the basis of an output value of an imaging element of the adjustment camera. Furthermore, at the time of using the camera, output value adjustment processing that matches the output of the imaging element with the output of the reference machine is executed, by acquiring the band-corresponding gain from a memory, and multiplying the output of the imaging element by the acquired band-corresponding gain.