In order to perform image processing in which relative positional deviation of image outputs between two image pickup units is suppressed, an image pickup apparatus comprises: an imaging device that has a first image pickup unit configured to perform imaging in a first cycle and generate a first image signal, and a second image pickup unit configured to perform imaging in a second cycle shorter than the first image pickup unit and generate a plurality of second image signals; a relation degree information generation unit configured to generate relation degree information between the first image signal and the second image signal based on overlap between an exposure time period in the first image pickup unit and an imaging timing in the cycle of the second image pickup unit; a region selection unit configured to select a specific region from among the second image signals based on the relation degree information generated by the relation degree information generation unit; and an image processing unit configured to perform predetermined image processing on the first image signal based on the specific region selected by the region selection unit.

An imaging device includes a light separator configured to separate light into a plurality of light bands. The imaging device further includes a plurality of imaging elements. Each of the plurality of imaging elements is configured to receive one of the plurality of light bands and to generate a corresponding signal. Each of the plurality of imaging elements has a pixel size for generating a 4K resolution image signal including at least 3840 horizontal pixels and at least 2160 vertical pixels on a display. A registration error among the plurality of imaging elements is equal to or less than a threshold of 20% of a pixel size in the plurality of image elements.

A photographing apparatus is disclosed. A photographing apparatus according to one embodiment comprises: a first image sensor; a second image sensor; and at least one processor functionally coupled to the first image sensor and the second image sensor, wherein the at least one processor may be configured to: obtain, by using the first image sensor, a first image which includes a first pixel, a second pixel adjacent to the first pixel, and a third pixel adjacent to the second pixel in an area other than the area in which the second pixel and the first pixel are adjacent; obtain, by using the second image sensor, a second image which includes a fourth pixel associated with the first pixel on the basis of the position thereof, and a fifth pixel adjacent to the fourth pixel and associated with the second pixel on the basis of the position thereof; determine whether a difference in luminance between the first pixel and the second pixel falls within a designated range; and, when the difference in the luminance between the first pixel and the second pixel falls within the designated range, generate color information corresponding to at least one of the first pixel and the second pixel at least on the basis of the color information of the fourth pixel and the color information of the fifth pixel.

Systems and methods for extended color processing on Pelican array cameras in accordance with embodiments of the invention are disclosed. In one embodiment, a method of generating a high resolution image includes obtaining input images, where a first set of images includes information in a first band of visible wavelengths and a second set of images includes information in a second band of visible wavelengths and non-visible wavelengths, determining an initial estimate by combining the first set of images into a first fused image, combining the second set of images into a second fused image, spatially registering the fused images, denoising the fused images using bilateral filters, normalizing the second fused image in the photometric reference space of the first fused image, combining the fused images, determining a high resolution image that when mapped through a forward imaging transformation matches the input images within at least one predetermined criterion.

An imaging device includes a light separator that separates light into light bands, and imaging elements that each receives one of the light bands and generates a corresponding signal. Each of the imaging elements has a pixel size of at most 2.5 m by 2.5 m. A registration error among the imaging elements is equal to or less than a threshold determined according to the pixel size.

An imaging device includes a light separator that separates light into light bands, and imaging elements that each receives one of the light bands and generates a corresponding signal. Each of the imaging elements has a pixel size of at most 2.5 m by 2.5 m. A registration error among the imaging elements is equal to or less than a threshold determined according to the pixel size.

The present invention provides an image processing device, an imaging apparatus, and an image processing method capable of acquiring images that are registered with high accuracy and include only a component of a desired wavelength range. In an image processing device according to an aspect of the invention, at least one image is captured with light having a plurality of wavelength ranges not overlapping one another (that is, including an intentional interference component in addition to a principal wavelength range), and at least one wavelength range of the plurality of wavelength ranges is common among the images. Accordingly, it is possible to detect correspondence points among a plurality of images based on information of such a common wavelength range, and to register the plurality of images with high accuracy. In addition, since an influence of light having a wavelength range other than a specific wavelength range is eliminated after registration, it is possible to acquire a plurality of images including only a component of a desired wavelength range.

In general, techniques are described that facilitate processing of color image data using both a mono image data and a color image data. A device comprising a monochrome camera, a color camera and a processor may be configured to perform the techniques. The monochrome camera may be configured to capture monochrome image data of a scene. The color camera may be configured to capture color image data of the scene. The processor may be configured to perform intensity equalization with respect to a luma component of either the color image data or the monochrome image data to correct for differences in intensity between the color camera and the monochrome camera.

Systems and methods for extended color processing on Pelican array cameras in accordance with embodiments of the invention are disclosed. In one embodiment, a method of generating a high resolution image includes obtaining input images, where a first set of images includes information in a first band of visible wavelengths and a second set of images includes information in a second band of visible wavelengths and non-visible wavelengths, determining an initial estimate by combining the first set of images into a first fused image, combining the second set of images into a second fused image, spatially registering the fused images, denoising the fused images using bilateral filters, normalizing the second fused image in the photometric reference space of the first fused image, combining the fused images, determining a high resolution image that when mapped through a forward imaging transformation matches the input images within at least one predetermined criterion.

In general, techniques are described that facilitate processing of color image data using both a mono image data and a color image data. A device comprising a monochrome camera, a color camera, and a processor may be configured to perform the techniques. The monochrome camera may capture monochrome image data of a scene. The color camera may capture color image data of the scene. A processor may determine a parallax value indicative of a level of parallax between the monochrome image data and the color image data and determine that the parallax is greater than the parallax threshold. The processor may further combine, in response to the determination that the parallax is greater than the parallax threshold, a luma component of the color image data with a luma component of the monochrome image data to generate a luma component of enhanced color image data.