Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. Lens stack arrays in accordance with many embodiments of the invention include lens elements formed on substrates separated by spacers, where the lens elements, substrates and spacers are configured to form a plurality of optical channels, at least one aperture located within each optical channel, at least one spectral filter located within each optical channel, where each spectral filter is configured to pass a specific spectral band of light, and light blocking materials located within the lens stack array to optically isolate the optical channels.

Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

A robot vision system for generating a 3D point cloud of a surrounding environment through comparison of unfiltered and filtered images of the surrounding environment. A filtered image is captured using a camera filter which tends to pass certain wavelength bandwidths while mitigating the passage of other bandwidths. A processor receives the unfiltered and filtered images, pixel matches the unfiltered and filtered images, and determines an image distance for each pixel based on comparing the color coordinates determined for that pixel in the unfiltered and filtered image. The image distances determined provides a relative distance from the digital camera to an object or object portion captured by each pixel, and the relative magnitude of all image distances determined for all pixels in the unfiltered and filtered images allows generation of a 3D point cloud representing the object captured in the unfiltered and filtered images.

Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

Imager arrays, array camera modules, and array cameras in accordance with embodiments of the invention utilize pixel apertures to control the amount of aliasing present in captured images of a scene. One embodiment includes a plurality of focal planes, control circuitry configured to control the capture of image information by the pixels within the focal planes, and sampling circuitry configured to convert pixel outputs into digital pixel data. In addition, the pixels in the plurality of focal planes include a pixel stack including a microlens and an active area, where light incident on the surface of the microlens is focused onto the active area by the microlens and the active area samples the incident light to capture image information, and the pixel stack defines a pixel area and includes a pixel aperture, where the size of the pixel apertures is smaller than the pixel area.

A system, method, and computer program product for generating a digital image is disclosed. The system includes a first image sensor configured to capture a first image that includes a plurality of chrominance values, a second image sensor configured to capture a second image that includes a plurality of luminance values, and an image processing subsystem configured to generate a resulting image by combining the plurality of chrominance values with the plurality of luminance values. The first image sensor and the second image sensor may be distinct image sensors optimized for capturing chrominance images or luminance images.

A method of operating an optical system is described. An image is detected. A digital image signal based on a spatially varying luminance level of the detected image is received. Horizon data, separate from the digital image signal, indicating the position of a horizon where a sky is adjacent the horizon, is received. The position of the horizon is determined based on the received horizon data. A fused image signal is provided based on the received digital image signal and the determined position of the horizon where a sky region indicative of the sky is provided with an enhanced color fused with the varying luminance level of the detected image. A full color display displays a fused image based on the fused image signal. A corresponding optical system is also described.

A head-mounted display (101) includes: an imaging assembly (203) imaging scenery seen by a user to generate a source picture; a storage unit (202) storing color correction factors used for correcting brightness of each of a red, green and blue color components included in a picture; a correction picture generator (212) performing color correction processing for enhancing a color component with a relatively low color correction factor stored in the storage unit (202), of the red, green and blue color components forming the source picture, to generate a correction picture; a picture display assembly (207) displaying the correction picture in the field of view of the user under a condition where he/she is able to perceive the outside world; and a special picture processor (213) performing picture processing for overlaying the correction picture and the source picture on each other for display.

An imaging device may have a monochrome image sensor and a bi-chromatic image sensor. A beam splitter may split incident light between the two image sensors. The monochrome image sensor may have an array of broadband image sensor pixels that generate broadband image signals. The bi-chromatic image sensor may have an array of red and blue image pixels that generate red and blue image signals. The image sensors may be coupled to processing circuitry that performs processing operations on only the broadband image signals to produce monochrome images, or on the red, blue, and broadband image signals to produce color images. Processing operations used to produce color images may include chroma-demosaicking and/or point filter operations.

A dual-aperture zoom camera comprising a Wide camera with a respective Wide lens and a Tele camera with a respective Tele lens, the Wide and Tele cameras mounted directly on a single printed circuit board, wherein the Wide and Tele lenses have respective effective focal lengths EFL.sub.W and EFL.sub.T and respective total track lengths TTL.sub.W and TTL.sub.T and wherein TTL.sub.W/EFL.sub.W>1.1 and TTL.sub.T/EFL.sub.T<1.0. Optionally, the dual-aperture zoom camera may further comprise an optical OIS controller configured to provide a compensation lens movement according to a user-defined zoom factor (ZF) and a camera tilt (CT) through LMV=CT*EFL.sub.ZF, where EFL.sub.ZF is a zoom-factor dependent effective focal length.