The present disclosure relates to an image pickup apparatus, an image pickup method, a program, and an image processing apparatus that enable a color image to be generated on the basis of an infrared image and a visible image captured using an image pickup device that uses a focal plane reading system.

An image pickup apparatus according to the present disclosure includes: an image pickup device that generates, in a one frame period, sub-frame images in a number corresponding to 3 or more sub-frame periods; an infrared light irradiation unit that turns on/off irradiation of infrared light onto an image pickup range in a time length unit that is the same as the sub-frame period in the one frame period; and a color image generation unit that generates a color image in a predetermined frame rate on the basis of an infrared image based on the sub-frame image in which a period during which the infrared light is irradiated is included in an exposure time and a visible image based on the sub-frame image in which the period during which the infrared light is irradiated is not included in the exposure time. The present disclosure is applicable to a surveillance camera, for example.

An in-vehicle camera is provide which has a case and a lens, and which is attached in a vehicle interior so that the lens is exposed at a top face of the case and the top face is opposed to a windshield or another window. The top face has an angular shape bent at a ridge line passing through the top face. The lens is positioned in the vicinity of the ridge line. The in-vehicle camera includes a hood attached to a front portion of the case, the front portion being positioned at a front side of the case with respect to the lens.

A multi-aperture imaging system comprising a first camera with a first sensor that captures a first image and a second camera with a second sensor that captures a second image, the two cameras having either identical or different FOVs. The first sensor may have a standard color filter array (CFA) covering one sensor section and a non-standard color CFA covering another. The second sensor may have either Clear or standard CFA covered sections. Either image may be chosen to be a primary or an auxiliary image, based on a zoom factor. An output image with a point of view determined by the primary image is obtained by registering the auxiliary image to the primary image.

According to the invention, an image sensor is disclosed. The image sensor may include a plurality of pixels. Each pixel of a first portion of the plurality of pixels may include a near-infrared filter configured to block red, green, and blue light; and pass near-infrared light. Each pixel of a second portion of the plurality of pixels may be configured to receive at least one of red, green, or blue light; and receive near-infrared light.

Various embodiments include methods and apparatuses for forming and using pixels for image sensors. In one embodiment, an image sensor having at least two pixel electrodes per color region, and having at least two modes is disclosed. The example image sensor includes a first, unbinned, mode; and a second, binned, mode. In the first, unbinned mode, the at least two pixel electrodes per color region are to be reset to substantially similar levels. In the second, binned mode, a first pixel electrode of the at the least two pixel electrodes is to be reset to a high voltage that results in efficient collection of photocharge, and a second pixel electrode of the at the least two pixel electrodes is to be reset to a low voltage that results in less efficient collection of photocharge. Other methods and apparatuses are disclosed.

Provided are a pixel interpolation apparatus, imaging apparatus, program, and integrated circuit that appropriately perform pixel interpolation on an image signal obtained by a single-chip image sensor with a four-color array filter whatever array pattern the four-color array filter has. An imaging apparatus includes an imaging circuitry, a signal processing circuitry, and a pixel interpolation processing circuitry. The apparatus calculates a degree of correlation for pairs in two orthogonal directions for an image signal obtained by the imaging circuitry including a single-chip image sensor having a four-color array filter using pixel data in an area around a target pixel, and performs pixel interpolation using the correlation degree as a determination criterion. When a color component pixel with a color identical to a color of a color component pixel subjected to pixel interpolation is not located in the direction having the high correlation, the pixel interpolation apparatus obtains a change ratio in a direction orthogonal to the direction having the high correlation by using a pixel value resulting from color space conversion in the direction orthogonal to the direction having the high correlation, and performs pixel interpolation processing based on the change ratio.

A color filter array for an image sensing device is disclosed. The color filter array includes a plurality of pixels and a control unit. The plurality of pixels is utilized for generating a plurality of pixel data of an image. The control unit is utilized for controlling the plurality of pixels. In addition, each of the plurality of pixels is divided into a plurality of sub-pixels corresponding to the same color. When outputting the plurality of pixel data, each of the plurality of pixels accumulates pixel value of at least one of the plurality of sub-pixels in each of the plurality of pixels as the pixel data outputted by each of the plurality of pixels.

Methods of fixing defective pixels are described wherein a predicted value for a target pixel in a target color channel is determined based on the values of nearby pixels, wherein the target pixel value can be selectively replaced with the predicted value. The predicted value is determined by determining a candidate value for each of a plurality of directions using: (i) a gradient of pixel values in one color channel along the respective direction and (ii) a pixel value of a pixel in the target color channel which is aligned with the target pixel along the respective direction. Using gradients can provide better predicted values than averaging nearby pixel values since rates of change of pixel values are taken into account. The median of the candidate values may be used in order to reduce the impact of other defective pixels on the predicted value for the target pixel.

To facilitate design and development of an apparatus that processes images.

An image processing apparatus includes a separation unit, a Bayer image signal supply unit, and a signal processing unit. The separation unit separates and removes, in input image signals in which pixel signals each including an invisible light component are arranged in an array different from a Bayer array, the invisible light components from the pixel signals. The Bayer image signal supply unit arranges the pixel signals from which the invisible light components have been removed in the Bayer array and supplies the pixel signals as Bayer image signals. The signal processing unit subjects the Bayer image signals to predetermined signal processing.

An apparatus includes a first sensor unit that includes a plurality of first sensors arranged in the first direction which include a first sensor configured to receive a first image formed by light with a first wavelength, a second sensor unit that includes a plurality of second sensors arranged in the first direction which include a second sensor configured to receive a second image formed by light with a second wavelength , and a controller configured to control the first and second sensor units. The controller controls the plurality of first sensors under a first common exposure condition, and controls the plurality of second sensors in the second sensor unit under a second common exposure condition.