The invalid pixel detection unit 32 detects an invalid pixel from a non-polarized image and a plurality of polarized images for different polarization directions obtained by performing image pickup using the polarization image pickup unit 20. For example, the invalid pixel detection unit 32 detects a saturated pixel having a pixel value larger than a preset saturation detection threshold and a black-crushed pixel having a pixel value smaller than a preset black crushing detection threshold as invalid pixels from a non-polarized image. The polarization information generation unit 33 performs processing of generating polarization information on the basis of the non-polarized image and polarized images and switches the processing of generating the polarization information depending on the detection result of the invalid pixels in the invalid pixel detection unit 32, so as to generate the polarization information without using the invalid pixels. Correct polarization information can be acquired.

Apparatuses, systems, and techniques to receive, at one or more processor associated with an image signal processing (ISP) pipeline, a compressed image generated by an image sensor, wherein the compressed image is captured at a first bit-depth associated with the image sensor and is compressed to a second bit-depth that is lower than the first bit-depth, and wherein the ISP is associated with a third bit-depth that is lower than the first bit-depth and higher than the second bit-depth; and decompress the compressed image according to a power curve to generate a partially decompressed image having the third bit-depth, wherein a plurality of regions of the partially decompressed image are decompressed at separate decompression amounts based on a corresponding pixel value of each region of the plurality of regions.

According to certain embodiments, an image sensor including a pixel, the pixel including a micro lens, a plurality of photodiodes, and a color filter disposed between the plurality of photodiodes and the micro lens; a processor operatively connected to the image sensor; and a memory operatively connected to the processor, wherein the memory stores one or more instructions that, when executed by the image sensor, cause the image sensor to perform a plurality of operations, the plurality of operations comprising: determining whether the image sensor is in a high-resolution mode; when the image sensor is in the high-resolution mode, calculating a disparity based on signals detected from the plurality of photodiodes; when the disparity is not greater than a threshold value, applying a first remosaic algorithm to the signals; and when the disparity is greater than the threshold value, applying a second remosaic algorithm to the signals.

There is provided with an image processing apparatus. A noise reduction unit generates a noise-reduced image in which noise is reduced from an input image in which a plurality of types of pixels that represent mutually different types of color information are arranged in one plane. An extraction unit generates a high-frequency emphasized image in which a high-frequency component of the input image is emphasized. A demosaicing unit generates a demosaiced image having a plurality of planes that each represent one type of color information by demosaicing processing to the noise-reduced image. A generation unit generates an output image by correcting the demosaiced image by using the high-frequency emphasized image.

An image processing device includes a memory, and a processor configured to execute obtaining image data from an imaging device in which pixel groups of multiple colors are repeatedly arranged, each of the pixel groups including multiple pixels; and determining a direction in which change in a pixel value is small at a position of a target pixel group, based on a changed amount of the pixel values of at least one of pairs of pixels that are arranged around the target pixel group, and included in multiple other pixel groups having colors that are different from a color of the target pixel group.

A solid-state imaging device includes a color filter unit disposed on a pixel array unit including pixels two-dimensionally arranged in a matrix and a conversion processing unit disposed on a substrate having the pixel array unit thereon. The color filter unit has a color arrangement in which a color serving as a primary component of a luminance signal is arranged in a checkerboard pattern and a plurality of colors serving as color information components are arranged in the other area of the checkerboard pattern. The conversion processing unit converts signals that are output from the pixels of the pixel array unit and that correspond to the color arrangement of the color filter unit into signals that correspond to a Bayer arrangement and outputs the converted signals.

Provided are an image composition apparatus for composing color images with black-and-white images including infrared components, and an image composition method thereof The image composition method includes generating a first image signal with color information and a second image signal including infrared components without color information, dividing the first image signal into a brightness signal and a color signal, composing the brightness signal of the first image signal with a brightness signal of the second image signal to generate a composed brightness signal, and composing the composed brightness signal with the color signal of the first image signal to generate a color image.

A lens and colour filter assembly contains lens units, and each lens unit is assigned to a single-colour colour filter unit. The lens and colour filter assembly may be combined with pixel units such that a plurality of monochromatic, low-resolution images can be obtained, and the monochromatic images refer to shifted versions of the same image object. By a super-resolution technique comprising shift-compensation a mosaicked image is obtained which is then demosaiced. In the resultant image only few artefacts appear. Simple colour filter arrays allow a simplified fabrication process and provide less chromatic aberrations at less computational effort.

A system for obtaining color imagery using SPADs includes a SPAD array that has a plurality of SPAD pixels. Each of the plurality of SPAD pixels includes a respective color filter positioned thereover. The system is configurable to capture an image frame using the SPAD array and generate a filtered image by performing a temporal filtering operation using the image frame and at least one preceding image frame. The at least one preceding image frame is captured by the SPAD array at a timepoint that temporally precedes a timepoint associated with the image frame. The system is also configurable to, after performing the temporal filtering operation, generate a color image by demosaicing the filtered image.

Embodiments of the invention provide a method and system that allows parameters of a desired target image to be determined from hyperspectral imagery of scene. The parameters may be representative of various aspects of the scene being imaged, particularly representative of physical properties of the scene. For example, in some medical imaging contexts, the property being imaged may be blood perfusion or oxygenation saturation level information per pixel. In one embodiment the parameters are obtained by collecting lower temporal and spatial resolution hyperspectral imagery, and then building a virtual hypercube of the information having a higher spatial resolution using a spatiospectral aware demosaicking process, the virtual hypercube then being used for estimation of the desired parameters at the higher spatial resolution. Alternatively, in another embodiment, instead of building the virtual hypercube and then performing the estimation, a joint demosaicking and parameter estimation operation is performed to obtain the parameters. Various white level and spectral calibration operations may also be performed to improve the results obtained. While establishing functional and technical requirements of an intraoperative system for surgery, we present iHSI system embodiments that allows for real-time wide-field HSI and responsive surgical guidance in a highly constrained operating theatre. Two exemplar embodiments exploiting state-of-the-art industrial HSI cameras, respectively using linescan and snapshot imaging technology, were investigated by performing assessments against established design criteria and ex vivo tissue experiments. We further report the use of one real-time iHSI embodiment during an ethically-approved in-patient clinical feasibility case study as part of a spinal fusion surgery therefore successfully validating our assumptions that our invention can be seamlessly integrated into the operating theatre without interrupting the surgical workflow.