In various examples, image processing techniques are presented for reducing artifacts in images for autonomous or semi-autonomous systems and applications. Systems and methods are disclosed for deferring at least a portion of a strength associated with a color correction matrix (CCM) of an image processing pipeline to one or more other stages associated with the image processing pipeline. For instance, the CCM is used to determine a first CCM and a second, deferred CCM. The first CCM may be associated with an earlier stage of the image processing pipeline while the deferred CCM is associated with a later stage of the image processing pipeline. In some examples, the deferred CCM is combined with another matrix, such as a color space conversion matrix. By deferring part of the CCM, the image processing pipeline may reduce a number of artifacts and/or eliminate the artifacts that occur when processing image data.

Systems and methods of the invention merge information from multiple image sensors to provide a high dynamic range (HDR) video. The present invention provides for real-time HDR video production using multiple sensors and pipeline processing techniques. According to the invention, multiple sensors with different exposures each produces an ordered stream of frame-independent pixel values. The pixel values are streamed through a pipeline on a processing device. The pipeline includes a kernel operation that identifies saturated ones of the pixel values. The streams of pixel values are merged to produce an HDR video.

The invention provides methods for broadcasting video in a dual HDR/LDR format such that the video can be displayed in real time by both LDR and HDR display devices. Methods and devices of the invention process streams of pixels from multiple sensors in a frame-independent manner to produce an HDR video signal in real time. That HDR video signal is then tone-mapped to produce an LDR video signal, the LDR signal is subtracted from the HDR signal to calculate a residual signal, and the LDR signal and the residual signal are merged into a combined signal that is broadcast via a communications network.

Techniques are provided to encode and decode image data comprising a tone mapped (TM) image with HDR reconstruction data in the form of luminance ratios and color residual values. In an example embodiment, luminance ratio values and residual values in color channels of a color space are generated on an individual pixel basis based on a high dynamic range (HDR) image and a derivative tone-mapped (TM) image that comprises one or more color alterations that would not be recoverable from the TM image with a luminance ratio image. The TM image with HDR reconstruction data derived from the luminance ratio values and the color-channel residual values may be outputted in an image file to a downstream device, for example, for decoding, rendering, and/or storing. The image file may be decoded to generate a restored HDR image free of the color alterations.

An imaging device may have a monochrome image sensor and a bi-chromatic image sensor. A beam splitter may split incident light between the two image sensors. The monochrome image sensor may have an array of broadband image sensor pixels that generate broadband image signals. The bi-chromatic image sensor may have an array of red and blue image pixels that generate red and blue image signals. The image sensors may be coupled to processing circuitry that performs processing operations on only the broadband image signals to produce monochrome images, or on the red, blue, and broadband image signals to produce color images. Processing operations used to produce color images may include chroma-demosaicking and/or point filter operations.

Video processing techniques and pipelines that support capture, distribution, and display of high dynamic range (HDR) image data to both HDR-enabled display devices and display devices that do not support HDR imaging. A sensor pipeline may generate standard dynamic range (SDR) data from HDR data captured by a sensor using tone mapping, for example local tone mapping. Information used to generate the SDR data may be provided to a display pipeline as metadata with the generated SDR data. If a target display does not support HDR imaging, the SDR data may be directly rendered by the display pipeline. If the target display does support HDR imaging, then an inverse mapping technique may be applied to the SDR data according to the metadata to render HDR data for display. Information used in performing color gamut mapping may also be provided in the metadata and used to recover clipped colors for display.

Matching color information in an optical system can include splitting an image forming beam into a bright intensity beam and a dark intensity beam, detecting, using multiple sensors, a color value for a light component from the bright intensity beam and the dark intensity beam, determining color values for the remaining light components associated with the bright intensity beam and the dark intensity beam, and transforming the color values associated with the dark intensity beam to calibrate the color values of the dark intensity beam against the color values of the light intensity beam, the color values of the light intensity beam including color inaccuracies.

An electronic device includes an image sensor including a color filter having a plurality of color channels, a memory storing a plurality of color conversion matrices and instructions, and a processor. The processor is configured to obtain noise information of a color image captured by the image sensor, select a target matrix from among the plurality of the color conversion matrices based on the obtained noise information, and generate a color converted image by applying the selected target matrix to the color image.

Embodiments of this application relate to the technical field of video shooting and provide a video processing method and apparatus, an electronic device, and a storage medium. The video processing method includes: in a first night mode or in a second night mode, obtaining a video shot through a camera lens; in the first night mode, processing a video image of a first exposure frame, where processing is used for reducing noise and increasing a luminance; in the first night mode, fusing a video image of a first exposure frame processed and the video image of a second exposure frame not processed, to obtain a fused video; and in the second night mode, fusing the video image of a first exposure frame and the video image of a second exposure frame, to obtain a fused video.

The invention provides an autonomous vehicle with a video camera that merges images taken a different light levels by replacing saturated parts of an image with corresponding parts of a lower-light image to stream a video with a dynamic range that extends to include very low-light and very intensely lit parts of a scene. The high dynamic range (HDR) camera streams the HDR video to a HDR system in real timeas the vehicle operates. As pixel values are provided by the camera's image sensors, those values are streamed directly through a pipeline processing operation and on to the HDR system without any requirement to wait and collect entire images, or frames, before using the video information.