Image Processing System Creating A Field Sequential Color Using Delta Sigma Pulse Density Modulation For A Digital Display

Abstract

A device and method of an image processing system where a Field Sequential Color Delta Sigma Pulse Density Modulation is used for digital displays, where the digital displays are non-emissive. The device and method are a digital driving solution using Delta Sigma Encoding where N bit-per-component symbols at F1 frame-rate-per-second are represented using M bits-per-component symbols at F2 frame-rate-per-second, where N≥M and F2≥F1. The F2 frames are sent to a sequential color picker, which outputs frames with one color, followed by the next in a sequential pattern which reduces power consumption, increases color saturation, increases contrast, and increases brightness.

Claims

1. An image processing system comprising an oversampling module wherein an N bits-per-component image or video is converted to an M bits-per-component Field Sequential Color (FSC) image or video using an oversampling frequency, wherein the oversampling frequency is a ratio of an incoming video frequency and a refresh frequency of a display.

2. The image processing system of claim 1, wherein the N bits-per-component image or video is displayed on the display over time after being converted to the M bits-per-component FSC image or video.

3. The image processing system of claim 1, wherein the N bits-per-component image or video displays High Dynamic Range (HDR) content and M bits-per-component at a frequency F2 creates an equivalent to image or video having N bits-per-component at a frequency F1 wherein N bits-per-component at the frequency F2 is not possible to achieve due to display driver constraints.

4. The image processing system of claim 1, wherein a reduced power display is created by setting M to be less than or equal to N and displaying the FSC image or video.

5. The image processing system of claim 1, wherein at least one of a brightness, a contrast, and a Pixel Per Inch (PPI) of the display is increased by setting M is less than or equal to N and creating an equivalent image to an image having N bits-per-component and displaying the FSC image or video.

6. The image processing system of claim 1, wherein a Color Breakup (CBU) of the display is reduced by setting M is less than or equal to N and creating an equivalent image to an image having N bits-per-component and displaying the FSC image or video.

7. The image processing system of claim 1, wherein the display or an FSC stack for the display is devoid of any color filter.

8. The image processing system of claim 7, wherein the display is a liquid crystal display.

9. The image processing system of claim 7, further comprising a column driver controlling colors separately.

10. The image processing system of claim 1, further comprising at least one Blue LED, wherein the at least one Blue LED has a wavelength being greater than 455 nm.

11. The image processing system of claim 1, further comprising a sequential color picker.

12. The image processing system of claim 1, wherein an M bits-per-component at a frequency F2 is greater than or equal to three times a harmonic of a motion of a video.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present disclosure, a brief description of the drawings is given below. The following drawings are only illustrative of some of the embodiments of the present disclosure and for a person of ordinary skill in the art, other drawings or embodiments may be obtained from these drawings without an inventive effort.

[0024] FIG. 1 illustrates a typical stack for RGB OLEDs without color filters.

[0025] FIG. 2 illustrates a typical stack for a White OLED with color filters.

[0026] FIG. 3 illustrates a typical stack for LCD with color filters.

[0027] FIG. 4 illustrates a typical power consumption for LCD with color filter components.

[0028] FIG. 5 illustrates an FSC stack for LCD without (i.e. devoid of) color filters.

[0029] FIG. 6 is a diagram showing components of the image processing system and particularly showing per component delta sigma FSC modulation flow.

[0030] FIG. 7 is the diagram showing the oversampler implemented as an N-bit adder.

[0031] FIG. 8 illustrates three RGB frames integrated by the eye.

[0032] FIG. 9 illustrates that the individual multi-color (typically RGB) BLU can be steep which increases the color saturation and the Blue wavelength can move to be greater than 455 nm.

[0033] FIG. 10 illustrates that the BLU can be scanned which will decrease the power further.

[0034] FIG. 11 illustrates that Delta Sigma PDM is asynchronous input to output.

[0035] FIG. 12A illustrates a conventional LCD and how scattering occurs off the column drivers and color filters.

[0036] FIG. 12B illustrates that removing the Color Filter and two thirds of the Column Drivers reduces the scattering, thus increasing the contrast by 2×-3×.

[0037] FIG. 13 illustrates a typical Delta Sigma digital circuit used for compact disc (CD) audios.

[0038] FIG. 14 illustrates how a column driver turns on the pixels for an entire column. The TCON or AP choses the appropriate value. The column driver does the work to turn on the TFTs.

[0039] FIG. 15 illustrates that the TFT sits within the pixel. TFT blocks the available light which reduces the aperture ratio. Making the TFT smaller increases the aperture ratio.

DETAILED DESCRIPTION

[0040] The technical solutions of the present disclosure will be clearly and completely described below with reference to the drawings. The embodiments described are only some of the embodiments of the present disclosure, rather than all of the embodiments. All other embodiments that are obtained by a person of ordinary skill in the art on the basis of the embodiments of the present disclosure without an inventive effort shall be covered by the protective scope of the present disclosure.

[0041] In the description of the present disclosure, it is to be noted that the orientational or positional relation denoted by the terms such as “center”, “upper”, “lower”, “left”, “right”, “vertical”, “horizontal”, “inner” and “outer” is based on the orientation or position relationship indicated by the figures, which only serves to facilitate describing the present disclosure and simplify the description, rather than indicating or suggesting that the device or element referred to must have a particular orientation, or is constructed or operated in a particular orientation, and therefore cannot be construed as a limitation on the present disclosure. In addition, the terms “first”, “second” and “third” merely serve the purpose of description and should not be understood as an indication or implication of relative importance.

[0042] In the description of the present disclosure, it should be noted that unless otherwise explicitly specified and defined, the terms “install”, “link” and “connect” shall be understood in the broadest sense, which may, for example, refer to fixed connection, detachable connection or integral connection; may refer to mechanical connection or electrical connection; may refer to direct connection or indirect connection by means of an intermediate medium; and may refer to communication between two elements. A person of ordinary skill in the art would understand the specific meaning of the terms in the present disclosure according to the specific situations.

[0043] The invention is an image processing system or method implemented within a display driver and is a novel way to display images, whether the image(s)/video(s) is/are still or moving. As the modulation scheme within the image processing system, the invention produces a Field Sequential Color (FSC) using Delta Sigma (ΔΣ) Pulse Density Modulation (PDM). The system uses ΔΣ PDM and oversamples the input therefore, breaking the input into digital components (see FIG. 7). The output frequency must be greater than or equal to the input harmonic frequency. The digital components can be represented using 1-bit, 2-bits, 3-bits and so on and this is known as the bit-depth. The sequential color picker outputs red, green, or blue frames (see FIG. 8) which are integrated by the eye. Using ΔΣ PDM, the images are created over time and the human eye (or a camera) integrates them. For each subpixel, a loop of summations is created running at the output frequency. A subpixel value is added to the residual from the previous iteration of the loop using the following formula:

New_Value=Input_Video/Image_Value+Residual_Value;

Output_Video/Image=NearestValueEqualorUnder

Residual_Value=New_Value−Output_Video/Image.

[0044] As an example, if the initial Residual_Value=0 and the Input_Video/Image_Value=0.89 (0 1 scale), then the New_Value=0.89+0=0.89. If M=2, the four possible values are 0, 1/3, 2/3, 1.0. Therefore, the Output_Video/Image=0.67 (Nearest Value that is equal or under is 2/3). The Residual=0.89-0.67=0.22. On the next frame, if the Input_Video/Image_Value does not change, then the New_Value=0.89+0.22=1.11. Thus, the Output_Video/Image=1.0 and the Residual=1.11-1.0=0.11. Thus, the residual value is saved, and the residual value will be used on the next frame at the same pixel location. The next frame will occur at time increments of F 1. F2/F1 will define how many Outputs occur for the Input.

[0045] A series of shifters and adders are implemented by using different technologies. For example, using a semiconductor-based technology (see FIG. 7), each subpixel can be operated on independently. The end result is a massively parallel architecture.

[0046] The ΔΣ PDM output is not an image. In fact, the output has no frame properties; the ΔΣ PDM output is frame-less (see FIG. 6 and FIG.7). The output after the eye integrates is an image (see FIG. 8). The image quality produced by FSC ΔΣ PDM depends on oversampling frequency and the bit-depth of the digital component (refer to Table 1). As the bit-depth decreases, the oversampling frequency must increase to produce equivalent images/videos.

[0047] Since the human eye is the integrator, the image must be resolved within the human eye integration time which is 0.6 seconds. This makes ΔΣ PDM faster than PWM (refer to Table 2). The advantages of this approach are reduced power consumption, increased color saturation, increased brightness, increased contrast, and increased PPI.

[0048] Definition of Terms

[0049] Dithering hides banding by noisily transitioning from one color to another. This does not increase the bits-per-component.

[0050] Pixel is a point within the image. A pixel is made up of three or four components such as red, green, and blue (RGB), or cyan, magenta, yellow, and black (CMYK). Components are also referred to as sub-pixels/subpixels. Throughout this document we refer to bits-per-component (bpc), which is also known as bits per subpixel.

[0051] The present invention, which discloses a system and method thereof creating a Field Sequential Color (FSC) using Delta Sigma (As) Pulse Density Modulation (PDM) for digital displays, is described in detail below in reference to the figures.

[0052] FIGS. 1-2 illustrate the typical stack for an emissive display (OLED). FIG. 3 illustrates the typical stack for a non-emissive display (LCD). FIG. 4 illustrates the typical power consumption of an LCD with a color filter. FIG. 5 illustrates an FSC stack for LCD without color filters of applicant's invention. FIGS. 6-12 illustrate details of applicant's invention, where a detailed description will be proved below. FIG. 13 shows a traditional ΔΣ block diagram used when making audio compact discs (CD).

[0053] FIG. 1 illustrates a typical stack for RGB OLEDs without color filters. This stack is difficult to manufacture and suffers from color uniformity issues, mainly due to the blue color. Blue has a shorter life span than the other colors.

[0054] FIG. 2 illustrates a typical stack for a White OLED with color filters which is easier to manufacture, but the color filters are 40% efficient, which means that they block the light intensity.

[0055] FIG. 3 illustrates a typical stack for LCD with color filters. Generally, 4-8% of the light from the light source gets through the stack. Most of the power is consumed going through the color filters. The color filters are 40% efficient, which means that they block the light intensity. To compensate, the BLU must increase the intensity, which increases the power requirements.

[0056] FIG. 4 illustrates a typical power consumption for LCD with color filters. Most of the power is consumed by the backlight unit (BLU). The backlight unit (BLU) takes 67% of the power budget.

[0057] FIG. 5 illustrates an LCD without color filters. 87% of the BLU power will be saved by removing the color filters. The total power savings will be 87%*67%=58%. Thus, since the disclosed LCD and/or the FSC stack for the LCD, as shown in FIG. 5, is devoid of any color filter, 87% of the BLU power will be saved and the total power savings will be 87%*67%=58%.

[0058] The LCD comprises an analyzer; a liquid-crystal (LC); a thin film transistor (TFT) array; and a polarizer. The backlight unit (BLU) comprises a diffuser film; a color converter, which can be a quantum dot (QD) color converter or any equivalent color converter; and at least one blue LED or there are a plurality of blue LED's.

[0059] FIG. 6 is the system used in this invention. As shown in FIG. 6, the video/image 2 is inputted into the system. The inputted video/image 2 can be any (N) bit-per-component (bpc) at any F1 frame rate per second (fps). Each pixel location in the video/image has a value. The output will be at a new value using M-bpc at F2 fps. The ratio of F2/F1 is the oversampling frequency. As the oversampling frequency increases, fewer bits-per-component (M) are needed to display the video. The temporal average of the M-bpc values at the oversampling frequency represent the N-bpc value. A display 7 is made from Emissive or Non-Emissive material. Then a backplane consisting of thin-film transistors drive the display. A Timing Control or Application Processor (AP) decides which transistors are on or off. The FSC algorithm is inside the Timing Controller or AP. FSC reduces the number of columns (number of transistors) and thus the PPI increases.

[0060] When comparing the present invention to a traditional ΔΣ block diagram of FIG. 13, input 1 is the video input; an impulse is the Residual. Counter is the Oversampling output. Summing Interval is M-bpc at F2. Buffer is a low pass filter (the human eye or camera).

[0061] FIG. 6 is described in more detail below.

[0062] Reference number 1 is a residual from the previous iteration. The first iteration is pre-defined. A good first-order approximation of the residual is a distribution of a random number across the image. The residual is divided into its color components. The color components can be any color space such as RGB (Red-Green-Blue) or CMYK (Cyan-Magenta-Yellow-black). These color components are often termed sub-pixels/subpixels.

[0063] Reference number 2 is a video/image. The video is also divided into its color components. The video can be inputted at any frame rate F1 (0=still image; 15 fps, 24 fps, etc.).

[0064] Reference number 3 is an oversampling module. The oversampling module 3 can be software (i.e. code or algorithm) and/or hardware such as a chip, an application processor (AP) and/or a timing controller (TCON). The oversampling module 3 can be implemented in many different ways depending on the underlying hardware. A common way is to use an N-bpc adder. Box 1 is added to Box 2 for each component. If the summation overflows the N-bpc adder, then the output value is incremented. For example, if M=1, the video input is 81 (0-255 range), the residual from the previous frame is 200 (0-255 range), then the summation=281 (0-255 range). This creates an overflow; output value=1 and the residual for the next iteration=26. The output value, defined in M-bpc, does not define a color level per frame. ΔΣ PDM is frameless. Instead, the M-bpc values are integrated over time by the eye to form the image. The M-bpc values averaged over the oversampling frequency will approximate the original input video at N-bpc. They will be equivalent if the oversampling frequency is high enough as show in Table 1.

[0065] Reference number 4 is a module providing the desired F2 fps. The module 4 can be software (i.e. code or algorithm) and/or hardware such as a chip, an application processor (AP) and/or a timing controller (TCON). F2 is nominally set to the display's frequency and M is set to achieve the desired goal. The goal may be equivalency, bandwidth reduction, power reduction, or Mura (i.e. lack of uniformity) correction.

[0066] Reference number 5 is an output value in M-bpc. In the above example, the value=1.

[0067] Reference number 6 is a Sequential Color Picker. In an example, the sequential color picker chooses among Red, Green or Blue. When the sequential color picker chooses Red, the sequential color picker Nulls (zeros) all information about Green and Blue.

[0068] Reference number 7 is an output to the display that will show the value M.

[0069] A low pass filter which will integrate the output values over time. Nominally, this is a human eye. The low pass filter can alternatively be a camera running at the input frequency (F1) or alternatively any device that performs a low pass filter function.

[0070] FIG. 7 illustrates the oversampling built using an N-bit adder (3). The description is the same as FIG. 6. In addition, an N-bit adder (3) is where the Sum=Input+Residual and an N-bit-register is 0, 0.33, 0.67, 1.0.

[0071] FIG. 8 illustrates the final data as a Red, Green, or Blue only Frame. This can be also CMYK or any other color space. FIG. 8 is illustrating an output after all of the calculations are performed and is showing output values as a red, green, or blue frame.

[0072] FIG. 9 illustrates that individual LEDs will make up the backlight and is showing how to make the display saturated (i.e. very colorful). The X-axis is in nm and illustrates that the Red Color or Blue Color or Green Color can be very thin (saturated). The Y axis is the brightness of the LEDs. These LEDs can be made very steep which has two affects: the color saturation will improve, and the Blue LED(s) wavelength can be moved to be greater than 455 nm (Blue light ratio between 415-455 nm has been shown to be harmful in some studies). In order to reduce the blue light toxicity factor, the industry has used a blue reduction filter. However, the blue reduction filter removes 20% of the brightness of the display. This invention has solved the blue light toxicity by moving the Blue LED to greater than (i.e. >) 455 nm, while also improving the color saturation and improving the brightness of the display.

[0073] FIG. 10 illustrates that the BLU can be scanned and is showing how scanning the backlight will reduce motion blur. This will save power as the backlight does not need to be on for the entire frame. Motion blur can be improved using scanned backlights [0074] Backlight is divided into rows [0075] Light is scanned down the display at frame rate [0076] One or more rows can be illuminated at a time [0077] This removes the blur effect.

[0078] FIG. 11 illustrates the asynchronous input to output of the FSC Delta Sigma PDM. A video is inputted at a given frame rate. After FSC ΔΣ PDM, the output is asynchronous from the input.

[0079] FIG. 12A illustrates scattering within a conventional LCD. The backlight scatters off the column drivers and color filters.

[0080] FIG. 12B illustrates that the contrast will be increased by 2×-3× because the color filters and 2/3 of the column drivers are removed, which reduces the scattering of light.

[0081] FIG. 13 illustrates a traditional ΔΣ block diagram for building audio compact discs (CDs).

[0082] FIG. 14, sets the required voltage of the pixels in the activated row and thus supplies voltages to the LCD panel. The column driver turns on RGB pixel per row. The pixel shows Red/Green/Blue at the same time, but PPI is less as three subpixels make up a pixel in the RGB case.

[0083] FIG. 15 shows an LCD panel comprises a matrix of pixels, divided into for example, red, green, and blue “sub-pixels”. In FIG. 15, the grey is the TFT which is mainly opaque (blocks light). The more transistors and capacitors equals more light blocked. The subpixel is Red Green or Blue. The pixel is the a group of three (if RGB). The column driver turns on a pixel (RGB). For 1080 P, this means there are 1920*1080*3=6 Million pixels to turn on. For the disclosed invention of FSC, it is Red or Green or Blue, which means only 1920*1080=2 Million Pixels have to be turn on.