Dynamic frame rate conversion in active matrix display
11694653 · 2023-07-04
Assignee
Inventors
Cpc classification
G09G2320/0686
PHYSICS
G09G2320/106
PHYSICS
G09G2340/0435
PHYSICS
G09G2360/16
PHYSICS
G09G3/20
PHYSICS
International classification
Abstract
The present invention provides a motion content based dynamic frame rate conversion method for displaying a video by a display device, comprising: detecting motion content of the video and generating a control signal for controlling a display color depth based on the motion detection result. The video is displayed with a lower color depth at a higher frame rate than a standard configuration of the display device if the motion detection result indicates that the video contains appreciable amount of motion content; and the video is displayed with a higher color depth at a lower frame rate than the standard configuration of the display device if the motion detection result indicates that the video is relatively static. The present invention can facilitate the display device to dynamically convert its display output formats according to motion content of the video to further optimize the display quality.
Claims
1. A motion content based dynamic frame rate conversion method for displaying a video by a display device, comprising: detecting, by a dynamic motion detection apparatus, motion content of the video and generating, by the dynamic motion detection apparatus, a motion detection result; receiving, by a frame rate controller, the motion detection result; and generating, by the frame rate controller, a control signal for controlling a display color depth based on the motion detection result; wherein: the video is displayed with a lower color depth at a higher frame rate than a standard configuration of the display device if the motion detection result indicates that the video contains appreciable amount of motion content; the video is displayed with a higher color depth at a lower frame rate than the standard configuration of the display device if the motion detection result indicates that the video is relatively static; the detection of motion content of the video comprises: storing, by a storage unit, the regional brightness values of the first frame into a first brightness data array and the regional brightness values of the second frame into a second brightness data array; comparing, by a brightness change detector, the first and second brightness data arrays to obtain an array of brightness differences; detecting, by the brightness change detector, change of brightness for each region of the display screen by comparing each element of the array of brightness differences with one or more voting threshold values; and generating, by the brightness change detector, a vote based on a comparison result for each element of the array of brightness differences calculating, by a majority vote logic unit, a sum of the votes generated for all elements of the array of brightness differences; determining, by the majority vote logic unit, the motion detection result by comparing the calculated sum of the votes with one or more motion detection threshold values; and generating, by the majority vote logic unit, a motion detection signal to the frame rate controller.
2. The motion content based frame rate conversion method according to claim 1, the detection of motion content of the video further comprises: partitioning, by a brightness accumulator, a display screen of the display device into a plurality of regions; and calculating, by the brightness accumulator, a plurality of regional brightness values for a first frame and a plurality of regional brightness values for a second frame which is A F number of frames after the first frame.
3. A dynamic motion detection apparatus for detecting motion content in a video, comprising: a brightness accumulator configured to: partition a display screen of the display device into a plurality of regions; and calculate a plurality of regional brightness values for a first frame and a plurality of regional brightness values for a second frame which is ΔF number of frames after the first frame; a storage unit configured to store the regional brightness values of the first frame into a first brightness data array and the regional brightness values of the second frame into a second brightness data array; a brightness change detector configured to: compare the first and second brightness data arrays to obtain an array of brightness differences; detect change of brightness for each region of the display screen by comparing each element of the array of brightness differences with one or more voting threshold values; and generate a vote based on a comparison result for each element of the array of brightness differences a majority vote logic unit configured to calculate a sum of the votes generated for all elements of the array of brightness differences; determine the motion detection result by comparing the calculated sum of the votes with one or more motion detection threshold values; and generate a motion detection signal based on the motion detection result.
4. A frame rate convertible active matrix display device comprising the dynamic motion detection apparatus according to claim 3.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Embodiments of the invention are described in more details hereinafter with reference to the drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
DETAILED DESCRIPTION OF THE INVENTION
(13) In the following description, methods for driving an active matrix display for frame-rate conversion and the display device for implementing the same are set forth as preferred examples. It will be apparent to those skilled in the art that modifications, including additions and/or substitutions may be made without departing from the scope and spirit of the invention. Specific details may be omitted so as not to obscure the invention; however, the disclosure is written to enable one skilled in the art to practice the teachings herein without undue experimentation.
(14)
(15) The timing controller 12 may comprise a dynamic motion detection apparatus 121 configured to detect motion content of a video and generate a motion detection signal (V_MD); a frame rate controller 122 configured to receive the motion detection result and the input color data, and generate a control signal (V_Ctrl) for controlling a display color depth, a dithering and directional modulation-based frame rate conversion apparatus 123 configured to receive the control signal and convert the input color data to the output color data based on directional modulation and dithering such that the video can be displayed without observable color depth degradation even if the display frame rate is higher than the standard configuration of the display device. The timing controller 12 may further comprise a frame buffer 124 connected to the frame rate controller 122 and configured to store color data.
(16) In particular, if the motion detection result for the video indicating that the video contains appreciable amount of motion content, the display device will display the video with a lower color depth (e.g., 4 bits per color components) at a higher frame rate (e.g. 120 Hz) than a standard configuration of the display device. If the motion detection result indicating that is the video is relatively static, the display device will display the video with a higher color depth (e.g., 8 bits per color component) at a lower frame rate (e.g. 60 Hz) than the standard configuration of the display device.
(17)
(18) The directional delta modulation generator 310 may be configured to receive a plurality of input color data (R_In, G_In and B_In) representing RGB color components of an input pixel color, a plurality of synchronization signals (V_Sync) and a control signal (V_Ctrl); and generate a plurality of modulated data (R_Mod, G_Mod and B_Mod) for the plurality of input color data respectively.
(19) Each dithering module 320 may comprise a respective residue line buffer 322 configured to track a residual error in dithering conversion and generate a respective residual data (R_Res/G_Res/B_Res); and a respective adapter (or adder) 321 configured to receive a respective input color data (R_In/G_In/B_In), a respective modulated data (R_Mod/G_Mod/B_Mod) from the directional delta modulation generator 310, and a respective residual data (R_Res/G_Res/B_Res) from a respective residue line buffer 322 to adapt the respective input color data to generate a respective adapted color data (R_AD/G_AD/B_AD) by adding the respective input color data with the respective modulated data and the respective residual data.
(20) Each dithering module 320 may further comprise a dithering engine 323 configured to receive a respective adapted color data (R_AD/G_AD/B_AD) from a respective adapter and generate a respective output color data (R_Out/G_Out/B_Out).
(21) Depending on the frame-rate conversion target, each dithering engine 323 may be configured to perform K-bit dithering to convert the respective adapted color data to an output color data for representing a color depth of K bits per component, where K is an integer equal to or great than 1 which may be selected by the control signal (V_ctrl) from the frame rate controller 122. In particular, the respective adapted color data is compared against (2.sup.K-1) dithering threshold values to generate a respective output color data with 2.sup.K possible output color levels.
(22) As shown in
(23) As shown in
(24) A color space cube for representing the pixel color may be divided into a number of sub-color space cubes depending on the color depth to be displayed. For instance, in a RGB color space, with a color depth of K bits per component in each RGB direction, a color space cube may be divided into an 8.sup.K number of sub-color space cubes and there are K number of quantized color levels in each RGB direction. Accordingly, each sub-color space cube corresponds to a set of RGB color levels.
(25)
(26) The directional delta modulation generator 310 may be further configured to determine a modulation direction by comparing each of the color components of the pixel color against a modulation threshold value and obtaining a flag value of modulation for each of the color components. For instance, if a color component of the pixel has a value equal to or greater than the modulation threshold value, the flag value of modulation for the color component is set as “1”, otherwise, the flag value of modulation for the color component is set as “0”.
(27) Accordingly, the flag values of modulation for each of the color components may be used to construct a modulation direction unit vector U.sub.m (x.sub.m, y.sub.m, z.sub.m) in the RGB color space to represent the modulation direction, where x.sub.m, y.sub.m, and z.sub.m are RGB components of the unit vector respectively. Each of the RGB components x.sub.m, y.sub.m, and z.sub.m of the modulation direction unit vector U.sub.m may have a binary value (“1” or “0”) determined by comparing RGB component values of the pixel color against with the modulation threshold value respectively. For instance, if the R component value of the pixel color is equal to or greater than the modulation threshold value, x.sub.m is set to “1”, otherwise x.sub.m is set to be “0”. In other words, whether delta modulation is applied in a color component (direction) in the color space depends on whether the component value of the pixel color in that color component (direction) is equal to or greater than the modulation threshold value.
(28)
(29) Referring to
(30) Referring to
(31) Referring to
(32) Referring to
(33) Referring to
(34) Referring to
(35) Referring to
(36) The directional delta modulation generator 310 may be further configured to apply a delta modulation based on the determined flag value to each of the color components of the pixel to obtain a modulated data for the color component.
(37) The delta modulation may be performed across a sequence of image frames over a modulation cycle using a sequence of N delta modulation values to obtain N modulated data for each of the color components. Within the modulation cycle, an i.sup.th modulated data for the color component obtained in the i.sup.th frame may be given by:
X.sub.mi=X.sub.oi+d.sub.i, for i=1,2, . . . , N,
(38) where X.sub.mi is the i.sup.th modulated data obtained in the i.sup.th frame, X.sub.oi is the original value of the input color data in the i.sup.th frame, d.sub.i is the delta modulation value used in the i.sup.th frame which may have a positive or negative value, and N is the total number of frames within a modulation cycle.
(39) Preferably, the sequence of N delta modulation values d.sub.i may be selected to have a sum equal to zero, that is, Σ.sub.i=1.sup.Nd.sub.i=0, in order to apply the delta modulation across the frames in a balanced manner.
(40) Within the modulation cycle, the dithering engine 323 may be further configured to determine a N number of color levels for the color component of the pixel based on the N modulated data obtained across the N number of frames respectively.
(41) The i.sup.th color level for the color component based on the modulated data X.sub.mi obtained in the i.sup.th frame may be determined with an algorithm given by:
(42)
(43) where C.sub.i is the i.sup.th color level obtained in the i.sup.th image frame, L.sub.k is the k.sup.th color levels defined in the color space with the color depth of K bits per component to be displayed.
(44) The dithering engine 323 may be further configured to average the N color levels for the color components of the pixel determined across the sequence of frames over the modulation cycle to obtain an average display color value C.sub.avg, which is given by:
(45)
and set the average display color value as the output color level.
(46)
(47)
which is represented as a dot within the sub-color space squares corresponding to L.sub.k. By way of example, the delta modulation values (di) used in the modulation cycle are set as: d.sub.1=0, d.sub.2=−δ, d.sub.3=δ, d.sub.4=0, d.sub.5=−2δ, and d.sub.6=2δ, where δ is a predefined delta value.
(48) Referring to
(49)
and less than L.sub.k. That is,
(50)
The color levels of the pixel color component in the 6 frames are determined as: C.sub.1=L.sub.k, C.sub.2=L.sub.k−1, C.sub.3=L.sub.k, C.sub.4=L.sub.k, C.sub.5=L.sub.k−1, and C.sub.6=L.sub.k. The average display color value C.sub.avg is equal to (2 L.sub.k−1+4L.sub.k)/6, which is a color level between L.sub.k−1 and L.sub.k.
(51) Referring to
(52) Referring to
(53)
That is,
(54)
The color levels of the pixel color component in the 6 frames are determined as: C.sub.1=L.sub.k, C.sub.2=L.sub.k, C.sub.3=L.sub.k+1, C.sub.4=L.sub.k, C.sub.5=L.sub.k, and C.sub.6=L.sub.k+1. As a result, the average display color value C.sub.avg is equal to (4L.sub.k+2L.sub.k+1)/6, which is a color level value between L.sub.k and L.sub.k+1.
(55) It can be observed from
(56)
and less than L.sub.k, has an average display color value between L.sub.k−1 and L.sub.k over the modulation cycle; the color component of the pixel for the case in
(57)
has an average display color value between L.sub.k and L.sub.k+1 over the modulation cycle. In other words, the display image is smoothened by applying the directional modulation therefore the observable degradation of color depth due to frame rate conversion can be eliminated.
(58) In some embodiments, with the dithering and directional modulation-based frame rate conversion apparatus, an input image source having 60 Hz frame rate at color depth of 8 bits per component may be converted to an output image source with color depth of 2 bits per component displayed at 240 Hz frame rate, color depth of 3 bits per component displayed at 180 Hz frame rate or color depth of 4 bits per component displayed at 120 Hz frame rate.
(59) The dithering and directional modulation-based frame rate conversion apparatus can be configured to support conversion of image sources (say, from a computer graphic card) having various frame rates. For example, an input image source having 60 Hz frame rate at color depth of 8 bits per component may be converted to an output image source with color depth of 2 bits per component displayed at 240 Hz frame rate, color depth of 3 bits per component displayed at 180 Hz frame rate or color depth of 4 bits per component displayed at 120 Hz frame rate.
(60) In some embodiments, the output image source may have a mix of different display formats of frame rates including but not limited to, 240 Hz, 200 Hz, 180 Hz, 150 Hz, 120 Hz, 100 Hz, 80 Hz and 60 Hz.
(61) Referring to
(62) Referring to
(63) Referring to
(64) Referring to
(65)
(66) In some embodiments, the vote may have a first voting value for the element if the comparison result is that the element is equal to or lower than the first voting threshold value; a second voting value which is higher than the first voting value for the element if the comparison result is that the element is higher than the first voting threshold value and lower than the second voting threshold value; or a third voting value which is higher than the second voting value for the element if the comparison result is that the element is equal to or higher than the second voting threshold value.
(67) The process of dynamic motion detection may further include: i) calculating, by a majority vote logic unit 940, a sum of the votes generated for all elements of the array of brightness differences; j) determining, by the majority vote logic unit 940, the motion detection result by comparing the calculated sum of the votes with one or more motion detection threshold values; and k) generating, by the majority vote logic unit 940, a motion detection signal (V_MD) to the frame rate controller 122 based on the motion detection result.
(68) In some embodiments, if the calculated sum is equal to or higher than a motion detection threshold value, the motion detection result may be determined as that the video includes appreciable amount of motion contents. Based on the determined motion detection result, the frame rate controller 122 may determine to display the video with a lower color depth at a higher frame rate than the standard configuration of the display device (e.g., with a color depth of 4 bits per color components and at a frame rate of 120 Hz). If the calculated sum is smaller than the motion detection threshold value, the motion detection result may be determined as that the video is relatively static. Based on the determined motion detection result, the frame rate controller 122 may determine to display the video with a higher color depth at a lower frame rate than the standard configuration of the display device (e.g., with a color depth of 8 bits per color components and at a frame rate of 60 Hz).
(69) A new round of dynamic motion detection may be performed by: taking a previous second frame in a previous round of motion detection as a new first frame for the new round of motion detection; calculating brightness values of a new second frame which is Δ F frames after the new first frame; overwriting the brightness data array which has stored brightness values of a previous first frame in the previous round of motion detection with the calculated brightness values for the new second frame; and repeating the above steps f) to k). As there is no need to calculate the brightness values for the new first frame, the computation time for new round of motion detection can be greatly reduced.
(70)
(71) Referring to
(72) Referring to
(73) Referring to
(74) Referring to
(75) The embodiments disclosed herein may be implemented using general purpose or specialized computing devices, computer processors, or electronic circuitries including but not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), and other programmable logic devices configured or programmed according to the teachings of the present disclosure. Computer instructions or software codes running in the general purpose or specialized computing devices, computer processors, or programmable logic devices can readily be prepared by practitioners skilled in the software or electronic art based on the teachings of the present disclosure. In some embodiments, the present invention includes computer storage media having computer instructions or software codes stored therein which can be used to program computers or microprocessors to perform any of the processes of the present invention. The storage media can include, but are not limited to ROMs, RAMs, flash memory devices, or any type of media or devices suitable for storing instructions, codes, and/or data.
(76) The embodiments were chosen and described in order to best explain the principles of the invention and its working principle and practical application, thereby enabling others skilled in the art to understand the invention for various embodiments and with various modifications that are suited to the particular use contemplated. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations will be apparent to the practitioner skilled in the art.