VIDEO SIGNAL PROCESSING APPARATUS, VIDEO SIGNAL PROCESSING METHOD, AND IMAGING APPARATUS
20230215131 · 2023-07-06
Assignee
Inventors
Cpc classification
H04N5/66
ELECTRICITY
H04N23/741
ELECTRICITY
G06V10/60
PHYSICS
International classification
G06V10/60
PHYSICS
H04N23/741
ELECTRICITY
Abstract
This imaging signal processing apparatus includes: a first video signal processing unit that generates, from a pixel signal obtained by an imaging unit capable of obtaining a pixel signal with a first dynamic range, a luminance reduction video signal with a second dynamic range narrower than the first dynamic range; and a second video signal processing unit that generates, from the pixel signal, an information signal of a luminance change component of a luminance region of the luminance reduction video signal and a gain up signal of a pixel signal of the luminance region and adds the information signal and the gain up signal to the luminance reduction video signal.
Claims
1. A video signal processing apparatus, comprising: a first video signal processing unit that generates, from a pixel signal obtained by an imaging unit capable of obtaining a pixel signal with a first dynamic range, a luminance reduction video signal with a second dynamic range narrower than the first dynamic range; and a second video signal processing unit that generates, from the pixel signal, an information signal of a luminance change component of a luminance region of the luminance reduction video signal and a gain up signal of a pixel signal of the luminance region and adds the information signal and the gain up signal to the luminance reduction video signal.
2. The video signal processing apparatus according to claim 1, wherein the first video signal processing unit is configured to generate, from the pixel signal, the luminance reduction video signal by knee processing.
3. The video signal processing apparatus according to claim 2, wherein the second video signal processing unit is configured to generate the gain up signal by multiplying the pixel signal of the luminance region of the luminance reduction video signal by a gain determined in accordance with a luminance value of the pixel signal.
4. The video signal processing apparatus according to claim 3, wherein the first dynamic range is a high dynamic range (HDR), and the second dynamic range is a standard dynamic range (SDR).
5. The video signal processing apparatus according to claim 4, further comprising a first output unit that outputs an output video signal generated by the second video signal processing unit.
6. The video signal processing apparatus according to claim 5, wherein the first output unit is configured to output the output video signal to a view finder.
7. The video signal processing apparatus according to claim 1, wherein the second video signal processing unit is further configured to generate an information signal of a luminance change component of a region belonging to a higher-luminance side as compared to the luminance region of the luminance reduction video signal.
8. The video signal processing apparatus according to claim 7, wherein the second video signal processing unit is configured to add the generated information signal of the luminance change component of the region belonging to the higher-luminance side with the information signal of the luminance change component of the luminance region of the luminance reduction video signal and the gain up signal to the luminance reduction video signal.
9. The video signal processing apparatus according to claim 1, further comprising a third video signal processing unit that generates, from the pixel signal obtained by the imaging unit, a video signal with the first dynamic range.
10. The video signal processing apparatus according to claim 9, further comprising a second output unit that outputs the video signal with the first dynamic range.
11. The video signal processing apparatus according to claim 9, wherein the second video signal processing unit is configured to generate, from the pixel signal, an information signal of a luminance change component of a region belonging to a higher-luminance region as compared to the luminance region of the luminance reduction video signal and further add the information signal to the video signal with the first dynamic range.
12. A video signal processing method, comprising: generating, from a pixel signal obtained by an imaging unit capable of obtaining a pixel signal with a first dynamic range, a luminance reduction video signal with a second dynamic range narrower than the first dynamic range; and generating, from the pixel signal, an information signal of a luminance change component of a luminance region of the luminance reduction video signal and a gain up signal of a pixel signal of the luminance region and adding the information signal and the gain up signal to the luminance reduction video signal.
13. An imaging apparatus, comprising: an imaging unit capable of obtaining a pixel signal with a first dynamic range; a first video signal processing unit that generates, from the pixel signal obtained by the imaging unit, a luminance reduction video signal with a second dynamic range narrower than the first dynamic range; and a second video signal processing unit that generates, from the pixel signal, an information signal of a luminance change component of a luminance region of the luminance reduction video signal and a gain up signal of a pixel signal of the luminance region and adds the information signal and the gain up signal to the luminance reduction video signal.
Description
BRIEF DESCRIPTION OF DRAWINGS
[0028]
[0029]
[0030]
[0031]
[0032]
[0033]
[0034]
[0035]
[0036]
[0037]
[0038]
[0039]
[0040]
MODE(S) FOR CARRYING OUT THE INVENTION
[0041] Hereinafter, embodiments according to the present technology will be described with reference to the drawings.
First Embodiment
[0042] [General Configuration]
[0043]
[0044] An imaging system 1 includes an imaging unit 10, a video signal processing apparatus 20, a view finder 30, an operation input unit 40, and a control unit 50.
[0045] The imaging unit 10 includes an optical block 101 and an imaging element unit 102.
[0046] The optical block 101 has a lens, a focus adjustment mechanism, a shutter mechanism, an aperture (iris) mechanism, and the like. The optical block 101 forms an image of reflected light from a subject on an imaging plane of an imaging element of the imaging element unit 102 with a lens. The imaging element unit 102 has an imaging element, an analog signal processing circuit, an A/D conversion circuit, and the like. The imaging element includes a charge coupled device (CCD) image sensor, a complementary metal oxide semiconductor (CMOS) image sensor, and the like. The imaging unit 10 is capable of obtaining a pixel signal with a relatively wide first dynamic range, for example, a high dynamic range (HDR).
[0047] The video signal processing apparatus 20 generates, from a pixel signal with a first dynamic range obtained by the imaging unit 10, a video signal with a second dynamic range narrower than the first dynamic range, such as an SDR, for example.
[0048] The video signal processing apparatus 20 includes a first video processing unit 210 and a second video processing unit 220.
[0049] The first video processing unit 210 has a pre-processing unit 211, a gain adjustment unit 212, a matrix unit 213, a detail processing unit 214, a knee processing unit 215, and an SDR gamma processing unit 216.
[0050] The pre-processing unit 211 performs, for example, signal correction processing such as defect correction and lens aberration correction on the pixel signal obtained by the imaging unit 10.
[0051] The gain adjustment unit 212 performs gain adjustment such as RGB gain adjustment for white balance adjustment.
[0052] The matrix unit 213 obtains color image data that is the first video signal with a first dynamic range by performing DeBayer processing, linear matrix processing, or the like on a pixel signal passing through the gain adjustment unit 212.
[0053] The detail processing unit 214 processes details with respect to the color image data that is the first video signal.
[0054] The knee processing unit 215 generates a luminance reduction video signal with a second dynamic range narrower than the first dynamic range by performing knee processing of the color image data that is the first video signal.
[0055] The SDR gamma processing unit 216 subjects the color image data with the second dynamic range to gamma processing for display on a display.
[0056] The second video processing unit 220 has a dynamic contrast extraction unit 221, an original signal gain up processing unit 222, a first adder 223, and a second adder 224.
[0057] The dynamic contrast extraction unit 221 generates an information signal of a luminance change component of a luminance region of the luminance reduction video signal as a contrast emphasis signal.
[0058] The original signal gain up processing unit 222 generates a signal obtained by multiplying the pixel signal of the luminance region of the luminance reduction video signal by a gain determined in accordance with the luminance value as a gain up signal.
[0059] The first adder 223 combines the contrast emphasis signal with the gain up signal.
[0060] The second adder 224 generates an output video signal by adding the composite signal of the contrast emphasis signal and the gain up signal output from the first adder 223 to the luminance reduction video signal with the second dynamic range generated by the first video processing unit 210.
[0061] A formatter/output unit 227 converts the output video signal output from the second adder 224 into a transmission format of SDR video and outputs it to the view finder 30, for example.
[0062] The SDR video can be transmitted to a camera control unit (CCU) or an external video device such as an external display through a camera cable other than the view finder 30.
[0063] The operation input unit 40 receives input of a variety of operation instructions, mode settings, and the like from a user. The operation input unit 40 includes a button, a switch, a dial, and a touch panel sensor provided on a VF-17 screen, for example.
[0064] The control unit 50 includes a central processing unit (CPU), a random-access memory (RAM), and a read only member (ROM). Programs to be executed by the CPU, various parameters, and the like are stored in the RAM or the ROM. The control unit 50 interprets information about user operation input received by the operation input unit 40 and controls the imaging system 1 in accordance with the interpreted input information.
[0065] [Regarding Problem]
[0066]
[0067] Provided that the range of the SDR is 100%, the range of the HDR is 600% or 1200%, for example. As one of methods for generating an SDR video signal from an HDR video signal, there is a method of performing knee correction on the HDR video signal on the basis of a predetermined luminance value (knee correction point) as shown in
[0068] In view of this, there is a method of obtaining an SDR video signal with contrast of a high-luminance region emphasized by generating, from an original signal obtained by an imaging unit, an information signal of a high-luminance change component of a region on a higher-luminance side with respect to a knee correction point and adding it to a luminance reduction video signal with the SDR. However, this method cannot provide an effect of improving the visibility of a region other than the high-luminance region as a matter of course. In view of this, there can be a method of adding a contrast emphasis signal of a low-luminance region to a luminance reduction video signal with respect to the knee correction point. However, since an amount of change in luminance of the low-luminance region is smaller than that of the high-luminance region, it is difficult to improve the visibility of video of a dark portion significantly.
[0069] [Operations of Dynamic Contrast Extraction Unit 221 and Original Signal Gain Up Processing Unit 222]
[0070]
[0071]
[0072]
[0073] By adding the contrast emphasis signal S1 and the gain up signal S2 to the original signal S0 in this manner, an SDR video signal with visibility improved in terms of both contrast and brightness can be generated. Accordingly, the visibility of the SDR video displayed on the view finder 30 or the like, for example, and the focus accuracy for the dark portion of the video can be improved.
[0074] [Regarding Gain of Original Signal Gain Up Processing Unit 222]
[0075] As described above, the original signal gain up processing unit 222 generates the signal obtained by multiplying the pixel signal by the gain determined in accordance with the luminance value of the pixel signal to avoid exceeding of the luminance value of the output video signal as the gain up signal. Here, a gain determined depending on the luminance value of the pixel signal will be described.
[0076] Alternatively, as shown in
First Modified Example
[0077]
[0078] With this configuration, the visibility of video of both the region on the lower-luminance side and the region on the higher-luminance side with respect to the knee correction point can be improved. It is possible to improve the visibility of the video of the region on the higher-luminance side where light up is done while adjusting the focus on the region on the lower-luminance side when imaging a dark portion for example.
Second Modified Example
[0079]
[0080] The matrix unit 241 obtains color image data by performing DeBayer processing, linear matrix processing, or the like on a pixel signal passing through the gain adjustment unit 212.
[0081] The detail processing unit 242 processes details of the color image data.
[0082] The knee processing unit 243 performs knee processing on the color image data.
[0083] The OETF unit 244 performs gradation compression with an optical-electro transfer function (OETF) for the HDR with respect to the color image data and performs gamma signal processing.
[0084] A formatter/output unit 245 converts the color image data passing through the OETF unit 244 into a transmission format of HDR video.
[0085] The HDR video signal generated by the third video processing unit 240 is transmitted to a camera control unit (CCU) or an external video device such as an external display through a camera cable as main video.
[0086] In the imaging system to which the third video processing unit 240 is added as described above, the contrast emphasis signal generated by the dynamic contrast extraction unit 231 for the high-luminance region may be added by an adder 225 to the HDR video signal generated by the third video processing unit 240. Accordingly, the HDR video signal with contrast emphasized is obtained.
Third Modified Example
[0087] [Regarding Mode Setting]
[0088] In the imaging system 1 described above, as for generation of the above-mentioned contrast emphasis signal and gain up signal, the control unit 50 controls, for example, on/off of the gain up function, the contrast emphasis function for the low-luminance region, and the contrast emphasis function for the high-luminance region by the user selecting a mode.
[0089] A user interface (UI) screen for the user setting is displayed on the view finder 30. As for this UI, the user can input settings through the operation input unit 40 for example. The input settings are received and retained by the control unit 50.
[0090] The mode includes a general contrast emphasis mode, a contrast emphasis mode for a high-luminance region, a contrast emphasis mode for a region other than the high-luminance region, a contrast emphasis mode for a low-luminance region, a contrast emphasis mode for the high-luminance region+the low-luminance region, and the like.
[0091] (General Contrast Emphasis Mode)
[0092] On the general contrast emphasis mode, contrast components are generated with respect to all luminance regions (all regions on the lower-luminance side and the higher-luminance side with respect to the knee correction point).
[0093]
[0094] (Contrast Emphasis Mode for High-Luminance Region)
[0095] The contrast emphasis mode for the high-luminance region is a mode of adding contrast components of the region belonging to the higher-luminance side with respect to the knee correction point, for example, the highest-luminance region to the output video signal of the first video processing unit 210.
[0096]
[0097] (Contrast Emphasis Mode for Region Other than High-Luminance Region)
[0098] The contrast emphasis mode for the region other than the high-luminance region is a mode of adding contrast components of a region other than the highest-luminance region in the region on the higher-luminance side with respect to the knee correction point and the region on the lower-luminance side with respect to the knee correction point to the video signal generated by the third video processing unit 240 and the luminance reduction video signal output from the first video processing unit 210. This mode is used especially in a scenario with the fog or the like. Fog tends to increase the luminance of the video signal. Therefore, if the contrast of a foggy region in video is emphasized, the focus is put on the fog. It is a mode used for avoiding such a situation.
[0099]
[0100] (Contrast Emphasis Mode for Low-Luminance Region)
[0101] The contrast emphasis mode for the low-luminance region is a mode of adding contrast components of the low-luminance region and the gain up signal to the luminance reduction video signal output by the first video processing unit 210. This mode corresponds to the above-mentioned first embodiment.
[0102] (Contrast Emphasis Mode for High-Luminance Region+Low-Luminance Region)
[0103] The contrast emphasis mode for the high-luminance region+the low-luminance region is equivalent to the one described in the first modified example. On this mode, the dynamic contrast extraction unit 221 for the low-luminance region is set by the control unit 50 to generate a contrast emphasis signal of the region on the lower-luminance side with respect to the knee correction point. On the other hand, the dynamic contrast extraction unit 231 for the high-luminance region is set by the control unit 50 to generate an information signal of a high-luminance change component of a region belonging to the higher-luminance side with respect to the knee correction point, for example, the highest-luminance region as the contrast emphasis signal. The original signal gain up processing unit 222 is turned on. This mode can improve the visibility of the video of both the region on the lower-luminance side with respect to the knee correction point and the region on the higher-luminance side. Note that on this mode, the region on the higher-luminance side for which the contrast emphasis signal is generated is not necessarily the highest-luminance region. For example, it may be a middle-luminance region or the like in the region on the higher-luminance side with respect to the knee correction point, a region belonging to the higher-luminance side.
[0104] It should be noted that the present technology may also take the following configurations.
(1) A video signal processing apparatus, including:
[0105] a first video signal processing unit that generates, from a pixel signal obtained by an imaging unit capable of obtaining a pixel signal with a first dynamic range, a luminance reduction video signal with a second dynamic range narrower than the first dynamic range; and
[0106] a second video signal processing unit that generates, from the pixel signal, an information signal of a luminance change component of a luminance region of the luminance reduction video signal and a gain up signal of a pixel signal of the luminance region and adds the information signal and the gain up signal to the luminance reduction video signal.
(2) The video signal processing apparatus according to (1), in which the first video signal processing unit is configured to generate, from the pixel signal, the luminance reduction video signal by knee processing.
(3) The video signal processing apparatus according to (1) or (2), in which
[0107] the second video signal processing unit is configured to generate the gain up signal by multiplying the pixel signal of the luminance region of the luminance reduction video signal by a gain determined in accordance with a luminance value of the pixel signal.
(4) The video signal processing apparatus according to any one of (1) to (3), in which
[0108] the first dynamic range is a high dynamic range (HDR), and
[0109] the second dynamic range is a standard dynamic range (SDR).
(5) The video signal processing apparatus according to any one of (1) to (4), further including
[0110] a first output unit that outputs an output video signal generated by the second video signal processing unit.
(6) The video signal processing apparatus according to (5), in which
[0111] the first output unit is configured to output the output video signal to a view finder.
(7) The video signal processing apparatus according to any one of (1) to (6), in which
[0112] the second video signal processing unit is further configured to generate, from the pixel signal, the information signal of the luminance change component of the luminance region of the luminance reduction video signal and generate an information signal of a luminance change component of a region belonging to a higher-luminance side as compared to the luminance region of the luminance reduction video signal.
(8) The video signal processing apparatus according to (7), in which
[0113] the second video signal processing unit is configured to add the generated information signal of the luminance change component of the region belonging to the higher-luminance side with the information signal of the luminance change component of the luminance region of the luminance reduction video signal and the gain up signal of the pixel signal of the luminance region to the luminance reduction video signal.
(9) The video signal processing apparatus according to any one of (1) to (8), further including
[0114] a third video signal processing unit that generates, from the pixel signal obtained by the imaging unit, a video signal with the first dynamic range.
(10) The video signal processing apparatus according to (9), further including
[0115] a second output unit that outputs the video signal with the first dynamic range.
(11) The video signal processing apparatus according to (9), in which
[0116] the second video signal processing unit is configured to generate, from the pixel signal, an information signal of a luminance change component of a region belonging to a higher-luminance region as compared to the luminance region of the luminance reduction video signal and further add the information signal to the video signal with the first dynamic range.
(12) A video signal processing method, including:
[0117] generating, from a pixel signal obtained by an imaging unit capable of obtaining a pixel signal with a first dynamic range, a luminance reduction video signal with a second dynamic range narrower than the first dynamic range; and
[0118] generating, from the pixel signal, an information signal of a luminance change component of a luminance region of the luminance reduction video signal and a gain up signal of a pixel signal of the luminance region and adding the information signal and the gain up signal to the luminance reduction video signal.
(13) The video signal processing method according to (12), further including
[0119] generating, from the pixel signal, the luminance reduction video signal by knee processing.
(14) The video signal processing method according to (12) or (13), further including
[0120] generating the gain up signal by multiplying the pixel signal of the luminance region of the luminance reduction video signal by a gain determined in accordance with a luminance value of the pixel signal.
(15) The video signal processing method according to any one of (12) to (14), in which
[0121] the first dynamic range is a high dynamic range (HDR), and
[0122] the second dynamic range is a standard dynamic range (SDR).
(16) The video signal processing method any one of (12) to (15), further including
[0123] outputting an output video signal generated by the second video signal processing unit.
(17) The video signal processing method according to (16), further including
[0124] outputting the output video signal to a view finder.
(18) The video signal processing method according to any one of (12) to (17), further including
[0125] generating, from the pixel signal, the information signal of the luminance change component of the luminance region of the luminance reduction video signal and generating an information signal of a luminance change component of a region belonging to a higher-luminance side as compared to the luminance region of the luminance reduction video signal.
(19) The video signal processing method according to (18), further including
[0126] adding the generated information signal of the luminance change component of the region belonging to the higher-luminance side with the information signal of the luminance change component of the luminance region of the luminance reduction video signal and the gain up signal of the pixel signal of the luminance region to the luminance reduction video signal.
(20) The video signal processing method according to any one of (12) to (19), further including
[0127] generating, from the pixel signal obtained by the imaging unit, a video signal with the first dynamic range.
(21) The video signal processing method according to (20), further including
[0128] outputting the video signal with the first dynamic range.
(22) The video signal processing method according to (20), further including
[0129] generating, from the pixel signal, an information signal of a luminance change component of a region belonging to a higher-luminance region as compared to the luminance region of the luminance reduction video signal and further add the information signal to the video signal with the first dynamic range.
(23) An imaging apparatus, including:
[0130] an imaging unit capable of obtaining a pixel signal with a first dynamic range;
[0131] a first video signal processing unit that generates, from the pixel signal obtained by the imaging unit, a luminance reduction video signal with a second dynamic range narrower than the first dynamic range; and
[0132] a second video signal processing unit that generates, from the pixel signal, an information signal of a luminance change component of a luminance region of the luminance reduction video signal and a gain up signal of a pixel signal of the luminance region and adds the information signal and the gain up signal to the luminance reduction video signal.
(24) The imaging apparatus according to (23), in which
[0133] the first video signal processing unit is configured to generate, from the pixel signal, the luminance reduction video signal by knee processing.
(25) The imaging apparatus according to (23) or (24), in which
[0134] the second video signal processing unit is configured to generate the gain up signal by multiplying the pixel signal of the luminance region of the luminance reduction video signal by a gain determined in accordance with a luminance value of the pixel signal.
(26) The imaging apparatus according to any one of (23) to (25), in which
[0135] the first dynamic range is a high dynamic range (HDR), and
[0136] the second dynamic range is a standard dynamic range (SDR).
(27) The imaging apparatus according to any one of (23) to (26), further including
[0137] a first output unit that outputs an output video signal generated by the second video signal processing unit.
(28) The imaging apparatus according to (27), in which
[0138] the first output unit is configured to output the output video signal to a view finder.
(29) The imaging apparatus according to any one of (23) to (28), in which
[0139] the second video signal processing unit is further configured to generate, from the pixel signal, the information signal of the luminance change component of the luminance region of the luminance reduction video signal and generate an information signal of a luminance change component of a region belonging to a higher-luminance side as compared to the luminance region of the luminance reduction video signal.
(30) The imaging apparatus according to (29), in which
[0140] the second video signal processing unit is configured to add the generated information signal of the luminance change component of the region belonging to the higher-luminance side with the information signal of the luminance change component of the luminance region of the luminance reduction video signal and the gain up signal of the pixel signal of the luminance region to the luminance reduction video signal.
(31) The imaging apparatus according to any one of (23) to (30), further including
[0141] a third video signal processing unit that generates, from the pixel signal obtained by the imaging unit, a video signal with the first dynamic range.
(32) The imaging apparatus according to (31), further including a second output unit that outputs the video signal with the first dynamic range.
(33) The imaging apparatus according to (31), in which
[0142] the second video signal processing unit is configured to generate, from the pixel signal, an information signal of a luminance change component of a region belonging to a higher-luminance region as compared to the luminance region of the luminance reduction video signal and further add the information signal to the video signal with the first dynamic range.
REFERENCE SIGNS LIST
[0143] 10 imaging unit [0144] 20 video signal processing method [0145] 30 view finder [0146] 210 first video signal processing unit [0147] 220 second video signal processing unit [0148] 221 dynamic contrast extraction unit [0149] 222 original signal gain up processing unit