Image capturing apparatus and method of controlling the same
09742984 · 2017-08-22
Assignee
Inventors
Cpc classification
H04N23/81
ELECTRICITY
H04N25/702
ELECTRICITY
H04N25/75
ELECTRICITY
H04N25/704
ELECTRICITY
H04N9/68
ELECTRICITY
International classification
H04N9/68
ELECTRICITY
Abstract
An image capturing apparatus comprises an image pickup device with a plurality of pixels each of which includes at least two photoelectric conversion portions, a readout unit configured to read out a first image signal and an added signal obtained by adding the first image signal and a second image signal, a subtraction unit configured to subtract the first image signal from the added signal, a focus detection unit configured to detect a focus state based on the first and the second image signal, and a limiter unit configured to suppress an output of the first photoelectric conversion portion and an output of the second photoelectric conversion portion not to exceed a predetermined threshold, wherein the limiter unit suppresses the output of the first photoelectric conversion portion and the second photoelectric conversion portion for different color filters.
Claims
1. An imaging apparatus including an image sensor having a plurality of pixels each including at least two photoelectric conversion portions and a focus adjustment unit for performing a phase-difference detection type focus adjustment using image signals detected by the photoelectric conversion portions the imaging apparatus comprising: a readout unit configured to readout a first image signal from each of the photoelectric conversion portions and added image signal as a sum of the first image signal; a subtraction unit configured to obtain a second image signal by subtracting the first image signal from the added signal; and a suppression unit configured to suppress at least the second image signal not to exceed a predetermined value before performing the focus adjustment, wherein the subtraction unit subtracts a luminance signal generated from the first image signal of the luminance signal generated from the added image signal, and wherein the suppression unit suppresses the luminance signal of the second image signal obtained by the subtraction unit.
2. The imaging apparatus according to claim 1, wherein the suppression unit suppresses the first image signal not to exceed the predetermined value.
3. The imaging apparatus according to claim 1, further comprising a generation unit configured to generate the luminance signal corresponding to each of the first image signal and the added image signal.
4. The imaging, apparatus according to claim 3, wherein the image sensor includes on-chip Bayer arrangement color filters, and wherein the generation unit generates the luminance signal by combining the image signals corresponding to the color mosaic filters of red, green, and blue.
5. The imaging apparatus according to claim 1, further comprising a detection unit configured to detect a defocus amount in a predetermined area based on the first image signal and the second image signal to perform the focus adjustment.
6. The imaging apparatus according to claim 5, further comprising a processing unit configure to generate a 3D image based on the first image signal and the second image signal.
7. The imaging apparatus according to claim 1, wherein the predetermined value is less than or equal to ½ an upper limit value, the upper limit value being a maximum amount of light which the photoelectric conversion portions can receive.
8. The imaging apparatus according to claim 1, further comprising a determination unit configured to perform saturation determination for at least the second image signal, wherein the suppression unit suppresses the second image signal in a case that the determination unit determines the second image signal is in a saturation state.
9. A method of controlling an imaging apparatus including an image sensor having a plurality of pixels each including at least two photoelectric conversion portions and a focus adjustment unit for performing a phase-difference detection type focus adjustment using image signals detected by the photoelectric conversion portions, the method comprising: reading out a first image signal from each of the photoelectric conversion portions and added image signal as a sum of the first image signal; obtaining a second image signal by subtracting the first image signal from the added image signal; and suppressing at least the second image signal not to exceed a predetermined value before performing the focus adjustment, wherein in the obtaining, a luminance signal generated from the first image signal is subtracted from the luminance signal generated from the added image signal, and wherein in the suppressing, the luminance signal of the second image signal obtained by the subtraction is suppressed.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
DESCRIPTION OF THE EMBODIMENTS
(8) An embodiment of the present invention will be described in detail below with reference to the accompanying drawings.
(9)
(10) Reference numeral 107 denotes an image pickup device which includes pixels capable of performing focus detection and is composed of a CMOS sensor and its peripheral circuits. A two-dimensional, single-CCD color sensor in which light-receiving pixels arranged M pixels in the horizontal direction by N pixels in the vertical direction and an on-chip Bayer arrangement primary color mosaic filter is disposed is used as the image pickup device 107. Each pixel of the image pickup device 107 includes a plurality of photoelectric conversion portions and a color filter.
(11) Reference numeral 111 denotes a zoom actuator which carries out a variable magnification operation by rotating a cam cylinder (not shown) manually or by the actuator to drive the first lens group 101 through the third lens group 105 along the optical axis; 112, a stop actuator which controls the diameter of the opening of the stops 102 and adjusts the amount of light for imaging, and also controls the exposure time in imaging of a still image; and 114, a focus actuator which drives the third lens group 105 along the optical axis to adjust the focus.
(12) Reference numeral 121 denotes a CPU which includes an arithmetic unit, ROM, RAM, A/D converter, D/A converter, and communication interface circuit for performing various kinds of control of the camera body. In addition, based on predetermined programs stored in the ROM, the CPU 121 drives the various circuits of the camera, and executes a series of operations of focus control (AF), imaging, image processing, recording, and the like.
(13) Reference numeral 122 denotes an image pickup device drive circuit which controls the image capturing operation of the image pickup device 107 and A/D-converts obtained image signals to transmit the converted image signals to the CPU 121; 123, an image processing circuit which performs processing such as color interpolation, y conversion, and image compression on the images obtained by the image pickup device 107; and 124, a phase difference calculation processing circuit serving as a focus detection unit, which obtains the image shift amount between an A image and a B image by correlation calculation using, as an AF A image signal and an AF B image signal, signals obtained from the two photoelectric conversion portions of each pixel of the image pickup device 107, thereby calculating a focus shift amount (focus state). Furthermore, reference numeral 125 denotes a focus drive circuit which controls to drive the focus actuator 114 based on the focus detection result to drive the third lens group 105 along the optical axis, thereby performing focus adjustment; 126, a stop drive circuit which controls to drive of the stop actuator 112, thereby controlling the opening of the stop 102; and 127, a zoom drive circuit which drives the zoom actuator 111 according to the zoom operation of the user.
(14) Reference numeral 128 denotes a limiter unit which suppresses the added signal of the A image signal and the B image signal not to exceed a predetermined threshold; and 129, a control unit which generates an A image and (A+B) image by controlling the limiter unit when the added signal of the A image signal and the B image signal exceeds the predetermined threshold in the limiter unit 128.
(15) Reference numeral 131 denotes a display unit such as an LCD which displays information about the imaging mode of the camera, a preview image in imaging, a confirmation image after imaging, and an in-focus state display image in focus detection; 132, an operation unit which is constituted by a power switch, imaging start switch, zoom operation switch, imaging mode selection switch, and the like; and 133, a detachable flash memory which records captured images including a moving image and a still image.
(16)
(17) In this embodiment, the photoelectric conversion portion of each of all the pixels is divided into two areas in the X direction. With respect to a photoelectric conversion signal of each divided area, a signal can be independently read out from the first photoelectric conversion portion 211a for each color filter but a signal cannot be independently read out from the second photoelectric conversion portion 211b. The signal of the second photoelectric conversion portion 211b is calculated by subtracting the signal of the first photoelectric conversion portion 211a from a signal read out after adding the outputs of the first photoelectric conversion portion and the second photoelectric conversion portion.
(18) The signals from the first photoelectric conversion portion 211a and the second photoelectric conversion portion 211b can be used not only for focus detection using the phase difference detection method in a method (to be described later) but also for generating a 3D (3-dimensional) image formed by a plurality of images having parallax information. On the other hand, information obtained by adding the outputs of the divided photoelectric conversion portions is used as a general captured image.
(19) Pixel signals when performing focus detection using the phase difference detection method will now be described. In this embodiment, the microlens 211i and the divided photoelectric conversion portions 211a and 211b of
(20)
(21) Note that the image pickup device 107 according to this embodiment has two types of readout modes. The first readout mode is called an all-pixel readout mode, which is used to capture a high-resolution still image. In this case, signals are read out from all the pixels. The second readout mode is called a thinning readout mode, which is used to only record a moving image or display a preview image. In this case, since the number of pixels required is smaller than the total number of pixels, the apparatus reads out signals from only pixels remaining after thinning out the pixel groups at a predetermined ratio in both the X and Y directions.
(22) It is only necessary to read out an (A+B) image for a general image for imaging. To detect a phase difference in a plurality of distance measurement areas, however, an A image signal and a B image signal are read out and the image shift amount between the A image and the B image is detected by correlation calculation, thereby calculating a defocus amount.
(23)
(24) Referring to
(25) The pixel 211 includes, from the lowermost layer, photoelectric conversion portions 211a and 211b, wiring layers 211e to 211g, a color filter 211h, and the on-chip microlens 211i. The on-chip microlens 211i projects the photoelectric conversion portions 211a and 211b onto the exit pupil plane of the imaging optical system. Projection images are represented by EP1a and EP1b.
(26) If the stop 102 is in the full-aperture state (for example, F2.8), the outermost portion of the light beam passing through the imaging optical system is represented by L(F2.8). The projection images EP1a and EP1b are not eclipsed by the stop opening. On the other hand, if the stop 102 is in the stopped-down-aperture state (for example, F5.6), the outermost portion of the light beam passing through the imaging optical system is represented by L(F5.6). The outer sides of the projection images EP1a and EP1b are eclipsed by the stop opening. Note that at the center of the image plane, the eclipsed states of the projection images EP1a and EP1b are symmetrical with respect to the optical axis, and the amounts of light received by the photoelectric conversion portions 211a and 211b are equal.
(27) A measure taken when the output value of the photoelectric conversion portion of this embodiment exceeds an upper limit value (is saturated) will be explained next. Each of the photoelectric conversion portions of each pixel receives a light amount from the light beam passing through the imaging optical system, and outputs a signal corresponding to the light amount by photoelectric conversion. In the case of a high-luminance object with a large amount of light, however, the upper limit value of the light amount which can be accumulated in the photoelectric conversion portions 211a and 211b is exceeded to cause leakage of charges to the adjacent photoelectric conversion portions, resulting in crosstalk. Crosstalk occurs between the A image signal generated from the photoelectric conversion portion 211a and the B image signal generated from the photoelectric conversion portion 211b, resulting in image shift amount error between the A image signal and the B image signal. Therefore, an error occurs in the defocus amount obtained by detecting the image shift amount by correlation calculation, thereby disabling setting of a desired object in the in-focus state.
(28) In this embodiment, in a process of generating a B image signal, a B image signal is generated by subtracting an A image signal from an (A+B) image signal. An outputtable upper limit value is set for an image signal. In this embodiment, the same upper limit value is set for the image signals of the A, B, and (A+B) images. When the image signal of the A image reaches the outputtable upper limit value, the output signal of the (A+B) image also reaches the upper limit value. As a result, the (A+B) image signal and the A image signal also reach the upper limit value. That is, when the A image signal takes the upper limit value, the (A+B) image signal takes the same upper limit value, and the B image signal is generated by subtracting the A image signal from the (A+B) image signal, thereby outputting 0. In this case, the A image signal takes the upper limit value and the B image signal takes 0, resulting in generation of an error image signal. Therefore, even if the image shift amount between the A image and the B image is detected by correlation calculation to calculate a defocus amount, a desired object cannot be set in the in-focus state. Furthermore, even if the A image signal has not reached the upper limit value, when the (A+B) image is in a saturated state, the image signal is lost upon generating a B image. In this case, even if a defocus amount is calculated from the image shift amount between the A image and the B image by correlation calculation, a desired object cannot be set in the in-focus state.
(29) As described above, to set a high-luminance object in the in-focus state even if each pixel is saturated, it is necessary to control the image signals so that the A image signal and the (A+B) image signal do not reach the upper limit value. In the embodiment, the limiter unit 128 for suppressing the A image signal and the B image signal not to exceed the predetermined threshold is provided and the control unit 129 for controlling the limiter unit 128 is provided, thereby controlling the image signals not to reach the upper limit value.
(30) In this embodiment, since the A image signal is converted into a luminance signal by adding the pixel values of the color filters of green (to be referred to as G1 hereinafter) and red (to be referred to as R hereinafter) of the odd-numbered rows and those of the color filters of blue (to be referred to as B hereinafter) and green (to be referred to as G2 hereinafter) of the even-numbered rows, a threshold is set for each of G1, R, B, and G2. With this arrangement, even if the value of the specific color G1, R, B, or G2 reaches the upper limit value, the limiter unit 128 sets a threshold, and the control unit 129 suppresses each image signal not to exceed the threshold.
(31) For the B image signal, the limiter unit 128 sets a threshold with respect to its luminance signal. This is done for the following reason. That is, processing of generating a B image for each of G1, R, B, and G2 is equivalent to processing of temporarily storing each of the A image and (A+B) image for each of G1, R, B, and G2, and generating G1, R, B, and G2 of a B image. The scales of a circuit for storing a signal, a circuit for generating a signal, and the like become large. Therefore, the signal of a B image is generated from the luminance signals of the A image and (A+B) image. For this reason, for the B image, the limiter unit 128 sets a threshold with respect to its luminance signal, and the control unit 129 suppresses the luminance signal not to exceed the threshold.
(32) A saturation determination method for controlling a signal from each photoelectric conversion portion of the embodiment not to exceed the upper limit value will be described with reference to
(33)
(34) A case in which saturation determination is performed will be described with reference to
(35) In an area in which the peripheral portion of the image pickup device, that is, the image height is large, the diameter of the exit pupil becomes small due to eclipse (vignetting) of the imaging optical system. The received light amount of the pixel decreases, and the received light amounts of the two photoelectric conversion portions become unequal. As the opening diameter of the stop decreases, the nonuniformity of the received light amount becomes significant. As described above, the received light amounts of the two photoelectric conversion portions 211a and 211b in one pixel may be different from each other. Saturation determination when the values of the A image signal and the B image signal as the signals from the two photoelectric conversion portions 211a and 211b are different from each other will be explained with reference to
(36)
(37) In this embodiment, as shown in
(38) An operation according to this embodiment will be described with reference to the flowchart shown in
(39) As described above, after the (A+B) image signal is converted into a luminance signal, the B image is generated and saturation determination is performed for the luminance signal of the B image. Although it is possible to perform saturation determination for the R, G1, B, and G2 signals of the B image, an additional memory for storing these signals is required, thereby increasing the circuit scale. As in this embodiment, by adding the R, G1, B, and G2 signals of each of the (A+B) image and the B image to obtain a corresponding luminance signal, and performing saturation determination for the B image using its luminance signal, it is possible to decrease the circuit scale and calculate a desired defocus amount. The above procedure allows calculation of a defocus amount from the image shift amount between the A image and the B image by correlation calculation.
(40) While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
(41) This application claims the benefit of Japanese Patent Application No. 2013-082499, filed Apr. 10, 2013, which is hereby incorporated by reference herein in its entirety.