ANALOG-TO-DIGITAL CONVERTER, SENSOR AND APPARATUS
20240107190 ยท 2024-03-28
Inventors
Cpc classification
H04N23/667
ELECTRICITY
H04N25/60
ELECTRICITY
H04N25/78
ELECTRICITY
International classification
H04N25/60
ELECTRICITY
H04N25/78
ELECTRICITY
Abstract
An analog-to-digital converter comprises: an analog-to-digital conversion unit that performs analog-to-digital conversion using ?? modulation on an image signal output from pixels; a setting unit that sets an operating frequency of the analog-to-digital conversion unit; and a generation unit that generates a clock signal having the operating frequency and supplies it to the analog-to-digital conversion unit. The setting unit sets the operating frequency based on at least one of a digital gain to be applied to the signal output from the analog-to-digital conversion unit and a shooting mode.
Claims
1. An analog-to-digital converter comprising: one or more processors and/or circuitry which function as: an analog-to-digital conversion unit that performs analog-to-digital conversion using ?? modulation on an image signal output from pixels; a setting unit that sets an operating frequency of the analog-to-digital conversion unit; and a generation unit that generates a clock signal having the operating frequency and supplies it to the analog-to-digital conversion unit, wherein the setting unit sets the operating frequency based on at least one of a digital gain to be applied to the signal output from the analog-to-digital conversion unit and a shooting mode.
2. The analog-to-digital converter according to claim 1, wherein the generation unit divides and multiplies a frequency of a predetermined clock signal to generate the clock signal having the operating frequency.
3. The analog-to-digital converter according to claim 1, wherein the setting unit sets the operating frequency so that bit precision of a signal obtained by applying a digital gain to the image signal which has undergone the analog-to-digital conversion by the analog-to-digital conversion unit is constant regardless of magnitude of the digital gain.
4. The analog-to-digital converter according to claim 3, wherein, given that a predetermined digital gain is M, the setting unit obtains the operating frequency by multiplying a predetermined frequency by M/N in a case where the digital gain to be applied to the image signal is N.
5. The analog-to-digital converter according to claim 3, wherein the digital gain is determined based on at least one of ISO sensitivity and a correction method including peripheral illumination correction.
6. The analog-to-digital converter according to claim 1, wherein the shooting mode is one of a plurality of shooting modes including at least one of a high-quality still image mode, a still image mode, a Log moving image mode, a moving image mode, a live view mode, and a Hybrid Log Gamma (HLG) method and a Perceptual Quantization (PQ) method of High Dynamic Range (HDR) mode.
7. The analog-to-digital converter according to claim 6, wherein the setting unit obtains the operating frequency by multiplying a predetermined frequency by a first magnification in a case of the PQ method, a second magnification smaller than the first magnification in a case of the Log moving image mode or the HLG method, a third magnification smaller than the second magnification in a case of the high-quality still image mode, a fourth magnification lower than the third magnification in a case of the still image mode, a fifth magnification lower than the fourth magnification in a case of the moving image mode, and a sixth magnification lower than the fifth magnification in a case of the live view mode.
8. The analog-to-digital converter according to claim 1, wherein the setting unit obtains the operating frequency based on at least one of the digital gain and the shooting mode, and in a case where the obtained operating frequency is a frequency that can be generated by the generation unit, sets the obtained operating frequency without changing it.
9. The analog-to-digital converter according to claim 1, wherein the setting unit obtains the operating frequency based on at least one of the digital gain and the shooting mode, and in a case where the obtained operating frequency is over an upper limit of a frequency that can be generated by the generation unit, sets the upper limit of the frequency.
10. The analog-to-digital converter according to claim 1, wherein the setting unit obtains the operating frequency based on at least one of the digital gain and the shooting mode, and in a case where the obtained operating frequency is below a lower limit of a frequency that can be generated by the generation unit, sets the lower limit of the frequency.
11. A sensor comprising: a plurality of pixels; and an analog-to-digital converter comprising one or more processors and/or circuitry which function as: an analog-to-digital conversion unit that performs analog-to-digital conversion using ?? modulation on an image signal output from the pixels; a setting unit that sets an operating frequency of the analog-to-digital conversion unit; and a generation unit that generates a clock signal having the operating frequency and supplies it to the analog-to-digital conversion unit, and wherein the setting unit sets the operating frequency based on at least one of a digital gain to be applied to the signal output from the analog-to-digital conversion unit and a shooting mode.
12. The sensor according to claim 11, wherein the generation unit divides and multiplies a frequency of a predetermined clock signal to generate the clock signal having the operating frequency.
13. The sensor according to claim 11, wherein the setting unit sets the operating frequency so that bit precision of a signal obtained by applying a digital gain to the image signal which has undergone the analog-to-digital conversion by the analog-to-digital conversion unit is constant regardless of magnitude of the digital gain.
14. The sensor according to claim 11, wherein the shooting mode is one of a plurality of shooting modes including at least one of a high-quality still image mode, a still image mode, a Log moving image mode, a moving image mode, a live view mode, and a Hybrid Log Gamma (HLG) method and a Perceptual Quantization (PQ) method of High Dynamic Range (HDR) mode.
15. The sensor according to claim 11, wherein the setting unit obtains the operating frequency based on at least one of the digital gain and the shooting mode, and in a case where the obtained operating frequency is a frequency that can be generated by the generation unit, sets the obtained operating frequency without changing it.
16. An apparatus comprising: a sensor that includes: a plurality of pixels; and an analog-to-digital converter comprising one or more processors and/or circuitry which function as: an analog-to-digital conversion unit that performs analog-to-digital conversion using ?? modulation on an image signal output from the pixels; a setting unit that sets an operating frequency of the analog-to-digital conversion unit; and a generation unit that generates a clock signal having the operating frequency and supplies it to the analog-to-digital conversion unit; and a signal processing unit that processes the image signal converted by the analog-to-digital converter, wherein the setting unit sets the operating frequency based on at least one of a digital gain to be applied to the signal output from the analog-to-digital conversion unit and a shooting mode.
17. The apparatus according to claim 16, wherein the generation unit divides and multiplies a frequency of a predetermined clock signal to generate the clock signal having the operating frequency.
18. The apparatus according to claim 16, wherein the setting unit sets the operating frequency so that bit precision of a signal obtained by applying a digital gain to the image signal which has undergone the analog-to-digital conversion by the analog-to-digital conversion unit is constant regardless of magnitude of the digital gain.
19. The apparatus according to claim 16, wherein the shooting mode is one of a plurality of shooting modes including at least one of a high-quality still image mode, a still image mode, a Log moving image mode, a moving image mode, a live view mode, and a Hybrid Log Gamma (HLG) method and a Perceptual Quantization (PQ) method of High Dynamic Range (HDR) mode.
20. The apparatus according to claim 16, wherein the setting unit obtains the operating frequency based on at least one of the digital gain and the shooting mode, and in a case where the obtained operating frequency is a frequency that can be generated by the generation unit, sets the obtained operating frequency without changing it.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the disclosure, and together with the description, serve to explain the principles of the disclosure.
[0010]
[0011]
[0012]
[0013]
[0014]
[0015]
[0016]
[0017]
[0018]
DESCRIPTION OF THE EMBODIMENTS
[0019] Hereinafter, embodiments will be described in detail with reference to the attached drawings. Note, the following embodiments are not intended to limit the scope of the disclosure, and limitation is not made to an invention that requires a combination of all features described in the embodiments. Two or more of the multiple features described in the embodiments may be combined as appropriate. Furthermore, the same reference numerals are given to the same or similar configurations, and redundant description thereof is omitted.
First Embodiment
[0020]
[0021] In
[0022] Note that the lens unit 101 may be configured integrally with the image capturing apparatus, or may be configured to be detachable. A mechanical shutter 103 is controlled by a shutter actuation device 104. The image sensor 105 photoelectrically converts the optical image of the subject formed by the lens unit 101 and outputs an image signal.
[0023] A signal processing circuit 106 performs digital gain processing for applying a digital gain to the image signal output from the image sensor 105, various corrections, data compression, etc., and outputs image data. A memory unit 107 is used to temporarily store the image data. A system control unit 108 performs various operations and controls the entire image capturing apparatus. An I/F unit 109 is an interface for performing recording and reading out on a recording medium 110, and the recording medium 110 is a semiconductor memory such as a flash memory for holding image data, and the like, and is detachably configured. A display unit 111 displays various information and captured images.
[0024] Next, an operation at the time of shooting in the image capturing apparatus having the above configuration will be briefly described.
[0025] When a main power supply switch (not shown) is turned on, power supply to the system control unit 108 is started, and power supply to the imaging circuits such as the signal processing circuit 106 is also started. Then, when a release button (not shown) is pushed or the like to instruct shooting, shooting operation is started. After the shooting operation is finished, an image signal output from the image sensor 105 is subjected to image processing including various corrections and digital gain processing in the signal processing circuit 106, and the obtained image data is written to the memory unit 107 in response to an instruction from the system control unit 108. The image data held in the memory unit 107 is recorded on the detachable recording medium 110 such as a semiconductor memory through the I/F unit 109 under the control of the system control unit 108.
[0026] Also, ROW image data may be sent to an external computer or the like via an external I/F unit (not shown), and image processing may be performed in the computer or the like.
[0027]
[0028] The image sensor 105 includes a pixel section 200, a vertical scanning unit 202, a timing generator (TG) 203, a circuit control unit 204, a CLK generation circuit 205, column output lines 206, a signal readout unit 207 and an output unit 209.
[0029] A plurality of pixels 201 are arranged in a matrix in the pixel section 200. For simplicity of explanation, the pixels 201 are shown as an array of 4?4 pixels in the pixel section 200, but in practice a large number of pixels, for example several millions or more, are arranged. The signal readout unit 207 has a plurality of readout circuits 208.
[0030] The vertical scanning unit 202 selects the pixels 201 of the pixel section 200 in units of row and supplies plural kinds of actuation signals to each pixel 201 of the selected row. As a result, the pixel signals of the pixels 201 in the selected row are output to the signal readout unit 207 via the column output lines 206. The output pixel signals are converted into digital signals in the respective readout circuits 208 and output to the outside of the image sensor 105 via the output unit 209.
[0031] The readout circuit 208 A/D-converts the input pixel signal into a digital signal of a predetermined number of bits. Here, a so-called ?? A/D conversion method is used for the A/D conversion.
[0032] The output unit 209 converts the digital signal of each pixel into a predetermined signal format and outputs it to the outside of the image sensor 105 from the transmission line.
[0033] The TG 203 sends a timing signal to the vertical scanning unit 202, and the vertical scanning unit 202 generates control signals for actuating the pixels 201 based on the timing signal and actuates the pixels 201. The circuit control unit 204 controls the CLK generation circuit 205 and the signal readout unit 207 based on the timing signal from the TG 203. Further, the CLK generation circuit 205 generates a clock signal to be supplied to the signal readout unit 207 based on a clock signal supplied from the TG 203 and control by the circuit control unit 204.
[0034]
[0035] The readout circuit 208 includes an analog-to-digital converter (?? ADC) 300 using ?? modulation and a digital filter 301. The ?? ADC 300 converts the pixel signal input via the column output line 206 into a digital signal using ?? modulation. The digital filter 301 removes out-of-band quantization noise shifted to a higher frequency by the ?? modulation. Also, the digital filter 301 reduces the output rate and/or converts the pixel signal into a multi-bit signal by thinning out and/or taking moving average of the high-rate output of the ?? ADC 300.
[0036]
[0037] As shown in
[0038] The subtraction circuit 400 outputs the difference between the pixel signal input via the column output line 206 and the output signal of the DAC 403 to the integration circuit 401. Note that, as an embodiment of the disclosure, a configuration in which a sample-and-hold circuit is provided between the column output line 206 and the subtraction circuit 400 may be used. In that case, the pixel signal output from the sample-and-hold circuit is input to the subtraction circuit 400.
[0039] The integration circuit 401 has an integrator that integrates the input signal from the subtraction circuit 400. As the integration circuit 401, a general integration circuit such as a gm-C integration circuit using a transconductor or an RC integration circuit using an operational amplifier may be used.
[0040] The comparator 402 compares a reference voltage Vref with a voltage signal output from the integration circuit 401 in synchronization with the clock signal, and outputs the comparison result as a 1-bit digital signal. For example, when the voltage signal output from the integration circuit 401 is lower than the reference voltage Vref, 0 is output, and when it is equal to or higher than the reference voltage Vref, 1 is output. This digital signal is supplied to the digital filter 301 and the DAC 403.
[0041] The DAC 403 converts the digital signal output from the comparator 402 into a predetermined analog signal, and outputs it to the subtraction circuit 400. For example, when the digital signal output is 1, an analog signal with a predetermined level is output. Various kinds of circuits can be used as the configuration of the digital-to-analog conversion circuit.
[0042]
[0043] The CLK generation circuit 205 is composed of a frequency divider A 500, a Pulse-Locked Loop (PLL) 501, and a frequency divider B 502.
[0044] The frequency divider A 500 and the frequency divider B 502 have their division ratios controlled by the circuit control unit 204. The frequency divider A 500 divides the frequency of the clock f.sub.M given by the TG 203 by 1/M and inputs it to the PLL 501. The frequency divider B 502 divides the output of the PLL 501 by 1/N and inputs it to the PLL 501 again.
[0045] Since the PLL 501 multiplies the input clock so that the phases and frequencies of the inputs from the frequency divider A 500 and the frequency divider B 502 match, the output f.sub.OS of the PLL 501 is f.sub.M?N/M. Since f.sub.OS is the frequency of the clock signal supplied to the ?? ADC 300, it is the operating frequency of the ?? ADC 300, that is, the oversampling frequency.
[0046]
[0047] The voltage waveforms shown in
[0048]
[0049]
[0050] In this case, the change in the voltage signal that occurs up to time 10 in the example shown in
[0051] Tables 1 and 2 shown below show the relationship between ISO sensitivity setting of the image capturing apparatus, oversampling rate (hereinafter referred to as OSR), which is the ratio between the Nyquist frequency and the oversampling frequency, and bit precision.
TABLE-US-00001 TABLE 1 A/D BIT OUTPUT BIT ISO OSR PRECISION Dgain PRECISION 100 n 14 1 14 200 n 14 2 13 400 n 14 4 12 800 n 14 8 11 1600 n 14 16 10
TABLE-US-00002 TABLE 2 A/D BIT OUTPUT ISO OSR PRECISION Dgain BIT PRECISION 100 n 14 1 14 200 2n 15 2 14 400 4n 16 4 14 800 8n 17 8 14 1600 16n 18 16 14
[0052] In Table 1, the OSR is n regardless of the ISO sensitivity, and the bit precision of A/D conversion including processing by a decimation filter is 14 bits at ISO100. At this time, it is necessary to apply a digital gain (Dgain) to raise the ISO sensitivity to ISO200, ISO400, etc. Therefore, the bit precision of the final output becomes 1/Dgain, and in the case of ISO1600, it becomes 10 bits. In this case, the resulting image is adversely affected by tone jumps, quantization noise multiplied by gain, and so forth.
[0053] In order to eliminate these effects, in this embodiment, as shown in Table 2, the OSR is changed according to the ISO sensitivity. For example, in ISO1600, by setting the OSR to 16n, the bit precision of A/D conversion becomes 18 bits, and even if multiplied by Dgain of 16, the bit precision of the final output becomes 14 bits. Also, by increasing the OSR, the effect of noise shaping is increased, and the quantization noise is reduced.
[0054] In the above example, the case where the OSR is determined according to the digital gain due to the ISO sensitivity has been described, but the OSR may be changed according to the digital gain resulting from various corrections such as peripheral illumination correction. For example, in a case where a lens with a large peripheral light falloff is attached and a digital gain is applied to the periphery of an image according to the image height in the peripheral light falloff correction, by changing the OSR according to the digital gain as shown in Table 2, it is possible to reduce tone jumps and noise in the periphery of the image even after the correction is applied.
[0055] As described above, according to the first embodiment, by changing the OSR according to the digital gain, it is possible to avoid the decrease in bit precision due to the digital gain, and an image from which tone jump and an increase in quantization noise are reduced can be obtained.
[0056] In the first embodiment, the first-order ?? modulator constituting the first-order loop filter is used, but the disclosure can be applied to a circuit configuration that uses a second-order or higher-order ?? modulator in order to stabilize the feedback loop. Also, in the incremental ?? ADC, although the digital signal waveforms are different from those in
Second Embodiment
[0057] Next, a second embodiment of the disclosure will be described.
[0058] In the above-described first embodiment, the case where the OSR is changed according to the digital gain has been explained, but in the second embodiment, the case where the OSR is changed according to the shooting mode or the like will be explained. Note that the configuration of the image capturing apparatus according to the second embodiment is the same as that described with reference to
[0059] Table 3 shows an example of the shooting modes in the second embodiment, and the relationship between OSR and bit accuracy.
TABLE-US-00003 TABLE 3 MODE OSR A/D BIT PRECISION High quality still image 2n 15 Still image n 14 Log moving image 4n 16 Moving image n/4 12 LV n/16 10 HLG 4n 16 PQ 8n 17
[0060] In Table 3, the OSR in the still image mode is n and the bit precision is 14 bits, and the OSR in the moving image mode is n/4 and the bit precision is 12 bits. The difference in bit precision between the still image mode and the moving image mode is due to restrictions on the data transfer rate from the image sensor 105 to the signal processing circuit 106, for example.
[0061] In a case where the high-quality still image mode is implemented as a mode capable of acquiring a good image with less noise than in the still image mode, an image with 15-bit accuracy is obtained by setting the OSR to 2n.
[0062] In the live view (LV) mode, the image to be displayed on the display unit 111 is acquired without being recorded on the recording medium 110, so the image quality requirements are not high. Therefore, the OSR is set to low, n/16, for the purpose of reducing power consumption by reducing the amount of data.
[0063] On the other hand, in the Log moving image mode for obtaining a wide dynamic range, the OSR is increased to 4n. A Log moving image have input/output characteristics as shown in
[0064] As shown in
[0065] Therefore, although the Log moving image will be recorded as a 10-bit moving image file, by increasing the OSR and reducing quantization noise through noise shaping, it is possible to reduce noise and improve gradation in dark areas.
[0066]
[0067] As in the case of log moving images, by allocating more signal levels to dark areas in the HLG method than in the SDR mode, the gradation in the dark areas becomes richer, but the noise in the dark areas becomes more conspicuous. This tendency is even more pronounced in the PQ method. Therefore, by increasing the OSR and reducing the quantization noise by noise shaping, it is possible to reduce noise and improve gradation in dark areas.
[0068] As described above, according to the second embodiment, by changing the OSR according to the shooting mode, and the like, it is possible to obtain an image with image quality suitable for each shooting mode, and the like, and reduce power consumption.
Third Embodiment
[0069] Next, a third embodiment of the disclosure will be described.
[0070] In the third embodiment, an OSR switching control, which is a combination of the OSR switching control described in the first embodiment and the OSR switching control described in the second embodiment, will be described.
[0071]
[0072] When the power of the image capturing apparatus is turned on, the set shooting conditions are acquired in step S901. In step S902, the OSR to be set is determined based on the combination of the ISO sensitivity and the shooting mode according to the obtained current shooting conditions. For example, in the case of ISO400 and moving image mode, the OSRs are 4n and n/4 from Tables 2 and 3, respectively, so the OSR to be set is n.
[0073] In step S903, it is determined whether the OSR obtained in step S902 can be set. In one embodiment, the OSR can be set without restrictions, but in reality, there are circuit setting limits for clock frequency division and multiplication. If the OSR is within the settable range, the determined OSR is set in step S904. If the OSR is equal to or less than the settable lower limit, the lower limit is set as the OSR in step S905. If the OSR is equal to or higher than the settable upper limit, the upper limit is set as the OSR in step S906.
[0074] In step S907, it is determined whether an instruction has been given to change the ISO sensitivity and/or the shooting mode that requires to change the OSR. If the OSR needs to be changed, the process returns to step S902, and the OSR is obtained again based on the changed ISO sensitivity and/or shooting mode.
[0075] If there is no change in ISO sensitivity and/or shooting mode that requires to change the OSR, it is determined in step S908 whether shooting is instructed by pressing the release button or the like. If there is no shooting instruction, the process returns to step S907.
[0076] On the other hand, if shooting is instructed, in step S909 moving image recording or still image shooting are performed, and after a signal is processed by the signal processing circuit 106, the obtained image data is held in the memory unit 107 and recorded on the recording medium 110 via the I/F unit 109, and the processing is ended.
[0077] As described above, according to the third embodiment, by switching the OSR according to the combination of ISO sensitivity and shooting mode, it is possible to obtain an image with image quality suitable for each combination as well as to reduce power consumption.
[0078] In the example shown in
[0079] While the disclosure has been described with reference to exemplary embodiments, it is to be understood that the disclosure is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
[0080] This application claims the benefit of Japanese Patent Application No. 2022-152806, filed Sep. 26, 2022, which is hereby incorporated by reference herein in its entirety.