Image sensor and image capturing apparatus

Abstract

An image sensor comprises a plurality of pixels, each pixel including: a light-receiving element that outputs an output voltage that varies in response to a photon entering; at least one comparator that compares the output voltage with a plurality of mutually-different threshold voltages and outputs a single signal each time the output voltage varies so as to exceed either of the threshold voltages; and at least one counter that counts a number of signals output by the comparator upon comparing the output voltage with the plurality of threshold voltages and outputs a count value, for each of the threshold voltages.

Claims

1. An image sensor comprising a plurality of pixels, each pixel including: an avalanche photodiode that is driven in Geiger mode and outputs an output voltage that varies in response to a photon entering; a quenching resistor; at least one comparator that compares the output voltage with a first threshold voltage or a second threshold voltage different from the first threshold voltage and outputs a single signal each time the output voltage varies so as to exceed either of the first and second threshold voltages; and at least one counter that counts a number of signals output by the comparator upon comparing the output voltage with the first and second threshold voltages and outputs a count value, for each of the first and second threshold voltages.

2. The image sensor according to claim 1, wherein the comparator includes: a first comparator that compares the output voltage with the first threshold voltage and outputs a single signal each time the output voltage varies so as to exceed the first threshold voltage; and a second comparator that compares the output voltage with the second threshold voltage and outputs a single signal each time the output voltage varies so as to exceed the second threshold voltage, and the counter includes: a first counter that counts a number of signals output from the first comparator and outputs a first count value; and a second counter that counts a number of signals output from the second comparator and outputs a second count value.

3. The image sensor according to claim 2, wherein a difference between the output voltage and the first threshold voltage when a photon is not entering is lower than a difference between the output voltage and the second threshold voltage when a photon is not entering.

4. The image sensor according to claim 1, wherein each of the plurality of pixels further includes a switch that switches between the first threshold voltage and the second threshold voltage and supplies the voltage to the comparator; and the counter counts a number of signals output from the comparator when comparing the output voltage with the first threshold voltage and outputs a first count value, and counts a number of signals output from the comparator when comparing the output voltage with the second threshold voltage and outputs a second count value.

5. The image sensor according to claim 4, wherein a difference between the output voltage and the first threshold voltage when a photon is not entering is lower than a difference between the output voltage and the second threshold voltage when a photon is not entering.

6. The image sensor according to claim 1, wherein the plurality of pixels are all the pixels in the image sensor.

7. The image sensor according to claim 1, further comprising: a determinator that determines that a pixel for which a difference between the count values obtained for the first and second threshold voltages is greater than or equal to a predetermined threshold is a saturated pixel.

8. The image sensor according to claim 7, wherein a predetermined count value is output to the exterior of the image sensor from the pixel determined to be a saturated pixel.

9. An image sensor comprising: a pixel array including a plurality of first pixels and a plurality of second pixels, each pixel having an avalanche photodiode that is driven in Geiger mode and outputs an output voltage that varies in response to a photon entering, wherein each of the first pixels includes: a first quenching resistor; a first generator that generates a first threshold voltage; a first comparator that compares the output voltage with the first threshold voltage and outputs a single signal each time the output voltage varies so as to exceed the first threshold voltage; and a first counter that counts a number of signals output from the first comparator and outputs a first count value, and each of the second pixels includes: a second quenching resistor; a second generator that generates a second threshold voltage different from the first threshold voltage; a second comparator that compares the output voltage with the second threshold voltage and outputs a single signal each time the output voltage varies so as to exceed the second threshold voltage; and a second counter that counts a number of signals output from the second comparator and outputs a second count value.

10. The image sensor according to claim 9, wherein a difference between the output voltage and the first threshold voltage when a photon is not entering is lower than a difference between the output voltage and the second threshold voltage when a photon is not entering.

11. The image sensor according to claim 9, further comprising: a determinator that determines that a pixel for which a difference between the count values obtained for the first and second threshold voltages is greater than or equal to a predetermined threshold is a saturated pixel.

12. The image sensor according to claim 11, wherein a predetermined count value is output to the exterior of the image sensor from the pixel determined to be a saturated pixel.

13. An image capturing apparatus comprising: the image sensor comprising a plurality of pixels, each pixel including: an avalanche photodiode that is driven in Geiger mode and outputs an output voltage that varies in response to a photon entering; a quenching resistor; at least one comparator that compares the output voltage with a first threshold voltage or a second threshold voltage different from the first threshold voltage and outputs a single signal each time the output voltage varies so as to exceed either of the first and second threshold voltages; and at least one counter that counts a number of signals output by the comparator upon comparing the output voltage with the first and second threshold voltages and outputs a count value, for each of the first and second threshold voltages; and a determinator that determines that a pixel for which a difference between the count values obtained for the first and second threshold voltages output from the image sensor is greater than or equal to a predetermined threshold is a saturated pixel.

14. An image capturing apparatus comprising: the image sensor comprising: a pixel array including a plurality of first pixels and a plurality of second pixels, each pixel having an avalanche photodiode that is driven in Geiger mode and outputs an output voltage that varies in response to a photon entering, a first quenching resistor; wherein each of the first pixels includes: a first generator that generates a first threshold voltage; a first comparator that compares the output voltage with the first threshold voltage and outputs a single signal each time the output voltage varies so as to exceed the first threshold voltage; and a first counter that counts a number of signals output from the first comparator and outputs a first count value, and each of the second pixels includes: a second quenching resistor; a second generator that generates a second threshold voltage different from the first threshold voltage; a second comparator that compares the output voltage with the second threshold voltage and outputs a single signal each time the output voltage varies so as to exceed the second threshold voltage; and a second counter that counts a number of signals output from the second comparator and outputs a second count value; and a determinator that determines that a pixel for which a difference between the count values obtained for the respective threshold voltages output from the image sensor is greater than or equal to a predetermined threshold is a saturated pixel.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention, and together with the description, serve to explain the principles of the invention.

(2) FIG. 1 is a block diagram illustrating the overall configuration of an image capturing apparatus according to first and second embodiments.

(3) FIGS. 2A and 2B are diagrams illustrating the configuration of an image sensor according to the first and second embodiments.

(4) FIG. 3 is a diagram illustrating the configuration of part of a pixel and a pixel computation unit according to the first embodiment.

(5) FIGS. 4A to 4C are diagrams illustrating examples of an output voltage value at a cathode end of an ADP and a comparator output, according to the first and second embodiments.

(6) FIG. 5 is a flowchart illustrating a counter-saturation pixel determination process and a replacement process according to the first embodiment.

(7) FIGS. 6A and 6B are diagrams illustrating an example of a pixel arrangement according to a variation on the first embodiment.

(8) FIG. 7 is a diagram illustrating the configuration of part of a pixel and a pixel computation unit according to the second embodiment.

(9) FIG. 8 is a diagram illustrating an example of a pixel arrangement according to a variation on the second embodiment.

(10) FIGS. 9A to 9E are diagrams pertaining to a photon counting type image sensor according to a conventional technique.

DESCRIPTION OF THE EMBODIMENTS

(11) Exemplary embodiments of the present invention will be described in detail in accordance with the accompanying drawings.

First Embodiment

(12) An image capturing system using a photon counting type image sensor according to a first embodiment will be described here. Note that the first embodiment describes a photon counting type image sensor having two comparators in all pixels in an image sensor, with different reference signals being input to the respective comparators.

(13) FIG. 1 is a block diagram illustrating the overall configuration of an image capturing apparatus according to the first embodiment. In FIG. 1, a lens unit 201 is constituted by a plurality of lenses including a zoom lens, and changes a focal length from a wide end to a tele end under the control of a lens drive unit 202.

(14) A mechanical shutter 203, and an aperture stop 204 (a light amount adjusting member) in the following stage, constitute an exposure amount adjusting mechanism that mechanically controls an illumination time of light incident on an image sensor 206. The shutter 203 and the aperture stop 204 are driven and controlled by a shutter/aperture drive unit 205.

(15) A subject image that has traversed the lens unit 201 including the zoom lens is formed on the image sensor 206 at an exposure amount adjusted to an appropriate amount by the shutter 203 and the aperture stop 204. The subject image, which is formed on a plurality of pixels in the image sensor 206, is converted into two-dimensional digital data in the image sensor 206, which is then sent to an image signal processing circuit 207. The image sensor 206 will be described in detail later.

(16) The image signal processing circuit 207 generates image data by carrying out various types of image signal processing such as low-pass filtering for reducing noise, shading correction, WB adjustment, and the like, as well as various types of correction such as defect correction, dark shading correction, and black subtraction, compression, and the like.

(17) A central control/processing unit 210 carries out control of and various types of operations for the image capturing apparatus as a whole. A timing generator (TG) 208 generates a drive pulse for driving the image sensor 206 on the basis of a control signal from the central control/processing unit 210. A first memory unit 209 temporarily stores the image data.

(18) A recording medium control interface (I/F) unit 211 records image data to, and reads out image data from, a recording medium 213, which is a removable storage medium such as semiconductor memory. A display unit 212 displays image data and the like. An external interface (I/F) unit 214 is an interface for communicating with an external computer or the like.

(19) A second memory unit 215 stores various types of parameters, such as processing results from the central control/processing unit 210, shooting conditions, and so on. Information regarding driving conditions of the image capturing apparatus set by a user through an operating unit 216 is sent to the central control/processing unit 210, and the image capturing apparatus is controlled as a whole on the basis of that information.

(20) FIGS. 2A and 2B illustrate the overall structure of the image sensor 206. In the present embodiment, as an example, a sensor substrate 301 and a circuit board 302 are stacked so as to be electrically connected, providing the image sensor with a stacked structure.

(21) As illustrated in FIG. 2A, a pixel array is formed on the sensor substrate 301 by arranging a plurality of pixels 303 two-dimensionally. The configuration of the pixel 303 will be described in greater detail later. A pixel computation unit 304 and a signal processing circuit 305 are formed on the circuit board 302.

(22) The pixel computation unit 304 is electrically connected to the pixels on the sensor substrate 301 by bumps or the like. The pixel computation unit 304 outputs control signals for driving the pixels 303, and carries out various types of processing upon receiving comparator outputs from the pixels 303.

(23) The pixel computation unit 304 includes a counter circuit that measures the number of pulse signals from the comparators output in response to photons entering the corresponding pixels. A count value obtained by the pixel computation unit 304 is output to the exterior of the image sensor 206 by the signal processing circuit 305.

(24) FIG. 2B is a diagram illustrating part of a color filter array used in the image sensor 206, which is included in the pixel array illustrated in FIG. 2A. This color filter arrangement is known as a Bayer array, in which a first color filter of red (R), a second color filter of green (Gr), a third color filter of green (Gb), and a fourth color filter of blue (B) are arranged in a repeating pattern. Among primary color filter arrangements, this arrangement provides a high resolution and excellent color reproduction.

(25) The configuration of the pixel 303 and part of the pixel computation unit 304 will be described next with reference to FIG. 3.

(26) The pixel 303 is constituted by a quenching resistor 101, an APD 102 serving as a light-receiving element, a first comparator 103, a second comparator 104, and resistors R.sub.A1, R.sub.A2, R.sub.B1, and R.sub.B2 for generating threshold voltages Vth.sub.A and Vth.sub.B. The constituent elements of the pixel 303 are arranged on the sensor substrate 301. Note that the other pixels included in the pixel array also have the same configuration. The pixel computation unit 304 includes a first counter 105 and a second counter 106 corresponding to each pixel 303, and those counters are arranged on the circuit board 302.

(27) The anode end of the APD 102 is grounded, while the cathode end is connected to the quenching resistor 101. A reverse bias voltage from a voltage HVDD is applied across the APD 102 via the quenching resistor 101. At this time, a voltage difference between the voltage HVDD and GND is set to be greater than or equal to a breakdown voltage for putting the APD 102 into Geiger mode.

(28) A voltage V.sub.APD at the cathode end of the APD 102 (an output voltage) is input to one input terminal of each of the first comparator 103 and the second comparator 104. Threshold voltages Vth.sub.A and Vth.sub.B, respectively obtained by the resistors R.sub.A1 and R.sub.A2 and the resistors R.sub.B1 and R.sub.B2 dividing a reference voltage Vref, are input to the other input terminal of each of the first comparator 103 and the second comparator 104.

(29) The voltage V.sub.APD at the cathode end of the APD 102 and the threshold voltage Vth.sub.A are input to the first comparator 103, and a pulse signal is output when the voltage V.sub.APD drops below and then returns above the level of the threshold voltage Vth.sub.A. Likewise, the voltage V.sub.APD at the cathode end of the APD 102 and the threshold voltage Vth.sub.B are input to the second comparator 104, and a pulse signal is output when the voltage V.sub.APD drops below and then returns above the level of the threshold voltage Vth.sub.B.

(30) The pulse signals output from the first comparator 103 and the second comparator 104 are input to the first counter 105 and the second counter 106, respectively, where the numbers of pulse signals output from the comparators are measured.

(31) A count-saturation detection process, and a process for replacing the output value of a count-saturated pixel with a count value of a count-saturated level, according to the first embodiment, will be described.

(32) FIG. 4A illustrates how the voltage V.sub.APD varies when seven photons P1 to P7 are incident in that order during a period from time t1 to time t7. The vertical axis represents the voltage V.sub.APD and the horizontal axis represents time. In other words, FIG. 4A illustrates how the voltage V.sub.APD varies from a voltage V.sub.0, arising when in a photon entry standby state, to a voltage V.sub.min, when the avalanche phenomenon occurs and the voltage value drops to a minimum.

(33) FIGS. 4B and 4C illustrate changes in the pulse signals output from the first comparator 103 and the second comparator 104, respectively. The vertical axis represents the comparator output and the horizontal axis represents time.

(34) As illustrated in FIG. 4A, the threshold voltages Vth.sub.A and Vth.sub.B of the first comparator 103 and the second comparator 104 are values between the voltage V.sub.0 when in the photon entry standby state and the minimum voltage V.sub.min when the avalanche phenomenon occurs. The threshold voltage Vth.sub.B is set to a lower value than the threshold voltage Vth.sub.A (Vth.sub.B<Vth.sub.A).

(35) Time t1 to time t7 indicate the timings at which the photons P1 to P7 enter the APD 102, respectively. The periods from time t.sub.1 to time t.sub.2, from time t6 to time t7, and from time t7 to time t8 are periods where the photons P1, P6, and P7 enter during a time period longer than the dead time. The photons P1, P6, and P7 therefore cause the voltage V.sub.APD to shift from the voltage V.sub.0 to the voltage V.sub.min, after which the cathode of the APD 102 is charged by the voltage HVDD and the voltage converges once again on the voltage V.sub.0 of the photon entry standby state.

(36) On the other hand, the periods from time t2 to time t4 and from time t4 to time t6 are periods where the photons P2, P3, P4, and P5 enter during a time period shorter than the dead time. Thus while the photons P2 and P4 cause the voltage V.sub.APD to shift from the voltage V.sub.0 to the voltage V.sub.min, the voltage reaches V.sub.min again due to the avalanche phenomenon caused by the photons P3 and P5 before the voltage across the APD 102 can completely returns to the voltage V.sub.0. The cathode is then charged by the voltage HVDD, and the voltage across the APD 102 returns to V.sub.0 in the photon entry standby state.

(37) At this time, as illustrated in FIG. 4B, the first comparator 103, which is input with the threshold Vth.sub.A as a threshold voltage, outputs individual pulse signals in response to the variation in the voltage V.sub.APD caused by the photons P1, P6, and P7 entering. However, the variation in the voltage V.sub.APD caused by the photons P2 and P3 is output as a single, combined pulse signal. Likewise, the variation in the voltage V.sub.APD caused by the photons P4 and P5 entering is output as a single, combined pulse signal.

(38) On the other hand, as illustrated in FIG. 4C, the second comparator 104, which is input with the threshold Vth.sub.B as a threshold voltage, outputs individual pulse signals in response to the variation in the voltage V.sub.APD caused by the photons P1 to P7 entering.

(39) Here, if the address (X,Y) of the pixel 303 in the two-dimensional pixel array of the image sensor 206 is (x,y), the count values obtained from the first counter 105 and the second counter 106 are denoted by KA(x,y) and KB(x,y), respectively (where 1≤X≤xmax and 1≤Y≤ymax). In this case, KA(x,y)=5 and KB(x,y)=7 in the example illustrated in FIGS. 4A to 4C.

(40) The count values KA(x,y) and KB(x,y) obtained from the pixel 303 are output to the signal processing circuit 305. The signal processing circuit 305 carries out a process for determining whether or not the pixel 303 is counter-saturated, and a process for replacing the pixel output value for a pixel determined to be counter-saturated (called a “counter-saturated pixel” hereinafter). The processing by the signal processing circuit 305 will be described next with reference to FIG. 5.

(41) First, in step S10, the count values KA(x,y) and KB(x,y) are obtained from the pixel 303, and in step S11, a difference ΔK between KA(x,y) and KB(x,y) is calculated through the following equation (1).
ΔK=KB(x,y)−KA(x,y)  (1)

(42) In step S12, the difference ΔK obtained through equation (1) is compared with a predetermined counter saturation determination threshold KthS (where KthS>0). If the difference ΔK is greater than or equal to the counter saturation determination threshold KthS, the pixel is determined to be a count-saturated pixel, and the process moves to step S13. However, if the difference ΔK is less than the counter saturation determination threshold KthS, the pixel is determined to not be a count-saturated pixel, and the process moves to step S15.

(43) In step S13, a determination flag J(x,y) is set to 1, indicating that the pixel is a count-saturated pixel, and in step S14, a count value KS of the count-saturated level (where KS>0) is selected as a pixel output value Iout(x,y). Note that, for example, the APD 102 can be exposed for a predetermined exposure time to generate a graph such as that illustrated in FIG. 9E, and the obtained maximum count value that has been changed in accordance with the actual exposure time can be used as the count value KS.

(44) On the other hand, in step S15, the determination flag J(x,y) is set to 0, indicating that the pixel is not a count-saturated pixel, and in step S16, the count value KA(x,y) is selected as the pixel output value Iout(x,y).

(45) In step S17, the determination flag J(x,y) set in step S13 or step S15, and the pixel output value Iout(x,y) selected in step S14 or step S16, are output, and the process ends. The signal processing circuit 305 carries out this processing for each pixel.

(46) The processing from step S10 to step S17 is carried out for all the pixels 303 (1≤X≤xmax, 1≤Y≤ymax) in the image sensor 206, and the determination flag J(x,y) and the pixel output value Iout(x,y) are found for each.

(47) The signal processing circuit 305 calculates a total number of saturated pixels KAS, which is the number of pixels for which the determination flag J(x,y) is set to 1, among the determination flags J(x,y) of all the pixels 303 found through the above-described processing. The pixel output values Iout(x,y) for all the pixels 303 found through the above-described processing and the total number of saturated pixels KAS are output to the image signal processing circuit 207, where various types of image signal processing, correction processing, and the like are carried out on the pixel output values Iout(x,y).

(48) The total number of saturated pixels KAS is also sent to the central control/processing unit 210 through the image signal processing circuit 207. If the total number of saturated pixels KAS is greater than or equal to a predetermined number, the central control/processing unit 210 determines that too many photos are entering the image sensor 206 and outputs a command to the shutter/aperture drive unit 205 to reduce the aperture stop 204 by one stop. The present invention is not limited thereto, however, and light reduction control may be carried out using an optical device such as an ND filter.

(49) According to the first embodiment as described thus far, in a photon counting type image sensor, it can be determined whether or not each pixel is in a count-saturation state. Pixels determined to be in a count-saturation state can have the count values replaced with count values of a count-saturated level, which are then output from the image sensor. Furthermore, if the number of pixels in the count-saturation state is greater than or equal to a predetermined number, control for reducing the number of photons entering the image sensor can be carried out on the basis of the data output from the image sensor.

(50) The foregoing describes an example in which the processing illustrated in FIG. 5 is carried out by the signal processing circuit 305 in the image sensor 206. However, the present invention is not limited thereto, and the processing may instead be carried out by a circuit outside the image sensor 206. For example, it is conceivable that the count values KA(x,y) and KB(x,y) obtained from the first counter 105 and the second counter 106 be output to the image signal processing circuit 207, with the processing illustrated in FIG. 5 being carried out by the image signal processing circuit 207.

Variation on First Embodiment

(51) The foregoing first embodiment describes a case where all the pixels 303 in the image sensor 206 have two comparators, with different reference signals being input to the respective comparators. However, the configuration may be such that two comparators are provided only for one out of every several pixels. A pixel having two comparators will be called a “determination pixel” hereinafter.

(52) FIGS. 6A and 6B illustrates examples of layouts when only some of the plurality of pixels 303 in the pixel array serve as determination pixels, where the determination pixels are indicated by hatching. The other pixels are normal pixels having only a single comparator, as illustrated in FIG. 9A.

(53) FIG. 6A illustrates a case where only the Gb pixels are determination pixels, whereas FIG. 6B illustrates a case where only one out of every four Gb pixels is a determination pixel. With a configuration in which only some of the pixels are determination pixels, whether or not a determination pixel is count-saturated can be determined as described with reference to the flowchart in FIG. 5. That determination result is then used to estimate whether or not pixels in the area near the determination pixel are count-saturated. The estimation method is not particularly limited in the present invention. For example, if more than a predetermined number of adjacent determination pixels are count-saturated, a determination of count saturation may be made, or a count saturation can be estimated from a distribution of determination pixels determined to be count-saturated pixel.

Second Embodiment

(54) A second embodiment of the present invention will be described next. The second embodiment will describe a case where, in a photon counting type image sensor in which all pixels in the image sensor 206 have a single comparator, the number of photons from the same subject is counted twice. In the present second embodiment, the overall configuration of the image capturing apparatus and the entire configuration of the image sensor 206 are the same as those described in the first embodiment with reference to FIGS. 1, 2A, and 2B, and thus descriptions thereof will be omitted here.

(55) The configuration of the pixel 303 and part of the pixel computation unit 304 according to the second embodiment will be described next with reference to FIG. 7. The configuration illustrated in FIG. 7 is used instead of the configuration described in the first embodiment with reference to FIG. 3.

(56) The pixel 303 includes a quenching resistor 701, an APD 702, a comparator 703, a selector switch 704, and the resistors R.sub.A1, R.sub.A2, R.sub.B1, and R.sub.B2 for generating the threshold voltages Vth.sub.A and Vth.sub.B input to the comparator. The constituent elements of the pixel 303 are arranged on the sensor substrate 301. Note that the other pixels included in the pixel array have the same configuration. The pixel computation unit 304 includes a counter 705 corresponding to each pixel 303, and the counter is arranged on the circuit board 302.

(57) The anode end of the APD 702 is grounded, while the cathode end is connected to the quenching resistor 701. A reverse bias voltage from a voltage HVDD is applied across the APD 702 via the quenching resistor 701. At this time, a voltage difference between the voltage HVDD and GND is set to be greater than or equal to a breakdown voltage for putting the APD 702 into Geiger mode.

(58) The voltage V.sub.APD at the cathode end of the APD 702 is input to one input terminal of the comparator 703. The switch 704 is connected to the other input terminal of the comparator 703. Switching the switch 704 makes it possible to input one of the threshold voltages Vth.sub.A and Vth.sub.B, obtained by dividing the reference voltage Vref using the resistors R.sub.A1 and R.sub.A2 or the resistors R.sub.B1 and R.sub.B2.

(59) When the voltage V.sub.APD at the cathode end of the APD 702 and the threshold voltage Vth.sub.A are input to the comparator 703, and a pulse signal is output when the voltage V.sub.APD drops below and then returns above the level of the threshold voltage Vth.sub.A. Likewise, when the voltage V.sub.APD at the cathode end of the APD 702 and the threshold voltage Vth.sub.B are input to the comparator 703, and a pulse signal is output when the voltage V.sub.APD drops below and then returns above the level of the threshold voltage Vth.sub.B.

(60) The pulse signals output from the comparator 703 are input to the counter 705, and the numbers of the pulse signals are measured.

(61) Operations of the pixel 303 and the pixel computation unit 304 configured as described above, according to the second embodiment, will be described with reference to FIGS. 4A to 4C.

(62) First, the switch 704 is switched so that the threshold voltage at the comparator 703 goes to Vth.sub.A, and the first photon count is taken. Immediately after the first photon count, the switch 704 is switched to change the threshold voltage at the comparator 703 from Vth.sub.A to Vth.sub.B, after which the second photon count is taken.

(63) Assuming the time periods of the first photon count and the second photon count are sufficiently short and a constant amount of photons are entering from the subject, the voltage V.sub.APD will vary in an almost identical manner in both the first photon count period and the second photon count period.

(64) Assuming that a photon furthermore enters at the same points in time in both the first photon count period and the second photon count period, the voltage V.sub.APD will vary in the manner illustrated in FIG. 4A. In this case, in the first photon count period, the output of the comparator 703 is the pulse signals as illustrated in FIG. 4B. Likewise, in the second photon count period, the output of the comparator 703 is the pulse signals as illustrated in FIG. 4C.

(65) Here, assume that the address of a pixel 303 in the two-dimensional pixel array of the image sensor 206 is (x,y). Assume also that the count value obtained from the counter 705 in the first photon count is KA(x,y), and the count value obtained from the counter 705 in the second photon count is KB(x,y). In this case, KA(x,y)=5 and KB(x,y)=7 in the example illustrated in FIGS. 4A to 4C.

(66) Thereafter, the count saturation determination process and the process for replacing the output value of a count-saturated pixel with a count value of a count-saturated level, described in the first embodiment with reference to FIG. 5, are carried out, and the pixel output value Iout(x,y) and the determination flag J(x,y) are obtained. The total number of saturated pixels KAS is obtained from the determination flags J(x,y) of all of the pixels 303.

(67) According to the second embodiment as described thus far, in a photon counting type image sensor, it can be determined whether or not each pixel in the image sensor is in a count-saturation state, even if each pixel has only one comparator, by switching the threshold voltage.

(68) Pixels determined to be in a count-saturation state can have the count values replaced with count values of a count-saturated level, which are then output from the image sensor. Furthermore, if the number of pixels in the count-saturation state is greater than or equal to a predetermined number, control for reducing the number of photons entering the image sensor can be carried out on the basis of the data output from the image sensor.

(69) Like the variation on the first embodiment described with reference to FIGS. 6A and 6B, some of the plurality of pixels 303 in the pixel array may be pixels configured as illustrated in FIG. 7.

Variation on Second Embodiment

(70) The foregoing second embodiment describes a case where all the pixels 303 in the image sensor 206 are configured as illustrated in FIG. 7, and the threshold voltage is switched. However, the present variation will describe another configuration in which different threshold voltages are used. Although the pixel 303 according to the present variation has the same configuration as that illustrated in FIG. 9A, one of the threshold voltages Vth.sub.A and Vth.sub.B is supplied to the comparator.

(71) FIG. 8 illustrates an example of a layout when different threshold voltages are supplied to the comparators of the pixels 303 in the pixel array. The shaded pixels 303 indicate pixels where the threshold voltage for the comparator is fixed to Vth.sub.B, whereas the other pixels indicate pixels where the threshold voltage for the comparator is fixed to Vth.sub.A.

(72) In the example illustrated in FIG. 8, the threshold voltage is fixed to Vth.sub.B only for the Gb pixels. At this time, the processing illustrated in FIG. 5 is carried out for the Gr and Gb count values, and it is determined whether or not the Gr and Gb pixels are count-saturated. On the basis of this result, it is estimated whether or not the adjacent R and B pixels are also count-saturated, specifically, a similar determination result as the Gr and Gb is obtained.

(73) The method for generating the threshold voltage Vth.sub.A and the threshold voltage Vth.sub.B is the same as that described in the first embodiment. In other words, with the layout illustrated in FIG. 8, the reference voltage Vref is divided by the resistors R.sub.B1 and R.sub.B2 in the pixels 303 that are shaded, whereas the reference voltage Vref is divided by the other resistors R.sub.A1 and R.sub.A2 in the other pixels 303. By using such a configuration, where one of a plurality of threshold voltages is supplied to the comparator of each pixel, whether or not each pixel is in a count-saturation state can be determined without increasing the size of the pixel circuits.

(74) While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

(75) This application claims the benefit of Japanese Patent Application No. 2018-139621 filed on Jul. 25, 2018 which is hereby incorporated by reference herein in its entirety.