Electric camera

Abstract

An electric camera includes an image sensing device with a light receiving surface having N vertically arranged pixels and an arbitrary number of pixels arranged horizontally, N being equal to or more than three times the number of effective scanning lines M of a display screen of a television system, a driver to drive the image sensing device to vertically mix or cull signal charges accumulated in individual pixels of K pixels to produce, during a vertical effective scanning period of the television system, a number of lines of output signals which corresponds to 1/K the number of vertically arranged pixels N of the image sensing device, K being an integer equal to or less than an integral part of a quotient of N divided by M, and a signal processing unit having a function of generating image signals by using the output signals of the image sensing device.

Claims

1. An electric camera comprising: an image sensing device with a light receiving surface having an array of pixels arranged vertically and horizontally, and configured to output signals to generate image signals based on signal charges accumulated in the pixels with an interval of a plurality of vertical pixels; a signal processing unit configured to generate the image signals by using the signals outputted by the image sensing device; and a zoom operation unit configured to operate to continuously change a magnification factor; wherein, as the zoom operation unit is operated to continuously change the magnification factor, the image sensing device outputs the signals to generate the image signals based on the signal charges accumulated while changing the interval of the plurality of vertical pixels stepwise; wherein the electric camera has at least three states, which have different intervals of the plurality of vertical pixels, including: a first state having a first interval of the plurality of vertical pixels in which first image signals are generated using first signals based on first signal charges accumulated in a first area of the light receiving surface with a first magnification factor, a second state having a second interval of the plurality of vertical pixels in which second image signals are generated using second signals based on second signal charges accumulated in a second area of the light receiving surface with a second magnification factor, and a third state having a third interval of the plurality of vertical pixels in which third image signals are generated using third signals based on third signal charges accumulated in a third area of the light receiving surface with a third magnification factor, wherein, while a state is between the first state and the second state when the zoom operation unit is operated to continuously change the magnification factor, the electric camera is configured to change areas of the light receiving surface to be used in accordance with changing the magnification factor between the first magnification factor and the second magnification factor and keep the first interval constant, and wherein, while a state is between the second state and the third state when the zoom operation unit is operated to continuously change the magnification factor, the electric camera is configured to change areas of the light receiving surface to be used in accordance with changing the magnification factor between the second magnification factor and the third magnification factor and keep the second interval constant.

2. The electric camera according to claim 1, wherein the image sensing device outputs the signals which are mixed or culled signals accumulated in the pixels with the interval of the plurality of vertical pixels.

3. The electric camera according to claim 1, wherein, as the zoom operation unit is operated to continuously change the magnification factor during recording of a moving video in a moving video mode, the image sensing device outputs the signals to generate the image signals based on the signal charges accumulated while changing the interval of the plurality of vertical pixels stepwise.

4. The electric camera according to claim 1, wherein the electric camera has a static image mode for taking a static image and, when in the static image mode, the signal processing unit generates the static image by using all signals accumulated in all of an effective number of the vertical pixels.

5. An electric camera comprising: an image sensing device with a light receiving surface having an array of pixels arranged vertically and horizontally, and configured to output signals to generate image signals based on signal charges accumulated in the pixels with an interval of a plurality of vertical pixels; a processor circuitry configured to control generating the image signals by using the signals outputted by the image sensing device and to continuously change a magnification factor in accordance with a zoom operation; wherein, as the processor circuitry controls to continuously change the magnification factor in accordance with the zoom operation, the image sensing device outputs the signals to generate the image signals based on the signal charges accumulated while changing the interval of the plurality of vertical pixels stepwise; wherein the electric camera has at least three states, which have different intervals of the plurality of vertical pixels, including: a first state having a first interval of the plurality of vertical pixels in which first image signals are generated using first signals based on first signal charges accumulated in a first area of the light receiving surface with a first magnification factor, a second state having a second interval of the plurality of vertical pixels in which second image signals are generated using second signals based on second signal charges accumulated in a second area of the light receiving surface with a second magnification factor, and a third state having a third interval of the plurality of vertical pixels in which third image signals are generated using third signals based on third signal charges accumulated in a third area of the light receiving surface with a third magnification factor, wherein, while a state is between the first state and the second state when continuously changing the magnification factor in accordance with the zoom operation, the electric camera is configured to change areas of the light receiving surface to be used in accordance with changing the magnification factor between the first magnification factor and the second magnification factor and keep the first interval constant, wherein, while a state is between the second state and the third state when continuously changing the magnification factor in accordance with the zoom operation, the electric camera is configured to change areas of the light receiving surface to be used in accordance with changing the magnification factor between the second magnification factor and the third magnification factor and keep the second interval constant.

6. An operating method for an electric camera comprising: receiving light with an image sensing device having a light receiving surface including an array of pixels arranged vertically and horizontally, and outputting signals to generate image signals based on signal charges accumulated in the pixels with an interval of a plurality of vertical pixels; generating the image signals by using the signals outputted by the image sensing device; continuously changing a magnification factor of a zoom operation; and wherein, as the zoom operation continuously changes the magnification factor, the outputting of the signals of the image sensing device includes outputting the signals to generate the image signals based on the signal charges accumulated while changing the interval of the plurality of vertical pixels stepwise; wherein the electric camera has at least three states which have different intervals of the plurality of vertical pixels, and wherein: the generating of the image signals includes, in a first state of the at least three states having a first interval of the plurality of vertical pixels, generating first image signals using first signals output by the image sensing device based on a first area of the light receiving surface with a first magnification factor, the generating of the image signals includes, in a second state of the at least three states having a second interval of the plurality of vertical pixels, generating second image signals using second signals output by the image sensing device based on a second area of the light receiving surface with a second magnification factor, and the generating of the image signals includes, in a third state of the at least three states having a third interval of the plurality of vertical pixels, generating third image signals using third signals output by the image sensing device based on a third area of the light receiving surface with a third magnification factor, wherein, while a state is between the first state and the second state when the zoom operation continuously changes the magnification factor, changing areas of the light receiving surface to be used in accordance with changing the magnification factor between the first magnification factor and the second magnification factor and keeping the first interval constant, and wherein, while a state is between the second state and the third state when the zoom operation continuously changes the magnification factor, changing areas of the light receiving surface to be used in accordance with changing the magnification factor between the second magnification factor and the third magnification factor and keeping the second interval constant.

7. The operating method according to claim 6, wherein the outputting of the signals with the image sensing device includes outputting mixed or culled signals accumulated in the pixels with the interval of the plurality of vertical pixels.

8. The operating method according to claim 6, wherein, as the zoom operation continuously changes the magnification factor during recording of a moving video in a moving video mode, the outputting of the signals with the image sensing device includes outputting the signals to generate the image signals based on the signal charges accumulated while changing the interval of the plurality of vertical pixels stepwise.

9. The operating method according to claim 6, wherein the electric camera has a static image mode for taking a static image and, wherein, in the static image mode, the generating of the image signals includes generating the static image by using all signals accumulated in all of an effective number of the vertical pixels.

10. An operating method for an electric camera comprising: receiving light with an image sensing device having a light receiving surface having an array of pixels arranged vertically and horizontally, and outputting signals to generate image signals based on signal charges accumulated in the pixels with an interval of a plurality of vertical pixels; controlling with processor circuitry generating the image signals by using the signals outputted by the image sensing device and controlling to continuously change a magnification factor in accordance with a zoom operation; wherein, as the zoom operation is controlled with the processor circuitry to continuously change the magnification factor, the outputting of the signals of the image sensing device includes outputting the signals to generate the image signals based on the signal charges accumulated while changing the interval of the plurality of vertical pixels stepwise; wherein the electric camera has at least three states which have different intervals of the plurality of vertical pixels, and wherein: the generating of the image signals includes, in a first state of the at least three states having a first interval of the plurality of vertical pixels, generating first image signals using first signals output by the image sensing device based on first signal charges accumulated in a first area of the light receiving surface with a first magnification factor, the generating of the image signals includes, in a second state of the at least three states having a second interval of the plurality of vertical pixels, generating second image signals using second signals output by the image sensing device based on second signal charges accumulated in a second area of the light receiving surface with a second magnification factor, and the generating of the image signals includes, in a third state of the at least three states having a third interval of the plurality of vertical pixels, generating third image signals using third signals output by the image sensing device based on third signal charges accumulated in a third area of the light receiving surface with a third magnification factor, wherein, while a state is between the first state and the second state when the zoom operation continuously changes the magnification factor, changing areas of the light receiving surface to be used in accordance with changing the magnification factor between the first magnification factor and the second magnification factor and keeping the first interval constant, and wherein, while a state is between the second state and the third state when the zoom operation continuously changes the magnification factor, changing areas of the light receiving surface to be used in accordance with changing the magnification factor between the second magnification factor and the third magnification factor and keeping the second interval constant.

11. The electric camera according to claim 5, wherein the image sensing device outputs the signals which are mixed or culled signals accumulated in the pixels with the interval of the plurality of vertical pixels.

12. The electric camera according to claim 5, wherein, as the processor circuitry controls to continuously change the magnification factor during recording of a moving video in a moving video mode, the image sensing device outputs the signals to generate the image signals based on the signal charges accumulated while changing the interval of the plurality of vertical pixels stepwise.

13. The electric camera according to claim 5, wherein the electric camera has a static image mode for taking a static image and, when in the static image mode, the processor circuitry controls to generate the static image by using all signals accumulated in all of an effective number of the vertical pixels.

14. The operating method according to claim 10, wherein the outputting of the signals with the image sensing device includes outputting mixed or culled signals accumulated in the pixels with the interval of the plurality of vertical pixels.

15. The operating method according to claim 10, wherein, as the zoom operation continuously changes the magnification factor during recording of a moving video in a moving video mode, the outputting of the signals with the image sensing device includes outputting the signals to generate the image signals based on the signal charges accumulated while changing the interval of the plurality of vertical pixels stepwise.

16. The operating method according to claim 10, wherein the electric camera has a static image mode for taking a static image and, wherein, in the static image mode, the generating of the image signals includes generating the static image by using all signals accumulated in all of an effective number of the vertical pixels.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is a block diagram showing the configuration of a first embodiment of an electric camera according to the present invention.

(2) FIG. 2 is a schematic diagram showing the structure of an image sensing device in the first embodiment of the electric camera of the invention.

(3) FIG. 3 is a drive pulse timing diagram in the first embodiment of the electric camera of the invention.

(4) FIG. 4 is a schematic diagram showing a mixing operation in the first embodiment of the electric camera of the invention.

(5) FIG. 5 is a schematic diagram showing a readout area in the first embodiment of the electric camera of the invention.

(6) FIG. 6 is a schematic diagram showing a mixing operation in the first embodiment of the electric camera of the invention.

(7) FIG. 7 is a block diagram showing the configuration of a second embodiment of an electric camera according to the present invention.

(8) FIG. 8 is a schematic diagram showing a mixing operation in the second embodiment of the electric camera of the invention.

(9) FIG. 9 is a schematic diagram showing a readout area in the second embodiment of the electric camera of the invention.

(10) FIG. 10 is a schematic diagram showing the structure of an image sensing device in a third embodiment of the electric camera according to the present invention.

(11) FIG. 11 is a drive pulse timing diagram in the third embodiment of the electric camera of the invention.

(12) FIG. 12 is a schematic diagram showing an interpolation operation in the third embodiment of the electric camera of the invention.

(13) FIGS. 13A and 13B are schematic diagrams showing the arrangement of color filters in the image sensing device in a fourth embodiment of the electric camera according to the present invention.

DESCRIPTION OF THE EMBODIMENTS

(14) Now embodiments of the present invention will be described by referring to the accompanying drawings. FIG. 1 is a block diagram showing the configuration of one embodiment of an electric camera according to the invention.

(15) In FIG. 1, reference number 1 represents a lens, 2 an aperture, 3 an image sensing device, 4 a drive circuit, 5 a gain adjust circuit, 6 an analog-digital (A/D) conversion circuit, 7 a signal processing circuit, 8 a vertical interpolation circuit to perform interpolation in a vertical direction, 9 a horizontal interpolation circuit to perform interpolation in a horizontal direction, 10 a recording unit including recording media such as magnetic tape, semiconductor memory and optical disk to record a video signal, 11 a control circuit to control these constitutional elements according to the operating state, 12 an encoder circuit to modulate the video signal into a standard television signal, 13 a digital-analog (D/A) conversion circuit, 14 a mode selector switch to change over the operation mode between the moving video taking and the still image taking, 15 a record button to start or stop the recording, 16a and 16b gyro sensors to detect vertical image-unstability and lateral image-unstability, respectively, and 17 an image-unstability decision circuit to determine the image-instability from signals output from the gyro sensors.

(16) In the above configuration, light coming from the lens 1 through the aperture 2 is focused on a light receiving surface of the image sensing device 3 where it is converted into an electric signal. In this embodiment the image sensing device 3 is of a CCD type. FIG. 2 shows the structure of this image sensing device 3. In FIG. 2, denoted 30 are pixels each formed of a photodiode, which are arranged horizontally and vertically in a grid pattern. On these grid-arrayed pixels three types of color filters that pass yellow (Ye), green (G) and cyan (Cy), respectively, are arranged in such a way that the combination of these three colors is repeated horizontally every three pixels and that the filters of the same colors are lined vertically in so-called vertical stripes. Although an arbitrary number of pixels may be used, this embodiment has an array of 1200 pixels vertically and 1600 pixels horizontally. A vertical transfer unit 32 is a CCD which is driven by three phase pulses V1, V2, V3. This CCD has a three-gate structure in which each pixel corresponds to three phase pulses and thus can vertically transfer a signal charge of each pixel independently. Transfer gates 31 for transferring the charge of each pixel to the vertical transfer unit 32 are commonly connected to a gate of the vertical transfer unit 32 that corresponds to the V2 pulse. An operation to transfer the charge from each pixel to the vertical transfer unit 32 in response to a peak value of the pulse applied to the commonly connected gate and an operation to transfer the charge vertically are performed separately. A horizontal transfer unit 33 horizontally transfers the charges supplied from the vertical transfer units 32 and outputs them successively through an output amplifier 34 from the output terminal.

(17) Referring back to FIG. 1, the operation performed when the moving video mode is selected by the mode selector switch 14 will be explained. The number of vertically arranged pixels on the image sensing device in this embodiment is 1200, so if the number of effective scanning lines in the field of the NTSC system is assumed to be 240 lines, then vertically mixing five pixels (=1200 pixel rows/240 scanning lines) can match the number of lines of output signals from the image sensing device to the number of effective scanning lines.

(18) However, in this embodiment, to realize the image stabilizing function described later, four vertically arranged pixels are mixed together during motion image taking mode. When four vertically arranged pixels are to be cyclically mixed together, the signals from the area of 960 pixels (=240 scanning lines4 pixels) out of the 1200 vertically arranged pixels are used as effective signals and the remaining 240 pixels (=1200 (all pixels)960 (effective pixels)) are not used for image forming. FIG. 3 shows the timing of a vertical drive pulse for the image sensing device in this operation mode, with V1, V2 and V3 representing three phase drive pulses applied to each gate of the CCD or vertical transfer unit 32.

(19) In FIG. 3, in a period T1 included in the vertical blanking period, the drive pulse V2 is held high to transfer the signal charge accumulated in each pixel to under the V2 gate of the vertical CCD. Next, in a period T2, while the V2 pulse is still at middle level, the V3 pulse is raised from low level to middle level; next, while the V3 pulse is at middle level, the V2 pulse is changed from middle level to low level, after which the V1 pulse is changed from low level to middle level; next, while the V1 pulse is at middle level, the V3 pulse is changed from middle level to low level, after which the V2 pulse is changed to middle level and finally the V1 pulse is changed from middle level to low level. With this sequence of pulse operations, the signal charges under the V2 gate for one pixel row are transferred and held again under the V2 gate.

(20) By repeating this series of operations, the signal charges for a desired number of pixel rows can be transferred. In FIG. 3, during a period T3 included in the vertical blanking period before the vertical effective scanning period (the vertical scanning period minus the vertical blanking period which corresponds to the actually displayed image) and during a period T4 included in the vertical blanking period after the vertical effective scanning period, the above transfer operation for one pixel row is repeated a total of 240 times to transfer the signal charges of the 240 pixel rows not used for image generation to the horizontal transfer unit 33 during the vertical blanking period. For example, if this transfer operation is performed 120 times during the period T3 and 120 times during the period T4, the signal charges from upper 120 pixel rows and lower 120 pixel rows on the light receiving surface are transferred to the horizontal transfer unit 33 during the period T3 and period T4 within the vertical blanking period. During each of the subsequent periods T5 and T6 in the vertical blanking period, the horizontal transfer unit 33 is driven for a predetermined period to output the charges transferred to the horizontal transfer unit 33 from the output terminal. These charges are not used as valid signals as they are output during the vertical blanking period.

(21) Next, in the vertical effective scanning period of FIG. 3, the above one-pixel-row transfer operation is performed four times during each horizontal blanking period to transfer the signal charges of four pixel rows to the horizontal transfer unit 33 where they are mixed together. Then, during a horizontal effective scanning period (the horizontal scanning period minus the horizontal blanking period which corresponds to the actually displayed image), the horizontal transfer unit 33 is driven to read out the signal charges from the horizontal transfer unit to produce an output signal conforming to the television system. If the above operation is performed on the A field and if, on the B field, the number of pixel rows transferred during the period T3 is set to 122 rows and that during the period T4 is set to 118 rows, then the combination of four pixel rows to be cyclically mixed together shifts by two rows between the two fields, thus allowing the interlaced scanning to be performed as shown in FIG. 4. (FIG. 4 shows the light receiving surface of the image sensing device and its relation to the displayed screen is vertically inverted.)

(22) Let us return to FIG. 1. The output signal from the image sensing device 3 is adjusted in gain by the gain adjust circuit 5 and then converted by the A/D conversion circuit 6 into a digital signal. The digital signal is then processed by the signal processing circuit 7 that performs color signal processing and luminance signal processing, such as generation of color signals, gamma correction, white balance processing and outline enhancement. The image sensing device in this embodiment has an array of vertical stripes of yellow (Ye), green (G) and cyan (Cy) color filters, so the color signals for Ye, G and Cy are obtained as a series of color points from one line of output signals at all times no matter how many pixels are vertically combined. From these color signals three primary color signals R, G, B can be obtained from the following calculations.
R=YeG
B=CyG
G=G

(23) The R, G and B signals undergoes the white balance processing and gamma correction processing in the signal processing circuit 7 and are then converted into color difference signals such as RY, BY or U and V. The luminance signals and the color difference signals are then entered through the vertical interpolation circuit 8 into the horizontal interpolation circuit 9. In this operation state the signals just pass through the vertical interpolation circuit 8 without being processed. The horizontal interpolation circuit 9 performs interpolation on the signals in the horizontal direction.

(24) FIG. 5 shows the light receiving surface of the image sensing device. As described above, in the operating state of this embodiment, the signals read out during the vertical effective scanning period correspond to an area having 960 of the 1200 vertically arranged pixels and a horizontal width of 1600 pixels, as shown shaded at A in FIG. 5. If the entire light receiving surface of the image sensing device has a 4-to-3 (width to height) aspect ratio, the shaded area A is more laterally elongate than this aspect ratio. Hence, if the signals of all horizontal pixels of the light receiving surface are displayed, for example, on an NTSC standard television monitor with the 4-to-3 aspect ratio, the image displayed is compressed horizontally and looks vertically elongate, compared with the original image. It is therefore necessary to output during the horizontal effective scanning period only those signals coming from a pixel area with the horizontal width conforming to the aspect ratio of the television system, as shown by a shaded area B. When the television system has an 4-to-3 aspect ratio, the number of pixels in the horizontal width of the shaded area B is 1280 (=960 (vertical effective pixels)4/3).

(25) Returning back to FIG. 1, the horizontal interpolation circuit 9 performs interpolation processing on the signals from the horizontal 1280 pixels to expand the signals so that they can be output over the entire horizontal effective scanning period. It also performs switching among different clocks as required. With the above operation, an area having 960 pixels in height and 1280 pixels in width is demarcated from the light receiving surface as signals conforming to the television system. Then, the luminance signal and the color difference signal are encoded by the encoder circuit 12 into television signals, which are then converted by the D/A conversion circuit 13 into analog signals for output. When the recording is specified by the record button 15, the signals are recorded by the recording unit 10. At this time, the signals may be compressed in the MPEG (Moving Picture Expert Group) format.

(26) Next, the image stabilizing operation will be explained. Image-unstability information obtained by the gyro sensors 16a, 16b that detect vertical and horizontal image-unstabilities is entered into the image-unstability decision circuit 17, which checks the received information for the amount and direction of the image-unstability and converts them into the number of pixels in vertical and horizontal directions on the light receiving surface of the image sensing device. Based on the converted pixel numbers, the position of an extracted area (effective pixel area) on the light receiving surface is shifted in a direction that cancels the image-unstability. This can correct the image-unstability. The positional shifting of the extracted area is performed as follows. The shifting in the vertical direction can be made by changing the number of pixel rows transferred during the periods T3 and T4 in FIG. 3 and the shifting in the horizontal direction made by changing the interpolation start position in the horizontal interpolation circuit 9.

(27) The operation during the moving video mode has been described above. Next, the operation performed when the static image mode is selected by the mode selector switch 14 will be explained.

(28) In the static image mode, too, until the recording is requested by the record button 15, the camera outputs signals compatible with the television system to monitor the angle of view. Unlike the moving video photographing, all of the effective pixels on the image sensing device are used in this embodiment during the still image photographing to produce signals with as high a resolution as possible. Hence, during the monitoring the television signals need to be generated from the signals coming from the entire pixel area.

(29) The image sensing device of this embodiment has 1200 vertically arranged pixels, and the number of lines of output signals from the image sensing device can be made to match the number of effective scanning lines of NTSC system, which is assumed to have 240 scanning lines, by vertically mixing five pixels (=1200/240). To make the image sensing device operate in this manner, the one-pixel-row transfer operation is performed five times during each horizontal blanking period in the vertical effective scanning period shown in the pulse timing diagram of FIG. 3. With this operation, the signal charges of five pixel rows can be mixed by the horizontal transfer unit 33. As for the transfer operations during the periods T3 and T4 in the vertical blanking period, because the interlaced scanning is carried out, only two pixel rows are transferred during the period T3 on the B field, with no transfer operations performed in other vertical blanking periods (In this embodiment, 1200/240=5 with no remainder produced, so no further transfer is necessary; if, however, a remainder occurs, the remaining pixels need only be transferred during the periods T3 and T4).

(30) The charges mixed by the horizontal transfer unit 33 are read out by driving the horizontal transfer unit 33 during the horizontal effective scanning period. With the above operations, the signal charges of all pixels on the Image sensing device can be read out in a manner conforming to the television system. The output signal from the image sensing device 3 is, as during the moving image photographing, adjusted in gain by the gain adjust circuit 5 and converted by the A/D conversion circuit 6 into a digital signal, which is then subjected to the color signal processing and the luminance signal processing in the signal processing circuit 7 before being entered into the vertical interpolation circuit 8. During the static image monitoring, the vertical interpolation circuit 8 performs a vertical gravity center correction on the received signals.

(31) FIG. 6 shows combinations of pixels to be cyclically mixed on the A field and the B field and also the vertical position of the gravity center of the mixed signals. In the interlaced scanning, scanning lines of the A field and the B field are located at the centers of adjoining scanning lines on other field. Hence, the signal samplings in the camera system for the two fields must be 180 degrees out of phase in the vertical direction. In the operating state of this embodiment, however, because five pixels are mixed together, the gravity centers of the output signals for the A field and the B field are deviated 36 degrees (= pixel or 1/10 the line-to-line distance on the same field) from the ideal sampling phase difference of 180 degrees, as shown in FIG. 6. To correct this requires generating a signal from two adjoining output lines by interpolation. For example, if we let an nth output line on a field be Sn and an (n+1)th output line on the same field be Sn+1, then a signal Sn obtained by calculating Sn=(Sn 9/10)+(Sn+1 1/10) is one whose gravity center is shifted by 1/10 output line from the gravity center of the nth output line toward the (n+1)th output line. The above calculation can also be performed on the signals for the B field to correct the gravity center deviation of sampling. In this embodiment, however, to equalize the effects of interpolation of the A field and the B field, the following calculations are performed to correct the nth output line by 1/20 line toward the (n1) line on the A field and by 1/20 line toward the (n+1)th line on the B field.
Sn=(Sn 19/20)+(Sn1 1/20)A field:
Sn=(Sn 19/20)+(Sn+1 1/20)B field:

(32) While this embodiment performs the interpolation based on the calculation of two adjoining lines of output signals, a greater number of lines may be used for the interpolation processing. The output signal of the vertical interpolation circuit 8 is supplied to the horizontal interpolation circuit 9, which in this operating state does nothing but passes the signal. Then, as in the case of the moving image photographing, the signal is encoded by the encoder circuit 12 into a television signal, which is then converted by the D/A conversion circuit 13 into an analog signal for output. As described above, the television signals can be generated from all of the pixel area of the image sensing device also during the static image mode.

(33) Next, the operation performed when the recording is requested by the record button 15 will be described. During the monitoring in the static mode, the signals are mixed together inside the image sensing device to reduce the number of signals and thereby generate television signals. During recording, however, the mixing processing is not performed and all the pixel signals need to be read out independently of each other in order to produce high resolution signals. To realize this, the one-pixel-row transfer operation is performed only once during each horizontal blanking period in the vertical effective scanning period shown in the pulse timing diagram of FIG. 3. This causes only the signal charges for one pixel row to be transferred into the horizontal transfer unit 33, which is then driven to read out the signal charges for one pixel row. Repeating this operation the number of times equal to the number of vertically arranged pixel rows enables the signal charges of all pixel rows to be read out independently of each other. The transfer operation is not done during the periods T3 and T4 in the vertical blanking period.

(34) The signal charges thus read out are adjusted in gain by the gain adjust circuit 5 and converted by the A/D conversion circuit 6 into digital signals, which are then subjected to the color signal processing and the luminance signal processing in the signal processing circuit 7, after which the signals are supplied through the vertical interpolation circuit 8 and the horizontal interpolation circuit 9 to the recording unit 10 where they are recorded. At this time, no interpolation processing is performed by the vertical interpolation circuit 8 or horizontal interpolation circuit 9. The recording unit 10 may compress the signals in the JPEG (Joint Photographic Expert Group) format, for example. Because during the static image recording, no television signal can be generated, an image immediately before starting the recording or a single color image is output as the television signal for monitoring purpose. With the above operation, high resolution signals obtained from all the pixels of the image sensing device can be recorded. Although in this embodiment the recording unit is used commonly for the moving video mode and for the static image mode, dedicated recording units may be provided separately for these modes.

(35) As explained above, since there is no limit on the number of vertically arranged pixels in this embodiment, an image sensing device with a large enough pixel number even for still images can be used to photograph highly detailed still images and satisfactory moving images.

(36) Further, the signal mixing and the vertical signal transfer during the vertical blanking period allow the signals from the image sensing device with a large number of pixels to be read out in a manner that conforms to the television system. This in turn can reduce image quality degradation and realize the moving image photographing with an image stabilizing function and the monitoring during still image photographing.

(37) Only the output signals from that horizontal segment which virtually corresponds to the television system's aspect ratio with respect to the vertical segment are extracted and output over the entire horizontal effective scanning period of the television system. This ensures that the output signals obtained conform to the television system's aspect ratio regardless of the extracted vertical segment position.

(38) Further, the image sensing device is driven in such a way as to shift the position of the pixels to be cyclically mixed together every display cycle of the television system in order to output interlaced signals. With this arrangement, the interlaced scanning can be performed even when an image sensing device with a large number of pixels is used.

(39) Further, the output signals produced by the mixing are interpolated so that the gravity centers of the output signals interlaced every display cycle have a phase difference of 180 degrees in the vertical direction. This ensures that the interlaced output signals have no deviation from the ideal 180-degree phase difference during interlacing even when an interlace phase deviation would normally occur, as when odd numbered pixels are mixed together.

(40) In this embodiment, the image sensing device has 1200 vertically arranged pixels, and four pixels are mixed together during the moving video mode and five pixels during the static image mode. Because the area of image-stabilization pixels may or may not be used and set to any desire size, the number of pixels to be cyclically mixed together in each mode needs only to be equal to or less than the integral part of a quotient of the number of vertically arranged pixels divided by the number of television system's effective scanning lines (in the above example, 5 or less). (The number of vertically arranged pixels does not need to be divisible and, in the above example, may be more than 1200).

(41) The number of vertically arranged pixels for static image photographing needs only to be three or more times the number of effective scanning lines on each field of the television system. In this embodiment the vertically adjoining pixels are mixed together to reduce the number of output lines from the image sensing device during the vertical effective scanning period. The number of lines of output signals can also be reduced by a so-called culling operation, by which only one line of signal charges of pixels is read out for every predetermined number of lines.

(42) While in this embodiment the vertical transfer unit of the image sensing device is formed as a CCD that is driven by three phase pulses for each pixel, the image sensing device may have any desired structure as long as it can realize the mixing or culling of pixels that meets the above conditions.

(43) Although this embodiment described the case of NTSC system, the invention can also be applied to other television systems, such as PAL standard, with different numbers of effective scanning lines.

(44) In summary, a variety of constructions essentially equal in the working principle to this embodiment can be realized by the use of an image sensing device that has an arbitrary number of vertically arranged pixels N three or more times the number of effective scanning lines M of each field of the television system and which allows the vertical mixing or culling of that number of pixels which is at least one of integers equal to or less than the integral part of a quotient of N divided by M (a number of lines of output signals corresponds to 1/K the number of vertically arranged pixels N of the image sensing device).

(45) Next, another embodiment of the present invention will be described by referring to FIG. 7 showing the configuration of the embodiment. The configuration shown in FIG. 7 differs from that of FIG. 1 in that it has a view angle change switch 18. In FIG. 7 constitutional elements identical with those shown in FIG. 1 are assigned like reference numbers and explanations on the constitutional elements performing the same operations as those in FIG. 1 are omitted here.

(46) The operations of the moving video mode, the monitoring during the static image mode and the static image recording are similar in normal condition to the operations of the previous embodiment which was explained with reference to the configuration diagram of FIG. 1. An operation performed when during the moving video mode a request to change the angle of view is made by the view angle change switch 18 will be described.

(47) In the normal condition of this embodiment, as explained in the previous embodiment, the mixing of four vertically arranged pixels, the vertical transfer during the vertical blanking period and the horizontal interpolation processing are performed to extract an area of 960 pixels in height and 1280 pixels in width from the entire pixel area to generate television signals. When a view angle change (which means a zooming function without image quality degradations in the vertical direction) is requested by the view angle change switch 18, three vertically arranged pixels are mixed together and the signals from the excess vertically arranged pixels are read out during the vertical blanking periods before and after the vertical effective scanning period.

(48) In this embodiment, signals of 480 pixels (=12002403) or 160 lines of output signals after mixing (=480/3) need to be read out during the vertical blanking period. This allows the signals of 720 vertically arranged pixels to be read out as 240 lines of output signals conforming to the television system. To carry out this reading requires, in the pulse timing diagram of FIG. 3, transferring the signals of three pixel rows during each horizontal blanking period in the vertical effective scanning period and also transferring a total of 480 pixel rows (=160 lines of output signals) during the T3 and T4 periods in the vertical blanking period. The combinations of pixels to be cyclically mixed together are changed from one field to another to achieve the interlaced scanning.

(49) The output signals of the image sensing device 3 are supplied to the gain adjust circuit 5. Because the signal level produced as a result of the 3-pixel mixing is the signal level of the 4-pixel mixing, the again of the gain adjust circuit 5 is increased to 4/3 the gain of the 4-pixel mixing to make the 3- and 4-pixel-mixed input signal levels to the subsequent circuit equal. Then, the signals are processed by the A/D conversion circuit 6 and the signal processing circuit 7 before being supplied to the vertical interpolation circuit 8. The combinations of pixels to be cyclically mixed on the A field and the B field and the vertical positions of the gravity centers of the mixed signals are shown in FIG. 8. As in the static image monitoring of the previous embodiment, the phase difference between the two fields is 180 degrees. Because the sampling phases of the fields are deviated from the ideal 180-degree phase difference, the vertical position of the gravity centers are corrected by the vertical interpolation circuit 8. The amount of phase deviation in this operating state is 60 degrees (= pixel or the line-to-line distance on the same field). To correct the phase deviation evenly on the both fields, the following calculations should be performed.
Sn=(Sn 11/12)+(Sn1 1/12)A field:
Sn=(Sn 11/12)+(Sn+1 1/12)B field:

(50) As described earlier, the interpolation processing may use three or more output lines. Next, the horizontal interpolation circuit 9 horizontally expands the signals from that horizontal segment which corresponds to the 4-to-3 aspect ratio with respect to the 720 vertically arranged pixels (i.e., signals from a horizontal 960-pixel segment (=1600720/1200) in this operating state) so that the expanded signals can be output during the entire horizontal effective scanning period. With the above operation, an area of 720 pixels in height and 960 pixels in width can be extracted from the light receiving surface.

(51) Next, when the view angle change is requested again by the view angle change switch 18, two vertically arranged pixels are mixed together, an area of 480 vertically arranged pixels is read out during the vertical effective scanning period, and the horizontal interpolation circuit 9 expands the signals from the horizontal 640-pixel segment and outputs the expanded signals during the entire horizontal effective scanning period to extract an area of 480 pixels in height and 640 pixels in width. (During the two-pixel mixing, because the interlace phase deviation does not occur, the gravity center position correction by the vertical interpolation circuit 8 is not performed.) If a further view angle change is requested by the view angle change switch 18, the operation is restored to a normal state where four vertically arranged pixels are mixed together.

(52) As a result of the above operation, the area extracted from the light receiving surface of the image sensing device can be changed to three different areas: (A) 960 pixels high by 1280 pixels wide, (B) 720 pixels high by 960 pixels wide and (C) 4880 pixels high by 640 pixels wide. That is, the angle of view can be changed to three different angles. With the area A produced by the 4-pixel mixing taken as a reference, the area B produced by the 3-pixel mixing can provide an image enlarged by 1.33 times and the area C produced by the 2-pixel mixing can provide an image enlarged by two times. It should be noted here that because the three different areas are chosen by changing the number of pixel to be cyclically mixed together in order to make the number of the effective output lines from the imaging device agree with the number of the effective scanning lines of the television system, the angle of view can be changed while maintaining a good image with no image quality degradation in the vertical direction, when compared with an ordinary so-called digital zoom which generates, effective scanning lines of signals by interpolating a small number of output lines. During the monitoring of a static image, too, it is possible to perform the similar operation of changing the angle of view by changing the number of pixel rows to be cyclically mixed together.

(53) As described above, in addition to the advantages provided by the previous embodiment, this embodiment can also realize the view angle change with little image quality degradation even for still images by using an image sensing device with a large number of pixels and changing the number of pixels to be cyclically mixed together.

(54) Further, because changes in signal level caused when the number of pixels to be cyclically mixed is changed are absorbed by the gain adjust means, the input signal level to the subsequent signal processing means can be kept constant.

(55) While in this embodiment, the view angle change is performed by the view angle change switch 18, the angle of view may be changed continuously by a zoom switch. In this case, when the magnification factor does not reach the value that is obtained by changing the pixel mixing, the digital zoom performs the ordinary interpolation processing. In this embodiment, when the magnification factor is 1 or more and less than 1.33, the 4-pixel mixing is performed; for the factor of 1.33 or more and less than 2, the 3-pixel mixing is done; and for the factor of 2 or higher, the 2-pixel mixing is carried out. The mixing operation may be interlocked with an optical zooming mechanism.

(56) Regardless of the number of pixels in the image sensing device, the structure of the image sensing device or the television system employed, this embodiment, as in the previous embodiment, may also use an image sensing device that has an arbitrary number of vertically arranged pixels N three or more times the number of effective scanning lines M of each field and which allows the vertical mixing or culling of those numbers of pixels which are at least two of integers equal to or less than the integral part of a quotient of N divided by M. The use of this image sensing device can form a variety of constructions essentially equal in the working principle to this embodiment.

(57) Next, a further embodiment of the present invention will be described. The overall configuration of this embodiment is similar to that of FIG. 1, except that the inner structure of the image sensing device 3 is different. The configuration of the image sensing device in this embodiment is shown in FIG. 10. In FIG. 10, denoted 30 are pixels formed of photodiodes, which are arranged horizontally and vertically in a grid pattern. On these grid-arrayed pixels three types of color filters that pass yellow (Ye), green (G) and cyan (Cy), respectively, are arranged in so-called vertical stripes.

(58) In this embodiment, the image sensing device has an array of pixels measuring 864 pixels vertically and 1152 pixels horizontally. A vertical transfer unit 32 is a CCD which is driven by six phase pulses V1, V2, V3, V4, V5, V6. This CCD has a two-gate structure in which each pixel corresponds to two phase pulses and six gates corresponding to the six phase pulses are repeated for every three pixels. Transfer gates 31 for transferring the signal charge of each pixel to the vertical transfer unit 32 are commonly connected to respective gates of the vertical transfer unit 32 corresponding to the pulses V1, V3, V5. An operation to transfer the signal charge from each pixel to the vertical transfer unit 32 in response to peak values of pulses applied to the commonly connected gates and an operation to transfer the charge vertically are performed separately.

(59) A horizontal transfer unit 33 horizontally transfers the charges supplied from the vertical transfer units 32 and outputs them successively through an output amplifier 34 from the output terminal. This image sensing device, unlike the one in the previous embodiment, cannot vertically transfer all pixels independently of each other, but can mix together the signal charges of three vertically adjoining pixels inside the vertical transfer unit 32 before transferring them.

(60) First, the operation performed in this embodiment during the moving video mode will be explained. In the image sensing device of this embodiment the effective number of vertically arranged pixels is 864. If three pixels are vertically mixed, the signals of the 720 (=2403) of the 864 vertically arranged pixels can be used as the effective signals and the remaining 144 (=864720) pixels can be used as the image-unstability correction pixel area.

(61) FIG. 11 shows the timings of vertical drive pulses for the image sensing device of FIG. 10 during this operation mode, with V1, V2, V3, V4, V5, V6 representing the six phase drive pulses applied to the respective gates of the CCD or vertical transfer unit 32. In FIG. 11, during a period T1 included in the vertical blanking period, the drive pulses V1, V3 and V5 are held high to cause the signal charge of each pixel to be transferred to under the V1, V3 and V5 gates of the vertical CCD. Then, the V2 and V4 pulses are changed from low level to middle level to mix the charges of the adjoining three pixels. After the mixing, the V5 pulse is changed from middle level to low level to hold the mixed signal charges under the V1, V2, V3, V4 gates.

(62) Next, a series of operations performed during a period T2 (changing the drive pulses from middle level to low level or from low level to middle level in the order of V1, V2, V3, V4, V5 and V6) causes one mixed output line (3 pixel rows) to be transferred and held again under the V1, v2, V3, V4 gate. By repeating this series of operations, a desired number of output lines of mixed signal charges can be transferred.

(63) In FIG. 11, during a period T3 included in the vertical blanking period before the vertical effective scanning period and during a period T4 included in the vertical blanking period after the vertical effective scanning period, the transfer operation for one output line is repeated a total of 144 times to transfer 144 output lines of signal charges not used for image forming to the horizontal transfer unit 33 at high speed during the vertical blanking periods. During subsequent periods T5 and T6 in the vertical blanking periods, the horizontal transfer unit 33 is driven for predetermined periods to output the signal charges transferred to the horizontal transfer unit 33 from the output terminal.

(64) Next, in the vertical effective scanning period of FIG. 11, the one-output-signal-line transfer operation is performed during each horizontal blanking period. Then, during the horizontal effective scanning period, the horizontal transfer unit 33 is driven to read out the signal charges from the horizontal transfer unit 33. With this operation the signal charges of three pixels mixed together can be read out in a way conforming to the television system. As shown in FIG. 11, the signals for the A field are mixed by changing the V2 and V4 pulses to middle level after transferring the signals from the pixels to the vertical transfer unit 32. The signals for the B field, on the other hand, are mixed by changing the V2 and V6 pulses to middle level. With this mixing method, the combinations of pixels to be cyclically mixed together can be changed from one field to another, thereby realizing the interlaced scanning. The output signals from the image sensing device are processed in the similar manner to that of the previous embodiment. A vertical interpolation circuit 8 performs the gravity center correction, as in the 3-pixel mixing in the previous embodiment, and a horizontal interpolation circuit 9 performs interpolation processing to match the aspect ratio with that of the television system.

(65) Next, the operation during the monitoring in the static image mode will be explained. It is assumed that the still image photographing is done by using all effective pixels of the image sensing device, as in the previous embodiment. The image sensing device of this embodiment has 864 vertically arranged pixels and, when 3-pixel mixing is done as in the moving video taking, the number of output lines is 288 (=864/3), which means that these signal lines cannot be read out in a manner conforming to the television system. Hence, during the monitoring in the static image mode, vertical 6-pixel mixing is performed. The 6-pixel mixing can be achieved by transferring to the horizontal transfer unit 33 in each horizontal blanking period two output lines of signal charges each of which line has been generated by vertically mixing three pixels within the vertical transfer unit 32. The 6-pixel mixing can reduce the number of output lines from the image sensing device down to 144 (=864/6) lines. The output signals of the image sensing device that were reduced to 144 output lines are interpolated by the vertical interpolation circuit 8 to transform the 144 output lines of signals into 240 lines of signals, which conform to the television system. To generate 240 lines of signals from the 144 lines requires interpolation processing that generates five lines from three lines (144/240=).

(66) FIG. 12 shows how the interpolation is performed using two adjoining output lines. Let three output lines of the image sensing device be n, n+1 and n+2. The five output lines of signals can be generated from the following calculations.
n=n
n+1=n/2+(n+1)/2
n+2=n+1
n+3=(n+1)/2+(n+2)/2
n+4=n+2

(67) Three or more output lines of signals may be used for interpolation processing. With the above operation, television signals can be generated by using signals of all pixels of the image sensing device also during the monitoring in the static image mode.

(68) Next, the operation performed when the recording is requested by the record button 15 will be explained. In the recording process, the mixing processing is not performed and signals of all pixels need to be read independently of each other in order to obtain high-resolution signals. To realize this, the aperture 2 is first closed and then, during the period T2 in the pulse timing diagram of FIG. 11, only the V1 pulse is raised to high level to transfer the signal charge of only the pixel adjacent to the V1 gate to the vertical transfer unit 32. Then, the vertical transfer unit 32 and the horizontal transfer unit 33 are successively driven to read out the signal charges. Similarly, the V3 pulse is raised to high level to read the signal charge of the pixel adjacent to the V3 gate, followed by raising the V5 pulse to high level to read the signal charge of the pixel adjacent to the V5 gate. With the above processing, the signal charges of all pixels can be read out independently in three successive operations. The signal charges thus read out are recorded in the recording unit 10. At this time, they are rearranged properly to reconstruct the pixel arrangement on the light receiving surface of the image sensing device.

(69) As described above, this embodiment offers the following advantages. If the number of vertically arranged pixels is not an integral multiple of the number of scanning lines of the television system, the signals conforming to the television system can be generated from the whole area of effective pixels by performing the pixel mixing and the vertical interpolation.

(70) In this embodiment, as in the previous embodiment, regardless of the number of pixels in the image sensing device, the structure of the image sensing device or the television system employed, a variety of constructions essentially equal in the working principle to this embodiment can be realized by using an image sensing device that has an arbitrary number of vertically arranged pixels N three or more times the number of effective scanning lines M of each field and which allows the vertical mixing or culling of that number of pixels which is greater by at least one than the integral part of a quotient of N divided by M.

(71) Next, a further embodiment of the present invention will be explained. This embodiment differs from the previous embodiments in that the image sensing device have different arrangements of color filters. FIGS. 13A and 13B show arrangements of color filters in this embodiment. These color filters in both examples are arranged in vertical stripes and, regardless of the number of pixels to be vertically mixed or culled, the R, G, B primary color signals can be generated from one line of output signals. FIG. 13A shows a color filter arrangement on the image sensing device in which white filters (W=passing all colors) are used instead of the green (G) filters used in the previous embodiment. The R, G, B signals can be obtained by the following calculations.
R=WCy
G=Ye+CyW
B=WYe

(72) When this color filter arrangement is used, a higher sensitivity can be obtained than the color filter arrangement of the previous embodiment. FIG. 13B show a color filter arrangement that uses, in stead of complementary colors, color filters that pass the primary colors R, G, B. This color filter arrangement can directly produce the primary color signals, R, G, B and can provide a camera with good color purity and good color S/N.

(73) With the above color filter arrangements, it is possible to produce color signals corresponding to the three kinds of color filters from each line of output signals at all times no matter how many pixels are vertically mixed or culled. Therefore, color signals conforming to the television system can be generated easily.