METHOD AND APPARATUS FOR THE CORRECTION OF GEOMETRIC AND DEPTH OF FIELD PROBLEMS DURING IMAGE TAKING

Abstract

An imaging equipment includes an image display; an image sensor; a motor; and an inclinometer, where the image display concurrently displays the image being captured by the image sensor, the motor automatically rotates the image sensor around the center of the motor with respect to at least one of the image sensor's axes, to make the sensor level on the basis of data acquired from a inclinometer, the motor allows the user to control the motor to shift the image sensor along at least one of the image sensor's axes, and the motor allows the user to control the motor to rotate the image sensor with respect to at least one of the image sensor's axes to have the motor tilt and swing the image sensor in such a way as to move two or more points, selected by the user, to lie on a same focus plane.

Claims

1. A method for the perspective control of images from an imaging equipment having an inclinometer, an image sensor that captures an image, an image display that concurrently displays the image captured by the image sensor, and a motor that moves the image sensor, three Cartesian axes being defined by a y axis, an x axis, and a z axis that intersect at a center of the image sensor, the method comprising the steps of: having the inclinometer measure and report an angle theta[z] of the image sensor's z axis with respect to a level orientation of the image sensor and concurrently having the image display display the image captured by the image sensor; based on the report of the angle theta[z], having the motor make a correction by rotating the image sensor around the center of the motor with respect to the image sensor's z axis, by the motor rotating the image sensor by an angle of minus theta[z] to thereby reduce the angle theta[z] of the image sensor's z axis to a zero value, said connection by said motor rotation leveling the image thereby avoiding a keystoning effect in the image; and controlling the motor to move the image sensor linearly along at least one of the image sensor's y axis and x axis, the linear movement along the y axis being up to a first specified threshold along the y axis and the linear movement along the x axis being up to a second specified threshold along the x axis, said linear movement along at least one of the image sensor's y axis and x axis reducing or avoiding a need to tilt the imaging equipment in order to capture a portion of interest in the scene, wherein said first and second specified thresholds can be set independently to respective first and second default values, thus allowing the perspective control without user intervention, and wherein said first and second default values can be overridden by a user to thereby override the first and second specified thresholds.

2. A method for the automatic leveling of images from an imaging equipment having an image display, an image sensor that captures an image, and a motor that rotates the image sensor, three Cartesian axes being defined by a y axis, an x axis, and a z axis that intersect at a center of the image sensor, and in-body image stabilization that avoids camera shake, said stabilization provided by said motor or by an additional motor, the method comprising the steps of: having an inclinometer report an angle theta of the image sensor's y axis with respect to a level orientation of the image sensor and concurrently having the image display the image captured by the image sensor; and based on the report of the angle theta, having the motor rotate the image sensor around the center of the motor with respect to the image sensor's y axis, by the motor rotating the image sensor by an angle of minus theta, and using the in-body image stabilization to provide finer corrections for camera shake.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIGS. 1-4 show an example embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] FIG. 1 shows an example of imaging equipment, for example, a camera (1) with a lens (2).

[0024] The image sensor (3) captures the image and said capture is concurrently shown on the image display (4), and/or in an electronic view finder (5).

[0025] The camera includes an electronic level or inclinometer and computing hardware, which may be placed anywhere in the camera, together or separately.

[0026] For the purpose of explanation, three Cartesian axes are defined. The origin of these axes is the center of the image sensor (3).

[0027] The z axis (6) connects the origin with the center of the lens, the y axis (7) is parallel to the left (or right) side of the sensor, and the x axis (8) is parallel to the top (or bottom) side of the sensor.

[0028] The camera also has a processor-controlled motor (9) which can rotate the sensor with respect to the y and z axis and optionally the x axis, and, in some embodiments, shift the sensor with respect to the x or y axis, or both.

[0029] FIG. 2 shows a camera tilted with respect to the y axis (7), and level with respect to the z axis, and so producing a tilted horizon (11). The dotted axis (12) indicates the level y axis. The angle θy (10) is with respect to the level y axis (7), and is therefore a positive value in the figure.

[0030] FIG. 3 shows a camera tilted by an angle θz (14) with respect to the level z axis (13). As discussed below, if θz is not null, the camera is not level, i.e., pointing straight but it points either upwards or downwards. The scene is vertically tilted, making a square look like a trapezoid, with the shorter side either up or down depending on whether the camera points upwards or downwards (vertical keystoning). In FIG. 3, the camera is tilted by a negative angle θz (14) with respect to the level z axis (13). In the invention, to correct the tilt, the motor rotates the image sensor with respect to the z axis by +θz degrees as shown in FIG. 4 so that the image sensor is at a null angle θz (14) with respect to the level z axis (13).

[0031] The description of the preferred embodiments refers to the camera in a landscape orientation, since the description for the portrait orientation is the same with the y and x axes interchanged. The preferred embodiment can be applied to any imaging equipment.

[0032] The electronic level reports the angle θy and θz, which reports the deviation of the y and z axis, respectively, from level. These angles can be positive, negative or null (zero).

[0033] I. Camera Rotation with Respect to the y Axis (Tilted Horizon):

[0034] if θy is not null, the camera is not level and the horizon is tilted. To correct the tilt, the motor rotates the images sensor with respect to the y axis by −θy degrees. Such a correction is done continuously on the basis of the output of the electronic level.

[0035] By using in-body image stabilization some embodiments provides finer corrections for camera shake.

[0036] II. Camera Rotation with Respect to the z Axis (Vertical Keystoning):

[0037] if θz is not null, the camera is not level, i.e. pointing straight but it points either upwards or downwards. The scene is vertically tilted, making a square look like a trapezoid, with the shorter side either up or down depending on whether the camera points upwards or downwards (vertical keystoning).

[0038] To correct the tilt, the motor rotates the image sensor with respect to the z axis by +θz degrees. Compare FIGS. 3-4.

[0039] The two corrections are done continuously on the basis of the output of the electronic level. In a preferred embodiment, corrections can be excluded in order to allow for intentional tilts. In another embodiment, corrections can be automatically excluded when the camera is not capturing stills or video, as to save power. In yet another embodiment, corrections can be applied at the shutter button half press.

[0040] III. Sensor Shift for Perspective Control

[0041] Especially when photographing architecture and in particular high buildings, the camera must usually be tilted with respect to the z axis, in order to capture the entire building in the image. This, however, produces a vertical keystoning. On the other hand, if the camera is kept level, only the bottom part of the building will usually be captured.

[0042] In order to overcome these problems, the sensor can be shifted along the y axis in such a way that the bottom of the building moves towards the bottom of the image, and a larger portion of the building is captured by the sensor.

[0043] A default shift for landscape orientation and for portrait orientation may be provided.

[0044] The sensor can also be continuously shifted up and down (on the y axis), or left and right (on the x axis) by the user, until a specified threshold that usually depends on the lens circle of coverage. In a preferred embodiment, shifts are controlled through one or two controls, either physical controls or virtual ones on the display. The image display itself gives a concurrent feedback on the effect on perspective control of the current shift. Because shifts are primarily used for perspective control, a preferred embodiment forces the sensor to be level.

[0045] IV. Sensor Swing for Eliminating Horizontal Keystoning.

[0046] Horizontal keystoning (a square looks like a trapezoid with the smaller side either on right or on the left) occurs when the sensor is not perfectly parallel to the scene. This problem can be corrected by rotating the sensor with respect to the x axis until the horizontal keystoning disappears. Since the inclinometer by itself is unable to determine the angle of deviation from parallel, in one embodiment the rotation is manually applied through physical or virtual controls.

[0047] In another embodiment, in which lenses report the focusing distance from the sensor plane, the user selects two (or more) points on the image that should be at the same distance from the sensor plane, and the camera automatically computes the angle of rotation around the x axis.

[0048] The inclinometer may measure and report an angle theta[z] of the image sensor's z axis with respect to a level orientation of the image sensor. The image display concurrently displays the image captured by the image sensor. Then, based on the report of the angle theta[z], the motor makes a correction by rotating the image sensor around the center of the motor with respect to the image sensor's z axis. The motor rotating the image sensor by an angle of minus theta[z] therebys reduce the angle theta[z] of the image sensor's z axis to a zero value, and the connection by the motor rotation leveling the image thereby avoids a keystoning effect in the image.

[0049] The method includes controlling the motor to move the image sensor linearly along at least one of the image sensor's y axis and x axis, the linear movement along the y axis being up to a first specified threshold along the y axis and the linear movement along the x axis being up to a second specified threshold along the x axis, the linear movement along at least one of the image sensor's y axis and x axis reducing or avoiding a need to tilt the imaging equipment in order to capture a portion of interest in the scene. In this regard, these first and second specified thresholds can be set independently to respective first and second default values, thus allowing the perspective control without user intervention, with the user being able to override the default values and thereby override the first and second specified thresholds.

[0050] V. Combination of Movements for Varying the Focus Plane

[0051] When no tilt (rotation with respect to the z axis) or swing (rotation with respect to the x axis) movements are applied to the sensor, the focus plane (i.e. the plane on which all points are in focus) is parallel to the sensor.

[0052] Tilt and swing movements, however, can be used to have a focus plane that is not parallel to the sensor, and in which objects whose distance from the imaging equipment is different lie in fact on the same focus plane, and are therefore in focus at the same time. The focus plane, as a function of tilt and swing angles, varies according to the Scheimpflug principle.

[0053] An embodiment lets the user vary the tilt and swing angles manually, until the focus plane is satisfactorily placed.

[0054] In a preferred embodiment, the user selects two or more points that should lie on the desired focus plane, and the imaging equipment computes the correct (or best possible) tilt and swing angles according to the Scheimpflug principle, and automatically applies them.

[0055] In all situations in which points must be selected on the image, a variant allows the selection of points through a touch-sensitive display, and another variant allows this operation performed on larger displays, working in tethering with the imaging equipment.

[0056] Although only a few embodiments have been disclosed in detail above, other embodiments are possible and the inventors intend these to be encompassed within this specification.

[0057] The specification describes specific examples to accomplish a more general goal that may be accomplished in another way. This disclosure is intended to be exemplary, and the claims are intended to cover any modification or alternative which might be predictable to a person having ordinary skill in the art. For example, the methods can be applied to other search and data management applications.

[0058] Those of skill would further appreciate that the various illustrative logical blocks, modules, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, and steps have been described above generally in terms of their functionality.

[0059] Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the exemplary embodiments of the invention.

[0060] Also, the inventor intends that only those claims which use the words “means for” are intended to be interpreted under 35 USC 112, sixth paragraph. Moreover, no limitations from the specification are intended to be read into any claims, unless those limitations are expressly included in the claims.

[0061] Secondary storage defines generic storage devices (including main memory) that maintain information that the computer can access, and may use any type of technology for this purpose.

[0062] The previous description of the disclosed exemplary embodiments is provided to enable any person skilled in the art to make or use the present invention.

[0063] Various modifications to these exemplary embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.