AUTOFOCUS METHOD AND ASSOCIATED OPTICAL IMAGING SYSTEM

Abstract

To improve the accuracy and the speed of an autofocus method, using which a present best focal plane (13) may be found in an automated manner, which enables a best possible image quality for an object (3), which is located at a specific working distance (11) to an optical imaging system (1), it is provided that at least one parameter used during a z-scan (17) be adapted in an automated manner as a function of a presently set optical zoom level and/or a current estimated value of the working distance (11). During the z-scan (17), a present location of a focal plane (12) of the optical imaging system (1) is displaced within a scanning range (14) along an optical z-axis (8) of the imaging system (1), wherein the individual focal planes (12) are each evaluated to identify the best focal plane (13) among them.

Claims

1. An autofocus method for automated finding of a present best focal plane (13), the method comprising: by tuning a focus lens (6) of an optical imaging system (1), displacing a location of a focal plane (12) of the optical imaging system (1) in a z-scan (17) within a scanning range (14) along an optical z-axis (8), and automatically adapting at least one parameter of the z-scan (17) as a function of at least one of a) a currently set optical zoom level of the imaging system (1) or b) a current estimated value for a working distance (11) between the imaging system (1) and an object (3) visualized using the imaging system (1), to thus accelerate finding the best focal plane (13).

2. The autofocus method as claimed in claim 1, wherein the at least one parameter comprises at least one of a length (16) of a scanning range (14), a number of the focal planes (12) optically scanned using the z-scan (17), a spatial scanning frequency of the z-scan (17), an adjustment speed of the focus lens (6), or an optical zoom level of the imaging system (1) used during the z-scan (17).

3. The autofocus method as claimed in claim 2, wherein the spatial scanning frequency of the z-scan (17) is increased when the length of the scanning range (14) of the z-scan (17) is shortened.

4. The autofocus method as claimed in claim 1, wherein by adapting the at least one parameter at least one of a) a depth of field is actively adapted by changing a zoom level of the imaging system (1), or b) a present change of the depth of field is compensated for, by adapting at least one of a spatial scanning frequency or a length (16) of the z-scan (17).

5. The autofocus method as claimed in claim 1, wherein the estimated value for the working distance (11) is ascertained using at least one of a) an additional sensor, b) an additional item of location information with respect to a present spatial location of the imaging system (1), or c) an additional camera.

6. The autofocus method as claimed in claim 2, wherein the at least one parameter comprises the optical zoom level, and upon increase of the optical zoom level at least one of a) a length (16) of the scanning range (14) is shrunken or b) a number of scanned focal planes (12) within the scanning range (14) is reduced.

7. The autofocus method as claimed in claim 2, wherein upon decrease of the estimated value for the working distance (11), at least one of a) the length (16) of the scanning range (14) is shrunken, or the number of scanned focal planes (12) within the scanning range (14) is reduced.

8. The autofocus method as claimed in claim 2, wherein the scanning range (14) is traversed step-by-step in that the location of the focal plane (12) is displaced step-by-step in a step width (15) and wherein the step width (15) is adapted as a function of the presently set optical zoom level and/or the estimated value for the working distance (11), preferably wherein the step width (15) is reduced upon increase of the zoom level and accompanying increasing image magnification and/or upon decrease of the estimated value for the working distance (11).

9. The autofocus method as claimed in claim 1, wherein the scanning range (14) is continuously traversed in that the location of the focal plane (12) is continuously displaced within the scanning range (14), by at least one of a) the focus lens (6) moving at constant movement speed, b) the focus lens (6) moving continuously, or c) the location of the focal plane (12) being displaced continuously along the z-axis, and a mean scanning speed, at which the location of the focal plane (12) is displaced along the z-axis (8), is adapted as a function of at least one of a presently set optical zoom level or an estimated value for the working distance (11), so that the scanning speed is reduced at least one of a) upon increase of the zoom level and accompanying increasing image magnification, or b) upon decrease of the estimated value for the working distance (11).

10. The autofocus method as claimed in claim 9, further comprising operating an image sensor (9), via which the different focal planes (12) are acquired or scanned as individual images (19), in a rolling shutter mode, so that different image areas (20a, 20b, 20c) of these individual images correspond to different z-positions (z1, z2, z3) along the optical z-axis (8), since the respective individual image (19) is recorded while the location of the focal plane (12) changes, and adapting an evaluation area (21) within the respective individual image (19), which is evaluated to assess the respective focal plane (12), as a function of at least one of a) a mean scanning speed or b) a number of focal planes (12) to be acquired.

11. The autofocus method as claimed in claim 1, wherein to find the best focal plane (13), at least two z-scans (27, 28) are carried out in succession within the scanning range (14), and in each case the location of a current focal plane (12) is displaced within the scanning range (14), wherein the two z-scans (27, 28) differ by at least one of a respective length (18), a step width (15) used, a scanning speed used, or a respective location within the scanning range (14), and the two z-scans (27, 28) at least partially overlap.

12. The autofocus method as claimed in claim 11, wherein the at least two z-scans (27, 28) comprise a coarse scan (27) and a fine scan (28) following with respect to time, the coarse scan (27) takes place at a lower zoom level at lower image magnification than the fine scan (28), and at least one of a) before carrying out the coarse scan (27), initially setting a minimal zoom level is set, or b) executing the coarse scan (27) over a maximum possible scanning range (14).

13. The autofocus method as claimed in claim 11, wherein the at least two z-scans (17 a, 28) comprise a coarse scan (27) and a fine scan (28) following with respect to time, and at lest one of: a local z-resolution of the fine scan (28) is higher than a local z-resolution of the coarse scan (27), a length (18a) of the coarse scan (27) is greater than a length (18b) of the fine scan (28), a step width (15a) of the coarse scan (27) is greater than a step width (15b) of the fine scan (28), or a scanning speed of the coarse scan (27) is higher than a scanning speed of the fine scan (28).

14. An optical imaging system (1) for visualizing an object (3) during a medical intervention, the optical imaging system comprising: a zoom optics unit (4), which is adjustable by a zoom actuator (5), to adapt an optical zoom level, a focus lens (6), which is tunable by a focus actuator (7), to adapt a location of a focal plane (12) along an optical z-axis (8), an image sensor (9) for recording image data, and a controller (10) for activating the focus actuator (7), wherein the controller (10) is configured to implement an autofocus and to activate at least one of the focus actuator (7) or the zoom actuator (5) as a function of at least one of a) a zoom level presently set with the zoom actuator (5) or an estimated value for a present working distance (11) between the imaging system (1) and the object (3).

15. The optical imaging system (1) as claimed in claim 14, wherein the controller (10) is configured to activate the at least one of the focus actuator (7) or the zoom actuator (5) in order to carry out an autofocus method to determine a present best focal plane (13) including the steps of by tuning the focus lens (6), displacing a location of the focal plane (12) in a z-scan (17) within a scanning range (14) along the optical z-axis (8), and automatically adapting at least one parameter of the z-scan (17) as a function of at least one of a) a currently set optical zoom level of the imaging system (1) or b) a current estimated value for a working distance (11) between the imaging system (1) and an object (3) visualized using the imaging system (1), to thus accelerate finding the best focal plane (13).

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The invention will be described in more detail on the basis of exemplary embodiments, but is not restricted to these exemplary embodiments. Further designs of the invention can be obtained from the following description of a preferred exemplary embodiment in conjunction with the general description, the claims, and the drawings.

[0058] In the following description of various preferred embodiments of the invention, elements corresponding in their function receive corresponding reference numerals even with differing design or formation.

[0059] In the figures:

[0060] FIG. 1 shows a schematic sketch of a typical application situation of an imaging system according to the invention, which is used to observe an object,

[0061] FIG. 2 shows individual components of the imaging system from FIG. 1 as well as actuators and a controller,

[0062] FIG. 3 shows a greatly simplified illustration of the curve of an image contrast along the optical z-axis,

[0063] FIG. 4 shows an illustration of two z-scans carried out in different situations, to find the location of a best focal plane in each case,

[0064] FIG. 5 shows a further illustration of an autofocus method according to the invention, which comprises a coarse scan and two following fine scans,

[0065] FIG. 6 shows an individual image recorded using an image sensor of the imaging system shown in FIG. 1, and

[0066] FIG. 7 shows the individual steps when running an autofocus method according to the invention, which comprises a coarse scan and a fine scan.

DETAILED DESCRIPTION

[0067] FIG. 1 shows an optical imaging system 1 according to the invention, which is configured to visualize an object 3 during a medical intervention. For this purpose, the optical imaging system 1 includes a zoom optics unit 4, which is adjustable by means of a zoom actuator 5 in order to adapt an optical zoom level, as illustrated in FIG. 2, which shows the internal components of the optical imaging system 1 in detail. In addition, the imaging system 1 includes a focus lens 6, which is tunable by means of a focus actuator 7 in order to displace the present location of a focal plane 12 along the optical z-axis 8, as illustrated in FIGS. 1 and 2. A controller 10 is provided for activating the focus actuator 7 and the zoom actuator 5.

[0068] The dashed vertical lines in FIG. 1 indicate the z-positions (for example z.sub.1 to z.sub.8), in which the focal plane 12 is positioned during a z-scan 17. I.e., the respective dashed vertical line in FIG. 1 indicates the position of that plane which—as a result of the tuning of the focus lens 6—is presently sharply imaged on the image sensor 9 and is thus optically scanned.

[0069] As can be seen in FIG. 2, the zoom optics unit 4 includes two zoom lenses 24, each designed as achromatic lenses, wherein the front zoom lens can be displaced with the aid of the zoom actuator 5 along the optical axis 8, in order to thus zoom into the object 3 more or less strongly. The optical components 4 and 6 are combined with a front lens 23 in a compact objective 26. The front lens 23 is designed as a scattering lens and thus enables a comparatively large working distance 11 between the front lens 23 and the object 3. Together with the image sensor 9 illustrated in FIG. 2, these components 4, 6, and 23 form a video camera 2.

[0070] As shown in FIG. 1, by tuning the focus lens 6 (which can be designed either as a lens displaceable along the z-axis 8 and/or as a tunable lens), the present focal plane 12 can be displaced, for example, step-by-step in a step width 15 of Δz.sub.1 in positive scanning direction 29 (i.e., in the positive z-direction 8—cf. FIG. 1) along the optical z-axis 8.

[0071] If the focus lens 6 shown in FIG. 2 is designed, for example, as a displaceable lens and if it is displaced at continuous movement speed along the z-axis 8, the present focal plane 12 will thus also accordingly be continuously displaced along the z-axis 8, wherein this movement can depend nonlinearly on a present z-position of the focus lens 6. By reading the image sensor 9 at regular time intervals, upon the continuous displacement of the present focal plane 12, an entire set of different (present) focal planes 12 can thus be optically scanned in each case, as illustrated by the dashed vertical lines in FIG. 1. By selecting the tuning speed of the focus lens 6 and/or the readout frequency of the image sensor 9, therefore not only the respective length 16 of the scanning range 14 traveled in the respective z-scan 17 can be adapted, but also the effective step width 15, thus the distance between two adjacent focal planes 12 which are scanned in the scope of such a z-scan 17 with the aid of the image sensor 9.

[0072] FIG. 3 shows a greatly simplified possible curve of an image sharpness (vertical axis) as a function of the z-coordinate. As can be seen, it can occur in real application situations that in addition to a global maximum 22 of the image sharpness, further local maxima 21 also exist. The best focal plane 13 illustrated in FIG. 1 corresponds here to the global maximum 22 of the image sharpness in FIG. 3. That is to say, when the focus lens 6 is tuned so that the present focal plane 12, as illustrated in FIG. 1, is positioned at the present working distance 11 of the object 3 from the front lens 23, the object 3 is thus imaged in optimum image sharpness on the image sensor 9. In other words, the best focal plane 13 therefore corresponds just to that focal plane 12 scanned with the aid of the imaging system 1 which is closest to the plane in which the object 3 is located. As FIG. 4 shows, the best focal plane 13, which is determined using an autofocus method according to the invention, can still deviate somewhat from the actual best focal plane 13*, because the latter cannot be determined with arbitrarily high z-resolution.

[0073] As explained hereinafter, the controller 10 of the imaging system 1 is configured to implement an autofocus method according to the invention. For this purpose, the controller 10 activates the focus actuator 7 and/or the zoom actuator 5 accordingly, and does so depending on which zoom level is presently set with the aid of the zoom actuator 5 (for example in reaction to a preceding input of the user of the imaging system 1) and/or depending on a present estimated value for the (actual) working distance 11, which presently exists between the imaging system 1 and the object 3.

[0074] It is to be considered in this case that in a typical application situation, thus, for example, if the imaging system 1 is designed in the form of an exoscope and is positioned at the end of a robot arm at different working distances 11 to the object 3 to be observed, both the zoom level (depending on the desire of the user) and also the working distance 11 can change from situation to situation. In order to accelerate finding the best focal plane 13 in such situations, the controller 10 selects both the length 16 of the scanning range 14, within which a specific number of focal planes 12 is scanned, and therefore also the number of these focal planes 12 independently. Furthermore, the controller 10, depending on the situation, also adapts the optical zoom level of the imaging system 1 which is used during the described z-scan 17, thus when the imaging system 1 scans the different focal planes 12.

[0075] For this purpose, the controller 10 checks which zoom level is currently set and moreover queries a current estimated value for the working distance 11 between the imaging system 1 and the object 3. To ascertain the estimated value for the working distance 11, the controller 10 can evaluate, for example, an additional item of location information, which has a reference to the present spatial location of the imaging system 1, for example a current position of a robot arm, on which the imaging system 1 is fastened and using which the imaging system 1 is moved in space relative to the object 3. Alternatively or additionally thereto, however, the imaging system 1 can also have an additional sensor, for example a contactless distance sensor, or, for example, an additional camera. This is because such devices can also be used to ascertain a current estimated value for the working distance 11 in each case.

[0076] For example, FIG. 4 shows that when the object 3 is located at a comparatively short working distance z.sub.1, the resulting depth of field (DOF) is small, while the depth of field enlarges when the working distance enlarges, for example to the value z.sub.2. The controller accordingly shrinks the length 16 of the scanning range 14 and thus also reduces the number of scanned focal planes 12 within the scanning range 14 when the estimated value for the working distance 11 decreases. In this case, for example, the step width 15 between the individual scanned focal planes 12 can be maintained. An improvement of the z-resolution when finding the best focal plane 13 is in particular achieved, however, as illustrated in FIG. 4, when the step width 15 is reduced upon decrease of the estimated value for the working distance 11 (either alternatively or additionally to the adaptation of the length 16 of the scanning range 14). This suggests itself in particular if the scanning range 14 is traversed step by step in that the location of the respective focal plane 12 is displaced step-by-step in a step width 15 (for example Δz.sub.1 as illustrated on the left in FIG. 4).

[0077] In addition, it is reasonable to consider the presently set optical zoom level of the imaging system 1 to accelerate the finding of the best focal plane 13. If the zoom level has increased, for example, i.e., if a large zoom focal length is presently used (telephoto), so that only a small image detail of the object 3 is imaged on the image sensor 9 and accordingly a high magnification of the object 3 is achieved, it thus makes sense to shrink the length 16 of the scanning range 14 or to reduce the number of scanned focal planes 12 within the scanning range 14. This is because the depth of field will decrease at high optical zoom level, similarly as with a comparatively short working distance 11, so that the best focal plane 13 has to be found in a smaller z-range along the optical z-axis 8.

[0078] As already explained, the scanning range 14, which is illustrated in FIG. 1, can also be continuously traversed, in that the location of the focal plane 12 is continuously displaced within the scanning range 14, while the image sensor 9 is read out. For example, for this purpose the focus lens 6 can be moved at constant movement speed. In contrast, if the focus lens 6 is designed as a tunable liquid lens, the location of the focal plane 12 can thus be displaced continuously along the z-axis 8 by continuously tuning the focus lens 6. In such cases, the mean scanning speed at which the location of the focal plane 12 is displaced along the z-axis 8 can be adapted as a function of the presently set optical zoom level and/or an estimated value for the (present) working distance 11. The controller 10 can thus reduce the scanning speed upon increase of the zoom level and accompanying increased image magnification and/or upon decrease of the estimated value for the working distance 11. This is because if the image sensor 9 is read out at a constant (for example maximum image) frequency during the continuously executed z-scan 17, the z-resolution thus increases with decreasing scanning speed, because then the z-distances between the individual focal planes 12, which are each scanned during the readout of the image sensor 9, decrease. In other words, by reducing the scanning speed for a given maximum image frequency of the image sensor 9, the z-resolution in the determination of the best focal plane 13 can thus be increased.

[0079] The image sensor 9 can also, however, be operated in a rolling shutter mode, for example. In this case, different image areas 20a, 20b, and 20c of individual images 19, which are acquired using the image sensor 9, can correspond to different z-positions z.sub.1, z.sub.2, z.sub.3 along the optical z-axis 8, as illustrated in FIG. 6, the block arrow indicating the direction of the rolling shutter therein. In this case, an evaluation area 21 within the respective individual image 19, which is evaluated by the controller 10 to assess the respective focal plane 12, can be adapted, for example, as a function of a mean scanning speed of the z-scan 17 and/or a number of focal planes 12 to be acquired within the scanning range 14. For example, it makes sense to shrink the evaluation area 21 in the vertical direction in FIG. 6 (thus in the direction of the rolling shutter) when the scanning speed is increased. This is because in this case a larger z-area is covered in the acquisition of a single individual image 19 (i.e., a larger scanning distance is optically traversed in the z-direction during the acquisition of the individual image 19), so that the evaluation area 21 is to be shrunken to maintain a high z-resolution.

[0080] As already illustrated in FIG. 1, at least two z-scans 27, 28 can also be executed in succession within the scanning range 14 to find the best focal plane 13. The two z-scans 27, 28 can differ here in their respective length 18 and also the respective step width 15 used or, for example, also in the scanning speed used and not least also in their respective location (for example mean z-position) within the scanning range 14. In the example of FIG. 1, a first coarse z-scan 27 (having comparatively large distance between the individual optically scanned planes 12) extends over the entire length 16 of the scanning range 14. The fine scan 28 executed thereafter with respect to time, in contrast, has a smaller step width 15 Δz.sub.2<Δz.sub.1 in relation to the first coarse scan 27 and a shorter length 18. Moreover, the coarse scan 27 is executed at a lower zoom level and thus with less image magnification than the fine scan 28. More precisely, to carry out the coarse scan 21, initially a minimal zoom level is set by the controller 10, which is settable by means of the zoom optics unit 4, and the coarse scan 27 is executed over the maximum possible scanning range 14 which can be covered by tuning the focus lens 6. Only then is there a return to a zoom level, which has previously been selected by a user to record a video image data stream, to carry out the following fine scan 28. Due to the smaller step width Δz.sub.2 of the fine scan 28, the local z-resolution achievable using this scan is higher than that of the coarse scan 27 previously carried out. At the same time, finding the best focal plane 13 is accelerated with respect to time by this two-step procedure, since the fine scan is not over the entire scanning range 14, rather only within a smaller z-range previously identified with the aid of the coarse scan 27, in which the best focal plane 13 is located.

[0081] These individual method steps are illustrated once again in FIG. 7, wherein it is apparent on the basis of the black arrows that initially the individual focal planes 12 are approached step-by-step at the z-coordinates z.sub.1 to z.sub.8 and recorded with the aid of the image sensor 9 (=coarse scan 27, black block arrows), and that subsequently the first focal plane 12 of the fine scan 28 is approached at the z-coordinate z.sub.a. From there, subsequently the fine scan 28 is carried out step-by-step up to the focal plane 12 at the z-coordinate z.sub.h. The fine scan 28 thus takes place in the opposite scanning direction as the coarse scan 27.

[0082] In summary, to improve the accuracy and the speed of an autofocus method using which a present best focal plane 13 may be found in an automated manner, which enables a best possible image quality for an object 3 located at a specific working distance 11 to an optical imaging system 1, it is provided that at least one parameter used during a z-scan 17 be adapted in an automated manner as a function of a presently set optical zoom level and/or a current estimated value of the working distance 11. During the z-scan 17, a present location of a focal plane 12 of the optical imaging system 1 within a scanning range 14 is displaced along an optical z-axis 8 of the imaging system 1, wherein the individual focal planes 12 are each evaluated to identify the best focal plane 13 among them (cf. FIG. 4). This evaluation can preferably be carried out on the basis of image information which is acquired using the image sensor 9 of the imaging system 1 (=image-based autofocus).

LIST OF REFERENCE SIGNS

[0083] 1 optical imaging system [0084] 2 video camera [0085] 3 object [0086] 4 zoom optics unit (displaceable or tunable) [0087] 5 zoom actuator [0088] 6 focus lens (displaceable or tunable) [0089] 7 focus actuator [0090] 8 optical z-axis [0091] 9 image sensor [0092] 10 controller [0093] 11 working distance (between 1 and 3) [0094] 12 (present) focal plane (specified by 1 or 6) [0095] 13 best focal plane (to image 3 optimally sharp on 9) [0096] 14 scanning range (along 8 with respect to 12) [0097] 15 step width (distance between 12, after step-by-step adaptation of the location of 12) [0098] 16 length (of 14) [0099] 17 z-scan within 14 [0100] 18 length of 17 [0101] 19 individual image (recorded using 9) [0102] 20 image area (within 19) [0103] 21 local maximum [0104] 22 global maximum [0105] 23 front lens [0106] 24 zoom lens [0107] 25 (bidirectional) control line [0108] 26 objective [0109] 27 coarse scan [0110] 28 fine scan [0111] 29 (present) scanning direction (of 17)