Microscopy method and microscope for producing an image of an object

Abstract

A microscopy method is for producing an electronic image of an object, wherein the object is imaged with an adjustable optical imaging scale on an image detector. The method includes: selecting a parameter for the electronic image, wherein the parameter can be influenced by the optical imaging scale and differs from the image field dimensions, and setting a setpoint value range for the parameter, setting a total imaging scale for the electronic image, wherein adjusting or controlling the parameter of the electronic image is implemented such that, at the same time, the parameter of the electronic image lies in the specified setpoint value range with a tolerance and the set total imaging scale is obtained, wherein the optical imaging scale forms a basis for a manipulated variable of the adjustment or closed-loop control and a digital image magnification is carried out on the basis of the set total imaging scale.

Claims

1. A microscopy method for producing an electronic image of an object, the method comprising: setting a total imaging scale for the electronic image; providing a cost function depending on at least two of: a signal-to-noise ratio, a depth of field range, working distance, an object illumination intensity, and an image resolution of the electronic image as parameters; illuminating and imaging the object with an optical imaging scale onto an electronic image detector providing electronic image data, wherein the optical imaging scale is adjusted in a closed-loop control to minimize the cost function in said closed-loop control; generating the electronic image by digital post magnification of the electronic image data to the total imaging scale; and, wherein the cost function is represented by an equation
KF(Z,V).sup.2=W1*((resolution(Z,V)−limit.sub.resolution)/limit.sub.resolution).sup.2+W2*((depth of field range(Z,V)−limit.sub.depth of field range)/limit.sub.depth of field range).sup.2+W3*(limit.sub.illuminance/(illuminance(Z,V)−limit.sub.illuminance)).sup.2 wherein W1, W2, and W3 are weighting factors, V is the optical imaging scale, Z is the digital magnification, “limit.sub.resolution” is a predetermined minimal resolution, “limit.sub.depth of field range” is a predetermined depth of field range, and “limit.sub.illuminance” is a predetermined maximum illuminance.

2. The microscopy method of claim 1, wherein the closed-loop control comprises: producing a preliminary electronic image of the object with the optical imaging scale realizing the total imaging scale; reducing the optical imaging scale until the value of the cost function lies within a specified setpoint value range with a tolerance; and, performing digital post magnification of the preliminary electronic image to generate the electronic image having the total imaging scale.

3. The microscopy method of claim 1, wherein the cost function is minimized such that an image resolution of the electronic image lies above a specified minimum value.

4. The microscopy method of claim 1, wherein additionally the working distance is adjusted in said closed-loop control.

5. The microscopy method of claim 1, wherein additionally the object illumination intensity is adjusted in said closed-loop control.

6. The microscopy method of claim 1, wherein the optical imaging scale is adjusted such that an image field completely covers a detector surface of the electronic image detector.

7. The microscopy method of claim 1, wherein the object is illuminated with illumination radiation in an illumination area, wherein the size of the illumination area is adjusted such that an extension of the illumination area corresponds to the region of the object visible in a final electronic image and regions of the object which are located in an optical object field but not visible in the electronic image are not illuminated.

8. The microscopy method of claim 1, wherein said closed-loop control is achieved via proportional closed-loop control, proportional-integral closed-loop control, proportional-integral-derivative closed-loop control, or fuzzy closed-loop control.

9. The microscopy method of claim 1, wherein an absolute optical imaging scale, a proportion of the optical imaging scale in the total imaging scale or a ratio of optical imaging scale to digital post magnification is adjusted in the control.

10. A microscope for producing an electronic image of an object, the microscope comprising: an image detector configured to generate electronic image data; an objective lens, a tube lens and an adjustable optical zoom lens configured to conjointly produce an optical image on said image detector with an adjustable optical imaging scale; a control device including a processor and being configured to produce an electronic image of the object from the electronic image data and to adjust the adjustable optical imaging scale; said control device being further configured to: set a total imaging scale for the electronic image; provide a cost function depending on at least two of: a signal-to-noise ratio, a depth of field range, a working distance, an object illumination intensity and an image resolution as a parameter; illuminate and image the object with an optical imaging scale onto the electronic image detector and adjust the optical imaging scale in a closed-loop control to minimize the cost function in said closed-loop control; generate the electronic image by digital post magnification of the electronic image data to the total imaging scale; and, wherein the control device is further configured such that the cost function is represented by the following equation
KF(Z,V).sup.2=W1*((resolution(Z,V)−limit.sub.resolution)/limit.sub.resolution).sup.2+W2*((depth of field range(Z,V)−limit.sub.depth of field range)/limit.sub.depth of field range).sup.2+W3*(limit.sub.illuminance/(illuminance(Z,V)−limit.sub.illuminance)).sup.2 wherein W1, W2, and W3 are weighting factors, V is the optical imaging scale, Z is the digital magnification, “limit.sub.resolution” is a predetermined limit resolution, “limit.sub.depth of field range” is a predetermined depth of field range, and “limit.sub.illuminance” is a predetermined maximum illuminance.

11. The microscope of claim 10, wherein said control device is further configured to: produce a preliminary electronic image of the object with the optical imaging scale realizing the total imaging scale; reduce the optical imaging scale until the cost function lies within a specified setpoint value range with a tolerance; and, perform digital post magnification of the preliminary electronic image to generate a final electronic image having the total imaging scale.

12. The microscope of claim 10, wherein said control device is further configured to adjust the working distance in the closed-loop control.

13. The microscope of claim 10, wherein said control device is further configured to adjust the object illumination intensity in the closed-loop control.

14. The microscope of claim 10, wherein said image detector defines a detector surface; said control device is further configured to adjust the optical imaging scale such that an image field completely covers said detector surface of said electronic image detector.

15. The microscope of claim 10, wherein said control device is further configured to control the microscope such that the object is illuminated with illumination radiation in an illumination area defining a size, wherein the size of the illumination area is adjusted such that an extension of the illumination area corresponds to a region of the object visible in a final electronic image and regions of the object which are located in an optical object field but not visible in the final electronic image are not illuminated.

16. The microscope of claim 10, wherein the control device is further configured to implement proportional closed-loop control, proportional-integral closed-loop control, proportional-integral-derivative closed-loop control or fuzzy closed-loop control.

17. The microscope of claim 10, wherein said control device is further configured to additionally adjust one of the following in the closed-loop control: an absolute optical imaging scale, a proportion of the optical imaging scale in the total imaging scale or a ratio of optical imaging scale to digital magnification.

18. A microscopy method for producing an electronic image of an object, the method comprising: setting a total imaging scale for the electronic image; providing a specified parameter for the electronic image, wherein the specified parameter includes at least one of the following: a signal-to-noise ratio in the electronic image, a depth of field range, a working distance being a distance between the microscope and the object, and an object illumination intensity; specifying a setpoint value range for the specified parameter; illuminating and imaging the object with an adjustable optical imaging scale onto an electronic image detector providing electronic image data while open-loop or closed-loop controlling the specified parameter to have a value in the specified setpoint value range with a tolerance, wherein at least the adjustable optical imaging scale forms a basis for a manipulated variable of said open-loop or closed-loop controlling; generating the electronic image by performing digital image magnification on the electronic image data to the total imaging scale; wherein a cost function is evaluated and minimized in said open-loop or closed-loop controlling by adjusting said adjustable optical imaging scale and digital magnification, the cost function linking several of the signal-to-noise ratio, the depth of field range, the working distance, the object illumination intensity and resolution; wherein the cost function is represented by an equation
KF(Z,V).sup.2=W1*((resolution(Z,V)−limit.sub.resolution)/limit.sub.resolution).sup.2+W2*((depth of field range(Z,V)−limit.sub.depth of field range)/limit.sub.depth of field range).sup.2+W3*(limit.sub.illuminance/(illuminance(Z,V)−limit.sub.illuminance)).sup.2 wherein W1, W2, and W3 are weighting factors, V is the optical imaging scale, Z is the digital magnification, “limit.sub.resolution” is a predetermined limit resolution, “limit.sub.depth of field range” is a predetermined depth of field range, and “limit.sub.illuminance” is a predetermined maximum illuminance.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The invention will now be described with reference to the drawings wherein:

(2) FIG. 1 shows a schematic illustration of an embodiment of a microscope;

(3) FIG. 2 shows a schematic illustration for illustrating a microscopy method carried out with the microscope; and,

(4) FIGS. 3A and 3B show block diagrams for illustrating the essential method steps of the method of FIG. 2.

DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE INVENTION

(5) FIG. 1 schematically shows a microscope M for imaging an object O. Preferably, the microscope M is a surgical microscope which images part of a patient during a surgical intervention. The microscope M images the object O from an object field on an image field, lying on a detector area 6 of an image detector 8, via an objective 2 and a tube lens 4. A zoom optical unit 10 is present in the beam path, the zoom optical unit altering the optical imaging scale, for example, the size of the image field in the case of an unchanging size of the object field, and including to this end a drive not denoted in any more detail. The optical imaging scale, with which the object field is imaged on the image field by the objective 2, the zoom optical unit 10 and the tube lens 4, is therefore adjustable. The image detector 8 is read by a controller 12, which converts the data obtained in this way into an electronic image, which is displayed on a display device 14.

(6) The object O is illuminated by an illumination source 16, which emits the illumination radiation 18 toward the object O; the size of the illuminated spot is preferably adjustable, for example, by a radiant field stop (not illustrated).

(7) The working distance between the microscope M and the objective O is adjustable in one embodiment. To this end, provision is made of an adjustment unit 20 (not illustrated) with corresponding support means, which adjust the microscope M including at least the objective 2, the zoom optical unit 10, the tube lens 4 and the image detector 8 together in relation to the object O. This has been disclosed for surgical microscopes from the prior art. Here, the object field dimension is then also adjusted unless the zoom optical unit 10 is embodied in such a way that it also allows an adjustment of the object field dimension. Such varifocal objectives are possible in embodiments.

(8) The control device 12 is not only connected to the image detector 8 and the drive of the zoom optical unit 10 but optionally also to the illumination device 16 in embodiments with adjustable radiant field dimensions.

(9) To present the electronic image on the display device 14, the control device 12 can not only adjust the imaging scale by way of the zoom optical unit 10 but can also adjust a digital magnification, implemented in the electronic image data from the image detector 8, before the electronic image is displayed on the display device 14. Preferably, the control device 12 can also adjust the exposure time and the resolution of the image detector 8, the latter by pixel binning, for example.

(10) Further optionally, the control device 12 includes an input device 22, by means of which a user can adjust the mode of operation of the microscope M. A user, for example a surgeon, can specify or adjust the imaging scale of the microscope M using the control device 12.

(11) Preferably, the microscope M is embodied as a stereo microscope, of the type as is known from DE 10 2018 110 643 B3. The content of this publication is incorporated herein by reference.

(12) The functionality for improving a parameter of the image is explained below with reference to FIG. 2. The microscopy method carried out in the process brings about an adjustment/closed-loop control of the parameter by virtue of a desired total imaging scale initially being set on the control device 12. The latter is then divided between an optical imaging scale serving as manipulated variable by way of an appropriate adjustment of the zoom optical unit 10 and a digital magnification, brought about in the image data prior to the display on the display device 14, such that the parameter lies within a predetermined setpoint value range or misses the latter by no more than a tolerance. In the process, the parameter to be optimized can likewise be selected on the control device 12 or is fixedly specified in other embodiments, that is, already stored in the control device 12 and set thereby.

(13) Here, the adjustment/closed-loop control is initially explained below in a very simple embodiment. The latter initially produces a preliminary image of the object, with the total imaging scale only being caused by the optical imaging scale, that is, the setting of the zoom optical unit 10. A digital magnification does not yet occur here. Subsequently, the optical imaging scale is reduced until the parameter reaches the desired setpoint value range (possibly with the tolerance being taken into account). Naturally, this occurs if the parameter in the preliminary region is not in the setpoint value range. If the parameter was placed into the desired setpoint value range in this way, the digital magnification ensures that the reduction of the optical imaging scale is compensated such that, total, the set total magnification entered at the control device 12, that is, the total imaging scale, is reached again.

(14) Here, the reduction in the optical imaging scale can additionally or alternatively also be realized by an adjustment of the working distance if the corresponding device 20 to this end is present.

(15) By way of example, the signal-to-noise ratio of the image can be improved as a parameter since the light intensity incident on the image detector 8 is increased by altering the optical imaging scale.

(16) To elucidate this, the object O is represented as a flower in a schematic illustration in FIG. 2. The object O reflects or emits light, which is supplied by the optical units 2, 10, 4 to the image detector 8. To produce a preliminary image, illustrated in FIG. 2 as path A, the zoom optical unit 10 is adjusted in such a way that the object O is imaged on the detector area 72 of the image detector 8 with the total imaging scale set in advance. Here, the dimensions of the image field 71 fit to the detector area 72 (provided with reference sign 6 in FIG. 1).

(17) Subsequently, a check is carried out as to whether the signal-to-noise ratio is within a specified setpoint value range. This can be implemented by way of a known noise analysis. The specified setpoint value range has a lower limit, which is adjustable by the user of the microscope M. By way of example, this lower limit can be chosen in such a way that the final image has a desired, acceptable signal-to-noise ratio. If the signal-to-noise ratio is not in the specified setpoint value range, the optical imaging scale of the zoom optical unit 10 is reduced until the signal-to-noise ratio lies in the specified setpoint value range. This is illustrated as path B in FIG. 2.

(18) The reduction in the optical imaging scale causes the image field 71 to significantly underfill the detector area 72. On the other hand, all of the imaging radiation from the object field reaches the detector area 72. Since not all pixels of the detector area 72 are illuminated, the number of photons per illuminated pixel is increased in comparison with the preliminary image. Now, the final digital image is produced by virtue of a digital magnification being carried out by way of a digital zoom factor. In this way, only pixels with an improved signal-to-noise ratio contribute to the image (path A).

(19) To ensure that the region of the object O visible in the preliminary image corresponds with the section of the object O visible in the final image, the final image is digitally post-magnified with the zoom factor which corresponds to the quotient of total imaging scale and reduced optical imaging scale. The digitally magnified image is presented to the observer. The signal-to-noise ratio has been improved in comparison with the preliminary image on account of the higher light intensity per pixel during the imaging with the reduced magnification scale.

(20) In another embodiment, the method is used to improve the depth of field range of the image. Here, too, a check is carried out as to whether the depth of field range of the image lies within a specified setpoint value range. Should this not be the case, the optical imaging scale is reduced until the depth of field range lies in the specified setpoint value range and the final image is subsequently digitally magnified to the total imaging scale.

(21) The principal procedure is explained again on the basis of FIGS. 3A and 3B, with the working distance optionally being altered in this case. With reference to FIG. 3A, in a first step S1, the position of the microscope M is adjusted in relation to the object O and hence the working distance is set. Further, a total imaging scale is set, for example by virtue of the user adjusting the optical imaging scale to the desired total imaging scale. In this way, the section of the object O visible in the image and the magnification thereof is set in step S1. Further, a preliminary image of the object O is produced.

(22) In a subsequent step S2, at least one parameter of the image, such as the signal-to-noise ratio or the depth of field range, for example, is set.

(23) In step S3, a check is carried out as to whether the parameter of the preliminary image lies within a specified setpoint value range or a setpoint value range likewise set in step S2. Should this not be the case, the optical imaging scale is reduced until the parameter lies within the specified setpoint value range. This can be implemented by altering the working distance and/or the zoom optical unit. Subsequently, the final image of the object O is produced by virtue of there being a digital magnification to the total magnification scale 5. The reduction of the optical imaging scale and compensating digital magnification are implemented until the parameter lies in the specified setpoint value range.

(24) An optional condition for the reduction of the imaging scale is that the resolution of the digitally post-magnified final image reaches a specified minimum value of the resolution

(25) In a subsequent optional step S4, the illuminance of the illumination device 50 is reduced in this way until the signal-to-noise ratio reaches a threshold. In this way, the illuminance can be reduced without the signal-to-noise ratio falling below a specified limit value.

(26) Further, the area 28 of the object O illuminated by the illumination device 16 can optionally be adapted to the digital zoom in such a way that the illuminated area 28 corresponds with the section displayed in the final image. Thus, only the region of the object O visible in the digitally post-magnified image is illuminated. This means that the optical image field images regions that are not illuminated in the object O. In this way, the radiation generated by the illumination device 16 can be concentrated on the relevant regions.

(27) Another embodiment of the microscopy method, which is illustrated schematically in FIG. 3B, again starts with step S1, as explained above. Here, it is not only a simplified incremental procedure for adjustment/closed-loop control that is now pursued; instead, there is closed-loop control with a control loop to the effect of the value of the parameter being placed in the setpoint value range in a control loop, with the corresponding manipulated variable being the optical imaging scale, for example, the adjustment of the zoom optical unit 10. At the same time, a digital magnification ensures that the total imaging scale set is maintained. Ultimately, the closed-loop control therefore adjusts the ratio between optical imaging scale and digital magnification.

(28) In step S5, the cost function KF presented below is minimized.
KF(Z,V).sup.2=W1*((resolution(Z,V)−limit.sub.resolution)/limit.sub.resolution).sup.2+W2*((depth of field range(Z,V)−limit.sub.depth of field range)/limit.sub.depth of field range).sup.2+W3*(limit.sub.illuminance/(illuminance(Z,V)−limit.sub.illuminance)).sup.2

(29) The cost function KF depends on the resolution of the image, the illuminance and the depth of field range, with the weighting factors W1, W2 and W3 allowing individual terms to be preferred. V is the optical imaging scale; Z represents the digital zoom factor. The product of optical imaging scale V and the digital zoom factor Z yields the total magnification scale. The limit values of the resolution “limit.sub.resolution” and of the depth of field range “limit.sub.depth of field” are preferably adjusted manually by the observer. The limit value of the illuminance “limit.sub.illuminance” is set on the basis of the object O, in particular; by way of example, this limit value can be the maximum illuminance before radiation damage is caused in the object O.

(30) In steps S6 and S7, a control loop is carried out to minimize the characteristics ascertained in the cost function KF. In the process, a check is carried out in step S7 as to whether the parameters lie in the specified setpoint value range. If the parameters lie in the specified setpoint value range, the method ends in step S7. In one embodiment, the control device 12 contains a controller, for example a P-controller, a PI-controller or a PID-controller or a fuzzy controller.

(31) It is understood that the foregoing description is that of the preferred embodiments of the invention and that various changes and modifications may be made thereto without departing from the spirit and scope of the invention as defined in the appended claims.