METHOD FOR OPERATING A STEREOSCOPIC MEDICAL MICROSCOPE, AND MEDICAL MICROSCOPE

Abstract

The invention relates to a method for operating a stereoscopic medical microscope, wherein deteriorated and/or invalid calibration data are recognized, wherein for this purpose mutually corresponding image representations of at least one feature arranged in capture regions of cameras of a stereo camera system of the medical microscope are captured by means of the cameras, the captured image representations are evaluated by means of feature-based image processing, wherein the at least one feature is recognized in this case in the captured image representations and a misalignment and/or a decalibration of the cameras of the stereo camera system are/is recognized on the basis of the at least one feature recognized; and wherein at least one measure is carried out depending on an evaluation result. Furthermore, the invention relates to a medical microscope.

Claims

1. A method for operating a stereoscopic medical microscope, wherein deteriorated and/or invalid calibration data are recognized, wherein for this purpose mutually corresponding image representations of at least one feature arranged in capture regions of cameras of a stereo camera system of the medical microscope are captured by means of the cameras, the captured image representations are evaluated by means of feature-based image processing, wherein the at least one feature is recognized in this case in the captured image representations and a misalignment and/or a decalibration of the cameras of the stereo camera system are/is recognized on the basis of the at least one feature recognized, and wherein at least one measure is carried out depending on an evaluation result.

2. The method as claimed in claim 1, wherein during the evaluating on the basis of the at least one feature recognized an offset at least in one direction between the mutually corresponding image representations captured is determined and provided as evaluation result.

3. The method as claimed in claim 1, wherein during the evaluating on the basis of the at least one feature recognized a rotation between the mutually corresponding image representations captured is determined and provided as evaluation result.

4. The method as claimed in claim 1, wherein for the purpose of checking a focus position calibration, a sharpness of the at least one feature in the image representations respectively captured by the cameras is determined, wherein a difference between the sharpnesses determined is determined and provided as evaluation result.

5. The method as claimed in claim 1, wherein during the capturing of the image representations, the at least one feature is injected into a respective capture region and/or a respective beam path of the cameras of the stereo camera system by means of at least one injection device of the medical microscope.

6. The method as claimed in claim 1, wherein the at least one feature, for the purpose of capturing, is arranged in the capture regions and/or in an intermediate image plane of the medical microscope in an automated manner by means of a feature actuator system and/or wherein the stereo camera system, for the purpose of capturing, is moved in an automated manner by means of an actuator system, such that the at least one feature is arranged in the capture regions.

7. The method as claimed in claim 1, wherein capturing the image representations, recognizing the deteriorated and/or invalid calibration data and carrying out the at least one measure are/is carried out in an automated manner during starting up and/or shutting down and/or during an idle state of the medical microscope.

8. The method as claimed in claim 1, wherein capturing the image representations, recognizing the deteriorated and/or invalid calibration data and carrying out the at least one measure are/is carried out during regular operation of the medical microscope.

9. The method as claimed in claim 1, wherein the evaluation result is compared with at least one predefined limit value, wherein the at least one measure Q is selected depending on a comparison result.

10. The method as claimed in claim 9, wherein a first limit value has been predefined or is predefined, wherein as measure calibration data are determined and/or adapted if the evaluation result is greater than or equal to the predefined first limit value.

11. The method as claimed in claim 10, wherein the evaluation result is stored in a maintenance database operated for anticipatory maintenance if the evaluation result lies below the predefined first limit value.

12. The method as claimed in claim 9, wherein a second limit value has been predefined or is predefined, wherein as measure a service message is generated and displayed and/or sent if the evaluation result is greater than or equal to the predefined second limit value.

13. A medical microscope, comprising: a stereo camera system having cameras, configured for capturing in each case mutually corresponding image representations of capture regions of the cameras, and an image processing device, configured for processing the captured image representations, wherein the image processing device is furthermore configured to recognize deteriorated and/or invalid calibration data and for this purpose to instigate capturing of mutually corresponding image representations of at least one feature arranged in the capture regions of the cameras of the stereo camera system, to evaluate the captured mutually corresponding image representations by means of feature-based image processing, wherein the at least one feature is recognized in this case in the captured image representations and a misalignment and/or a decalibration of the cameras of the stereo camera system are/is recognized on the basis of the at least one feature recognized, and to instigate at least one measure depending on an evaluation result.

Description

[0036] FIG. 1 shows a schematic illustration of one embodiment of the medical microscope;

[0037] FIG. 2 shows a schematic flow diagram of one embodiment of the method for operating a stereoscopic medical microscope.

[0038] FIG. 1 shows a schematic illustration of one embodiment of the medical microscope 1. The medical microscope 1 comprises a stereo camera system 2 having a left camera 21 and a right camera 2r. Furthermore, the medical microscope 1 comprises an image processing device 3. The image processing device 3 comprises a computing device 3-1 and a memory 3-2. The image processing device 3 can be part of a control device 4 of the medical microscope 1 and/or be provided by said control device. The medical microscope 1 is a surgical microscope, for example. The method described in this disclosure is explained in more detail below on the basis of the medical microscope 1.

[0039] In principle, the control device 4 and/or the image processing device 3 can also be embodied as microscope-external devices which are provided for example by means of an external data processing device, for example by means of a desktop, laptop or tablet computer, or else by means of a cloud-based solution.

[0040] The left camera 21 captures a left image representation 101 of a left capture region 301 via a stereoscopic imaging optical unit 5 and the right camera 2r captures a right image representation 10r— corresponding thereto—of a right capture region 30r. In particular, provision is made for the cameras 21, 2r to generate raw signals by means of image sensors, the image representations 101, 10r being generated from said raw signals by way of signal technology, calibration data being taken into account here, which correct for example an offset and/or a rotation between the captured image representations 101, 10r by way of signal technology. The captured image representations 101, 10r can subsequently be displayed on a display device 6 of the medical microscope 1.

[0041] The image processing device 3 is configured for processing the captured image representations 101, 10r. The image processing device 3 is furthermore configured to recognize deteriorated and/or invalid calibration data and for this purpose to instigate capturing of mutually corresponding image representations 101, 10r of at least one feature 31 arranged in the capture regions of the cameras 21, 2r of the stereo camera system 2. The image processing device 3 evaluates the mutually corresponding image representations 101, 10r captured, wherein the at least one feature 31 is recognized in this case in the captured image representations 101, 10r and a misalignment and/or a decalibration of the cameras 21, 2r of the stereo camera system 2 are/is recognized on the basis of the at least one feature 31 recognized. The image processing device 3 instigates at least one measure 20 depending on an evaluation result 15 obtained.

[0042] It can be provided that during the evaluating on the basis of the at least one feature 31 recognized an offset 15-1 at least in one direction between the mutually corresponding image representations 101, 10r captured is determined and provided as evaluation result 15 by means of the image processing device 3. The offset 15-1 comprises for example a difference between positions of image elements which correspond to the at least one feature 31 in the captured image representations 101, 10r. By way of example, at least one line difference (offset in the y-direction) can be determined. However, additionally or alternatively, a column difference (offset in the x-direction) can also be determined.

[0043] Furthermore, other error measures can also be determined, for example a Euclidean distance between said image elements in the two captured image representations 101, 10r.

[0044] It can be provided that during the evaluating on the basis of the at least one feature 31 recognized a rotation 15-2, in particular a rotation angle or rotation difference angle, between the mutually corresponding image representations 101, 10r captured is determined and provided as evaluation result 15. In this case, the rotation 15-2 denotes in particular a rotation 15-2 around an image center point of the image representations 101, 10r. Methods from the field of computer vision and/or methods of artificial intelligence known per se can be used for determining the rotation 15-2.

[0045] It can be provided that for the purpose of checking a focus position calibration, a sharpness of the at least one feature 31 in the image representations 101, 10r respectively captured by the cameras 21, 2r is determined, wherein a difference 15-3 between the sharpnesses determined is determined and provided as evaluation result 15.

[0046] It can be provided that during the capturing of the image representations 101, 10r, the at least one feature 31 is injected into a respective capture region 301, 30r and/or a respective beam path of the cameras 21, 2r of the stereo camera system 2 by means of a left injection device 71 and a right injection device 7r of the medical microscope 1. In principle, it is also possible to use just one (common) injection device situated in a common optical path of the cameras 21, 2r. The at least one feature 31 is injected into a respective beam path of the imaging optical unit 5 in particular at the position of the injection devices 71, 7r (by means of a semitransparent mirror), such that said at least one feature can be captured in each case by image sensors of the cameras 21, 2r. The injection devices 71, 7r are controlled for example by means of the control device 4 of the medical microscope 1. The information to be injected, in particular the at least one feature 31 to be injected, is likewise generated and provided by the control device 4. An injection device can be for example an Integrated Data Injection System (IDIS) from Carl Zeiss Meditec AG.

[0047] It can be provided that the at least one feature 31, for the purpose of capturing, is arranged in the capture regions 301, 301 in an automated manner by means of a feature actuator system (not shown) and/or that the stereo camera system 2, for the purpose of capturing, is moved in an automated manner by means of an actuator system 8, such that the at least one feature 31 is arranged in the capture regions 301, 30r. For this purpose, the actuator system 8 is controlled by means of the control device 4. The feature actuator system for arranging the at least one feature 31 in the capture regions 301, 30r can comprise a pivot arm, for example, by means of which a calibration object with the at least one feature 31 can be pivoted into the capture regions 301, 30r.

[0048] It can be provided that capturing the image representations 101, 10r, recognizing the deteriorated and/or invalid calibration data and carrying out the at least one measure 20 are/is carried out in an automated manner during starting up and/or shutting down and/or during an idle state of the medical microscope 1. This can be initiated by the control device 4 and/or by the image processing device 3. By way of example, checking the calibration can be a process that is permanently integrated in a sequence of starting up and/or shutting down the medical microscope 1. An idle state can be recognized for example by means of the image processing device 3, wherein the latter for this purpose continually evaluates captured image representations 101, 10r and instigates the checking if (in particular whenever) the content imaged in the image representations 101, 10r does not change for a predefined minimum duration. Furthermore, an idle state can also be recognized by monitoring parameters or changes in parameters of the medical microscope 1. If no parameters, in particular no control parameters of an actuator system of the medical microscope 1, are changed for a predefined time, then an idle state is recognized. Furthermore, a software interrogation can also take place, which establishes whether or not the medical microscope 1 is in an idle state. An idle state can also be recognized, in particular, if no interaction with the medical microscope 1 takes place for a predefined duration.

[0049] It can also be provided that capturing the image representations 101, 10r, recognizing the deteriorated and/or invalid calibration data and carrying out the at least one measure 20 are/is carried out during regular operation of the medical microscope 1.

[0050] It can be provided that the evaluation result 15 is compared with at least one predefined limit value 16, wherein the at least one measure 20 is selected depending on a comparison result. A limit value 16 can have been predefined or be predefined for example for an offset 15-1, a rotation 15-2, a difference 15-3 between the determined sharpnesses (focus position) and/or for some other error measure. The comparing takes place in the image processing device 3 and/or in the control device 4.

[0051] It can be provided that a first limit value 16-1 has been predefined or is predefined, wherein as measure 20 calibration data are determined and/or adapted if the evaluation result 15 is greater than or equal to the predefined first limit value 16-1.

[0052] It can furthermore be provided that the evaluation result 15 is stored in a maintenance database operated for anticipatory maintenance if the evaluation result 15 lies below the predefined first limit value 16-1.

[0053] It can be provided that a second limit value 16-2 has been predefined or is predefined, wherein as measure 20 a service message is generated and displayed and/or sent if the evaluation result 15 is greater than or equal to the predefined second limit value 16-2.

[0054] FIG. 2 shows a schematic flow diagram of one embodiment of the method for operating a stereoscopic medical microscope. The medical microscope is configured for example like the embodiment shown in FIG. 1.

[0055] The method is started in a method step 100 when the medical microscope starts up for (regular) operation and/or when the medical microscope shuts down after (regular) operation.

[0056] In a method step 101, at least one feature is injected into beam paths of cameras of a stereo camera system by means of injection devices (e.g. IDIS). The at least one feature comprises a checkered pattern, for example. The checkered pattern is then injected into the left and right beam paths (or alternatively into a common beam path/path), such that the respective checkered patterns can be captured by the left and right cameras.

[0057] In a method step 102, mutually corresponding image representations of the at least one feature arranged in the capture regions of the cameras of the stereo camera system of the medical microscope are captured by means of the cameras. In particular, the checkered patterns respectively injected into the beam paths are captured. The captured image representations are evaluated by means of feature-based image processing. In particular, in this case, positions of the individual checkered squares in the checkered patterns are determined, for example with the aid of edge filters.

[0058] In method step 103, an x-/y-offset and/or a rotation, in particular a rotation angle, between the captured image representations and/or, in a manner corresponding thereto, between the cameras of the stereo camera system are/is determined and provided as evaluation result. It can also be provided that an aggregate error measure is determined and provided as evaluation result, which can comprise an x-offset and/or a y-offset and/or a rotation and/or further variables.

[0059] In method step 104, the evaluation result is compared with predefined limit values in order thereby to determine measures which are intended to be carried out and/or instigated. Method step 104 comprises method steps 105-110. In this case, it can be provided that the variables determined in method step 103 are compared with respective predefined limit values or that a (single) aggregate error measure is compared with respective predefined limit values.

[0060] Method step 105 involves checking whether the evaluation result (individual value or aggregate error measure) is less than a first predefined limit value. If this is the case, then in method step 106 the evaluation result is stored in a maintenance database operated for anticipatory maintenance.

[0061] Method step 107 involves checking whether the evaluation result (individual value or aggregate error measure) is greater than or equal to the first predefined limit value. If this is the case, then in a method step 108 calibration data are determined and/or adapted in a manner known per se. It can furthermore be provided that after determining and/or adapting the calibration data, the at least one feature, in particular the injected checkered patterns, are once again captured and evaluated in order to check an effectiveness of the determined and/or adapted calibration data.

[0062] Method step 109 involves checking whether the evaluation result (individual value or aggregate error measure) is less than a second predefined limit value. In this case, the second predefined limit value is chosen to be, in particular, greater than the first predefined limit value. If this is the case, then in method step 110 a service message is generated and displayed and/or sent, for example to a central server of a manufacturer of the medical microscope and/or of a maintenance service provider. In this way, in the case of a very severe misalignment and/or decalibration, a service action can be instigated in an automated manner.

[0063] The method is subsequently ended 111.

[0064] One of the advantages of the method described in this disclosure and of the medical microscope is the possibility of constantly monitoring a quality of captured image representations of the cameras of the stereo camera system in an automated manner, and being able to take suitable measures in the event of a decrease in this quality. Overall, a three-dimensional visualization with consistently high quality can be provided as a result.

LIST OF REFERENCE SIGNS

[0065] 1 Medical microscope [0066] 2 Stereo camera system [0067] 21 Left camera [0068] 2r Right camera [0069] 3 Image processing device [0070] 3-1 Computing device [0071] 3-2 Memory [0072] 4 Control device [0073] 5 Stereoscopic imaging optical unit [0074] 6 Display device [0075] 71 Left injection device [0076] 7r Right injection device [0077] 8 Actuator system [0078] 101 Left image representation [0079] 10r Right image representation [0080] 15 Evaluation result [0081] 15-1 Offset [0082] 15-2 Rotation [0083] 15-3 Difference (focus positions) [0084] 16 Limit value [0085] 16-1 First limit value [0086] 16-2 Second limit value [0087] 20 Measure [0088] 301 Capture region (left camera) [0089] 30r Capture region (right camera) [0090] 31 Feature [0091] 100-111 Method steps of the method