MICROSCOPY SYSTEM AND METHOD FOR OPERATING THE MICROSCOPY SYSTEM
20220346887 · 2022-11-03
Inventors
- Stefan Saur (Aalen, DE)
- Christian Voigt (Abtsgmuend, DE)
- Marco Wilzbach (Stuttgart, DE)
- Christoph Hauger (Aalen, DE)
Cpc classification
G02B21/365
PHYSICS
A61B34/20
HUMAN NECESSITIES
A61B90/20
HUMAN NECESSITIES
International classification
A61B34/20
HUMAN NECESSITIES
A61B90/00
HUMAN NECESSITIES
Abstract
A microscopy system includes a microscope, a stand configured to mount the microscope and including a drive device configured to move the microscope, a detection device configured to detect a spatial position of a target fastened to a body part or to an instrument, wherein the position detection device includes the target with at least one marker element and an image capture device configured to optically capture the target. The microscopy system further includes at least one control device configured to operate the microscopy system according to the detected position of the target, wherein the position detection device is configured to determine the position of the target by evaluating a two-dimensional image of the image capture device. In addition, a method for operating the microscopy system is provided.
Claims
1. A microscopy system comprising: a microscope; a stand configured to mount the microscope and including a drive device configured to move the microscope; a position detection device including a target with a marker element and an image capture device configured to optically capture the target, an identity being assigned to the marker element, the identity being identifiable on an image basis, the position detection device being configured to detect a spatial position of the target by evaluating a two-dimensional image captured by the image capture device; a control device configured to: control an operation of the microscopy system according to the spatial position of the target detected by the position detection device and according to the identity, and activate a predetermined operation mode depending on the identity, wherein a size of a working space of the microscopy system is adjustable based on the identified identity of at least one marker element, wherein the working space designates a spatial region in which the microscope of the microscopy system can be moved, and wherein the working space is formed with a predefined size about a current spatial position of the microscope upon activation of the operating mode.
2. A microscopy system comprising: a microscope, wherein the microscope comprises a microscope body, and wherein optical elements for beam guidance and/or beam shaping and/or beam deflection are arranged in a housing of the microscope body; a stand configured to mount the microscope and including a drive device configured to move the microscope; a position detection device including a target with a marker element and an image capture device configured to optically capture the target and an illumination device for illuminating the target, the position detection device being configured to detect a spatial position of the target by evaluating a two-dimensional image captured by the image capture device; a control device configured to: control an operation of the microscopy system according to the spatial position of the target detected by the position detection device, wherein the illumination device is arranged on and fixed in position relative to the microscope body, and wherein the illumination device generates light with a wavelength from the infrared range and no light with a wavelength from the visible range.
3. The microscopy system according to claim 1, wherein the control device is configured to at least one of (a) control the drive device according to the spatial position of the target, and (b) adjust at least one of an operating parameter, a movement parameter, and an operating mode of the microscope.
4. The microscopy system according to claim 1, wherein the image capture device of the position detection device is arranged in or on the microscope to permit beams detected by the image capture device to run through an optical element of the microscope.
5. The microscopy system according to claim 1, wherein the image capture device of the position detection device is arranged in or on the microscope to prevent beams detected by the image capture device from running through an optical element of the microscope.
6. The microscopy system according to claim 1, wherein the microscope includes a transparent element arranged between the image capture device of the position detection device and a detection region to be imaged.
7. The microscopy system according to claim 1, wherein position detection is carried out according to a sequence of at least two images.
8. The Microscopy system according to claim 7, wherein the sequence of the at least two images is captured by high dynamic range (HDR) imaging.
9. The microscopy system according to claim 1, wherein: the target includes a marker element, the marker element has an elliptical marker body or an elliptical marker surface and a geometric center of the elliptical marker body or of the elliptical marker surface, the marker body or the marker surface is filled with color spectrum points distributed radially with respect to the geometric center, and a color value of each color spectrum point of the marker/marker surface is determined according to an angle between a horizontal line through the geometric center, and a further line through the geometric center and the corresponding color spectrum point.
10. The microscopy system according to claim 1, wherein a position-based control is performed in real time.
11. The microscopy system according to claim 1, further comprising an illumination device configured to illuminate the target.
12. The microscopy system according to claim 11, wherein the illumination device generates light with a wavelength outside a visible range in an infrared wavelength range.
13. The microscopy system according to claim 2, wherein at least one of an intensity of an illumination, a working distance, and an exposure time of the image capture device are adjustable according to a distance of the target from the image capture device.
14. The microscopy system according to claim 1, wherein at least one of: the microscopy system includes a filter configured to filter beams detected by the image capture device, the image capture device only detects beams of a defined wavelength or of a wavelength range, or the image capture device is a monochrome image capture device.
15. The microscopy system according to claim 1, further comprising a means for activating a position-based control of the microscopy system.
16. The microscopy system according to claim 1, wherein the image capture device is a wide-angle camera.
17. The microscopy system according to claim 1, wherein an open-loop or closed-loop control of a movement of the microscope is effected according to at least one of an operating parameter of the microscope, and an operating parameter of the image capture device of the position detection device.
18. The microscopy system according to claim 1, wherein at least one of a number and a type of the enabled degrees of freedom of a movement of the microscope are adjustable.
19. The microscopy system according to claim 1, wherein at least one of (a) a size of a working space of the microscopy system, and (b) an admissible maximum speed of a movement is adjustable.
20. The microscopy system according to claim 1, further comprising a graphic user interface operated according to the spatial position of the target.
21. A method for operating a microscopy system according to claim 1, the method comprising: determining a spatial position of the target relative to an image capture device of a position detection device; and controlling an operation of the microscopy system according to the spatial position.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0144] The disclosure will now be described with reference to the drawings wherein:
[0145]
[0146]
[0147]
[0148]
[0149]
[0150]
[0151]
DESCRIPTION OF EXEMPLARY EMBODIMENTS
[0152] Identical reference signs hereinafter designate elements having identical or similar technical features.
[0153]
[0154] The stand 3 shown is an example of a kinematic structure for mounting and moving the microscope 2. A person skilled in the art will of course know that other kinematic structures may also be used.
[0155] The stand 3 includes drive devices (not shown) for moving the microscope 2. Here, the drive devices can, for example, permit a rotational movement of moving parts of the stand 3 about rotation axes 4, 5, 6 and a rotation axis parallel to the vertical axis z.
[0156] The microscopy system 1 moreover includes a position detection device for detecting a position of an instrument 19 that can be held and moved by a user 8. The user 8 can be a surgeon, for example. The position detection device includes at least one target 9 with at least one marker element 22 (see
[0157]
[0158] The position of the instrument 19 can be detected by the position detection device, by the position of the target 9 being determined, in particular on an image basis, wherein the position of the instrument 19 can then be determined on account of the fixed arrangement of the target 9 on the instrument 19. A relative position between target 9 and instrument 19 may be known beforehand and may be determined, for example, by registration. If only changes of position of the head are to be detected, this registration is not absolutely necessary.
[0159]
[0160]
[0161] Before the microscopy system 1 is put into operation, the coordinate system of the position detection device can be registered with the reference coordinate system shown. In other words, a transformation rule can be determined for transforming the position in the coordinate system of the position detection device into the reference coordinate system.
[0162] The position of the target 9 can be detected by evaluating exactly one two-dimensional image of the image capture device 10.
[0163] Moreover, the depicted control device 7, which can include a microcontroller for example, is configured in such a way that it can actuate the at least one drive device of the stand 3 according to the detected position of the target 9 or according to the detected change of position of the target 9 and thus of the instrument. Alternatively or in addition, at least one operating parameter and/or movement parameter of the microscope 2 or an operating mode of the microscope 2 can be adjusted by the control device 7, as explained above, according to the detected position of the target 9 or according to the detected change of position of the target 9 and thus of the instrument 19.
[0164]
[0165] In addition,
[0166]
[0167] It is possible, although not shown here, that the image capture device 10 is also arranged inside the housing of the microscope body 16 in such a way that, in the radiation direction, it lies behind a lens of the microscope 2. In this case, the lens and also the glass plate 18 are arranged between the image capture device 10 and the patient 13 who is to be observed.
[0168] An acquisition range of the image capture device 10 at least partially overlaps with an acquisition range of the microscope for magnified depiction of the patient or of regions of the body of the patient 13.
[0169] By moving the instrument 19, for example with his hands, it is possible that the user 8 moves the target 9 and thus changes the position of the latter. The change of position can be detected here by the position detection device, wherein the control device 7 then controls an operation of the microscopy system 1, in particular of the at least one drive device and/or of the microscope 2, according to the detected position of the target 9. In particular, the movement of the microscope 2 can track the movement of the instrument 19.
[0170]
[0171] The image capture device 10 is here arranged on the microscope 2 in such a way that the image acquisition range at least partially overlaps, or completely overlaps, with the image acquisition range of the microscope 2 for the magnified view.
[0172]
[0173]
[0174] The inner marker surface has varying color values or grey values. In particular, it is filled with color spectrum points, which are distributed radially with respect to the geometric center. A color value or grey value of each color spectrum point of the marker surface is defined, and can thus be determined, according to an angle between a horizontal line through the geometric center and a further line through the geometric center and the corresponding color spectrum point.
[0175]
[0176] It is then moreover possible to control the microscopy system 1 shown in
[0177]
[0178] The spatial position can be determined here in a reference coordinate system (see explanations concerning
[0179] In particular, a movement of the microscope 2 can be controlled on a position basis, in particular in such a way that an optical axis 17 of the microscope 2 is arranged in a desired orientation.
[0180] Alternatively or in addition, operating parameters of the microscope, for example a zoom and/or a focus, can be adjusted according to the detected position, in particular the detected change of position, and/or an operating mode of the microscope 2.
[0181] In a parameter adjustment mode, a zoom of the microscope 2 can be adjusted, for example, according to the detected relative position. In a movement control mode, a drive device of a stand 3 of the microscopy system 1 can be actuated in such a way that the microscope 2 is controlled with a desired movement. In particular, a change of position of the target 9 can be determined, wherein the drive device can then be actuated in such a way that the microscope 2 executes the same change of position (position tracking mode).
[0182] Moreover, it is also possible to control operation of an illumination device 20 (see
[0183] It is moreover possible that an identity of a marker element is identified in the second step S2, in which case the operation of the microscopy system 1, in particular also the movement, is then controlled according to the identified identity.
[0184]
[0185] It is possible that the microscopy system 1 shown in
[0186] In this case in particular, an evaluation can be made, before or during the second step S2, as to which operating mode is activated. It is also possible to evaluate whether an operating mode is activated and if so which one. Depending on the activated operating mode, further steps of the method can then be carried out. In particular, in the second step S2, movement control is effected only if the microscopy system 1 is in a movement mode that can be controlled on a position basis.
[0187] The position-based control or a desired operating mode can be activated, for example, by the user 8 actuating a switch, for example a foot-operated switch, by evaluation of a position or a change of position and/or by identification of a marker element 22.
[0188] Moreover, in the initial step S0, it is possible to determine a number and/or type of the degrees of freedom enabled for a movement of the microscope 2.
[0189] In regard to the first step Si and second step S2, reference may be made to the explanations concerning
[0190] It is possible that the position detection is carried out according to a sequence of at least two, typically three, images generated successively by the image capture device 10 (see FIG. 1). For this purpose, in the first step S1, a first image can be generated in a first subsidiary step, a second image can be generated in a subsequent second subsidiary step, and a third image can be generated in a subsequent third subsidiary step. These images can be generated with different imaging parameters or operating parameters of the image capture device 10, for example with different aperture stops.
[0191] In summary, microscopy system is provided including a microscope 2, a stand 3 for mounting the microscope 2, wherein the stand 3 comprises at least one drive device for moving the microscope 2, at least one position detection device for detecting a spatial position of a target 9, wherein the position detection device comprises the at least one target 9 with at least one marker element 22, 22a, . . . , 22d and an image capture device 10 for optical acquisition of the target 9, at least one control device 7 for controlling the operation of the microscopy system 1 according to the detected position of the target 9, wherein the position detection device is designed to determine the position of the target 9 by evaluating a two-dimensional image generated by the image capture device 10 of the position detection device.
[0192] According to an exemplary embodiment, the control device 7 is a control device 7 for controlling the at least one drive device according to the detected position of the target 9 and/or a control device for adjusting at least one operating parameter and/or movement parameter and/or an operating mode of the microscope 2.
[0193] According to another exemplary embodiment, the image capture device 10 of the position detection device is arranged in or on the microscope 2 in such a way that beams detected by the image capture device 10 run through at least one optical element of the microscope 2.
[0194] According to yet another exemplary embodiment, the image capture device 10 of the position detection device is arranged in or on the microscope 2 in such a way that beams detected by the image capture device 10 do not run through at least one optical element of the microscope 2.
[0195] According to an exemplary embodiment, a transparent element of the microscope 2 is arranged between the image capture device 10 of the position detection device and the detection region that is to be imaged.
[0196] According to another exemplary embodiment, the position detection can be carried out according to a sequence of at least two images.
[0197] According to yet another exemplary embodiment, the sequence can be generated in an
[0198] HDR method.
[0199] According to an exemplary embodiment, the target 9 comprises a marker element 22, 22a, . . . , 22d, wherein the marker element 22, 22a, . . . 22d has an elliptical marker body or an elliptical marker surface and a geometric center of this marker body or of this marker surface, wherein the marker body or the marker surface is filled with color spectrum points that are distributed radially with respect to the geometric center, wherein a color value of each color spectrum point of the marker/marker surface is determined according to an angle between a horizontal line through the geometric center and a further line through the geometric center and the corresponding color spectrum point.
[0200] According to an exemplary embodiment, the position-based control can be performed in real time.
[0201] According to another exemplary embodiment, the microscopy system 1 comprises an illumination device 20 for illuminating the target.
[0202] According to yet another exemplary embodiment, the illumination device 20 generates light with a wavelength outside the visible range, in particular in the infrared wavelength range.
[0203] According to an exemplary embodiment, the intensity of the illumination and/or a working distance and/or an exposure time of the image capture device 10 is adjustable according to the distance of the target 9 from the image capture device 10.
[0204] According to an exemplary embodiment, the microscopy system 1 comprises at least one means for filtering the beams detected by the image capture device 10, and/or in that the image capture device 10 only detects beams of a defined wavelength or of a wavelength range, in particular of the infrared wavelength range, and/or in that the image capture device 10 is a monochrome image capture device.
[0205] According to another exemplary embodiment, the microscopy system 1 comprises a means for activating a position-based control of the microscopy system.
[0206] According to an exemplary embodiment, the image capture device 10 is a wide-angle camera.
[0207] According to an exemplary embodiment, an identity is assigned to a marker element 22, 22a, . . . , 22d, wherein this identity is identifiable on an image basis.
[0208] According to another exemplary embodiment, an operation of the microscopy system 1 is controllable according to the identified identity.
[0209] According to yet another exemplary embodiment, an open-loop or closed-loop control of the movement of the microscope 2 is effected according to at least one operating parameter of the microscope 2 and/or at least one operating parameter of the image capture device 10 of the position detection device.
[0210] According to an exemplary embodiment, a number and/or a type of the enabled degrees of freedom of the movement of the microscope 2 is adjustable.
[0211] According to an exemplary embodiment, a size of a working space of the microscopy system 1 and/or an admissible maximum speed of the movement is adjustable.
[0212] According to another exemplary embodiment, a graphic user interface is operated according to the position of the target 9.
[0213] According to an exemplary embodiment, a position of the at least one target 9 relative to the at least one image capture device 10 of the position detection device is determined, wherein an operation of the microscopy system 1 is controlled according to the determined position.
[0214] In the foregoing description, the terms “exhibit”, “have”, “comprise” or “include” or any grammatical deviations therefrom are used in a non-exclusive way. Accordingly, these terms can refer either to situations in which, besides the feature introduced by these terms, no further features are present, or to situations in which one or more further features are present. For example, the expression “A exhibits B”, “A has B”, “A comprises B” or “A includes B” may refer both to the situation in which no further element aside from B is provided in A (that is to say to a situation in which A is composed exclusively of B) and to the situation in which, in addition to B, one or more further elements are provided in A, for example element C, elements C and D, or even further elements.
[0215] Furthermore, in the foregoing description the terms “at least one” and “one or more” and grammatical modifications of these terms or similar terms, if they are used in association with one or more elements or features and are intended to express the fact that the element or feature can be provided singly or multiply, in general are used only once, for example when the feature or element is introduced for the first time. When the feature or element is subsequently mentioned again, the corresponding term “at least one” or “one or more” is generally no longer used, without restriction of the possibility that the feature or element can be provided singly or multiply.
[0216] Furthermore, in the foregoing description the terms “preferably”, “in particular”, “by way of example” or similar terms are used in conjunction with optional features, without alternative embodiments thereby being restricted. In this regard, features introduced by these terms are optional features, and there is no intention to restrict the scope of protection of the claims, and in particular of the independent claims, by these features. In this regard, the invention, as will be recognized by the person skilled in the art, can also be carried out using other configurations. Similarly, features introduced by “in one embodiment of the invention” or “in one exemplary embodiment of the invention” are to be understood to be optional features, without this being intended to restrict alternative refinements or the scope of protection of the independent claims. Furthermore, all possibilities of combining the features introduced by these introductory expressions with other features, whether optional or non-optional features, are intended to remain unaffected by said introductory expressions.
[0217] It is understood that the foregoing description is that of the exemplary embodiments of the disclosure and that various changes and modifications may be made thereto without departing from the spirit and scope of the disclosure as defined in the appended claims.
LIST OF REFERENCE NUMERALS
[0218] 1 microscopy system [0219] 2 microscope [0220] 3 stand [0221] 4 rotation axis [0222] 5 rotation axis [0223] 6 rotation axis [0224] 7 control device [0225] 8 user [0226] 9 target [0227] 10 image capture device [0228] 12 signal link and/or data link [0229] 13 patient [0230] 14 operating table [0231] 15 eyepiece [0232] 16 microscope body [0233] 17 optical axis [0234] 18 glass plate [0235] 19 instrument [0236] 20 illumination device [0237] 22 marker element [0238] 22a marker element [0239] 22b marker element [0240] 22c marker element [0241] 22d marker element [0242] 23 support device [0243] S0 initial step [0244] S1 first step [0245] S2 second step [0246] S3 third step [0247] E plane [0248] HB half-space