METHOD FOR OBSERVING AN OBJECT, NON-TRANSIENT COMPUTER READABLE STORAGE MEDIUM AND A MEDICAL OBSERVATION APPARATUS

Abstract

The invention relates to a method for observing an object (33) using a medical observation apparatus (1), such as a microscope (3), a non-transient computer readable storage medium (95) and a medical observation apparatus (1). Solutions of the art are expensive, bulky and prevents further usage of a microscope (3) is e.g. a robotic arm has a malfunction. The inventive method and medical observation apparatus (1) solves those problems by directing an optical assembly (7) to an object (33) located in a field of view (31), and by keeping the object (33) in focus when the optical assembly (7) is manually shifted, essentially perpendicularly to a viewing axis (17) of the optical assembly (7). The inventive apparatus (1) comprises an optical assembly (7) providing an optical viewing axis (17) and a lens adjustment assembly (29) which is configured to direct the optical assembly (7) to the object (33) in dependence on position data (65).

Claims

1. A method for observing an object (33) using a medical observation apparatus (1), the method comprising the steps of: directing an optical assembly (7) to the object (33) located in a field of view (31), the optical assembly (7) having a viewing axis (17); and automatically keeping the object (33) in the field of view (31) when the optical assembly (7) is manually moved essentially perpendicularly to the viewing axis (17) of the optical assembly (7).

2. The method according to claim 1, wherein the method further comprises the steps of: receiving position data (65) from a position sensor (41); and readjusting the optical assembly (7) based on the position data (65).

3. The method according to claim 1, wherein the method further comprises the steps of: recognizing at least one pattern (93) of the object (33); retrieving position data (65) based on a variation of the at least one pattern (93); and applying the position data (65) for readjusting the optical assembly (7) to keep the object (33) in focus.

4. The method according to claim 3, wherein recognizing the at least one pattern (93) comprises stereoscopic imaging of the object (33).

5. The method according to claim 2, wherein the method further comprises the steps of: reading-out assignment data (90) from an assignment table (89); correlating the position data (65) with correction data (83); and readjusting the optical assembly (7) based on the correlated correction data (83).

6. The method according to claim 2, wherein the method further comprises the steps of: calculating correction data (83) from the position data (65); and readjusting the optical assembly (7) based on the calculated correction data (83).

7. The method according to claim 1, wherein the medical observation apparatus (1) is a microscope (3).

8. A non-transient computer readable storage medium (95) comprising a program for executing the method according to claim 1.

9. A medical observation apparatus (1) for observing an object (33), the apparatus (1) comprising: a housing (5) supporting the optical assembly (7); an optical assembly (7) supported by the housing, the optical assembly (7) having an optical viewing axis (17) and a field of view (31); and a lens adjustment assembly (29) connected to the optical assembly (7); wherein the optical assembly (7) is manually movable essentially perpendicularly to the optical viewing axis (17) from one position (13a) to another position (13b), and wherein the lens adjustment assembly (29) is configured to automatically direct the optical assembly (7) to the object (33) based on position data (65) representative of the position of the manually moved optical assembly (7).

10. The medical observation apparatus (1) according to claim 9, wherein the lens adjustment assembly (29) includes a position sensor (41) for generating the position data (65).

11. The medical observation apparatus (1) according to claim 10, wherein the position data (65) represents position and/or angular orientation of the optical assembly (7).

12. The medical observation apparatus (1) according to claim 10, wherein the position sensor (41) comprises a pattern recognition module (84) for identifying, in input image data (73), at least one structure (94) and an orientation of the structure (94) relative to the optical assembly (7).

13. The medical observation apparatus (1) according to claim 12, wherein the pattern recognition module (84) comprises a stereoscopic imaging module (85).

14. The medical observation apparatus (1) according to claim 9, wherein the lens adjustment assembly (29) includes a storage module (87) storing assignment data (90) which correlate the position data (65) with correction data (83), and wherein a direction of the optical assembly (7) is adjusted based on the correction data (83).

15. The medical observation apparatus (1) according to claim 9, wherein the lens adjustment assembly (29) includes a calculation module (91) configured to compute correction data (83) based on the position data (65), and wherein a direction of the optical assembly (7) is adjusted based on the correction data (83).

16. The medical observation apparatus (1) according to claim 9, wherein the lens adjustment assembly (29) comprises a controller (39) having an input interface (77) for receiving the position data (65), and further having an output interface (79) for providing correction data (83) to a movable readjustment assembly (45a) for readjusting the optical assembly (7) to keep the object in focus.

17. The medical observation apparatus (1) according to claim 9, wherein the medical observation apparatus (1) is a microscope (3).

Description

BRIEF DESCRIPTION OF THE DRAWING VIEWS

[0041] In the figures

[0042] FIG. 1 shows a schematic drawing of the inventive medical observation apparatus and its working principle; and

[0043] FIG. 2 shows a simplified schematic representation of the inventive medical observation apparatus.

DETAILED DESCRIPTION OF THE INVENTION

[0044] FIG. 1 illustrates the working principle of the inventive medical observation apparatus 1, which is embodied as a microscope 3.

[0045] The figure illustrates a manually moveable housing 5 which supports an optical assembly 7 which may consist of or comprise a lens system or objective 9, including devices for beam detection such as mirrors or prisms.

[0046] FIG. 1 further shows two possible movements 11 of the housing 5 and thereby also the optical assembly 7.

[0047] The optical assembly 7 (received within the housing 5) is shown in a first position 13a, a second position 13b and a third position 13c.

[0048] If the first position 13a represents the initial position 15, each of the movements 11 are oriented essentially perpendicularly to a viewing axis 17, wherein according to the position 13a-13c of the optical assembly 7, also the corresponding viewing axis 17 is in the first 13a, the second 13b or the third position 13c.

[0049] As can be seen, the housing 5 is shiftable with respect to a frame (not shown) of the medical observation apparatus 1. In particular, the housing 5 may be guided, e.g. by a parallelogram device (not shown), to be moved only translationally. Preferably, the housing itself is not tiltable, only the optical assembly 7 may be tiltable with respect to the housing 5, and be preferably not shiftable relative to the housing 5 perpendicular to the viewing axis 17.

[0050] The three viewing axes 17 cross each other in a center point 19, wherein this center point 19 defines a virtual sphere 21. A radius 23 of said virtual sphere 21 corresponds to a working distance 25 of the optical assembly 7 in the first position 13a.

[0051] In the second 13b and third position 13c, a second 25b and third working distance 25c is set, respectively. Both working distances 25b and 25c are larger than the working distance 25. A description how a changed working distance 25 will be taken care of by the inventive medical observation apparatus 1 will be given in FIG. 2.

[0052] In order to achieve that the viewing axes 17 are directed to the center point 19 for each position 13a-13c, the optical assembly 7 is tilted.

[0053] In the first position 13a of the optical assembly 7, a first tilt angle 27a corresponds to zero degree, wherein in the second 13b and third position 13c a second 27b and a third tilt angle 27c are measured, respectively.

[0054] The tilt of the optical assembly 7 is performed by a lens adjustment assembly 29 which will be described in FIG. 2.

[0055] The optical assembly 7 defines a field of view 31 which is schematically shown in FIG. 1. In each position 13a-13c a first 31a, a second 31b and a third field of view 31c are defined, respectively. Each field of view 31 extends into the drawing plane but is shown in a view as seen along the corresponding viewing axis 17.

[0056] It can be seen that the field of view 31 rotates around the center point 19, if an object 33 located in the field of view 31 is regarded from different angles. The optical assembly 7 is automatically tilted at each of the position 13a to 13c to maintain the essentially same field of view 31, in particular to be directed to always the same location in the field of view 31. Preferably, this location is at the center of the field of view 31. For effecting the tilting, a drive system (not shown) may be provided. The drive system may comprise a separate drive, such as an electric motor, for each rotational axis, about which the optical assembly 7 may be tilted.

[0057] This is described in more detail by a first reference point 35a, a second reference point 35b and a third reference point 35c which are exemplarily drawn.

[0058] An image 37 obtained for the three different positions 13a-13c is schematically shown, wherein in the first position 13a only the first 35a and third reference point 35c are shown, the Image 37 in the second position 13b shows the first 35a and the second reference point 35b, whereas in the third position 13c, the image 37 does not show the second reference point 35b.

[0059] It is to be noted that the reference points 35a-35c do not correspond to points of a structure used for pattern recognition. They are solely shown for explanation of the different perspective.

[0060] In FIG. 2 a simplified schematic representation of the inventive medical observation apparatus 1 is shown in more detail.

[0061] The housing 5 comprises the before mentioned lens adjustment assembly 29, which is composed of several components. Those components comprise (in the embodiment shown) a controller 39, a position sensor 41, two stereoscopic cameras 43, a rotational stage 45 and an image sensor 47.

[0062] An optical system 49 comprises one or more lenses 51 (only one is shown in FIG. 2), a tunable lens 53, a beamsplitter 55, the image sensor 47 and optical observation means 57.

[0063] An optical path 59 is drawn from the object 33 through the lens 51, through the tunable lens 53 and through a beam path correction assembly 61. The beam path correction assembly 61 assures that the optical path 59 through the beamsplitter 55 is not changed if the optical assembly 7 is rotated by the rotational stage 45. In the upper part of the figure, the optical path 59 is drawn discontinuously for the sake of the size of FIG. 2.

[0064] It is to be noted that the rotational stage 45 solely represents one possibility to tilt the viewing axis 17. Different means which are configured to tilt the viewing axis 17 are conceivable as well.

[0065] The optical assembly 7 is rotated around a center of rotation 63, wherein the arrangement of the lens 51 and the tunable lens 53 with respect to the center of rotation 63 may be different in a different embodiment of the medical observation apparatus 1.

[0066] The position sensor 41 provides position data 65 via a position data interface 42, which data 65 is represented by a rectangular shaped electric signal and which is provided to a position data input port 67 of the controller 39.

[0067] The stereoscopic cameras 43 deliver stereoscopic image data 69 represented by a triangular shaped electric signal, which are provided to the controller 39 via a stereoscopic image data input port 71.

[0068] The image sensor 47 generates image data 73 represented by an electric signal with two spikes, wherein the image data 73 may be input to the controller 39 via a image data input port 75.

[0069] The position data input port 67, the stereoscopic image data input port 71 and the image data input port 75 represent an input interface 77 of the controller 39.

[0070] The controller 39 also has an output interface 79 which is embodied as a correction data output port 81 via which correction data 83 (which is indicated by a sequence of a triangular and rectangular shaped electric signal) may be provided to the rotational stage 45, which may be referred to as movable readjustment assembly 45a.

[0071] The controller 39 further comprises a pattern recognition module 84, a stereoscopic imaging module 85, a storage module 87 in which an assignment table 89 (schematically shown) is stored and a calculation module 91.

[0072] The position recognition module 64 may be configured to identify at least one pattern, such as a blood vessel, in the stereoscopic image data 69, and to track the pattern in the images obtained at the various positions 13a-c.

[0073] The position sensor 41 may comprise the pattern recognition module 84, i.e. in this embodiment the generation of position data 65 by the position sensor 41 is based, at least partly, on pattern recognition.

[0074] The generation of correction data 83 may be based on position data 65 provided by the position sensor 41, wherein (a) the controller 39 reads assignment data 90 from the assignment table 89 from the storage module 87 and correlates the position data 65 with a necessary correction data 83 or (b) the controller 39 provides the position data 65 to the calculation module 91 which subsequently calculates the correction data 83.

[0075] It is also possible that the stereoscopic image data 69 provided by the stereoscopic cameras 43 may be applied to correlate (via the assignment table 89) or calculate (via the calculation module 91) the correction data 83 which is provided to the movable readjustment assembly 45a.

[0076] Furthermore, the correction data 83 may also be retrieved from image data 73 provided by the image sensor 47, wherein the pattern recognition module 84 of the controller 39 is adapted to identify a preferentially three-dimensional pattern 93 of a structure 94 at or in the object 33 and calculates position data 65 from the pattern 93. For example, a pattern 93 that has been identified, manually or automatically, in the initial position 13a, will have different geometry from the other positions 13b, 13c. The amount and shape of distortion of the pattern 93 allows computing the tilt of the viewing axis due to the position change. Position information from the optical assembly such as distance setting and/or focal length allow determining the position of the optical assembly relative to the identified pattern. This allows adjusting the optical assembly without the need of sensors acquiring position data directly from housing elements such as the housing.

[0077] For tilting the optical assembly, a simple control loop may be implemented which drives the tilt of the optical assembly to counteract any relative movement of the at least one identified pattern in subsequent image data. Thus, the identical pattern 93 is simply kept at a constant location within the field of view. Alternatively or additionally, the tilting may be computed by triangulation of the at least one identified pattern.

[0078] The beam path correction assembly 61 is adapted to correct the beam path 59 such that even after a rotation of the optical assembly 7 (see FIG. 1) the optical path 59 is correctly focused on the image sensor 47 and into the optical observation means 57.

[0079] A change of the working distance 25 may move the object 33 out of focus of the optical assembly 7, wherein this misalignment may be compensated by the tunable lens 53, which is configured to alter an effective focal length (not shown) of the optical assembly 7.

[0080] The medical observation apparatus 1 may be controlled by a computer 97 which reads a non-transient computer readable storage medium 95 which comprises a program for executing the inventive method.

REFERENCE NUMERALS

[0081] 1 medical observation apparatus [0082] 2 microscope [0083] 5 housing [0084] 7 optical assembly [0085] 9 objective [0086] 11 movement [0087] 13a first position [0088] 13b second position [0089] 13c third position [0090] 15 initial position [0091] 17 viewing axis [0092] 19 center point [0093] 21 virtual sphere [0094] 23 radius [0095] 25 working distance [0096] 25b second working distance [0097] 25c third working distance [0098] 27a first tilt angle [0099] 27b second tilt angle [0100] 27c third tilt angle [0101] 29 lens adjustment assembly [0102] 31 field of view [0103] 31a first field of view [0104] 31b second field of view [0105] 31c third field of view [0106] 33 object [0107] 35a first reference point [0108] 35b second reference point [0109] 35c third reference point [0110] 39 controller [0111] 41 position sensor [0112] 42 position data interface [0113] 43 stereoscopic camera [0114] 45 rotational stage [0115] 45a moveable readjustment assembly [0116] 47 image sensor [0117] 49 optical system [0118] 51 lens [0119] 53 tunable lens [0120] 55 beam splitter [0121] 57 optical observation means [0122] 59 optical path [0123] 61 beam path correction assembly [0124] 63 center of rotation [0125] 65 position data [0126] 67 position data input port [0127] 69 stereoscopic image data [0128] 71 stereoscopic image data port [0129] 73 image data [0130] 75 image data input port [0131] 77 input interface [0132] 79 output interface [0133] 81 correction data output port [0134] 83 correction data [0135] 84 parallel recognition molecule [0136] 85 stereoscopic imaging module [0137] 87 storage module [0138] 89 assignment table [0139] 90 assignment data [0140] 91 calculation module [0141] 93 pattern [0142] 94 structure [0143] 95 non-transient computer readable storage medium [0144] 97 computer