METHOD AND SYSTEM FOR REPRODUCING AN INSERTION POINT FOR A MEDICAL INSTRUMENT

Abstract

The invention relates to a method for displaying an injection point for a medical instrument. The method comprises the following steps: Providing at least one marker on a surface of an object, with such marker exhibiting the property that it can be recorded both tomographically, in particular fluoroscopically, and also optically; Generating tomographic image data that can be used to reconstruct a fluoroscopic image of the at least one marker, located on the surface of the object, together with the object; Determining the insertion point for the medical instrument on the surface of the object relative to the at least one marker in the coordinate system of the tomographic image data; Generating visual image data that can be used to reconstruct a visual image of the at least one marker, located on the surface of the object, together with the object; Transforming the coordinate of the insertion point in the coordinate system of the tomographic image data into the coordinate system of the visual image data using the relative position of the insertion point to the at least one marker; and Displaying the insertion point for the medical instrument in real time in a view of the object.

Claims

1. A method for displaying an insertion point for a medical instrument, with such method comprising the following steps: Providing at least one marker on a surface of an object, with such marker exhibiting the property that it can be recorded both tomographically, in particular fluoroscopically, and optically; Generating fluoroscopic or/and tomographic image data that can be used to reconstruct a fluoroscopic or/and tomographic image of the at least one marker, located on the surface of the object, together with the object; Determining the insertion point for the medical instrument on the surface of the object relative to the at least one marker in the coordinate system of the fluoroscopic or/and tomographic image data; Generating visual image data that can be used to reconstruct a visual image of the at least one marker, located on the surface of the object, together with the object; Transforming the coordinate of the insertion point in the coordinate system of the fluoroscopic or/and tomographic image data into the coordinate system of the visual image data using the relative position of the insertion point to the at least one marker; and Displaying the insertion point for the medical instrument in real time in a view of the object.

2. The method according to claim 1, wherein the visual image data is generated as three-dimensional visual image data

3. The method according to claim 1, comprising the following steps: Determining an insertion angle and/or a puncture depth for the medical instrument relative to the at least one marker in the coordinate system of the fluoroscopic and/or tomographic image data; Transforming the insertion angle or/and the puncture depth determined in the coordinate system of the fluoroscopic and/or tomographic image data into the coordinate system of the visual image data using a relative orientation of the insertion angle and/or using a relative distance of the puncture depth to the at least one marker; and Displaying the insertion angle and/or the puncture depth for the medical instrument in real time in the view of the object.

4. The method according to claim 1, wherein the visual image data is generated continuously and at least the insertion point determined in the coordinate system of the fluoroscopic or/and tomographic image data is transformed into the coordinate system of the respectively last generated visual image data, and the display of at least the insertion point for the medical instrument in the view of the object is shown in real time.

5. The method according to claim 1, wherein the view of the object is a visual image of the surface that has been reconstructed from the visual image data generated, and the insertion point for the medical instrument is displayed in the visual image.

6. The method according to claim 1, wherein the insertion point for the medical instrument is displayed on a transparent optical display through which the view of the real surface is visible, wherein the insertion point is displayed on the transparent display perspectively correct in relation to the view of the real surface.

7. The method according to claim 1, wherein the insertion point for the medical instrument is displayed as an optical marker on the real surface of the object.

8. The method according to claim 3, wherein the insertion point and the insertion angle and/or the puncture depth are displayed in the form of a digital representation of a virtual tool in real time in the view of the object.

9. The method according to claim 8, comprising the following steps: Optical recording of position and orientation of the medical instrument relative to the at least one marker in the coordinate system of the visual image data generated, Determining whether the recorded position and orientation of the medical instrument corresponds to the position and orientation of the displayed virtual tool, and if this is the case: Signaling that the recorded position and orientation of the medical instrument corresponds to the position and orientation of the displayed virtual tool.

10. The method according to claim 9, comprising the following steps: If the recorded position and orientation of the medical instrument does not correspond to the position and orientation of the displayed virtual tool: Calculating a trajectory between the recorded position and orientation of the medical instrument and the position and orientation of the displayed virtual tool; and Displaying a virtual directional indication in real time in the view of the object, wherein the directional indication preferably shows the direction in which the medical instrument has to be moved in order to achieve alignment of the position and orientation of the medical instrument with the position and orientation of the virtual tool displayed in the view of the object.

11. The method according to claim 8, comprising the following steps: Aligning the digital representation of the virtual tool in real time relative to the at least one marker in relation to a recording axis, along which the visual image data is generated.

12. A medical system for displaying an insertion point for a medical instrument, with such system comprising the following: A marker that is designed in such a way that the marker can be recorded both tomographically, in particular fluoroscopically, and also optically; An imaging modality for generating fluoroscopic or/and tomographic image data; A camera for generating visual image data; A computing unit that is designed to Determine the insertion point for the medical instrument on the surface of the object relative to the at least one marker in the coordinate system of the fluoroscopic or/and tomographic image data; Transform the coordinate of the insertion point in the coordinate system of the fluoroscopic or/and tomographic image data into the coordinate system of the visual image data using the relative position of the insertion point to the at least one marker; and A display unit for displaying the insertion point for the medical instrument in real time in a real or reconstructed view of the object.

13. The medical system according to claim 12, wherein the camera is a light field camera, a stereo camera, a triangulation system, or a TOF camera.

14. The medical system according to claim 12, wherein the display unit is an optical display that is operatively connected to the computing unit and on which the insertion point for the medical instrument can be visualized by means of the computing unit.

15. The medical system according to claim 12, wherein the display unit is a video projector that is autocalibrated with the camera and designed to display the insertion point for the medical instrument on the real surface of the object as an optical marker.

16. The medical system according to claim 12, wherein the at least one marker is created with adhesive tape that can be adhesively attached on the surface of the object.

17. The medical system according to claim 16, wherein the adhesive tape contains BaSO.sub.x so that the adhesive tape can be detected fluoroscopically.

18. The medical system according to claim 12, wherein the at least one marker contains at least one fluoroscopically detectable element and/or at least one optically detectable element.

19. The medical system according to claim 18, wherein the fluoroscopically detectable element can be made up of a metal and is designed in such a way that it can be identified as a tomographically detectable element in a tomographic image.

20. The medical system according to claim 18, wherein the optically detectable element can be a light emitting diode that is designed to emit electromagnetic radiation in a defined wavelength range.

21. The medical system according to claim 20, wherein the defined wavelength range comprises infrared radiation and the camera features an infrared sensor for detecting the infrared radiation emitted by the light emitting diode.

22. A computer program that is designed to determine an insertion point for a medical instrument on a surface of an object relative to a marker in the coordinate system of generated tomographic image data and to transform the coordinate of the insertion point in the coordinate system of the fluoroscopic or/and tomographic image data into the coordinate system of generated visual image data using a relative position of the insertion point to the marker.

23. A computer-readable storage medium where the computer program according to claim 22 is permanently stored.

Description

[0080] The invention will now be explained in more detail using schematically depicted exemplary embodiments and referencing the figures. The figures show the following:

[0081] FIG. 1: A flow chart of a method for displaying an insertion point for a medical instrument.

[0082] FIG. 2: A schematic diagram of a medical system for displaying an insertion point for a medical instrument.

[0083] FIG. 1 shows a flow chart of a method for displaying an insertion point for a medical instrument.

[0084] The sequence of the method is as follows:

[0085] Initially (step S1), at least one marker is provided on a surface of an object. The properties of the marker allow for tomographic, in particular fluoroscopic, and also optical detection. For example, the marker can be created with adhesive tape that is adhesively attached to the surface in a regular or irregular pattern, thereby creating the marker. In order for the marker to be tomographically detectable, it will preferably have barium sulfate, which is visible in a fluoroscopic image of the marker, distributed across the surface or in selected areas of the adhesive tape. The marker can also be provided on the surface of the object by adhesively attaching double-sided adhesive foil with a precut pattern to the surface. The carrier foil of the double-sided adhesive foil can be peeled off in such a way that only adhesive tape remains in the pre-cut pattern on the surface, thereby creating the marker. The marker is thus created by a predefined pattern of the adhesive foil. For example, a detected deformation of the predefined pattern can indicate a movement of the object. The marker can also be created with a carrier material that has fluoroscopically and optically detectable elements arranged on it. The fluoroscopically detectable elements can be metal balls, and the optically detectable elements can be light emitting diodes, for example. The fluoroscopically and optically detectable elements are preferably arranged in a known spatial relationship to one another.

[0086] Subsequently (step S2), tomographic image data is generated that can be used to reconstruct a fluoroscopic image of the at least one marker, arranged on the surface of the object, together with the object. The tomographic image data can be generated with an X-ray device, for example, which features an X-ray source and an X-ray detector. To generate the tomographic image data, the object is arranged between the X-ray source and the X-ray detector in such a way that the X-rays emitted by the X-ray source penetrate the marker and at least that partial area of the object where a target area to be punctured is located, and are then detected by the X-ray detector.

[0087] Subsequently (step S3), the insertion point for the medical instrument on the surface of the object is determined relative to the at least one marker provided on the surface in the coordinate system of the tomographic image data. For example, the insertion point for the medical instrument can initially be determined in a fluoroscopic image reconstructed from the tomographic image data, e.g. implemented by a doctor or using software. A computing unit can then mathematically determine the coordinate of the specified insertion point in the coordinate system of the tomographic image data. Since the position of the marker in the coordinate system of the tomographic image data is known, the spatial relationship between the marker and the insertion point in the coordinate system of the tomographic image data can be determined. In particular, the relative position of the insertion point to the marker in the coordinate system of the tomographic image data is then known.

[0088] In addition to the insertion point, the insertion angle and/or puncture depth for the medical instrument relative to the at least one marker in the coordinate system of the tomographic image data can also be determined. It is then known in the coordinate system of the tomographic image data where, at what angle and how deep the medical instrument should be inserted into the object for a puncture.

[0089] Subsequently (step S4), visual image data is generated that can be used to reconstruct a visual image of the at least one marker, arranged on the surface of the object, together with the object. The visual image data is generated using a camera that is preferably designed to continuously create visual image data as three-dimensional visual image data.

[0090] Subsequently (step S5), the coordinate of the insertion point in the coordinate system of the tomographic image data is transformed into the coordinate system of the visual image data using the relative position to the at least one marker. If the insertion angle and/or the puncture depth were also determined in the coordinate system of the tomographic image data, transformation of the insertion angle, using the relative orientation of the insertion angle to the at least one marker, and/or of the puncture depth, using the relative distance of the puncture depth to the at least one marker, into the coordinate system of the visual image data is also performed.

[0091] Subsequently (step S6), the insertion point for the medical instrument and—if identified—also the insertion angle and/or the puncture depth are displayed in a view of the object. If available, the insertion point, insertion angle and puncture depth for the medical instrument are preferably displayed together in the view of the object.

[0092] The view of the object can be a real view or a reconstructed view. A real view can be a direct view of the real surface or an indirect view of the real surface, for example through a transparent optical display. In a direct view of the object, the insertion point for the medical instrument can be displayed by means of an optical marker, for example. In an indirect view of the object through a transparent optical display, the insertion point for the medical instrument can be displayed in such a way that it is displayed in real time and perspectively correct in relation to the surface. Furthermore, in the indirect view of the object, the insertion angle and puncture depth can also be displayed in real time and perspectively correct in relation to the surface. It is possible to display the insertion point, insertion angle and puncture depth in the form of a digital representation of a virtual tool in real time in the indirect view of the object. A reconstructed view of the object can be a photograph reconstructed from generated visual image data, in particular a real-time image recording of the object. The reconstructed view of the object can, for example, be displayed on a monitor, e.g. a computer monitor or the monitor of VR glasses. The reconstructed view can display the insertion point, insertion angle and puncture depth, e.g. in the form of a digital representation of a virtual tool.

[0093] It is possible to only display the insertion point in a single view. It is also possible to display the insertion point, insertion angle and puncture depth together in one view. It is also possible to display the insertion point, insertion angle and puncture depth in one view and, in an additional view, only the insertion point. In this case, the insertion point is displayed in two different views, i.e. redundantly. For example, it is possible to display the insertion point, insertion angle and puncture depth in the form of a digital representation of a virtual tool in an indirect view of the object and, in addition, the insertion point by means of an optical marker in a direct view. A user can then choose between the two views, for example. An optical marker can also be integrated into an indirect view of the object.

[0094] FIG. 2 shows a schematic diagram of a medical system 200 for displaying an insertion point for a medical instrument (not shown).

[0095] The medical system 200 comprises an X-ray device 204, a camera 206, a computing unit 208, a marker 210, and two display units 214a, 214b. The medical system 200 is in particular suitable for implementing the method described with reference to FIG. 1.

[0096] The marker 210 can be arranged on an object to be punctured 216 (not part of the medical system 200), for example a patient. The marker 210 is then preferably arranged on the object 116 in such a way that it follows a movement of the object, so that there is no relative movement between the object 216 and the marker 210. Preferably, the marker 210 is adhesively attached to the object 216. For example, the marker 210 can be created with adhesive tape that is adhesively attached to the surface of the object 216 in a regular or irregular pattern. The marker 210 is designed in such a way that it can be recorded both tomographically, in particular fluoroscopically, and also optically. In order for the marker 210 to be recorded optically, it is preferably created in a color and/or form that ensures a visible contrast to the surface of the object 210 in a visual image recording. In order for the marker 210 to be also visible in a tomographic recording, it can have barium sulfate as a contrast medium in defined areas.

[0097] The X-ray device 204 can be a computer tomograph (CT) device, for example, and comprises an X-ray source and an X-ray detector (not shown). In order to generate tomographic image data, the object 216 is arranged between the X-ray source and the X-ray detector of the X-ray device 204 in such a way that the generated tomographic image data can be used to reconstruct a tomographic image of the marker 210 together with the object 216. A reconstructed tomographic image can initially be used to plan a puncture of the object 216, for example to specify an insertion point on the surface of the object 210.

[0098] The computing unit 208 can determine the coordinate of the insertion point in the coordinate system of the tomographic image data 218 relative to the position of the marker 210. The computing unit 208 is also designed to determine the insertion angle and the puncture depth for the medical instrument in the coordinate system of the tomographic image data 218.

[0099] To determine the insertion point, the insertion angle and the puncture depth in the coordinate system of the tomographic image data 218, the computing unit accesses and processes the tomographic image data generated by the X-ray device 204. The computing unit 208 is also operatively connected to the camera 206 to access and process visual image data generated by the camera.

[0100] So that the insertion point 202a, the insertion angle and the puncture depth can be displayed in a view 220 of the surface of the object 216, the computing unit 208 is designed to transform the coordinate of the insertion point determined in the coordinate system of the tomographic image data, as well as the insertion angle and the puncture depth into the coordinate system 222 of the visual image data generated by the camera 206. The computing unit 208 is designed to use the relative position of the insertion point to the at least one marker for transforming the coordinate of the insertion point. The relative position of the insertion point to the at least one marker 210 can be used for transforming the coordinate of the insertion point, because the position of the marker 210 is known in both the coordinate system of the tomographic image data 218 and in the coordinate system of the visual image data 222. The position of the marker 210 can thus be used as a reference for transforming the coordinate of the insertion point from the coordinate system of the tomographic image data 218 into the coordinate system of the visual image data 222.

[0101] The computing unit 208 is further operatively connected to the display unit 214a and designed to display the insertion point 202a for the medical instrument in real time and perspectively correct in a view 220 of the surface.

[0102] The medical system 200 features two display units 214a, 214b. The medical system 200 can also have only one of the two display units 214a, 214b, or an alternative display unit. The display unit 214b is a video projector that is autocalibrated with the camera 206 and designed to display the insertion point 202b as an optical marker.

[0103] The display unit 214a is a transparent optical display that can be mounted, together with the camera 206, on a frame, e.g. an eyeglasses frame.

[0104] The view 220 on the transparent optical display 214a is an indirect view of the real surface of the object 216 that shows the insertion point 202a. The insertion point 202a can be displayed in real time and perspectively correct in the view 220. Instead of the optical display 214a, or in addition to it, the medical system 200 can also feature a monitor that displays the insertion point in a reconstructed view of the object.