NAVIGATION, TRACKING AND GUIDING SYSTEM FOR THE POSITIONING OF OPERATORY INSTRUMENTS WITHIN THE BODY OF A PATIENT

Abstract

A navigation, tracking and guiding system for the positioning of operatory instruments inside the body of a patient. The system includes a control unit, a viewer and detecting means for determining the spatial position of the viewer. The system further includes a sensor associated to an operatory instrument and insertable inside the internal portion of the body of the patient. The control unit is configured to project on the viewer an image of the state of the internal portion.

Claims

1.-29. (canceled)

30. A navigation, tracking and guiding system for positioning of operatory instruments within the body of a patient, the system comprising: a control unit configured to receive multiple information data related to an internal state of the body of the patient; a viewer configured for an operator to see at least one internal portion of the body of the patient through said viewer; and special position detecting means for determination of a spatial position of said viewer, said control unit being configured to project on said viewer an image of the internal state of said at least one internal portion of the body of the patient, said image being obtained by processing said multiple information data on the basis of said spatial position of said viewer, the system further comprising a probe associated to an operatory instrument and insertable within the internal portion of the body of the patient, said probe comprising i) at least one optical guide having dispersion zones of a luminous flux generated inside said optical guide and ii) dispersion detecting means of dispersion of said luminous flux in order to identify a spatial arrangement of the probe when inserted within the body of the patient, said control unit being further configured to project on said viewer the image of said probe based on the identified spatial arrangement.

31. The system according to claim 30, wherein said dispersion zones are defined by first dispersion zones arranged in sequence along at least one portion of said optical guide.

32. The system according to claim 31, wherein said dispersion zones are further defined by second dispersion zones arranged in sequence along at least a portion of said optical guide and arranged radially in a staggered configuration with respect to a central axis of said at least one optical guide.

33. The system according to claim 32, wherein said second dispersion zones are disposed at an angle of 90 with respect to said first dispersion zones, said angle being measured with respect to a central axis of said at least one optical guide.

34. The system according to claim 32, wherein said probe comprises two parallel optical guides, said first dispersion zones and said second dispersion zones being defined respectively on one of said two optical guides.

35. The system according to claim 30, wherein said at least one optical guide is connected to a light source, and has a reflecting wall arranged at one free terminal end, wherein a directional coupler connected to an oscilloscope is arranged between said laser source and said reflective wall.

36. The system according to claim 30, further comprising a video conversion assembly connected to the control unit.

37. The system according to claim 30, wherein said multiple information data related to the internal state of the body of the patient are obtained by at least one scan.

38. The system according to claim 30, wherein said viewer is arranged along an operator body portion of a patient body visual axis.

39. The system according to claim 30, wherein said viewer is a facial viewer wearable by the operator.

40. The system according to claim 30, further comprising outer surface detecting means of the outer surface of the body of the patient connected with the control unit.

41. The system according to claim 40, wherein said outer surface detecting means of the outer surface of the body of the patient comprises at least three detectable first physical markers suitable to be arranged on the outer surface of the internal portion of the body of the patient, said control unit being configured to align a first internal state of virtual markers of the image projected on the viewer with said first physical markers.

42. The system according to claim 40, wherein said outer surface detecting means of the outer surface of the body of the patient comprises at least one of: an ultrasound transducer, an inertial measurement unit and a measuring encoder, in order to determine vital parameters of the patient in real time.

43. The system according to claim 30, wherein said viewer comprises a gyroscope, a compass, and an inertial measurement unit.

44. The system according to claim 43, wherein said viewer further comprises a depth sensor configured to record a reconstructed volume of the patient.

45. The system according to claim 40, further comprising a second physical marker disposed on the probe and suitable to be disposed, in use, outside the body of the patient, wherein the outer surface detecting means of the outer surface of the body of the patient is configured to further detect the physical position of the second marker.

46. The system according to claim 30, further comprising an anthropomorphic robot arm with five degrees of freedom, configured to provide Cartesian coordinates X, Y, Z of the probe with respect to a reference system predetermined by the system.

47. The system according to claim 30, further comprising a compactness sensor of the internal tissues of the patient.

48. A navigation, tracking and guiding method for positioning of operatory instruments within the body of a patient, the method comprising the steps of: providing a viewer configured for an operator to see at least an internal portion of the body of the patient through said viewer, providing spatial position detecting means for determination of spatial position of said viewer; providing a control unit to perform the steps of: receiving multiple information data related to the internal state of the body of the patient, processing said multiple information data based on the spatial position of the viewer; and projecting on said viewer an image of the internal state of said at least an internal portion of the body of the patient as a function of the performed intervention, the method further comprising the steps of: providing a probe associated to an operatory instrument and insertable within the internal portion of the body of the patient, said probe comprising i) at least one optical guide having dispersion zones of a luminous flux generated inside said optical guide and ii) dispersion detecting means of dispersion of said luminous flux in order to identify a spatial arrangement of said probe when inserted within the body of the patient, and projecting on said viewer the image of said probe based on the identified spatial arrangement, by the control unit.

49. The method according to claim 48, further comprising the step of obtaining said multiple information data relating to the internal state of the body of a patient by at least one scan.

50. The method according to claim 48, further comprising the step of arranging outer surface detecting means of the outer surface of the body of the patient connected with the control unit.

51. The method according to claim 50, wherein said step of providing outer surface detecting means comprises: providing at least three detectable first physical markers suitable to be arranged on the outer surface of the body portion of the patient; detecting the dynamic positioning of the first physical markers to send multiple information data to the control unit; and aligning the first virtual markers of the image of the internal state projected on the viewer with the first physical markers arranged on the body of the patient through the control unit.

52. The method according to claim 51, wherein said step of providing outer surface detecting means comprises arranging a second physical marker disposed on the probe and suitable to be disposed, in use, outside the body of the patient, and further detecting the physical position of the second marker using the outer surface detecting means of the outer surface of the body of the patient.

53. The method according to claim 48, wherein: said step of processing said multiple information data on the basis of said spatial position of said viewer comprises processing through segmentation of organs and pathologies; a 3D representation is performed with pathology segmented and separated from the rest of the volume as a function of the performed segmentation; and said step of projecting on said viewer an image of the internal state of the at least one internal portion of the body of the patient as a function of the performed processing is performed by projecting a joint visualization of said organs and pathologies.

54. The method according to claim 48, wherein: said step of processing said multiple information data on the basis of said spatial position of said viewer comprises processing through segmentation of organs and post-treatment necrosis; a 3D representation is performed with a pathology segmented and separated from the rest of the volume as a function of the performed segmentation; said step of projecting on said viewer an image of the internal state of the at least one internal portion of the body of the patient as a function of the performed processing is performed by projecting a joint visualization of the pathology and the necrosis.

55. The method according to claim 48, further comprising the step of recording a reconstructed volume of the patient by a depth sensor, thereby allowing the operator to explore organs and pathologies within the body of the patient while looking at the same patient.

56. The method according to claim 48, wherein the method is a computer-operated method.

57. A non-transitory computer-readable medium performing at least the steps of the method of claim 56 when run on a computer.

Description

[0044] Further features and advantages of the present invention will become more apparent from the indicative, and therefore non-limiting description of a preferred but not exclusive embodiment of a navigation, tracking and guiding system for the positioning of operatory instruments within the body of a patient, as illustrated in the accompanying drawing, wherein FIG. 1 is a perspective schematic view of a navigation, tracking and guiding system for the positioning of operatory instruments inside the body of a patient according to the present invention during an operating configuration.

[0045] With reference to the attached FIGURES, 1 generally indicates a navigation, tracking and guiding system for the positioning of operatory instruments within the body of a patient, from now on simply indicated as system 1.

[0046] The system 1 comprises a control unit 2 configured to receive a plurality of information related to the internal state of the body of a patient P.

[0047] Preferably, the plurality of information regarding the internal state of the body of a patient P occurs at least through a scan, for example RX (X-ray), MRI (magnetic resonance imaging), CT (computerized axial tomography), PET-CT (computerized positron emission tomography).

[0048] The scan can be carried out in situ or be pre-loaded into the control unit 2.

[0049] The system 1 comprises a viewer 3 configured in such a way that an operator, not shown, can see at least one internal portion Pi of the body of the patient P through the viewer 3.

[0050] The system also comprises first detecting means 4 for determining the spatial position of the viewer 3.

[0051] The control unit 2 is configured to project on the viewer 3 an image of the internal state of the internal portion Pi of the body of a patient P; in particular, the image is obtained by developing the plurality of information on the basis of the spatial position of the viewer 3.

[0052] In other words, the control unit 2 can project an augmented reality image of the interior of the body of a patient P on the viewer 3 that varies depending on the spatial arrangement of the viewer 3.

[0053] Preferably, the viewer 3 is arranged along an operatorbody portion of the patient P visual axis so as to ensure the best ergonomic conditions for the operator and avoid any coordination problems.

[0054] Preferably, according to a preferred embodiment of the present invention illustrated in FIG. 1, the viewer 3 is defined by a facial viewer wearable by the operator (also called Head Mounted Display, HMD), for example eyeglasses with at least partially transparent lenses.

[0055] Preferably, the viewer 3 comprises a gyroscope, a compass, and an inertial measurement unit; advantageously, these elements allow a correct and precise identification of the spatial position of the viewer 3.

[0056] Preferably, the viewer 3 also comprises a depth sensor adapted to record a reconstructed volume of the patient, which advantageously allows the operator to explore organs and pathologies within the body of the patient P while looking at the same patient P.

[0057] Advantageously, the system 1 according to the present invention further comprises a probe, not illustrated, associated (i.e. inserted internally) to an operatory instrument 5 and insertable within the portion Pi of the body of the patient P.

[0058] The probe comprises at least one optical guide, not shown, having dispersion zones of a luminous flux generated inside the optical guide and detecting means for the detection of the dispersion of the luminous flux, in order to identify the spatial arrangement of the probe when inserted within the patient P.

[0059] The control unit 2 is in fact also configured to project on the display 3 the image of the probe, based on the identified spatial arrangement.

[0060] Thanks to the present invention, therefore, the operator can display on the viewer 3 what would not otherwise be visible to the naked eye. In fact, the control unit 2 transmits on the viewer 3 an augmented reality image that, in addition to showing the internal state of the body of the patient P, also shows the motion of the probe (and therefore of the operatory instrument 5 associated to it) inside the internal portion Pi of the body of the same patient P.

[0061] The actual image of the patient P visible through the viewer 3 is therefore superimposed on a virtual image projected in transparency, showing the organs and the internal tissues of the patient P together with the probe portion inserted within the patient P that otherwise would not be visible.

[0062] Thanks to the present invention, therefore, the operator can operate in absolute safety and precision on the affected area without the need of having to open the body of the patient P to display the operating area and the position/movement of the operatory instruments 5.

[0063] As mentioned above, in order to identify the spatial arrangement of the probe when inserted within the patient P, an optical guide is provided, inside which a luminous flux flows; by modulating and measuring the optical power losses reflected through the dispersion areas, it is possible to determine, and therefore, display on the viewer 3 by means of the control unit 2, the position of the probe within the patient P.

[0064] Therefore, it is possible to check and display in real time the correct handling of surgical instruments 5 that incorporate probes of the type object of the present invention in order to operate accurately and reliably.

[0065] In other words, the optical guide is micro-machined along its central axis in order to introduce reflected optical power losses varying according to the entity and direction of the curvature to which it is subjected.

[0066] The dispersion zones are preferably realized by means of a procedure of micro-machining of the optical guide, which consists of the direct mechanical abrasion of the outer casing of the guide (also called cladding) in order to locally reduce the degree of light confinement in the core.

[0067] The part of the optical guide subjected to the selective removal of the cladding allows light, no longer confined within the core, to escape towards the outside, resulting in a loss of reflected power.

[0068] The optical power loss increases or decreases following the positive or negative curvature of the optical guide, thus power loss appears to be directly proportional to the curvature of the sensitive area (also called core) of the optical guide.

[0069] So, according to the principle of operation of the present system 1, the integration of measurement of the probe bending is provided, with position data originating from the first detecting means 4 for determining the spatial position of the viewer 3.

[0070] According to a possible embodiment of the present invention, the spatial reference system is realized with an articulated arm with five degrees of freedom, which allows to provide Cartesian coordinates X, Y, Z of the probe, with respect to the operatory field reference system.

[0071] Preferably, the dispersion zones are defined by a series of first dispersion zones arranged in sequence along at least one portion of said optical guide.

[0072] Preferably, the dispersion zones are further defined by a series of second dispersion zones arranged in sequence along at least one portion of said optical guide and arranged radially staggered with respect to a central axis of the optical guide.

[0073] Thanks to the staggered configuration of the two series of dispersion zones, it is possible to obtain a precise estimate of the spatial arrangement of the probe.

[0074] Even more preferably, the second dispersion zones are arranged at a 90 angle with respect to the series of first dispersion zones, in which the angle is measured with respect to a central axis of the optical guide.

[0075] Preferably, the probe comprises two parallel optical guides, in which the series of first dispersion zones and the series of second dispersion zones are defined respectively on one of the two optical guides.

[0076] Preferably, the optical guide is connected to a light source, in particular a laser source, not shown, and has a reflecting wall arranged at a free terminal end, wherein between the laser source and the reflective wall a directional coupler is arranged, connected to an oscilloscope.

[0077] Advantageously, closing with reflective material the free end of the optical fibre, it is possible to generate a return of the light in the opposite sense.

[0078] Preferably, the system comprises a video conversion assembly, not shown, connected to the control unit 2.

[0079] Preferably, the video conversion assembly comprises: at least two VGA-HDMI converters, at least one BNC-HDMI converter, and at least two HDMI ports. Still more preferably, the converters are arranged in a single container and the five video signals define the input of a switcher and scaler with five inputs and a single HDMI output. Video signals are picked up by a switcher and sent in HDMI standard to the viewer 3.

[0080] The video signals are sent to the viewer 3 by a mirroring device or by an encoder; alternatively, it is possible to use a local server.

[0081] According to a possible embodiment of the present system, it is expected that the video signals can be selected by the operator by means of a pressure device, for example a pedal.

[0082] The pedal is useful for ergonomics during surgery in which the doctor/operator must look different monitors, both during the operation and during the diagnosis, as well as during the targeting in the field of biopsy.

[0083] Preferably, the system 1 comprises second detecting means 6 of the outer surface of the body of the patient P, not illustrated, connected with the control unit 2 and comprising for example a camera or a stereoscopic camera, preferably integral with the viewer 3.

[0084] Advantageously, the second detecting means 6 of the outer surface of the body of the patient P allow to record the operation while the operator performs it; also, if installed on the viewer 3, they are especially useful as regards both instructions, because it is possible to obtain directly the clinical point of view, and legal aspects, because they record the entire operation.

[0085] Preferably, the viewer 3 comprises a data transmission and reception unit, not illustrated, preferably via Wi-Fi, connected with the first detecting means for determining the spatial position of the viewer 3 and/or connected with the second detecting means 6 of the outer surface of the body of the patient P.

[0086] To allow the display of an augmented reality picture as faithful and consistent as possible with the actual internal state of the patient P, it is necessary to take into account the vital parameters of the patient P (breathing, heartbeat, etc.). In fact, the radiological scans can only provide a static image of the interior of the patient P.

[0087] For this purpose, it is necessary to identify the variation of the spatial configuration of the outer surface of the body of the patient P that the operator looks through the viewer 3 in order to obtain a correct overlap/projection of the image processed by the control unit 2 on the actual image of the patient P.

[0088] The invention provides to arrange seconds detecting means 6 of the outer surface of the body of the patient P connected with the control unit 2.

[0089] In particular, the invention provides for arranging at least three first physical markers suitable to be arranged on the outer surface of the body portion Pi of the patient P and detectable by the second detecting means 6 themselves; detecting the dynamic positioning of the first physical markers to send a plurality of information to the control unit 2; and by the control unit 2, aligning the first virtual markers of the image of the internal state projected on the viewer 3 with the first physical markers arranged on the body of the patient P.

[0090] In other words, the second detecting means 6 of the outer surface of the body of the patient P comprise at least three first physical markers (preferably electromagnetic or optical), not illustrated in the attached FIGURES, suitable to be arranged on the outer surface of the body portion Pi of the patient P and detectable by the second detecting means 6 themselves. The second detecting means 6 of the outer surface of the body of the patient P detect the dynamic positioning of the first physical markers to send a plurality of information to the control unit 2, which is advantageously configured to align the first virtual markers of the image of the internal state projected on the viewer 3 with the first physical markers arranged on the body of the patient P.

[0091] This makes it possible to generate a precise and good-quality augmented reality image that instantly reflects the actual state of the patient inside, the still images having been corrected with the vital parameters of the patient P.

[0092] The invention provides for arranging a second physical marker disposed on the probe and suitable to be disposed in use outside of the body of the patient P, and also to detect the physical position of the second marker using the second detecting means 6 of the outer surface of the body of the patient P.

[0093] In other words, the system 1 comprises a second physical marker disposed on the probe and suitable to be placed in use outside of the body of the patient P, in which the second detecting means 6 of the outer surface of the body of the patient P are configured to also detect the physical position of the second marker.

[0094] Advantageously, in this way, it is possible to accurately identify the positioning of the probe inside the body of the patient P and view it in the augmented reality projected on the viewer 3.

[0095] It is useful to point out here that the spatial reference may be provided by the fact of anthropomorphic arm employed by the operator, then, in this configuration, the second physical mark can represent an additional piece of information (substantially redundant) but useful to increase the reliability of the system on the position of the probe handle.

[0096] Preferably the second detecting means 6 of the outer surface of the body of a patient comprise at least one of an ultrasound transducer, an inertial measurement unit and a measuring encoder, so as to determine in real time said patient's vital parameters.

[0097] In particular, the measuring encoder is a system composed of at least two arms joined by a joint whose motion is detected by an encoder.

[0098] The ends of the two arms are fixed to the patient's chest and consequently move according to the patient's chest breathing motion. In so doing, they reveal a dynamic pattern of the respiratory cycle. Every moment of the pattern will be matched with the target position detected at that moment, so as to match each phase of the respiratory cycle, determined in this manner, with a position of the nodule.

[0099] Preferably, the system 1 comprises a compactness sensor of the internal tissues of the patient P, preferably a loss-modulation optical fibre interferometric sensor.

[0100] Advantageously, thanks to the compactness of the sensor, it is possible to integrate in the plurality of information detected also a measure of the so-called stiffness, which allows to supply characterization parameters of tissues crossed by the probe.

[0101] The present system 1 can be advantageously used during, before, or after surgery. In fact, the system 1 makes it possible for the operator to show an overlap of internal organs and pathologies in 3D, aligned with the real anatomical structures.

[0102] Moreover, the present system 1 makes it possible to display operatory instruments inside the body of the patient P and represent their deformation through different anatomical structures.

[0103] It can be used both for minimally invasive procedures and for standard surgical interventions. The only difference is that, in minimally invasive procedures, all the reconstructed volume of the interior of the body of the patient P is aligned with the body of the patient P; while in the second case, the alignment is made between a specific part of an organ and the same part of the organ in the reconstructed volume. For example, FIG. 1 shows schematically a real segment of the liver and the reconstructed volume of the same.

[0104] The system of this invention has many advantages.

[0105] The invented solution allows to view subjectively with respect to the operator the images in 2, 3 or 4 dimensions in the exact location where the structure they are referred to are located, with high spatial resolution, and to increase the resolution, accuracy and recognition the correct positioning of the operatory instruments 5.

[0106] In addition, the invented solution allows to detect the position and the bending of the probes and of the deformable instruments.

[0107] The system, unlike the solutions used to solve the same problems, allows greater precision, even without electromagnetic systems, through mechanical systems and/or computer vision, together or separately.

[0108] The system of the present invention can be used for the preparation or the performance of surgical, laparotomy, endoscopy or minimally invasive interventions, percutaneous or transosseous, or during interventions by laparotomy or endoscopy. The system is also valid for performing percutaneous or radiologically-guided diagnostic procedures, such as, by way of example, biopsy or needle aspiration.

[0109] Advantageously, the invention provides that the step of processing the plurality of information on the basis of the spatial position of the viewer 3 comprises processing through segmentation of organs and diseases;

[0110] Preferably, the step of processing includes making 3D renderings of radiological scans.

[0111] The invention also involves the creation of a 3D representation with the pathology segmented and separated from the rest of the volume as a function of the performed segmentations.

[0112] The invention further comprises a step of projecting on the viewer 3 an image of the internal state of the at least one internal portion Pi of the body of the patient P as a function of the performed processing, realized by projecting a joint visualization of organs and pathologies.

[0113] As an alternative, advantageously, the invention provides that the step of processing the plurality of information on the basis of the spatial position of the viewer 3 comprises processing via segmentation of organs and post-treatment necrosis;

[0114] The invention also involves the creation of a 3D representation with the pathology segmented and separated from the rest of the volume as a function of the performed segmentations.

[0115] Preferably, the step of processing includes making 3D renderings of radiological scans.

[0116] The invention further comprises a step of projecting on the viewer 3 an image of the internal state of the at least one internal portion Pi of the body of the patient P as a function of the performed processing, realized by projecting a joint visualization of the pathology and necrosis.

[0117] In other words, in a first of the two alternatives, the invention provides the use of a computer program capable of producing segmentations of organs and pathologies (such as, for example, tumours, etc.) and a 3D rendering of radiological scans. Moreover, the software is also able to compare 3D renderings from different scans. It consists of software codes that include segmentation algorithms.

[0118] The program is, in particular, a web application that can take images from different radiological scans (MRI, CT, PET-CT) and transform them into a 3D representation with the pathology segmented and separated from the rest of the volume (for example, with a different colour). Segmentation is fully automatic, that is without user intervention, and does not need any correction.

[0119] In addition, in the second of the two alternatives, the program also measures if the treatment was successful or not. In fact, using this software, organs and post-treatment necrosis are segmented in the images (CT-PET, MRI and CT scans), and the volume is recorded before the treatment and after the treatment, and a joint display of the pathology and the necrosis is performed.

[0120] The program expects to receive an image processing request of an internal portion Pi of the body of the patient P and to transmit data representative of the request to the control unit 2.

[0121] In other words, a web client requests the execution of a script, the script is placed into a scheduler that will manage the queue/the order of the script to be run on the server and, once the scheduler will give the go-ahead, the server will process the files with the required script and write the files in the shared storage.

[0122] The program expects to receive the image of a probe as previously defined and to display the image.

[0123] In other words, the web client will find the generated files or the requested layers and will view them on the viewer 3, particularly on the HMD.

[0124] Another embodiment of the invention involves the registration of a reconstructed volume of a patient using a depth sensor. This sensor advantageously allows the operator to explore organs and pathologies within the body of the patient P while watching the same patient P.

[0125] This is the solution of a second computer program, based

[0126] on appropriate means adapted to implement the so-called computer vision.

[0127] This second solution involves the use of depth sensors and of a stereoscopic camera, jointly or separately, in such a way that the two volumes are aligned. Said camera is used both in AR recognition and for the generation of a disparity map which allows to have more information regarding the depth of the scene; for this second reason, the camera must also have the possibility to adjust the interocular distance of the same, in this case in order to operate with different depth ranges (for example a setting to two/three fixed presets, for a long range and a more accurate short range).

[0128] It should be hereby specified that said depth sensor alone would be sufficient to define the depth map of the scene, making it unnecessary to compute a disparity map using a stereoscopic camera, but given that this type of sensors is often susceptible to strong light and infrared ray sources which can interfere with the reading, it is possible and, in some cases, necessary to integrate both technologies to obtain a more precise alignment between reality and augmented reality.

[0129] A navigation, tracking and guiding system/method for the positioning of operatory instruments within the body of a patient in which augmented reality is used as an operator interface, so as to allow the operator to operate on the patient in a precise, reliable, safe and efficient manner, has been described.