CONTROL OF A ENDOVASCULAR ROBOTIC INSTRUMENT

Abstract

A control device (100) for controlling movement of a medical instrument is presented. The control device comprises a limiting indicator providing unit (101) configured to provide an indicator (12) of a limiting position for an instrument (10) beyond which the position of the instrument is considered as out-of-limit, and a position providing unit (102) configured to provide a position (11) of the instrument from a received image including a portion of the instrument and/or retrieved movement of the instrument. The control device further comprises a controller (103) configured to control a movement of the instrument based on the indicator of the limiting position and the position of the instrument such that the position of the instrument maintains a predetermined relation with respect to the limiting position. This allows for an improved movement control for medical and other instruments.

Claims

1. A control device for controlling movement of a medical instrument, wherein the control device comprises: a non-transitory memory that stores instructions; and a processor that executes the instructions, wherein, when executed by the processor, the instructions cause the processor to: provide an indicator of a limiting position for an instrument beyond which position of the instrument is considered as out-of-limit, provide the position of the instrument from an image including at least one of a portion of the instrument and retrieved movement of the instrument, and control a movement of the instrument based on the indicator of the limiting position and the position of the instrument such that the position of the instrument maintains a predetermined relation with respect to the limiting position.

2. The control device as defined in claim 1, wherein the instructions further cause the processor to provide a representation of the indicator in the image so as to graphically represent the limiting position.

3. The control device as defined in claim 2, wherein the instrument is a medical instrument, and wherein the instructions further cause the processor to control a movement of the instrument along a tubular anatomical structure, wherein a portion of the tubular anatomical structure is included in the image, and wherein the representation of the indicator is provided across an imaged tubular structure.

4. The control device as defined in claim 3, wherein the instructions further cause the processor to provide the representation of the indicator with a size that is provided across an imaged tubular structure and dynamically changes with a potential change of width of the imaged tubular structure.

5. The control device as defined in claim 1, wherein the instrument is a first instrument of two instruments, and a second instrument of the two instruments is configured to induce a motion upon the first instrument when the second instrument is moved.

6. The control device as defined in claim 5, wherein the second instrument of the two instruments is manually driven and is configured to induce a motion upon the first instrument, wherein the instructions further cause the processor to control the first instrument so as to automatically maintain the predetermined relation with respect to the limiting position while the second instrument is manually driven.

7. The control device as defined in claim 5, wherein the instructions further cause the processor to: provide a target indicator indicative of a target position for the second instrument to be moved, and provide a second position that comprises a position of the second instrument.

8. The control device as defined in claim 7, wherein the instructions further cause the processor to at least one of: control a movement of the second instrument based on the target indicator and the second position such that the second position approaches the target position; trigger an alert for a user when the second instrument approaches the target position beyond an alert limit; control a movement of the second instrument based on the target indicator and the second position such that the movement of the second instrument slows down when the second position is beyond an approach limit, and control a counteracting force applied to the second instrument or to a tool driving the second instrument, the counteracting force being opposite to a manual force exerted by the user who would manually drive the second instrument, the counteracting force being exerted when the second position is beyond an approach limit, wherein the approach limit is the target position and the counteracting force is sufficiently high to prevent the user to manually drive the second instrument beyond the target position, wherein the counteracting force gradually increases between the approach limit and the target position.

9. The control device as defined in claim 5, wherein the first instrument and the second instrument are medical instruments, wherein the instructions further cause the processor to control a movement of the first instrument and the second instrument along a tubular anatomical structure.

10. The control device of claim 9, wherein the first instrument is an inner device and the second instrument is an outer device, wherein the image includes a portion of the first instrument and the second instruments and a branched intersection of a plurality of branches of the tubular anatomical structure, including a main branch and a target branch which is branched from the main branch, and wherein the instructions further cause the processor to maintain the inner device at a target point in a target branch of the tubular anatomical structure, when the controller causes retraction of the outer device, by further advancing the inner device by an amount compensating for the retraction of the outer device.

11. The control device as defined in claim 1, wherein the instructions further cause the processor to determine a distance between the position of the instrument for which the indicator of the limiting position is provided and the limiting position, and to control the movement of the instrument based on the distance.

12. The control device as defined in claim 11, wherein the position of the instrument for which the indicator of the limiting position is provided from an image of a portion of the instrument, wherein the limiting position refers to a position in the image, and wherein the distance is determined based on the image.

13. The control device as defined in claim 1, wherein the instructions further cause the processor to provide the indicator for the limiting position based on an imaged anatomical feature comprising a new branch where the instrument enters into, if the instrument is moved at least one of along a tubular anatomical structure, and on a user input through a user interface configured to associate a user input with at least one of a location in the image and on stored predefined setting parameters.

14. The control device as defined in claim 13, wherein the instructions further cause the processor to provide the indicator of the limiting position based further on the position of the instrument.

15. The control device as defined in claim 1, wherein the instructions further cause the processor to detect when the position of the instrument to be moved does not change although the instrument is controlled to move beyond a threshold distance, and to modify the movement of the instrument or to trigger an alert to a user.

16. An apparatus for moving a medical instrument, the apparatus comprising: the control device as defined in claim 1, a drive controllable by the processor of the control device to drive the movement of the instrument to be moved.

17. The apparatus as defined in claim 16, further comprising the instrument which is a first instrument of two instruments, and a second instrument of the two instruments, wherein at least one of: a) the first instrument refers to a flexible elongate guiding instrument and the second instrument refers to an instrument to be guided along the first instrument, and b) the second instrument refers to a flexible elongate guiding instrument and the first instrument refers to an instrument to be guided along the first instrument.

18. A method for controlling movement of a medical instrument, the method comprising: providing an indicator of a limiting position for an instrument beyond which position of the instrument is considered as out-of-limit, providing the position of the instrument from an image including at least one of a portion of the instrument and retrieved movement of the instrument, and controlling a movement of the instrument based on the indicator of the limiting position and the position of the instrument such that the position of the instrument maintains a predetermined relation with respect to the limiting position.

19. A non-transitory computer-readable medium having stored a computer program which comprises instructions which, when executed by a processor, causes the processor to: provide an indicator of a limiting position for an instrument beyond which position of the instrument is considered as out-of-limit, provide the position of the instrument from an image including at least one of a portion of the instrument and retrieved movement of the instrument, and control a movement of the instrument based on the indicator of the limiting position and the position of the instrument such that the position of the instrument maintains a predetermined relation with respect to the limiting position.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In the following drawings:

[0077] FIG. 1 shows schematically and exemplarily a control device for controlling movement of a medical instrument,

[0078] FIG. 2 shows, in its four parts given by FIG. 2A to FIG. 2D, schematically and exemplarily different stages during a catheter exchange,

[0079] FIG. 3 shows schematically and exemplarily an apparatus for moving a medical instrument,

[0080] FIG. 4 shows schematically and exemplarily a display including a user interface for controlling movement of a medical instrument, and

[0081] FIG. 5 shows schematically and exemplarily a method for controlling movement of a medical instrument.

DETAILED DESCRIPTION OF EMBODIMENTS

[0082] FIG. 1 shows schematically and exemplarily a control device 100 for controlling movement of a medical instrument. The control device 100 comprises a limiting indicator providing unit 101 configured to provide an indicator 12 of a limiting position for an instrument 10. The indicator 12 can also be referred to as a limiting indicator. Beyond the limiting position, the position of the instrument 10 is considered out-of-limit. Moreover, the control device 100 comprises a position providing unit 102 configured to provide a position 11 of the instrument 10 from a received image including a portion of the instrument and/or retrieved movement of the instrument, and a controller 103 configured to control a movement of the instrument 10 based on the indicator 12 of the limiting position and the position of the instrument such that the position of the instrument maintains a predetermined relation with respect to the limiting position.

[0083] FIG. 2A to FIG. 2D illustrate a control mechanism implemented by the control device according to a particular embodiment in its application to the situation of catheter exchange. In this embodiment, the instrument 10 is a guidewire for a medical catheter. The guidewire is considered a first medical instrument, while the catheter is considered a second medical instrument 20. FIG. 2A shows a state in an endovascular procedure in which a target position inside a vasculature of a patient has been reached by the guidewire and the catheter. In order to reach the target position, the guidewire has first been inserted into the patient and through the vasculature, whereafter the catheter has been inserted by help of the guidewire, namely by sliding along it. As is often the case in endovascular procedures, once the target position has been reached, an exchange of devices is required. In the illustrated case, the catheter needs to be exchanged for a new one, meaning that the present catheter needs to be removed from the body and a new one needs to be inserted over the guidewire. Since the catheter is in physical contact with the guidewire, its retraction, or pullback, along the guidewire can lead to an accidental withdrawal or other induced motions of the guidewire. If the induced motion becomes too large, i.e. if the guidewire is accidentally repositioned too far from its initial position prior to commencement of the exchange process, it may no longer be possible to reach the target position with the new catheter to be installed. It may then be necessary that the guidewire is corrected in position again, which can increase the procedure time beyond an expectable degree and/or lead to other medical complications. Particularly in emergency situation such as strokes, any delay or other procedural complication can have serious health consequences for the patient. In fact, the induced motion can also in itself become dangerous for the patient. For instance, snapping motions of elastic guidewires may be triggered when a distal part thereof is accidentally pulled out of a vessel branch.

[0084] In the illustrated embodiment, the control device 100 further comprises a target indicator providing unit configured to provide a target indicator 22, the target indicator 22 being indicative of a target position for the second instrument 20, which is in this case the catheter, and a second position providing unit configured to provide a second position 21, the second position being a position of the second instrument 20, i.e. the catheter. Moreover, the controller 103 is in this case configured to not only control the movement of the first instrument 10, i.e. in this case the guidewire, but also a movement of the second instrument 20 being the catheter, wherein the movement of the second instrument 20, i.e. the catheter, is controlled based on the target indicator 22 and the second position 21 such that the second position 21 approaches the target position.

[0085] From FIG. 2A to FIG. 2D, the catheter is retracted from an initial position towards a target position, the latter being indicated by an intersection of the target indicator 22 with the line as which the catheter appears in the image. As the relevant position of the catheter based on which the movement is controlled, its tip 21 is assumed. While the tip 21 of the catheter 20 moves from its initial position towards the target position along the guidewire 10, the tip 11 of the guidewire 10 is accidentally withdrawn, which can be considered an induced motion. This is noticed by the controller 103 by virtue of being provided with the guidewire's tip position 11 by the position providing unit 102, which may use artificial neural network-aided image segmentation for this purpose. Moreover, the limiting indicator 12 provided by the limiting indicator providing unit 101 is, in the illustrated embodiment, a line segment 12 similar to the line segment 22 corresponding to the target indicator 22 for the catheter, wherein, however, the line segment corresponding to the limiting indicator 12 is located distal relative to the target indicator 22. The predetermined relation which the position of the guidewire 10 being the first instrument is to maintain with respect to the limiting position indicated by the indicator 12 could in this case be formulated as the condition that a distance between the guidewire's tip position 11 and any position on the line segment 12 should stay larger than zero, preferably larger than zero by a predetermined margin. In this way, as will be appreciated from FIG. 2C and FIG. 2D, the controller 103 can counterbalance or inhibit a withdrawal of the guidewire by the retracted movement of the catheter if it notices that the predetermined relation is at risk of not being maintained, namely if it is determined that the tip of the guidewire is about to cross the line segment corresponding to the indicator 12.

[0086] A guidewire like the one shown in FIG. 2A to FIG. 2D can be understood as an example of a flexible elongate guiding instrument by which a second instrument like the catheter also illustrated by FIG. 2A to FIG. 2D can be guided. The guidance is in this case facilitated by means of physical contact between the guidewire and the catheter, meaning that the catheter slides along the guidewire. Typically, the catheter, which can itself also be understood as a flexible elongate instrument, would be designed in a tube or sleeve shape having a diameter sufficient to allow the catheter to slide over the guidewire in an axial direction while radially surrounding the guidewire. The guidewire may be specifically designed so as to be easily navigable. This allows the catheter, which may by itself be less easily navigable than the guidewire, to be navigated to a treatment site in a patient's body by pushing it along the guidewire. A guidewire could also be regarded as an inner instrument with respect to the catheter, wherein the catheter could be regarded as an outer instrument with respect to the guidewire being the inner instrument.

[0087] It will be understood that, while movements of the catheter can induce motions of the guidewire, also the reverse situation may happen, i.e. that movements of the guidewire induce motions of the catheter. While the former situation may happen, for instance, during retraction of the catheter for catheter exchange, the latter situation may arise, for instance, if the guidewire is further protracted or if it is retracted while the catheter has already been partially or completely slid over the guidewire. Moreover, both of the two instruments may be moved simultaneously, in which case their respective movements induce motions upon each other. Particularly in such a case, indicators for limiting positions and target indicators may be defined for each of the two instruments.

[0088] While a guidewire and a catheter are examples of medical instruments, it will be understood that the control mechanisms described with respect to the illustrated embodiment are equally applicable to other instruments. In particular, the instruments controllable by the controller 103 are not necessarily moved along a tubular anatomical structure in a patient's body, such as along vessels as illustrated by FIG. 2A to FIG. 2D. Instead, the controller 103 may be configured to control a movement of the instruments in other environments as well.

[0089] Movements of the one or more instruments may be controlled by the controller 103 based on their respective position and the respective limiting position indicator and/or target indicator. In the illustrated embodiment, which exemplifies the case that the catheter 20 is being retracted while the guidewire should remain in place, the controller 103 is configured to determine a first distance based on the guidewire's tip position 11 in the image, as provided by the position providing unit 102, and the indicator 12 corresponding to the distal line segment visible in the image and being provided by the limiting indicator providing unit 101, wherein the determined distance is indicative of a distance between the position 11 of the tip of the guidewire 10 and the limiting position for the guidewire 10. The latter is given in this case by the axial position along the vessel branch on which the line segment 12 appears in the image.

[0090] In principle, many distance measures could be chosen for the distance determination. For instance, the shortest two-dimensional Euclidean distance between the tip of the guidewire as it appears in the image and points on the line segment 12, or an intersection thereof with the vessel structure, could be considered. It may however be preferred that an anatomical distance measure is applied, such that the controller 103 determines an anatomical distance between the position 11 of the tip of the guidewire and the position of the indicator 12. An anatomical distance may be understood as a distance along an anatomical structure in which the one or more instruments are being moved, such as a vessel structure in the illustrated embodiment. Hence, an anatomical distance determination may need to be further based on the respective anatomical structure. Anatomical distances may more accurately reflect the physically possible pathways for the instruments in the anatomical structure. For instance, in the case of very curved vessels, the tip of a guidewire may be located around the corner as seen from a limiting indicator like the line segment 12, such that a Euclidean distance would correspond to a pathway of the catheter tip cutting the corner, i.e. leaving the vessel.

[0091] According to one particular anatomical distance measure applicable in the illustrated embodiment, an arc length of the curve as which the guidewire 10 appears in the image would be taken as a basis for controlling the movement of the guidewire 10, wherein the arc length would be measured between the guidewire's tip position 11 and the point of intersection between the curve as which the guidewire 10 appears in the image and the line segment 12 corresponding to the indicator of the limiting position for the guidewire 10. Hence, in this particular case, the only position indicated by the indicator of the limiting position and entering the movement control would be the position, i.e. the location, of the point of intersection. It should be realized that this particular distance measure would yield results that can come the closer to a Euclidean distance measure the straighter and/or shorter the part of the guidewire extending beyond the line segment 12 is or becomes. Whether anatomical or not, applicable distance measures include three-dimensional distance measures. Three-dimensional distances can be determined, for instance, based on three-dimensional image data.

[0092] In the illustrated embodiment, in order to retract the catheter 20, the controller 103 is configured to determine a second distance based on the position of the catheter's tip 21 as provided by the second position providing unit based on the image, and the target indicator 22 corresponding to the larger, proximal line segment seen in FIG. 2B to FIG. 2D. The second distance can be determined based on the same or a different distance measure as compared to the first distance, i.e. when compared to the catheter and the line segment 22. The second distance is used as a basis for controlling the movement of the catheter 20.

[0093] Images like those shown in FIG. 2A to FIG. 2D can be acquired by an imaging modality and thereafter provided in terms of image data to the control device 100, particularly the controller 103. For this purpose, the control device 100 may comprise an image data providing unit configured to provide the image data. One or more images may be acquired, and thereafter provided in terms of image data by the image data providing unit, of only the instrument to be moved by the controller 103, or also of one or more further instruments whose movements may induce motion upon the instrument to be moved. In the illustrated embodiment, both instruments, i.e. the guidewire 10 and the catheter 20, are imaged in a single, common image. In this case, spatially two-dimensional X-ray images are collected over time, wherein FIG. 2A to FIG. 2D each show one instant in time. Apart from X-ray, different imaging modalities may be used, such as MRI or ultrasound, for instance.

[0094] For any instrument being imaged, the instrument's position may be determined at least in part based on the image, i.e. the image data corresponding to the image. Concretely, for instance, the position of the tips of the guidewire and the catheter in the images shown in FIG. 2A to FIG. 2D may be determined to just correspond to the tips' pixel positions in the images. The positioning of the instruments may be carried out by the controller 103 also based on non-image data, such as past control data from which past movements can be retrieved. However, an exclusively image-based positioning of the instruments may, as in the illustrated embodiment, often be already accurate enough for a reliable movement control.

[0095] As explained above, in FIG. 2A to FIG. 2D, the first and the second distance, i.e. the distances based on which the movement of the guidewire 10 and the catheter 20 are controlled, respectively, could be approximately equal to the length of the portion of the respective instrument that extends, as viewed from a proximal to a distal side, beyond the respective line segment 12, 22. These lengths, which could be understood as anatomical distances since they naturally lead along the vessel structure, can be measured based on the image data by, for instance, integration along the respective instrument as visible in the image. Such an integration is hence naturally also carried out along the vessel structure, whereas in a formula for these lengths no explicit reference any anatomical structure would be needed. If, for instance, the guidewire 10 is represented internally as a set of consecutive segments, the arc length based on which the distance used for controlling its movement can be measured in terms of the sum of the lengths of the segments extending distally beyond the limiting indicator 12. Should the guidewire 10 have been withdrawn so far that it no longer intersects the indicator 12, then the closest distance between its tip position and any point on the indicator 12 could be used for controlling the guidewire such that its tip moves back past the indicator 12 again. Hence, the distance measure used to determine the distance based on which the respective instrument is controlled may depend on the instrument's position relative to its respective limiting positing and/or target position.

[0096] The indicator 12 is provided by the limiting indicator providing unit 101 based on anatomical information. The anatomical information may again be derived from image data, particularly the image data based on which also the position of one or more of the instruments has been determined. As illustrated by FIG. 2A to FIG. 2D, for instance, the indicator 12 may indicate the limiting position with respect to a part of the anatomical structure in which the first instrument is moved. Moreover, the indicator 12 may be adapted to a geometry of the anatomical structure. For instance, given a preliminary limiting position for a first instrument like the guidewire 10, which may be indicated by a user or correspond to a fixed, predefined position, the indicator 12 may be determined by the limiting indicator providing unit 101 to lie at roughly this preliminary position, but such that it indicates a limiting position for all physically possible pathways of the first instrument passing the rough preliminary position. In the illustrated case of the indicator 12 corresponding to a line segment in an image, for instance, a user may choose a rough position in the image where he/she intends the indicator 12 to be placed, wherein then the limiting indicator providing unit 101 could fix the corresponding line segment to have a location at an extent so as to fully close off the vessel in a region closest to where the user has preliminarily intended the indicator 12 to be.

[0097] Moreover, the limiting indicator providing unit 101 may be configured to provide the indicator 12 based further on the position of the first instrument 10. In this case, a user input may no longer be needed, since the anatomical information and the position of the first instrument 10 may, according to a possibly predefined criterion, uniquely determine where the indicator 12 is to be placed. For instance, as will be appreciated from FIG. 2A and FIG. 2B, by requiring the indicator 12 to correspond to a line segment positioned between a current tip position 11 of the guidewire and the closest branch of the vessel structure proximal to the current tip position 11, and moreover to lie transverse to a longitudinal axis of the vessel segment in which the guidewire tip 11 is positioned, the line segment 12 is substantially fixed.

[0098] Optionally, a length of any line segments corresponding to the indicator 12 and the target indicator 22 can be chosen to comply with a dimension of the anatomical structure at a position of the respective indicator, wherein the dimension is preferably chosen to be measured in a direction transverse to a movement direction of the respective instrument. The line segments may then be displayed as an overlay over the image in which also the one or more instruments are displayed in the anatomical structure. In FIG. 2A to FIG. 2D, for instance, the indicating line segments 12, 22 extend across the respective vessel sections, such that any movement of the respective instrument 10, 20 is likely or almost certain to cross the respective line segment 12, 22 some time. In the case of the line segment 22 corresponding to the target indicator for the catheter 20 during retraction, the diameter of the vessel section proximal to the target indicator's position has been extrapolated to some extent, such that the line segment 22 closes off the imaginary extension of the proximal vessel section in a distal direction. The line segments 12, 22 may also be moved, and/or changed in size, according to temporal changes in the anatomical structure. For instance, the line segments 12, 22 may be moved with the vasculature motion and/or modified to comply in length with a temporally changing width of the vasculature, as causable by blood pulses, for instance.

[0099] The limiting position indicated by the indicator 12 could be regarded as a pinning position, and the indicator 12 itself could be regarded as a virtual anchor that anchors or pins the first instrument, i.e. the guidewire 10 in the illustrated embodiment, to a predefined position, at least if it approaches this position from, for instance, its distal side. As indicated above, the indicator 12 can be determined automatically or semi-automatically, i.e. automatically up to being based on a user input, for instance. While it may be defined relative to anatomical information derived from an image, preferably an image acquired while the instrument is being moved, such as during a medical procedure, the indicator 12 may also be generated from a preoperative model of the anatomical structure in which it is to be moved.

[0100] In some other embodiments, a guidewire may be actively anchored to a location. For example, when the guidewire is in a side branch while the catheter is in the proximal larger branch, the motion of the catheter may begin to retract the guidewire. This presents a challenge in manual manipulation because even a skilled professional needs to manipulate both of the interventional devices in four degrees of freedom simultaneously with only two hands. Using locations of the interventional devices in images, the robotic device may counteract this by advancing the guidewire relative to the catheter such that the guidewire maintains its depth in the side branch. This is accomplished by active guidewire servoing, with the feedback metric being the location of the tip of the guidewire inside the side vessel. When the tip of the guidewire is retracting due to external forces such as catheter motion, the guidewire may be automatically advanced in proportional to the retraction amount from the initial position. This minimizes the movement of guidewire escaping the cannulated side branch. In other embodiments, the guidewire may be further extended into the side branch to provide additional stabilization.

[0101] The controller 103 may also advance the inner device by a distance past the outer device, while keeping the outer device static. The controller 103 may servo-control the robotic device to retract the outer device by a distance relative to the inner device, keeping the inner device static. The controller can also retract the inner device relative to the outer device. The controller 103 may actively advance and retract the inner device relative to the initial position of the tip of the inner device relative to the image. In this way, the controller 103 may anchor the inner device to the vessel/branch. Advancing the inner device past the outer device may be performed when the controller 103 is retracting or rotating the outer device.

[0102] In some other embodiments, a guidewire position in a cannulated vessel may be slipping, and when this is recognized, the movement of the catheter may be stopped automatically and a new maneuver suggested. Additionally, when the catheter is not rotated in the correct orientation to be facing the opposite direction of the cannulated vessel, the incorrect orientation may be detected and communicated to the user along with a suggested correction. Alternatively, the correction may be automatically implemented once the incorrect orientation is detected. As a result, the guidewire may be actively anchored to the vessel by automatically correcting the catheter orientation.

[0103] Furthermore, while the indicator 12 and the target indicator 22 may be graphically represented for display to a user in terms of line segments, they could also be graphically represented differently, such as by circles, broken line segments, rectangles, areas, or volumes, for instance. More complex forms like circles, areas or volumes can be constructed from line segments. The graphical representation of the indicators 12, 22 can correspond to their internal representation in which they are provided as input to the controller 103. The indicators 12, 22 can also comprise textual labels, glyphs and/or animations, some of which may dynamically represent the distance measure, or generally a respective metric, based on which the movement of the first and the second instrument is controlled, i.e. which may be actively considered for robot motion.

[0104] However, the graphical representations chosen for the indicators 12, 22 could also be simplified versions of more complex representations defined only internally by the control device 100. For example, a rectangular area can be used to define a desired region to keep an instrument like the guidewire pinned in, wherein the region be would more accurately defined as being bounded by vessel walls and additionally two lines across the vessel at an axial, i.e. vessel-axial, distance to each other. It is understood that examples like the latter generalize to three-dimensional graphical representations by, for instance, generalizing a rectangle to a cube or a cylinder or a line segment to a square or a disc.

[0105] The controller 103 may be configured to control the movement of the one or more instruments by providing control signals as output, wherein the control signals are interpreted by a drive, which could also be regarded as a motor and could comprise, for instance, a servo system, so as to effect the respective movements. In such a configuration, particularly due to the respective environment in which the one or more instruments are moved, the actual movements effected may not correspond to the intended movements corresponding to the control signals provided as output by the controller 103. Such a mismatch between intended and actually effected movements may be identified in images acquired of the instruments while being moved, and precautionary measures can be taken in order to decrease risks associated with such a mismatch. For instance, if the controller 103 controls an instrument so as to move, but no movement of the instrument is observed in the acquired image, the controller 103 could decrease the speed at which movement of the instrument is to be effected, such as by the drive. This can decrease the risk of spring effects potentially generating unwanted large displacements of the instrument. More generally, a control algorithm carried out by the controller 103 may take into account slacks observed in an instrument to be moved. In other words, the controller 103 may be configured to detect when the position 11 of the guidewire tip does not change even though the guidewire 10 is controlled to move, and slow down any movement of the guidewire 10 in this case. Moreover, the controller 103 may be configured to detect when the position 21 of the catheter tip does not change even though the catheter 20 is controlled to move, and to slow down any movement of the catheter 20 in this case.

[0106] FIG. 3 shows schematically and exemplarily an apparatus 200 for moving one or more medical instruments like the guidewire 10 and the catheter 20 from above. The apparatus 200 comprises the control device 100 and a drive 210 controllable by the controller 103 of the control device 100 to drive the movement of the one or more medical instruments. As illustrated, the apparatus 200 further comprises an imaging device 220 that is configured to capture images of the one or more instruments. The apparatus 200 also comprises a display device 230 configured to display the one or more instruments. In particular, the images may be acquired by the imaging device 220 during an ongoing medical procedure carried out using the one or more instruments 10, 20, wherein the images may be displayed on the display 230 to a user of the one or more instruments. Based on the displayed images and possibly further graphical elements on the display 230 as determined by the control device 100, the user may provide a user input, such as via a user interface of the control device, which may correspond to the display 230 itself, wherein a control of the instruments, mediated by the drive 210, may be carried out based on the user input. In the example of FIG. 3, the medical procedure could be, for instance, an endovascular procedure on a patient 30.

[0107] FIG. 4 shows schematically and exemplarily a display screen displayable on the display device 230 of the apparatus 200 illustrated by FIG. 3. In the left part of the display screen, an image of a guidewire 10 and a catheter 20 representing two medical instruments inside, i.e. inserted in, a tubular test structure can be seen, wherein a line segment corresponding to an indicator 12 of a limiting position has been placed in a left arm of the test structure for the guidewire 10. The image shows a state in which, for test purposes, the catheter 20 has been rotated while being retracted in order to perturb the guidewire 10. Due to the controller 103 controlling the movement of the guidewire such that the guidewire maintains a predetermined relation with respect to the limiting position indicated by the line segment 12, thereby virtually anchoring the guidewire tip, the attempt of perturbing the guidewire is not successful.

[0108] In the right part of the display screen shown in FIG. 4, a user interface is presented via which a user may provide control inputs for the guidewire 10 and the catheter. For each of the two instruments, the user may demand the respective instrument to be moved forward or backward, i.e. to be protracted or retracted with one of two predefined speeds (single arrow or double arrow), and to rotate around its longitudinal axis either clockwise or anticlockwise, again with one of two predetermined speeds (single arrow or double arrow). For each of the instruments, a distance indicative of how far they are inserted into the tubular structure and an angle indicative of their rotation state is displayed. Moving the instruments by use of the arrow buttons so as to explicitly effect particular translational and/or rotational movements could be regarded as a semi-automatic control. In contrast, by pressing the Auto buttons shown in FIG. 4, a user can initiate an otherwise fully automatic movement of the respective instrument, wherein the initiated movement can correspond to a movement of the respective instrument to a previously indicated target position, particularly while limiting a movement of the respective other instrument to an indicated limiting position. For safety purposes, the user has to press the respective Auto button continuously to keep the movement going. In its upper part, the user interface shown in FIG. 4 comprises a button for linking a movement of the two instruments to be controlled (indicated by two interlocked chain elements). If this button is clicked, clicking either of the Auto buttons triggers an automatic movement of both instruments.

[0109] FIG. 5 shows schematically and exemplarily a method 300 for controlling movement of a medical instrument, wherein the method includes, in a step 301, providing an indicator 12 of a limiting position for an instrument 10 beyond which the position of the instrument 10 is preferably considered as out-of-limit. In a step 302 of the method 300, a position 11 of the instrument 10 is provided from a received image including a portion of the instrument and/or retrieved movement of the instrument. Furthermore, the method 303 includes a step 303 of controlling a movement of the instrument 10 based on the indicator 12 of the limiting position and the position 11 of the instrument 10 such that the position 11 of the instrument 10 maintains a predetermined relation with respect to the limiting position.

[0110] Navigating endovascular instruments under fluoroscopic guidance is inherently challenging and any delays and complication can have serious health consequences for the patients in emergency situations such as stroke. Once a target location inside the vasculature is reached, an exchange of instruments is often required. During the exchange process (e.g., removing a current catheter and installing a new one over a guidewire), one or mode instruments can be accidentally dislodged (guidewire is retracted accidentally, for instance) from the target position, requiring a repeat of the navigation process and thus leading to increase in procedure time and/or complications. It has been found, inter alia, that robotically controlled instruments and image interpretation can be leveraged in a closed-loop controller to provide a reliable and automatic means to move or exchange instruments while keeping one or more of them in a desired target position inside vessels, effectively pinning the instruments to the vessel location.

[0111] In a preferred embodiment, active stabilization of the tip of a instrument in a desired target position inside a vessel can be achieved, even if another instrument is retracted, thanks to the following system (device or apparatus) for controlling coaxial elongated instruments, the system comprising a) an imaging unit arranged to provide a 2D X-ray image including imaged coaxial elongated instruments, wherein the coaxial movable elongated instruments include an inner instruments (guidewire) and an outer instruments (catheter), b) a unit for providing a limitation, configured to provide a line (limitation line) overlaid on and crossing the imaged inner instrument (or the imaged outer instrument) such that the tip of the inner instrument (or of the outer instrument) is beyond the limitation linethis may also be referred to as a virtual anchoring, c) a unit for image analysis, configured to identify, in the image, and during the motion of the inner and/or the outer instrument, whether the tip of the inner instrument (or of the outer instrument) is crossing or is approaching the limitation line, and to provide a corresponding output information, and d) a (robotic) controller arranged to drive the inner instrument (or of the outer instrument) and configured to adapt the motion of the inner instrument (or of the outer instrument) based on the output information, such that the tip of the inner instrument (or of the outer instrument) stays as much as possible beyond the limitation line. As described above in more detail, the limitation line can be adapted in length to and overlaid across a width of an imaged vasculature, and be moved with a vasculature motion detected in the image. Moreover, the limitation line can be provided through a user interface, or automatically by the system based on anatomical features (e.g. aperture of a vessel).

[0112] Optionally, the limitation providing unit is configured to provide a second limitation line (overlaid on and crossing the imaged outer instrument (or the imaged inner instrument) such that the tip of the outer instrument (or of the inner instrument) is beyond the limitation line, and the image analyzing unit is configured to identify, in the image, and during the motion of the inner and/or the outer instrument, whether the tip of the outer instrument (or of the inner instrument) is crossing or is approaching the second limitation line, and to provide a corresponding second output information. The controller may then be arranged to drive the outer instrument (or of the inner instrument) and to adapt the motion of the outer instrument (or of the inner instrument) based on the second output information, such that the tip of the outer instrument (or of the inner instrument) stays as much as possible beyond the limitation line. This might be particularly useful to require retracting movement of the catheter to the second limitation line (robotically controlled) while keeping the guidewire at the first limitation line. Particularly the limitation line for the instrument being retracted can also be regarded as a target line, since a retraction up to this line may be desired.

[0113] In a particular example, the system could be a robotic system for instrument navigation in neurovascular procedures. A particular target location could in that case be, for instance, the left vertebral artery of a patient. A controller of the system, which could also be regarded as a control device, might, in an embodiment, follow a control flow as follows: A.) The user selects catheter exchange on the user interface and an X-ray image is taken, B.) the system automatically creates target locations in the X-ray image from the device tracking informationone for catheter (distal to location where catheter exits image) and one for guidewire (at the current guidewire tip location, perpendicular to distal section), represented by line segments. The user can adjust these through a GUI. The user presses a button or display region representable by X-ray/robot enable and the robot automatically servoes the two devices independently to their respective target with live fluoroscopy for feedback control, C.) the robot continues to servo the catheter by retracting the catheter to the catheter's target, and D.) the catheter retraction completes. During the retraction of the catheter, the guidewire tip may be retracted to its own target. Over the whole or parts of the workflow, an incorporation of respective visual cues on a corresponding fluoroscopy image allows a physician to clearly monitor the location of the guidewire tip inside the catheter.

[0114] If image-based, the feedback control method to move the instruments can use Euclidean geometry in the image space. The instruments in the images can, for instance, be segmented and represented as ordered line segments, just like the respective limitation or target lines. The control objective can then be to move one of the instruments as close as possible to a target line. The target line could correspond to the distal end of the other instrument (tip). However, any portion of the instrument can be used to calculate, for instance, the 2D Euclidean distance between the closest point on the target line to the closest point on the instrument to be moved. This distance can then be used to protract the instrument to reach the line. In the case when the instrument crosses the target line segment, the length of the instrument past the intersection is preferably used as the metric to retract the instrument. A dead-zone tolerance can be used to minimize dithering about the target line.

[0115] For the case of guidewire stabilization, the distances to the target can be relatively small, thus servo motions will be relatively small, such that the instrument appears static or pinned. But for the use case of catheter retraction, the motions will likely be fast to reach the target, since the target line is relatively far from the tip of the catheter.

[0116] In a preferred embodiment, the controller, or a controlled robot, pins the guidewire while it also retracts the catheter to be exchanged. However, alternative combinations are also possible: e.g., catheter pinned, guidewire retracted, or other combination when three or more instruments are involved. While a fully automatic control over all instruments is possible, also a user can be included in the control loop. For instance, a user may manually control one or more instruments while the robot is controlling the other. In an example, once the guidewire target(s) are defined, the user is able retract the catheter (manually or robotically) while the robot is actively pinning the guidewire to the target location in the image. In case of a robotically controlled catheter, the degrees of freedom of the catheter can be limited to retraction or rotation only, or both could be allowed. In the case where the guidewire is rapidly moving away from the target location, the velocities of the catheter controller could be lowered to improve target stabilization, of the guidewire. An extension of this considers the type of motion that is applied by the user and compensates for potential big changes in the length/position of the guidewire, for example fast rotations of a curved catheter.

[0117] Although the above embodiments mainly related to medical applications, it will be understood that they generalize to the control of non-medical instruments, particularly in environments different from tubular structures like vasculature in a patient.

[0118] Moreover, while the above embodiments mainly referred to cases of one or two instruments being controlled, any other number of instruments could be controlled. That is to say, indicators of a limiting positions and/or a target position could be provided for a plurality of instruments, wherein the controller of the instruments may be configured to control the plurality of instruments based on the indicators. In particular, motions induced upon more than one instrument by an environment and/or more than one other instrument could be balanced or inhibited by providing corresponding indicators, which could be referred to as induced motion limiting indicators, for more than one or two instrument. Along with the induced motion limiting indicators, target indicators could be provided for the plurality of instruments, such that the plurality of instruments could also be controlled to move towards respective target positions.

[0119] In fact, the control of the one or more instruments is also not limited to being position-based, particularly not to being only position-based. Additionally or alternatively, for instance, velocities of the instruments and/or forces on the instruments may be taken into account. Therefore, in particular, an indicator of a limiting position provided for a respective instrument could indicate the limiting position for this instrument also indirectly via other quantities. For instance, the limiting position may be indirectly defined as the position in which the instrument has a predefined limiting velocity or in which a predefined limiting force acts upon it. Likewise, the target indicator for a respective instrument may indicate the target position for this instrument only indirectly via other quantities like the instrument's velocity or a force on the instrument, such that the target position could be indirectly defined as the position in which the instrument has a predefined target velocity or in which a predefined target force acts upon it. Correspondingly, additionally or alternatively to a position providing unit for providing the position of the respective instrument, a respective velocity and/or force providing unit may be included in the control device. The predetermined relation to be maintained by the respective instrument being controlled could then also be indirectly defined, namely via the other quantities, such as in terms of a limiting force or velocity, or a target force or velocity, et cetera.

[0120] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

[0121] In the claims, the word comprising does not exclude other elements or steps, and the indefinite article a or an does not exclude a plurality.

[0122] A single unit or device may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0123] As used herein, a providing unit can be a receiving unit configured to receive a respective quantity from another unit or device, and to provide the received quantity. However, a providing unit can also be a storage in which a previously measured or otherwise acquired quantity has been stored and from which this quantity can be retrieved for providing the same. A providing unit also can be or comprise the unit or device which has previously measured or otherwise received the quantity. Before providing a previously received or stored quantity, the quantity may be processed by the respective providing unit.

[0124] Procedures like the providing of indicators and positions, the control of movements, et cetera, performed by one or several units or devices, can be performed by any other number of units or devices. These procedures can be implemented as program code means of a computer program and/or as dedicated hardware.

[0125] A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0126] Any reference signs in the claims should not be construed as limiting the scope.

[0127] A control device for controlling movement of a medical instrument is presented. The control device comprises a limiting indicator providing unit configured to provide an indicator of a limiting position for an instrument beyond which the position of the instrument is considered as out-of-limit, and a position providing unit configured to provide a position of the instrument from a received image including a portion of the instrument and/or retrieved movement of the instrument. The control device further comprises a controller configured to control a movement of the instrument based on the indicator of the limiting position and the position of the instrument such that the position of the instrument maintains a predetermined relation with respect to the limiting position. This allows for an improved movement control for medical and other instruments.

[0128] One and/or other of the units as herein used (e.g. limiting indicator providing unit, position providing unit, target indicator providing unit, second position providing unit) and/or controller may be hardware and/or software-based components. In particular these units and/or controller may be found as codes of computer program, or instructions, stored in a non-transitory memory, and arranged to be executed by a processor causing the control device to implement the control according to the different embodiments of the invention.

[0129] The control device may be a computer system or computer device or computer unit including at least one memory, where, if a plurality of memories, memories in the computer system communicate with each other and the processor via a bus. Either or a plurality of said memories may be considered representative examples of the memory of the control device, and store instructions used to implement some or all aspects of methods and processes described herein. Memory(ies) described herein and according to the invention is (are) tangible storage medium(s) for storing data and executable software instructions and is (are) non-transitory during the time software instructions are stored therein. As used herein, the term non-transitory is to be interpreted not as an eternal characteristic of a state, but as a characteristic of a state that will last for a period. The term non-transitory specifically disavows fleeting characteristics such as characteristics of a carrier wave or signal or other forms that exist only transitorily in any place at any time. Memory(ies) is (are) article(s) of manufacture and/or machine components. It (they) is (are) computer-readable medium(s) from which data and executable software instructions can be read by a computer (or a processor). Memory(ies) may be implemented as one or more of random access memory (RAM), read only memory (ROM), flash memory, electrically programmable read only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, a hard disk, a removable disk, tape, compact disk read only memory (CD-ROM), digital versatile disk (DVD), floppy disk, Blu-ray disk, or any other form of storage medium known in the art. The memories may be volatile or non-volatile, secure and/or encrypted, unsecure and/or unencrypted.

[0130] Memory is an example of a computer-readable storage medium. Computer memory is any memory which is directly accessible to a processor. Examples of computer memory include, but are not limited to RAM memory, registers, and register files. References to computer memory or memory should be interpreted as possibly being multiple memories. The memory may for instance be multiple memories within the same computer system. The memory may also be multiple memories distributed amongst multiple computer systems or computing devices.

[0131] The computer system or device or unit may further include a video display unit, such as a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid-state display, or a cathode ray tube (CRT), for example. Additionally, the computer system or device or unit may include an input device, such as a keyboard/virtual keyboard or touch-sensitive input screen or speech input with speech recognition, and a cursor control device, such as a mouse or touch-sensitive input screen or pad. The computer system or device or unit also optionally includes a disk drive unit, a signal generation device, such as a speaker or remote control, and/or a network interface device.

[0132] The disk drive unit may include a computer-readable medium in which one or more sets of software instructions (software) are embedded. The sets of software instructions are read from the computer-readable medium to be executed by the processor. Further, the software instructions, when executed by the processor, perform one or more steps of the methods and processes as described herein. In an embodiment, the software instructions reside all or in part within a memory (e.g. a main or static memory) and/or the processor during execution by the computer system or device or unit. Further, the computer-readable medium may include software instructions or receive and execute software instructions responsive to a propagated signal, so that a device connected to a network communicates voice, video or data over the network. The software instructions may be transmitted or received over the network via the network interface device.

[0133] In an embodiment, dedicated hardware implementations, such as application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic arrays and other hardware components, are constructed to implement one or more of the methods described herein. One or more embodiments described herein may implement functions using two or more specific interconnected hardware modules or devices with related control and data signals that can be communicated between and through the modules. Accordingly, the present disclosure encompasses software, firmware, and hardware implementations. Nothing in the present application should be interpreted as being implemented or implementable solely with software and not hardware such as a tangible non-transitory processor and/or memory.

[0134] In accordance with various embodiments of the present disclosure, the methods described herein may be implemented using a hardware computer system that executes software programs. Further, in an exemplary, non-limited embodiment, implementations can include distributed processing, component/object distributed processing, and parallel processing. Virtual computer system processing may implement one or more of the methods or functionalities as described herein, and a processor described herein may be used to support a virtual processing environment.