Robot Assisted Surgical System with Clutch Assistance

Abstract

A robot-assisted surgical system includes a robotic manipulator for robotic positioning of a surgical instrument, and a user input device moveable by a user to cause the robotic manipulator to move the surgical instrument. The system is configured to define virtual boundaries in a workspace of the user input device, based on range limits or user ergonomic limits of the user input device. The system alerts the user if the user input device is moved into proximity of the virtual boundary. This cues the user that it would be useful to clutch and reposition the user input device.

Claims

1. A robot-assisted surgical system comprising: a robotic manipulator configured for robotic positioning of a surgical instrument in a body cavity, at least one haptic user input device moveable by a user, at least one processor and at least one memory, the at least one memory storing instructions executable by said at least one processor to: define virtual boundaries in a workspace of the user input device, the virtual boundaries defined based on range limits or user ergonomic limits of the user input device; receive user input in response to movement of the input device by a user; cause the manipulator to move the first surgical instrument in response to the user input, casing an alert to the user to be generated in response to movement of the user input device in proximity of the virtual boundary, the alert alerting the user to clutch and reposition the user input device.

2. The system of claim 1, wherein the user input device is a haptic input device, and wherein the alert is an activation of actuators of the haptic input device.

3. The system of claim 2, wherein the alert is an activation of the actuators to cause the user input device to push against the user in a direction opposed to the direction of movement of the input device.

4. The system of claim 1, wherein the alert causes vibration of the user input.

5. The system of claim 1, wherein the alert is an auditory alert.

6. The system of claim 1, wherein the alert is a visual alert displayed on a display observable by the user.

7. The system of claim 2, wherein the memory stores instructions executable by the processor to recognize predetermined input from the user input as a clutch instruction, and to suspend the input/output relationship between the user input and the manipulator in response to the clutch instruction.

8. The system of claim 7, wherein the predetermined input is selected from any of the following sensed at the user input device: direction of force/torque frequency of force/torque number of instances of force/torque over a time period duration of application of force/torque direction and/or distance of displacement of the control point

9. The system of claim 1, wherein the memory stores instructions executable by the processor to suspend the input/output relationship between the user input and the manipulator in response to movement of the user input device in proximity of the virtual boundary.

10. The system of claim 7, wherein the memory stores instructions executable by the processor to cause actuators of the input device to move the user input device to a predetermined starting position in response to suspension of the input/output relationship between the user input and the manipulator.

11. The system of claim 7, wherein the memory stores instructions executable by the processor to, in response to user repositioning of the user input device during suspension of the input/output relationship between the user input and the manipulator, defining second virtual boundaries in the workspace of the user input device, and response to user interaction with the virtual boundaries using the user input device, re-engaging the input/output relationship between the user input and the manipulator.

12. The system of claim 11, wherein the user interaction is selected from any of the following sensed at the user input device: direction of force/torque frequency of force/torque number of instances of force/torque over a time period duration of application of force/torque direction and/or distance of displacement of the control point.

13. The system of claim 7, wherein the memory stores instructions executable by the processor to, in response to user repositioning of the user input device during suspension of the input/output relationship between the user input and the manipulator, causing a haptic, auditory or visual alert to the user indicating to the user in response to a determination that the user input device has been re-positioned to a suitable starting position.

14. The system of claim 9, wherein the memory stores instructions executable by the processor to, in response to user repositioning of the user input device during suspension of the input/output relationship between the user input and the manipulator, causing a haptic, auditory or visual alert to the user indicating to the user in response to a determination that the user input device has been re-positioned to a suitable starting position.

15. The system of claim 9, wherein the memory stores instructions executable by the processor to cause actuators of the input device to move the user input device to a predetermined starting position in response to suspension of the input/output relationship between the user input and the manipulator.

16. The system of claim 9, wherein the memory stores instructions executable by the processor to, in response to user repositioning of the user input device during suspension of the input/output relationship between the user input and the manipulator, defining second virtual boundaries in the workspace of the user input device, and response to user interaction with the virtual boundaries using the user input device, re-engaging the input/output relationship between the user input and the manipulator.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIG. 1 shows an example of a robot-assisted surgical system;

[0009] FIG. 2 is a functional block diagram illustrating features of a first method according to the disclosed principles.

[0010] FIG. 3 is a functional block diagram illustrating features of a second method according to the disclosed principles.

[0011] FIG. 4 is a functional block diagram illustrating features of a third method according to the disclosed principles.

DETAILED DESCRIPTION

[0012] Although the inventions described herein may be used on a variety of robotic surgical systems, the embodiments will be described with reference to a system of the type shown in FIG. 1. In the illustrated system, a surgeon console 12 has two input devices such as handles 17, 18. The input devices 12 are configured to be manipulated by a user to generate signals that are used to command motion of a robotically controlled device in multiple degrees of freedom. In use, the user selectively assigns the two handles 17, 18 to two of the robotic manipulators 13, 14, 15, allowing surgeon control of two of the surgical instruments 10a, 10b, and 10c disposed at the working site at any given time. To control a third one of the instruments disposed at the working site, one of the two handles 17, 18 is operatively disengaged from one of the initial two instruments and then operatively paired with the third instrument. A fourth robotic manipulator, not shown in FIG. 1, may be optionally provided to support and maneuver an additional instrument.

[0013] One of the instruments 10a, 10b, 10c is a camera that captures images of the operative field in the body cavity. The camera may be moved by its corresponding robotic manipulator using input from a variety of types of input devices, including, without limitation, one of the handles 17, 18, additional controls on the console, a foot pedal, an eye tracker 21, voice controller, etc. The console may also include a display or monitor 23 configured to display the images captured by the camera, and for optionally displaying system information, patient information, etc.

[0014] A control unit 30 is operationally connected to the robotic arms and to the user interface. The control unit receives user input from the input devices corresponding to the desired movement of the surgical instruments, and the robotic arms are caused to manipulate the surgical instruments accordingly.

[0015] The input devices 17, 18 are configured to be manipulated by a user to generate signals that are processed by the system to generate instructions used to command motion of the manipulators in order to move the instruments in multiple degrees of freedom.

[0016] One or more of the degrees of freedom of the input devices are coupled with an electromechanical system capable of providing tactile haptic feedback to the surgeon, and optionally providing gravity compensation for the user input, and/or. It should be understood that the concepts described in this application are not limited to any particular user input device configuration. Alternative configurations include, without limitation, those described in co-pending application Ser. No. 16/513,670, entitled HAPTIC USER INTERFACE FOR ROBOTICALLY CONTROLLED SURGICAL INSTRUMENTS (Atty Ref: TRX-10610, attached at the Appendix), and user interfaces or haptic devices known to those of skill in the art or developed in the future.

[0017] The surgical system allows the operating room staff to remove and replace surgical instruments carried by the robotic manipulator, based on the surgical need. Once instruments have been installed on the manipulators, the surgeon moves the input devices to provide inputs into the system, and the system processes that information to develop output commands for the robotic manipulator in order to move the instruments and, as appropriate, operate the instrument end effectors. The user interface may be one that allows the surgeon to select motion scaling factors as well as to clutch and reposition the handles to a more comfortable position. In some cases, the surgeon may desire a fine scaling motion, while in others he may prefer larger motion, relative to the movement of the user interface.

[0018] The concept described in this application involves the use of the haptic input device to create virtual tactile boundaries that define the useful ergonomic workspace of the input device to enable more automatic and natural clutch activation. When the user, while moving the user input, reaches these boundaries, which may be near the end of range of motion of the device and/or that the user's pose is no longer comfortable or effective position, a number of different haptic features could be used to assist the user with clutching. These include the following:

1. Virtual Walls with Force/Motion Gesture Clutch

[0019] With this feature, depicted in FIG. 2, haptic constraints are created to act as virtual walls at the edges of the useful workspace of the device. In the control system, the real time position and orientation of the control point near the input handle is monitored, such as using input from position sensors associated with the user input device. When the position of the control is determined to be outside of the predetermined defined virtual boundaries (edges of the walls) stored in the system's memory, the control system will generate a force/torque at the control point in the opposite direction. In one possible implementation, the magnitude of this correcting force/torque is proportional to the difference between the position of the control point and the position of the wall. This distance term is then multiplied by a constant value to set the magnitude of force that is applied to the handle by the motors. In this example, the virtual walls would feel like springs of spring rate equal to the constant value described previously, trying to push the user back into the workspace.

[0020] To cause the system to clutch, rather than using the conventional approach of depressing the foot pedal of the work station or engaging other switch/input, the user could move the haptic user input in the direction of this virtual boundary, to press against this virtual boundary. This action serves as input to the control system that the user would like to disable the output motion of the manipulator. Examples of gestures that might be used to function as a haptic clutch include, without limitation:

[0021] direction of force/torque

[0022] frequency of force/torque

[0023] number of instances of force/torque over a time period

[0024] duration of application of force/torque

[0025] direction and/or distance of displacement of the control point

[0026] In one exemplary embodiment, the user moves the user input device twice in succession, to press twice against the virtual wall, or moves the user input device and holds against the wall for a certain time, or presses hard/far enough into the wall to cause the system to recognize the input as an instruction to clutch. The choice of which gestures to use as a haptic clutch could be made by evaluating user preferences and programming the system to recognize those gestures as instructions to perform clutching.

2. Virtual Boundary with Haptic Alert

[0027] In an alternative configuration depicted in FIG. 3, a haptic or vibratory alert is used to indicate to the user that he/she is in a position in which clutching and repositioning is recommended. This mode would allow the user to keep operating up to the limits of the device but provides an alert to remind the user to clutch rather than continuing to operate in a poor ergonomic position. This might be particularly useful with surgeons who are new to using robotic surgical systems. For those surgeons, clutching is a new feature to learn, and so a regular reminder to clutch and re-center for comfort could be beneficial. This concept would provide those reminders near the limits of the input device, similar to a warning track in the outfield of a baseball field.

3. Virtual Boundary with Visual or Audible Alert

[0028] This concept is identical in purpose and function to concept (2) except that a visual or audible alert on the monitor displaying the surgical field or from the user interface console is provided instead of a haptic alert.

4. Virtual Boundary with Auto-Clutch

[0029] In this final option, depicted in FIG. 4, the system automatically clutches to disable output motion when the virtual boundary is reached. For a more experienced user, this type of control mode could be very efficient as he/she would know from experience that he/she is near the workspace limits and expect to be auto-clutched soon. Upon auto-clutching, the user could quickly re-center his/her hands and continue operating. A haptic, visual, or audible alert may also be used to immediately notify the surgeon that he/she has been clutched out at the moment that the virtual boundary is reached.

[0030] Once the control system deactivates the output motion input motion relationship, the system can assist with repositioning the user's hands and re-enabling the control of the output device in any of the following ways:

1. Autonomous Motion to Optimal Start Position

[0031] Once the system is clutched out via one of the methods discussed above, the manipulator autonomously moves the handle into the optimal start position. The user can then resume the procedure and clutch back in simply by moving the handles. Alternatively, a haptic constraint can be applied so that the user needs to apply a force greater than some threshold for the system to clutch back in and enable control of the output device.

2. Virtual Wall at Optimal Start Position with Force/Motion Gesture Clutch (1+ Planes)

[0032] Once the system is clutched out via one of the methods discussed above, the user can reposition his/her hand back to the optimal starting position. At this optimal start position, the user will feel the virtual walls being created by the control system and he/she can use force/position gestures (as described previously) to clutch back in. The virtual walls may be 1 or more virtual planes enabling the user some flexibility in start pose. If 1 plane is used, it may allow the user to clutch back in at any height or depth but only at the lateral position at which the virtual plane is created. The choice of position and orientation of the virtual wall can be made by the control system based on which virtual boundary was crossed when clutching out. For example, if the maximum right side limit was hit, the system may create a haptic wall positioned at the optimal start position such that the user will feel the wall after moving the handle back to the left.

3. Haptic, Visual, or Audible Alert at Optimal Start Position

[0033] This option is identical to the previous option (2) except that an alert is provided to the user via haptic vibration, visual alert, and/or audible alert once the virtual boundary is crossed that defines the optimal start pose. This will tell the user that this is a good ergonomic position at which to clutch back in.

[0034] The disclosed concepts provide several advantages over existing technology. Current robotic systems use buttons, pedals, surgeon presence sensors, or other switches to clutch in and out for hand repositioning. This can, at times, be cumbersome, especially for new users. Such prior methods also require the use of a finger, foot, etc to actuate the clutch which prevents them from being used for other purposes. Furthermore, those steps take time, require training, and leave room for ergonomic and usability improvements. Clutching is a critical advantage that robotic surgery offers over manual surgical methods as it enables improved ergonomics, strength, and confidence for surgeons performing the procedures. The disclosed methods of using virtual boundaries, virtual walls, and force/motion gesture clutching to clutch in and out of control of the output device enables users to maintain hands on the input device, use hands, fingers, and feet for other tasks, reminds users to clutch when outside of ergonomic limits, and accelerates the clutching process by increasing ease of use and increasing surgeon robot collaboration. These methods should be much more natural to a user and should help increase effective utilization of robotic clutching.

[0035] I believe that the use of virtual boundaries and/or virtual walls to provide haptic, audible, or visual alerts to the user to encourage ergonomic adjustment and suggest the use of clutching is novel for surgical robotics. Automatic clutching at virtual boundaries is also novel to my understanding. I also believe that the use of virtual walls with haptic force/motion gesture clutching is entirely new in our field. As far as I know, all of the functionality described in the technical details section is novel.

[0036] All prior patents and applications referred to herein, including for purposes of priority, are incorporated herein by reference.