IDENTIFICATION AND ASSIGNMENT OF INSTRUMENTS IN A SURGICAL SYSTEM USING CAMERA RECOGNITION

Abstract

In a surgical system in which surgical instruments are manipulated using robotic arms that move in response to user input from a plurality of input devices. Images capturing an instrument positioned within view of a camera are received. At least one characteristic of the surgical instrument is determined by the system based on the image data. The system automatically assigns the surgical instrument to a user input device, or controls movement of the robotic manipulator carrying the instrument, based on the at least one characteristic. The surgical instrument is then robotically maneuvered by the corresponding robotic arm based on user input to the user input device.

Claims

1.-2 (canceled)

3. A robotic surgical system, comprising: a surgical instrument moveable by a robotic manipulator within a work area; a camera positioned to capture an image of a portion of the surgical instrument; a processor configured to determine, based on the image data from the sensor, at least one characteristic of the surgical instrument and to cause robotic control of the surgical instrument based on the determined characteristic.

4. The system of claim 3, wherein the camera is positioned in the work area inside a patient's body cavity to capture an image of a portion of the surgical instrument inside the body cavity.

5. The system of claim 4, wherein the system further includes an image display and wherein the camera is positioned to capture an image of the work area for display on the image display.

6. The system of claim 4, wherein the system further includes an image display, and wherein the system includes a second camera positioned to capture an image of the work area for display on the image display

7. The system of claim 3, wherein the camera is positioned to capture an image of a portion of the surgical instrument outside the patient's body cavity.

8. The system of claim 3 wherein the characteristic pertains to a shape, color, color pattern, marking, texture, QR code, bar code or other code on the instrument.

9. The system of claim 3 wherein the characteristic is an instrument type, length, geometry, mass or weight, jaw open-close range, jaw type, or an identification of operable degrees of freedom.

10. A method of robotically controlling a surgical instrument, comprising the steps of: receiving image data for an instrument positioned within view of a camera; determining, based on the image data, at least one characteristic of the surgical instrument; and robotically controlling the surgical instrument based on the at least one characteristic.

11. The method of claim 10, wherein the method includes capturing the image data using a camera positioned in a work area inside a patient's body cavity.

12. The method of claim 11, wherein the method further displaying images from the camera on an image display.

13. The method of claim 11, wherein the camera is a first camera and wherein the method further includes capturing second images of the work area using a second camera and displaying the second images, but not the images from the first camera, on an image display.

14. The method of claim 10, wherein the method includes capturing the images of a portion of the surgical instrument outside the patient's body cavity.

15. The method of claim 10 wherein the characteristic pertains to a shape, color, color pattern, marking, texture, QR code, bar code or other code on the instrument.

16. The method of claim 10, wherein the characteristic is an instrument type, length, geometry, mass or weight, jaw open-close range, jaw type, or an identification of operable degrees of freedom.

17. A method of robotically controlling a surgical instrument, comprising the steps of: receiving image data for an instrument positioned within view of a camera; determining, based on the image data, at least one characteristic of the surgical instrument; and automatically assigning the surgical instrument to a user input device based on the at least one characteristic; and robotically controlling the surgical instrument based on user input to the user input device.

18. The method of claim 17, wherein the method includes capturing the image data using a camera positioned in a work area inside a patient's body cavity.

19. The method of claim 17, wherein the method includes capturing the images of a portion of the surgical instrument outside the patient's body cavity.

20. The method of claim 17 wherein the characteristic pertains to a shape, color, color pattern, marking, texture, QR code, bar code or other code on the instrument.

21. The method of claim 17, wherein the characteristic is an instrument type, length, geometry, mass or weight, jaw open-close range, jaw type, or an identification of operable degrees of freedom.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG. 1 is a perspective view of one type of surgical robotic system.

[0011] FIG. 2 shows a camera image display used to facilitate tool assignment.

DETAILED DESCRIPTION

[0012] This application describes use the laparoscopic camera to identify instruments and communicate that back to the surgeon console. Additionally, the method may allow for automatic assignment of the instrument to the manipulator arm via recognition of which side of the screen the instrument enters.

[0013] A control unit provided with the surgical system includes a processor able to execute programs or machine executable instructions stored in a computer-readable storage medium (which will be referred to herein as memory). Note that components referred to in the singular herein, including memory, processor, control unit etc. should be interpreted to mean one or more of such components. The control unit, among other things, generates movement commands for operating the robotic arms based on surgeon input received from the input devices at the surgeon console.

[0014] The memory includes computer readable instructions that are executed by the processor to perform the methods described herein. These include methods of using image input from the laparoscopic camera to identify and recognize eye tracking input in a sequence for assigning user input devices to selected surgical instruments via their shape, color (IR spectrum, visual or patterns), or markings (QR codes, etched or other laser markings, bar code) camera (detection of shapes, colors, markings).

[0015] In use, after an instrument has been positioned such that an image of its relevant features can be captured by the camera, the processor receives image data input based on the captured images of the instrument. The image may be captured outside the body or inside the body. It may be captured before or after the instrument is mounted to the robotic arm. Outside the body, an image of the end effector of the instrument may be captured before the instrument is inserted into the body using the laparoscopic camera or another camera located outside the body, or an image of a more proximal part of the instrument may be captured outside the body (regardless of whether the end effector has been inserted into the body) using the laparoscopic camera before that camera is positioned inside the body, or using an externally positioned camera. Inside the body, image capture for instrument recognition may be performed using the laparoscopic camera or auxiliary image sensors (which may be broadly referred to using the term camera) used for computer vision applications.

[0016] After receiving image data, the processor determines or derives information about the instrument using, for example, other information stored in the memory. This information can include a correlation of the image data (i.e. data relating to the instrument shape, etched markings, QR code, color etc.) to the instrument type and/or to specific instrument geometric properties or drive parameters for that instrument. The system may then control operation of the instrument using the appropriate drive parameters for the instrument type. The system may also automatically assign the instrument to one of the user input devices based on the determined instrument type. For example, if the surgical procedure to be performed is one to be carried out with an instrument of type 1 controlled by the user's right hand and an instrument of a second type, type 2, controlled by the user's left hand, the processor, upon receiving image data indicating that a type 1 instrument had been mounted to a surgical manipulator, may automatically assign that instrument to the right hand user input device. In another embodiment, different regions may be displayed on the visual display.

[0017] These regions could be used to automatically assign an instrument to the right/left user input device. An example of such an image display is shown in FIG. 2.