ARM WITH A COMBINED SHAPE AND FORCE SENSOR

Abstract

A bend sensor is used to determine force applied to a robotic arm. The force may be an external force applied to the arm, an internal actuation force, or both. In some aspects, a stiffening element is used to restore the arm to a minimum kinematic energy state. In other aspects, the stiffening element is eliminated, and the arm is fully actuated.

Claims

1-19. (canceled)

20. A robotic surgical system comprising: a surgical instrument, wherein the surgical instrument includes: an elongate, hollow shaft having a proximal end, a distal end, and a flexible section; a sensor apparatus configured to generate sensor data about the flexible section; and a force transmission mechanism coupled to the proximal end of the shaft; and a processor communicatively coupled to at least the sensor apparatus, wherein the processor is configured to receive the sensor data about the flexible section from the sensor apparatus and to use the sensor data with information regarding mechanical and material property data for the surgical instrument to determine at least one of an internal actuation force applied by the force transmission mechanism or external force information for the surgical instrument.

21. The robotic surgical system of claim 20 wherein the processor is further configured to receive three-dimensional imaging data for tracking the surgical instrument.

22. The robotic surgical system of claim 20 wherein the external force information includes information about at least one of a magnitude or a direction of an external force applied to the surgical instrument.

23. The robotic surgical system of claim 20 wherein the sensor apparatus comprises an optical fiber shape sensor.

24. The robotic surgical system of claim 20 wherein the sensor apparatus comprises a piezo sensor.

25. The robotic surgical system of claim 20 wherein the surgical instrument further includes a passive stiffening element coupled to the distal end of the shaft.

26. The robotic surgical system of claim 25 wherein the passive stiffening element includes an internal spring.

27. The robotic surgical system of claim 20 wherein the processor is further configured to output an indication to a user.

28. The robotic surgical system of claim 27 wherein the indication provides a haptic force at a control device.

29. The robotic surgical system of claim 27 wherein the indication provides a visual cue at a control device.

30. The robotic surgical system of claim 27 wherein the indication is a visual indicator.

31. The robotic surgical system of claim 20 wherein the processor is further configured to output the external force information in a control loop for receipt by an actuation mechanism for actuating the surgical instrument.

32. The robotic surgical system of claim 31 further comprising the actuation mechanism.

33. The robotic surgical system of claim 20 wherein the flexible section is continuously flexible.

34. The robotic surgical system of claim 20 wherein the mechanical and material property data includes dimensional information of the surgical instrument.

35. The robotic surgical system of claim 20 wherein the dimensional information comprises spacing between joints of the surgical instrument.

36. The robotic surgical system of claim 35 wherein the mechanical and material property data includes friction information.

37. The robotic surgical system of claim 35 wherein the mechanical and material property data includes stretch property information for a cable actuated by the force transmission mechanism.

38. The robotic surgical system of claim 20 wherein the surgical instrument further includes an end effector at the distal end.

39. The robotic surgical system of claim 20, wherein the processor is further configured to use image based tracking to determine position and orientation information for the surgical instrument.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0018] FIG. 1 is a diagrammatic view that illustrates a bask inventive concept.

[0019] FIG. 2 is a diagrammatic view of elements of an arm and the associated forces that will be calculated in accordance with an aspect of the subject invention.

[0020] FIG. 3 is diagrammatic view of one embodiment of the subject invention.

[0021] FIG. 4 is diagrammatic view of an articulated arm.

DETAILED DESCRIPTION

[0022] Arms, including instruments and structures used to support and move such instruments, may be substantially rigid or flexible. Flexible instruments may be classified into two types: (i) continuously flexible arms that do not have discrete joints (i.e., infinite degrees of freedom), and (ii) arms that have a series of two or more short links interconnected by joints having at least one degree of freedom (the number of degrees of freedom being sufficient to provide the required flexibility). Many medical devices, such as catheters and small diameter minimally invasive instruments, have such a continuously flexible property. Many other medical devices, such as relatively large diameter endoscopes that include one or more working channels for minimally invasive instruments, are made of such a series of short, interconnected links. It is possible to have a hybrid flexible arm that includes at least one continuously flexible portion and at least one link portion. Arms with both rigid and flexible (continuous, linked, or hybrid) portions are also possible.

[0023] Such arms may be actuated to move by various methods that provide a necessary actuation force. One exemplary method is actuation with one or more tendons (e.g., cables) that run along the length of an arm. A pulling force (tension) in one or more of the tendons bends the arm or a segment in the arm. Actuation force may be provided from various sources, such as manual crank or a servomotor. Other mechanical actuation methods, such as providing pushing force (compression) on tendons, using gears, etc. are known and may be used.

[0024] An arm may include a passive stiffening element that applies a restoring force to a default pose for the arm. Stiffening elements may be of various forms, such as an internal spring that runs inside the flexible arm or an external sheath that surrounds the flexible arm. When no bending forces are applied to such an arm, the stiffening element enforces a minimum energy shape or kinematic configuration. If a bending force is applied to the arm, the stiffening element opposes the bending force on the portion of the arm that is being moved.

[0025] Both internal actuation force and external force may be applied to the arm. An example of an external force is the reactive force the arm experiences if it is pushed against tissue. The stiffening element opposes both externally applied force and the internal actuation force. In addition to opposing the actuation force, the stiffening element also spreads the actuation force across the bending portion of the arm so that, for instance, the angle between each link in a series of links is about equal.

[0026] It is possible to use a bend or shape sensor to detect not just the minimal bends associated with the normal actuation of the arm, but also the perturbation generated by an external force, of the stiffening element away from its minimum energy shape. It is therefore possible to detect the magnitude and direction of the applied external force in addition to the shape of the arm.

[0027] The subject invention can be implemented using fiber optic shape sensing technology of the type disclosed in the '019 application, referenced above. (See also, U.S. patent application Ser. No. 12/164,829, (filed Jun. 30, 2008), which is incorporated by reference.) As discussed below, other shape sensing technologies could be used.

[0028] Aspects of the invention may be applied to several arm configurations. One illustrative configuration is to have a non-actuated continuously flexible arm with a stiffening element. A simple example of this first illustrative configuration is the optical fiber bend sensor alone since it is continuously flexible and possesses an inherent stiffness. A second configuration is to have an actuated continuous or discretely flexible arm with a stiffening element. This second configuration, on which much of the following description is based, is underactuated, because actuation does not fully control the shape. A third configuration is to have an actuated discrete link flexible arm without an additional stiffening element. Since the stiffening element is not present in this third configuration, the arm's actuation mechanism must be able to completely determine the arm's kinematic position (i.e., every degree of freedom is actuated). In each of these illustrative configurations, the mechanical properties of these configurations (e.g., dimensions (such as link size, distance between joints, fit tolerances, and the like), material properties (such as known or estimated friction, cable “stretchiness”, and the like), etc.) are known so that such information can be combined with information from the sensor to derive the actuation force and/or a force externally applied to the arm.

[0029] FIG. 1 illustrates a basic inventive concept. An arm is illustrated as a tubular member 10, which includes an internal stiffening element (not shown). In pose 10a, tubular member 10's curved shape is caused by internal tensioning (actuation) forces. If a force F is applied to the end of the member 10 in pose 10a, it will begin to bend in an “S” like shape as illustrated by pose 10b, since the internal stiffening element must deflect to create a restoring force equal to force F. If a shape sensor is used to measure the bend in member 10, the information it generates can also be used to measure the external force. In FIG. 1, X represents the distance between the arm's minimum energy shape under the internal actuation force and the arm's actual shape (the result of the external force). For stiffness matrix K that may depend upon X, F is the force (F=KX).

[0030] FIG. 2 provides a more detailed example. In this example, the elongate arm 200 is attached at one end (the proximal end) to a fixed, known position 202. The position of the distal end 204 is determined by a number of forces acting on the arm. First, the arm contains a stiffening element that provides forces that restore the arm to a certain preferred shape. Second, some actuation mechanism (e.g., a servo actuator) actively influences the shape of the arm. Finally, an external force is applied at the distal end 204 of the arm. The total shape of the arm is measured by a sensing apparatus (not shown).

[0031] As shown in FIG. 2, the elongate arm 200 is a planar device composed of three rigid links L (links L may be relatively short, to form a flexible configuration that approximates a continuously flexible arm, or they may be relatively longer). Two revolute joints 206a couple the ends of the links L in series, and a third revolute joint 206b couples the proximal end of the arm to base 202. The axes of the revolute joints are parallel, so that the arm articulates in a plane (x-y, as shown). The stiffening element is a combination of three equal torsional springs S, each with stiffness k, that are attached at each joint 206. The actuation mechanism is a single cable C attached at the distal tip 204 and routed along the links, and the sensing apparatus (not shown) measures the angles of the three joints 206. The external force is a linear force in the arm's plane of motion that is applied at the distal tip, as illustrated by the arrow.

[0032] The three forces (stiffness, actuation, external) act at different locations and in different ways on the arm, but the forces can be compared by writing a force balance for the torques applied at each of the three joints. For a general arm with n joints, standard robotics theory and geometry are used to make this force balance:

0=τ=Wu−Kθ+Bf

where τ is the total combined n-dimensional torque vector, which should balance to zero if the system is not moving, u is the c-dimensional vector of cable forces, θ is the n-dimensional vector of joint angles, and f is the e-dimensional vector of external tip forces. In other words, n is the number of mechanical degrees of freedom, c is the degree of actuation, and e is the number of independent external force directions. For the example in FIG. 2, (n,c,e)=(3,1,2), and the matrices W, K, and B are:

[00001]$W=\left[\begin{array}{c}w\cos \left({\theta }_1\right)\\ w\cos \left({\theta }_2\right)\\ w\cos \left({\theta }_3\right)\end{array}\right]$$K=\left[\begin{array}{ccc}k& 0& 0\\ 0& k& 0\\ 0& 0& k\end{array}\right]$$B=-l\left[\begin{array}{cc}\cos \left({\theta }_1\right)+\cos \left({\theta }_1+{\theta }_2\right)+\cos \left({\theta }_1+{\theta }_2+{\theta }_3\right)& \sin \left({\theta }_1\right)+\sin \left({\theta }_1+{\theta }_2\right)+\sin \left({\theta }_1+{\theta }_2+{\theta }_3\right)\\ \cos \left({\theta }_1+{\theta }_2\right)+\cos \left({\theta }_1+{\theta }_2+{\theta }_3\right)& \sin \left({\theta }_1+{\theta }_2\right)+\sin \left({\theta }_1+{\theta }_2+{\theta }_3\right)\\ \cos \left({\theta }_1+{\theta }_2+{\theta }_3\right)& \sin \left({\theta }_1+{\theta }_2+{\theta }_3\right)\end{array}\right]$

[0033] The force balance equation can be used to determine the unknown external forces by f measuring the shape θ of the arm. The approach assumes that external tip forces influence the shape differently than the (unknown) cable forces u, such that if sufficient shape variables θ are available (i.e., n is large enough), u and f can be recovered at the same time.

[0034] The force balance equation can be used to recover both the external force f and the actuation force u, provided that sufficient shape information is available and the actuation and external forces influence the shape independently. For the example of FIG. 2 and similar tendon-actuated arms, these conditions are met unless the system is at a kinematic singularity.

[0035] The force balance can be rewritten, separating the known from the unknown variables, as follows:

[00002]$\left[\begin{array}{cc}W& B\end{array}\right].Math.\left[\begin{array}{c}u\\ f\end{array}\right]=K\theta$

[0036] If the matrix [W B] has full column rank c+e (this is the mathematical formulation of the condition “independent shape influence”), the unknown variables u and f can be uniquely recovered, for example using a pseudo-inverse as:

[00003]$\left[\begin{array}{c}u\\ f\end{array}\right]={\left[\begin{array}{cc}W& B\end{array}\right]}^{+}K\theta ={\left(\left[\begin{array}{c}{W}^r\\ B^r\end{array}\right].Math.\left[\begin{array}{cc}W& B\end{array}\right]\left[\begin{array}{c}{W}^r\\ B^r\end{array}\right]\right)}^{-1}\left[\begin{array}{c}{W}^r\\ B^r\end{array}\right]K\theta$

[0037] Intuitively, the way the force information is obtained is by comparing the measured shape to a shape that could be achieved using only cable actuation. For example, without external forces, the system in FIG. 2 will acquire a circular shape with all angles θ equal, and solving the previous equation will return a solution (u,f) with f=0. If θ describes a shape that is not a circle, then the equation translates this “non-circle” shape to external forces f that must have caused it.

[0038] Finally, it is important to note that this derivation relies on the facts that the arm is only partially actuated (c<n), only certain external forces are applied (e<n), and the shape measurement is rich enough to independently describe both effects (c+e≤n). These dimensional relations, plus the condition that the matrix [W B] must have full column rank, can be used to generalize to more complex mechanisms.

[0039] It should be understood that the more degrees of freedom of movement in the system, the easier it is to mathematically derive the external forces on the system (the best case, there fore1is with the infinite degrees of freedom of a continuously flexible arm). The example above also focused on recovering an external force applied to the distal tip. It should be understood that depending on the structure of the arm, external force(s) applied between the distal and proximal ends could also be determined. And, if sufficient shape information is available an external force component directed along the arm's longitudinal axis can be determined.

[0040] FIG. 3 is a schematic diagram of another illustrative embodiment of the subject invention. In this illustration, the arm is a minimally invasive surgical instrument 300 that includes an elongated1hollow, substantially rigid shaft 302 coupled at its distal end to a flexible wrist section 304. A surgical end effector 306, such as a tissue grasper, is connected to the distal end of the wrist section. A force transmission mechanism 310 is coupled to the proximal end of the shaft. Teleoperated servomotor actuators engage the transmission mechanism 310 at mechanical interface 311, which in turn passes actuating forces through shaft 302 via cables (not shown) to cause the wrist section mechanism to bend in pitch and/or yaw and to actuate the end effector. (See U.S. Pat. No. 6,817,974 (filed Jun. 28, 2002), U.S. Pat. No. 5,807,377 (filed May 16, 1997), and U.S. Pat. No. 6,461,372 (filed Mar. 8, 2000), each of which is incorporated herein by reference, for More details about this type of structure).

[0041] In accordance with an aspect of the subject invention, the instrument 300 is illustratively provided with an optical fiber 312 that includes a plurality of fiber Bragg gratings for sensing shape. Fiber 312 extends through wrist 304 and optionally into some or all of end effector 306. Information about these types of fibers is described in detail in the above-cited patent applications. Other bend sensors could be used. For example, other optical fiber shape sensing technologies that do not require FBGs (e.g., Rayleigh scattering methods) may be used. Magnetic/electric field sensors can be placed along the length of the flexible arm. Alternatively, piezo sensors can be placed at the joints or distributed along body of the arm. Other sensing possibilities include the use of sliding sensing cables that provide position information of joints, or the use of three-dimensional imaging and machine vision approaches to derive instrument position, orientation, and shape. Image-based tool tracking technology may also be used. (See e.g., U.S. Patent Application Publications No. US 2009/0088897 A1 (filed Sep. 30, 2007), US 2009/0088773 A1 (filed Sep. 30, 2007), and US 2009/0088634, A1 (filed Sep. 30, 2007), each of which is incorporated herein by reference.)

[0042] The proximal end of fiber 312 is coupled to fiber strain interrogator/detector unit 314. Unit 314 functions to send light down the fiber and to collect and measure reflected light. As noted above, changes in the shape of the wrist 304 strain the gratings associated with the wrist, which in turn will change the character of the returned light in a manner which can be measured.

[0043] The output of unit 314 is supplied to an electronic data processor 316. Processor 316 uses the information to determine both the shape of the arm as well as any external forces applied to the arm as discussed above. The external force information may then be used in a teleoperation loop to output an indication of the forces haptically at a master control device (e.g., finger or hand grip), or to output the force indications to the user in some other way (e.g., using visual, vibro-tactile, etc. methods). (See, U.S. Pat. No. 6,424,885, (filed Aug. 13, 1999) incorporated herein by reference.) The external force information may also be routed in a control loop to a controller 318 that automatically controls one or more of the servomotor actuators to keep instrument 300's arm stationary as the external force is applied to or withdrawn from the arm. The shaft 302 and wrist 304 arm configuration is illustrative of various arm configurations as described above.

[0044] FIG. 4 illustrates another type of robotic arm 400 that could be adapted for use with the subject invention. Arm 400 includes a plurality of spaced apart disks 402. Each of the disks 402 is rigidly attached to a center spine 404. The center spine exerts a spring (stiffening) force that attempts to straighten arm 400 if bent.

[0045] Each of three tendons 406 are rigidly attached to the most distal disk 402 and pass through holes in the remaining disks. Arm 400 is not constrained to planar movement, and it is activated by changing the relative tension among the three tendons. In free space, the arm will bend in a smooth arc so as to minimize the energy stored in the center spine. If a force is applied to the arm, however, its shape will change in a manner that can be measured. As with the embodiment of FIG. 3, the embodiment of FIG. 4 is provided with an apparatus as described above for monitoring the change in shape of the arm. One suitable apparatus would be an optical fiber with Bragg gratings, as discussed above.

[0046] The ability to recover both the tendon forces and the external forces acting on a movable arm can be used to control the arm. For example, with this information, the control system (FIG. 3) can be programmed to compensate for various drive train dynamics, for example, friction acting on actuation cables.

[0047] Referring now to the first illustrative configuration mentioned above (non-actuated continuously flexible arm with a stiffening element), the approach for determining external forces using bend sensors can be applied to flexible probes that contain no actuating mechanisms for internally controlling the motion of the probe. In a simple example, the probe can be a spring element having bend sensors applied thereto. When the distal tip of the probe hits a surface, it will bend into some shape. The bend sensors can be used to determine that shape. The information about the shape can be used to determine the force vector applied to the tip. This concept can apply to a probe within finite degrees of freedom, such as a continuously flexible device or to a device with a finite number of degrees of freedom. An illustrative probe is a whisker-like device that has a known, inherent stiffness.

[0048] In the prior art, strain gauges have been mounted on rigid links of an arm to determine some information about lateral forces on the arm that cause shaft bending and shear loads. In the case of the flexible probe discussed above, however, the interest is in deriving bend information that modifies the shape of the probe rather than strain information. Such bend information can be derived using the Luna fiber system mentioned above.

[0049] Referring now to the third illustrative condition (actuated discrete link flexible arm without a stiffening element), the subject invention can also be used with robotic arms that are fully actuated with no explicit stiffening elements. Information about external forces can be derived by using the bend sensors to determine how the links deflect (since the links will, in fact, have finite stiffness). In this system, the bend sensor information from the links is used to determine force, and the bend sensor information from the joints is used to determine joint position. Both force and joint position information are usable in a control system, as illustrated above.

[0050] In another aspect it can be seen that a similar approach may be used for sensing twist force around a longitudinal axis (i.e., the axis between proximal and distal ends) of an arm. The combination of an optical fiber that is used to sense twist (e.g., a four core fiber with one core aligned with the fiber's longitudinal axis and the other three cores equidistantly spaced around the center core) and a torsion spring that seeks a minimum torsional energy state may be used to determine torque on the linkage arm or continuously flexible arm. The torque sensing multicore fiber may also be used to sense bend, and also lateral force, as described above.

[0051] While the subject invention has been described with reference to several embodiments, various changes and modifications could be made therein, by one skilled in the art, without varying from the scope and spirit of the subject invention as defined by the appended claims. Principles and elements illustrated in one or more of the drawings or described above in relation to one aspect, embodiment, or implementation of the invention are not limited solely to that one aspect, embodiment, or implementation, but may be incorporated in various other aspects, embodiments, and implementations. For example, shape sensor apparatus components and electronic data processors that receive information extracted from shape sensor apparatus components may be incorporated as necessary in the various aspects, embodiments, and implementations illustrated and described herein.