TELESURGICAL SYSTEM WITH INTRINSIC HAPTIC FEEDBACK BY DYNAMIC CHARACTERISTIC LINE ADAPTATION FOR GRIPPING FORCE AND END EFFECTOR COORDINATES

Abstract

A teleoperation system is provided, having a slave having a drive unit which drives a gripping end effector, wherein a kinematic coordinated end effector and a gripping force f effector can be determined with a camera which is preferably integrated in the slave and which is aligned with the end effector; a master, which is remote from the slave, with at least one operating unit on which a user can exert a gripping head F.sub.G, the gripping force being transmitted to the slave, and a visual user interface representing the image of the camera; and where F.sub.G is linearly dependent on the kinematic coordinate and the F.sub.effector.

Claims

1. A teleoperating system comprising: a slave (10) which has a drive unit which drives a gripping end effector, wherein a kinematic coordinate of the end effector and a gripping force F.sub.effector can be determined with a camera (9) which is preferably integrated in the slave and which is aligned with the end effector, a master (1) which is remote from the slave, having at least one operating unit (2, 3) on which a user can apply a gripping force F.sub.G, the gripping force being transmitted to the slave, and a visual user interface 4), which represents the image of the camera, where F.sub.G is linearly dependent on the kinematic coordinate and the F.sub.effector, or vice versa.

2. The teleoperation system of claim 1, wherein the F.sub.effector is determined by one or more of the following approaches: deduction of the force from the drive units of the drive unit in the slave or from a control computer; measuring the current in the drive unit; measuring the force in a kinematic structure between the end effector and the drive unit; structure-integrated measurement by force sensors in parallel kinematics; force/torque sensors on the drive unit; measurement of the force directly between the end effector and the surrounding tissue.

3. The teleoperation system as claimed in claim 1, wherein the operating unit is as rigid as possible and has only the flexibility required for the gripping force detection.

4. The teleoperation system as claimed in claim 1, wherein the operating unit has a defined resilience and is thus designed for a defined deflection and thus enables gripping force detection, whereby an actuator in the operating unit can be dispensed with.

5. The teleoperation system as claimed in claim 1, wherein the gripping force F.sub.G is determined by deriving the interaction force between the operating unit and the user by one or more of the following methods: force measurement between the fingers differential force measurement between the fingers discharge of the force from the deflection or deformation of a non-rigid operating unit.

6. The teleoperation system as claimed in claim 1, wherein:
F.sub.G=Kinematic coordinate*F.sub.effector Or
F.sub.G=Kinematic coordinate+F.sub.effector or
F.sub.G=Kinematic coordinate*(F.sub.effector+F.sub.min)+F.sub.G.sub._.sub.offset Where F.sub.min is the force to initially move the effector, and F.sub.G.sub._.sub.offset is the force to allow the sensor to respond in the operating unit, and preferably, possible factors for scaling the forces to adjust the described relationships to any of the manipulated environment conditions.

7. The teleoperation system as claimed in claim 1, characterized by a unit for generating tactile haptic feedback on the operating unit, wherein a frequency is transmitted by a sensor in the slave, which is sent to the unit for generating tactile haptic feedback Which is preferably in the range from about 50 to 1000 HZ.

8. The teleoperating system according to claim 7, wherein the tactile haptic feedback generating unit is one or more of the following: force output by inertial mass motors eccentric motors piezoelectric actuators.

9. The teleoperating system as claimed according to claim 7, wherein an acting force direction of the tactile haptic feedback generating unit exerts no or only minimal forces in the direction of the gripping force F.sub.G, in order to reduce control instability in the system.

10. The teleoperating system as claimed according to claim 7, wherein the frequency detected by a sensor in the slave is filtered as a function of ambient values in order to obtain stability in a control loop.

11. The teleoperating system as claimed according to claim 7, wherein the sensor in the slave is one or more of the following: (SAW) sensors for detecting surface oscillations in the kinematic components or at the end effector.

12. The teleoperating system as claimed according to claim 1, wherein an additional digital representation of the current end effector coordinate can be superimposed in the camera image, preferably by one or more of the following: angle indication, strokes which move towards each other, a stylized gripper that moves, color traces, representation of the force acting on the end effector on the display, deflection.

13. The teleoperating system as claimed according to claim 1, wherein a control computer is designed to carry out a differential force measurement on the operating unit by measuring the gripping force for the thumb and index finger separately from one another, and preferably the respective smaller or larger of the two measured values for The gripping force.

14. A slave for a teleoperation system, according to claim 1, comprising: at least three tripods arranged as tripod, each having two active degrees of freedom in the form of translation and rotation, and each being driven by means of a drive into the degrees of freedom; with an end effector which is connected to the push rods via kinematic chains, wherein the kinematic chains are designed in such a way that the end effector can be aligned and can be opened and closed in three dimensions by means of translation or rotation of the push rods.

15. The slave according to claim 14, wherein a kinematic chain is formed as a main chain, the rotation of which leads to a rotation of the end effector and the displacement thereof leads to a displacement of the end effector.

16. The slave according to claim 15, wherein two chains are formed as side chains, the displacement of which leads to a displacement of the end effector, and the rotation thereof leads to an opening or closing or bending.

17. The slave according to claim 16, wherein the rotations of the secondary chains are converted into a linear movement via a spindle and a carriage, which opens or closes the end effector.

18. The slave according to claim 14, wherein the kinematic main chain has at least four degrees of freedom and/or the kinematic secondary chain has at least six degrees of freedom.

19. The slave as claimed in claim 14, wherein the subchain is connected to the main chain by means of swivel joints, wherein the swivel joints are designed as U-shaped clamping elements.

Description

BRIEF DESCRIPTION

[0055] FIG. 1 shows the structure of an exemplary teleoperation system based on a master-slave structure;

[0056] FIG. 2 shows the system structure of a conventional teleoperation system;

[0057] FIG. 3 shows the system structure of a pseudohaptic system;

[0058] FIG. 4 shows the system structure of a combined teleoperating system impedance admittance structure as well as an additional pseudohaptic degree of freedom and a structure for superposition of high-frequency haptic feedback;

[0059] FIG. 5 shows an end effector with different positions of the gripping arms;

[0060] FIG. 6 shows an end effector with completely opened gripper arms;

[0061] FIG. 7 shows an end effector with partially closed gripping arms;

[0062] FIG. 8 shows an end effector with completely closed gripper arms;

[0063] FIG. 9 shows an end effector without force action between the gripper arms, since no tissue contact is yet present;

[0064] FIG. 10 shows an end effector with an effective end effector gripping force during tissue contact;

[0065] FIG. 11 shows a rigidly designed control means in the master as well as the direction of the gripping force under engagement of the user;

[0066] FIG. 12 shows a control means which is designed with a defined resilience as well as the direction of the gripping force under engagement of the user;

[0067] FIG. 13 shows the relationship between the gripping force and the closing angle without influencing the coupling characteristic by the effective end effector force. Characteristic curve 1 and characteristic curve 2 differ by the predetermined force F.sub.min;

[0068] FIG. 14 shows, in contrast to FIG. 13, the relationship between gripping force and closing angle starting from an optionally applicable offset of the gripping force;

[0069] FIG. 15 shows the haptically perceptible gripping force difference in the case of visually perceived equal end effector closing angle by variation of the coupling characteristic between gripping force and closing angle;

[0070] FIG. 16 shows an exemplary characteristic curve with the influence of different end effector gripping forces based on the multiplicative evaluation of the relationship between gripping force and closing angle with the acting end effector gripping force.

[0071] FIG. 17 shows, in contrast to FIG. 16, the characteristic curve profile with the influence of different end effector gripping forces on the basis of the additive assessment of the relationship between gripping force and closing angle with the acting end effector gripping force.

[0072] FIG. 18 shows the embodiment of the slave consisting of end effector 1, TCP 2, parallel kinematic mechanism 3, push rods 4, camera channel 5, shaft 6 and drive unit 7;

[0073] FIG. 19 shows an enlargement of the embodiment in FIG. 18 with end effector 1, TCP 2, parallel kinematic mechanism 3, push rods 4, camera channel 5, shaft 6;

[0074] FIG. 20 shows the embodiment of an control means with control means 1, TCP of the control unit 2, base 3, drives of the control means 4;

[0075] FIG. 21 shows the embodiment of a rigid control means with gripping elements 1, 2, fingers 3, base 4 of the control means, fastening element 5 for fastening to the TCP of the control means, force sensor elements 6 between grip elements and base of the control means;

[0076] FIG. 22 shows a sectional as well as the exploded drawing of the embodiment of an control means with grip elements 1, drives for tactile feedback 2, force sensor elements 3, base of the control means 4, fastening element for TCP of the control means;

[0077] FIG. 23 shows a detailed construction of the slave.

DESCRIPTION OF THE EMBODIMENT

[0078] The invention is described below with reference to a teleoperation system for minimally invasive surgery, which is not to be understood as limiting. This transmits control information from the user to an intracorporal manipulator and represents the interaction forces between the end effector of the intracorporal manipulator and tissue as haptic and pseudo-haptic feedback on the control unit.

[0079] FIG. 1 shows the structure of an exemplary teleoperation system based on a master-slave structure with master 1, control unit 2, control means 3, visual user interface (screen 4), parallel kinematic mechanism 5, end effector 6, tool center point 7, working channel 8, camera channel 9, Slave 10, operating table 11.

[0080] The slave is shown in FIG. 1. It consists of a parallel kinematic mechanism to which TCP an end effector is mounted. The position of the TCP and thus the position of the end effector can be adjusted by the defined longitudinal displacement of the push rods. The individual push rods are moved by separate actuators in the drive unit. The slave's shaft has a channel for a camera and a working channel. The end effector consists of two gripping arms, between which the end effector gripping force (F.sub.effector) acts. The closing angle (Phi) is the angle between the two gripping arms of the effector. Both the acting force (F.sub.effector) and the closing angle (Phi) are thus determined. In addition to the force between the end effector grippers (F.sub.effector), the interaction forces between the end effector and the environment are derived. The slave is connected to the master by means of a corresponding cable.

[0081] FIGS. 2 and 3 show in comparison the difference of a conventional system to the system of the present invention. It can be seen here that feedback via actuators is not given in the present invention. FIG. 4 shows the system structure of a combined teleoperating system, impedance admittance architecture as well as an additional pseudohaptic degree of freedom and a structure for superposition of high-frequency haptic feedback. The systems of FIGS. 2 and 3 have been combined together here.

[0082] The master consists of two control units according to FIG. 20 for the left hand and the right hand. These control units have a control means according to FIGS. 11, 12, 20, 21, 22. The user operates with the control means via the control unit. The control means is connected to the control unit at the TCP of the kinematics of the control unit. Control inputs for the slave are entered into the system by means of user input into the control means and thus into the control unit. By means of actuators mounted in the base of the control means, haptic feedback can be generated with regard to the interaction forces measured at the slave between the end effector and the environment and can be output to the user via the control means.

[0083] FIG. 18 shows an embodiment of the slave consisting of end effector 1, TCP 2, parallel kinematic mechanism 3, push rods 4, camera channel 5, shaft 6 and a drive unit 7.

[0084] FIG. 19 shows an enlargement of the embodiment in FIG. 18 with end effector 1, TCP 2, parallel kinematic mechanism 3, push rods 4, camera channel 5, shaft 6.

[0085] FIG. 20 shows the embodiment of a control mean with control means 1, TCP of the control unit 2, base 3 and the drive of the control unit 4.

[0086] FIG. 21 shows the embodiment of a rigid control means with gripping elements 1, 2, fingers 3, base 4 of the control means, fastening element for fastening 5 on the TCP of the control means on the control unit, force sensor elements 6 between grip elements and base of the control means.

[0087] The gripping force of the user is used as the control variable for the closing angle phi of an intracorporal end effector (see, for example, FIGS. 6 to 17) instead of a position measurement of movable elements of the control means. For this purpose, a force sensor system is used in the control means for detecting the gripping force (for example, FIGS. 12 and 22). By adjusting the required gripping force F.sub.G, max for the complete closing of the end effector, the behavior of the end effector can be influenced in the form of a (linear) characteristic phi (F.sub.G) and modified in a manner adapted to the situation (FIGS. 13-17). A haptic sense impression is thereby produced by the correlation of gripping force itself introduced into the user interface and the visually perceived closing angle of the end effector. See FIGS. 7-9.

[0088] In order to generate the haptic feedback, no actuator is necessary in this case since the user generates the force necessary for a haptic sense impression by virtue of its gripping force. A necessary prerequisite is a direct view of the end effector by the user. The fundamental function of this pseudohaptic feedback is known from the realm of virtual reality.

[0089] The force F.sub.G or also F.sub.grip is determined on the control means as shown in FIGS. 11,12.

[0090] In order to ensure haptic feedback of a material in the end effector/gripper, the characteristic curve (FIG. 13-17), which represents the relationship between gripping force at the user interface and the closing angle of the end effector, can be varied (see FIGS. 5-10).

[0091] This takes place as a function of the force required for closing or actuating the end effector. This corresponds to the interaction force F.sub.effector due to the adjusting force balance.

[0092] The variation of the characteristic curve phi (FG) is thereby possible by adding the measured output end effector force

phi=phi (F.sub.G+F.sub.effector) as well as by multiplying the measured end effector force phi=phi (F.sub.GF.sub.effector). The two cases describe a differently strong weighting of the respectively effective end effector force (F.sub.effector). In both cases, the necessary gripping force, which is necessary to achieve a certain closing angle phi, changes. In connection with the visual feedback on the opening of the gripper, an impression is thus obtained for the user of the material at the end effector, since the interaction force F.sub.effector is inter alia material-dependent.

[0093] FIG. 16 shows an exemplary characteristic curve with the influence of different acting end effector gripping forces on the basis of the multiplicative evaluation of the relationship between gripping force and closing angle with the acting end effector gripping force. Characteristic curve 0 shows the course of the coupling characteristic curve without an effective end effector gripping force. Characteristic curves 1 and 2 show the course of the coupling characteristic curve for gripped materials of different stiffnesses. Characteristic curves 3 shows the course of a characteristic curve in which the end effector gripping force is so high that the manipulated variable for the closing angle saturates at the maximum possible useful gripping force.

[0094] FIG. 17 shows, in contrast to FIG. 16, the characteristic curve under influence of different end effector gripping forces on the basis of the additive assessment of the relationship between gripping force and closing angle with the acting end effector gripping force. The characteristic curve 2 describes the intervention on a material which is stiffer in comparison to characteristic curve 1.

[0095] Preliminary tests show that a coupling of the gripping force and the kinematic component via a multiplication provides the better results and thus makes it easier for the user to differentiate between different material properties. Moreover, it is found that scaling factors and calculation methods can be selected depending on the nature of the environment of the end effector in order to get the optimal dynamic of the haptic perception for distinguishing special material parameters.

[0096] A necessary prerequisite for this method is the derivation of the interaction force F.sub.effector between the gripping arms of the end effector (FIGS. 9-10). The dynamic requirements for these measurements are low, since the man's ability to exercise comprises only a small, almost quasi-static range. Therefore, the derivation of the force output values of the actuators and in the end effector is sufficient by integrating a sensor away from the end effector. This not only reduces the dynamic requirements but also the requirements for the space, weight and overload resistance of the sensor used.

[0097] The haptic feedback of the gripping force thus shown is quasi-static and can therefore be unsatisfactory when used for the representation of certain properties, such as the surface texture and the differentiation of materials and the like.

[0098] Therefore, in a further embodiment, this disadvantage is compensated in a simple manner by the integration of a highly dynamic actuator in the control means (piezo, voice coil, eccentric motor, etc.) with very small deflections required. Due to the properties of the human haptic perception, the introduction direction cannot be clearly distinguished in the case of highly dynamic signals, so that haptic feedback, which is felt in several degrees of freedom, can be represented with a one-dimensional movement of the actuator.

[0099] FIG. 22 shows a section as well as the exploded drawing of the embodiment of a control means with grip elements 1, actuators for tactile feedback 2, force sensor elements 3, a base of the control means 4 and a fastening element for TCP of the control means.

[0100] The measurement of the high-frequency signals could be carried out by measuring accelerations with miniaturized acceleration sensors arranged sterilisably in the end effector.

[0101] By comparing teleoperation systems with haptic feedback known from the literature, the invention not only expands the haptically perceptible range, but also reduces the design complexity of the entire control means. By using serially arranged actuators, a frequency distribution is possible for the haptic feedback. Instead of an actuator with a large bandwidth and, at the same time, great deflections in the base of the control means, the high-frequency portion of the haptic feedback is generated by a dynamic actuator with small deflections. In the end effector, the complexity of the sensor system is reduced so that multi-dimensional, highly dynamic force sensors can be replaced by a one-dimensional force sensor system and a multi-dimensional acceleration measurement. The latter is easier to integrate into the end effector since it does not have to be integrated into the main force flow direction. In addition, peripheral requirements for the sensors in terms of dynamics, overload resistance and sterilizable packaging are decreasing.

[0102] FIG. 23 shows one of two parallel kinematic mechanisms of the slave, which is also referred to as a manipulator in the following. Each manipulator has up to six degrees of freedom so that the TCP 1 can be positioned in the space. Furthermore, an end effector 2 attached to the TCP can be rotated about its longitudinal axis 3, angled (deviation) and its closing angle (Phi) can be changed.

[0103] The parallel kinematic mechanism consists of kinematic chains composed of rigid or flexible struts and joints. In general, a large number of solutions are conceivable for the implementation of the joints. Thus, in addition to rigid joints, solid body joints or flexible elements, e.g. Springs, film joints, folding bellows and NiTi wires could be used.

[0104] In order to move the intracorporal manipulator, six motors are preferably installed in the extra-corporal drive unit per manipulator. A different number of motors and gearboxes are conceivable. The movements generated are transmitted via three pushing rods 4 into the intracorporal region. Two active degrees of freedom are transmitted via a push rod in the form of translation q10-q30 and rotation q40-q60. The intracorporal movements are shaped by the parallel kinematic mechanism, which consists of a kinematic main chain 18 and up to four kinematic secondary chains 8, 9, 14, 15, such that a displacement of the push rods leads to a change in the position of the TCP, a rotation of the main chain Q40 rotates the end effector arbitrarily about its longitudinal axis, and a rotation q50 and q60 opens or closes the gripper. This is achieved e.g. by means of corresponding spindles which can also be seen in FIG. 23.

[0105] In detail, the parallel kinematic mechanism consists of a tripod-like substructure composed of the kinematic main chain 18 with four degrees of freedom and two kinematic secondary chains 8, 9 each with six degrees of freedom. These kinematic chains are connected to the main shaft 5 via pivot joints. In order to prevent jamming, these joints are realized as U-shaped clamping elements 6, 7. The rotation of the main chain is routed directly to the end effector via a universal joint located at the base so that the latter can be rotated freely about its longitudinal axis.

[0106] The rotations of the two remaining pushing rods are also transferred via cross joints along the first and second secondary chain, and are finally converted into a respective displacement via a spindle and a slide 10, 11. Via the third and fourth secondary chains 14, 15, each of which has four degrees of freedom, these movements are transmitted to sleds 21, 22 guided on the main shaft. In order to limit the forces occurring within the mechanism, compliances are integrated in the third and fourth secondary chain, in order to prevent jamming of the sled elements, these are also designed as a U-shaped bracket. The friction moment occurring within the rotary joints 12, 13 is dissipated via the secondary chains. For this purpose, the clamping elements 6 and 7 are connected to the elements 21 and 22 by means of a respective pendulum support.

[0107] Each of the displacements generated on the main shaft moves a push rod within the main shaft, this movement being applied to one of the two jaws of the end effector, e.g. by means of a cam disk or a toggle lever 16, 17. The pushing rods are guided by an elongated hole with respect to the main shaft and are locked against twisting by means of a pin. In order to obtain the rotation of the main shaft, the carriage movements 21, 22 produced on the main shaft are transmitted via pivot joints 12, 13 to the pushrod located in the shaft. As a result, the grippers can be opened or closed via a uniform rotation q50 and q60. If the push rods rotate counter-clockwise, the gripper is angled. The described state is invertible.

[0108] The parallel kinematic mechanism described has the following transfer characteristics:

1. The position of the TCP is independent of the rotations q40-q60 and is influenced by the shifts q10-q30 alone.
2. The push rods are arranged in a colinear manner so that the working space in the z direction is limited only by the maximum travel distance of the pushing rods. In the z-direction, a constant translation ratio of 1 results.
3. The longitudinal rotation of the end effector depends solely on the rotation q40.
4. The opening angle and the angle of inclination are mainly determined by the rotations q50 and q60.

[0109] In order to reference the kinematics with respect to the base plate (19), stops are attached (20) at the ends of the push rods.

[0110] The invention is not limited to the above-described embodiments but is intended to be defined by the claims.