AUTONOMOUS CORRECTION OF ALIGNMENT ERROR IN A MASTER-SLAVE ROBOTIC SYSTEM
20180271607 ยท 2018-09-27
Inventors
Cpc classification
Y10S901/31
GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
B25J11/008
PERFORMING OPERATIONS; TRANSPORTING
A61B2034/301
HUMAN NECESSITIES
Y10S901/06
GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
International classification
B25J11/00
PERFORMING OPERATIONS; TRANSPORTING
A61B34/00
HUMAN NECESSITIES
Abstract
A method and apparatus for correcting an alignment error between an end effector of a tool associated with a slave and a master actuator in a master-slave robotic system in which an orientation of the end effector is remotely controlled by an orientation of the master actuator by producing, transmitting, receiving and/or generating control signals: master actuator orientation signals (R.sub.MCURR), end effector orientation signals (R.sub.EENEW) and master-slave misalignment signals (R.sub.). Producing control signals from the master to the slave involves receiving and transmitting an enablement signal when the enablement signal is active and not active. In response, computing the master-slave misalignment signals (R.sub.) and detecting a second difference between the master actuator orientation (R.sub.MCURR) and the end effector orientation (R.sub.EENEW) signals. The processor adjusts the master-slave misalignment signals (R.sub.) to reduce the alignment difference to satisfy an alignment criterion so that subsequent generations of the end effector orientation signals (R.sub.EENEW) cause the tool to be physically aligned with the master within the alignment criterion.
Claims
1. A method of correcting an alignment error between an end effector of a tool associated with a slave and a master actuator associated with a master in a master-slave robotic system in which an orientation of the end effector is remotely controlled by an orientation of the master actuator by producing and transmitting control signals at the master for controlling the slave, the method comprising: causing a processor associated with the master to receive master actuator orientation signals (R.sub.MCURR) representing the orientation of the master actuator relative to a master reference frame; causing the processor to generate end effector orientation signals (R.sub.EENEW) representing the end effector orientation relative to a slave reference frame, in response to: said master actuator orientation signals (R.sub.MCURR); master base orientation signals (R.sub.MBASE) representing previous-saved values of said master actuator orientation signals (R.sub.MCURR); and slave base orientation signals (R.sub.EEBASE) representing previous-saved values of said end effector orientation signals (R.sub.EENEW); causing the processor to produce control signals based on said end effector orientation signals, for transmission from the master to the slave; causing the processor to receive an enablement signal for selectively enabling said control signals to be transmitted from the master to the slave whereby the master transmits said control signals to the slave when said enablement signal is active and does not transmit said control signals to the slave when said enablement signal is not active and such that when said enablement signal is active, changes in the orientation of the master actuator cause corresponding changes in the orientation of the end effector and such that when said enablement signal is not active, changes in the orientation of the master actuator do not cause corresponding changes in the orientation of the end effector; when said enablement signal transitions from said not active state to said active state, causing the processor to: save said values of said master actuator orientation signals (R.sub.MCURR) as said master base orientation signals (R.sub.MBASE) to create said previous-saved values of said master actuator orientation signals (R.sub.MCURR); save said values of said end effector orientation signals (R.sub.EENEW) as said slave base orientation signals (R.sub.EEBASE) to create said previous-saved values of said end effector orientation signals (R.sub.EENEW); causing the processor to detect a difference, between the master actuator orientation signals (R.sub.MCURR) and the end effector orientation signals (R.sub.EENEW), the difference representing a difference in physical alignment between the tool and the master relative to their respective reference frames; in response to detecting said difference, causing the processor to adjust said saved slave base orientation signals (R.sub.EEBASE) to ultimately have the same values as said saved master base orientation (R.sub.MBASE) values so that subsequent generations of said end effector orientation signals (R.sub.EENEW) cause said control signals to cause said tool to satisfy an alignment criterion.
2. The method of claim 1 further comprising causing said processor to autonomously adjust said saved slave base orientation (R.sub.EEBASE) signals.
3. The method of claim 1 wherein causing the processor to detect a difference in orientation signals between the master actuator orientation signals (RMCURR) and the end effector orientation signals (R.sub.EENEW) comprises computing a difference rotation matrix that carries the end effector orientation signals into the master actuator orientation signals according to the relation:
R.sub.EE.sub._.sub.TO.sub._.sub.MASTER=R.sub.EENEW.sup.1R.sub.MCURR Where: R.sub.EENEW.sup.1 is a inverse matrix of the end effector orientation signal R.sub.EENEW represented by a 33 matrix; and R.sub.M.sub._.sub.CURR is the master actuator orientation signal at a current time step represented by a 33 matrix.
4. The method of claim 3 further comprising causing the processor to determine an angle of rotation associated with the difference rotation matrix according to the relation:
.sub.EE.sub._.sub.TO.sub._.sub.MASTER=a cos(0.5 trace(R.sub.EE.sub._.sub.TO.sub._.sub.MASTER)1).
5. The method of claim 4 further comprising causing the processor to determine whether the angle of rotation associated with the difference rotation matrix (R.sub.EE.sub._.sub.TO.sub._.sub.MASTER) meets a criterion.
6. The method of claim 5 further comprising causing the processor to determine an angular speed of rotation of the difference rotation matrix (R.sub.EE.sub._.sub.TO.sub._.sub.MASTER) which represents the difference in rotation between the previous-saved master handle orientation signals and the current master handle orientation signals.
7. The method of claim 6 further comprising causing the processor to determine whether the angular speed of rotation of the difference rotation matrix meets a criterion.
8. The method of claim 7 further comprising causing the processor to determine a misalignment axis and an incremental correction angle by which the slave base orientation signals (R.sub.EEBASE) is to be rotated about the misalignment axis.
9. The method of claim 8 further comprising causing the processor to: generate a correction rotation matrix for adjusting the current slave base orientation (R.sub.EEBASE) signals by the incremental correction angle in a misalignment plane; and adjust said current slave base orientation (R.sub.EEBASE) signals with the correction matrix.
10. The method of claim 9 further comprising causing the processor to produce new end effector orientation signals (R.sub.EENEW) using said adjusted current slave base orientation (R.sub.EEBASE) signals and causing the processor to generate said control signals using said new end effector signals (R.sub.EENEW).
11. A method of correcting an alignment error between an end effector of a tool associated with a slave and a master actuator associated with a master in a master-slave robotic system in which an orientation of the end effector is remotely controlled by an orientation of the master actuator by producing and transmitting control signals at the master for controlling the slave, the method comprising: causing a processor associated with the master to receive master actuator orientation signals (R.sub.MCURR) representing the orientation of the master actuator relative to a master reference frame; causing the processor to generate end effector orientation signals (R.sub.EENEW) representing the end effector orientation relative to a slave reference frame, in response to: said master actuator orientation signals (R.sub.MCURR); master-slave misalignment signals (R.sub.), representing a product of previously saved values of said master actuator orientation signals (R.sub.MCURR) and said end effector orientation signals (R.sub.EENEW); causing the processor to produce control signals based on said end effector orientation signals, for transmission from the master to the slave; causing the processor to receive an enablement signal for selectively enabling said control signals to be transmitted from the master to the slave whereby the master transmits said control signals to the slave when said enablement signal is active and does not transmit said control signals to the slave when said enablement signal is not active and such that when said enablement signal is active, changes in the orientation of the master actuator cause corresponding changes in the orientation of the end effector and such that when said enablement signal is not active, changes in the orientation of the master actuator do not cause corresponding changes in the orientation of the end effector; when said enablement signal transitions from said not active state to said active state, causing the processor to: compute the master-slave misalignment signals (R.sub.) as a difference between the master actuator orientation signals (R.sub.MCURR) and the end effector orientation signals (R.sub.EENEW), the misalignment signals representing a difference in physical alignment between the tool and the master relative to their respective reference frames; causing the processor to detect a second difference, between the master actuator orientation signals (R.sub.MCURR) and the end effector orientation signals (R.sub.EENEW) the second difference representing the difference in physical alignment between the tool and the master relative to their respective reference frames; and in response to detecting said second difference, causing the processor to adjust said master-slave misalignment signals (R.sub.) to reduce the alignment difference to satisfy an alignment criterion so that subsequent generations of said end effector orientation signals (R.sub.EENEW) cause said control signals to cause said tool to be physically aligned with said master within the alignment criterion.
12. The method of claim 11 further comprising causing the processor to: generate a correction rotation matrix for adjusting the current master-slave misalignment (R) signals by an incremental correction angle in the misalignment plane; and adjust said current master-slave misalignment (R.sub.) signals with the correction matrix.
13. The method of claim 12 further comprising causing the processor to produce new end effector orientation signals (R.sub.EENEW) using said adjusted master-slave misalignment (R.sub.) signals and causing the processor to generate said control signals using said new end effector orientation signals (R.sub.EENEW).
14. A computer readable medium encoded with codes for directing a processor to execute the method of claim 1.
15. An apparatus for correcting an alignment error between an end effector of a tool associated with a slave and a master actuator associated with a master in a master-slave robotic system in which an orientation of the end effector is remotely controlled by an orientation of the master actuator by producing and transmitting control signals at the master for controlling the slave, the apparatus comprising: means associated with the master for receiving master actuator orientation signals (R.sub.MCURR) representing the orientation of the master actuator relative to a master reference frame; means for generating end effector orientation signals (R.sub.EENEW) representing the end effector orientation relative to a slave reference frame, in response to: said master actuator orientation signals (R.sub.m); master base orientation signals (R.sub.MBASE) representing previous-saved values of said master actuator orientation signals (R.sub.MCURR); and slave base orientation signals (R.sub.EEBASE) representing previous-saved values of said end effector orientation signals (R.sub.EENEW); means for producing control signals based on said end effector orientation signals, for transmission from the master to the slave; means for receiving an enablement signal for selectively enabling said control signals to be transmitted from the master to the slave whereby the master transmits said control signals to the slave when said enablement signal is active and does not transmit said control signals to the slave when said enablement signal is not active and such that when said enablement signal is active, changes in the orientation of the master actuator cause corresponding changes in the orientation of the end effector and such that when said enablement signal is not active, changes in the orientation of the master actuator do not cause corresponding changes in the orientation of the end effector; means responsive to a transition of the enablement signal from said not active state to said active state, for: saving said values of said master actuator orientation signals (R.sub.MCURR) as said master base orientation signals (R.sub.MBASE) to create said previous-saved values of said master actuator orientation signals (R.sub.MCURR); saving said values of said end effector orientation signals (R.sub.EENEW) as said slave base orientation signals (R.sub.EEBASE) to create said previous-saved values of said end effector orientation signals (R.sub.EENEW); means for detecting a difference, between the master actuator orientation signals (R.sub.MCURR) and the end effector orientation signals (R.sub.EENEW), the difference representing a difference in physical alignment between the tool and the master relative to their respective reference frames; means for adjusting said saved slave base orientation signals (R.sub.EEBASE) to ultimately have the same values as said saved master base orientation (R.sub.MBASE) values so that subsequent generations of said end effector orientation signals (R.sub.EENEW) cause said control signals to cause said tool to satisfy an alignment criterion, in response to detecting said difference.
16. The apparatus of claim 15 further comprising means for autonomously adjusting said saved slave base orientation (R.sub.EEBASE) signals.
17. The apparatus of claim 15 wherein said means for detecting a difference in orientation signals between the master actuator orientation signals (R.sub.MCURR) and the end effector orientation signals (R.sub.EENEW) includes: means for computing a difference rotation matrix that carries the end effector orientation signals into the master actuator orientation signals according to the relation:
R.sub.EE.sub._.sub.TO.sub._.sub.MASTER=R.sub.EE.sub._.sub.NEW.sup.1R.sub.M.sub._.sub.CURR Where: R.sub.EE.sub._.sub.NEW.sup.1 is a inverse matrix of the end effector orientation signal R.sub.EE.sub._.sub.NEW represented by a 33 matrix; and R.sub.M.sub._.sub.CURR is the master actuator orientation signal at a current time step represented by a 33 matrix.
18. The apparatus of claim 17 further comprising means for determining an angle of rotation associated with the difference rotation matrix according to the relation:
.sub.EE.sub._.sub.TO.sub._.sub.MASTER=a cos(0.5 trace(R.sub.EE.sub._.sub.TO.sub._.sub.MASTER)1).
19. The apparatus of claim 18 further comprising means for determining whether the angle of rotation associated with the difference rotation matrix (R.sub.EE.sub._.sub.TO.sub._.sub.MASTER) meets a criterion.
20. The apparatus of claim 19 further comprising means for determining an angular speed of rotation of the difference rotation matrix (R.sub.EE.sub._.sub.TO.sub._.sub.MASTER) R.sub.DIFF which represents the difference in rotation between the previous-saved master handle orientation signals and the current master handle orientation signals.
21. The apparatus of claim 20 further comprising means for determining whether the angular speed of rotation of the difference rotation matrix (R.sub.DIFF) meets a criterion.
22. The apparatus of claim 21 further comprising means for determining a misalignment axis and an incremental correction angle by which the slave base orientation signals (R.sub.EEBASE) is to be rotated about the misalignment axis.
23. The apparatus of claim 22 further comprising: means for generating a correction rotation matrix for adjusting the current slave base orientation (R.sub.EEBASE) signals by the incremental correction angle in a misalignment plane; and means for adjusting said current slave base orientation (R.sub.EEBASE) signals with the correction matrix.
24. The apparatus of claim 23 further comprising means for producing new end effector orientation signals (R.sub.EENEW) using said adjusted current slave base orientation (R.sub.EEBASE) signals and means for generating said control signals using said new end effector signals (R.sub.EENEW).
25. An apparatus for correcting an alignment error between an end effector of a tool associated with a slave and a master actuator associated with a master in a master-slave robotic system in which an orientation of the end effector is remotely controlled by an orientation of the master actuator by producing and transmitting control signals at the master for controlling the slave, the apparatus comprising: means associated with the master for receiving master actuator orientation signals (R.sub.MCURR) representing the orientation of the master actuator relative to a master reference frame; means for generating end effector orientation signals (R.sub.EENEW) representing the end effector orientation relative to a slave reference frame, in response to: said master actuator orientation signals (R.sub.MCURR); master-slave misalignment signals (R.sub.), representing a product of previously saved values of said master actuator orientation signals (R.sub.MCURR) and said end effector orientation signals (R.sub.EENEW); means for producing control signals based on said end effector orientation signals, for transmission from the master to the slave; means for receiving an enablement signal for selectively enabling said control signals to be transmitted from the master to the slave whereby the master transmits said control signals to the slave when said enablement signal is active and does not transmit said control signals to the slave when said enablement signal is not active and such that when said enablement signal is active, changes in the orientation of the master actuator cause corresponding changes in the orientation of the end effector and such that when said enablement signal is not active, changes in the orientation of the master actuator do not cause corresponding changes in the orientation of the end effector; means responsive to a transition of the enablement signal from said not active state to said active state, for: computing the master-slave misalignment signals (R.sub.) as a difference between the master actuator orientation signals (R.sub.MCURR) and the end effector orientation signals (R.sub.EENEW), the misalignment signals representing a difference in physical alignment between the tool and the master relative to their respective reference frames; means for detecting a second difference, between the master actuator orientation signals (R.sub.MCURR) and the end effector orientation signals (R.sub.EENEW) the second difference representing the difference in physical alignment between the tool and the master relative to their respective reference frames; and means responsive to detecting said second difference, for adjusting said master-slave misalignment signals (R.sub.) to reduce the alignment difference to satisfy an alignment criterion so that subsequent generations of said end effector orientation signals (R.sub.EENEW) cause said control signals to cause said tool to be physically aligned with said master within the alignment criterion.
26. The apparatus of claim 25 further comprising: means for generating a correction rotation matrix for adjusting the current master-slave misalignment (R.sub.) signals by an incremental correction angle in the misalignment plane; and means for adjusting said current master-slave misalignment (R.sub.) signals with the correction matrix.
27. The apparatus of claim 26 further comprising: means for producing new end effector orientation signals (R.sub.EENEW) using said adjusted master-slave misalignment (R.sub.) signals; and means for generating said control signals using said new end effector orientation signals (R.sub.EENEW).
28. An apparatus for correcting an alignment error between an end effector of a tool associated with a slave and a master actuator associated with a master in a master-slave robotic system in which an orientation of the end effector is remotely controlled by an orientation of the master actuator by producing and transmitting control signals at the master for controlling the slave, the apparatus comprising: a processor associated with the master operably configured to: receive master actuator orientation signals (R.sub.MCURR) representing the orientation of the master actuator relative to a master reference frame; generate end effector orientation signals (R.sub.EENEW) representing the end effector orientation relative to a slave reference frame, in response to: said master actuator orientation signals (R.sub.m); master base orientation signals (R.sub.MBASE) representing previous-saved values of said master actuator orientation signals (R.sub.MCURR); and slave base orientation signals (R.sub.EEBASE) representing previous-saved values of said end effector orientation signals (R.sub.EENEW); produce control signals based on said end effector orientation signals, for transmission from the master to the slave; receive an enablement signal for selectively enabling said control signals to be transmitted from the master to the slave whereby the master transmits said control signals to the slave when said enablement signal is active and does not transmit said control signals to the slave when said enablement signal is not active and such that when said enablement signal is active, changes in the orientation of the master actuator cause corresponding changes in the orientation of the end effector and such that when said enablement signal is not active, changes in the orientation of the master actuator do not cause corresponding changes in the orientation of the end effector; when said enablement signal transitions from said not active state to said active state: save said values of said master actuator orientation signals (R.sub.MCURR) as said master base orientation signals (R.sub.MBASE) to create said previous-saved values of said master actuator orientation signals (R.sub.MCURR); save said values of said end effector orientation signals (R.sub.EENEW) as said slave base orientation signals (R.sub.EEBASE) to create said previous-saved values of said end effector orientation signals (R.sub.EENEW); detect a difference, between the master actuator orientation signals (R.sub.MCURR) and the end effector orientation signals (R.sub.EENEW), the difference representing a difference in physical alignment between the tool and the master relative to their respective reference frames; adjust said saved slave base orientation signals (R.sub.EEBASE) to ultimately have the same values as said saved master base orientation (R.sub.MBASE) values so that subsequent generations of said end effector orientation signals (R.sub.EENEW) cause said control signals to cause said tool to satisfy an alignment criterion, in response to detecting said difference.
29. The apparatus of claim 28 wherein said processor is further configured to autonomously adjust said saved slave base orientation (R.sub.EEBASE) signals.
30. The apparatus of claim 28 wherein said processor is further configured to detect said difference in orientation signals between the master actuator orientation signals (R.sub.MCURR) and the end effector orientation signals (R.sub.EENEW) by computing a difference rotation matrix that carries the end effector orientation signals into the master actuator orientation signals according to the relation:
R.sub.EE.sub._.sub.TO.sub._.sub.MASTER=R.sub.EE.sup.+NEW.sup.1R.sub.M.sub._.sub.CURR Where: R.sub.EE.sub._.sub.NEW.sup.1 is a inverse matrix of the end effector orientation signal R.sub.EE.sub._.sub.NEW represented by a 33 matrix; and R.sub.M.sub._.sub.CURR is the master actuator orientation signal at a current time step represented by a 33 matrix.
31. The apparatus of claim 30 wherein said processor is further configured to determine an angle of rotation associated with the difference rotation matrix according to the relation:
.sub.EE.sub._.sub.TO.sub._.sub.MASTER=a cos(0.5 trace(R.sub.EE.sub._.sub.TO.sub._.sub.MASTER)1).
32. The apparatus of claim 31 wherein said processor is further configured to determine whether the angle of rotation associated with the difference rotation matrix (R.sub.EE.sub._.sub.TO.sub._.sub.MASTER) meets a criterion.
33. The apparatus of claim 32 wherein said processor is further configured to determine an angular speed of rotation of the difference rotation matrix (R.sub.EE.sub._.sub.TO.sub._.sub.MASTER) R.sub.DIFF which represents the difference in rotation between the previous-saved master handle orientation signals and the current master handle orientation signals.
34. The apparatus of claim 33 wherein said processor is further configured to determine whether the angular speed of rotation of the difference rotation matrix (R.sub.DIFF) meets a criterion.
35. The apparatus of claim 34 wherein said processor is further configured to determine a misalignment axis and an incremental correction angle by which the slave base orientation signals (R.sub.EEBASE) is to be rotated about the misalignment axis.
36. The apparatus of claim 35 wherein said processor is further configured to: generate a correction rotation matrix for adjusting the current slave base orientation (R.sub.EEBASE) signals by the incremental correction angle in a misalignment plane; and adjust said current slave base orientation (R.sub.EEBASE) signals with the correction matrix.
37. The apparatus of claim 36 wherein said processor is further configured to produce new end effector orientation signals (R.sub.EENEW) using said adjusted current slave base orientation (R.sub.EEBASE) signals and generate said control signals using said new end effector signals (R.sub.EENEW).
38. An apparatus for correcting an alignment error between an end effector of a tool associated with a slave and a master actuator associated with a master in a master-slave robotic system in which an orientation of the end effector is remotely controlled by an orientation of the master actuator by producing and transmitting control signals at the master for controlling the slave, the apparatus comprising: a processor associated with the master operably configured to: receive master actuator orientation signals (R.sub.MCURR) representing the orientation of the master actuator relative to a master reference frame; generate end effector orientation signals (R.sub.EENEW) representing the end effector orientation relative to a slave reference frame, in response to: said master actuator orientation signals (R.sub.MCURR); master-slave misalignment signals (R.sub.), representing a product of previously saved values of said master actuator orientation signals (R.sub.MCURR) and said end effector orientation signals (R.sub.EENEW); produce control signals based on said end effector orientation signals, for transmission from the master to the slave; receive an enablement signal for selectively enabling said control signals to be transmitted from the master to the slave whereby the master transmits said control signals to the slave when said enablement signal is active and does not transmit said control signals to the slave when said enablement signal is not active and such that when said enablement signal is active, changes in the orientation of the master actuator cause corresponding changes in the orientation of the end effector and such that when said enablement signal is not active, changes in the orientation of the master actuator do not cause corresponding changes in the orientation of the end effector; in response to a transition of the enablement signal from said not active state to said active state: compute the master-slave misalignment signals (R.sub.) as a difference between the master actuator orientation signals (R.sub.MCURR) and the end effector orientation signals (R.sub.EENEW), the misalignment signals representing a difference in physical alignment between the tool and the master relative to their respective reference frames; detect a second difference, between the master actuator orientation signals (R.sub.MCURR) and the end effector orientation signals (R.sub.EENEW) the second difference representing the difference in physical alignment between the tool and the master relative to their respective reference frames; and responsive to detecting said second difference, adjusting said master-slave misalignment signals (R.sub.) to reduce the alignment difference to satisfy an alignment criterion so that subsequent generations of said end effector orientation signals (R.sub.EENEW) cause said control signals to cause said tool to be physically aligned with said master within the alignment criterion.
39. The apparatus of claim 38 wherein said processor is further configured to: generate a correction rotation matrix for adjusting the current master-slave misalignment (R.sub.) signals by an incremental correction angle in the misalignment plane; and adjust said current master-slave misalignment (R.sub.) signals with the correction matrix.
40. The apparatus of claim 39 wherein said processor is further configured to: produce new end effector orientation signals (R.sub.EENEW) using said adjusted master-slave misalignment (R.sub.) signals; and generate said control signals using said new end effector orientation signals (R.sub.EENEW).
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0012] In drawings which illustrate embodiments of the invention,
[0013]
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
DETAILED DESCRIPTION
[0022] Referring to
[0023] In the embodiment shown, the master subsystem 52 comprises a workstation 56 having first and second input devices 58 and 60 and a viewer 62 in communication with a master apparatus 64 comprising at least one processor. The first and second input devices 58 and 60 are operable to be actuated by respective hands of an operator such as a surgeon, for example, who will perform the laparoscopic surgery by manipulating the first and second input devices of the master subsystem 52 to control corresponding laparoscopic tools 66 and 67 on the slave subsystem 54.
[0024] The viewer 62 may include an LCD display 68, for example, for displaying images acquired by a camera 70 on the slave subsystem 54, to enable the user to see the RECTIFIED SHEET (RULE 91) laparoscopic tools 66 and 67 inside the patient while manipulating the first and second input devices 58 and 60 to cause the tools to move in desired ways to perform the surgery. The first and second input devices 58 and 60 produce position and orientation signals that are received by the master apparatus 64 and the master apparatus produces slave control signals that are transmitted by wires 72 or wirelessly, for example, from the master subsystem 52 to the slave subsystem 54.
[0025] The slave subsystem 54 includes a slave computer 74 that receives the slave control signals from the master subsystem 52 and produces motor control signals that control motors 76 on a drive mechanism of a tool controller 78 of the slave subsystem 54, to extend and retract wires (not shown) of respective tool positioning devices 79 and 81 to position and to rotate the tools 66 and 67. Exemplary tool positioning devices and tools for this purpose are described in PCT/CA2013/001076, which is incorporated herein by reference. The tool positing devices 79 and 81 extend through an insertion tube 61, a portion of which is inserted through a small opening 63 in the patient to position end effectors 71 and 73 of the tools 66 and 67 inside the patient, to facilitate the surgery.
[0026] In the embodiment shown, the workstation 56 has a support 80 having a flat surface 82 for supporting the first and second input devices 58 and 60 in positions that are comfortable to the operator whose hands are actuating the first and second input devices 58 and 60.
[0027] In the embodiment shown, the slave subsystem 54 includes a cart 84 in which the slave computer 74 is located. The cart 84 has an articulated arm 86 mechanically connected thereto, with a tool holder mount 88 disposed at a distal end of the articulated arm.
[0028] In the embodiment shown, the first and second input devices 58 and 60 are the same, but individually adapted for left and right hand use respectively. In this embodiment, each input device 58 and 60 is an Omega.7 haptic device available from Force Dimension, of Switzerland. For simplicity, only input device 60 will be described, it is being understood that input device 58 operates in the same way.
[0029] Referring to
[0030] The arms 94, 96, 98 facilitate translational movement of the handle 102 and hence the handle position 104, in space, and confine the movement of the handle position within a volume in space. This volume may be referred to as the handle translational workspace.
[0031] The handle 102 is mounted on a gimbal mount 106 having a pin 108. The base plate 90 has a calibration opening 110 for receiving the pin 108. When the pin 108 is received in the opening 110, the input device 60 is in a calibration position that is defined relative to a fixed master Cartesian reference frame comprising orthogonal axes x.sub.r, y.sub.r, z.sub.r generally in the center of the handle translational workspace. In the embodiment shown, this master reference frame has an x.sub.rz.sub.r plane parallel to the flat surface 82 and a y.sub.r axis perpendicular to the flat surface. In the embodiment shown, the z.sub.r axis is parallel to the flat surface 82 and is coincident with an axis 112 passing centrally through the control unit 92 so that pushing and pulling the handle 102 toward and away from the center of the control unit 92 along the axis 112 in a direction parallel to the flat surface 82 is movement in the z.sub.r direction.
[0032] The control unit 92 has sensors (not shown) that sense the positions of the arms 94, 96, 98 and the rotation of the handle 102 and produces signals representing the handle position 104 (i.e. the center of the handle 102) in the workspace and the rotational orientation of the handle 102 relative to the fixed master reference frame x.sub.r, y.sub.r, z.sub.r. In this embodiment, these position and orientation signals are transmitted on wires 111 of a USB bus to the master apparatus 64. More particularly, the control unit 92 produces current handle position signals and current handle orientation signals that represent the current position and orientation of the handle 102 by a current handle position vector
[0033] For example, the current handle position vector
where x.sub.1, y.sub.1, and z.sub.1 represent coordinates of the handle position 104 within the handle workspace relative to the fixed master reference frame, x.sub.r, y.sub.r, z.sub.r.
[0034] The current handle rotation matrix R.sub.MCURR is a 33 matrix where the columns of the matrix represent the axes of the handle reference frame x.sub.1, y.sub.1, z.sub.1 written in the fixed master reference frame x.sub.r, y.sub.r, z.sub.r. R.sub.MCURR thus defines the current rotational orientation of the handle 102 in the handle translational workspace, relative to the x.sub.r, y.sub.r, z.sub.r master reference frame.
[0035] The current handle position vector P.sub.MCURR and current handle rotation matrix R.sub.MCURR are transmitted in the current handle position and orientation signals on wires 111 of the USB bus, for example, to the master apparatus 64 in
[0036] In addition, in the embodiment shown, referring to
[0037] Referring now to
[0038] In the embodiment shown, end effector 73 includes a pair of gripper jaws. Orthogonal axes x.sub.2, y.sub.2 and z.sub.2 of an end effector Cartesian reference frame have an origin on the end effector axis, for example, at the intersection at the tip of the gripper jaws of the end effector 73. The origin of the end effector reference frame may be referred to as the end effector position 150 relative to the fixed slave reference frame x.sub.s, y.sub.s, z.sub.s. Due to the mobility of the tool positioning device 81 and the mobility of the end effector 73 itself, the end effector position 150 can be placed at discrete positions within a volume in space. This volume may be referred to as the end effector translational workspace.
[0039] New end effector positions and end effector orientations are calculated by an end effector position and orientation calculation block 116 encoded in the master apparatus 64 shown in
[0040] For example, the new end effector position vector
where x.sub.2, y.sub.2, and z.sub.2 represent coordinates of the end effector position 150 within the end effector translational workspace relative to the x.sub.s, y.sub.s, z.sub.s fixed slave reference frame.
[0041] The end effector rotation matrix R.sub.EENEW is a 33 matrix where the columns of the R.sub.EENEW matrix represent the axes of the end effector reference frame x.sub.2, y.sub.2, z.sub.2 written in the fixed slave reference frame x.sub.s, y.sub.s, z.sub.s. R.sub.EENEW thus defines a new orientation of the end effector 73 in the end effector translational workspace, relative to the x.sub.s, y.sub.s, z.sub.s reference frame.
[0042] Referring back to
[0043] Generally, the end effector position and orientation calculation block 116 includes codes that direct the master apparatus 64 to produce the new end effector position and orientation signals, referred to herein as
[0044] The kinematics block 118 includes codes that direct the master apparatus 64 to produce configuration variables in response to the newly calculated end effector position and orientation signals
[0045] The motion control block 120 includes codes that direct the master apparatus 64 to produce the slave control signals, in response to the configuration variables.
[0046] The base setting block 216 is executed asynchronously, whenever the enablement signal transitions from an inactive state to an active state, such as when the user releases the footswitch 170. The base setting block 216 directs the master apparatus 64 to set new reference positions and orientations for the handle 102 and end effector 73, respectively as will be described below.
[0047] Referring back to
[0048] Referring to
[0049] The kinematics block 118 receives newly calculated end effector position and orientation signals
[0050] Referring to
[0051] Referring to
[0052] The s-segment 130 extends from the first position 330 to a third position 334 defined as an origin of a third reference frame having axes x.sub.5, y.sub.5, and z.sub.5 and is capable of assuming a smooth S-shape when control wires (not shown) inside the s-segment 130 are pushed and pulled. The s-segment 130 has a mid-point at a second position 332, defined as the origin of a second position reference frame having axes x.sub.4, y.sub.4, z.sub.4. The s-segment 130 has a length L.sub.1, which in the embodiment shown may be about 65 mm, for example.
[0053] The distal segment 132 extends from the third position 334 to a fourth position 336 defined as an origin of a fourth reference frame having axes x.sub.6, y.sub.6, z.sub.6. The distal segment 132 has a length L.sub.2, which in the embodiment shown may be about 23 mm, for example.
[0054] The tool 67 also has an end effector length, which in the embodiment shown is a gripper length L.sub.3 that extends from the fourth position 336 to the end effector position 150 defined as the origin of axes x.sub.2, y.sub.2, and z.sub.2. The gripper length L.sub.3, in this embodiment, may be about 25 mm, for example. The slave base position 128, first position 330, second position 332, third position 334, fourth position 336 and end effector position 150 may collectively be referred to as tool reference positions.
[0055] As explained in PCT/CA2013/001076, hereby incorporated herein by reference in its entirety, by pushing and pulling on certain control wires inside the tool positioning devices 79 and 81, the s-segment 130 can be bent into any of various degrees of an S-shape, from straight as shown in
[0056] In addition, the distal segment 132 lies in a second bend plane containing the third position 334 and the fourth position 336. The second bend plane is at an angle .sub.dist to the x.sub.s-z.sub.s plane of the fixed slave reference frame. The distal segment 132 is bent in the second bend plane at an angle .sub.dist. Thus, by pushing and pulling the control wires within the tool positioning device 81, the fourth position 336 can be placed within another volume in space. This volume may be referred to as the distal workspace. The combination of the s-segment workspace plus the distal workspace can be referred to as the tool positioning device workspace, as this represents the total possible movement of the tools 66 and 67 as effected by the respective tool positioning devices 79 and 81.
[0057] The distance between the fourth position 336 and the end effector position 150 is the distance between the movable portion of the distal segment 132 and the tip of the gripper end effector 73 in the embodiment shown, i.e. the length L.sub.3. Generally, the portion of the gripper between the fourth position 336 and the end effector position 150 (L.sub.3) will be unbendable.
[0058] In the embodiment shown, the end effector 73 is a gripper jaw tool that is rotatable about the z.sub.2 axis in the x.sub.2-y.sub.2 plane of the end effector reference frame, the angle of rotation being represented by an angle relative to the positive x.sub.2 axis. Finally, the gripper jaws may be at any of varying degrees of openness from fully closed to fully open (as limited by the hinge). The varying degrees of openness may be defined as the gripper.
[0059] In summary therefore, the configuration variables provided by the kinematic block 118 codes are as follows: [0060] q.sub.ins: represents a distance from the slave base position 128 defined by axes x.sub.s, y.sub.s, and z.sub.s to the first position 330 defined by axes x.sub.3, y.sub.3 and z.sub.3 where the s-segment 130 of the tool positioning device 81 begins; [0061] .sub.prox: represents a first bend plane in which the s-segment 130 is bent relative to the x.sub.s-y.sub.s plane of the fixed slave reference frame; [0062] .sub.prox: represents an angle at which the first and second sections 320 and 322 of the s-segment 130 is bent in the first bend plane;
[0063] .sub.dist: represents a second bend plane in which the distal segment 132 is bent relative to the x.sub.s-y.sub.s plane of the fixed slave reference frame;
[0064] .sub.dist: represents an angle through which the distal segment 132 is bent in the second bend; [0065] : represents a rotation of the end effector 73 about axis z.sub.2; and [0066] Gripper: represents a degree of openness of the gripper jaws of the end effector 73. (This is a value which is calculated in direct proportion to a signal produced by an actuator (not shown) on the handle 102 indicative of an amount of pressure the operator exerts by squeezing the handle).
[0067] To calculate the configuration variables, it will first be recalled that the end effector rotation matrix R.sub.EENEW is a 33 matrix:
[0068] Since the last column of R.sub.EENEW is the z-axis of the end effector reference frame written relative to the fixed slave reference frame x.sub.s, y.sub.s and z.sub.s, the values .sub.dist, .sub.dist, and associated with the distal segment 132 can be calculated according to the relations:
[0069] These values can then be used to compute the locations of the third position 334, the fourth position 336, and the end effector position 150 relative to the fixed slave base position 128. The locations may be expressed in terms of vectors
[0070] where: [0071] is a unit vector in the x direction; [0072]
[0074] Once the vector from the fixed slave base position 128 to the third position 334 (
[0075] The ratio of (8b) and (8a) gives
.sub.prox=a tan 2(
[0076] where and
[0077] A closed form solution cannot be found for .sub.prox, thus .sub.prox must be found with a numerical equation solution to either of equations (8a) or (8b). A Newton-Raphson method, being a method for iteratively approximating successively better roots of a real-valued function, may be employed, for example. The Newton-Raphson method can be implemented using the following equations:
[0078] where is the unit vector in the x direction.
[0079] The equation (10) is equation (8a) rewritten in the form f(.sub.prox)=0. The Newton-Raphson method tends to converge very quickly because in the range 0<.sub.prox<, the function has a large radius of curvature and has no local stationary points. Following the Newton-Raphson method, successive improved estimates of .sub.prox can be made iteratively to satisfy equation (10) using the following relationship:
[0080] Finally, upon determination of .sub.prox, the following equation can be used to find q.sub.ins,
[0081] where: [0082]
[0084] The codes in the kinematics block 118 shown in
[0085] It will be appreciated that configuration variables are produced for each end effector 71 and 73 and therefore in the embodiment shown, two sets of configuration variables which will be referred to as left and right configuration variables respectively are produced and forwarded or otherwise made available to the motion control block 120.
[0086] Referring to
[0087] Referring to
[0088] Initially, therefore: [0089]
[0091] In other words, the master base reference frame and the handle reference frame coincide at startup.
[0092] Thereafter, the master base position vector
[0093] Referring to
[0094] Initially, therefore: [0095]
[0097] In other words, the slave base reference frame and the end effector reference frame coincide at startup.
[0098] The slave base position
[0099] Referring to
[0100] To produce new end effector position signals
and
R
[0101] Where:
[0110] The following describes how the master apparatus 64 is controlled by the codes in the end effector position and orientation calculation block 116 to effect autonomous alignment of the orientation of the end effector 73 with the handle 102 after clutching and to effect autonomous alignment of the z-axes of the handle 102 and end effector 73 for wrist roll management.
[0111] Referring to
R.sub.EE.sub._.sub.TO.sub._.sub.MASTER=R.sub.EENEW.sup.1R.sub.MCURR
[0112] Where: R.sub.EENEW.sup.1 is the inverse matrix of the end effector rotation matrix R.sub.EENEW represented by a 33 matrix stored in the eighth store 164 in
[0114] Then, block 206 directs the master apparatus 64 to compute an angle of rotation associated with R.sub.EE.sub._.sub.TO.sub._.sub.MASTER (.sub.EE.sub._.sub.TO.sub._.sub.MASTER) by the relation:
.sub.EE.sub._.sub.TO.sub._.sub.MASTER=a cos(0.5 trace(R.sub.EE.sub._.sub.TO.sub._.sub.MASTER)1)
[0115] This angle of rotation (.sub.EE.sub._.sub.TO.sub._.sub.MASTER) represents the alignment difference between the orientation of the handle 102 and the newly calculated end effector orientation.
[0116] In a special case, applicable to the embodiment described here, it is desirable that to be aligned, only the z-axes of the reference frames described by R.sub.EENEW and R.sub.MCURR be coincident. In this case the handle 102 and the end effector 73 point in the same direction relative to their respective fixed reference frames (x.sub.r,y.sub.r,z.sub.r and x.sub.s,y.sub.s,z.sub.s, respectively) and the roll about their z-axis is not considered.
[0117] In this special case therefore, blocks 204 and 206 shown in
.sub.EE.sub._.sub.TO.sub._.sub.MASTER=a cos(R.sub.EENEW(1,3)*R.sub.M.sub._.sub.CURR(1,3)+R.sub.EENEW(2,3)*R.sub.M.sub._.sub.CURR(2,3)+R.sub.EENEW(3,3)*R.sub.MCURR(3,3)) [0118] Where (i,j) represent matrix row (i) and column (j) indices.
[0119] This computation represents the angle obtained from the dot product of the z-axes of the handle reference frame and end effector reference frame.
[0120] Referring back to
[0121] After executing either block 206 or block 208, the angle of rotation by which the handle 102 and end effector 73 are out of alignment, i.e. the alignment error, is given by .sub.EE.sub._.sub.TO.sub._.sub.MASTER
[0122] The master apparatus 64 is then directed to block 210 which causes it to determine whether the alignment error .sub.EE.sub._.sub.TO.sub._.sub.MASTER meets a criterion, such as being above a threshold value. If the alignment error is not above the threshold value, the current handle orientation R.sub.MCURR and new end effector orientation R.sub.EENEW are considered to be aligned.
[0123] Then, block 215 directs the master apparatus 64 to signal the motion control block 120 of
[0124] Block 159 then directs the master apparatus 64 to copy the newly calculated end effector position vector
[0125] The end effector position and orientation calculation block 116 is thus completed, and the calculated
[0126] If at block 210 the alignment error is above the threshold value, block 214 directs the master apparatus 64 to produce a rotation matrix that carries the previous handle orientation into current handle orientation (R.sub.DIFF), according to the relation:
R.sub.DIFF=R.sub.MPREV.sup.1R.sub.MCURR [0127] Where: R.sub.MPREV.sup.1 is the inverse of the previous handle rotation matrix stored in the tenth store 145 of
[0129] Referring to
[0130] Then, block 216 directs the master computer 64 to compute an angle of rotation (.sub.DIFF) associated with R.sub.DIFF by the relation:
.sub.DIFF=a cos(0.5 trace(R.sub.DIFF)1)
[0131] Then, block 218 directs the master computer 64 to compute an angular speed of the rotation (.sub.DIFF) associated with R.sub.DIFF by the relation:
.sub.DIFF=sr.Math..sub.DIFF [0132] Where: sr=sample rate in Hz at which
[0133] Then, block 220 directs the master apparatus 64 to determine whether the angular speed .sub.DIFF meets a second criterion such as being above a threshold speed to initiate auto alignment. (This may avoid any automated motion when the user is performing slow fine movements that may be undesirable.)
[0134] If not, the master apparatus 64 is directed to block 222 which corresponds to location E on
[0135] If at block 220 the angular speed .sub.diff is above the threshold speed, block 224 directs the master computer 64 to determine whether the system is configured to allow wrist roll misalignment by reading the status of the wrist roll management signal set by the operator. If the wrist roll management signal is not active, block 226 directs the master apparatus 64 to determine a misalignment axis e.sub.ERR, i.e. the axis of rotation associated with R.sub.EE.sub._.sub.TO.sub._.sub.MASTER by the relation
[0136] Alternatively, if the system is configured to allow wrist roll misalignment, block 228 directs the master apparatus 64 to determine a unit vector in the direction of the misalignment axis e.sub.ERR by the relation:
[0137] Now referring to
L=|.sub.DIFF.Math..sub.ERR|
[0138] Then, block 234 directs the master apparatus 64 to compute a correction angle .sub.C, as a function of the angle of the master rotation .sub.DIFF and the component of the master rotation in the misalignment plane L by the relation:
.sub.C=B.sub.DIFFf(L) [0139] Where: B is a scaling factor that defines the fraction of .sub.DIFF that should be used in the correction angle .sub.C, for example B=0.5. [0140] f(L) is a function that alters the profile of how the correction angle .sub.C changes with the plane of orientation change, for example f(L)=L.sup.3
[0141] Then, block 236 directs the master apparatus 64 to compute a correction matrix R.sub.C(.sub.C, e.sub.ERR) to rotate by the correction angle .sub.c about the misalignment axis e.sub.ERR.
[0142] The correction matrix R.sub.c is determined by the relation:
R.sub.C(.sub.C,e.sub.ERR)=e.sub.ERRe.sub.ERR.sup.T+cos(.sub.C)*(Ie.sub.ERRe.sub.ERR.sup.T)+sin(.sub.C)E.sub.ERR
[0143] Where: E.sub.ERR=the cross product matrix of vector e.sub.ERR
[0144] Then, block 238 directs the master apparatus 64 to modify the end effector base matrix using the correction matrix, R.sub.c by the relation:
R.sub.EEBASE=R.sub.EEBASER.sub.C
[0145] The R.sub.EEBASE value calculated by block 238 is then saved in the sixth store 154, of
[0146] Then, block 244 directs the master apparatus 64 to re-compute the end effector rotation matrix R.sub.EENEW using the new end effector base rotation matrix R.sub.EEBASE and store this new end effector orientation as R.sub.EENEW in store 164, of
R.sub.EENEW=RR.sub.EEBASE [0147] Where: R is a rotation matrix describing the rotation between the master base rotation matrix and R.sub.MBASE the current handle rotation matrix R.sub.MCURR.
R=R.sub.MCURRR.sub.MBASE.sup.1
[0148] Then, block 240 directs the master apparatus 64 to location E in
[0149] Alternatively referring to
[0150] Referring back to
[0151] It will be appreciated that the above routine is executed by the end effecter position and orientation calculation block 116 after each sample of the
[0152] Generally, when the enablement signal is in the inactive state, the handle 102 on input device 61 can be moved and rotated and the calculations of
[0153] The above clutching effect is achieved by causing movements of the handle 102 and movements of the end effector 73 to be made relative to the last-saved master base position
[0154] While the above described clutching effect is desirable to match the range of translational movement of the end effector 73 with the range of movement of the handle 102 and to reposition the hands of the operator to a comfortable position, it is not desirable for clutching to result in reorientation of the handle within the master rotational workspace because orientation control can become unnatural or unintuitive to the operator when there is a misalignment between the handle 102 and end effector 73. In the absence of a mechanical means to maintain the orientation of the handle 102 it would be difficult for the operator to rotate the handle 102 to cause it to be exactly aligned with the end effector 73 on release of the footswitch 170 so that normal operation can be resumed. In this regard, the codes of the end effector position and orientation calculation block 116 direct the master apparatus 64 to detect a difference, between the current handle orientation signals R.sub.MCURR and the new end effector orientation signals R.sub.EENEW, the difference representing a difference in physical alignment between the end effector 73 and the handle 102 relative to their respective fixed reference frame. In response to detecting the difference, the codes cause the master apparatus 64 to adjust the saved slave base orientation signals R.sub.EEBASE to ultimately have values close to the same values as the saved master base orientation signals R.sub.MBASE so that subsequent generations of the end effector orientation signals R.sub.EENEW cause the slave control signals produced by the motion control block 120 to cause the end effector 73 to be physically aligned with the handle 102 relative to their respective fixed reference frames.
[0155] This technique of adjusting the saved slave base orientation signals R.sub.EEBASE also has applications in providing a wrist-roll management feature, where wrist roll is measured as variations of orientation of the handle 102 relative to only the z-axis. The wrist roll management feature would have the effect of correcting only for misalignment for the direction in which the end effector 73 and the handle 102 are pointing and not the rotation about the axis 134 of the end effector 73.
[0156] Generally, the above described system may cause smooth autonomous motion of the end effector 73 toward alignment with the handle 102, when there is a misalignment between the handle 102 and the end effector 73 without compromising control of the end effector 73 for the operator. In addition, if the alignment error .sub.EE.sub._.sub.TO.sub._.sub.MASTER exceeds the threshold value, the alignment error will always be reduced no matter what direction the handle 102 is moving, unless e.sub.DIFF and e.sub.ERR are parallel and f(A) is such that f(A)=0 when A=0.
[0157] While specific embodiments of the invention have been described and illustrated, such embodiments should be considered illustrative of the invention only and not as limiting the invention as construed in accordance with the accompanying claims.