SYSTEMS AND METHODS FOR WIRELESS LOCALIZATION INTEGRATION

Abstract

A wireless localization system including a pad with an exciter coil and a sensor coil, a tool or surgical robot including a wireless tag configured to generate a signal in response to a magnetic field generated by the exciter coil. The signal is detected by the sensor coil and a processor configured to determine the location of the tool.

Claims

1. A wireless localization system comprising: a pad including an exciter coil and a sensor coil; a tool including a wireless tag configured to generate a signal in response to a magnetic field generated by the exciter coil; wherein the signal is detected by the sensor coil; and a processor configured to determine the location of the tool based on the signal detected by the sensor coil.

2. The system of claim 1, wherein the tool is one of a camera, an ultrasound probe, an electric impedance probe, a light probe, a microforce probe, an electrocautery tool, a needle, a swallowable capsule, a keypad, a stapler, a clamp, and a sponge.

3. The system of claim 1, wherein the wireless tag is a first wireless tag and the signal is a first signal, and wherein the system further includes a second wireless tag coupled to a tissue of a patient and configured to generate a second signal in response to the magnetic field generated by the exciter coil.

4. The system of claim 3, wherein the processor is configured to determine the location of the tool relative to the second wireless tag.

5. The system of claim 3, wherein the tissue that the second wireless tag is coupled to is one of a lung tissue, a bone tissue, a soft tissue, and an artery.

6. The system of claim 1, wherein the processor is further configured to determine the orientation of the tool.

7. A wireless localization system comprising: a surgical robot assembly including a robotic arm, a camera, and a tool coupled to the robotic arm; a pad including an exciter coil and a sensor coil; a first wireless tag coupled to a portion of the surgical robot assembly, the first wireless tag configured to generate a first signal in response to a magnetic field generated by the exciter coil, wherein the first signal is detected by the sensor coil; a second wireless tag coupled to a tissue of a patient, the second wireless tag configured to generate a second signal in response to the magnetic field generated by the exciter coil, wherein the second signal is detected by the sensor coil a processor configured to determine the location of the first wireless tag and the second wireless tag based on the first signal and the second signal detected by the sensor coil.

8. The system of claim 7, wherein the first wireless tag is coupled to the camera.

9. The system of claim 7, wherein the first wireless tag is coupled to the robotic arm.

10. The system of claim 7, wherein the sensor coil is a first sensor coil and the system further includes a second sensor coil coupled to the robotic arm.

11. The system of claim 7, further including a movable object including a third wireless tag, wherein the movable object is moved to various positions and detected by the camera to register the field of view of the camera.

12. The system of claim 11, wherein the movable object includes an outer shell, an inner sphere movable with respect to the outer shell, wherein the third wireless tag is positioned within the inner sphere.

13. The system of claim 12, wherein the inner sphere includes a weighted portion to orient the sphere in a default orientation with respect to gravity.

14. The system of claim 7, wherein the surgical robot assembly includes a control console, and wherein the location of the first wireless tag and the location of the second wireless tag are displayed on the control console.

15-30. (canceled)

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0055] FIG. 1 is a top view schematic of a wireless localization system include a wirelessly tracked tool in an operating room for a medical procedure.

[0056] FIG. 2 is a schematic view of a needle including a wireless tag and a patch with a wireless tag coupled to a patient skin.

[0057] FIG. 3A is a schematic view of a keypad and stylus with wireless tags.

[0058] FIG. 3B is an illustration of a localized probe and localized needle with visual markings.

[0059] FIG. 4 is a top view schematic of a wireless localization system including a wirelessly tracked surgical robotic system in an operating room for a medical procedure.

[0060] FIG. 5 is a side view schematic of a robotic arm with wireless tags and a sensor coil.

[0061] FIG. 6A is a schematic view of a movable object for registering a camera view.

[0062] FIG. 6B is a method for registering a camera to a wireless localization system.

[0063] FIG. 7 is a side view of a wireless localization tag including a plurality of tines.

[0064] FIG. 8 is a perspective view of a wireless tag including a plurality of self-deploying tines.

[0065] FIG. 9 is a perspective view of a wireless tag including an anchor.

[0066] FIG. 10 is a perspective view of a wireless tag including an anchor.

[0067] FIG. 11 is a schematic of a pad including four exciter coils generating three orthogonal magnetic fields.

[0068] FIG. 12 is a perspective view of a cross-section of a pad including an electromagnetically permeable layer and an electrically conductive layer.

[0069] FIG. 13 is a top view of a pad including four exciter coils and twelve sensor coils.

[0070] Before any embodiments are explained in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the following drawings. The invention is capable of other embodiments and of being practiced or of being carried out in various ways.

DETAILED DESCRIPTION

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. In case of conflict, the present document, including definitions, will control. Preferred methods and materials are described below, although methods and materials similar or equivalent to those described herein can be used in practice or testing of the present disclosure. All publications, patent applications, patents and other references mentioned herein are incorporated by reference in their entirety. The materials, methods, and examples disclosed herein are illustrative only and not intended to be limiting.

[0072] The terms “comprise(s),” “include(s),” “having,” “has,” “can,” “contain(s),” and variants thereof, as used herein, are intended to be open-ended transitional phrases, terms, or words that do not preclude the possibility of additional acts or structures. The singular forms “a,” “an” and “the” include plural references unless the context clearly dictates otherwise. The present disclosure also contemplates other embodiments “comprising,” “consisting of” and “consisting essentially of,” the embodiments or elements presented herein, whether explicitly set forth or not.

[0073] For the recitation of numeric ranges herein, each intervening number there between with the same degree of precision is explicitly contemplated. For example, for the range of 6-9, the numbers 7 and 8 are contemplated in addition to 6 and 9, and for the range 6.0-7.0, the number 6.0, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8, 6.9, and 7.0 are explicitly contemplated.

[0074] In the foregoing description of preferred embodiments, specific terminology has been resorted to for the sake of clarity. However, the invention is not intended to be limited to the specific terms so selected, and it is to be understood that each specific term includes all technical equivalents which operate in a similar manner to accomplish a similar technical purpose. Terms such as “top” and “bottom”, “front” and “rear”, “inner” and “outer”, “above”, “below”, “upper”, “lower”, “vertical”, “horizontal”, “upright” and the like are used as words of convenience to provide reference points.

[0075] Provided herein are systems, devices, assemblies, and methods for integrating a remotely located tag into medical procedures. While the specification focuses on medical uses in human tissues, it should be understood that the systems and methods find broader use, including non-human uses (e.g., use with non-human animals such as livestock companion animals, wild animals, or any veterinary settings). For example, the systems may be used in environmental settings, agricultural settings, industrial settings, or the like.

A. Wireless Tool Tracking Applications

[0076] In addition to being located within human tissue, the tags can be integrated into tools to wireless track the location and orientation of tools utilized in various medical procedures. With reference to FIG. 1, a wireless localization system 100 is schematically shown in an operating room (i.e., a doctor's office, a surgical suite, etc.) for a general medical procedure. The wireless localization system 100 includes a pad 104 positioned underneath a patient. In some embodiments, the pad 104 includes at least one exciter coil (e.g., exciter) and at least one sensor coil (e.g., witness station). A wireless tag 108 is coupled to a tissue of a patient and is configured to generate a wireless signal in response to a magnetic field generated by the exciter coil. In some embodiments, the wireless tag 108 is configured to generate a plurality of wireless signals (e.g., a first wireless signal and a second wireless signal) in response to a magnetic field generated by the exciter coil. As discussed further herein, the wireless tag 108 is coupled to tissue (e.g., lung tissue, bone tissue, soft tissue, an artery, etc.) depending on the medical procedure. Examples of such a wireless tag 108 are described in U.S. patent application Ser. No. 15/113,703 and is incorporated herein by reference in its entirety.

[0077] With continued reference to FIG. 1, a tool 112 includes at least one wireless tag 116 that is configured to generate a wireless signal in response to a magnetic field generated by the exciter coil of the pad 104. The wireless signal from the wireless tags 116 in the tool 112 and the wireless tag 108 in/on patient are detected by the sensor coil, for example, in the pad 104. As detailed herein, the tool 112, in some embodiments, is a surgical tool with wireless tags 116 embedded therein.

[0078] A processor 120 (shown in FIG. 1 as part of a wireless localization console 124) is configured to determine the location of the tool 112 and the location of the targeted area in the patient based on the signals from the wireless tags 108, 116 and measured by the sensor coil. As discussed further herein, the relative position of the tag 108 in the patient tissue and the tool 112 is utilized to improve the outcome of medical procedure. For example, the relative distance between the tag 108 and the tool 112 may be displayed visually (e.g., on a display 105) or by providing audio or haptic feedback to a user. In some embodiments, the system 100 is tracking the location of a plurality of wireless tags.

[0079] In the illustrated embodiment, the tool 112 includes two wireless tags 116 positioned along a longitudinal axis 113 of the tool 112 to determine the orientation of the tool 112 in addition to the position of the tool 112. In some embodiments, the tags 116 are positioned within an outer housing of the tool 112. In other embodiments, the tags 116 are positioned on an outer surface of the tool 112.

[0080] With continued reference to FIG. 1, the environment in which the wireless localization system 100 is positioned includes various active and reactive sources of electromagnetic noise. For example, the environment in FIG. 1 includes overhead lighting 106A, an anesthesia machine 106B, a display 105, a surgical bed 106C, and an electrocautery generator 106D. Each of which may affect the electromagnetic field present around the wireless localization system 100. As described herein, the wireless localization system 100 effectively tracks (e.g., localizes) a plurality of wireless tags in proximity to various active and reactive sources of electromagnetic noise

[0081] In some embodiments, the tool 112 is a camera. For example, some medical procedures utilize a camera to visualize the treatment area, but wireless tags implanted in a patient are not typically visible in the camera view. A solution presented herein is a camera with an integrated wireless tag to track the location of the camera and a processor to determine a coordinate transformation from the implant tracker coordinates to the camera field of view coordinates. As such, the location of the implanted tag (e.g., tag 108) can be superimposed on the camera view that is presented to the user (e.g., the display 105)—thereby improving a user's identification of the tag from the camera's field of view.

[0082] To determine the coordinate transformation, an object of known geometry is simultaneously imaged with the camera and localized with the wireless localization system. To track the camera, a wireless tag (e.g., tag 116) is fixed to the camera and also tracked with the wireless localization system. To track roll of the camera, or if the wireless tag cannot be co-axial with the camera, two wireless tags are utilized. Registration of the camera can occur in two ways: either the camera remains stationary and a moving object is tracked (see FIG. 6A), or a stationary object is registered to a moving camera. In some embodiments the camera is manually moved by a user. In other embodiments, the camera is moved by an actuator.

[0083] In some embodiments, the tool 112 is an ultrasound probe. Although ultrasound can be used to interrogate tissue and determine the position of target tissue, the imaging is two-dimensional and a separate probe must be used. A solution presented herein is to track the position and orientation of the ultrasound probe, simultaneously with the position of an implanted tag (e.g., tag 108). Ultrasound images at many positions and orientations may be acquired. Then, swap the ultrasound probe with a tool used to perform surgical excision, which is also tracked. Next, the ultrasound images are replayed according to the position of the electrosurgical tool, corrected by any relative shifts in implant position. Furthermore, ultrasound images can be segmented by tissue type, and a 3D model built up, registered to the implant position. After enough images are built up, the 3D model can be maintained relative to the implant position and used in live navigation.

[0084] In some embodiments, the tool 112 is an electrical impedance probe. It may be helpful to measure the 3D contours of the patient body including any external surfaces and internal organs (i.e., measure the geometry of excised tissue). A solution presented herein utilizes an electrical impedance probe to detect when the electrical impedance probe is in contact with the patient tissues. If the location of the probe tip is tracked the geometry of any part of the patient can be acquired by moving the probe tip around the skin.

[0085] In some embodiments, a tracked probe provides an ultrasound of patient tissue that is used to determine the location of a target with respect to tags positioned on the skin surface (e.g., embedded within an adhesive material adhered to a patient's skin). Once the ultrasound is segmented, direct feedback to the user can be provided on the needle placement, insertion orientation, and depth of insertion. In some embodiments, a motorized needle guide is tracked and adjusted to ensure proper needle insertion.

[0086] In some embodiments, the tool 112 is a light probe. Spectroscopy can be performed, for example, by illuminating tissue with broadband light and collecting reflected light but keeping track of the position of data collection must be performed manually. A solution presented herein collects the position and spectra data simultaneously, such that the location of anomalies or features of interest can be automatically collected and presented to the user.

[0087] Lung surgeons need the location of the tumor to be clearly visible during video-assisted thoracis surgery (VATS) and robot assisted thoracis surgery (RATS) procedures. Surgeon attempt position 2D or 3D cameras to define a point of view, and conventional video processing can overlay graphics and text on the video feed provided by the user. But video processing requires interconnections between systems, typically by cables, software, and site-specific configurations. A solution presented herein is projected light that overlays graphics and text on the tissue within the camera field of view. Alternatively, a tracked probe in the camera view can be equipped with one or more controlled light source(s).

[0088] A projected light implementation compatible with VATS and RATS procedures, in some embodiments, includes a robotically controlled laser pointer mounted near the camera, where two motor controlled axial degrees of freedom allow the system to always point the laser at the wireless tag, thereby illuminating the location on the tissue surface directly between the camera and the wireless tag. In some embodiments, the distance from the tissue surface to the wireless tag is determined by touching a wireless tracked probe to the tissue at the point of illumination, or by using the camera video feed to determine the location of the point of illumination. In such an embodiment, the camera is calibrated and tracked within the same reference frame as the wireless tag and light projector. In some embodiments, the projected light and machine vision processing is configured for structured light projection to determine tissue geometry more accurately. In some embodiments, rasterization is used to project other graphics and text. In some embodiments, the light projection technique has no moving parts. In some embodiments, a high-powered light is utilized to burn the tissue to create a durable mark on the tissue.

[0089] A tracked probe with a controlled light source compatible with VATS and RATS procedures, in some embodiments, includes a single light emitting diode (LED) attached to the tracked probe. In some embodiments, the LED brightness is controlled by the system based on the relative position and/or alignment of the probe to the wireless tag. For example, the user could manipulate the tracked probe until the user sees the illumination in the camera view, and when the brightness is maximized, the tracked probe is pointed directly at the wireless tag. In other embodiments, a tracked probe with a controlled light source compatible with VATS and RATS procedures includes multiple LEDs arranged in a pattern. In some embodiments, each LED brightness is controlled by the system based on the relative position and/or alignment of the probe to the wireless tag. For example, the user could observe the pattern that is configured to guide them to the wireless tag. In some embodiments, a set of LEDs arranged in a plane is lighted so that the location of the highest intensity corresponds to the position of the wireless tag.

[0090] In some embodiments, the tool 112 is a microforce probe. The clamping force of a robot may be easy to control, but there are advantages to also wirelessly monitoring the clamping force. In some instances, a robotic jaw is configured to grab an object, but there is no feedback on how much resistance the robotic jaws are experiencing. The solution presented herein is a wireless tag embedded into a small module with a wireless communications module and one or more sensors and mechanical interfaces to measure and transmit real time data such as force, temperature, pressure, etc. In some embodiments, the tool 112 is a specialized tracked tool that deflects in a way that is measurable via the angle between two wireless tags, and the deflection is related to the amount of force applied to what is captured within the jaws. Advantageously, no battery is required because the communications can be powered by the exciter via induction. In other words, the same exciter used for localization can also provide power to the microsensor. As such, size can be minimized. For example, communications chips can be as small as approximately 2.5 mm×approximately 2.5 mm without an antenna. The probe data can be utilized, for example, to generate haptic feedback to the user.

[0091] In some embodiments, the tool 112 is an electrocautery multi-tool. Conventional electrocautery tools include various cords or tubes that must be assembled by the user. A solution presented herein is a single device with smoke capture, illumination, electrocautery, and tip localization. The solution consolidates multiple cords into one and eliminates user assembly steps. It is beneficial for the localization and electrocautery system to be coordinated so that interference is mitigated. Advantageously, better cutting energy can be delivered by controlling the waveforms more precisely, and by using low energy to detect when the tip is in contact with tissue. In addition, low energy detection of the tip in contact with tissue can be used to build a 3D model of the patient.

[0092] In some embodiments, the tool 112 is an electrocautery tool accessory (e.g., a collar, a tip, etc.). Conventional electrocautery tools include a tip removable from the pen. A solution presented herein is an adaptor for an electrocautery tool (e.g., a bovie pen). In some embodiments, the adaptor is a small wireless device that attaches to the electrocautery tool like a collar. In other embodiments, a locatable tip (i.e., a tip with wireless tag embedded in the shaft) is provided such that the tip can be tracked wirelessly by the wireless localization system.

[0093] In some embodiments, the tool 112 is a stapler. In lung applications, a cut is optimally performed perpendicular to the lung plane. A solution presented herein is a stapler including at least one wireless tag configured to define the plane where the lung will be cut and the end of the stapler. In some embodiments, the wireless tag is oriented perpendicular to the plane of interest, which aligns an unknown roll coordinate of the wireless tag to be anywhere on the plane.

[0094] In some embodiments, the tool 112 is a tracked clamp configured to span the entire cut across a lung.

[0095] In some embodiments, the tool 112 is a needle that is utilized in a medical procedure. A solution presented herein includes a needle 200 with a wireless tag 202 (e.g., a SmartClip®) in the upper shaft that can be tracked. With reference to FIG. 2, the needle 200 includes the wireless tag 202 coupled to the needle 200 for localization of the needle 200. In some embodiments, a patch 204 including a wireless tag 205 is positioned on the skin of a patient 201 to localize the skin surface (e.g., to track location of skin surface). A known distance 203 between the wireless tag 202 and a tip 206 of the needle 200 is utilized to provide advantageous information. As such, the needle depth of insertion is tracked by the patch 204 that is coupled to the patient skin near an intended punction site. Also, the position and orientation of the patch 204 would allow the angle of the needle 200 with respect to the patient 201 to be calculated. Alternatively, the patch 204 is eliminated and the depth of insertion is tracked by having the user indicate when the tip 206 of the needle 200 is at the skin surface 201.

[0096] In some embodiments, the tool 112 is a capsule or pill that is swallowed by a patient. For example, convention capsules are swallowed by a patient to diagnose GI issues. A solution presented herein is a capsule with integrated wireless tags to provide wireless tracking of the capsule to increase the utility of the devices. The capsule with an integrated wireless tag can be enabled to deliver medication based on a command given over wireless communication. Medication delivery could be protected by requiring a large pulse of energy from an exciter. The large pulse can physically change the capsule between a medication deliver prohibited configuration to a medication delivery allowed configuration. In addition, a capsule with an integrated tag could implant a wireless tag on demand to mark tissue of interest.

[0097] In some embodiments, the tool 112 is a keypad and stylus (i.e., a sterile surgical interface). In surgery, sterile users cannot touch non-sterile user input devices. Bringing additional wired devices to serve as input devices requires additional sterile field management. Conventional wireless input devices are typically expensive and require batteries. The solution presented herein provides a keypad without electronics or digital communications with an embedded wireless tag that can be localized (i.e., a low-cost keypad). A localized stylus or electrocautery tool can touch the localized keypad.

[0098] With reference to FIG. 3A, a keypad 250 includes two wireless tags 251 and is localized by a wireless localization system (e.g., system 100). The system then determines which key on the keypad is touched by tracking the location of a tracked tool tip or stylus 252 with respect to the keypad 250. In other words, the tracked tool tip 252 location is known and the location of the buttons are known with respect to the wireless tags 251 in the keypad 250. As such, the localization system can detect the location of the tracked tool tip 252 and will register a button press by the tool tip 252 relative location to the keypad 250. In some embodiments, the keypad provides a mechanical response on each button which can be recognized by the localization system as a “click” based on the characteristic movement of the tracked tool (e.g., tracked stylus, tracked electrocautery tool, etc.). In some embodiments, the keypad is located by positioning a stylus with a single wireless localization tag at or on the keypad.

[0099] With reference to FIG. 3B, in some embodiments, the tool 112 is a localized probe 310 that is graspable by a robotic arm or tool 326. The localized probe 310 includes a conical tip 314 and a wireless tag 318 positioned within a housing 322. In the illustrated embodiment, the housing 322 is graspable by the robotic arm or tool 326. The probe 310 further includes a stem 330 with indicia 334 (e.g., visual markings) that can be detected and tracked by a camera. In other words, the localized probe 310 is a small wirelessly tracked probe that can be grasped by a robot or other instrument within the patient body, where that probe has a stem that acts as a visual reference. In some embodiments, the orientation of the probe 310 is constrained to define the orientation of the probe. In other words, the robotic arm 326 can be constrained to always keep the probe 310 oriented vertically, with the stem 330 pointing upwards.

[0100] With continued reference to FIG. 3B, in some embodiments, the tool 112 is a localized needle 350 that is graspable by a robotic arm or tool 352. The localized needle 350 includes a housing 354 with a wireless tag 358 and a needle portion 362 with indicia 366 (e.g., visual markings) that can be detected and tracked by a camera. In other words, the localized needle 350 includes an insertion depth dimensions readable by a camera. As such, in some embodiments, the localized probe is inserted into the patient organ until its tip is co-located with an implanted wireless tag so that it functions as a physical landmark visualized by a camera. In some embodiments, it could be tactilely sensed.

B. Robotic Integration Applications

[0101] As disclosed herein, wireless tags are integrated into robots and robotic apparatuses utilized in various medical procedures. An example of such a robotic system is the Intuitive da Vinci System. An example of a computer-assisted tele-operated surgery system and methods disclosed in U.S. Pat. No. 11,207,143, the entire contents of which are incorporated herein by reference. Unique challenges arise when attempting to integrate a wireless located tag into a robotic environment.

[0102] For example, patient anatomy and pre-operating imaging is insufficient for many surgical interventions. Wired-only navigation systems depend on pre-operative imaging and other methods to register detected locations to patient anatomy. As mentioned, some medical procedures utilize a camera, but wireless tags implanted in a patient are not typically visible in the camera view. A solution presented herein provides wireless tag localization in a robotic application. Precise localization of an implanted tag, co-registered to the surgeon's visual frame of reference, along with localization of tools, can provide intuitive dimensional feedback essential for successful interventions like excision, ablation, drug delivery, etc.

[0103] With reference to FIG. 4, a wireless localization system 400 is schematically shown in an operating room (i.e., a doctor's office, a surgical suite, etc.) for a general medical procedure. The environment in which the wireless localization system 400 is positioned includes various active and reactive sources of electromagnetic noise. For example, the environment in FIG. 4 includes overhead lighting 406A, an anesthesia machine 406B, and a surgical bed 406C. Each of which may affect the electromagnetic field present around the wireless localization system 400. The wireless localization system 400 includes a pad 404 positioned underneath a patient. In some embodiments, the pad 404 includes at least one exciter coil and at least one sensor coil (e.g., witness station). The wireless localization system 400 includes a surgical robot assembly 408 (e.g., a da Vinci System developed by Intuitive). In the illustrated embodiment, the surgical robot assembly 408 includes a robotic arm 412, a camera 416, and a tool coupled to the distal end of the robotic arm 412. In some embodiments, the surgical robot assembly 408 includes a single robotic arm. In other embodiments, the surgical robot assembly 408 includes at least one robotic arm. In the illustrated embodiment, the surgical robot assembly 408 includes three robotic arms 412. In some embodiments, the surgical robot assembly 408 includes a plurality of cameras. As explained in greater detail herein, the wireless localization system 400 integrates the surgical robot assembly 408 to provide improved localization of portions of the surgical robot assembly 408 relative to a target area in the patient. For example, visualization of a wireless tag 422 embedded within a patient can be visualized and overlayed on the robotic vision from the camera 416. In some embodiments, the surgical robot assembly includes a control console 430 where a surgeon 431 controls the robot and the location(s) of the wireless tags are displayed on the control console. In some embodiments, the control console 430 is in the same location (e.g., room) as the surgical robot and in other embodiments, the control console 420 is in a different location (e.g., a remote location, offsite location).

[0104] With continued reference to FIG. 4, a first wireless tag 418 is coupled to a portion (e.g., a robotic arm) of the surgical robot assembly 408, and a second wireless tag 422 is coupled to a target tissue of a patient. The first wireless tag 418 and the second wireless tag 422 are configured to generate wireless signals in response to a magnetic field generated by the at least one exciter coil in the pad 404. The wireless signals from the tags 418 and 422 are detected by the at least one sensor coil in the pad 404. A processor 426 then determines the location of the first wireless tag 418 and the second wireless tag 422 based on the signals measured by the at least one sensor coil.

[0105] With reference to FIG. 5, in some embodiments the each of the robotic arms 500 include two wireless tags 504 mounted along a longitudinal axis 506 aligned with the surgical tool mounted on a distal end of the arm. In some embodiments, a sensor coil 508 (similar to and/or in addition to the sensor coils in the patient pad 510) is coupled to the robot arm 500 in order to improve the sensing of wireless tag signals that may be poisoned far away from the patient pad 510. In the illustrated embodiment, the sensor coil 508 is positioned axially between the two wireless tags 504, along the longitudinal axis 506. In some embodiments, the sensor coil 508 is oriented orthogonal to the coils in the wireless tags 504.

[0106] One challenge to integrating wireless tags into robotic applications is effectively using cameras to measure anatomical landmarks. Use of camera to measure landmarks has drawbacks such as, challenges with lighting, lack of clear reference marks, tissue changes compared to as-imaged, presence of fluids, etc. A solution presented herein allows for co-registration to be achieved by registering the camera location with respect to wireless tags mounted on the camera and by tracking an object that goes to specific points in the camera's field of view. A wireless tag mounted to the camera is not mounted directly on the lens, so the relative position of the wireless tag and the lens of the camera is registered to improve localization. In some embodiments, field of view registration is accomplished by moving a tracked object to the upper-right, lower-right, upper-left and lower-left corners of the field of view when the position of the camera is known. In some embodiments, the camera is calibrated by placing a first wireless tag at a known close distance from the camera (e.g., within approximately 50 mm) to get the camera position, and then the orientation of the camera is determined by imaging a second wireless tag with the camera while the second wireless tag is far away from the camera. The wireless localization system can locate the magnetic center and the direction of the longitudinal axis of the wireless tag, but not necessarily which end of the tag is which. In order to consistently co-register the camera view to the system coordinate system, an additional degree of freedom is required to be constrained.

[0107] With reference to FIG. 6A, a tracked object 600 for camera registration is illustrated. The tracked object 600 includes a clear outer shell 605 and an inner sphere 606 that is freely movable with respect to the outer shell 605. In some embodiments the outer shell 605 is configured to be gripped or grabbed by a robotic tool. A wireless tag 604 is positioned within the inner sphere 606. In the illustrated embodiment, a weighted portion 602 (e.g., weighted bottom) uses gravity to ensure a wireless tag 604 in an inner sphere 606 always “points” a given direction (i.e., upwards and away from the patient pad) and thereby provides enough information to constrain the last degree of freedom. In other words, the inner sphere 606 includes a weighted portion 602 to orient the inner sphere 606 and the tag 604 in a default orientation with respect to gravity. The inner sphere 606 further includes a plurality of sections 608 of various colors or shades of black and white that are detectable by the camera. In the illustrated embodiment, the plurality of sections 608 includes four different color sections. In some embodiments the plurality of sections 608 includes eight different color sections. As such, the movable tracked object 600 includes the wireless tag 604 and is moved to various positions and detected by the camera to register the field of view of the camera

[0108] In some embodiments, the wireless tags are cylindrical and the roll about the longitudinal axis and the direction the tag is pointing are challenging to determine. A solution presented herein is to place a wireless tag within an asymmetrically weighted mount. The wireless tag in the mount creates an angle of approximately 20-70 degrees from a line extending between the mount's center of mass and the mount's axis of rotation. The asymmetric weight provides a constraint on the tag orientation sufficient to allow the determination of both the roll orientation and the pointing direction. In some embodiments, the mount is encased in an object having a known pattern (e.g., the plurality of sections 608). The known pattern is imaged with a camera, allowing the object to be simultaneously localized with a camera and the wireless localization system. Once simultaneously localized, the general transformation between the wireless localization system and the camera is determined.

[0109] If the position of the camera is not known, the four corners of the field of view can be measured at two different distances from the camera to determine the camera location. In some embodiments, the camera performs a set of pre-determined motions (e.g., spin, move forward, etc.). In other embodiments, the camera looks at the same point from different points of view. In some embodiments, configurations of the camera registration can be saved and loaded as a pre-set configuration.

[0110] With reference to FIG. 6B, a method 650 is disclosed to register a camera to a wireless localization system. The method 650 includes a STEP 654 of providing a camera unit with a wireless tag mounted to the camera unit. The method 650 also includes a STEP 658 of moving a localized probe to contact the camera unit. In other words, at the STEP 658 includes abutting a localized probe to the camera unit. The method 650 also includes a STEP 662 to position the localized probe in the camera field of view. In some embodiment, the localized probed position in the field of view is the same probe from STEP 658. In other embodiments, the localized probe positioned in the view of view is different from the localized probe used in STEP 658. The method 650 further includes a STEP 666 to calibrate the camera zoom utilizing an object of a known size that is positioned within the camera field of view. In some embodiments, a calibration outline is displayed to a user and the camera zoom is adjusted until the object of known size is contained within the calibration outline. In one embodiment, a known object with a 10 cm diameter is positioned within the camera field of view and the zoom on the camera is adjusted until the object fits within a calibration outline on the display. The method 650 further includes a STEP 670 of displaying a dimensionally accurate augmented reality. In other words, at the STEP 670, the camera field of view can be overlayed with a 3D interface. In some embodiments, two 3D interfaced are provided on separate video feeds with one to each eye so that the localization interface is overlayed on top of a 3D camera interface. As such, the method 650 provides a calibration process to register a camera view to a wireless localization system that visualizes the location of various wireless tags. In some embodiments, additional corrections are applied to account for various camera lens types (e.g., wide aperture lens adjustments).

[0111] Another challenge to integrating wireless tags into a surgical robot application is that metal object should be kept away from detectors or sensor coils because they create distortions in the electromagnetic field. In other words, surgical robots often include metal components (e.g., a metallic arm) that can distort the electromagnetic field relied upon by the wireless localization system. A solution presented herein is to utilize various noise reduction and signal processing techniques. In some embodiment, the system utilizes a phase sensitive signal processing.

[0112] Another challenge to integrating wireless tags into a surgical robot application is attaching wireless tags directly to metal components (e.g., a robotic arm) can cause the signal transmitted by the wireless tag to degrade by induced eddy currents within the metal. A solution presented herein is to include a thin layer of high permeability material (e.g., iron, manganese, zinc, silicon, aluminum, nickel, electrical steel, cobalt-iron, etc.) positioned between the antenna winding of the wireless tag and the metal. In some embodiments, the layer of high permeability material is selected based on the frequency range of the signal transmitted by the wireless tag.

[0113] Another challenge to integrating wireless tags into a surgical robot application is that range is limited by the ability to measure and a feature of interest on a patient can often be positioned far away from the pad. The solution presented herein is to embed a high gain detector (e.g., 508 of FIG. 5) orthogonal to emitters to sense the wireless tag signals. The high gain detector can be mounted on a robotic component (e.g., a robot arm) and the sensor moved close to the wireless tag.

[0114] Another challenge to integrating tags into a surgical robot application is that small wireless tags respond to only one direction of the excited field, and in some embodiment the robotic applications would have a plurality of wireless localization tags (i.e., clips, SmartClips®) being tracked. A solution presented herein utilizes a pad that creates three orthogonal directions of an exciter field. With reference to FIG. 11, an exciter pad 1102 is illustrated generating three electromagnetic fields just above the pad 1102 that are orthogonal to each other (e.g., X, Y, and Z direction electromagnetic fields). With three orthogonal directions of the exciter field, there is enough power transfer for wireless tags regardless of the wireless tag orientation. The preferred field direction for any wireless tag can be determined by measuring the total power received by the sensing system and choosing the field direction that has the largest total power. If multiple tags need to be localized, only those excited by the same field direction can be localized at the same time. By sequentially stepping through the minimum number of field directions which cause all wireless tags to be localizable, all tags are localized. The amount of time it takes to change field direction is important. To minimize the time, solid-state transistors are used to change the field polarity (in place of, or in addition to, electromechanical relays). In some embodiments, the electromechanical relays may take longer to switch than the solid-state transistors and electromechanical relays do create noise at the time of switching that must be allowed to decay out of the narrow bandwidth signal conditioning. An example pad with an exciter coil and a sensor coil is described in U.S. Pat. No. 10,278,779, the entire contents of which are incorporated herein by reference. 10115) In some embodiments, surgical robots perform an automated intervention (e.g., a set of steps based on a detailed surgical plan for robotic movement). Another challenge to integrating tags into robotic applications is that automated interventions could be based on a detailed surgical plan for robotic movements. A solution presented herein is automation safety that is enhanced by monitoring the robot movements by an independent processing system that compares the actual movements to the planned movements, and generates a signal when the plan diverges beyond allowed tolerances. A secondary signal is generated when the robot movements are not being tracked. In other words, in some embodiments, the wireless tags are utilized to track and double check the robotic components are moving according to the plan. As such, the wireless tags can wirelessly track the movement of robotic components and may be utilized to provide additional safety or a safety lockout.

C. General Surgical Applications

[0115] The solutions provided herein also have general surgical applications including, but not limited to: preventing items being left within patients, bone modeling, and soft tissue modeling.

[0116] A general problem with surgery arises when items (e.g., surgical instruments, sponges, etc.) are left within a patient after surgery is completed. The solution presented herein embeds wireless tags into such items so that they can be tracked wirelessly. Conventional fielded wireless object tracking system are incompatible with the wireless tag localization system. As described herein, fielded wireless object tracking is integrated into the tag localization system to reduce the amount of equipment required and/or reduce the amount of wireless interference. Items tagged with a tag are detectable with high assurance with the EnVisio® system. A large number of tags can be supported if localization requirements are relaxed. In some embodiments, the system keeps an ongoing count and tracks the number of items introduced to the general fielded area and decrease the number when tagged items leave the general fielded area (i.e., keeps a running total of items in the environment).

[0117] Another challenge is conventional robot-guided orthopedic systems rely on optical tracking of bones, which requires optical trackers to be surgically implanted into bone and that the system maintains direct line of sight of the optical trackers. The solution presented herein embeds two or more wireless tags (i.e., SmartClips®) into a bone, use pre-operative imaging and image segmentation to establish the bone geometry within the coordinate system defined by two or more wireless tags. In some embodiments, at time of surgery the bone is mounted to motor driven actuators controlled by software (e.g., a robotic arm), and the bone interface includes the wireless tag excitation and sensing sub-systems.

[0118] Another challenge is that pre-operative imaging is collected with soft tissue in different positions, orientations, and shape than it is presented in the operating room for surgery. The solution presented herein is to implant two or more wireless tags within soft tissue and/or on the skin of the patient. Pre-operating imaging can be conducted with the tags in place, and imaging can be imported to observe the new locations and orientations of the tags. The 3D image is deformed according to a volume maintaining algorithm that aligns the imaged tags to the live observed tags. In some embodiments, the volume maintaining algorithm is physics and physiology-based that may incorporate tissue density, elasticity, deformation, etc.

D. Wireless Tag Deployment

[0119] Tags can be delivered into soft tissue through needles, catheters, etc. with a conventional plunger mechanism. In some embodiments, additional mechanical features are added to secure the wireless tag to surrounding tissue. Securing the wireless tag is especially important during tissue excision involving manipulation of the tissue. With reference to FIG. 7, the wireless localization tag 702 is positioned within a housing 704 having at least one tine 706 formed at an axial end 708 of the housing 704. In the illustrated embodiment, the tines 706 extend along a longitudinal axis 712 of the tag 702. The tines 706 are configure to grip, secure, or otherwise anchor the wireless tag 702 to surrounding tissue.

[0120] With reference to FIG. 8, a wireless tag 802 includes a housing 804 with self-deploying tines 806 that engage in surrounding tissue and prevent movement. In the illustrated embodiment, the tines 806 deflect radially with respect to a longitudinal axis 810 of the tag 802 (housing 804). In some embodiments, the tines 806 are movable to a first position where the tines 806 are deflected radially inwards toward the longitudinal axis 810 (e.g., during deployment or delivery), and the tine 806 are movable to a second position where the tines 806 extend radially outwards from the longitudinal axis 810 (e.g., deployed and in position). In other words, the tines 806 deploy radially outward when in position. In some embodiments, the tines 806 are used to secure the wireless tag 802 to a patient's lungs.

[0121] With reference to FIG. 9, a wireless tag 902 includes a housing 904 with a tine 908. In the illustrated embodiment, the tine 908 is a spiral that extends in a plane 910 perpendicular to a longitudinal axis 912 of the tag 902. In some embodiments, the tine 908 is made of nitinol. The tine 908 is movable between a first position where the spiral tine 908 is deflected radially inward toward the longitudinal axis 912 (e.g., a circumferentially compressed deploying position) and a second position where the spiral tine 908 extends radially outward (e.g., a deployed position). In the illustrated embodiment, the spiral tine 908 remains in the plane 910 as the tine 908 moves between the first position and the second position.

[0122] With reference to FIG. 10, a wireless tag 1002 includes a housing 1004 with a tine 1006. In the illustrated embodiment, the tine 1006 is a deployable stent.

E. Targeted Radiation Delivery

[0123] When delivering high energy radiation to soft tissue of a patient, positioning the target tissue at the intended location is critical. Conventional techniques to determine the location to radiate include use of external landmarks (e.g., anatomy, tattoos) or x-ray-based imaging. Conventional techniques to detect patient movement after placement include monitoring an external tag, monitoring patient breaching via air flow.

[0124] A solution presented herein provides direct localization with a tag implanted at or near the site targeted for radiation. If a wireless tag is implanted at the site the radiation is intended to be delivered, the patient will necessarily be positioned such that the tag is at or near the focus of the targeted radiation systems. Therefore, the volume over which a wireless tag is placed is relatively small (e.g., 10 cm×10 cm×10 cm) compared to other procedures. The tag excitation and sensing elements should not overlap with the radiation to both prevent beam deflection and also prevent damage to the electronics. In some embodiments, it is advantageous to have the excitation and sensing elements in separate modules, such that the excitation field is much lower and electronic filters are sufficient to reject the excitation signal. In some embodiments, a Helmholtz coil configuration, with exciters at two locations with the wireless tag between the two, may also be advantageous. In other embodiments, a single pad with both exciter and sensors is placed alongside the patient during radiation treatment, instead of under the patient.

F. Perioperative Imaging

[0125] Radiopaque materials used in wireless tag signal excitation and sensing can obscure patient features when present between the imaging equipment source and detectors. In some procedures, a signal is provided when the wire, catheter, or similar device is at a certain location within the vessel relative to the implant.

[0126] A solution presented herein is a wireless tag positioned within a stent (e.g., a coronary vessel) when it is desired, for example, to retrieve the stent. A solution also includes a wired emitter is coupled to an outer wall of a catheter or similar device. The wired emitter provides enough magnetic field strength to power up the wireless tag. A sensor or set of sensors position outside the body listen for the wireless tag response (indicating the wired emitter is approaching the wireless tag). In some embodiments, the signal conditioning on the sensor signal includes a notch or low pass filter to block the exciter signal carried by the emitter. If the wireless tag is in range of the wired emitter, the frequency spectrum characteristic of the wireless can be sensed.

[0127] Alternatively, the implant includes a magnet, and the catheter includes a high-sensitivity magnetometer. This approach includes a “zeroing” procedure to reduce effect of variation in the local magnetic field due to the earth and nearby metallic components.

[0128] Alternatively, the tag includes a passive highly resonant LC circuit, and the reflected signal on the emitter is monitored. Changes in coupling between the wired emitter and implant alters the reflected signal, and a location where the reflected signal is affected the most would correspond to the position of maximum coupling, and the geometry of the emitter and tag are tailored such that the position of maximum coupling corresponds to a signal relative location of the catheter within the vessel.

G. Arterial Access

[0129] Gaining arterial access can be challenging especially for femoral access in either cardio or leg interventions. A solution presented herein is a wireless tag positioned along the artery at the location of intended access under ultrasound or imaging guidance. In some embodiments, the needle used for initial access is tracked with a wireless or wired beacon, then guided by the system for ideal placement and direction. High precision reduces bleeding and reduce the time required to gain access.

H. Different Environment

[0130] Changes in the complex magnetic permeability near the pad can influence the signals used to localize beacons. For example, metallic (high conductivity) materials force the magnetic field to zero by means of induced currents that act as signal source at the same frequency. Ferrous (high permeability) materials are much less common but also modify the fields. Pads positioned under the patient are placed as close as possible to beds with large metallic components. A mapping process is used to learn the influence of the bed on the fields, and remove the influence on localization. However, different environments range from including no metal to mounting a pad directly on metal. Solutions presented herein permit the pad to be used in these various environments.

[0131] A first challenge arises from large metal loops embedded in bed that can create an opposing magnetic field that effectively cancels the exciter field. A solution presented herein is to position an electrically conductive layer 1206 (e.g., conductive plate) on the bottom of the pad 1202. The conductive plate 1206 acts as a shield to changes in environment permeability below the pad 1202. With reference to FIG. 12, the conductive plate 1206 shapes the magnetic field (see arrow 1210) to contain the electromagnetic flux in the pad 1202. In some embodiments, the conductive plate is metallic. In some embodiments, the conductive layer is aluminum. In some embodiments, the conductive layer is mu-metal, cooper, or stainless steel. In some embodiments, the electrically conductive layer includes a conductivity of at least 20×10.sup.6 Siemens per meter (S/m).

[0132] A second challenge arises from the conductive layer affecting the exciter field because of currents induced in the metal and the resulting opposing magnetic field. With reference to FIG. 12, a solution presented herein is a pad 1202 with a first layer 1204 of a high electromagnetically permeability material and a second layer 1206 of electrically conductive material (e.g., the conductive plate). In some embodiments, the first layer 1204 has an electromagnetic permeability within a range of approximately 10 to approximately 5000. In some embodiments, the first layer 1204 is a ferrite core (e.g., oxides made from Iron, Manganese, and Zinc—manganese zinc ferrites). In some embodiments, the first layer 1204 is a composite including Iron, Silicon, Aluminum, and/or Nickel (e.g., a powder core)

[0133] In the illustrated embodiment, the first layer 1204 is positioned between a plurality of exciter coils 1212 and the second layer 1206. The electromagnetic field at second layer 1206 is lower and the corresponding currents induced lower. In the illustrated embodiment, the permeable layer 1202 is positioned between the exciter coil 1212 and the conductive layer 1206. As such, the pad 1202 includes a combination of materials that redirects and contains the magnetic flux on the bottom of the pad 1202, and the pad 1202 is configured to operate efficiently on various surgical beds (e.g., “bed agnostic”).

[0134] In some embodiments, the exciter coils 1212 are encapsulated with a high thermal conductivity encapsulating material 1214 (e.g., an epoxy). In some embodiments, the encapsulating material has a thermal conductivity greater than 1000 W/Kelvin. In some embodiments, the encapsulating material has a heat capacity greater than 1000 Joules per kg° C. In some embodiments, the encapsulating material 1214 has a dielectric strength of at least 400 Volts per mil. In some embodiments, ferrite material helps shape the magnetic field.

[0135] A third challenge arises because the presence of a high permeability material distorts the magnetic field directions. Distortion of the magnetic field directions creates challenges because it is desirable to position the sensors in a location and orientation where the sensor is orthogonal to the exciter field for each of the states of the exciter creating different field directions. The distortion is pronounced in multi-exciter systems where the phases of the current are varied to change the primary directionality of the induced magnetic field. This effect results in large changes in field direction, in the plane of the exciter, as the exciter is configured to create different field direction (e.g., FIG. 11).

[0136] With reference to FIG. 13, a solution presented herein is a pad 1301 with exciter coils 1300A, 1300B, 1300C, 1300D and the sensor coils 1304A-1304L arranged in a grid configuration. The sensor coils 1304A-1304L are placed in locations where the least change in field direction in the plane of the exciter occurs. In the illustrated embodiment, the sensor coils 1304A-1304L each define a sensor coil axis 1306A-1306L and the exciter coils 1300A-1300D each define an exciter coil axis 1302A-1302D. In this grid configuration, all sensors coils 1304A-1304L are oriented parallel to the tangent of adjacent exciter coil 1300A-1300D. For example, the sensor coil axes 1306A, 1306D, 1306F, and 1306C of sensors coils 1304A, 1304D, 1304F, 1304C are oriented parallel to the tangent of the exciter coil 1300A. In other words, the sensor coil axes 1306A, 1306D, 1306F, and 1306C do not intersect the exciter coil axis 1302A.

[0137] With continued reference to FIG. 13, the four exciter coils 1300A-1300D are positioned circumferentially around a center 1312 of the pad. In the illustrated embodiment, the sensors axes 1306A, 1306B, 1306F, 1306G, 1306K, and 1306L are aligned in a first direction (e.g., an X direction) and the sensor axes 1306C, 1306D, 1306E, 1306H, 1306I, and 1306J are aligned in a second direction (e.g., a Y direction). In the illustrated embodiment, the first direction and the second direction are perpendicular. In the illustrated embodiment, the excitor axes 1302A-1302D are aligned in a third direction perpendicular to the plane of view of FIG. 13 (e.g., a Z direction). In the illustrated embodiment, the sensor coils 1304A-1304L are positioned centered with at least one exciter coil (in line with at least one exciter axis 1302A-1302D). For example, the sensor coils 1304A, 1304D, 1304F, and 1304C are positioned circumferentially around the exciter coil 1300A. In the illustrated embodiment, there is a sensor coil 1304A-1304L positioned at the mid-point of each side of each exciter coil 1300A-1300D.

[0138] In some embodiments, the exciter coil is circular. In other embodiments, the exciter coil is rectangular or another suitable shape. In some embodiment, the electronics utilized for monitoring and controls of the pad are positioned under the conductive shield (e.g., the conductive shield is positioned between the electronics and the exciter coils).

[0139] In the illustrated embodiment, the high permeability material tolerates high fields without becoming saturated. In some embodiments, the internal induced field the high permeability material can support is higher than the external applied field.

[0140] Various features and advantages are set forth in the following claims.