SYSTEM FOR PROVIDING PROBE TRACE FIDUCIAL-FREE TRACKING

Abstract

A method for referencing a tracking system's coordinate frame to a rigid body's coordinate frame is disclosed. The method involves obtaining a 3D model depicting some of the surfaces of the rigid body. A probe is provided with an affixed tracking reference component. A second tracking reference component is attached to the rigid body. The method involves tracking locations of the probe as it moves along surfaces of the rigid body and then determining a transform that relates the probe locations to the 3D model of the rigid body. In one embodiment the rigid body is a dental mandible or maxilla of a patient and the 3D model is a surface extracted from a computed tomography image of the patient's jaw and teeth.

Claims

1. A method for referencing a tracking system's coordinate frame to a rigid body's coordinate frame, comprising the steps of: providing a 3D model depicting some of the surfaces of the rigid body; providing a probe with an affixed tracking reference component; providing a rigidly affixed tracking reference component on the rigid body; tracking locations of the probe as it moves along surfaces of the rigid body; and determining a transform that relates the probe locations to the 3D model of the rigid body.

2. The method of claim 1, wherein the 3D model is a surface extracted from a computed tomography image of a patient's anatomy.

3. The method of claim 1, wherein the tracking system's coordinate frame is attached to an electromagnetic field generator and the tracking reference components are electromagnetic sensor coils.

4. The method of claim 1, wherein the tracking system's coordinate frame is attached to a stereo optical tracking system.

5. The method of claim 4, wherein the tracking reference components include contrast patterns.

6. The method of claim 4, wherein the tracking reference components are arrays of reflective spheres.

7. The method of claim 4, wherein the tracking reference components are arrays of light-emitting devices.

8. The method of claim 1, wherein the probe is an instrument with a spherical tip.

9. The method of claim 1, wherein the probe is an instrument with a tip that includes a conical point.

10. The method of claim 1, wherein the rigid body is a dental mandible or maxilla of a patient.

11. The method of claim 10, wherein the probe is traced along and around the patient's teeth.

12. The method of claim 10, wherein the probe is traced along and around the patients jaw bone.

13. A method for referencing a tracking system's coordinate frame to a rigid body's coordinate frame, comprising the steps of: providing a 3D data set depicting some of the surfaces of the rigid body; providing a probe with an affixed tracking reference component; providing a rigidly affixed tracking reference component on the rigid body; providing an initial transform that approximately relates the probe locations to the 3D data set of the rigid body; tracking locations of the probe as it moves along surfaces of the rigid body; visually depicting the probe location in relation to the 3D dataset; providing a plurality of estimates of the error between a given probe location and a portion of the surfaces in the 3D dataset; and determining a transform that relates the probe locations to the 3D model of the rigid body.

14. The method of claim 13, wherein the estimate of error is a correspondence between a probe tip location and a plane in the 3D data set.

15. The method of claim 13, wherein the rigid body is a CT scan of at least a portion of a patient's oral cavity, and wherein the surfaces are teeth or portions of a jawbone, and wherein the approximate location of the probe relative to the 3D data is depicted on a display; the method further comprising the steps of generating a line on the display that is orthogonal to the approximate location of a tip of the probe, the line being at a first location; receiving signals to move the line relative to the 3D data set to a second location; calculating a distance of movement from the first location to the second location, wherein at least one of the estimates of error are determined based on the distance.

16. A method for referencing a tracking system's coordinate frame to a rigid body's coordinate frame, comprising the steps of: a. providing a computer display; b. displaying on the display a 3D data set depicting multiple surfaces of the rigid body, the rigid body including a person's oral cavity and at least some of the surfaces corresponding to the person's teeth or jawbone; c. providing a probe with an affixed tracking reference component, the probe having a probe tip; d. providing a rigidly affixed tracking reference component on the rigid body; e. receiving a signal when the probe tip is placed against a surface of the multiple surfaces; f. depicting on the display an approximate first location of the probe and probe tip on the 3D data set of the rigid body based on an initial transform; g. generating a line on the display that is orthogonal to the first location of the probe tip; h. receiving a signal to move the line relative to the 3D data set to a second location which corresponds to the depicted location in the 3D data set of an image corresponding to the surface of the multiple surfaces; i. determining an error value corresponding to the difference in location of the first and second locations and storing the error value; j. receiving a signal when the probe tip is placed against another surface of the multiple surfaces and repeating steps f through i; and k. determining a transform from the error values that relates the probe locations to the 3D model of the rigid body.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other aspects, features, and advantages of the present invention may be more apparent from the following more particular description of embodiments thereof, presented in conjunction with the following drawings.

[0023] FIG. 1 illustrates probe tracing of the lower jaw of a patient according to the present invention.

[0024] FIG. 2 is a pre-operative CBCT-derived iso-surface of a patient's jaw with the pertinent features, such as the teeth stored as a 3D mesh model.

[0025] FIG. 3 illustrates the trace of the teeth of the lower jaw in FIG. 1 created by the probe during surgery.

[0026] FIG. 4 illustrates the alignment/registration of the probe trace of FIG. 3 with the pre-operative CBCT scan.

[0027] FIG. 5A illustrates an enlargement of a portion of the probe trace prior to alignment with a portion of the teeth in the CBCT scan.

[0028] FIG. 5B illustrates the portion of the probe trace of FIG. 5A after alignment with the teeth in the CBCT scan.

[0029] FIG. 6A illustrates visualization of the interactive transform refinement prior to the alignment step.

[0030] FIG. 6B illustrates visualization of the interactive transform refinement after the alignment step.

[0031] FIG. 6C illustrates extraction of the point and plane for the interactive transform refinement

[0032] FIG. 7A illustrates several probe location points and corresponding CT planes prior to alignment by the interactive transform refinement.

[0033] FIG. 7B illustrates several probe location points and corresponding CT planes subsequent to alignment by the interactive transform refinement

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] A better understanding of various features and advantages of the present methods and devices may be obtained by reference to the following detailed description of illustrative embodiments of the invention and accompanying drawings. Although these drawings depict embodiments of the contemplated methods and devices, they should not be construed as foreclosing alternative or equivalent embodiments apparent to those of ordinary skill in the subject art.

[0035] Referring to the drawings, and initially to FIG. 1, a stereo tracking system 10 is disclosed which permits live tracking a surgical tool relative to a pre-operative CBCT scan. The stereo tracking system 10 includes a stereo camera system 12, which may be any conventional system that is capable of capturing images that can be used for determining three dimensional location of an item being viewed. Typically such systems include at least two cameras 12A, 12B mounted above the surgical area. Those skilled in the art are familiar with such systems and, thus, no further discussion of such systems are necessary.

[0036] In a preferred embodiment, the stereo tracking system includes a lighting system, such as with blue or violet lighting, for providing illumination of a surgical or pre-surgical area. The lighting is configured to reflect off tracking reference components 14, 16 located in the surgical or presurgical area to facilitate capture by the cameras. The tracking reference components are objects that are covered with contrast patterns 18 that are unique to each type of tracking reference component (i.e., the patterns on a tracking reference component attached to a patient 14 are different than the patterns on the tracking reference component attached to an instrument). The illuminated contrast patterns 18 are observed by each of the two stereo tracking cameras, whose relative position is precisely known. Correspondences between each pair of images allow the tracking system to determine the location and orientation of each tracking reference component relative to the two tracking cameras. Suitable tracking reference components are described in detail in pending U.S. patent application Ser. No. 14/487,987, 14/488,004, 29/539,394, and 29/519,637, the disclosures of which is incorporated herein by reference in their entirety.

[0037] In an embodiment, the stereo cameras 12 track three separate tracking reference components 14, 16 (not necessarily at the same time). One of the tracking reference components, the patient tracking component 14 is attached rigidly and removable to the patient's teeth 14, such as with an oral fixture 20 that includes a thermoplastic impression of several teeth and described in U.S. patent application Ser. No. 14/209,500, the disclosure of which is incorporated herein by reference in its entirety. The oral fixture 20 removably snaps or engages with the teeth and positions the patient tracking component 14 at a location that can been seen (captured) by the two cameras.

[0038] A second tracking reference component, the probe tracking component 16, is rigidly attached to a probe device 22 which is used to trace portions of the patient's teeth as will be discussed below. The third tracking reference component is the surgical tool tracking component (not shown) that is rigidly attached to the surgical device (not shown) used to drill an osteotomy and deliver the implant into the patient jaw. The surgical device or tool could be the same instrument as the probe device 22 with the probe tip 24 switched to a drill bit. As such, the reference tracking component 16 for the probe device 22 and the surgical drill could be the same.

[0039] Prior to surgery, a cone beam computed tomography (CBCT) scan is acquired, including the patient's surgical anatomy of interest (usually the entire mandible or the entire maxilla). CBCT scans are well known in the art so no further discussion is needed. The surgeon imports the CBCT image into a software image guidance system, such as the X-Guide™ Surgical Guidance System sold by X-Nav Technologies, LLC. and defines the desired location and orientation of the implants (i.e., creates a surgical plan). The surgeon also defines the locations of the two most distal molars (or where they would be if missing) and the most mesial location on the dental arch.

[0040] Next, the CBCT is processed using a computer graphics algorithm, such as marching cubes which is a high resolution computer graphics 3D surface constructions algorithm published by William E. Lorensen and Harvey E. Cline in: Computer Graphics, Vol. 21, Nr. 4, July 1987, which is used to extract a polygonal mesh from the CBCT scan (which is incorporated herein by reference in its entirety). Using the marching cubes algorithm, an iso-surface 30 at the level of the bone-air transition can be automatically extracted. See FIG. 2. An 800 HU (Hounsfield Unit), which is a scale based on radiodensities of various materials, is typically a suitable value for the iso-surface. The HU scale is well known to those skilled in the art. This iso-surface 30 that is created is a 3D surface model of the patient's jaw and teeth.

[0041] On the day of surgery, the surgical staff performs a series of calibrations. The staff first calibrates the probe device 22. As mentioned above the probe device 22 has an attached probe tracking reference component 16, preferably attached to the handle. The probe device 22 includes a probe tip 24 that has a small, precise, stainless steel sphere. The calibration is performed using the probe device 22 and a calibration plate. The calibration plate is a tracking reference component with a flat surface and a conical divot formed in it for receiving the sphere. The flat surface includes contrast patterns 18 similar to the ones described above, for example, the contract patterns described in U.S. Patent Applications 29/519,642 and 14/488,004 (the disclosures of which is incorporated herein by reference in its entirety). With the camera tracking system on, the probe device's sphere tip is inserted into the conical divot and the probe device is pivoted with respect to the calibration plate, generating relative estimates between the probe tracking reference component 16 and the calibration plate (for example, about 500 estimates). The location of the pivot point (i.e., the center of the spherical tip) is solved with respect to the probe tracking reference component's coordinate system by solving a linear least-squares system of equations such that the residual error over all relative estimates is minimized. Once the location of the probe tip sphere is known, the tracking system can determine the location of the center of the probe tip sphere based on the detected location of the probe tracking reference component 16.

[0042] The surgical staff also calibrates the drill that will be used to drill the osteotomy and to place the implant. The drill to be used has a tracking reference component integrated into the handle. As mentioned above, the drill and the probe device may be the same tool with only the tips switched. The drill calibration is performed in two steps. In the first step, a special drill bit, with an attached tracking reference component, is inserted into the chuck of the drill. The bit is rotated slowly (e.g., about 10 RPM) in such a way that the tracking system (i.e., the cameras 12A, 12B) can capture both tracking reference components and determine a suitable number of relative estimates between the two tracking reference components (e.g., approximately 500 estimated calculations.) The image guidance system uses a linear least squares system of equations to solve for the axis of rotation that minimizes the residual error over all accumulated estimates.

[0043] In the second step, a drill bit (or the implant) is inserted into the chuck of the drill and its tip is held against and perpendicular to the flat surface of the calibration plate. The tracking system is then activated and the doctor or technician moves (rotates and pivots) the tool about its tip so that the cameras can capture the relative position between the calibration plate and the drill tracking reference component. The combination of these steps determines the axis and location of the tip of the drill bit with respect to drill tracking reference component. The calibration procedure is described, for example, in U.S. patent application Ser. No. 14/209,500, the disclosure of which is incorporated herein by reference in its entirety. The second step is repeated for each drill bit or implant used during the surgery.

[0044] When the patient arrives, the surgical staff attaches the patient tracking reference component to the patient by affixing it to the patient's teeth using the oral fixture described above.

[0045] Using the probe device, the surgeon holds the probe tip near each of the three arch locations that were defined during planning, and interacts with the software to indicate that the tip is in the desired location. By simultaneously tracking both the patient tracking reference component 14, and the probe tracking reference component 14, the tracking system can locate where the probe tip 24 is with respect to the patient tracking reference component 14, which defines a floating coordinate system (coordinate system 32 in FIG. 3) because it has not yet been related to the patient's CBCT scan. Probing each of the three arch locations defines three notional, but inaccurate correspondences between the CBCT-derived coordinate system and the floating patient coordinate system, which is a sufficient number to perform absolute orientation to derive a best-fit transform. While too inaccurate in most cases to support surgical navigation, this provides an initial estimate of the true transform that relates the floating patient coordinate system to the pre-operative CBCT coordinate system (coordinate system 34 in FIG. 2). This initial estimate of the transform serves a dual purpose: (i) it allows for meaningful visual feedback while the surgeon is performing the probe trace in the next step, and (ii) once the probe trace has been performed, it facilitates the refinement of the transform because the initialization can be assumed to be close to the final transform.

[0046] The surgeon now begins the probe trace. Referring to FIG. 3, the surgeon places the probe tip on a tooth surface, and interacts with the tracking system to indicate the beginning of a probe trace. The surgeon slowly traces the probe's spherical tip over each of the patient's exposed teeth, tracing points on the occlusal, buccal, and lingual surfaces. With the tip of the probe still in contact with a tooth surface, the surgeon interacts with the tracking system to indicate the end of the trace. A sample of the probe trace is shown in FIG. 3 and identified as PT. Throughout the probe trace process, the tracking system simultaneously estimates the locations of the tracking reference components on the probe device 22 and on the patient, which in combination with the probe calibration generates a sequence of estimates of 3D points near (one probe sphere radius away from) the surface of the teeth. These probe trace points PT are still in a floating patient coordinate system (FIG. 3), and not yet related to the CBCT coordinate system.

[0047] The sequence of probe trace point estimates are processed in combination with the jaw and teeth iso-surfaces automatically extracted from the CBCT volume. The first step in this process is to offset (dilate) the iso-surface by the radius of the probe tip. FIG. 5A. In the preferred embodiment, this is done by offsetting each vertex outward along its normal direction, but in alternate embodiments this could be done by 1) creating a supersampled binary volume mask enclosed by the isosurface, 2) dilating the volume mask by adding any voxel whose voxel center is within one radius of an original volume mask voxel, 3) creating a new isosurface from the dilated voxel mask. In doing this offsetting, an ideal alignment of the probe trace points to the offset iso-surface would place all probe trace points in contact with (or very near to) the dilated iso-surface. The second step in this process is to align the probe trace points to the dilated iso-surface. FIG. 5B. The system uses a well-known iterative closest point (ICP) algorithm, which is typically used for aligning two surfaces to each other. ICP algorithm determines the transform that best aligns the trace points to the dilated iso-surface, and in doing so, determines the relationship between the floating coordinate system 32 and the CBCT coordinate system 34. FIG. 4. At this stage, whenever the probe tracking reference component 16 is tracked by the system simultaneously with the patient tracking reference component 14, the location of the probe sphere center can be related back to a 3D coordinate 34 in the CBCT volume, and the sphere center depicted with respect to the CBCT depiction of the patient anatomy (FIG. 4). Similarly, whenever the drill tracking reference component is tracked simultaneously with the patient tracking reference component 14, the axis and tip of the drill bit (or implant) can be related back to the CBCT volume, and the drill's location and orientation can be depicted with respect to the CBCT depiction of the patient anatomy and implant plan.

[0048] The surgeon is now free to use the drill to deliver the implant using any conventional dental implant guidance system, such as the X-Guide® Guidance System described in the above reference pending applications.

[0049] There are several variations that are possible in the present invention. For example, the probe tip does not need to be spherical, it can, instead, have a defined point. As mentioned above, the probe device and the drill can be the same device, provided a sphere-ended or pointed drill bit is used. Other bit geometries can be used with diminished accuracy.

[0050] The present invention does not need to be used with an optical stereo tracking system. Instead, any tracking system that can determine the location and orientation of two components in a single reference frame is suitable for use with the above described fiducial free tracking system. Electromagnetic trackers can be used if an E/M sensor is attached to each of the probe device and to the patient. Alternately the field generator may be attached to either the probe device or the patient, with a sensor being attached to the other. Likewise mechanical encoding arms can be used to track the probe and/or the patient.

[0051] The tracking reference component attached to the probe device does not always need to determine the probe device's location in a 6DOF sense. A SDOF tracking reference component can be used if it is aligned such that the axis of the center passes through the probe sphere center or tip. A 3DOF tracking reference component can be used if it is co-located with the probe sphere center or tip.

[0052] There are many variants on providing initialization transforms. No initialization transform is strictly needed. Initialization can be provided by using user-defined point correspondences between points on the trace and corresponding points on the CT, or by interactive manipulation of the trace in the CT coordinate frame.

[0053] Initialization by manipulation of the trace can be performed by graphically depicting the trace overlaid upon a 2D or 3D representation of the CBCT scan (e.g., 2D slices possibly including iso-surface outlines, volume renderings or iso-surface renderings), the trace being related to the CBCT scan by its current rigid transform estimate. The user can interact with a system to adjust the rigid transform, effectively translating or rotating the trace as a whole with respect to the underlying CT representation until they are visually aligned. This process can be assisted by correspondences consisting of points selected on the trace and corresponding points selected on the CT representation.

[0054] Existing tracking systems can also provide initialization through use of fiducials identified in the Pre-operative CBCT scan. In these cases, the probe trace can be used to refine the transform and correct for any errors due to removal and replacement of the patient attachment between CBCT and surgery.

[0055] Additional interactive refinement to initialization can be performed prior to or as an alternative to performing automated trace alignment. This process is depicted in FIG. 7A and FIG. 7B. In one such method, the surgeon places the tip 50 of a pointed probe 52 in contact with a point on the anatomy A in such a way that the probe tip 50 is aligned with its axis approximately normal to the surface it is in contact with. The surgeon indicates to the system to select this tip location as a point for refinement. A set of interactive CT representations is displayed. In one embodiment, this is a set of 3 orthogonal slices S as depicted in FIG. 6A, oriented with respect to the estimated probe tip's canonical axes (whose origin is at the probe tip, whose Z axis is aligned with the probe tip axis, with the probe handle aligned in the X/Z plane). The system places a line 54 on the display, drawn orthogonal to Z on each slice, which depicts the projection of a plane orthogonal to the probe tip axis. The surgeon can move this line up or down the probe axis until it just touches the surface of the anatomy A whose transition is visible in the CT, in an attempt to correct for any error in this axis. For example, the surgeon can use the roller wheel on a mouse to move the line upwards or downwards on the display to align it with the surface of the anatomy A. Once this adjustment is complete, at which point the system would look similar to FIG. 6B, the surgeon interacts with the system to finish the adjustment, and the system stores 1) the plane equation in CT coordinates, and 2) the probe tip location and direction in patient tracker coordinates as depicted in FIG. 6C. The surgeon defines additional points and corresponding planes in the same way, after which time, the system performs an optimization over the rigid body transform T relating the patient tracker coordinate system to the CT coordinate system. In one embodiment, this optimization minimizes the sum of squared distances between the planes and their corresponding patient tracker points transformed into CT coordinates via transform T. Variants to this optimization can add constraints on T and/or impose penalties on the distance between the transformed point and the center of the plane (where the original Z axis intersected the plane) in order to reduce the number of correspondences required or to improve robustness or convergence speed. The surgeon can repeat this process one or more times in order to further refine the estimate of T. This process can either be used to improve the initialization to, and therefore the robustness of automated trace alignment, or can be used instead of automated trace alignment. This can be desirable, e.g., in fully edentulous patients, where soft tissue surfaces probed with a spherical probe are difficult to automatically locate on the CT, and where instead a pointed probe can puncture through to the hard tissue, which can be more easily discerned interactively by the surgeon.

[0056] The method is not restricted to oral anatomy. It can be used on any rigid body part. Rigid does not necessarily mean hard. The probe trace could be performed on soft tissue as well, provided the deformation is limited between when the 3D model is created and when the trace is performed.

[0057] The method is not restricted to 3D models derived from a CBCT or CT scan of a patient. For example, the above described method would work equally well for relating the tracking system to a pre-operative intra-oral optical scan with respect to which an implant plan can be defined pre-operatively.

[0058] The method is not restricted to medical applications. It could, for example, be used in registering a known mechanical part on a CNC milling machine, prior to secondary machining.

[0059] While the above description refers to a surgical tool or instrument that includes a drill, the term “surgical instrument” or “surgical tool” is intended to cover other tools used during intraoral procedures, such as ablation tools for ablating tissue, including third molars in children.

[0060] The system or systems described herein may be implemented on any form of computer or computers and the algorithms and programs may be implemented as dedicated applications or in client-server architectures, including a web-based architecture, and can include functional programs, codes, and code segments. The computer system of the present invention may include a software program be stored on a computer and/or storage device (e.g., mediums), and/or may be executed through a network. The computer steps may be implemented through program code or program modules stored on a storage medium.

[0061] For the purposes of promoting an understanding of the principles of the invention, reference has been made to the preferred embodiments illustrated in the drawings, and specific language has been used to describe these embodiments. However, no limitation of the scope of the invention is intended by this specific language, and the invention should be construed to encompass all embodiments that would normally occur to one of ordinary skill in the art.

[0062] The computer processes herein may be described in terms of various processing steps. Such processing steps may be realized by any number of hardware and/or software components that perform the specified functions. For example, the described embodiments may employ various integrated circuit components, e.g., memory elements, processing elements, logic elements, look-up tables, and the like, which may carry out a variety of functions under the control of one or more microprocessors or other control devices. Similarly, where the elements of the described embodiments are implemented using software programming or software elements the invention may be implemented with any programming or scripting language such as C, C++, Java, assembler, or the like, with the various algorithms being implemented with any combination of data structures, objects, processes, routines or other programming elements. Functional aspects may be implemented in algorithms that execute on one or more processors. Furthermore, the embodiments of the invention could employ any number of conventional techniques for electronics configuration, signal processing and/or control, data processing and the like. The words “mechanism” and “element” are used broadly and are not limited to mechanical or physical embodiments, but can include software routines in conjunction with processors, etc.

[0063] The particular implementations shown and described herein are illustrative examples of the invention and are not intended to otherwise limit the scope of the invention in any way. For the sake of brevity, conventional electronics, control systems, software development and other functional aspects of the systems (and components of the individual operating components of the systems) may not be described in detail.

[0064] Finally, the steps of all methods described herein are performable in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. Numerous modifications and adaptations will be readily apparent to those skilled in this art without departing from the spirit and scope of the invention.