SURGICAL NAVIGATION SYSTEMS AND METHODS

Abstract

A system, including various apparatus and methods, for surgical navigation is provided. The system is configured to track the spine of a patient by capturing images via one or more cameras. The cameras are configured to capture images of one or more arrays. The system transmits the images to a computer system. The one or more arrays are releasably secured with the spine of the patient, such as by a spine pin or a spine clamp. The system can determine the spatial position and orientation of relevant anatomical features, implants, and instruments using and processing the captured images.

Claims

1-20. (canceled)

21. A computer-implemented method for surgical navigation, comprising: displaying a start-up menu having a selection area on a display screen; receiving a first input from the selection area having case-specific information; obtaining an image data set based on the first input; displaying a surgical planning menu on the display screen based on the first input; receiving a second input from the surgical planning menu including any one of (i) a level selection, (ii) a screw type selection, (iii) a screw size selection, or (iv) any combination of (i)-(iii); displaying a surgical planning screen on the display screen, the surgical planning screen having one or more views of the image data set; receiving a third input from the surgical planning screen including any one of (v) a determined anatomical measurement, (vi) a trajectory of a screw, or (vii) any combination of (v)-(vi); and generating a surgical plan based on the first input, the second input, and the third input.

22. The method of claim 21, wherein the case-specific information includes any one of (viii) an anatomical region that is subject of a surgical procedure, (ix) one or more levels of the surgical procedure, (x) a surgical approach, or (xi) any combination of (viii)-(x).

23. The method of claim 21, further comprising: retrieving a stored template and a surgeon profile; displaying a menu of the stored template and the surgeon profile on the display screen; and receiving a fourth input from the menu of the stored template and the surgeon profile comprising a selection of any one of (viii) the stored template, (ix) the surgeon profile, or (x) the stored template and the surgeon profile, wherein displaying the surgical planning menu and displaying the surgical planning screen is based on the fourth input.

24. The method of claim 21, further comprising: displaying a registration menu screen on the display screen; receiving a selection of a registration method from the registration menu screen; receiving one or more images; and performing registration using the one or more images based on the selection of the registration method.

25. The method of claim 21, further comprising: displaying an instrument setup menu on the display screen; receiving a fourth input from the instrument setup menu comprising a description of an instrument; displaying an array assignment menu on the display screen; receiving a fifth input from the array assignment menu comprising an assignment of an array to the instrument; receiving one or more images of the array and a second array configured to be releasably secured with a vertebra via a connection with a spine pin; determining a surgical instrument location and an anatomy location based on the one or more images; generating a surgical representation including a surgical instrument representation overlaid on an anatomy representation based on the fourth input, the fifth input, the surgical instrument location, and the anatomy location; and displaying the surgical representation on the display screen.

26. The method of claim 25, further comprising: generating a simulated trajectory on the surgical representation displayed on the display screen based on the surgical instrument location and the anatomy location.

27. The method of claim 26, further comprising: displaying a tool menu with the surgical representation on the display screen, the tool menu comprising options that include any one of (viii) a system type option, (ix) an implant type option, (x) a screw type option of the simulated trajectory, or (xi) any combination of (viii)-(x); receiving a sixth input from the tool menu comprising a selected option; and generating the selected option on the surgical representation displayed on the display screen based on the simulated trajectory.

28. The method of claim 21, further comprising: generating a view of a surgical site based on the surgical plan; and displaying the view of the surgical site on the display screen.

29. A system for surgical navigation, comprising: a display screen; and a computer system including a processor, wherein the computer system is configured to: display a start-up menu comprising a selection area on the display screen; receive a first input from the selection area comprising case-specific information; obtaining an image data set based on the first input; display a surgical planning menu on the display screen based on the first input; receive a second input from the surgical planning menu comprising any one of (i) a level selection, (ii) a screw type selection, (iii) a screw size selection, or (iv) any combination of (i)-(iii); display a surgical planning screen on the display screen, the surgical planning screen comprising one or more views of the image data set; receive a third input from the surgical planning screen comprising any one of (v) a determined anatomical measurement, (vi) a trajectory of a screw, or (vii) any combination of (v)-(vi); and generate a surgical plan based on the first input, the second input, and the third input.

30. The system of claim 29, wherein the case-specific information comprises any one of (viii) an anatomical region that is subject of a surgical procedure, (ix) one or more levels of the surgical procedure, (x) a surgical approach, or (xi) any combination of (viii)-(x).

31. The system of claim 29, wherein the computer system is further configured to: retrieve a stored template and a surgeon profile; display a menu of the stored template and the surgeon profile on the display screen; and receive a fourth input from the menu of the stored template and the surgeon profile comprising a selection of any one of (viii) the stored template, (ix) the surgeon profile, or (x) the stored template and the surgeon profile, wherein to display the surgical planning menu and to display the surgical planning screen is based on the fourth input.

32. The system of claim 29, wherein the computer system is further configured to: display a registration menu screen on the display screen; receive a selection of a registration method from the registration menu screen; receive one or more images; and perform registration using the one or more images based on the selection of the registration method.

33. The system of claim 29, wherein the computer system is further configured to: display an instrument setup menu on the display screen; receive a fourth input from the instrument setup menu comprising a description of an instrument; display an array assignment menu on the display screen; and receive a fifth input from the array assignment menu comprising an assignment of an array to the instrument.

34. The system of claim 33, further comprising: a second array being configured to be releasably secured with a vertebra via a connection with a spine pin; and at least one camera configured to track the array and the second array, wherein the computer system is further configured to: receive one or more images of the array and the second array from the at least one camera; determine a surgical instrument location and an anatomy location based on the one or more images; generate a surgical representation including a surgical instrument representation overlaid on an anatomy representation based on the fourth input, the fifth input, the surgical instrument location, and the anatomy location; and display the surgical representation on the display screen.

35. The system of claim 34, wherein the computer system is further configured to: generate a simulated trajectory on the surgical representation displayed on the display screen based on the surgical instrument location and the anatomy location.

36. The system of claim 35, wherein the computer system is further configured to: display a tool menu with the surgical representation on the display screen, the tool menu comprising options that include any one of (viii) a system type option, (ix) an implant type option, (x) a screw type option of the simulated trajectory, or (xi) any combination of (viii)-(x); receive a sixth input from the tool menu comprising a selected option; and generate the selected option on the surgical representation displayed on the display screen based on the simulated trajectory.

37. The system of claim 29, wherein the computer system is further configured to: generate a view of a surgical site based on the surgical plan; and display the view of the surgical site on the display screen.

38. The system of claim 37, wherein the view of the surgical site on the display screen comprises any one of (viii) a simulated trajectory based on the surgical plan, (ix) a bookmark screw based on the surgical plan, or (x) a combination of (viii) and (ix).

39. The system of claim 38, wherein the computer system is further configured to: receive a fourth input from the view of the surgical site; and based on the fourth input, removing from the view of the surgical site on the display screen any one of (xi) the simulated trajectory, (xii) the bookmark screw, or (xiii) a combination of (xi) and (xii).

40. The system of claim 29, wherein the one or more views of the image data set comprise any one of (viii) an axial view, (ix) a sagittal view, (x) a coronal view, or (xi) any combination of (viii)-(x).

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] FIG. 1 illustrates a computer system according to one embodiment of the navigation system.

[0017] FIG. 2 illustrates an optical tracking system according to an embodiment of the navigation system.

[0018] FIG. 3 illustrates an optical tracking system and a mobile cart according to an embodiment of the navigation system.

[0019] FIG. 4A illustrates an assembled tracker according to an embodiment of the navigation system.

[0020] FIG. 4B is an exploded view of the assembled tracker of FIG. 4A.

[0021] FIG. 4C is a cross section view of a spine pin according to an embodiment of the navigation system.

[0022] FIG. 5 is an exploded view of an array according to an embodiment of the navigation system.

[0023] FIG. 6A illustrates a front perspective view of an array according to one embodiment of the navigation system.

[0024] FIG. 6B illustrates a front perspective view of an array according to another embodiment of the navigation system.

[0025] FIG. 6C illustrates a front perspective view of an array according to an embodiment of the navigation system.

[0026] FIG. 6D illustrates a front perspective view of an array according to yet another embodiment of the navigation system.

[0027] FIG. 6E illustrates a front perspective view of an array according to an embodiment of the navigation system.

[0028] FIG. 6F illustrates an exploded view of the array of FIG. 6E.

[0029] FIG. 6G illustrates a front perspective view of an array according to another embodiment of the navigation system.

[0030] FIG. 6H illustrates an exploded view of the array of FIG. 6G.

[0031] FIG. 6I illustrates a front perspective view of an array according to yet another embodiment of the navigation system.

[0032] FIG. 6J illustrates an exploded view of the array of FIG. 6I.

[0033] FIG. 6K illustrates elevation views over 90 degree rotations of the array of FIG. 6I.

[0034] FIG. 7 illustrates a spine pin according to an embodiment of the navigation system.

[0035] FIG. 8A illustrates a spine pin according to another embodiment of the navigation system.

[0036] FIG. 8B illustrates a spine pin and an array (as a tracker assembly) according to an embodiment of the navigation system.

[0037] FIG. 8C illustrates a spine pin and an array (as a tracker assembly) according to yet another embodiment of the navigation system.

[0038] FIG. 8D illustrates a front perspective view of a clamp according to an embodiment of the navigation system.

[0039] FIG. 8E illustrates a front perspective view of a clamp according to another embodiment of the navigation system.

[0040] FIGS. 9A-9C illustrate the use of a spine tracker inserter affixing a spine pin and an array to a spinous process in accordance with an embodiment of the navigation system.

[0041] FIG. 10 illustrates arrays and spine pins removably secured with a spine of a patient.

[0042] FIGS. 11A-11H illustrate an instrument with an integrated 360 degree variously rotated around an axis according to an embodiment of the navigation system.

[0043] FIG. 12A illustrates a C-arm array mount according to an embodiment of the navigation system.

[0044] FIG. 12B illustrates the C-arm mount array of FIG. 12A mounted to a C-arm.

[0045] FIG. 13 illustrates calculations used to identify one or more spine pins from received images.

[0046] FIGS. 14A-14C illustrate images of a set of spine pins affixed to the spinous processes of first, second, and third vertebrae.

[0047] FIG. 15 depicts a segmentation process according to an embodiment of the navigation system.

[0048] FIGS. 16A-16C illustrate vertebrae segmentation from a CT model of the spine according to an embodiment of the navigation system.

[0049] FIGS. 17A-17D illustrate a segmented 3D image set and various 2D DRR views of the segmented 3D image set according to an embodiment of the navigation system.

[0050] FIGS. 18A and 18B illustrate images with a set of spine pins affixed to the spinous processes of first, second, third, fourth, fifth and sixth vertebrae captured by a C-arm according to an embodiment of the navigation system.

[0051] FIG. 19 illustrates a start-up menu on a display screen according to an embodiment of the navigation screen.

[0052] FIG. 20 illustrates a level selection menu on a display screen according to an embodiment of the navigation system.

[0053] FIG. 21 illustrates a template and surgeon profile menu on a display screen according to an embodiment of the navigation system.

[0054] FIG. 22 illustrates a setup screen menu of stored templates on a display screen according to an embodiment of the navigation system.

[0055] FIG. 23 illustrates a surgical planning screen displayed on a display screen according to an embodiment of the navigation system.

[0056] FIG. 24 illustrates another surgical planning screen displayed on a display screen according to an embodiment of the navigation system.

[0057] FIG. 25 illustrates a 2D-3D image registration menu screen displayed on a display screen according to an embodiment of the navigation system.

[0058] FIGS. 26A-26D illustrate a registration and calibration algorithm screen for the received images displayed on a display screen according to an embodiment of the navigation system.

[0059] FIGS. 27A-27D illustrate a verification screen for the resulting registration determination displayed on a display screen according to an embodiment of the navigation system.

[0060] FIG. 28 illustrates a 3D-3D image registration menu screen displayed on a display screen according to an embodiment of the navigation system.

[0061] FIG. 29 illustrates an instrument setup menu screen displayed on a display screen according to an embodiment of the navigation system.

[0062] FIG. 30 illustrates an array assignment menu screen displayed on a display screen according to an embodiment of the navigation system.

[0063] FIGS. 31-33 illustrate projected trajectory and screw screens displayed on a display screen according to an embodiment of the navigation system.

[0064] FIG. 34 illustrates a segmented vertebrae screen displayed on a display screen according to an embodiment of the navigation system.

[0065] FIGS. 35A and 35B illustrate a bookmarked screw feature displayed on a display screen according to an embodiment of the navigation system.

[0066] FIGS. 36A and 36B illustrate surgical plan menu screens displayed on a display screen according to an embodiment of the navigation system.

[0067] FIGS. 37 and 38 illustrate planned trajectories screens according to planned screws from pre-surgery displayed on a display screen according to another embodiment of the navigation system.

[0068] FIG. 39 illustrates a screen displaying the bookmarked screw feature displayed on a display screen according to another embodiment of the navigation system.

[0069] FIG. 40 is a flowchart that illustrates an embodiment of the navigation system.

[0070] FIG. 41 is a block diagram that illustrates an embodiment of the navigation system.

DETAILED DESCRIPTION

[0071] Illustrative embodiments of a system for surgical navigation system and related methods are described below. In the interest of clarity, not all features of an actual implementation are described in this specification. It will of course be appreciated that in the development of any such actual embodiment, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which will vary from one implementation to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the art having the benefit of this disclosure. The system for surgical navigation system and related methods disclosed herein boast a variety of inventive features and components that warrant patent protection, both individually and in combination.

[0072] As shown and described herein, the term “navigation” describes the ability to leverage intraoperative imaging in real-time to obtain spatial awareness between anatomical structures and instrumentation. The navigation system provides maximum surgeon visualization with minimal radiation exposure through an innovative and efficient solution during MIS and traditional open spine procedures. Specifically, the navigation system enables surgeons to perform 3D image-guided surgery with increased accuracy exposure during posterior and anterior column procedures. This provides benefits for surgeons and hospitals alike: 1) for degenerative and MIS surgeons who desire improved visualization while simultaneously reducing radiation exposure; 2) for deformity surgeons who desire real-time spinopelvic parameter assessment and anatomical orientation in complex spine procedures; and 3) for hospital administrators who desire to reduce the total cost of health care through more predictable outcomes form accurate implant placement and reduced morbidity of MIS procedures. Indeed, the system of the present disclosure provides for reduced total cost of care though facilitating more predictable outcomes, reduced OR time, reduced hospital length of stay, reduced post-op complication rates, and reduced revision operations (due to accurate and repeatable implant placement).

[0073] The navigation systems and methods described herein are compatible with spine procedures and the instruments and implants associated therewith. By way of example, the navigation system and methods described herein are compatible with open and MIS pedicle screw placements for thoracolumbar fusions, lateral interbody fusion procedures including lateral lumbar interbody fusion (XLIF), trauma procedures, maximum access surgery transforaminal lumbar interbody fusion (MAS TLIF), maximum access surgery posterior lumbar interbody fusion (MAS PLIF), lateral fixation procedures, corpectomies, anterior cervical discectomy and fusion (ACDF), and posterior cervical fusion (PCF). It is contemplated that the navigation systems and methods will integrate planning, such as the iGA platform by NuVasive, Inc., intraoperative monitoring, automated rod bending, etc. to provide a holistic view of the anatomy and foster enhanced procedural solutions.

[0074] In a first aspect, a navigation system 10 is provided. As shown in FIG. 1, the navigation system 10 may include one or more hardware components, one or more software components, and one or more auxiliary components. For example, the navigation system 10 may include a computer system 12 including a control unit 14 including at least one processor configured to execute computer executable instructions (i.e., software), and one or more display screens 16. The control unit 14 may be housed in a technology hub 18 having one or more locking wheels 20 disposed thereon such that the technology hub 18 may be easily positionable around the OR. The technology hub 18 may include one or more arms 22 connecting to the display screens 16. The control unit 14 may be configured for executing the application software and algorithms, and communicating and interfacing with other system components associated with the navigation system 10, such as auxiliary displays 24, remote control devices 26, such as tablets or phones, and mobile computing devices 28, such as intraoperative neuromonitoring technician laptops, and cloud remote and cloud planning systems 30.

[0075] The computer system 12 may receive universal imaging inputs, meaning that it has the ability to work with a pre-operative CT input, a pre-operative MRI input, a 3D C-arm input, or an intraoperative CT input. The imaging inputs may be formatted according to industry standards, such as the Digital Imaging and Communications in Medicine (DICOM) standard, the Picture Archive and Communication System (PACS) standard, Phase Alternating Line (PAL) standard, and National Television System Committee (NTSC) standard. The system 12 may receive the input over one or more networks (e.g., wired or wireless local area network, such as a hospital PACS) or via USB, CD, DVD, DVI, composite video, or analog video. Advantageously, as discussed herein, the present system 10 employs automatic registration with intra-operative and pre-op CT images, the system 10 is configured to perform segmentation of each vertebral body through image recognition, and the system 10 is configured to register individual vertebral bodies such that the spine can be dynamically tracked during the surgical procedure.

[0076] The one or more display screens 16 may be touch screens such that they include a graphical user interface (GUI) with which the user can directly input commands by touching the screen 16. The system 10 offers intuitive and convenient system interaction with the software and hardware available to surgeons (and users others within the surgical field) and other hospital personnel (outside the surgical field). While various descriptions of the aspects of the present disclosure may refer to a surgeon, or surgeons, it is to be understood that the functionality of such aspects may extend to other users, as contextually appropriate, such that the term “surgeon(s)” supports the term “user(s).” The software may be primarily controlled through the touch screen graphical user interface on the one or more display screens 16, which controls the navigation system 10. In one embodiment, the system 10 includes a secondary control through the one or more remote control devices 26.

[0077] The navigation system 10 receives data and inputs from various other parts of the system 10, including the 3D imaging data and optical camera(s) 34, 36 that track the vertebrae and surgical instruments (discussed in further detail below), surgeon inputs, and processing to provide real-time navigation information to the surgeon or OR personnel. The surgeon/OR personnel can interact with the navigation software from the sterile field for navigation view settings, instrument selection/calibration real-time implant planning and sizing, administrative features, and option selection. The software is controlled without interfering with other intraoperative computer-assisted modalities and the system 10 is able to easily transition between navigation modes and other modes, for example, intraoperative neuromonitoring (TOM) services, NUVAMAP O.R., and BENDINI software modes.

[0078] In a second aspect of the navigation system 10, the system 10 includes an optical tracking system 32, shown in FIGS. 2 and 3. The optical tracking system 32 may provide real-time location of objects (e.g., one or more body locations such as vertebral bodies of the spine, and one or more instruments for use in surgery) in relationship to each other as the objects move through space. The optical tracking system 32 may be in communication with the control unit 14 of the computer system 12 of the navigation system 10. The optical tracking system 32 may include one or more cameras that are infrared (IR) cameras 34 and/or visible light cameras 36 (i.e., sense and transmit data from the IR or visible light spectrums). Each camera 34, 36 may be selected between IR and visible light modes under software control by the control unit 14. The optical tracking system 32 senses (i.e., sees) the location of one or more tracking arrays 38 (FIGS. 4A and 4B) within the field of view of the system 32. The tracking arrays 38 may be positioned on one or more anatomical features 4 of a subject 2, such as vertebral bodies of a human undergoing spinal surgery, and one or more surgical instruments 6. The optical tracking system 32 provides the navigation system 10 with dynamic 3D position information corresponding to the anatomical features 4 and/or the surgical instruments 6 being tracked.

[0079] The optical tracking system 32 may be configured in any suitable orientation. In one embodiment, illustrated in FIG. 2, the optical tracking system 32 includes a first and a second IR camera 34 flanking a first visible light camera 36. The cameras 34, 36 may be discrete units or connected together by a camera base 37. The cameras 34, 36 may be compact enough to be positioned within a sterile field of a surgical procedure without interfering with the procedure. The cameras 34, 36 may contain a high number of pixels. As used herein, the term, “pixel” is used to refer to a single scalar element of a multi-component representation (also referred to as a photosite). The cameras 34, 36 may capture at least 1 megapixel, at least 2 megapixels, at least 5 megapixels, at least 10 megapixels, at least 12 megapixels, at least 15 megapixels, or at least 20 megapixels. A thin, transparent barrier 40 may be placed over the lenses 41 of the cameras 34, 36. Advantageously, a high pixel count enables the barrier 40 to be placed over the lenses 41 of the cameras 34, 36 while the cameras 34, 36 are in-use without sacrificing the accuracy of the position of the sensed tracking arrays 38. The barrier 40 also enables the cameras 34, 36 to be draped and placed within the sterile field. Another benefit of embodiments of the system 10 having the barrier 40 is that the barrier 40 enables the cameras 34, 36 to be in close proximity with the tracking arrays 38, which further allows the arrays 38 to be reduced in dimensions such that the arrays 38 are less likely to interfere with the surgical procedure being performed.

[0080] The optical tracking system 32 may be used with markers 42 disposed on one or more arrays 38 (discussed below). The markers 42 may be small in size (e.g., 3 mm diameter or as small as technologically feasible) with a minimal tracking array footprint (less than 10 mm between markers 42 allowing for small array 38 size). In addition to tracking spatially track arrays 38, the optical tracking system 32 may track objects that have arrays 38 as the objects change orientation (e.g., rotation, yaw, roll). The optical tracking system 32 may be positioned within the OR to minimize the potential for line-of-sight disruptions with the subject 2 for the surgeon performing the surgical procedure.

[0081] In embodiments of the navigation system 10 where the cameras 34, 36 are placed outside of the sterile field, the cameras 34, 36 may be placed on a mobile cart 44 (FIG. 3) with one or more locking wheels 46 such that the cart 44 may be positioned variously by rolling the cart 44 within the OR. The cart 44 may be placed proximate to one end of a surgical bed. The cart 44 may comprise a base 48 for receiving the cameras 34, 36. The base 48 may be lockingly adjustable, including height, longitudinally, and laterally so that the cameras 34, 36 may be optimally positioned for the surgical procedure.

[0082] In embodiments of the navigation system 10 where the cameras 34, 36 are placed within the sterile field, the draped cameras 34, 36 may be configured to view the C-arm 194, arrays 38 (including on anatomical features 4 and/or instruments 6) by placing one or more cameras 34, 36 at one of the following locations: patient anchor attachment, bedrail attachment, cart attachment, an overhead boom/light attachment, or any combination thereof. Some embodiments of the navigation system 10, discussed below, include the optical tracking system 32 that allows a single (i.e., initial) set up of the cameras 34, 36 with no additional adjustments necessary or made during a surgical procedure, thereby improving surgical workflow efficiency by eliminating the need for hospital personnel to adjust and re-adjust the cameras 34, 36 during the operative procedure to “see” or calibrate the navigated instruments 6 or the markers 42.

[0083] Referring to FIGS. 4-12B, in a third aspect, the navigation system 10 includes one or more tracking arrays 38 that can be securely disposed on the anatomical features 4 and/or the surgical instruments 6. The navigation system 10 tracks the tracking arrays 38, such by information received by cameras 34, 36, to effectuate the navigation and tracking of anatomical features 4 and surgical instruments 6 as will be described in greater detail below. According to one or more implementations, the navigation system 10 may track the spine using tracker 50.

[0084] As shown in the embodiment of FIGS. 4A-4B, the tracker 50 (i.e., the assembly of the array 38 and pin 52) comprises the array 38 having one or more markers 42 disposed thereon. The arrays 38 may securely, or lockingly and releasably, engage with a spine pin 52. The markers 42 may be variously positioned on the array 38. The markers 42 may be positioned at various points on a first surface 56 of the array 38. The arrays 38 may be scattered over the first surface 56, as shown in FIGS. 4A and 4B. The spine pin 52 may be positioned with or engage one or more anatomical feature 4 (e.g., vertebrae) by use of a spine tracker inserter 54 (FIGS. 9A-9C). Advantageously, when engaged via the spine pin 52, navigation system 10 may sense and track the position of the arrays 38 engaged with the spine pin 52, thereby tracking the position of the vertebrae 4 with which the spine pin 52 is engaged. The tracker 50 may have a first end 58 and an oppositely disposed second end 60, the array 38 disposed at the first end 58 and the spine pin 52 disposed at the second end 60. At the terminus of the second end 60, the spine pin 52 may include a sharp tip 62 configured to engage and be removably insertable into anatomical feature 4. Thus, when the array 38 is connected to the spine pin 52 that is inserted with the anatomical feature 4, such as vertebrae, the tracker 50 enables individual vertebral level registration and dynamic spine tracking as will be explained in greater detail below. The spine pin 52 may include an axial retention feature 76 (e.g., a helical thread) proximate to the tip 62 for axially securing the pin 52 with the anatomical feature 4.

[0085] The tracker 50 may include one or more radiographic glyph elements 78, such as a sphere. The tracker 50 may include a first radiographic glyph element 78 proximate to where the array 38 interfaces with the spine pin 52, and a second radiographic glyph element 78 proximate to the axial retention feature 76. The distance 80 between the radiographic glyph elements 78 may be from 5 mm to 100 mm, from 15 mm to 80 mm, about 40 mm, or any value or subrange thereof. As shown in FIG. 4C, an array end 82 of the spine pin 52 may have a tri-lobular profile (i.e., cross-section) to prevent the rotation of the pin 52 when the pin 52 is engaged with the array 38. The array 38 may be coupled with the spine pin 52 after the spine pin 52 is secured with the anatomical feature 4. The spine pin 52 may serve as a fiducial for image registration with the system 10, allowing for registration within multiple degrees of freedom, such as at least 4 degrees of freedom, at least 5 degrees of freedom, or at least 6 degrees of freedom, as will be explained in greater detail below.

[0086] FIG. 5 illustrates an exploded view of the array 38 according to an embodiment of the system 10. The array 38 comprises a connecting portion 64 for securely coupling the array 38 at the first end 58 of the tracker 50. The connecting portion 64 is sized and dimensioned to securely couple with to a complementary surface (e.g., a reference surface) 66 of the tracker 50. The array 38 includes a body 68 for supporting the various components of the array 38, such as the markers 42 and a cover 70 for protecting the markers 42. The markers 42 may be disposed on opposite sides (e.g., on the first surface 56 and an oppositely facing second surface 72) of the array 38. The markers 42 may be at least partially received in complementarily-shaped recesses 74 in the body 68. The markers 42 and/or body 68 may be provided in a sterile form such that they can be utilized within the sterile surgical field. The markers 42 may be provided in sterile packaging. The protective cover 70 may be clear and thin, and protect the markers 42 from being obscured from the system 10, particularly the optical tracking system 32, by debris such as blood from the patient undergoing surgery. The cover 70 may be constructed of a polymer or glass, or may be a film. The cover 70 may be polarized to limit the field of view if so desired.

[0087] The markers 42 may be reflective and hemispherical in shape, as shown in FIG. 5, or spherical in shape. In embodiments of the system 10 having hemispherical markers 42, the hemispherical shape offers several advantages relative to a reflective sphere. Advantages of the hemispherical shape include the ability to be more fully recessed with recesses 74 such that the viewing angle is physically limited so that the markers 42 are not presented to the camera when other markers 42 are directly facing the cameras 34, 36 allowing for accurate marker 42 tracking by the system 10. Additionally, because the markers 42 are recessed, the cover 70 can be placed over the markers 42. When used in conjunction with other components of the system 10, such as high pixel count cameras 34, 36, the markers 42 may be dimensioned to be about 3 mm in size. The markers 42 create a distinctive pattern around the body 68 of the array 38 such that the cameras 34, 36 can sense and track the location and orientation of the anatomical feature 4 or an instrument 6.

[0088] FIGS. 6A-6K illustrate embodiments of the array 38. In FIGS. 6A-6D, the body 68 of the array 38 is elongated and comprises a plurality of arms 84 extending from the body 68. The body 68 may comprise a spine pin engagement feature 88 for releasably securing the spine pin 52. The array 38 is removable and repositionable such that the array 38 may be positioned with the spine pin 52 or a surgical instrument 6 in either of a right handed or a left handed orientation. Beneficially, this feature allows the surgeon to reposition the array 38 away from the side where the surgical procedure is being performed such that the array 38 does not interfere, whether by view or physical obstruction, with the surgical procedure. This feature also aids in the ability for the cameras 34, 36 to see the markers 42 without, or with fewer, obstructions, as the surgeon can alternate the array 38 between left and right orientations as the surgeon progresses, for example, down the spine, allowing the cameras 34, 36 to view multiple vertebrae levels. If line of sight due to position of the arrays 38 becomes an issue during the surgery, vertebrae that are not being operated on can have their respective array 38 removed from the pin 52 to declutter the operative area and then releasably resecured when the surgeon has moved past that location. The benefits of decluttering the operative area are enormous, as the minimized footprint of the present system reduces hospital employee demand (saving money and hospital resources), and integrates with existing technologies on a single system. The relatively small size of the arrays 38 also reduces patient and surgeon impact, allowing the surgeon to work freely and minimize OR time, which is beneficial for the patient.

[0089] The arms 84 may variously extend laterally, longitudinally, and vertically from the body 68 such that each arm 84 terminates in a different position relative to the body 68. Each arm 84 may include a marker 42, such as a spherical marker 42 (FIGS. 6A-6B). In the array 38, depicted in FIGS. 6C-6D, the arms 84 terminate to paddles 86 such that reflective markers 42, such as adhesive reflective dots, can be disposed on the paddles 86.

[0090] As shown in FIGS. 6E-6K, the array 38 may be a 360 degree array 38. The 360 degree array 38 may include the body 68 that is frustoconical, as shown in FIGS. 6E-6H. The frustoconical body 68 may comprise one or more arms 84 that variously (vertically and around the body) extend from the body 68 to spherical markers 42. The array 38 may include a base 90. The base 90 may interface with a collar 92 by a retention mechanism 94, such as an O-ring seal, that secures and seals the interface between the base 90 and the collar 92. The collar 92 may comprise one or more legs 96, such as an oppositely disposed pair, that upwardly extend away from the collar 92 and are configured for engaging corresponding apertures (which can alternatively be recesses) 98 in the body 68. The body 68 may comprise a top aperture 100 that is open to an internal chamber 102, the internal chamber 102 opens to a collar aperture 104. The arms 84 may extend away from the body 68 farther than the collar 92 extends from the body 68 (FIGS. 6E-6F).

[0091] As shown in FIGS. 6G-6H, the 360 degree array 38 may be configured to receive, or be securely and releasably coupled with, a shield 106. The shield 106 may extend 360 degrees around the body 68, the arms 84, and the markers 42. The collar 92 may extend away from the body 68 farther than the arms 84 extend from the body 68 (FIGS. 6G-6H). The collar 92 may comprise a groove 108 extending around, or proximate to, its perimeter for releasably securing the shield 106. The collar 92 may include one or more guides 110 that extend from the collar aperture 104 to the groove 108 to further guide and secure the shield 106 within the groove 108. A top cover 112 may be releasably secured over the shield 106 opposite of the collar 92, the top cover 112 including a groove 114 (not shown) similar to groove 108 for receiving the shield 106.

[0092] As illustrated in FIGS. 6I-6K, the 360 degree array 38 may be a 360 degree prismatic array 38. The body 68 may be a polygon (e.g., a prism), such as a ten-sided pyramid having a profile of two opposing frustums. The exterior surfaces, or faces, 116 of the body 68 may each have one or more pockets 118 disposed therein. The pockets 118 may have a pocket-shaped profile (i.e., continuously curved on a first end leading to substantially straight sidewalls that lead to a flat second end opposite the continuously curved first end). Markers 42, such as adhesive reflected dots, may be placed within the pockets 118. Advantageously, the pocket-shaped profile of the pockets 118, particularly the curved first end, assists in guiding and securing the markers 42. The body 68 may be integrally formed with the collar 92, with the retention mechanism 94 configured to be secured with the collar 92. The body 68 may be integrally formed in one or more cooperative sections, as shown in FIG. 6J. As illustrated in FIG. 6K, which shows the body 68 rotated 90 degrees (1, 2, 3, and 4), the pockets 118 may be in varying orientations and positions on each face 116 of the body such that no two faces 116 have the same pocket 118 position and orientation. Advantageously, when used with markers 42, this causes the marker 42 placement to be unique when viewing each 90 degree rotation of the body 68 and the respective faces 116. For example, some pockets 118 may be toward a center 120 of the body 68 while other pockets 118 are toward distal ends 122 of the body, and pockets 118 may be rotated variously between 0 degrees and 360 degrees relative to one another. Advantageously, this configuration allows each pocket 118 may be positioned uniquely on the face 116 relative to all pockets 118 on all faces 116, which is especially beneficial when using the array 38 with markers 42, as each marker is in a different relative position and orientation, which allows cameras 34, 36 to see more of the markers 42, and sense more spatial data, when the system 10 is in-use.

[0093] Referring to FIGS. 7-8E, a fourth aspect of the system 10 includes the spine pin 52 and/or a spinal clamp 124. The spine pin 52 and the spinal clamp 124 allow for secure and releasable attachment of the array 38, connecting the array 38 with the anatomical feature 4, such as a vertebrae, or more specifically, a spinous process. As shown in FIG. 7, the spine pin 52 may have distal radiographic glyph elements 126 that extend outwardly from the pin 52 by glyph arms 128. The glyph arms 128 may extend from the radiographic glyph elements 78 to the distal radiographic glyph elements 126, such that the glyph arms 128 connect the radiographic glyph elements 78 and the distal radiographic glyph elements 126. Advantageously, this configuration of glyph elements 78, 126 allows for the pin 52 to contribute to the tracking capabilities of the system 10 by functioning as radiographic fiducials.

[0094] Proximate to the depth retention feature 76 and opposite of the tip 62, the pin 52 may include a depth limiter 129 (e.g., an expanded circumferential body). The depth limiter 129 functions as a physical stop when the pin 52 is being inserted into the anatomical feature 4, such as to prevent the pin 52 from being inserted too deeply into the feature 4 and risk causing injury to the patient as a result thereof.

[0095] The axial retention feature 76 may include a helical thread, as shown. Meanwhile, the portion of the pin 52 between the axial retention feature 76 and a tip end 83 of the pin 52 may include bone thread 130. The bone thread 130 and the axial retention feature 76 may be a differential pitch to increase bone retention when the pin 52 is engaged with the anatomical feature 4 (e.g., a spinous process). The pin 52 may include a tool engagement feature 131 for engaging and retaining a tool, such as the spine tracker inserter 54 (shown in FIG. 9A) during installation of the pin 52 (FIGS. 9B and 9C). FIGS. 8B-8C illustrate the pin 52 engaged with the array 38.

[0096] In some embodiments of the system 10, a spinous clamp 124 is provided. The spinous clamp 124 serves to secure the array 38 with the anatomic feature 4 by clamping onto the feature 4 and providing an interface for releasably securing the array 38. The spinous clamp 124 may include a central body 132 that ends from an array end 134 of the clamp 124 to a clamp end 136 of the clamp 124. The clamp 124 may be symmetrical, as shown in FIG. 8D, or asymmetrical, as shown in FIG. 8E. The clamp 124 may comprise opposing first and second grasps 138, 140 that are configured to secure the anatomical feature 4, such as a spinous process. The grasps 138, 140 may be resiliently biased towards a first, closed position (as shown), and configured to translate to a second, open position by pivoting one or more wings 142 toward the central body 132. In the embodiment shown in FIG. 8D, the pivoting of the wings 142 causes laterally disposed arms 144 to translate, transitioning the grasps 138, 140 from the first position to the second position. The arms 144 may be contoured, such as continuously tapering inward toward their center, such as to minimize obstructing the view of the surgical area to the surgeon. The grasps 138, 140 may each include a plurality of spikes 146 to increase the grip between the grasps 138, 140 and the anatomical feature 4. At the array end 134 of the clamp 124, the clamp 124 may include an array orientation feature 148, such as a flat surface on the central body 132, to ensure that the array 38 is correctly clocked when engaging the array 38 with the clamp 124. The clamp 124 may include a driver retention element 150, such as a notch, to ensure that the clamp 124 is retained in a tool, such as the spine tracker inserter 54, during installation of the clamp 124 on the anatomical feature 4. The central body 132 may include radiographic glyph elements 78 and distal radiographic glyph elements 126.

[0097] As shown in FIG. 8E, the clamp 124 may be asymmetrical. The asymmetrical clamp has grasps 138, 140 that are a first jaw 152 and a second jaw 154, respectively. The first jaw 152 may be mobile, and the second jaw 154 may be fixed relative to the central body 132. The mobile first jaw 152 may be resiliently biased in a first, closed position. The mobile first jaw 152 may be hingedly connected with a support 156. The support 156 is in turn connected to a biased member 158. The biased member 158 may be disposed around the central body 132 and between two biasing members 160, such as springs, that retain the biased member 158 in the first, closed position. When the biased member 158 is translated along the central body 132, the biased member 158 translates the support 156, which in turn causes the mobile first jaw 152 to pivot to the second, open position, converting the translation movement of the biased member 158 to pivotal movement of the first mobile jaw 152. The biasing members 160 may be disposed around the central body 132.

[0098] FIG. 10 illustrates spine pins 52 coupled to arrays 38 in accordance with an embodiment of the system 10. As can be seen, left handed and right handed arrays 38 can be moved away from the operative corridor, advantageously allowing the surgeon unrestricted, or less restricted, access, while still maintaining tracking capabilities of the arrays 38. The arrays 38 of the present disclosure are especially useful when coupled with instruments 6, as the pattern of the markers 42 on the arrays 38 allow the tracking system 10 to determine the position of the array 38, and thus the instrument 6, in 3D space as the instrument 6 is rotated. This is especially true for axially symmetric instruments 6 that are rotated during use, as the instrument 6 itself appears unchanged as it rotates along the axis 162.

[0099] While arrays 38 have been shown to be affixed to a mating feature through the connection portion 64 of the array 38 on the complementary surface 66 of the spine pin 52, it is contemplated that any of the arrays 38 may be adapted to be affixed to a mating feature on the distal end of the surgical instrument 6 in a similar fashion.

[0100] FIGS. 11A-11H illustrate the proximal end of an axially symmetric surgical instrument 6 in various rotational positions around an axis 162 according to a fifth aspect of the present disclosure. The instrument comprises a handle 164 at a handle end 166 of the instrument 6. A shaft 168 extends from the handle 164 to a cylindrical array 170. The cylindrical array 170 includes a plurality of recesses 172. The recesses 172 may be uniquely patterned around the cylindrical array 170, as shown, and may be configured to at least partially receive the markers 42. As the instrument rotates 6 (as shown in FIGS. 11A-11H), at least three recesses 172, and thus associated markers 42, remain visible to the cameras 34, 36 so that the system 10 can identify the spatial position of the array 38 and its associated instrument 6. According to some embodiments of the system 10, the recesses 172 are configured so that the 3D center of the visible marker area remains stable as the cylindrical array 170 is rotated.

[0101] Referring to FIG. 12A-12B, a sixth aspect of the system 10 includes a C-arm array mount 174. The C-arm array mount 174 may be configured to attach to a C-arm 194 receiver end), such as commercially C-arms sold by GE OEC (9800 and 9990 series), Philips (Pulsera), Siemens (Arcadis), and Ziehm (Vision) which may be connected via a video cable (such as BNC cable) to the system 10. The C-arm array mount 174 may be constructed of a black polycarbonate glyph mount 176 operably connected to, or mounted to, a ring 178. The ring 178 may be constructed of, for example, aluminum. The mount 174 may include clamps 180, such as ring clamps, to pull a back face (not shown) of the glyph mount 176 so that the glyph mount 176 references the receiver end. The clamps 180 may press on an external perimeter of the receiver end casting, allowing the ring 178 to be positioned radially and rotationally so that a top most screw on the receiver end is centered (e.g., aligned) with a countersunk alignment hole 182 of the C-arm mount 174. The mount 174 may include an assembly alignment pin 186 for guiding the assembly of the glyph mount 176 with the ring 178.

[0102] The array 38 may be included into the array mount 174 at the back (i.e., opposite of the glyph mount 176) of the ring 178. The array 38 may include a mounting element 184, such as a thumbscrew, to secure the array 38 to the mount 174. The array 38 and the mounting element 184 may be constructed of a material(s) that enable them to be autoclaveable for repeated use between patients. Also beneficially, the present design avoids the use of specialized tools for installing or adjusting the ring 178 or the array 38. Thus, using the mount 174, the C-arm array mount 174 provides continuous line of sight from the array 38 to the navigation system 10, which can track the C-arm 194 in the vertical and horizontal orientations for all operative imaging levels. While a planar optical tracking array 38 is shown in FIG. 12A, it is contemplated that the 360 degree arrays 38 and 360 degree prismatic arrays 38 may also be utilized. The system 10 may utilize data from the sensed C-arm mount 174, enabling the use of hospital existing C-arms 194 and a calibration tool to accurately register pre-operative CT data sets to patients positioned for surgery in the OR. With reference to FIGS. 26A-26D, registration plates (not shown) may be placed over the drape of the C-arm 194 so that the drape is engaged with the ring 178. A/P, +25, −25, and lateral images may be taken, received by the system 10, and the system 10 can execute the registration and calibration algorithms for the received images as will be described in greater detail below.

[0103] In a seventh aspect of the system 10, and referring to FIG. 40, the system 10 includes computer executable instructions containing instructions to reconstruct 3D spine models from received imaging inputs (such as 2D and 3D medical images from pre-op and intra-op sources such as CT, CBCT, and MRI,) that are segmented into rigid bodies (e.g., vertebrae) based on tracked anatomical features 4. The computer executable instructions are configured to dynamically track and display each anatomical feature 4 in real time. The computer executable instructions of the system 10 are configured to track and correlate the real time location of surgical tools relative to each tracked anatomical feature. The computer executable instructions are configured to calculate and display a dynamic 3D spine model with streaming data of virtual tools overlaid over the displayed dynamic 3D spine model to augment direct anatomy visualization.

[0104] As can be seen in FIG. 41, the imaging input device (e.g., pre-op and intra-op sources) is in communication with the computer system 12. The computer system 12 is in communication with the touch screen display 16, which can display data from the system 10 to the surgeon and receive input data from the system. The computer system 12 is in communication with the optical tracking system 32, including the IR cameras 34 and the visible light cameras 36. The computer system 12 may control the cameras 34, 36 (views, IR/visible light functionality, etc.), cause the cameras 34, 36 to capture and transmit images, and receive image data from the cameras 34, 36.

[0105] The computer executable instructions may be configured to execute a registration step. As used herein, the term “registration” is used to mean the correlation of one or more reference points of a virtual 3D data set with one or more reference points of the patient's current anatomical position. A registration step may be configured to occur for each receiving image source. In a preferred embodiment, the registration step includes parameters regarding the physical flexibility of the patient's spine. The registration step may vary according to the received image data. For example, if the received image data is from a pre-operative CT data set synced to one or more intraoperative 2D c-arm images, registration includes image recognition of one or more spine pins and one or more image markers within two fluoro images will register each vertebrae and reconstructing a 3D model from the pre-op 3D image. If the registration is of an intra-op CT/CBCT, image recognition of one or more spine pins within the intra-op 3D image will register each vertebrae. Meanwhile, if the image data is from an MRI-CT, imagine recognition of the CT image will merge MR images to each CT segment/vertebrae. In a fluoro-MRI registration, the instructions may be configured to register the 3D MRI image without any CT images.

[0106] The computer executable instructions of the system 10 may be configured to identify the spine pin 52. The computer executable instructions may be configured to sense the spine pin 52 and use the sensed spine pin 52 to scale the intraoperative radiographic image to register patient space, image space, and navigation space. The pins 52 may have glyph elements 78, 126 (e.g., spheres) that have a known predetermined diameter. When imaged by the cameras 34, 36, each glyph element 78, 126 has one or more characteristic identifiers in each imaging modality that can be leveraged by image recognition techniques (such as the known predetermined diameter).

[0107] By way of example, for a CT, the imaged glyph element may have a characteristic of a halo around it that can be used to correlate the center of the radiographic halo to the center of the glyph element 78, 126. Also by way of example, for a fluoroscopic image, the imaged glyph element 78, 126 may be displayed as a black dot. The location of that center position may be restricted to an accuracy of a first 3D pixel (voxel). The addition of a second glyph element 78, 126 allows sub-voxel accuracy based on the known distance between the two glyph elements 78, 126 and the double halo (or double dot). When these imaged glyph elements 78, 126 are placed along a single axis, a five degree of freedom pin is generated. To generate all six degrees of freedom, a second pin 52 having five degrees of freedom may be located in the image so that the orientation of the spine may be determined. If it is desirable to use only one spine pin 52 and resolve all 6 degrees of freedom, a 6 degree of freedom spine pin 52 can be used which has at least two glyphs 78, 126 on a first axis and an additional glyph 78, 126 (e.g., a distal glyph 126) orthogonal to the axis of the pin 52. FIG. 13 illustrates calculations used to identify one or more spine pins 52 in an image. As can be seen in FIG. 13, using the positions and orientations in 6 degrees of freedom of the arrays 38, the computer executable instructions were configured to calculate the two glyph centers of each marker 42. FIGS. 14A-14C illustrate a set of spine pins 52 positioned within the spinous processes of three adjacent vertebrae identified in accordance with the method described herein. In some embodiments, the computer executable instructions may include image recognition configured to determine the geometric shape of the entire spine pin 52.

[0108] After the system 10 determines the orientation of the pin 52, whether in 5 degrees of freedom or six degrees of freedom, the computer executable instructions may be configured to determine a relative position for the pin(s) 52. The optical tracking system 32, through cameras 34, 36 can sense the arrays 38 coupled with the pins 52. If the pin 52 is a six degree of freedom pin 52 (e.g., has the distal glyph 126 in addition to a glyph 78), then only one array 38 is needed by the system for the computer executable instructions to determine the position of the pin 52. If the pin 52 is a five degree of freedom pin 52 (e.g., does not include the distal glyph 126), then two pins 52 and arrays 38 coupled with the pins 52 are required for the computer executable instructions to determine the directive of the spine relative to the cameras 34, 36. The optical tracking system 32 may be configured to sense left/right arrays 38 that are coupled to the pins 52, and the computer executable instructions may be configured to calculate a relative position for each pin 52 based on the sensed arrays 38. The computer executable instructions may be configured to compare the determined relative positions of the radiographic orientation of the pins 52 to confirm that they align within an acceptable error tolerance. One this correlation is confirmed by the system 10, any relative movement of the spine detected from the tracking system 32 can be used to update the segmented vertebra locations in the radiographic data. Moreover, as the tracking arrays 38 may have six degrees of freedom, even if the pin 52 has 5 degrees of freedom, one registration has been confirmed, all of the arrays 38 can be removed except the one that the surgeon desires to remain so that the operative area is unobstructed. Depending on surgeon preference, multiple spine pins 52 and arrays 38 may be left in place (i.e., connected to the anatomical feature 4) to show an indication of relative movement of the anatomical features 4 (individual vertebral bodies). If a traditional 3D radiographic view, similar to a CT, is preferred, the navigation system 10 can detect where the top of an instrument 6 is located and, based on the determined position of the anatomical feature 4, the system 10 can determine the most proximate vertebral body that is closest to the top, or intersecting the tip as a reference to determine the orientation of the 3D data set relative to the instrument.

[0109] The computer executable instructions may be configured to perform a segmentation step. As used herein, “segmentation” describes a process that identifies individual vertebrae within 3D image data so that the vertebrae can be separated and treated, manipulated, and displayed distinct from one another. The segmentation step may employ a segmentation algorithm that uses imaging processing and image recognition software to automate the spinal level segmentation process. FIG. 15 depicts a segmentation process where the computer executable instructions automatically identify and extract the spine curve, then detects and identifies each individual vertebra, until they are segmented from one another. One or more adaptive meshes may be applied to generate a segmented 3D model of the spine. Each vertebra or other anatomical feature 4 can be separately colored to visibly enhance the bone-soft tissue interface, or just the margin can be colored.

[0110] The computer executable instructions of the system 10 may be configured to perform segmental tracking. As used herein, “segmental tracking” refers to the ability of the system 10 to track each vertebra that is of clinical significance. Advantageously, segmental tracking includes a single registration even during the procedure, increased accuracy, and allowing the surgeon to continue to navigate while the spine moves during the placement of, for example, interbody implants, without losing accuracy of the tracking.

[0111] The computer executable instructions of the system 10 may be configured to perform 3D-2D image registration. As used herein, “3D-2D image registrations” refers to the receiving a 2D image (fluoro), calibrating the 2D image, and registering the location of the patient's anatomy to the 3D image (pre-op CT) while the patient is positioned for surgery using at least two perspectives. Using the 2D image, the 3D image can then be registered with the navigation system 10. The 3D-2D image registration may include a step of receiving a pre-operative CT into the system 10. The computer executable instructions may be configured to segment the vertebrae into individual levels from the CT model using a segmentation algorithm. FIG. 16A illustrates a CT model of a spine, and FIGS. 16B and 16C illustrate the segmenting out of three spinal levels of the spine of FIG. 16A. The computer executable instructions can extrapolate and generate, by a digitally reconstructed radiograph (DRR) algorithm, a set of 2D images based on the 3D CT model.

[0112] FIG. 17A illustrates a segmented 3D image data set. FIG. 17B illustrates a 2D DRR (axial view) of the 3D image data set. FIG. 17C illustrates the 2D DRR (sagittal view) of the 3D image data set. FIG. 17D illustrates the 2D DRR (coronal view) of the 3D data set. When spine pins 52 are inserted into the vertebral levels of interest, which can be done under fluoroscopic visualization using the C-arm 194, the geometry of the glyphs elements 78, 126 of the spine pins 52 provide the system 10 with the scaling necessary to register the patient space, image space, and navigation space, as discussed above. A C-arm 194, which may be tracked by the navigation system by the C-arm array mount 174, captures at least one image of at least one of the vertebral levels of interest in at least one view. The computer executable instructions may be configured to receive the at least one image. Exemplary images captured by the C-arm 194 are depicted in FIGS. 18A and 18B. In some embodiments, the registration can be verified. By way of a first example, using automated image processing and automated image recognition, the computer executable instructions may search, via an imaging algorithm, each generated segmental DRR to determine a match of a 2D representation of the 3D image data set with the image of the 2D intraoperative fluoroscopic image. By way of example, the algorithm may determine the match by a cross correlation value (via a similarity score) that exceeds a predetermined value in two views to result in a single solution. By way of a second example and with reference to FIGS. 27A-27D, a successful registration may show the DRR (e.g., a simulated C-arm shot) superimposed over a real C-arm image. Each level may be identified in a four shot C-arm series window and may show a green border over any images that align with 99.5% Z-score (or other predesignated level). Unregistered levels may not have a DRR associated (partial levels) to omit unnecessary information. The GUI may display a confirm button for each image that the surgeon can select to verify the registration. In some embodiments of the system 10, at least two verifications (of two images) must be performed to progress to the navigation screen of the GUI. In FIGS. 27A-27D, FIG. 27C has failed the verification and FIGS. 27A, 27C, and 27D have passed the verification. Next, the computer executable instructions may be configured to adjust the model position of each vertebral body from the Pre-Op CT position to the current operative patient position.

[0113] The computer executable instructions may be configured to cause a 3D-2D image registration menu screen to be displayed on the display screen 16 (FIG. 25).

[0114] The computer executable instructions of the system 10 may be configured to perform 3D-3D image registration. As used herein, “3D-3D image registration” refers to the capturing and receiving a 3D intraoperative image and importing/registering the 3D intraoperative image with the navigation system 10. Spine pins 52 may be inserted into the vertebral levels of interest (such as under fluoroscopic visualization). As described above, the geometry of the glyphs elements 78, 126 of the spine pins 52 provide the computer executable instructions with the scaling parameters necessary to register the patient space, image space, and navigation space. Additionally, as the pins 52 are imaged in the intraoperative 3D image, the pins 52 (particularly the metal of the construction of the pins 52) serves as seed points to optimize the segmentation algorithm. Once segmentation is performed, the computer executable instructions cause the intraoperative 3D imager to capture image(s) of the vertebral levels of interest. The computer executable instructions are configured to receive the intraoperative 3D image from the intraoperative 3D imager, and the image of the spine can be segmented. In some embodiments, first the pins 52 are segmented from bone, and next, vertebrae are segmented into individual levels from the CT model using the segmentation algorithm. The computer executable instructions may be configured to cause a 3D-3D image registration menu screen to be displayed on the display screen 16 (FIG. 28).

[0115] The computer executable instructions of the system 10 may be configured to perform 3D MRI-2D image registration. As used herein, “3D MRI-2D image registration” refers to the capturing and receiving a 2D image (fluoro) and registering the location of the patient's anatomy (e.g., the anatomical features 4) with the patient is positioned for surgery using at least two different perspectives. The computer executable instructions may be configured to register, based on the 2D image, the location of the patient's anatomy. Advantageously, 3D MRI-2D image registration does not require a pre-op CT and can be used in facilities that lack 3D imaging equipment. Another benefit to MRI-2D image registration is the facilitation of soft tissue navigation, as the computer executable instructions may be configured to generate and display spatial awareness between soft tissues and instruments 6. For example, using the system 10 in embodiments employing 3D MRI-2D image registration, a surgeon can visualize and locate vessels and nerve roots for procedures such as XLIF to provide safe access to a surgical target site. The computer executable instructions may be configured to cause a 3D MRI-2D image registration menu screen to be displayed on the display screen 16.

[0116] The computer executable instructions of the system 10 may cause the display screen(s) 16 to display various workflow and visualization of the generated images utilizing the above described registrations. For example, some embodiments of the system may include computer executable instructions that cause “wizards” (step-by-step guided instructions) to display to surgeons to walk the surgeons through navigation setup and usage. FIG. 19 illustrates an example start-up menu screen GUI on the display screen 16. From the selection area, the surgeon may input case-specific information. For example, the surgeon may select the anatomical region that is the subject of the surgical procedure, the levels of the surgical procedure (e.g., the level selection menu screen shown in FIG. 20), and the surgical approach (e.g., posterior, anterior, lateral). In FIG. 19, the surgeon selected a lumbar procedure, levels L3-L5, from a lateral approach. Selecting a “start” button moves the surgeon to the next step in the workflow. As can be seen, the surgeon can also save procedures, import saved procedures, and enter “demonstration” mode for, for example, training or surgical planning purposes. The surgeon can also access settings and saved procedures through a secure login access to the cloud. It will be appreciated that any of the computer executable instructions contained herein may be in the cloud or locally stored in the navigation system 10 (such as by computer system 12).

[0117] The computer executable instructions may retrieve stored templates and surgeon profiles, such as templates for preferred views (e.g., axial, probe's eye, frontal, coronal, sagittal, inline, etc.) to streamline initial setup of the system 10 with a patient. The computer executable instructions may cause the display screen 16 to display a menu of the templates and surgeon profiles to the surgeon and receive template and surgeon profile selections from the surgeon, as shown in FIG. 21. FIG. 22 illustrates a setup screen menu where stored templates for preferred views are displayed on the display screen 16. The computer executable instructions may be configured to receive the selected templates selections from the surgeon.

[0118] The computer executable instructions may be configured to enable the surgeon to load the pre-operative image data set and view, on the display screen 16, 2D and/or 3D image data for the purposes of planning the surgical procedure. FIG. 23 illustrates an example of a surgical planning screen displayed on the display screen 16. As can be seen in FIG. 23, the computer executable instructions may be configured to cause the display screen 16 to display a surgical planning menu where the surgeon (e.g., the surgeon or other OR personnel) can select the levels, screw types, and screw sizes for the surgical procedure.

[0119] As can be seen in FIG. 24, the computer executable instructions may be configured to cause the display screen 16 to display a surgical planning screen where a surgeon can access a axial, sagittal, and coronal views of the pre-operative image data set, determine one or more anatomical measurements (e.g., pedicle width, vertebral body width or vertebral body depth), and propose a surgical trajectory. The surgical plan may be saved for each individual vertebral body plan so the system 10 may present a 3D or C-arm view(s) of the individual vertebra body plan during the appropriate time in the surgical procedure. With reference to FIGS. 36A and 36B, according to some embodiments of the system 10, screws may be placed in each pedicle according to best fit (either manually via surgeon selected or automatically by the system 10). The surgical plan may be saved for each individual vertebral body plan so the system may present a 3D or C-arm view(s) of the individual vertebral body plan during the appropriate time in the surgical procedure. As can be appreciated, other surgical planning types may be incorporated into the computer executable instructions, such as NuvaMap surgical planning software by NuVasive, Inc.

[0120] The computer executable instructions may be configured to cause the display screen 16 to display an instrument setup menu (FIG. 29) and an array assignment menu (FIG. 30). The computer executable instructions may cause the system 10 to capture one or more images via the optical tracking system 32 and automatically recognize the array(s) 38. The system 10 may also utilize adapters for instruments 6, which allow the system 10 to recognize the instrument, identifying a first level describer (e.g., screwdriver for a screwdriver). The computer executable instructions may cause the display screen 16 to display a menu, such as a drop-down menu, so that the surgeon may select a final description for the instrument 6. Advantageously, this feature allows the surgeon to continue the operative workflow and seamlessly select navigated instruments 6.

[0121] The computer executable instructions may be configured, particularly after registration and setup steps are completed, to cause representations of one or more surgical instruments to be overlaid on the 2D representation of the 3D image data set in real time. By way of example, the representation of the instrument can be an engineering model (such as computer-aided design model), a wire frame model, or a simple shape (such as a cylinder, rectangular, or block). The computer executable instructions may be configured to further generate a simulated trajectory 188 (such as of a pedicle screw) based on the location of the surgical instrument with respect to the anatomy and display the simulated trajectory 188 on the 2D representation on the display screen 16, as shown in FIGS. 31-33. As can be seen in FIG. 32, the computer executable instructions may be configured to cause a tool menu to display, in which various options such as system type, implant type, and screw type of the simulated trajectory 188 can be selected by the surgeon. The computer executable instructions can be configured to display a simulated implant 190 (e.g., pedicle screw) based on the simulated trajectory and surgeon inputs regarding the tool menu. Once the surgeon is satisfied with the trajectory and implant selection, the system 10 can be used to navigate the real-world implant 191 (not shown) to its intended location, preferably following the simulated trajectory 188 as depicted in FIG. 33. While a positioning of a pedicle screw is described here, it will be appreciated that the navigation system 10 can provide navigation information for any number of instruments and implants.

[0122] The system 10 may comprise computer executable instructions configured to generate a simulated bookmarked screw 192 and cause the simulated bookmarked screw 192 to be displayed to the surgeon on the display screen 16. This feature is beneficial when a surgeon desires to insert two screws into a single vertebra aligned from a particular perspective (e.g., lateral). Shown in FIGS. 35A and 35B, using the generated segmental DRR, as discussed above, the computer executable instructions may be configured to generate a simulated bookmarked screw to capture the final location of a first placed screw in a vertebrae. When the surgeon is ready to place the second screw in the vertebrae, the simulated bookmarked screw is displayed over the DRR image such that the surgeon has a reference for alignment for placing the second screw. The simulated screw may be depicted by a solid, opaque representation of a screw, an outline of a screw with a transparent body, or a line. The computer executable instructions may be configured to generate the simulated screws in views other than DRR, such as a 3D sagittal sliced view. FIG. 35A illustrates a simulated bookmarked screw in DRR of spine for reference, and FIG. 35B illustrates a screw being placed over the simulated bookmarked screw.

[0123] In an eighth aspect, the system 10 generates a 3D post-op simulated CT. According to this aspect, the last locations of the dynamically tracked vertebrae and the final position of each placed screw may be processed by the system 10, via computer executable instructions, to generate a post-op 3D view of the patient. Beneficially, this 3D view is an accurate depiction of the patient immediately following surgery, such that an additional 3D scan at the end of the surgery is not required, resulting in less radiation exposure to the patient and OR staff, and a savings of time (i.e., lower risk of morbidity and cost savings).

[0124] The segmentation steps described above are particularly advantageous in that they provide for additional viewing options for the surgeon. Referring to FIGS. 37-39, the GUI may display different view of the surgical site based on surgeon preference. Referring to FIG. 37, the GUI may display different views of the surgical site based on surgeon preference. In some embodiments, only registered vertebral bodies may display and may be shown in current orientation (not pre-op CT orientation) based on registration. The adjacent vertebrae levels may be displayed in their new locations as solid engineering files, such as 3D.stl or other format files, and all may be tracked relative to the single patient reference array 38 (above operative level). In some embodiments, a DICOM image may be navigated for only a vertebral body that is currently being worked on by the surgeon. The Axial DRR or tool view of the CT slice and Lateral DRR (either left or right half) or Sagittal tool view CT is shown in FIG. 37. FIG. 38 illustrates a displayed planned trajectories according to planned screws from pre-surgery, as discussed above, which may be toggled on and off by the surgeon. The bookmarked screw feature may also be toggled on and off by the surgeon, shown in FIG. 39. The GUI may include a “Hold New Trajectory” selection for providing the surgeon the option to adjust a new trajectory of the planned screw based on real world surgical conditions and the surgeon's professional medical judgment.

[0125] In an ninth aspect of the system 10, the system 10 receives images captured from more than one camera 34, 36 such that a first camera 34, 36 has a field of view of an operative area and a second camera 34, 36 has a field of view of a registration apparatus (e.g., a C-arm 194). The system 10 may include computer executable instructions configured to calibrate the relative position of the first camera 34, 36 and the second camera 34, 36 by referencing an array 38. The cameras 34, 36 may be provided together, for example in the camera base 37, or separately. The system 10 may perform dynamic calibration when the cameras 34, 36 are tracking in the same direction or tracking from opposite directions.

[0126] For example, the system 10 may include a first infrared camera 34 within the sterile operative field to capture images of and track arrays 38 in communication with the spine, as discussed above. The system 10 may include a second infrared camera 34 positioned outside of the sterile operative field which captures images of and track a registration device. The second infrared camera 34 may capture images of and track at least one of the arrays 38 tracked by the first infrared camera 34, but may not track all of the arrays 38 tracked by the first infrared camera 34 due to field of view obstructions. However, as discussed above, the system 10 may receive these data and generate 3D models and display 2D representations of the 3D models on the display screen 16.

[0127] In another example, the system 10 utilizes more than one camera 34, 36 may be used to capture and track more than one array 38 that is connected to the anatomical feature 4 that is a single rigid segment, such as the S 1/pelvis complex, that may be treated as a single structure.

[0128] In yet another illustrative use, during a lateral spine procedure, for example, an XLIF procedure, a first infrared camera 34 may capture and track spine tracker arrays 38 while a second infrared camera 34 captures and tracks the operative corridor and an array 38 affixed to the iliac crest. Although neither the first nor the second cameras 34 may track the same array 38, the system 10 may use the registered and segmented a single, rigid structure allowing the system to calculate an indirect correlation between cameras 34 during registration so that, while neither camera 34 needs to track a common array 38, the two cameras 34 transmit images and data that the system 10 can receive to generate a unified 3D view of the surgical area.

[0129] In a tenth aspect of the system 10, the system 10 is configured to generate an augmented reality. The system 10 may include at least three cameras 34, 36, where the first and second infrared cameras 34 may be used for tracking the marker arrays 42, which may be infrared markers 42, while the third visible light spectrum camera 36 is focused on the operative area. This at least three cameras 34, 36 configuration allows computer executable instructions of the system 10 to generate an accurate portrayal of the position of the virtual tool location and tracking markers 42 by superimposing images captured by the infrared cameras 34 and the visible light spectrum camera 36. This overlapping of the virtual representation and the visible element can be used for segmented vertebral bodies, too, when the vertebral bodies have arrays 38 connected to them. The surgeon can then view the vertebral bodies and the surgical instruments from either of the infrared cameras 34 with overlap from the visible camera 36 such that system 10 can, for example, visually guide the surgeon through a predetermined surgical plan entered into the system 10 by the surgeon. The system 10 may include a wearable camera and array assembly that can be worn by the surgeon such that the system 10 can display, for example through augmented reality glasses, a simulated vertebrae and instrument over the patient.

[0130] In an embodiment of the system 10, the system 10 is configured to use the augmented reality feature for trouble shooting problems with one or more arrays 38. For example, if an array 38 is blocked or contaminated, the system 10 may be configured to display highlighted pixels in the visible light spectrum to display the position of the blocked or contaminated array 38. The system 10 may overlay on the highlighted pixels a symbol, such as a colored ball, of the tracked infrared position of the markers 42 of the array 38. This troubleshooting feature enables a surgeon to understand whether a line of sight issue exists with the array 38 or whether one of the markers 42 is defective.

[0131] Especially advantageous is that the presently described navigation system 10 enables calibration of the navigation system 10 without interrupting surgical work plan flow. For example, the system 10 can display enlarged pixels where the system 10 has determined the position of an instrument so that the surgeon can confirm that the OR conditions are calibrated with, and accurately displayed by, the simulation. If calibration is necessary, then the system 10 can use image recognition, such as by capturing and receiving images from the visible light cameras 36, to recalibrate itself. The calibrating may be motion activated, activated by, and calibrated by, the surgeon rotating the instrument 6 in view of the visible light camera 36.

[0132] An instrument 6 may have many attachments, such as a screw driver. The system 10 may contain computer executable instructions configured to capture and track the instrument 6 and associated screw via the cameras 34, 36, and calculate the diameter and length of the screw. The computer executable instructions may be configured to display a virtual representation of the appropriate screw onto the display screen 16. The virtual representation of the screw may have a color code associated with screw size, such as screw diameter and screw length for easy visual identification by the surgeon.

[0133] In an eleventh aspect, the system 10 may perform redundant spine tracking. As discussed above, the system 10 may include more than one vertebral tracking array 38. The system 10 may track the movement of each of the arrays 38, and if the relative movement of the arrays 38 exceeds a predetermined value, the surgeon can select, through the GUI, to track a secondary array 38 as a reference or reregister the vertebral body.

[0134] In a twelfth aspect, the system 10 tracks individual segmented vertebral bodies and displays individual movement of the vertebral bodies. Referring to FIG. 34, each vertebra can be tracked by the system 10 through the spine pins 52 and arrays 38, as discussed above, and segmented from the rest of the vertebrae. The system 10 may include computer executable instructions that are configured to, based on the tracked and segmented vertebrae, generate and display simulated vertebrae that the surgeon can view and select in any 2D or 3D perspective moving relative to the adjacent vertebra. The simulated individual segments may contain respective radiographic data that can be viewable when system 10 receives instructions from the surgeon to slice open the simulated segments. The outer shell of the simulated segment can be given a specific color to assist the surgeon in differentiating one vertebra from adjacent vertebra also being tracked and simulated. The computer executable instructions may be configured to display and hide a background (i.e., non-vertebral body tissues). The background may be displayed as a differential color from the simulated vertebrae, such as black or another contrasting color to assist the surgeon by lighting the vertebra in which the surgeon is interested. When the background is hidden from the 3D view, a deformable transform matrix can be applied to display a simulation of how the soft tissues have stretched. If CT and MRI images have been merged, then this deformable transformation matrix can be used to highlight the soft tissues, including using further colors to highlight different tissue types. In some embodiments of the system 10, the computer executable instructions are configured to receive input from a surgeon to highlight a border of a particular vertebral body. This feature is particularly advantageous for operating on osteoporotic patients, and the surgeon would benefit from this aid in discerning the border of the vertebral body.

[0135] In a thirteenth aspect, the system 10 may be used in systems and methods for surgical planning and assessment of spinal deformity correction, as described in International Application Number PCT/US17/20491, which is incorporated by reference herein in its entirety. Advantageously, the navigation system 10 can be combined with the surgical planning and assessment such that computer executable instructions of the system 10 are configured to capture and track, at the start of the surgical procedure, the positions and orientations of the vertebrae. The computer executable instructions may be configured to receive images from the cameras 34, 36 and update spinopelvic parameters, or calculate the change in foraminal or canal volume to assess proper decompression, automatically to determine compliance with an optimized surgical plan. Advantageously, this automatic tracking and updating eliminates the need to interrupt surgical workflow to manually capture additional C-arm images.

[0136] In a fourteenth aspect, the system 10 may be utilized with rod bending software and machines, such as the BENDINI spinal rod bending system by NuVasive, Inc., described in U.S. Pat. Nos. 7,957,831 and 8,549,888, which are incorporated by reference herein in their entireties. As spinal rod bending systems use digitalized locations of screws to generate bend instructions, the present system 10 streamlines the process through the reference (i.e., the arrays 38) affixed to more than one vertebrae, the system 10, via the optical tracking system 32, can capture the location, the position, and orientation of the screw when the screw was inserted into the spine. Advantageously, this automates the digitization step in the workflow for the use of spinal rod bending systems and allows the virtual rod preview to be dynamically updated as the surgery progresses, as the vertebrae, implant, and instruments can continue to be automatically and continually tracked over the surgical procedure. Advantageously, once the rod is created using the spinal rod bending system, the system 10 can be configured to generate and simulate directional information to the surgeon to facilitate navigation of the rod insertion.

[0137] In a fifteenth aspect, the system 10 provides for integrated neuromonitoring when used in conjunction with intraoperative neuromonitoring, such as the NVM5 neuromonitoring platform by NuVasive, Inc. Neuromonitoring platforms are set forth in, for example, U.S. Pat. Nos. 8,538,539; 8,548,579; 8,550,994; 8,556,808; 8,562,521; 8,591,432; 8,602,982; 8,628,469; 8,634,904; 8,663,100; 8,672,840; 8,679,006; 8,696,559; 8,708,899; 8,738,123; 8,747,307; 8,753,271; 8,764,649; 8,768,450; 8,784,330; 8,821,396; 8,942,801; 8,945,004; 8,956,283; 8,977,352; 8,989,866; and 9,037,250, which are hereby incorporated by reference in their entireties. The navigation features of the system 10 described herein may be integrated with EMG and free run activity, such that location of where EMG results occurred may be tracked and overlaid onto the 2D representations/views of the spine generated by the navigation system 10. By way of example only, implanted devices such as pedicle screws can be colored red, yellow, or green based on their proximity to the bone-soft tissue interface, such that close or breached may display the implanted devices as red, near may display the implanted devices as yellow, and an acceptable margin of distance may display the implanted devices as green.

[0138] The foregoing description illustrates and describes the processes, machines, manufactures, and other teachings of the present disclosure. Additionally, the disclosure shows and describes only certain embodiments of the processes, machines, manufactures, and other teachings disclosed, but as mentioned above, it is to be understood that the teachings of the present disclosure are capable of use in various other combinations, modifications, and environments and are capable of changes or modifications within the scope of the teachings as expressed herein, commensurate with the skill and/or knowledge of a person having ordinary skill in the relevant art. The embodiments described hereinabove are further intended to explain certain best modes known of practicing the processes, machines, manufactures, compositions of matter, and other teachings of the present disclosure and to enable others skilled in the art to utilize the teachings of the present disclosure in such, or other, embodiments and with the various modifications required by the particular applications or uses. Accordingly, the processes, machines, manufactures, and other teachings of the present disclosure are not intended to limit the exact embodiments and examples disclosed herein. Any section headings herein are provided only for consistency with the suggestions of 37 C.F.R. § 1.77 or otherwise to provide organizational queues. These headings shall not limit or characterize the invention(s) set forth herein.

[0139] While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and are herein described in detail. It should be understood, however, that the description herein of specific embodiments is not intended to limit the invention to the particular forms disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined herein. For example, any of the features of a particular example described herein may be used with any other example described herein without departing from the scope of the present invention. According to some embodiments, some or all of the surgical navigation systems, methods, and workflows described below can be used in conjunction with some or all of the systems, methods, and workflows described above.