Systems and methods are disclosed for use in electronic guidance systems for surgical navigation. A sensor is provided with an optical sensor, to provide optical information, and a measuring sensor, to provide measurements for determining a direction of gravity. The sensor communicates optical information and measurements to an inter-operative computing unit. In an embodiment, the inter-operative computing unit receives first optical information for a registration device and a patient anatomy and a measurement to determine a direction of gravity to perform a registration step. The inter-operative computing unit receives second optical information for the patient anatomy and an object and determines and presents measurements relative to the anatomy. The measurements relative to the anatomy are determined from the second optical information, and in relation to the registration of the anatomy of the patient.

A method for enhanced surgical navigation, and a system performing the method and displaying graphical user interfaces associated with the method. A 3D spatial map of a surgical site is generated using a 3D endoscope including a camera source and an IR scan source. The method includes detecting a needle tip protruding from an anatomy and determining a needle protrusion distance corresponding to a distance between the needle tip and a surface of the anatomy using the 3D spatial map. A position of a surgical tool in the 3D spatial map is detected and a determination is made by the system indicative of whether the needle protrusion distance is sufficient for grasping by the surgical tool. A warning is generated when it is determined that the needle protrusion distance is not sufficient for grasping by the surgical tool.

A method for creating a patient-specific surgical plan includes receiving one or more pre-operative images of a patient having one or more infirmities affecting one or more anatomical joints. three-dimensional anatomical model of the one or more anatomical joints is created based on the one or more pre-operative images. One or more transfer functions and the three-dimensional anatomical model are used to identify a patient-specific implantation geometry that corrects the one or more infirmities. The transfer functions model performance of the one or more anatomical joints as a function of anatomical geometry and anatomical implantation features. surgical plan comprising the patient-specific implantation geometry may then be displayed.

A computer assisted system is disclosed that includes an optical tracking system and one or more computing devices. The optical tracking system includes an RGB sensor and is configured to capture color images of an environment in the visible light spectrum and tracking images of fiducials in the environment in a near-infrared spectrum. The computer assisted system is configured to generate a color image of the environment using the color images, identify fiducial locations using the tracking images, generate depth maps from the color images, reconstruct three-dimensional surfaces of structures based on the depth maps, and output a display comprising the reconstructed three-dimensional surface and one or more surgical objects that are associated with the tracked fiducials. The computer assisted system can further include a monitor or a head-mounted display (HMD) configured to present augmented reality (AR) images during a procedure.

A system and method are provided for aligning a tool with a targeted axis in tissue to perform a medical procedure. medical plan is registered to the location of the tissue using a computer-assisted medical system, where the medical plan include a planned position for the targeted axis based on pre-procedure data. The tool is aligned with the planned position for the targeted axis using a computer- assisted medical system. The computer-assisted medical system includes a hand- held device having a handle and a working portion adjustable relative to the handle so as to orient the tool. A computing system is also provided comprising a tracking system and a control system for registering the medical plan to the location of the tissue, tracking the hand-held device relative to the tissue and the medical plan, and adjusting the working portion of the hand-held device relative to its handle.

Registration of a surgical image acquisition device (e.g. an endoscope) using preoperative and live contour signatures of an anatomical object is described. A control unit includes a processor configured to compare the real-time contour signature to the database of preoperative contour signatures of the anatomical object to generate a group of potential contour signature matches for selection of a final contour match. Registration of an image acquisition device to the surgical site is realized based upon an orientation corresponding to the selected final contour signature match.

Cutting tools, systems and methods for navigated procedures are provided. A cutting tool (e.g. oscillating blade, etc.) for a power tool has an optically trackable feature in a defined positional relationship relative to a cutting feature of the cutting tool. The trackable feature may include reflective material applied to a surface (e.g. a recessed blade surface). The trackable feature is be imaged by a camera integral with or attached to the power tool and provided to a computing unit of a navigation system to determine a relative pose of the cutting feature and camera. The camera may also track a patient's bone such that the computing unit may determine a relative position of the bone and camera. The unit then computes a relative pose of the cutting feature with respect to the patient's bone and provides same for determining display information and/or to a robotic controller for procedural control.

An image processing apparatus includes a processor including hardware, the processor being configured to: estimate, based on image information from a medical device that includes at least an endoscope, a plurality of procedure actions of an operator of the endoscope; perform different supports respectively for procedures by the operator, according to an estimation result of the procedure actions; and output a display for the supports to a display.

A surgical robotic system includes: a surgical console having a display and a user input device configured to generate a user input and a surgical robotic arm having a surgical instrument configured to treat tissue and being actuatable in response to the user input; and a video camera configured to capture video data that is displayed on the display. The system also includes a control tower coupled to the surgical console and the surgical robotic arm. The control tower is configured to: process the user input to control the surgical instrument and to record the user input as input data; train a machine learning system using the input data and the video data; and execute the at least one machine learning system to determine probability of failure of the surgical instrument.

A technique for determining a need for a re-registration of an optical patient tracker with medical image data of a patient is presented. A camera system is configured to generate camera image data for tracking the tracker. The camera system comprises an acceleration sensor configured to generate inertial data indicative of an acceleration of the camera system. A method implementation of the technique comprises the following steps performed by a processor: receiving image data from the camera system and analyzing the received image data for a positional change of the tracker indicative of at least one of a drift of the tracker and an impact on the tracker; receiving inertial data acquired by the acceleration sensor and analyzing the received inertial data, or data derived therefrom, with respect to at least one first predetermined condition indicative of an impact on the camera system; and generating, in case a positional change of the tracker indicative of at least one of a drift of the tracker and an impact on the tracker is identified based on the image data and the at least one first predetermined condition is not fulfilled, at least a re-registration signal.