The present disclosure relates to systems and methods for robotic, computer-aided or virtual/augmented reality assisted procedures, including with use a of patient-specific or patient-matched, customized apparatus for assisting in various surgical procedures. In varying embodiments, patient-specific guides may comprise embedded markers, chips, circuits, or other registerable components for providing information to a robotic or computer-aided device. Other apparatus described herein may be aligned and/or matched with the robotic or augmented reality equipment or another apparatus during a surgical procedure.

In certain embodiments, the systems, apparatus, and methods disclosed herein relate to robotic surgical systems with built-in navigation capability for patient position tracking and surgical instrument guidance during a surgical procedure, without the need for a separate navigation system. Robotic based navigation of surgical instruments during surgical procedures allows for easy registration and operative volume identification and tracking. The systems, apparatus, and methods herein allow re-registration, model updates, and operative volumes to be performed intra-operatively with minimal disruption to the surgical workflow. In certain embodiments, navigational assistance can be provided to a surgeon by displaying a surgical instrument’s position relative to a patient’s anatomy. Additionally, by revising pre-operatively defined data such as operative volumes, patient-robot orientation relationships, and anatomical models of the patient, a higher degree of precision and lower risk of complications and serious medical error can be achieved.

A system is disclosed that allows for determining a position of an instrument in an object space. The position may include a three-dimensional location and at least one degree of freedom of orientation, or any appropriate number of degrees of freedom. The tracked position may be based on imaging the instrument, including imaging an external contour of the instrument. The movement of the instrument can be determined based on a three-dimensional determination of a movement of the instrument contour

The combination of a marker system with binocular AR glasses to create a soft tissue procedure system used for surgery, biopsies, etc. A marker localization system is functionally integrated with augmented reality (AR) glasses. This soft tissue system can be applicable for AR surgeries in soft tissues involving mobile and static anatomies anywhere in the body, such as for example, breast, soft tissue, lungs, lymph nodes, liver surgeries, etc. By the placement of single or multiple markers and localizing the markers with a locator, such as a hand-held locator as a non-limiting example, the above-mentioned system provides a real-time intraoperative coordinate frame for the virtual projection of the markers (and the associated ROI) on/in the surgical field/biopsy field of view, using commercial off-the-shelf computer hardware (laptops, tablets, etc.) for the required image processing.

Methods and systems for providing treatment planning information for a neurology procedure, including neurosurgical procedures. A database containing historical data about historical procedures is accessed. Historical data relevant to a neurology procedure is determined, based on a determination of similarity to a set of data characterizing the neurology procedure. Historical instances of procedure parameters relevant to the neurology procedure are determined and displayed.

A fiducial marker includes a fastener and a feedback component to provide a registration signal when engaged by a probe. The feedback component includes light-up cap, a conducting component, a magnetic component and an RFID tag. A cap for registering fiducial markers with robotic surgical systems includes a housing, a socket in the housing for coupling to a fastener, an access port in the housing, a switch disposed in the housing proximate the access port, and a sensory indicator device coupled to the switch, wherein the sensory indicator device produces a signal when activated through the access port to confirm marker contact. Methods of registering a fiducial marker fastener, such as with robotic surgical systems, include manipulating a probe to align with a signal-producing feedback component attached to or integrated with the fastener, and engaging the feedback component with the probe to activate a sensory feedback indicator.

The present disclosure relates to systems and methods to perform a surgical procedure. The systems and methods utilize MRI-compatible fiducial markers including a body having at least one feature configured to receive an MRI-compatible and MRI-visible material and to allow registration of a navigational tool. The MRI-compatible fiducial markers can be affixed to a bone of a patient. The registered navigational tool can be used to advance a surgical tool along a navigational path to perform a surgical procedure.

For three-dimensional segmentation from two-dimensional intracardiac echocardiography imaging, the three-dimension segmentation is output by a machine-learnt multi-task generator. Rather than the brute force approach of training the generator from 2D ICE images to output a 2D segmentation, the generator is trained from 3D information, such as a sparse ICE volume assembled from the 2D ICE images. Where sufficient ground truth data is not available, computed tomography or magnetic resonance data may be used as the ground truth for the sample sparse ICE volumes. The generator is trained to output both the 3D segmentation and a complete volume (i.e., more voxels represented than in the sparse ICE volume). The 3D segmentation may be further used to project to 2D as an input with an ICE image to another network trained to output a 2D segmentation for the ICE image. Display of the 3D segmentation and/or 2D segmentation may guide ablation of tissue in the patient.

Embodiments of the invention provide systems and methods for providing augmented reality to a surgeon with the steps of acquiring at least one preoperative medical scan image of a region of interest in which the surgery is to be performed and introducing a fiducial device non-invasively or minimal invasively into a body lumen present in the region of interest of a patient. The fiducial device will be in particular introduced into a lumen, which can be clearly identified in the medical scan image. The fiducial device is adapted to emit infrared light either by using infrared light source or by using fluorescent material placed in or in the fiducial device ad to be excited by illumination light or exciting light. After acquiring at least one imaging light live image and one infrared live image the tissue structure of the body lumen with the fiducial device placed therein is clearly detectable within the live image and can reliably registered and matched to the scan image for presenting the overlaid images and to the surgeon.

In general, devices, systems, and methods for fiducial identification and tracking are provided.