Disclosed are systems, devices, and methods for performing treatment along a lumen network, an exemplary method comprising receiving image data of a patient's lungs, mapping one or more luminal networks inside the patient's lungs based on the received image data, identifying a treatment target in the image data, determining a luminal pathway to the treatment target via at least one of the luminal networks, configuring treatment parameters for treatment of the treatment target and at least one of the luminal networks, navigating a tool inside at least one of the luminal networks to the treatment target, treating the treatment target with a primary treatment modality, and treating the luminal pathway of at least one of the luminal networks leading to or from the treatment target with a secondary treatment modality.

A navigated surgery system includes at least one processor that is operative to obtain a 2D medical image slice of anatomical structure of a patient. The operations further obtain a 3D graphical model of anatomical structure. The operations determine a pose of a virtual cross-sectional plane extending through the 3D graphical model of the anatomical structure that corresponds to the anatomical structure of the 2D medical image slice. The operations control the XR headset to display the 2D medical image slice of the anatomical structure of the patient, display the 3D graphical model of the anatomical structure, and display a graphical object oriented with the pose relative to the 3D graphical model of the anatomical structure.

A surgical navigation system includes: a tracker (125) for real-time tracking of a position and orientation of a robot arm (191); a source of a patient anatomical data (163) and a robot arm virtual image (166); a surgical navigation image generator (131) generating a surgical navigation image (142A) including the patient anatomy (163) and the robot arm virtual image (166) in accordance to the current position and/or orientation data provided by the tracker (125); a 3D display system (140) showing the surgical navigation image (142A).

Provided is a control device including a control unit that calculates a first positional relationship between an eye of an observer observing an object displayed on a display unit and a first point in a master-side three-dimensional coordinate system, and controls an imaging unit that images the object so that a second positional relationship between the imaging unit and a second point corresponding to the first point in a slave-side three-dimensional coordinate system corresponds to the first positional relationship.

Systems and methods for surgical imaging are disclosed herein. A head-mountable device (HMD) can include a display configured to provide an image within a field of view of an environment of the HIVID. At least one fiducial marker can be arranged on a surgical patient. At least one sensor can be configured to track a position of the at least one fiducial marker. Three-dimensional image information indicative of one or more internal features of the patient is provided. Based on information from the at least one sensor, a position of the surgical patient can be determined. Based on the determined position of the surgical patient, the HIVID can display at least a portion of the three-dimensional image information superimposed on at least a portion of the surgical patient within the field of view.

An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.

In accordance with one aspect of the present disclosure, provided is a surgical heads-up display system comprising a heads-up display and an articulable arm coupled to the heads-up display. The heads-up display is configured to receive a portable electronic device and to redirect a display output of the portable electronic device. The articulable arm is configured to enable selective positioning and maintenance of the heads-up display in a selected position such that the display output of the portable electronic device is redirected in a chosen direction.

The invention relates to a method for displaying the position and orientation of a linear instrument (1) navigated with respect to a 3D medical image (V), wherein: said linear instrument (1) is coupled to a guide (2); the guide (2) is tracked by a navigation system with respect to the 3D image (V); a plane (4) containing the axis of the linear instrument (1) is virtually attached to the guide (2); a slice reformatted in the 3D image and containing said plane (4) is displayed.

Another object of the invention is a system for carrying out said method.

Devices and methods for performing a surgical step or surgical procedure with visual guidance using an optical head mounted display are disclosed.

A mediated-reality system for surgical applications incorporates pre-operative images and real-time captured images of a surgical site into a visualization presented on a head-mounted display worn by a surgeon during a surgical procedure. The mediated-reality system tracks the surgeon's head position and generates real-time images of the surgical site from a virtual camera perspective corresponding to the surgeon's head position to mimic the natural viewpoint of the surgeon. The mediated-reality system furthermore aligns the pre-operative images with the real-time images from the virtual camera perspective and presents a mediated-reality visualization of the surgical site with the aligned pre-operative three-dimensional images or a selected portion thereof overlaid on the real-time images representing the virtual camera perspective.