An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.

Devices and methods for performing a surgical step or surgical procedure with visual guidance using an optical head mounted display are disclosed.

The present disclosure relates to a surgical navigation system for the alignment of a surgical instrument and methods for its use, wherein the surgical navigation system may comprise a head-mounted display comprising a lens. The surgical navigation system may further comprise tracking unit, herein the tracking unit may be configured to track a patient tracker and/or a surgical instrument. Patient data may be registered to the patient tracker. The surgical instrument may define an instrument axis. The surgical navigation system may be configured to plan one or more trajectories based on the patient data. The head-mounted display may be configured to display augmented reality visualization, including an augmented reality position alignment visualization and/or an augmented reality angular alignment visualization related to the surgical instrument on the lens of the head-mounted display.

To support easy arrangement of a robot, an information processing apparatus includes: a projection image generating section which generates a projection image that projects information specifying a range of motion of a target part of a medical arm onto the range of motion on the basis of first information regarding the range of motion and second information regarding a position and a posture of a projection device that projects an image in an operating room; and an output instruction section that outputs a projection instruction for the projection image to the projection device.

An endoscope system includes an endoscope, at least one monitor, a first measuring instrument configured to measure a first direction, at least one second measuring instrument configured to measure a second direction, a reception circuit configured to receive the image from the endoscope, and a processor. The first direction corresponds to a horizontal component of an imaging direction of the endoscope. The second direction corresponds to a horizontal component of a reference direction. The reference direction is a direction in which the at least one monitor faces or is a direction toward or away from an image display surface of the at least one monitor. The processor is configured to calculate an angle between the first direction and the second direction, process the image based on the angle, and display the processed image on the at least one monitor. The processing includes a rotation processing or a horizontal flip processing.

Presented herein are methods, systems, devices, and computer-readable media for image guided surgery. The systems herein allow a physician to use multiple instruments for a surgery and simultaneously provide image-guidance data for those instruments. Various embodiments disclosed herein provide information to physicians about procedures they are performing, the devices (such as ablation needles, ultrasound wands or probes, scalpels, cauterizers, etc.) they are using during the procedure, the relative emplacements or poses of these devices, prediction information for those devices, and other information. Some embodiments provide useful information about 3D data sets. Additionally, some embodiments provide for quickly calibratable surgical instruments or attachments for surgical instruments.

A surgical navigation system includes a source of a patient anatomy data, wherein the patient anatomy data comprises a three-dimensional reconstruction of a segmented model comprising at least two sections representing parts of the anatomy. A surgical navigation image generator is configured to generate a surgical navigation image comprising the patient anatomy. A 3D display system is configured to show the surgical navigation image wherein the display of the patient anatomy is selectively configurable such that at least one section of the anatomy is displayed and at least one other section of the anatomy is not displayed.

A method of assessing inter-device communication pairing in a surgical setting, may include transmitting, by a first intelligent medical device, wireless communication data within the surgical setting, receiving, by a second intelligent medical device, the wireless communication data from the first intelligent medical device, determining, by the second intelligent medical device, communication pairing data indicative of an inter-device communication pairing of the second intelligent medical device with the first intelligent medical device, transmitting, by the second intelligent medical device, the communication pairing data to a modular control tower, and displaying, by the modular control tower on a display device, an augmented reality display comprising one or more virtual objects indicative of the inter-device communication pairing. An interactive surgical system may include multiple intelligent medical devices and displays which can form communication pairs in this manner.

Disclosed herein is a method of operating a medical instrument (100, 200, 400, 500). The medical instrument comprises a user interface (108) with a display. The method comprises receiving (300) an anatomical segmentation (122) identifying a location of an anatomical structure (416) and receiving (302) a target zone segmentation (124) identifying a location of a volume (416) at least partially within the anatomical segmentation. The method further comprises displaying (304) a planning graphical user interface (112) using the display. The planning graphical user interface comprises a first panel (130) configured for rendering a cross sectional view of the anatomical segmentation (136) and the target zone segmentation (138). The planning graphical user interface comprises a second panel (132) configured for displaying a first three-dimensional model (140) of the anatomical segmentation and the target zone segmentation. The planning graphical user interface further comprises a third panel (134) configured for displaying a second three-dimensional model (142) of a remaining portion of the target zone segmentation. The planning graphical user interface further comprises an ablation selector (144, 144′, 146) configured for providing an ablation zone. The method further comprises repeatedly: receiving (306) the ablation zone from the ablation selector; and updating (308) the remaining portion by removing the ablation zone from the remaining portion.

Techniques are described for guiding a joint replacement surgery. In some examples, a system includes a visualization device comprising one or more sensors; and processing circuitry configured to determine, based on data generated by the one or more sensors, one or more size parameters of a bone resection surface viewable via the visualization device; select, based on the one or more size parameters of the bone resection surface and from a plurality of implants, an implant; and output for display, via the visualization device, a graphical representation of the selected implant relative to the bone resection surface.