A method facilitates touchless control of medical equipment devices in an OR. The method involves: providing a three-dimensional control menu, which comprises a plurality of menu items selectable by the practitioner by one or more gestures made in a volumetric spatial region corresponding to the menu item; displaying an interaction display unit (IDU) image corresponding to the three-dimensional control menu to provide indicia of any selected menu items; estimating a line of sight of a practitioner; and when the estimated line of sight is directed within a first spatial range around a first medical equipment device, determining that the practitioner is looking at the first medical equipment device. Then the method involves providing a first device-specific three-dimensional control menu displaying a first device-specific IDU image.

Aspects of the present disclosure relate to systems, devices and methods for performing a surgical step or surgical procedure with visual guidance using an optical head mounted display. Aspects of the present disclosure relate to systems, devices and methods for displaying, placing, fitting, sizing, selecting, aligning, moving a virtual implant on a physical anatomic structure of a patient and, optionally, modifying or changing the displaying, placing, fitting, sizing, selecting, aligning, moving, for example based on kinematic information.

The invention relates to a method for operating an augmented reality observation system in a surgical application, wherein a viewing direction of a user is registered by means of a viewing direction sensor system of an AR observation apparatus, wherein the registered viewing direction is evaluated by means of a control device, and wherein at least one property of at least one controllable light source in the environment is altered by means of the control device by means of a control signal on the basis of the registered viewing direction. Further, the invention relates to an augmented reality observation system for a surgical application.

Systems and methods for a system include a manipulator configured to support an instrument moveable within an instrument workspace, the instrument having an instrument frame of reference; an input device configured to receive a movement command from an operator; and a control system. The control system is configured to determine a difference between an orientation of the instrument and an orientation of a field of view of the instrument workspace; adjust, based on the difference, a mapping to apply to the movement command to generate an implementable movement command for the instrument; further adjust the mapping based on an ergonomic offset to provide for a difference between an orientation of the input device and the orientation of the instrument; map, based on the adjusted mapping, the movement command to a motion of the instrument in the instrument frame of reference; and cause the instrument to move according to the motion.

Described herein is a surgical instrument guide for use with a robotic surgical system, for example, during spinal surgery. In certain embodiments, the guide is attached to or is part of an end effector of a robotic arm, and provides a rigid structure that allows for precise preparation of patient tissue (e.g., preparation of a pedicle) by drilling, tapping, or other manipulation, as well as precise placement of a screw in a drilled hole or affixation of a prosthetic or implant in a prepared patient situation.

A system for use in surgery includes a central body, a visualization system operably connected to the central body, a video rendering system, a head-mounted display for displaying images from the video rendering system, a sensor system, and a robotic device operably connected to the central body. The visualization system includes at least one camera and a pan system and/or a tilt system. The sensor system tracks the position and/or orientation in space of the head-mounted display relative to a reference point. The pan system and/or the tilt system are configured to adjust the field of view of the camera in response to information from the sensor system about changes in at least one of position and orientation in space of the head-mounted display relative to the reference point.

An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.

Methods and devices are disclosed for intra-operative viewing of pre- and intra-operative 3D patient images.

A method of designing an orthopedic implant comprising: (a) iteratively evaluating possible shapes of a dynamic orthopedic implant using actual anatomical shape considerations and kinematic shape considerations; and, (b) selecting a dynamic orthopedic implant shape from one of the possible shapes, where the dynamic orthopedic implant shape selected satisfies predetermined kinematic and anatomical constraints.

Aspects of the present disclosure relate to systems, devices and methods for performing a surgical step or surgical procedure with visual guidance using an optical head mounted display. Aspects of the present disclosure relate to systems, devices and methods for displaying, placing, fitting, sizing, selecting, aligning, moving a virtual implant on a physical anatomic structure of a patient and, optionally, modifying or changing the displaying, placing, fitting, sizing, selecting, aligning, moving, for example based on kinematic information.