A camera tracking system receives patient reference tracking information indicating pose of a patient reference array tracked by a patient tracking camera relative to a patient reference frame. A local XR headset view pose transform is determined between a local XR headset reference frame and the patient reference frame. Remote reference tracking information is received indicating pose of a remote reference array tracked by a remote reference tracking camera. A remote XR headset view pose transform is determined between a remote XR headset reference frame of a remote XR headset and the remote reference array. A 3D computer image is transformed from a local pose determined using the local XR headset view pose transform to a remote pose determined using the remote XR headset view pose transform. The transformed 3D computer image is provided to the remote XR headset for display with the remote pose relative to the remote XR headset reference frame.

Devices and methods for performing a surgical step or surgical procedure with visual guidance using an optical head mounted display are disclosed.

An insertion system having a processor including hardware, configured to: determine, for each of a plurality of times, one or more positions of an insertion section configured to be inserted into an insertion subject; determine, for the each of the plurality of times, state of the insertion section relative to the insertion subject based on the one or more positions for the each of the plurality of times; determine, for the each of the plurality of times, whether an attention state of the insertion section that restricts insertion of the insertion section into the insertion subject has occurred, based on the state of the insertion section relative to the insertion subject determined; and control a monitor to display attention state information indicating the one or more positions of the insertion section at which the occurrence of the attention state of the insertion section is determined over the plurality of times.

A technique is presented for computer-assisted planning of fastener placement in vertebrae that are to be stabilized by a pre-formed spinal rod, wherein two or more fasteners are to couple the rod to the vertebrae. At least one of a target shape of the spine and a shape of the pre-formed rod is defined in patient-specific shape data, a bone density of at least one of the vertebrae is defined in patient-specific bone density data, and patient-specific anatomical data are provided. A force distribution along a length of the rod may be calculated based on the shape data and the anatomical data. A planning result for fastener placement may be generated based on the bone density data of a given vertebra and the calculated force distribution.

A surgical navigation system includes a first tracking unit, a second tracking unit and a processing unit. The first tracking unit captures a first infrared image of a position identification unit that includes a reference target fixed on a patient and an instrument target disposed on a surgical instrument. The second tracking unit captures a second infrared image of the position identification unit. The processing unit performs image recognition on the first and second infrared images with respect to the position identification unit, and uses, based on a result of the image recognition, a pathological image and one of the first and second infrared images to generate an augmented reality image. When both the first and second images have both the reference target and the instrument target, one of the first image and the second image with a higher accuracy is used to generate the augmented reality image.

Automated user preferences can be applied on head-mounted devices, such as smartglasses. A head-mounted device is configured for this purpose. Upon recognizing a three-dimensional orientation of the head-mounted device as a three-dimensional orientation for which an associated set of parameters is stored, the parameters associated with the recognized three-dimensional orientation for the image sensor are automatically loaded and set.

Disclosed is a method for providing guidance to a user on where to arrange an object of interest in an operating room. The method includes obtaining a spatial pose of an item arranged within the operating room and obtaining a requirement on a spatial relationship between an object of interest and the item. The method also includes determining (a) a first spatial portion in which the object of interest is recommended to be arranged, the first spatial portion being larger than the object of interest, and/or (b) a second spatial portion in which the object of interest is not allowed to be arranged. Moreover, the method includes triggering display of a visualization of the first and/or second spatial portion to guide a user on where to arrange the first object of interest in the operating room.

An apparatus includes a base portion configured to selectively couple with a robotic arm. A shaft extends distally form the base portion and terminates into a distal end. A sleeve is slidably coupled to the shaft. A colpotomy cup is fixedly secured to a portion of the sleeve. One or more sensors are configured to detect a force applied to the sleeve, the shaft, or both the sleeve and the shaft.

Tracking constellations for use with surgical instruments, and associated systems and methods, are disclosed herein. In some embodiments, a tracking constellation includes (i) a support, (ii) a plurality of first standoffs extending from the support to a first height, and (iii) a plurality of second standoffs extending from the support to a second height different than the first height. The tracking constellation can further include a plurality of markers mounted to corresponding ones of the first standoffs or the second standoffs. The markers can lay in a common plane. The support can extend at a generally orthogonal angle to an instrument when the tracking constellation is coupled to the instrument such that the common plane is angled relative to the instrument.

The system for assisting a surgeon in the context of surgical operations includes a mobile wearable computer worn or carried by a user of the system, at least one perception sensor for collecting anatomical and/or instrumental measurement data in an operating scene, a processor adapted to process the collected anatomical and/or instrumental measurement data collected by the at least one perception sensor, a memory connected to the processor and adapted to store anatomical and/or instrumental measurement data, a robotic effector connected to the processor and adapted to carrying an instrument, wherein movements of the robotic effector are slaved to the anatomical and/or instrumental measurement data, wherein the anatomical and/or instrumental data delimit an area of action within which the movements of the robotic arm are restricted in the operating scene.