Devices and methods for performing a surgical step or surgical procedure with visual guidance using an optical head mounted display are disclosed.

An insertion system having a processor including hardware, configured to: determine, for each of a plurality of times, one or more positions of an insertion section configured to be inserted into an insertion subject; determine, for the each of the plurality of times, state of the insertion section relative to the insertion subject based on the one or more positions for the each of the plurality of times; determine, for the each of the plurality of times, whether an attention state of the insertion section that restricts insertion of the insertion section into the insertion subject has occurred, based on the state of the insertion section relative to the insertion subject determined; and control a monitor to display attention state information indicating the one or more positions of the insertion section at which the occurrence of the attention state of the insertion section is determined over the plurality of times.

A technique is presented for computer-assisted planning of fastener placement in vertebrae that are to be stabilized by a pre-formed spinal rod, wherein two or more fasteners are to couple the rod to the vertebrae. At least one of a target shape of the spine and a shape of the pre-formed rod is defined in patient-specific shape data, a bone density of at least one of the vertebrae is defined in patient-specific bone density data, and patient-specific anatomical data are provided. A force distribution along a length of the rod may be calculated based on the shape data and the anatomical data. A planning result for fastener placement may be generated based on the bone density data of a given vertebra and the calculated force distribution.

A surgical navigation system includes a first tracking unit, a second tracking unit and a processing unit. The first tracking unit captures a first infrared image of a position identification unit that includes a reference target fixed on a patient and an instrument target disposed on a surgical instrument. The second tracking unit captures a second infrared image of the position identification unit. The processing unit performs image recognition on the first and second infrared images with respect to the position identification unit, and uses, based on a result of the image recognition, a pathological image and one of the first and second infrared images to generate an augmented reality image. When both the first and second images have both the reference target and the instrument target, one of the first image and the second image with a higher accuracy is used to generate the augmented reality image.

To support easy arrangement of a robot, an information processing apparatus includes: a projection image generating section which generates a projection image that projects information specifying a range of motion of a target part of a medical arm onto the range of motion on the basis of first information regarding the range of motion and second information regarding a position and a posture of a projection device that projects an image in an operating room; and an output instruction section that outputs a projection instruction for the projection image to the projection device.

Surgical systems and methods are provided that utilize augmented reality navigation and visualization techniques for transferring aspects of a preoperative surgical plan to an actual surgical site. The surgical systems and methods may be utilized to achieve accurate alignment of a surgical positioning object, such as a guide pin for guiding surgical reaming procedures, between the preoperative surgical plan and the intraoperative anatomy associated with the actual surgical site. Augmented reality may be utilized to achieve visualization of both an entry point and a drilling trajectory of the surgical positioning object in a manner that avoids occluding the intraoperative anatomy during the procedure.

Automated user preferences can be applied on head-mounted devices, such as smartglasses. A head-mounted device is configured for this purpose. Upon recognizing a three-dimensional orientation of the head-mounted device as a three-dimensional orientation for which an associated set of parameters is stored, the parameters associated with the recognized three-dimensional orientation for the image sensor are automatically loaded and set.

Disclosed is a method for providing guidance to a user on where to arrange an object of interest in an operating room. The method includes obtaining a spatial pose of an item arranged within the operating room and obtaining a requirement on a spatial relationship between an object of interest and the item. The method also includes determining (a) a first spatial portion in which the object of interest is recommended to be arranged, the first spatial portion being larger than the object of interest, and/or (b) a second spatial portion in which the object of interest is not allowed to be arranged. Moreover, the method includes triggering display of a visualization of the first and/or second spatial portion to guide a user on where to arrange the first object of interest in the operating room.

Methods and systems for displaying an overlay superimposed with an intraoperative image of surgical field in a medical ophthalmic procedure, such that the overlay appears at a desired depth within the image are provided. Methods and system for displaying an overlay superimposed with a stereoscopic intraoperative image pair of a surgical field in a medical ophthalmic procedure are provided.

Tracking constellations for use with surgical instruments, and associated systems and methods, are disclosed herein. In some embodiments, a tracking constellation includes (i) a support, (ii) a plurality of first standoffs extending from the support to a first height, and (iii) a plurality of second standoffs extending from the support to a second height different than the first height. The tracking constellation can further include a plurality of markers mounted to corresponding ones of the first standoffs or the second standoffs. The markers can lay in a common plane. The support can extend at a generally orthogonal angle to an instrument when the tracking constellation is coupled to the instrument such that the common plane is angled relative to the instrument.