A user interface for enabling and controlling functions of a surgical system, comprising: a foot-operated function switch configured to be switched to any one of multiple discrete states by the user's foot; a head tracker configured to track the user's head motions; and a processor coupled to the function switch and head tracker, and configured to: obtain an association between a plurality of sequences comprising at least one of the multiple discrete states, and a plurality of corresponding system functions of the surgical system, receive an indication of at least one of the discrete states from the function switch, identify a system function based on the indication and the association, receive a head motion from the head tracker, and apply the head motion to control the identified system function of the surgical system.

A wearable device can present virtual content to the wearer for many applications in a healthcare setting. The wearer may be a patient or a healthcare provider (HCP). Such applications can include, but are not limited to, access, display, and modification of patient medical records and sharing patient medical records among authorized HCPs.

A method of assessing inter-device communication pairing in a surgical setting, may include transmitting, by a first intelligent medical device, wireless communication data within the surgical setting, receiving, by a second intelligent medical device, the wireless communication data from the first intelligent medical device, determining, by the second intelligent medical device, communication pairing data indicative of an inter-device communication pairing of the second intelligent medical device with the first intelligent medical device, transmitting, by the second intelligent medical device, the communication pairing data to a modular control tower, and displaying, by the modular control tower on a display device, an augmented reality display comprising one or more virtual objects indicative of the inter-device communication pairing. An interactive surgical system may include multiple intelligent medical devices and displays which can form communication pairs in this manner.

An intelligent surgical tool control system, comprising a tool management system; an indicating means to indicate at least one surgical event; a communicable database for storing, for each item of interest, its identity, its present 3D position and at least one previous 3D position; and at least one processor to identify, from a surgical event, an output surgical procedure. The tool management system can comprise a maneuvering mechanism to maneuver a surgical tool in at least two dimensions; and a controller to control at least one of activation and deactivation of a surgical tool and articulation of a surgical tool. The indicating means can indicate a surgical event selected from movement of a moving element and presence of an item of interest, where movement is determinable if the current 3D position of the moving element is substantially different from a previous 3D position of the same.

A surgical robotic system has a robotic grasper, a user interface device (UID), and one or more processors communicatively coupled to the UID and the robotic grasper. The system detects a directive to engage or re-engage a teleoperation mode, determines that the system is in a non-teleoperation mode, receives a sequence of user actions through the UID, determines the UID matches a jaw angle or a grip force of the robotic grasper, and transitions into teleoperation mode. Other embodiments are also described and claimed.

A method for ocular simulated camera assisted robot training is provided. In some implementations, the method includes initializing, by a processor, a robotics assembly. The method further includes connecting, by the processor, to one or more computing devices. The method further includes operating, by the processor, the robotics assembly. The method further includes simulating, by the processor, an eye movement of a human or animal. The method further includes operating, by the processor, a laser to perform a determined exercise on an eye of the robotics assembly. Related systems, methods, and articles of manufacture are also described.

The present disclosure involves object recognition as a method of registration, using a stereoscopic camera on Augmented Reality (AR) glasses or an endoscope as the image capture technology. Exemplary objects include surgical tools, anatomical components or features, such as bone or cartilage, etc. By detecting just a portion of the object in the image data of the surgical scene, the present disclosure may register and track a portion of the patient's anatomy, such as the pelvis, the knee, etc. The present disclosure also optionally displays information on the AR glasses themselves, such as the entire pelvis, the femur, the tibia, etc. The present disclosure may include combinations of the foregoing features, and may eliminate the need for electromagnetic, inertial, or infrared stereoscopic tracking as the tracking technology.

A system for use in surgery includes a central body, a visualization system operably connected to the central body, a video rendering system, a head-mounted display for displaying images from the video rendering system, a sensor system, and a robotic device operably connected to the central body. The visualization system includes at least one camera and a pan system and/or a tilt system. The sensor system tracks the position and/or orientation in space of the head-mounted display relative to a reference point. The pan system and/or the tilt system are configured to adjust the field of view of the camera in response to information from the sensor system about changes in at least one of position and orientation in space of the head-mounted display relative to the reference point.

A system comprises an image capture device, an augmented reality (AR) display to display, and a processing device. The processing deice receives image data of a dental arch from the image capture device and processes the image data using a plurality of detection rules, where each detection rule detects one or more dental conditions. The processing device determines a dental condition for the dental arch based on the processing, determines a position of an area of interest on the dental arch, wherein the area of interest is associated with the dental condition, generates a visual overlay comprising an indication of the dental condition at the position of the area of interest, and outputs the visual overlay to the AR display, wherein the visual overlay is superimposed over a view of the dental arch on the AR display at the position of the area of interest.

Systems, methods, and computer-readable media are provided for controlling a robotically delivered manipulator. One system includes a robotic manipulator having a base and a surgical instrument holder configured to move relative to the base, a surgical instrument removably coupled to the surgical instrument holder, a user interface configured to present information related to least one of the robotic manipulator or the surgical instrument, a gesture detection sensor configured to detect a gesture made by a user representing a desired movement of the robotic manipulator, and a controller configured to actuate the robotic manipulator in a predetermined manner corresponding to the detected gesture.