A surgical system is provided. The surgical system includes a camera operable to capture images and/or video. A projector is operable to project light, and a controller is communicatively coupled with the camera and the projector. The controller is operable to track movement of bone in real-time during surgery based on the images and/or video captured by the camera, and control the projector to project the light including a cutting line on the bone to indicate a cutting plane for cutting the bone during surgery.

Devices and systems are provided for tracking a position and orientation of a handheld implement, such that the handheld implement may be trackable with an overhead tracking system. A support member secures one or more markers relative to a longitudinal portion of the handheld implement, and a marker plane containing the markers is orientated an angle relative to a longitudinal axis of the longitudinal portion. A marker assembly may include a support member for supporting the markers, and a connector for removably attaching the marker assembly to one or more handheld implements. The marker assembly may be configured to be removably attachable to a plurality of connection adapters, where each connection adapter is further connectable to a handheld implement, optionally at a calibrated position, such that a single connection adapter can be optionally employed to track a plurality of handheld implements. The handheld implement may be a medical instrument.

An optical marker for positioning a medical device outside a body and the medical device are disclosed. The optical marker comprises: a base having a concave or a convex, the concave or the convex being provided with a non-coplanar optical mark(s), and the optical mark(s) being visible on the entire surface of the concave or the convex; and a connecting portion connected to the base and used for connecting with the medical device. In the optical marker, the base provided with the optical mark(s) has a non-planar structure, being able to increase the recognizable angle of the medical device and improve the recognition accuracy and stability.

A surgical system is configured to augment the visualization environment presented to the surgeon by merging, in real-time, video feed and ultrasound imaging; tracking anatomy and instruments; identifying critical structures; generating and displaying 3-dimensional models of relevant anatomy; providing actionable guidance to the user; and enabling data collection and processing. The surgical system may include a tissue-marking surgical instrument configured to simultaneously identify critical structures beneath an organ surface and mark the organ surface at a location overlapping the identified critical structures.

Systems and methods are provided for performing intraoperative fusion of two or more volumetric image datasets via surface-based image registration. The volumetric image datasets are separately registered with intraoperatively acquired surface data, thereby fusing the two volumetric image datasets into a common frame of reference while avoiding the need for complex and time-consuming preoperative volumetric-to-volumetric image registration and fusion. The resulting fused image data may be processed to generate one or more images for use during surgical navigation.

A method and system perform an ophthalmic procedure on an eye having an optical path from the lens to the retina. An image of at least part of the eye is received in a data processing unit. The image includes the optical path. The data processing unit determines keep out zone(s) and identifies undesirable feature(s) based on the image. The keep out zone(s) include the retina. The data processing unit also selects one of the undesirable feature(s) for removal. At least part of the undesirable feature is outside of the keep out zone(s). Confirmation for removal of the undesirable feature is received in the data processing unit. In response to receiving the confirmation, a control unit controls a laser to perform laser removal the at least the portion of the undesirable feature without targeting any portion of the keep out zone(s).

A surgical system includes a detector, comprising an array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument. The surgical system also includes a processor configured to receive the first signal, generate a modified image of the surgical instrument that includes a control panel. The control panel includes one or more control elements representative of one or more operating parameters of the surgical instrument. The processor is further configured to receive an input to the control panel from a user, the input being effective to change one of the operating parameters. The processor is also configured to generate a command signal based on the input to change the one of the operating parameters.

Medical imaging systems and methods are provided herein that provide for navigation and procedure planning without segmentation. A method comprises receiving, by a medical imaging system having at least one processing device, three-dimensional image data of a patient anatomy. The method also comprises filtering the three-dimensional data to display a portion of the three-dimensional image data that is associated with the patient anatomy and receiving, at the processing device, input from an operator input device. The input comprises navigational directions for virtual movement within a space defined by the three-dimensional image data. The method also includes tracking the virtual movement, defining a tracked pathway based on the tracked virtual movement, and generating a model of the patient anatomy based on the tracked pathway. The model of the patient anatomy is a line model including one or more lines based on the tracked pathway.

The invention involves a system and method for increasing positional accuracy of surgical systems that utilize magnetic or electromagnetic sensors to provide positional awareness to a surgeon or robot performing the surgery. The system takes advantage of electromagnetic tracking through sensors. These sensors are very accurate and repeatable, while being compact enough to not inhibit surgical procedures. The accuracy and repeatability of the sensors is <1 mm within a predetermined 6 inch×6 inch performance motion box. The system is constructed and arranged to map the distortion patterns of the sensor and, in real time, correct the distortion pattern to provide accurate location of anatomical structures for performance of a surgery.

Embodiments described herein provide various techniques and systems for building machine-learning surgical tool presence/absence detection models for processing surgical videos and predicting whether a surgical tool is present or absent in each video frame of a surgical video. In one aspect, a process for ensuring patient safety during a laparoscopic or robotic surgery involving an energy tool is disclosed. The process can begin receiving a real-time control signal indicating an operating state of an energy tool during the surgery. Next, the process receives real-time endoscope video images of the surgery. The process simultaneously applies a machine-learning surgical tool presence/absence detection model to the real-time endoscope video images to generate real-time decisions on a location of the energy tool in the real-time endoscope video images. The process then checks the real-time control signal against the real-time decisions to identify an unsafe event and takes a proper action when an unsafe event is identified.