A surgical system includes a detector, comprising an array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument. The surgical system also includes a processor configured to receive the first signal, generate a modified image of the surgical instrument that includes a control panel. The control panel includes one or more control elements representative of one or more operating parameters of the surgical instrument. The processor is further configured to receive an input to the control panel from a user, the input being effective to change one of the operating parameters. The processor is also configured to generate a command signal based on the input to change the one of the operating parameters.

Methods, apparatuses, and systems for transcribing and performing analysis on patient data are disclosed. Data is collected from one or more medical professionals as well as sensors and imaging devices positioned on or oriented towards a patient. An analysis is performed on the patient data and the data is presented to a medical professional via a verbal interface in a conversational manner, allowing the medical professional to provide additional data such as observations or instructions which may be used for further analysis or to perform actions related to the patient's care.

Methods, apparatuses, and systems for maintaining and controlling surgical tools are disclosed. For each surgical tool, a surgeon can give verbal commands which can result in feedback provided by a synthesized voice or the execution of an action as instructed by the verbal command. The tools are monitored during use to ensure the tools remain within their operating parameters. The system alerts the surgeon should the tools approach their operational limitations.

Alignment precision technology, in which a system accesses image data of a bone to which a reference marker array is fixed. The system generates a three-dimensional representation of the bone and the reference markers, defines a coordinate system for the three-dimensional representation, and determines locations of the reference markers relative to the coordinate system. The system accesses intra-operative image data that includes the bone and a mobile marker array that is attached to an instrument used in a surgical procedure. The system co-registers the intra-operative image data with the three-dimensional representation by matching the reference markers included in the intra-operative image data to the locations of the reference markers. The system determines locations of the mobile markers in the co-registered image and determines a three-dimensional spatial position and orientation of the instrument relative to the bone.

A system includes a mobile device having one or more cameras to take images; a sensor detecting reflected light from one or more lasers and a diffuser to detect object range or dimension; code for motion tracking, environmental understanding by detecting planes in an environment, and estimating light and dimensions of the surrounding based on the one or more lasers; code to estimate a three-dimensional (3D) volume of an object from multiple perspectives and from projected laser beams to measure size or scale and determine locations of points on the object's surface in a plane or a slice using time-of-flight, wherein positions and cross-sections for different slices are correlated to construct a 3D model of the object, including object position and shape; the device receiving user request to select a content from one or more augmented, virtual, or extended reality contents and rendering a reality view of the environment.

A system and method for providing enhanced information to a surgeon is described. A three-dimensional reconstruction of a patient's anatomical structure selected for surgery and a representation of a surgical treatment apparatus are rendered on a display device. At least one attachment metric for a proposed attachment between the surgical treatment apparatus and the patient's anatomical structure is calculated using the three-dimensional position of the surgical treatment apparatus relative to the patient's anatomical structure. And, an indication of the attachment metric is rendered on the display device.

An exemplary system includes a memory storing instructions and a processor communicatively coupled to the memory. The processor may be configured to execute the instructions to: detect an intent of a user of a computer-assisted surgical system to use a robotic instrument attached to the computer-assisted surgical system to interact with a target object while the target object is located in a surgical space; determine a pose of the target object in the surgical space; and perform, based on the detected intent of the user to interact with the target object and the determined pose of the target object in the surgical space, an operation with respect to the target object.

A number of improvements are provided relating to computer aided surgery utilizing an on tool tracking system. The various improvements relate generally to both the methods used during computer aided surgery and the devices used during such procedures. Other improvements relate to the structure of the tools used during a procedure and how the tools can be controlled using the OTT device. Still other improvements relate to methods of providing feedback during a procedure to improve either the efficiency or quality, or both, for a procedure including the rate of and type of data processed depending upon a CAS mode.

A computer system receives readings from sensors embedded in a spinal implant implanted in a patient during surgery. The sensor readings are indicative of a load applied by a spine of the patient on the spinal implant. The load causes physical discomfort to the patient. A feature vector is extracted from the implant sensor readings using a machine learning module. The feature vector is indicative of the physical discomfort caused by the load. Electrical signals are generated using the machine learning module based on the feature vector. The machine learning module is trained based on patient data sets to generate the electrical signals to balance the load, such that the physical discomfort is reduced. The electrical signals are transmitted to one or more actuators embedded in the spinal implant to cause the one or more actuators to configure the spinal implant, such that the load is balanced.

An orthopedic system to monitor a parameter related to the muscular-skeletal system is disclosed. The orthopedic system includes electronic circuitry, at least one sensor, and a computer to receive measurement data in real-time. The orthopedic system comprises a first plurality of shims of a first type, a second plurality of a second type, a measurement module, and the computer. The measurement module houses the electronic circuitry and at least one sensor. The measurement module is adapted to be used with the first plurality of shims and the second plurality of shims. The measurement module has a medial surface that differs from a lateral surface by shape, size, or contour.