An image processing apparatus includes a processor including hardware, the processor being configured to: estimate, based on image information from a medical device that includes at least an endoscope, a plurality of procedure actions of an operator of the endoscope; perform different supports respectively for procedures by the operator, according to an estimation result of the procedure actions; and output a display for the supports to a display.

Systems, methods, software and techniques for generating a milling path for a tool of a surgical system are provided. The milling path is designed to remove a resection volume associated with an anatomical volume. A reference guide is defined with respect to the resection volume. Sections are defined along the reference guide in succession. Each section intersects the reference guide at a different intersection point and is at a specified orientation relative to the reference guide at the intersection point. Each section further intersects the resection volume. A section path is generated to be bounded within each section and defined relative to the resection volume. A plurality of transition segments are generated and each transition segment connects section paths of successive sections along the reference guide.

An object or body-mounted camera apparatus for recording surgery is provided that is adapted for tracking a relevant visual field of an on-going operation. To help maintain visibility and/or focus of the visual field, specific machine learning approaches are proposed in combination with control commands to shift a physical positioning or a perspective of the camera apparatus. Additional variations are directed to tracking obstructions based on the visual field of the camera, which can be utilized for determining a primary recording for use when there are multiple cameras being used in concert.

A system is suggested comprising an optical sensing means and a processing unit. The optical sensing means may include an optical guide with a distal end, wherein the optical guide may be configured to be arranged in a device to be inserted into tissue in a region of interest. The processing unit may be configured to receive information of a region of interest including different tissue types as well as of a path through the tissues, to determine a sequence of tissue types along the path, to determine a tissue type at the distal end of the optical guide based on information received from the optical sensing means, to compare the determined tissue type with the tissue types on the path, to determine possible positions of the distal end of the optical guide on the path based on the comparison of tissue types, and to generate a signal indicative for the possible positions.

A method for determining a shape of a lumen in an anatomical structure comprises reading information from a plurality of strain sensors disposed substantially along a length of a flexible medical device when the flexible medical device is positioned in the lumen. When the flexible medical device is positioned in the lumen, the flexible medical device conforms to the shape of the lumen. The method further comprises computationally determining, by a processing system, the shape of the lumen based on the information from the plurality of strain sensors.

A surgical system includes an arm extending from the base and having a distal end configured to be coupled to a tool, a first marker coupled in fixed relation to the base, and a tracking system. The tracking system is configured to collect first data indicative of a position of the first marker and collect second data indicative of a position an anatomical feature of a patient. The surgical system also includes a processor configured to calculate a position of the tool relative to the anatomical feature based on the first data and the second data.

Disclosed is a computer-implemented method of transmitting identification information of a medical instrument. The method encompasses comparing a digital image of an instrument tray and an instrument to a digital image of just the instrument tray to determine the identity of the instrument. A characteristic geometry such as its envelope is assigned to the instrument, and a characteristic quantity of the envelope such as its aspect ratio may be used to identify the instrument. Based on determining, from the image of the instrument and the instrument tray, the relative position between those two entities, the method determines whether the instrument has been taken from the instrument tray, and accordingly instructs a medical computing system about this determination. The medical computing system may then determine whether the correct instrument has been taken from the instrument has been taken from the instrument tray, for example by comparison with medical procedure planning data.

A computer assisted surgery system comprising: a computerised surgical apparatus; and a control apparatus; wherein the control apparatus comprises circuitry configured to: receive information indicating a first region of a surgical scene from which information is obtained by the computerised surgical apparatus to make a decision; receive information indicating a second region of the surgical scene from which information is obtained by a medical professional to make a decision; determine if there is a discrepancy between the first and second regions of the surgical scene; and if there is a discrepancy between the first and second regions of the surgical scene: perform a predetermined process based on the discrepancy.

A head stabilization device or HFD includes a plurality of integrated markers. The HFD may be in the form of a skull clamp, vacuum bag, combinations thereof, or other supporting structure. The integrated markers include MRI markers detectable by an MRI scanner and fiducial markers detectable by a registration tool of a navigation system. The markers provide a reference point relative to an operation site in a patient for use in a navigation guided procedure and aid in registering or calibrating the location of the patient with captured images used in the navigation guided procedure.

A surgical robotic system includes: a surgical console having a display and a user input device configured to generate a user input and a surgical robotic arm having a surgical instrument configured to treat tissue and being actuatable in response to the user input; and a video camera configured to capture video data that is displayed on the display. The system also includes a control tower coupled to the surgical console and the surgical robotic arm. The control tower is configured to: process the user input to control the surgical instrument and to record the user input as input data; train a machine learning system using the input data and the video data; and execute the at least one machine learning system to determine probability of failure of the surgical instrument.