A medical apparatus can include an instrument comprising a shaft and a jaw assembly coupled to an end of the shaft; an image capture device; and a controller operably coupled to the image capture device to receive image data from the image capture device. The image data is from images of material gripped between jaw members of the jaw assembly and captured by the image capture device, with the controller programmed to process the received image data using at least one of optical flow and digital image correlation. A medical apparatus can include an instrument comprising a shaft, and a jaw assembly coupled to an end of the shaft, the jaw assembly comprising a pair of jaw members having opposing surfaces configured to grasp material between the opposing surfaces, wherein at least a portion of the opposing surface of a first jaw member of the pair of jaw members is transparent.

An imaging device includes a hollow flexible shaft having a central longitudinal axis and an imaging window therein. An optical fiber extends within the hollow flexible shaft substantially along the central axis. A distal tip of the optical fiber is attached to the hollow flexible shaft and aligned with the imaging window so as to transfer an optical coherence tomography signal through the imaging window. A handle is attached to the hollow flexible shaft configured rotate the hollow flexible shaft at speeds of greater than 1,000 rpm.

Surgical navigation system: 3D display system with see-through visor; a tracking system for real-time tracking of: surgeon's head, see-through visor, patient anatomy and surgical instrument to provide current position and orientation data; a source of an operative plan, a patient anatomy data and a virtual surgical instrument model; a surgical navigation image generator to generate a surgical navigation image with a three-dimensional image representing simultaneously a virtual image of the surgical instrument corresponding to the current position and orientation of the surgical instrument and a virtual image of the surgical instrument, the see-through visor, the patient anatomy and the surgical instrument; the 3D display system configured to show the surgical navigation image at the see-through visor, such that an augmented reality image collocated with the patient anatomy in the surgical field underneath the see-through visor is visible to a viewer looking from above the see-through visor towards the surgical field.

Methods, systems, and computer-readable medium tracks locations of one or more surgical instruments. The method includes detecting a plurality of markers disposed on a distal end of a first surgical instrument within a field of view of a camera, calculating a position of the first surgical instrument based on a location of the plurality of markers within the field of view of the camera, determining the position of the first surgical instrument in relation to a second surgical instrument.

Robotic surgical systems and methods of operating robotic surgical systems are included. The methods include directing light at an optical element configured to be detected by an image capture device of the robotic surgical system, the optical element configured to reflect light having a wavelength within a predetermined range, detecting, using an image capture device capturing images of the optical element, an absence or a presence of the reflected light from the optical element, and providing a notification, in response to the detection by the image capture device of the absence of the reflected light from the optical element.

Surgical navigation system: 3D display system with see-through visor; a tracking system for real-time tracking of: surgeon's head, see-through visor, patient anatomy and surgical instrument to provide current position and orientation data; a source of an operative plan, a patient anatomy data and a virtual surgical instrument model; a surgical navigation image generator to generate a surgical navigation image with a three-dimensional image representing simultaneously a virtual image of the surgical instrument corresponding to the current position and orientation of the surgical instrument and a virtual image of the surgical instrument, the see-through visor, the patient anatomy and the surgical instrument; the 3D display system configured to show the surgical navigation image at the see-through visor, such that an augmented reality image collocated with the patient anatomy in the surgical field underneath the see-through visor is visible to a viewer looking from above the see-through visor towards the surgical field.

A camera and expandable bladed retractor system. The system includes a camera, with a distal-most optical element disposed over the proximal end of the lumen or surgical channel established by the retractor blades. The camera may be fixed to the retractor, either to the proximal end of a blade or to a frame member.

[Object] To provide a control apparatus, a control method, and a medical observation system.

[Solution] A control apparatus including: an acquisition unit configured to acquire dental mirror detection information from a picked up image acquired by an image pickup apparatus; and a control unit configured to, on the basis of the dental mirror detection information, perform at least one of image processing on the picked up image or arm control processing that controls an arm unit that supports the image pickup apparatus.

Assisting robotic arm setup in a surgical robotic system using augmented reality can include capturing a live video of a user setting up a robotic arm in a surgical robotic system. A visual guide representing a target pose of the robotic arm can be rendered onto the live video, resulting in an augmented live video for guiding the arm setup. The augmented live video can be displayed to the user while the user is following the visual guide to set up the robotic arm. The captured live video can be continuously processed to determine whether the robotic arm has reached the target pose.

In accordance with one aspect of the present disclosure, provided is a surgical heads-up display system comprising a heads-up display and an articulable arm coupled to the heads-up display. The heads-up display is configured to receive a portable electronic device and to redirect a display output of the portable electronic device. The articulable arm is configured to enable selective positioning and maintenance of the heads-up display in a selected position such that the display output of the portable electronic device is redirected in a chosen direction.