The invention relates to a correlated set for minimal invasive surgery comprising a surgical instrument and a pattern generating member, a surgical system, a training kit, a method of training and a meth of performing a minimal invasive surgery. The surgical instrument comprises a handle portion, a surgical tool and a body portion connecting the handle portion to the surgical tool. The pattern generating member comprises a pattern light source and a projector for projecting a light pattern. The projector is adapted for being at least temporarily fixed to the body portion of the surgical instrument such that a movement of said surgical tool results in a correlated movement of said projector.

An endoscope system according to the present invention includes: a second illumination unit that emits second illumination light; a first illumination unit that emits, simultaneously with the second illumination light, first illumination light that is light of a different wavelength band from the second illumination light and is for imaging two sets of image information about a subject different depths; an imaging unit that simultaneously images a first illumination image of the subject illuminated with the first illumination light and a second illumination image of the subject illuminated with the second illumination light; a separation processing unit that separates the two sets of image information from the first illumination image; and a separated-image creating unit that processes the second illumination image using the two sets of image information to create separated images.

An automated surgical system includes a visualization system, an imaging display system, and a surgical hub. The visualization system includes an illumination source and an imaging device. The surgical hub includes a processor and a memory device configured to store instructions. The instructions direct the surgical hub processor to control the illumination source to illuminate a surgical site and to control the imaging device to receive the imaging data. The surgical processor processes the imaging data to create display data that is transmitted over a cloud-based network to the imaging display system for display.

An automated surgical hub system is configured to receive first image data of a surgical site. The hub system controls at least one illumination source of the surgical site in a first manner and receives second image data of the site under illumination by the source in the first manner. The hub system calculates a three-dimensional model of the surgical site based on the second image data and integrates the three-dimensional model with the first image data.

A surgical visualization system includes multiple light sources, at least one optical sensor, at least one display device, and a controller. The controller is configured to control at least one light source to illuminate surgical tissue in a first manner, and control the optical sensor to receive an image from the surgical tissue illuminated in the first manner. The controller calculates a three-dimensional image from the tissue illuminated in the first manner. The controller is configured to control at the least one light source to illuminate surgical tissue in a second manner, and control the optical sensor to receive an image from the surgical tissue illuminated in the second manner. The controller calculates one or more subsurface critical anatomic structures from the tissue illuminated in the second manner and combines the display of the three-dimensional image and the anatomic structures on the at least one display device.

Various adaptive surgical visualization systems are disclosed. Surgical visualizations can compensate for obscured, incomplete, damaged, or interfered with portions of captured images by substituting those portions of the images with corresponding portions of other images. The other images could include images that were previously generated by the surgical visualization system or images that were generated using multispectral imaging techniques.

A visualization system including multiple light sources, an image sensor configured to detect imaging data from the multiple light sources, and a control circuit is disclosed. At least one of the light sources is configured to emit a pattern of structured light. The control circuit is configured to receive the imaging data from the image sensor, generate a three-dimensional digital representation of the anatomical structure from the pattern of structured light detected by the imaging data, obtain metadata from the imaging data, overlay the metadata on the three-dimensional digital representation, receive updated imaging data from the image sensor, and generate an updated three-dimensional digital representation of the anatomical structure based on the updated imaging data. The visualization system can be communicatively coupled to a situational awareness module configured to determine a surgical scenario based on input signals from multiple surgical devices.

A surgical system for use in a surgical procedure is disclosed. The surgical system includes a surgical visualization system and a control circuit. The configured to propose a portion of an organ to resect based on visualization data from the surgical visualization system, determine a first value of a non-visualization parameter of the organ prior to resection of the portion, and determine a second value of the non-visualization parameter of the organ after resection of the portion. Resection of the portion is configured to yield an estimated capacity reduction of the organ.

A surgical system for use in a surgical procedure is disclosed. The surgical system includes at least one imaging device and a control circuit configured to identify an anatomical organ targeted by the surgical procedure, generate a virtual three-dimensional (3D) construct of at least a portion of the anatomical organ based on visualization data from the at least one imaging device, identify anatomical structures relevant to the surgical procedure from the visualization data from the at least one imaging device, couple the anatomical structures to the virtual 3D construct, and overlay onto the virtual 3D construct a layout plan of the surgical procedure determined based on the anatomical structures.

A surgical visualization system that can include a structured light emitter, a spectral light emitter, an image sensor, and a control circuit is disclosed herein. The structured light emitter can emit a structured pattern of electromagnetic radiation onto an anatomical structure. The spectral light emitter can emit electromagnetic radiation including a plurality of wavelengths. At least one of the wavelengths can penetrate a portion of the anatomical structure and reflect off a subject tissue. The image sensor can detect the structured pattern of electromagnetic radiation reflected off the anatomical structure and the at least one wavelength reflected off the subject tissue. The control circuit can receive signals from the image sensor, construct a model of the anatomical structure, detect a location of the subject tissue, and determine a margin about the subject tissue, based on at least one signal received from the image sensor.