A multifunctional laryngoscope is provided that includes a handle comprising a proximal end and a distal end and a display screen on the handle. The laryngoscope includes a laryngoscope camera at the distal end of the handle and an introducer comprising an orientation sensor at a distal end of the introducer. The laryngoscope includes a processor programmed to execute instructions for receiving from a steering input a steering command in a first reference frame, and mapping the steering command to a second reference frame oriented to the distal end of the introducer based on an orientation signal from the orientation sensor.

An endoscope system includes an endoscope, a moving device that moves the endoscope, a storage unit, and a processor. The storage unit stores first position information and first rotation angle information, which defines a rotation angle of an endoscope image of a first region, on the first region in a subject and second position information and second rotation angle information, which defines a rotation angle of an endoscope image of a second region, on the second region in the subject. The processor calculates third rotation angle information on a third region in the subject based on the first and second position information, the first and second rotation angle information, and third position information on the third region. If the third region includes the current imaging region, the processor rotates the endoscope image based on the third rotation angle information and outputs the rotated endoscope image to the display device.

Errors in a blended stream that would result in non-display or obscuring of a live video stream from a medical device may be automatically detected, and a failover stream corresponding to the first live video stream may be displayed to medical personnel. For example, one or more second input streams that are being blended may contain no data or invalid data which may result in the blended stream not displaying (or obscuring) the live video stream (if the blended were displayed). Switching from blending to a failover buffer may occur within the time to process a single video image frame. Upon detection (prior to display) that the blended stream would not display the live video stream, display of the live video stream from the failover buffer may be initiated. Other aspects are also described and claimed.

Disclosed embodiments integrate a camera into an intraoral mirror. Integrating a camera into an intraoral mirror provides an efficient way to record and display what is visible to the healthcare provider in the mirror.

A system (11) includes a medical probe (36) for insertion into a cavity of an organ, which includes a position and direction sensor (60) and a camera (45), both operating in a sensor coordinate system (62). The system further includes a processor (44) configured to: receive, from an imaging system (21) operating in an image coordinate system (28), a three-dimensional image of the cavity including open space and tissue; receive, from the medical probe, signals indicating positions and respective directions of the medical probe inside the cavity; receive, from the camera, respective visualized locations inside the cavity; register the image coordinate system with the sensor coordinate system so as to identify one or more voxels in the image at the visualized locations, and when the identified voxels have density values in the received image that do not correspond to the open space, to update the density values of the identified voxels to correspond to the open space.

Embodiments described herein provide various examples of monitoring adverse events in the background while displaying a higher-resolution surgical video on a lower-resolution display device. In one aspect, a process for detecting adverse events during a surgical procedure can begin by receiving a surgical video. The process then displays a first portion of the video images of the surgical video on a screen to assist a surgeon performing the surgical procedure. While displaying the first portion of the video images, the process uses a set of deep-learning models to monitor a second portion of the video images not being displayed on the screen, wherein each deep-learning model is constructed to detect a given adverse event among a set of adverse events. In response to detecting an adverse event in the second portion of the video images, the process notifies the surgeon of the detected adverse event to prompt an appropriate action.

Communication apparatus and devices for surgical robotic systems are described. The communication apparatus can include a user console in communication with a communication device having a surgical tool. The communication device can include a microphone to convert a sound input into an acoustic input signal. The communication device can transmit the acoustic input signal to the user console for reproduction as a sound output for a remote operator. The surgical tool can include an endoscope having several microphones mounted on a housing. The surgical tool can be a sterile barrier having a microphone and a drape. The microphone(s) of the surgical tools can face a surrounding environment such that a tableside staff is a source of the sound input that causes the sound output, and a surgeon and the tableside staff can communicate in a noisy environment. Other embodiments are also described and claimed.

A system (11) includes a medical probe (36) for insertion into a cavity of an organ, which includes a position and direction sensor (60) and a camera (45), both operating in a sensor coordinate system (62). The system further includes a processor (44) configured to: receive, from an imaging system (21) operating in an image coordinate system (28), a three-dimensional image of the cavity including open space and tissue; receive, from the medical probe, signals indicating positions and respective directions of the medical probe inside the cavity; receive, from the camera, respective visualized locations inside the cavity; register the image coordinate system with the sensor coordinate system so as to identify one or more voxels in the image at the visualized locations, and when the identified voxels have density values in the received image that do not correspond to the open space, to update the density values of the identified voxels to correspond to the open space.

A medical system and an operating method of a medical system are disclosed. The medical system can operate an endoscope to perform an operation task with respect to a site of interest of an organ of concern in response to input from an operator. The medical system includes a controller that can record a model of the organ of concern generated during preoperative planning, and record the names of a plurality of sites in the organ of concern, and corresponding regions or points in the model with respect to a first coordinate system. The controller receives the name of the site of interest, and the operation task; associates the first coordinate system of the model with a second coordinate system of a display space displayed by a display image; and operates the endoscope with respect to the site of interest on the basis of the operation task.

Systems and methods for safe operation of a device include a control system having a memory and a processor coupled to the memory. The processor is configured to determine, using one or more sensors, whether an operator is in contact with an input control of a console coupled to the system. The processor is further configured to affect commanded motion of the device using the input control in response to the determination of operator contact with the input control. In some embodiments, affecting the commanded motion includes limiting commanded motion of the device using the input control in response to determining the operator is not in contact with the input control. In some embodiments, the limited commanded motion is an insertion motion of the elongate device, a retraction motion of the elongate device, or a steering motion of a distal end of the elongate device.