A method of classifying a polyp captured in a tissue image of an in vivo tissue area is disclosed. The method includes a polyp during a colonoscopy procedure, analyzing, by a trained machine learned model, the tissue image, wherein the trained machine learned model is trained to identify classification characteristics of a polyp based on two or more visual characteristics, and generating a classification prediction of the tissue image based on the two or more visual characteristics including a basis of the classification prediction.

An image processing system includes a processor configured to acquire, as a processing target image, an in-vivo image, operate in accordance with a trained model, and output a recognition result representing a result of recognition of a region of interest in the processing target image. The trained model is trained by having undergone pre-training using a first image group including images captured in a first observation method, and having undergone, after the pre-training, fine-tuning that uses a second image group including images captured in a second observation method, as well as that uses ground truth regarding the region of interest included in the second image group. The first observation method is an observation method using normal light as illumination light, and the second observation method is an observation method using special light as the illumination light or an observation method in which a pigment has been dispersed onto the subject.

Implementations include herein are visual navigation strategies and systems for lumen center tracking comprising a high-level state machine for gross (i.e., left/right/center) region prediction and curvature estimation and multiple state-dependent controllers for center tracking, wall-avoidance and curve following. This structure allows a navigation system to navigate even under the presence of significant occlusion that occurs during turn navigation and to robustly recover from mistakes and disturbances that may occur while attempting to track the lumen center. This system comprises a high-level state machine for gross region prediction, a turn estimator for anticipating sharp turns, and several lower level controllers for heading adjustment.

The application relates to devices and methods for analysing a colonoscopy video or a portion thereof, and for assessing the severity of ulcerative colitis in a subject by analysing a colonoscopy video obtained from the subject. Analysing a colonoscopy video comprises using a first deep neural network classifier to classify image data from the subject colonoscopy video or portion thereof into at least a first severity class (more severe endoscopic lesions) and a second severity class (less severe endoscopic lesions), wherein the first deep neural network has been trained at least in part in a weakly supervised manner using training image data from a plurality of training colonoscopy videos, the training image data comprising multiple sets of consecutive frames from the plurality of training colonoscopy videos, wherein frames in a set have the same severity class label. Devices and methods for providing a tool for analysing colonoscopy videos are also described.

The application relates to devices and methods for analysing a colonoscopy video or a portion thereof, and for assessing the severity of ulcerative colitis in a subject by analysing a colonoscopy video obtained from the subject. Analysing a colonoscopy video comprises using a first deep neural network classifier to classify image data from the subject colonoscopy video or portion thereof into at least a first severity class (more severe endoscopic lesions) and a second severity class (less severe endoscopic lesions), wherein the first deep neural network has been trained at least in part in a weakly supervised manner using training image data from a plurality of training colonoscopy videos, the training image data comprising multiple sets of consecutive frames from the plurality of training colonoscopy videos, wherein frames in a set have the same severity class label. Devices and methods for providing a tool for analysing colonoscopy videos are also described.

In an endoscope apparatus including a processor, the processor specifies the position of a specific region and sets a reference scale in a picked-up image that is obtained from the image pickup of a subject on which the specific region formed by auxiliary measurement light is formed. Then, the processor extracts a region of interest, determines a measurement portion, calculates a measured value obtained from the measurement of the measurement portion, on the basis of the reference scale, and generates a measured value marker using the measured value. Further, the processor creates a specific image in which the measured value marker is superimposed on the picked-up image.

A computer-implemented method is provided for detecting at least one feature of interest in images captured with an imaging device. The method includes receiving an ordered set of images from the captured images, the ordered set of images being temporally ordered and analyzing one or more subsets of the ordered set of images using a local spatio-temporal processing module, the local spatio-temporal processing module being configured to determine the presence of characteristics related to the at least one feature of interest in each image of each subset of images and to annotate the subset of images based on the determined characteristics in each image of each subset of images. The method further includes processing a set of feature vectors of the ordered set of images using a global spatio-temporal processing module, the global spatio-temporal processing module being configured to refine the determined characteristics associated with each subset of images, and calculating one or more values for each image using a timeseries analysis module, the numerical value being representative of the at least one feature of interest and calculated using the refined characteristics associated each subset of images and spatio-temporal information. Still further, the method may include generating a report, a data or electronic file, integration into another reporting system or electronic medical records, and/or generating an electronic display on the at least one feature of interest using the multiple values associated with each image of each subset of the ordered set of images.

A computer-implemented method is provided for detecting at least one feature of interest in images captured with an imaging device. The method includes receiving an ordered set of images from the captured images, the ordered set of images being temporally ordered and analyzing one or more subsets of the ordered set of images using a local spatio-temporal processing module, the local spatio-temporal processing module being configured to determine the presence of characteristics related to the at least one feature of interest in each image of each subset of images and to annotate the subset of images based on the determined characteristics in each image of each subset of images. The method further includes processing a set of feature vectors of the ordered set of images using a global spatio-temporal processing module, the global spatio-temporal processing module being configured to refine the determined characteristics associated with each subset of images, and calculating one or more values for each image using a timeseries analysis module, the numerical value being representative of the at least one feature of interest and calculated using the refined characteristics associated each subset of images and spatio-temporal information. Still further, the method may include generating a report, a data or electronic file, integration into another reporting system or electronic medical records, and/or generating an electronic display on the at least one feature of interest using the multiple values associated with each image of each subset of the ordered set of images.

Disclosed are various embodiments of systems and methods of deep learning for colorectal polyp screening and providing a prediction of neoplasticity of a polyp. A video of a colonoscopy procedure can be captured. Frames from the video or images associated with the colonoscopy procedure can be extracted. A model for classifying objects that appear in the frames or the images can be obtained. A classification can be determined for a polyp that appears in at least one of the frames or images based on applying the frames or images to an input layer of the model.

A computer aided assistance system for use in endoscopic colonoscopy procedures. The computer aided assistance system including: at least one videoendoscopic instrument configured to capture image data; a controller comprising hardware, the controller being connected with the at least one videoendoscopic instrument; and a display connected or integral with the controller, wherein the controller being configured to automatically select a treatment guideline based on a combination of both a size and a classification of a lesion shown in the image data and to display the selected treatment guideline on the display.