System and method for automated gross examination of tissues

Abstract

The various embodiments herein provide a system and method for automatic gross-examination of tissue samples. The apparatus is of cubicle shape comprising a bed where the specimen is placed, an ultrasound equipment mounted on top of cubicle box, a robotic arm mounted with a plurality of surgical blades, and a camera. The ultrasound technology is used to accurately understand the specimen, size and dimensions of a tumor that is studied. The robotic arm assisted surgical blades receive ultrasound output or camera output and accurately slice the specimen for further analysis. The information pertaining to gross-examination is stored in an external server connected to the apparatus and analyzed using artificial intelligence algorithms.

Claims

1. A system for automatic gross-examination of tissue samples, the system comprising: a stainless steel bed for placing a tissue sample; a piezoelectric glass lid mounted on the top of the stainless steel bed; a robotic arm, wherein the robotic arm comprises a plurality of surgical blades and wherein the surgical blades are configured to accurately slice a tumor specimen for analysis; a plurality of cameras configured to take videographs of the tumor specimen; an ultrasound cleaning mechanism, and wherein the ultrasound cleaning mechanism is provided to keep a plurality of instruments clean for a sequential processing; a server; and an ultrasound equipment mounted on top of the stainless steel bed for detecting a size and a dimension of the tumor specimen, and wherein the ultrasound equipment is also configured to automate a process of cutting the tumor specimen; wherein the server is configured to perform an analysis based on artificial intelligence and machine learning technologies and a result of the analysis is are communicated from the server, and wherein the server is configured to employ a plurality of classification and supervised learning algorithms or models and digital pathology for collaboration in the analysis of the samples, and wherein an artificial intelligence engine is provided in the server for classification, probabilistic modeling and advanced image analysis of gross-examination of tissues, and wherein the analytical models are specific and customized to each type of specimen being handled, and wherein a plurality of processes comprising automated image analysis, remote viewing, pathologists' collaboration, standard image segmentation and storage retrieval are included as a part of integrated applications.

2. The system according to claim 1, wherein the stainless steel bed is provided with a disposable cover for each specimen, and wherein the bed is configured to slide out to avoid accidental injury, and wherein a box with a modular design is provided to cover the bed and wherein the box is provided with lock-in mechanisms to ensure that all the parts are opened for enabling a manual cleaning process.

3. The system according to claim 1, wherein the robotic arm capable of moving in X-axis Y-axis and Z- axis is fixed to the top of the box, and wherein the blades are configured to extend out during a dissection process and are retracted back inside the arm when not in use, and wherein the camera is mounted on a 3D movable arm for accurate capturing of the image for the detailing of the specimen, and wherein the plurality of cameras is provided to capture the details of the specimen to be grossed.

4. The system according to claim 1, wherein the robotic arm, is configured to receive an output of the ultrasound equipment and to cut and slice the sample for analysis based on a command issued from the server after an analysis by the pathologist and analytics from the server, and wherein the robotic arm, is also configured for precise detection and dissection of specimen into cubes of preset sizes using the medical grade blades based on the output from the ultrasound equipement, and wherein the cubes are transferred with help of robotic arm into an automatic wax block for preparation, which are then subjected to analysis.

5. The system according to claim 1, wherein the plurality of cameras provides one or more images and the one or more images are analyzed through an image analysis of a gross-examination sample comprising the following steps: an analysis of the specimen is carried out by the ultrasound waves and the waves are converted into coordinates by a computer algorithm; an image is captured by a piezoelectric device with the help of ultrasound waves and the image is sent to the image analysis algorithm for further analysis; a total size of the tumor versus the total size of the specimen is identified from the sonic imaging and the location of the tumor is identified with respect to its boundaries from left to right; a size of the tumor as per general slicing is also captured and stored for further use and the lymph nodes are counted from the image analysis and are mapped to the co-ordinates and nodal dissection takes place; and, the specimen is sliced from left to right while enabling more slicing at the boundaries of the tumor and while slicing the tumor, the grittiness and the texture of the tumor are captured.

6. The system according to claim 5, wherein once the slicing is done, the robotic arm disengages and the tumor is held for further clinical purposes; and wherein the tumor is then dissected to obtain a block of tumor by the robotic arm as per the grossing principles; a predefined full report is generated with all the necessary information; and, the report and the block are sent for further clinical purposes.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The other objects, features and advantages will occur to those skilled in the art from the following description of the preferred embodiment and the accompanying drawings in which:

(2) FIG. 1 illustrates a perspective view of an apparatus for automatic gross-examination of tissue samples, according to one embodiment herein.

(3) FIG. 2 illustrates a flow diagram that provides the steps involved in a preliminary identification and recording information about a gross-examination sample, according to one embodiment herein.

(4) FIG. 3 illustrates a flow diagram that provides the steps involved in an image analysis of a gross-examination sample, according to one embodiment herein.

(5) FIG. 4 illustrates a flow diagram that provides the steps involved in generating an analysis report of a gross-examination sample after conducting an image analysis on the sample, according to one embodiment herein.

(6) FIG. 5 illustrates a system that enables texture and consistency analysis and reporting of a sample, according to one embodiment herein.

(7) FIG. 6 illustrates a system that enables the sensor-blade technology in the robotic arm in the apparatus, according to one embodiment herein.

(8) FIG. 7 illustrates a system that enables Lymph-node plucking with present apparatus, according to one embodiment herein.

(9) FIG. 8 illustrates a system that enables a development of predicting modeling tool for malignant potential based on the final HPE to integrate for a routine imaging with artificial intelligence, according to one embodiment herein.

(10) FIG. 9 illustrates a system for transferring information from an ultrasonic generator to correlating software for pathology image, according to one embodiment herein.

(11) Although the specific features herein are shown in some drawings and not in others. This is done for convenience only as each feature may be combined with any or all of the other features in accordance Faith the embodiments herein.

DETAILED DESCRIPTION OF THE INVENTION

(12) In the following detailed description, a reference is made to the accompanying drawings that form a part hereof, and in which the specific embodiments that may be practiced is shown by way of illustration. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments and it is to be understood that other changes may be made without departing from the scope of the embodiments. The following detailed description is therefore not to be taken in a limiting sense.

(13) The various embodiments herein provide system and method for an automated apparatus for the gross examination of tissue sample.

(14) According to one embodiment herein, an automated apparatus for the gross examination of tissue sample is provided. The apparatus is of cubicle or rectangular shape or other suitable shape comprising a bed where the specimen is placed, an ultrasound equipment mounted under the bed, camera mounted on a 3D movable arm for accurately capturing the image for the detailing of the specimen and a robotic arm capable of moving in X-axis, Y-axis and Z-axis, a plurality of surgical blades housed in the cubicle box and mounted in the robotic arm. A 3D movable camera captures the details of the specimen to be grossed, similar to naked eye but with objectification. The ultrasound technology is used to accurately understand the specimen, size and dimensions of a tumor that is studied. The ultrasound equipment also assists in the automation of the process of cutting the specimen. The surgical blades receive the output from the ultrasound equipment or other imaging devices to accurately slice the specimen for further analysis.

(15) According to one embodiment herein, a precision instrument, which is linked with an Artificial Intelligence engine for classification, probabilistic modeling and advanced image analysis of gross-examination of tissues, is provided. All these are analytical models that are specific and customized to each type of specimen being handled. Processes such as automated image analysis, remote viewing, pathologists' collaboration, standard image segmentation, storage retrieval etc. are included in the system as a part of the integrated applications.

(16) According to one embodiment herein, an apparatus for enabling automated gross-examination of tissues is provided. The apparatus comprises stainless steel bed for placing the tissue sample, cubes of multiple sizes to act as the base for the bed, mountable ultrasound equipment and retractable robotic blades are provided. The piezo-electric compatible bed fixed on stainless steel based motor enabled plate is provided with a disposable cover for each specimen. The Bed is configured to slide out and when bed is outside, to avoid accidental injury. The blades inside robotic are retracted inside. Only on completion of ultrasound and confirmation by pathologist the blades are extended out of the robotic arm.

(17) According to one embodiment herein, the apparatus is provided with a built-in ultrasound cleaning mechanism to keep the instruments clean for a sequential processing.

(18) According to one embodiment herein, the box is formed or fabricated in three sizes of 30/60/90 sqcm with bed arranged at a ⅔rd height from the base. The bed is provided on top with a piezoelectric glass mounted with the ultrasound equipment. The bed is connected to a motor for rotating the bed for assisting the dissection process of grossing-in and imaging operations. The box has a modular design with lock-in mechanisms to ensure that all the pans are opened and cleaned manually by an operator or a lab technician with a minimal training. The ultrasound equipment is arranged or configured to cover an entire surface area on top of the bed. Alternatively the ultrasound equipment is arranged as an array for covering specimen per sqmm.

(19) According to one embodiment herein, the robotic arm is mounted with medical grade surgical blades (like scalpel) with a retractable mechanism for safety. The 3 blades are configured to cover X-Y-Z axes As soon as the specimen is sliced, the blades are cleaned with an ultrasound mechanism. The equipment is provided with an automatic cleaning facility arranged inside and is cleaned later manually. An output of ultrasound is input to robotic arm, based on the command issued from the server after the analysis by the pathologist and analytics from server to cut and slice the sample for analysis. The output of ultrasound is input to robotic arm, for precise detection and dissection of specimen into cubes of preset sizes using the medical grade blades. The cubes are transferred with help of robotic arm into automatic wax block for preparation, which are then subjected to analysis.

(20) According to one embodiment herein, any analytics on AI and Machine learning is carried out in the server and the results are communicated the apparatus from the server. A plurality of classification (supervised learning) algorithms/models, and Digital pathology for collaboration are employed in the analysis of the samples.

(21) According to one embodiment herein, the apparatus is configured to perform automatic process of grossing of tissues by integrating the technologies of data mining, analytics, robotics, ultrasound and mobile computing.

(22) According to one embodiment herein, the apparatus is configured to storage data/information related to gross-examination of tissue samples on an external server.

(23) According to an embodiment herein, a system is configured to evaluate the grossed specimen by superimposing the images acquired from Ultrasound/X ray/MRI or approved modalities by medical bodies with pictorial images of the same taken by conventional/digital imaging based on an AI based correlation.

(24) According to an embodiment herein, a system is configured to standardize the consistency of the tissue on a defined scale for uniform reporting by “robotic arm—with specified material, using defined force and proportionate wedge angle of knife/dissecting instrument”.

(25) According to an embodiment herein, a system is configured to perform an accurate pathological dissection to obtain the samples of ideal and relevant areas for processing with the help of image guided robotic navigation of multidimensional blades/instruments, especially for the margins and depth of suspected tissues.

(26) According to an embodiment herein, a system is configured to carry out an accurate Lymphnodal dissection (plucking rather than cutting) with the help of imaging for standardized yield and thereby preserving the specimen architecture.

(27) According to an embodiment herein, a system is linked with an Artificial Intelligence engine for classification, probabilistic modeling and advanced image analysis of gross-examination of tissues.

(28) According to an embodiment herein, the system is further configured to create analytical models that are specific and customized to each type of specimen being handled.

(29) According to an embodiment herein, the system is further configured to perform accurate pathological dissection comprising processes such as automated image analysis, remote viewing, pathologists' collaboration and feedback loop, standard image segmentation, storage retrieval etc. which are included in the system as a part of the integrated applications.

(30) According to an embodiment herein, a method further comprises evaluating the grossed specimen by superimposing the images acquired from Ultrasound/X ray/MRI or approved modalities by medical bodies with pictorial images of the same taken by conventional/digital imaging based on a AI based correlation.

(31) According to an embodiment herein, the method further comprises the steps of standardizing the consistency of tie tissue on a defined scale for uniform reporting by “robotic arm—with specified material, using defined force and proportionate wedge angle of knife/dissecting instrument”.

(32) According to an embodiment herein, the method further comprises the steps of performing an accurate pathological dissection to obtain the samples of ideal and relevant areas fur processing with the help of image guided robotic navigation of multidimensional blades/instruments, especially for the margins and depth of suspected tissues.

(33) According to an embodiment herein, the method further comprises the steps of performing an accurate Lymphnodal dissection (plucking rather than cutting) with the help of imaging for standardized yield and thereby preserving the specimen architecture.

(34) According to an embodiment herein, the method further comprises the steps for performing a classification, a probabilistic modeling and an advanced image analysis of gross-examination of tissues by using an Artificial Intelligence engine.

(35) According to an embodiment herein, the method further comprises the steps of creating the analytical models that are specific and customized to each type of specimen being handled.

(36) According to an embodiment herein, the method further comprises the steps of performing an accurate pathological dissection comprising processes such as automated image analysis, remote viewing, pathologists' collaboration and feedback loop, standard image segmentation, storage retrieval etc. which are included in the system as a part of the integrated applications.

(37) FIG. 1 illustrates a perspective view of an apparatus for automatic gross-examination of tissue samples, according to an embodiment herein. With respect to FIG. 1, an automated apparatus for the gross examination of tissue sample is provided. The apparatus is of cubicle shaped box comprising a bed 110 where the specimen 108 is placed. An ultrasound equipment is mounted on top of the bed 110 and a robotic arm 102 capable of moving in X-axis, Y-axis and Z-axis is fixed to the top of the box. A plurality of surgical blades is mounted in the robotic arm 102. The blades are configured to extend out during a dissection process and are retracted back inside the arm 102 when not in use. The ultrasound technology is used to accurately understand/detect the specimen, size and dimensions of a tumor that is studied. The ultrasound equipment also automates the process of cutting the specimen 108. The surgical blades receive ultrasound output and accurately slice the specimen for further analysis. A camera 104a, is mounted on 3 D movable arm for accurate capturing of the image for the detailing of the specimen. A plurality of 3D movable cameras 104,104a, 104b is provided to capture the details of the specimen to be grossed, similar to naked eye but with objectification.

(38) According to one embodiment herein, a precision instrument, which is linked with an Artificial Intelligence engine for classification, probabilistic modeling and advanced image analysis of gross-examination of tissues, is provided. All these are analytical models that are specific and customized to each type of specimen being handled. Processes such as automated image analysis, remote viewing, pathologists' collaboration, standard image segmentation, storage retrieval etc. are included in the system as a part of the integrated applications.

(39) According to one embodiment herein, the apparatus comprises stainless steel bed 110 for placing the tissue sample. The bed is mounted with a piezo electric glass 112 on top. A ultrasound equipment is mounted on the bed. The retractable robotic blades are provided. The Stainless steel bed 110 is provided with a disposable cover for each specimen. The Bed 110 is configured to slide out and when bed is outside, to avoid accidental injury. The blades are retracted inside robotic arm, when not in use. Only on completion of ultrasound and confirmation. by pathologist, the blades are extended out of the robotic arm.

(40) According to one embodiment herein, the apparatus is provided with a built-in ultrasound cleaning mechanism to keep the instruments clean for a sequential processing.

(41) According to one embodiment herein, the box is formed or fabricated in three sizes of 30/60/90 sqcm with bed arranged at a ⅔rd height from the base. The box has a modular design with lock-in mechanisms to ensure that all the parts are opened and cleaned manually by an operator or a lab technician with a minimal training. The ultrasound equipment is arranged or configured to cover an entire surface area on top of the bed. Alternatively the ultrasound equipment is arranged as an array for covering specimen per sqmm.

(42) According to one embodiment herein, the robotic arm is mounted 102 with medical grade surgical blades (like scalpel) with a retractable mechanism for safety. The 3 blades are configured to cover X-Y-Z axes As soon as the specimen is sliced, the blades are cleaned with an ultrasound mechanism. The equipment is provided with an automatic cleaning facility arranged inside and is cleaned later manually. An output of ultrasound is input to robotic arm, based on the command issued from the server after the analysis by the pathologist and analytics from server to cut and slice the sample for analysis. The output of ultrasound is input to robotic arm, for precise detection and dissection of specimen into cubes of preset sizes using the medical grade blades. The cubes are transferred with help of robotic arm into automatic wax block for preparation, which are then subjected to analysis.

(43) According to one embodiment herein, any analytics on AI and Machine learning is carried out in the server 106 and the results are communicated the apparatus from the server 106. A plurality of classification (supervised learning) algorithms/models, and Digital pathology for collaboration are employed in the analysis of the samples.

(44) FIG. 2 illustrates a flow diagram that provides the steps involved in a preliminary identification and recording information about a gross-examination sample, according to one embodiment herein. The method comprises the following steps: Identification. of the nomenclature and taxonomy of a specimen (201); Placement of the specimen is on the ultrasound bed laterally depending on the size of the specimen (202). The specimen is stabilized with a robotic arm and the measurements of the specimen are captured by ultrasound technique (203); The specimen is videographed and contour shape is recorded (204); and, Measurements are analyzed by pathologists; when the pathologists approve the measurements, the measurements and shape of the specimen are recorded in the database. When the pathologists do not approve the measurements, the pathologist modifies the measurements and the measurements and shape of the specimen are recorded in the database (205).

(45) FIG. 3 illustrates a flow diagram that provides the steps involved in an image analysis of a gross-examination sample, according to one embodiment herein. The method comprises the following steps: The analysis of the specimen is carried out by the ultrasound waves and the waves are converted into co-ordinates by a computer algorithm (301); An image is captured by a piezoelectric device with the help of ultrasound waves and the image is sent to the image analysis algorithm for further analysis (302); The total size of the tumor versus the total size of the specimen is identified from the sonic imaging and the location of the tumor is identified with respect to its boundaries from left to right (303); The size of the tumor as per general slicing is also captured and stored for further use and the lymph nodes are counted from the image analysis and are mapped to the co-ordinates and nodal dissection takes place (304); and, The specimen is sliced from left to right while enabling more slicing at the boundaries of the tumor and while slicing the tumor, the grittiness and the texture of the tumor are captured (305).

(46) FIG. 4 illustrates a flow diagram that provides the steps involved in generating an analysis report of a gross-examination sample after conducting an image analysis on the sample, according to one embodiment herein. The method comprises the following steps: Once the slicing is done the robotic arm disengages and the tumor is held for further clinical purposes (401); The tumor is then dissected to obtain a block of tumor by the robotic arm as per the grossing principles (402); A predefined full report is generated with all the necessary information (403); and, The report and the block are sent for further clinical purposes (404).

(47) FIG. 5 illustrates a system that enables texture and consistency analysis and reporting of a sample, according to one embodiment herein. The system comprises an automated/manual robotic arm with 3D control 501, a module with pressure/time/power gradient control coupled with movement measurement technology 502 and a module for measurement of resistance/movement traversed with outputs based on programmed calculator for consistency. The embodiment also comprises a module 504 with a standardized scale with a validated score system to objectively document the consistency and texture, that is reported automatically with a pathologist/a technical expert interface to minimize false negatives and errors. The embodiment also comprises an artificial intelligence module 505 that comprises: input capturing in terms of force vs. movement vs. time vs, texture and coupling with graded output for dissecting to robotic arm; data integration with image [Visual/optical] vs. Scan [electromagnetic/piezoelectric/texture/tensile and other properties; artificial intelligence based algorithm for the forward and backward integration; and, automated typing into the pre-formatted texting taking inputs specific to organ.

(48) FIG. 6 illustrates a system that enables the sensor-blade technology in the robotic arm in the apparatus, according to one embodiment herein. The system comprises a module with an intact tissue sliced and the scanner integrated with blade 601, a module with a feedback loop from the technical interface/historical control/machine learning controls the dissecting pressure and distance to be traversed 602, a module with a pressure too low that undercuts and be augmented by positive feedback loop 603, a module with a pressure too high that overcuts and be inhibited by negative feedback loop 604; and a module with an accurate dissection with texture/consistency oriented outputs, which are objective and quantifiable 605.

(49) FIG. 7 illustrates a system that enables Lymph-node plucking with the present apparatus, according to one embodiment herein. The system comprises a Lymph-node sample 701, a module with a 3D controlled human interface enabled arm having inputs from the Imaging and. Scanning integrated with pathologist inputs 702, a module with a plurality of outputs to a plucker/rotator blade [not slicing, which is unique] that plucks without damaging the surrounding tissues 703, a module with image based mapping of the spherico ovaoidal structures, having high probabilistic chances of being Lymph nodes and coordinates to be sent to the robotic arm 704 and a final feedback loop coupled with artificial intelligence makes the prediction better with machine learning and technical interface for better nodal yield 705. The plucking/circular cutting minimizes damage to surroundings of the sample.

(50) FIG. 8 illustrates a system that enables a development of predicting modeling tool for malignant potential based on the final HPE to integrate for a routine imaging with artificial intelligence, according to one embodiment herein. The system comprises a module with grossing results coupled with final HPE from the master database/computer, which are specific to the tissues and organ 801, a module with results of imaging/scanning from the database corresponding to the specimen 802, a module comprising artificial intelligence based algorithm far machine learning to predict the characters unique far malignant vs. benign tissues 803 and final software that predicts malignant potential at the scanning level itself in the live organisms/humans 804.

(51) FIG. 9 illustrates a system for transferring information from an ultrasonic generator to correlating software for pathology image, according to one embodiment herein. The system comprises a cloud module with data from USG and final pathology from automated grossing machine 901, a module for analysis of textures as measured, with resistivity index 902, a module for analysis of image textures from camera and ultrasound 903, a module to send for the artificial algorithm for correlation to HPE 904, a module to send for artificial intelligence platform for pattern recognition and validated outputs 905 and an image-pathological correlating software.

(52) The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such as specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modifications. However, all such modifications are deemed to be within the scope of the claims.

(53) The embodiments herein provide a system and method for an automated apparatus for the gross examination of tissue sample. The embodiments herein enable an accurate pathological dissection to obtain the samples of ideal and relevant areas for processing. The automated apparatus also increases accuracy and reduces false-positive and false-negative results.

(54) The automated apparatus helps a pathologist to navigate to accurate and relevant zones in the specimen. The output of present embodiment's analysis is fed to a robotic arm, which has three-dimensional blades for precise detection and dissection of the specimen to required sizes.

(55) The embodiments herein provide a system for high speed and automated Grossing-in of specimens, to reduce the turn-around time.

(56) The embodiments herein provide a system and method to enable better lymph node harvesting technology, which is an important event in the grossing in as the majority of the technicians are semi-skilled and under trained. The embodiments herein also assist the pathologists to enable better lymph node harvesting technology with the help of imaging and robotics techniques.

(57) The embodiments herein provide a system and method to provide reproducible results, with objective parameters and objectification of currently subjective issues with minimum human interface and maximum accuracy.

(58) The embodiments herein provides access to technologically qualified inputs to serve the remote areas, which largely depend on tele-pathology and where grossing in errors lead to major misdiagnosis.

(59) The embodiments herein provide a system to enable grossing of high volumes of specimens in limited time with limited resources.

(60) The embodiments herein provide a system and method to enable accurate measurement and to prevent cross contamination with help of automated and standardized procedures.

(61) The embodiments herein provide system and method to enable digital documentation of the grossing process for review and corrections.

(62) The embodiments herein provide system and method to enable better and uniform reporting of grossing-in processes and results through artificial intelligence techniques.

(63) The embodiments herein reduce the risk accidental infection to pathologist/technicians during grossing. The embodiments herein further reduce skin and eye infections due to exposure to formalin.

(64) The apparatus herein is configured to significantly increase accuracy in slicing the specimen and preserve the integrity of gross specimen. The apparatus is configured to avoid a lot of problems in grossing like wrong depth during splicing, which are errors due inability to understand resistivity and hardness of the specimens and increase the ability to reach deep areas which are otherwise difficult to reach such as areas close to vessels, deep lungs, intramural tumors etc.

(65) It is also to be understood that the following claims are intended to cover all of the generic and specific features of the embodiments described herein and all the statements of the scope of the embodiments which as a matter of language might be said to fall there between.