Artificial Intelligence Training with Multiple Pulsed X-ray Source-in-motion Tomosynthesis Imaging System
20220313176 · 2022-10-06
Inventors
- Jianqiang Liu (Campbell, CA, US)
- Manat Maolinbay (Gilroy, CA, US)
- Chwen-yuan Ku (San Jose, CA, US)
- Linbo Yang (Pleasanton, CA, US)
Cpc classification
A61B6/4405
HUMAN NECESSITIES
G06V10/12
PHYSICS
A61B6/4452
HUMAN NECESSITIES
G01N23/18
PHYSICS
G06V10/25
PHYSICS
G06V10/62
PHYSICS
A61B6/4208
HUMAN NECESSITIES
A61B6/4007
HUMAN NECESSITIES
A61B6/56
HUMAN NECESSITIES
A61B6/4476
HUMAN NECESSITIES
A61B6/541
HUMAN NECESSITIES
G06T11/005
PHYSICS
A61B6/4275
HUMAN NECESSITIES
G16H50/20
PHYSICS
A61B6/54
HUMAN NECESSITIES
G06T11/006
PHYSICS
A61B6/405
HUMAN NECESSITIES
A61B6/0407
HUMAN NECESSITIES
G16H10/60
PHYSICS
A61B6/4283
HUMAN NECESSITIES
A61B6/08
HUMAN NECESSITIES
International classification
A61B6/02
HUMAN NECESSITIES
A61B6/00
HUMAN NECESSITIES
G01N23/18
PHYSICS
Abstract
Disclosed are image recognition Artificial Intelligence (AI) training methods for multiple pulsed X-ray source-in-motion tomosynthesis imaging system. Image recognition AI training can be performed three ways: first, using existing acquired chest CT data set with known nodules to generate synthetic tomosynthesis Images, no X-ray radiation applied; second, taking X-ray raw images with anthropomorphic chest phantoms with simulated lung nodules, applying X-ray beam on phantom only; third, acquiring X-ray images using multiple pulsed source-in-motion tomosynthesis images from real patients with real known nodules and without nodules. An X-ray image recognition training network that is configured to receive X-ray training images, automatically determine whether the received images indicate a nodule or lesion condition. After training, image knowledge is updated and stored at knowledge database.
Claims
1. A method of artificial intelligence training with multiple pulsed X-ray source-in-motion tomosynthesis imaging system, the method comprising: selecting a set of acquired CT chest 3D image data from real patients with known nodules for artificial intelligence training; generating synthetic tomosynthesis raw image data using CT forward projection data with geometry parameters from a set of multiple pulsed X-ray source-in-motion tomosynthesis imaging system; generating synthetic tomosynthesis images from raw image data using backward projection reconstruction; transmitting to image recognition software package as image input; determining whether chest images indicate a nodule or lesion condition based on X-ray image knowledge; and storing output X-ray images to knowledge data base.
2. The method of claim 1, wherein the artificial intelligence training includes implementing at least one of a deep learning network or a convolutional neural network.
3. The method of claim 1, further comprising transmitting a feedback signal to the CT forward and backward projection process, in response to the determining whether the received X-ray tomosynthesis images indicate a nodule or lesion.
4. A method of artificial intelligence training with multiple pulsed X-ray source-in-motion tomosynthesis imaging system, the method comprising: acquiring X-ray raw images from a set of multiple pulsed X-ray source-in-motion tomosynthesis imaging system using anthropomorphic chest phantoms with simulated lung nodules for artificial intelligence training; reconstructing to become tomosynthesis images with clinically standard view; transmitting to image recognition software package as input images; determining whether the tomosynthesis images indicate a nodule or lesion condition based on X-ray image knowledge; and storing output X-ray images in knowledge database.
5. The method of claim 4, wherein the artificial intelligence training includes implementing at least one of a deep learning network or a convolutional neural network.
6. The method of claim 4, further comprising: transmitting a feedback signal to the X-ray tomosynthesis imaging system, in response to the determining whether the received X-ray tomosynthesis images indicate a nodule or lesion condition.
7. The method of claim 4, further comprising: activating a feedback element in the X-ray tomosynthesis imaging system, based on the feedback signal, to provide a feedback effect to a user of the X-ray tomosynthesis imaging system.
8. The method of claim 4, further comprising: transmitting a feedback signal to the X-ray tomosynthesis imaging system, in response to the determining whether the received X-ray tomosynthesis images indicate a nodule or lesion.
9. An apparatus, comprising: an set of multiple pulsed X-ray source-in-motion tomosynthesis imaging system to acquire X-ray images of a human patient or an object for artificial intelligence training; and an set of X-ray image recognition software package configured to: receive the acquired X-ray tomosynthesis images; automatically determine whether the received X-ray images indicate a nodule or lesion in a human body or a flaw in an object.
10. The apparatus of claim 9, wherein the artificial intelligence training includes implementing at least one of a deep learning network or a convolutional neural network.
11. The apparatus of claim 9, the X-ray imaging system includes a graphical user interface (GUI) operable to receive a selection of a plurality of X-ray tomosynthesis images with clinically standard views.
12. The apparatus of claim 9, wherein the X-ray image recognition software package is operable to automatically determine whether the received X-ray tomosynthesis images indicate a nodule or lesion condition.
13. The apparatus of claim 9, wherein the X-ray image recognition software package is further configured to provide a feedback signal to the X-ray tomosynthesis imaging system, in response to determining whether the received X-ray tomosynthesis images indicate a nodule or lesion condition.
14. The apparatus of claim 9, the X-ray tomosynthesis imaging system including a feedback element, the X-ray tomosynthesis imaging system being configured to activate the feedback element, based on the feedback signal, to provide a feedback effect to a user of the the X-ray tomosynthesis imaging system.
15. The apparatus of claim 9, comprising: a primary motor stage moving freely on an arc rail with a predetermined shape; a primary motor that engages with said primary motor stage and controls a speed of the primary motor stage; a plurality of X-ray sources each mounted at the primary motor stage; a supporting frame structure that provides housing for the primary motor stage; and an X-ray flat panel detector to receive X-ray and transmit X-ray imaging data.
16. The apparatus of claim 9, comprising a pair of electrical deflection plates on each X-ray tube of X-ray source.
17. The apparatus of claim 9, comprising one or a pair of magnetic deflection coil yoke on each of X-ray tube of X-ray source.
18. The apparatus of claim 9, wherein one or more of the X-ray sources is activated using a predetermined scheme and an initial spatial position of the primary motor stage is adjustable by software.
19. The apparatus of claim 9, comprising code for: selecting a set of acquired CT chest 3D image data from real patients with known nodules for artificial intelligence training; generating synthetic tomosynthesis raw image data using CT forward projection data with multiple pulsed X-ray source-in-motion tomosynthesis geometry; generating synthetic tomosynthesis images from raw image data using backward projection reconstruction; transmitting to image recognition software package as image input; determining whether chest images indicate a nodule or lesion condition based on X-ray image knowledge; and storing output X-ray images to knowledge data base.
20. The apparatus of claim 9, comprising code for: scanning an object with an X-ray full view using multiple X-ray source-in-motion tomosynthesis imaging system; locating region of interest using artificial intelligence after an initial scan; determining a dynamic collimation map for each of X-ray tubes; applying a dynamic collimation map to motion control of the X-ray tubes; and re-scanning the object using a multi-leaf dynamic collimator with X-ray beam only limited on the region of interest.
Description
BRIEF DESCRIPTION
[0014] This application will become more fully understood from the following detailed description, taken in conjunction with the accompanying figures, wherein like reference numerals refer to like elements in which:
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
DETAILED DESCRIPTION
[0022] In the following paragraphs, the present invention will be described in detail by way of example with reference to the attached drawings. Throughout this description, the preferred embodiment and examples shown should be considered exemplars rather than limitations on the present invention. As used herein, the “present invention” refers to any one of the embodiments of the invention described herein and any equivalents. Furthermore, reference to various features of the “present invention” throughout this document does not mean that all claimed embodiments or methods must include the referenced features.
[0023] However, this invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided so that this disclosure will be thorough and complete and fully convey the invention's scope to those of ordinary skill in the art. Moreover, all statements herein reciting embodiments of the invention and specific examples thereof are intended to encompass structural and functional equivalents. Additionally, it is intended that such equivalents include both currently known equivalents and equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure).
[0024]
[0025] One embodiment of a tomosynthesis imaging system has four significant parts: digital radiography system, 3D slice projection equipment, multi pulsed source in motion control systems and image recognition software package. The 3D scanner uses an industrial computer for data processing, network devices for signal transfer, computers with sufficient storage for data analysis, and high-performance graphic cards to display the processed data. The imaging system can be connected to a hospital network and may transmit image files to the central hospital network. The system can transmit patient information and diagnosis result to the doctor's office. The system can reduce X-ray dose and speed up the diagnostic process through the use of multiple pulsed sources that can achieve seconds shot and thus reduces radiation dosage while decreasing scan time.
[0026]
[0027] The patient 7 in a diagnostic radiology procedure room lies on an examination table. The patient is undergoing imaging by a multiple pulsed X-ray source-in-motion tomosynthesis imaging system 1. The tomosynthesis imaging system may include external supports that position the tomosynthesis imaging system over the patient and support a gantry.
[0028]
[0029]
[0030] X-ray tomosynthesis imaging system typically produces a set of three dimensional 3D X-ray images each X-ray image representing a thin slice of an organ or other tissue of a patient. X-ray tomosynthesis imaging system often acquires additional views including anterior, posterior, and lateral images. The set of 3D X-ray images may be displayed in order or combined into a single composite view as the viewing direction rotates around the patient. For example, lung cancer screening requires capturing X-ray images that show adequate visualization of at least one complete lobe of a patient's lung. The determination of clinical acceptability can vary—different systems exist, whether it is based on anatomical structure or scanning the image in a pattern.
[0031] The X-ray detector 8 has an array of X-ray sensitive elements that sense X-rays and provide information that is indicative of a location of the region of interest. The information may be digitized or otherwise processed to determine this location. A storage device stores data representative of images generated by a tomosynthesis imaging system. This stored data includes image information acquired by tomosynthesis imaging and corresponding slice location data that indicates the position of each acquired image relative to other images in the dataset. These data are used for training artificial intelligence algorithms such as neural networks. The artificial intelligence system consists of at least one processing element coupled to a memory. The processing element may be configured to receive input data from at least one user through a user interface. Input data may include medical X-ray images and or images that are representative of the nodule or tumor detected in the patient's lung or chest. In addition the processing element may be configured to perform the functions described herein. For example, it may be configured to compare an image to reference images or predetermined diagnostic criteria. A suitable processing element may be provided by a general purpose computer executing software, hardware or a combination thereof.
[0032] To increase speed and accuracy in identifying potential medical issues from a patient scan, artificial intelligence (AI) is applied. To improve performance and reduce cost, the preferred embodiment applies AI as a diagnostics tool to perform data acquisition and make diagnostics decisions. AI must be trained first before AI can be used as a decision-maker in multiple pulsed source-in-motion tomosynthesis imaging systems. An AI program has to be trained first. AI software uses virtual images created by reconstruction from raw data from tomosynthesis imaging system or anthropomorphic chest phantoms with stimulated lung nodules, including GroundGlass Opacity (GGO) nodules and Chronic Obstructive Pulmonary Disease (COPD) nodules, to get training data sets. Anthropomorphic phantoms are made of materials with similar tissue characteristics to normal biological organisms. Due to their limited availability and likeness to actual patients, anthropomorphic phantoms can be used for various tasks.
[0033] Artificial intelligence (AI) or machine learning model is a mathematical algorithm that is trained to come to the same result or prediction that a human expert would when provided the same information. Deep learning neural networks, or artificial neural networks, attempts to mimic the human brain through a combination of data inputs, weights, and bias. These elements work together to accurately recognize, classify, and describe objects within the data. Deep learning is a subset of machine learning, which is itself a subset of artificial intelligence (AI). Deep neural network represents the type of machine learning when the system uses many layers of nodes to derive high-level functions from input information. The “deep” in deep learning refers to the many layers the neural network accumulates over time, with performance improving as the network gets deeper. A Convolutional Neural Network is a deep learning algorithm which can take in an input X-ray tomosynthesis image, assign importance by learnable weights and biases to various aspects or objects in the X-ray tomosynthesis image and be able to differentiate one from the other. Convolutional Neural Networks (CNNs) are one of the most popular neural network architectures. They are extremely useful at X-ray tomosynthesis image processing. Machine learning is about computers being able to think and act with less human intervention; deep learning is about computers learning to think using structures modeled on the human brain. Deep learning can analyze X-ray tomosynthesis mages in ways machine learning can't easily do.
[0034] With the trained AI software that can process X-ray tomosynthesis images in order to detect cancerous lesions or any other kind of lesions, radiologists reviews of X-ray scan images can be reduced significantly to reduce doctor workload. Tomosynthesis imaging with AI recognition of results will save time and cost especially for those patients who are too old or sick to undergo CT scanning. To train the AI system, three methods are disclosed in
[0035] As method ONE of AI training,
[0036] The geometry of a typical multiple pulsed X-ray source-in-motion tomosynthesis imaging system includes X-ray source to detector distance, X-ray source to object distance, X-ray scan sweeping angle, X-ray flat panel detector size and incremental angles etc. and the cone-beam projection geometry can be identified in
[0037] Generating tomosynthesis images using CT forward project data and backward projection is based on the fact that similar tissue attenuation can be used to get tomosynthesis images. In one method, the process uses forward project with thickness correction for creating CT scan image sets and back projection reconstruction to reconstruct those data sets into a tomosynthesis image format. One advantage of this method is that the tomosynthesis images are created with accurate knowledge of X-ray doses due to the high accuracy of forward projection imaging and another advantage is that the speed of tomosynthesis imaging is relatively fast. The training data set is sent to Intelligence (AI) image recognition training network as input. The training output is then stored as image knowledge database. In method one, there are no actual X-ray beam activities at the tomosynthesis imaging system.
[0038] The process of creating virtual patients from a CT image data set is detailed next. Registration of CT images, and 3D phantoms CT images can be registered to the anthropomorphic phantoms using point to point matching based on a small number of anatomical features in one embodiment. For example, edges on the phantom to help register the CT images to the reference volume typically four or five points per patient would be sufficient for registration. However more points can be used for improved accuracy. This process should also work with deformable templates instead of fixed landmarks. The output of this step will be a group of patient-specific target volumes corresponding to each patient's CT image data set. There will be a forward projection data set as well as a backward projection data set for each target volume. These target volumes are essentially stand-ins for each patient's CT image data set representing the same geometry but with the materials characterized by the different levels of voxel density that result from the CT scanning process. Material values may be estimated for all the voxels in the target volumes. It may be desirable to smoothen the target volumes to obtain accurate estimates of the material values at the interface between bone and soft tissue.
[0039] As method TWO of AI training,
[0040] As method THREE of AI training,
[0041] An external exposure control unit may be configured to receive data from the training network and adjust the radiation dose to a level that provides optimal development of the AI-based X-ray image recognition training network. The external exposure control unit may be any unit or combination of units that are capable of performing the desired adjustment of the radiation dose. As an example, the external exposure control unit may be an integral part of the imaging system or it may be a separate unit not shown connected to the imaging system. By way of a wired or wireless connection, an embodiment of the multiple pulsed source-in-motion tomosynthesis imaging system includes an artificial intelligence based X-ray image recognition training network that is configured to receive X-ray training images and to develop X-ray image knowledge base. On the received X-ray training images, an X-ray imaging system acquires X-ray images from a patient and the tomosynthesis imaging system includes an X-ray image recognition software package. The X-ray image recognition software package is configured to receive the X-ray image knowledge and receive the acquired X-ray images from the X-ray imaging system. Based on the X-ray image knowledge it determines whether the clinically standard views indicate normal function. The received X-ray images are transmitted to the X-ray image recognition training network for further training and development of updated X-ray image knowledge base.
[0042] A single tomosynthesis imaging system using one or more methods can receive input on the developed X-ray image knowledge. The input information can include any information about the knowledge such as but not limited to training knowledge obtained from images that have been analyzed by the training network. In this way, an X-ray image recognition software package in cooperation with the acquired X-ray image knowledge may determine whether images with the clinically standard views indicate lung normal healthy condition. In some embodiments, an X-ray data information system includes an X-ray image recognition training network that is configured to receive X-ray training images and to develop X-ray image knowledge based on the received X-ray training images.
[0043] The tomosynthesis imaging system is used for multiple pulsed source-in-motion X-ray image acquisition of a patient. The acquired images include projection images where each projection image represents an X-ray based forward projection of the acquired images from a specific acquisition angle. A virtual phantom is generated by using the projection images as training data set to develop X-ray image knowledge and image recognition neural network model. The developed image recognition neural network model can be used to reconstruct tomosynthesis images optionally. An artificial intelligence software package receives the generated tomosynthesis images and acquires updated X-ray image knowledge to generate further updated tomosynthesis images optionally. The generated tomosynthesis images are transmitted to the artificial intelligence software package to update the image recognition neural network model using the image recognition neural network model and or the artificial intelligence software package. an X-ray technician can select appropriate acquisition angles to provide a chosen tomosynthesis image of the patient that indicates a clinically standard view of the patient.
[0044] With the validated training data, the AI system can be trained. During operation, patient data are input to a radiography X-ray acquisition system. The system is configured to acquire at least one X-ray image from a patient in at least one projection plane. Wherein the X-ray image is at least one of forward projected and or backward projected and or reconstructed from an at least one image volume acquired by an X-ray imaging device. A tomosynthesis image reconstruction module is configured to reconstruct an X-ray image of an organ or other tissue from the at least one X-ray image of the patient. An X-ray image analysis module is configured to determine a distribution of nodules in the lung or other tissue. An artificial intelligence module is configured to create X-ray image knowledge based on the determined distribution of nodules. A software package running on a computer system includes an artificial intelligence AI training network that is configured to receive X-ray training images and to develop X-ray image knowledge based on the received X-ray training image. The software package also includes an X-ray image recognition software package. The X-ray image recognition software package is configured to receive the X-ray image knowledge receive the acquired X-ray images from the X-ray imaging device and based on the X-ray image knowledge determine whether the clinically standard views indicate normal function.
[0045] The resulting system can be used to speed up diagnosis. When capturing lung cancer screening X-ray images, a radiation technician conventionally acquires images that show the entirety of a patient's left lung, for example. A radiologist will then review the X-ray images and if necessary have the X-ray technician develop additional X-ray images that show the whole of the patient's right lung. Since this process is time consuming, it can add to a significant delay in diagnosing lung cancer. An appropriate combination of X-ray images will indicate which portions of the patient's lungs are clear and free of any abnormalities such as nodules. The AI system trained with anthropomorphic chest phantoms also known as synthetic phantoms using the processes described above can improve the operation of the diagnostic imaging equipment.
[0046] Methods for multiple pulsed source-in-motion tomosynthesis image recognition and learning algorithm are described next. A way for multi pulsed source in motion tomosynthesis image recognition using artificial intelligence includes receiving a plurality of X-ray images from an X-ray imaging device. The plurality of X-ray images are acquired while an X-ray source scans across an object as training data. The system provides training data to a multiple pulsed X-ray source-in-motion tomosynthesis image recognition system. Using artificial intelligence as a diagnostics tool for multiple pulsed X-ray source-in-motion tomosynthesis imaging system uses artificial intelligence (AI) to capture tomosynthesis images as much as possible and send them to AI module, which applies various algorithms such as single nodule detector, tumor cluster detection, and reject false detections, for example.
[0047] It can be seen from the above description that the embodiments of the present invention use multiple pulsed X-ray sources-in-motion tomosynthesis imaging systems 1, and artificial intelligence training networks for X-ray image recognition methods. The process is composed of two parts. One is to develop X-ray image knowledge based on the X-ray training images, the other, is to create such updated X-ray image knowledge by continuing the process of recognizing new and acquired X-ray images through updates to the X-ray image knowledge developed using the previous X-ray training images. By creating a reliable X-ray image knowledge base, the method can further increase the efficiency of subsequent diagnostics processes. Furthermore, the developed X-ray image knowledge is continuously updated as more X-ray images are received and can therefore adapt to changes in anatomy and physiology.
[0048] Although the invention is described above in terms of various exemplary embodiments and implementations, it should be understood that the various features, aspects, and functionality described in one or more of the individual embodiments are not limited in their applicability to the particular embodiment with which they are describe. But instead may be applied alone, or in various combinations, to one or more of the other embodiments of the invention. Whether or not such embodiments are described and whether or not such features are presented as being a part of an illustrated embodiment. Thus the breadth and scope of the present invention should not be limited by any of the above-described exemplary embodiments.