Abstract
Digital breast tomosynthesis represents an enhanced type of mammogram for detecting breast cancer. In this disclosure, data from digital breast tomosynthesis is reconstructed into a volumetric database with each voxel having a (x, y, z) coordinate and viewed in true 3D using geo-registered head display unit and geo-registered tools for overall enhanced diagnosis. The breast is imaged under various configurations and the internal architecture of an anatomic feature three-dimensionally analyzed. Additional dataset creation and three-dimensional imaging techniques are disclosed.
Claims
1. A method comprising: using a digital breast tomosynthesis examination wherein said digital breast tomosynthesis examination comprises a set of focal plane images; generating a 3D dataset from said digital breast tomosynthesis examination wherein said 3D dataset is different from said set of focal plane images, wherein said 3D dataset is comprised of voxels, wherein each voxel in said 3D dataset has a cubic shape, an x,y,z coordinate and a data unit, and wherein for each focal plane image in said set of focal plane images: performing an analysis of data units to classify a plurality of anatomic features on each focal plane image; and for each anatomic feature of said plurality of anatomic features, assigning an x,y,z coordinate and a data unit to a voxel to said 3D dataset wherein assigning an x,y coordinate of said x,y,z coordinate is performed using a first method, wherein assigning a z-coordinate of said x,y,z coordinate is performed using a second method, wherein said second method is different from said first method, wherein said first method is based on a location of each anatomic feature of said plurality of anatomic features within a focal plane image, and wherein said second method is based on a height of said focal plane image; and providing said 3D dataset for analysis of said plurality of anatomic features.
2. The method of claim 1 further comprising generating a 3D volume of microcalcifications in multiple breast configurations.
3. The method of claim 1 further comprising generating a 3D volume of microcalcifications at varying thresholds.
4. The method of claim 1 further comprising generating a 3D volume of microcalcifications displayed to denote varying certainty levels.
5. The method of claim 1 further comprising generating a 3D volume of microcalcifications at varying thickness of focal plane images.
6. The method of claim 1 further comprising analysis of a 3D volume of microcalcifications by at least one of the group comprising performing artificial intelligence, performing comparison over multiple time points, and performing comparison over multiple configurations.
7. The method of claim 1 further comprising viewing of a 3D volume of microcalcifications with an Extended Reality display.
8. Aft computer system comprising: a memory; a processor; a communications interface; an interconnection mechanism coupling the memory, the processor and the communications interface; and wherein the memory is encoded with an application that when performed on the processor provides a process for processing information, the process causing the computer system to perform the operations of: using a digital breast tomosynthesis examination wherein said digital breast tomosynthesis examination comprises a set of focal plane images; generating a 3D dataset from said digital breast tomosynthesis examination wherein said 3D dataset is different from said set of focal plane images, wherein said 3D dataset is comprised of voxels, wherein each voxel in said 3D dataset has a cubic shape, an x,y,z coordinate and a data unit, and wherein for each focal plane image in said set of focal plane images: performing an analysis of data units to classify a plurality of anatomic features on each focal plane image; and for each anatomic feature of said plurality of anatomic features, assigning an x,y,z coordinate and a data unit to a voxel to said 3D dataset wherein assigning an x,y coordinate of said x,y,z coordinate is performed using a first method, wherein assigning a z-coordinate of said x,y,z coordinate is performed using a second method, wherein said second method is different from said first method, wherein said first method is based on a location of each anatomic feature of said plurality of anatomic features within a focal plane image, and wherein said second method is based on a height of said focal plane image; and providing said 3D dataset for analysis of said plurality of anatomic features.
9. The computer system of claim 8 further comprising generating a 3D volume of microcalcifications in multiple breast configurations.
10. The computer system of claim 8 further comprising generating a 3D volume of microcalcifications at varying thresholds.
11. The computer system of claim 8 further comprising generating a 3D volume of microcalcifications displayed to denote varying certainty levels.
12. The computer system of claim 8 further comprising generating a 3D volume of microcalcifications at varying thickness of focal plane images.
13. The computer system of claim 8 further comprising analysis of a 3D volume of microcalcifications by at least one of the group comprising performing artificial intelligence, performing comparison over multiple time points, and performing comparison over multiple configurations.
14. The computer system of claim 8 further comprising viewing of a 3D volume of microcalcifications with an Extended Reality display.
15. A non-transitory computer readable medium having computer readable code thereon for medical imaging, the medium comprising instructions for: using a digital breast tomosynthesis examination wherein said digital breast tomosynthesis examination comprises a set of focal plane images; generating a 3D dataset from said digital breast tomosynthesis examination wherein said 3D dataset is different from said set of focal plane images, wherein said 3D dataset is comprised of voxels, wherein each voxel in said 3D dataset has a cubic shape, an x,y,z coordinate and a data unit, and wherein for each focal plane image in said set of focal plane images: performing an analysis of data units to classify a plurality of anatomic features on each focal plane image; and for each anatomic feature of said plurality of anatomic features, assigning an x,y,z coordinate and a data unit to a voxel to said 3D dataset wherein assigning an x,y coordinate of said x,y,z coordinate is performed using a first method, wherein assigning a z-coordinate of said x,y,z coordinate is performed using a second method, wherein said second method is different from said first method, wherein said first method is based on a location of each anatomic feature of said plurality of anatomic features within a focal plane image, and wherein said second method is based on a height of said focal plane image; and providing said 3D dataset for analysis of said plurality of anatomic features.
16. The non-transitory computer readable medium of claim 15 further comprising generating a 3D volume of microcalcifications in multiple breast configurations.
17. The non-transitory computer readable medium of claim 15 further comprising generating a 3D volume of microcalcifications at varying thresholds.
18. The non-transitory computer readable medium of claim 15 further comprising generating a 3D volume of microcalcifications displayed to denote varying certainty levels.
19. The non-transitory computer readable medium of claim 15 further comprising generating a 3D volume of microcalcifications at varying thickness of focal plane images.
20. The non-transitory computer readable medium of claim 15 further comprising analysis of a 3D volume of microcalcifications by at least one of the group comprising performing artificial intelligence, performing comparison over multiple time points, and performing comparison over multiple configurations.
21. The non-transitory computer readable medium of claim 15 further comprising viewing of a 3D volume of microcalcifications with an Extended Reality display.
Description
BRIEF DESCRIPTION OF FIGURES
(1) FIG. 1 is a flow diagram that illustrates a process of generating 3D volumetric dataset from a digital breast tomosynthesis dataset and using this dataset for enhanced viewing.
(2) FIG. 2 illustrates an apparatus for implementing the process of FIG. 1.
(3) FIG. 3A illustrates the breast with georegistration points.
(4) FIG. 3B illustrates the breast with georegistration points and a georegistration grid.
(5) FIG. 3C illustrates the breast with georegistration points, georegistration grid and the coordinate system.
(6) FIG. 4A illustrates an initial configuration of the X-ray detector, x-ray beam, breast, microcalcification within the breast, the detector and a first image.
(7) FIG. 4B illustrates movement of the X-ray detector and subsequent image.
(8) FIG. 5A illustrates the geometry of the X-ray tube, X-ray detector, compression paddle, breast and microcalcification.
(9) FIG. 5B illustrates the geometry of the microcalcification within the breast, the photon beams from the two X-ray tube positions, the X-ray detectors.
(10) FIG. 6A illustrates an artery, a vein, breast tissue, microcalcifications, and a 3D cursor.
(11) FIG. 6B illustrates filtering (i.e., subtraction) of the tissues external to the 3D cursor.
(12) FIG. 6C illustrates changing the transparency of the tissues within the 3D cursor.
(13) FIG. 6D illustrates filtering (i.e., subtraction) of all tissues both inside and external to the 3D cursor with the exception of the breast microcalcifications.
(14) FIG. 7A illustrates the initial viewing perspective.
(15) FIG. 7B illustrates changing the interocular distance and angular field of view.
(16) FIG. 7C illustrates the volume of interest (VOI) contained inside of the 3D cursor rotated 90 degrees.
(17) FIG. 7D illustrates changing the viewing perspectives by rotating it by 90 degrees.
(18) FIG. 8A illustrates the 3D cursor affixed to a geo-registered platform.
(19) FIG. 8B illustrates tilting of the geo-registered platform.
(20) FIG. 8C illustrates pointing to the microcalcifications inside of the 3D cursor with the geo-registered focal point pen.
(21) FIG. 9 illustrates a top down view of the mammographer's desk illustrating several of the tools with position and orientation tracking.
(22) FIG. 10 illustrates the conversion of a digital breast tomosynthesis dataset into a single voxelated dataset.
(23) FIG. 11A illustrates a low pressure of compression of the breast.
(24) FIG. 11B illustrates a high level of compression of the breast.
(25) FIG. 11C illustrates an initial configuration of a breast mass.
(26) FIG. 11D illustrates a subsequent configuration of a breast mass.
(27) FIG. 12A illustrates a low level of compression of the breast and a breast mass that is round in shape.
(28) FIG. 12B illustrates a high level of compression of the breast and the breast mass remains round in shape.
(29) FIG. 12C illustrates a low level of compression of the breast and a breast mass that is round in shape.
(30) FIG. 12D illustrates a high level of compression of the breast and the breast mass becomes flattened.
(31) FIG. 13 illustrates a digital breast tomosynthesis dataset performed with skin markers.
(32) FIG. 14 illustrates a flow diagram for generating a 3D volumetric dataset of microcalcifications from a digital breast tomosynthesis dataset.
(33) FIG. 15A illustrates a series of slices with a first threshold setting and a cluster of microcalcifications.
(34) FIG. 15B illustrates a series of slices with a second threshold setting and a cluster of microcalcifications.
(35) FIG. 15C illustrates a series of slices with a third threshold setting and a cluster of microcalcifications.
(36) FIG. 16 illustrates a flow diagram of viewing a 3D volumetric dataset of microcalcifications from a digital breast tomosynthesis dataset.
(37) FIG. 17A illustrates a cluster of microcalcifications at a first time point.
(38) FIG. 17B illustrates a cluster of microcalcifications at a second time point.
(39) FIG. 17C illustrates a cluster of microcalcifications at a third time point.
DETAILED DESCIPTION OF FIGURES
(40) FIG. 1 is a flow diagram that illustrates a process of generating 3D volumetric dataset from a tomosynthesis dataset and using this dataset for enhanced viewing. The flow diagrams do not depict the syntax of any particular programming language. Rather, the flow diagrams illustrate the functional information one of ordinary skill in the art requires to fabricate circuits or to generate computer software to perform the processing required in accordance with the present invention. It should be noted that many routine program elements, Such as initialization of loops and variables and the use of temporary variables are not shown. It will be appreciated by those of ordinary skill in the art that unless otherwise indicated herein, the particular sequence of steps described is illustrative only and can be varied without departing from the spirit of the invention. Thus, unless otherwise stated the steps described below are unordered meaning that, when possible, the steps can be performed in any convenient or desirable order. The first step 100 is to record precise geometry of the digital breast tomosynthesis equipment. The second step 101 is prior to commencing digital tomosynthesis exam, have the option to affix pin head size radiographically detectable markers on the surface of the patient's breast. The preferred option would be to have the mammography technologist place the markers, see U.S. patent application Ser. No. 16/509,592, IMPLANTABLE MARKERS TO AID SURGICAL OPERATIONS. The third step 102 is to perform digital breast tomosynthesis examination and collect data. Note that this can be performed under varying levels of compression of the breast. For example, a first 3D volume can be performed at a first level of compression of the breast. Then, a second 3D volume can be performed at a second level of compression of the breast. The fourth step 103 is to download DBT data into the 3D processing system along with associated meta data for particular DBT System including DBT system resolution, arc degrees, number of images taken. The fifth step 104 is to create a grid and associated X, Y, Z coordinate system which is consistent with the DBT system resolution and subtends the volume subtended by the pin head size radiographically detectable markers. The sixth step 105 is to run the mathematical process to convert the multiple 2D DBT images into a single 3D DBT dataset composed of voxels with each voxel having a unique (x, y, z) coordinate. The seventh step 106 is to plot the 3D DBT data in the X, Y, Z coordinate system. The eighth step 107 is to display the 3D DBT data in true 3D via an extended reality (e.g., augmented reality, mixed reality or virtual reality) headset for radiologist examination. The ninth step 108 is for the computer responds to radiologist commands issued via the control unit to invoke the following: establishing view point; rotating, zooming, flying through the 3D volume and/or adding false color to denote selected tissue types; invoking tissue filtering to improve visualization of microcalcifiction and/or tumerous tissue; creation of a 3D cursor and movement thereof to regions of interest and re-size/re-shape, as desired; remove tissue external to 3D cursor, as desired; positioning the focal point pen to tissue of interest and create symbols/notes, as desired; move contents of 3D cursor to geo-registered pedestal and affix contents to hand held pedestal; and, move hand held pedestal with affixed contents, as desired. View the reconstructed 3D dataset via standard slice-by-slice scrolling or via advanced imaging techniques including those described in U.S. Pat. No. 8,384,771, Method and apparatus for three dimensional viewing of images, U.S. Pat. No. 9,349,183, Method and apparatus for three dimensional viewing of images, U.S. Pat. No. 9,473,766, Method and apparatus for three dimensional viewing of images, U.S. Pat. No. 9,980,691, Method and apparatus for three dimensional viewing of images, U.S. patent application Ser. No. 15,904,092, Processing 3D medical images to enhance visualization, U.S. patent application Ser. No. 16/195,251, INTERACTIVE VOXEL MANIPULATION IN VOLUMETRIC MEDICAL IMAGING FOR VIRTUAL MOTION, DEFORMABLE TISSUE, AND VIRTUAL RADIOLOGICAL DISSECTION, U.S. patent application Ser. No. 16/524,275, USING GEO-REGISTERED TOOLS TO MANIPULATE THREE-DIMENSIONAL MEDICAL IMAGES, PCT/US19/47891, A VIRTUAL TOOL KIT FOR RADIOLOGISTS, U.S. patent application Ser. No. 16/509,592, IMPLANTABLE MARKERS TO AID SURGICAL OPERATIONS, U.S. patent application Ser. No. 16/563,985, METHOD AND APPARATUS FOR THE INTERACTION OF VIRTUAL TOOLS AND GEO-REGISTERED TOOLS, and PCT/US19/23968, RADIOLOGIST-ASSISTED MACHINE LEARNING WITH INTERACTIVE, VOLUME-SUBTENDING 3D CURSOR.
(41) FIG. 2 illustrates an apparatus for implementing the process illustrated in FIG. 1. A radiologic imaging system 200 (i.e., digital breast tomosynthesis) is used to generate 2D medical images 202 of a breast 204. The 2D medical images 202 are provided to an image processor 206, that includes processors 208 (e.g., CPUs and GPUs), volatile memory 210 (e.g., RAM), and non-volatile storage 212 (e.g. HDDs and SSDs). A program 214 running on the image processor implements one or more of the steps described in FIG. 1. 3D medical images are generated from the 2D medical images and displayed on an IO device 216. The IO device may include an extended reality display (e.g., mixed reality, virtual reality or augmented reality headset), monitor, tablet computer, PDA (personal digital assistant), mobile phone, or any of a wide variety of devices, either alone or in combination. The IO device may include a touchscreen, and may accept input from external devices (represented by 218) such as a keyboard, mouse, joystick, and any of a wide variety of equipment for receiving various inputs. However, some or all the inputs could be automated, e.g. by the program 214.
(42) FIGS. 3A, 3B and 3C illustrate the breast with georegistration points on the skin surface, georegistration grid on the skin surface and coordinate system. FIG. 3A illustrates the breast 301 with georegistration points 302. FIG. 3B illustrates the breast 303 with georegistration points 304 and a georegistration grid 305. FIG. 3C illustrates the breast 306 with georegistration points 307, georegistration grid 308 and the coordinate system 309.
(43) FIGS. 4A and 4B illustrate the X-ray detector, x-ray beam, breast, microcalcification within the breast and the detector. In FIG. 4A, the x-ray tube is shown. The x-ray beam 402 is shown. The breast 403 is shown. 404 represents a microcalcification within the breast. 405 represents the image acquired. 406 represents the microcalcification within the image of the breast. 407 represents the image of the breast. In FIG. 4B, 408 represents the x-ray tube, which has been moved to a new position. 409 represents the x-ray beam from this new position. 410 represents the microcalcification in the breast, which has not moved. 411 represents the breast, which has also not moved. 412 represents the image acquired. 413 represents the image of the microcalcification, which has shifted in position as compared to 406 due to the new angle of the x-ray. 414 again demonstrates the image of the breast. 415 represents the distance that the microcalcification has shifted in image 412 as compared to image 405. Note that structures closer to the detector have less shift within the image than structures farther from the detector.
(44) FIGS. 5A and 5B illustrate the geometry required to compute the (x, y, z) coordinate of a microcalcification. In FIG. 5A, the geometry of the X-ray tube, X-ray detector, compression paddle, breast and microcalcification are illustrated. 501 illustrates the x-ray tube at the initial position directly above the breast in a position to send x-rays are emitted vertically toward the floor. 502 illustrates the x-ray tube at a subsequent position in a position to send x-rays obliquely toward the floor. 503 illustrates the path of X-ray photons passing from the X-ray tube 502 through the microcalcification 507 and then to the detector 508. Similarly, 504 illustrates the path of X-ray photons passing from the X-ray tube 501 through the microcalcification 507 to the detector 508. 505 illustrates the paddle that compresses the breast, the properties of which have minimal X-ray attenuation. 506 is the patient's breast containing the microcalcification 507. In FIG. 5B, the geometry of the microcalcification within the breast, the photon beams from the two X-ray tube positions, the X-ray detectors is illustrated. 509 represents the location of the microcalcification, which will be assigned an (x, y, z) coordinate. The coordinate of the microcalcification of interest 509 is located at the intersection of the vertically oriented X-ray beam 510 and the obliquely oriented X-ray beam 513. The angle between the vertically oriented X-ray beam 510 and the obliquely oriented X-ray beam 513 is α 512. The spot where the vertically oriented X-ray beam hits the detector is 511. Note that the angle between the vertically oriented X-ray beam 510 and the detector is denoted as 516 and is 90 degrees. The spot where the obliquely oriented X-ray beam (from 502 and 503 in FIG. 5A) hits the detector is 515. The distance between the spot where the vertically oriented X-ray beam hits the detector 511 and the spot where the obliquely oriented X-ray beam hits the detector is 506. The angle between the detector, a portion of which is shown as 506, and the obliquely oriented X-ray beam 513 is β. To illustrate the calculation of the coordinates of the microcalcification from this geometry, the following assumptions will be made: 512 will be assumed to be 30 degrees; and, 506 will be assumed to be 2 cm. According to the law of a right triangle, angle β will equal 60 degrees. The tangent of 60 degrees equals the vertical height of the microcalcification in the z-direction divided by the distance 506. Therefore, the height of the microcalcification over the detector is √12 cm, which will be the z-coordinate of the microcalcification. The x-coordinate of the microcalcification is the spot on the detector from the vertically oriented X-ray beam 510. The y-coordinate of microcalcification is also the spot on the detector from the vertically oriented X-ray beam 510. X, Y, Z axes 516 are shown. For example, if a linear calcification is oriented vertically, on the image wherein the x-ray beam is vertically oriented with respect to the floor, the calcification would appear as a dot. If the x-ray beam is at an angle with respect to the calcification, then the calcification would appear as a line.
(45) FIGS. 6A through 6D illustrate a cluster of microcalcifications within a volume-subtending 3D cursor with external tissues subtracted, transparency of internal structures altered and tissues both inside and outside of the 3D cursor subtracted except for the cluster of microcalcifications. In FIG. 6A, 600 illustrates an artery. 601 illustrates the breast tissue. 602 illustrates the 3D cursor. 603 illustrates a cluster of microcalcifications. 604 illustrates a vein. In FIG. 6B, note that the tissues external to the 3D cursor 605 have been subtracted. Note that the tissues inside of the 3D cursor 606 are unchanged from FIG. 6A. In FIG. 6C, the items shown are the same as FIG. 6B with the exception that the transparency (or grayscale) of the artery and vein 607 is altered to improve visualization of the microcalcifications. Transparency can be performed in a variety of ways, such as sparse sampling of the voxels segmented to a particular structure. Alternatively, or additionally, the opacity level of a particular voxel can be altered. Alternatively, or additionally, prioritized volume rendering can be perform as described in U.S. Provisional Patent Application 62/846,770 A METHOD OF PRIORITIZED VOLUME RENDERING TO IMPROVE VISUALIZATION OF PRIORITIZED ITEMS WITHIN A 3D VOLUME. In FIG. 6D, all tissues both inside and outside of the 3D cursor 608 with the exception of the microcalcifications are eliminated. This overall effort aims to improve visualization of microcalcifications for a user wearing an extended reality headset. By rotating, zooming, tilting or turning one's head, converging, one can better visualize and understand the true 3D distribution of microcalcifications.
(46) FIGS. 7A through 7D illustrate viewing of the isolated cluster of microcalcifications using an augmented reality or virtual reality head mounted display. 700 illustrates the head display unit, which includes the transmit/receive element 706, the inertial measurement unit 707 and the georegistration point 708. 701 illustrates the left eye viewing perspective with left eye field of view. 702 illustrates the right eye viewing perspective with right eye field of view. 703 illustrates the 3D cursor. 704 illustrates the x, y, z coordinate system. 705 illustrates the microcalcifications. FIG. 7A illustrates the initial viewing perspective. FIG. 7B illustrates changing the interocular distance and angular field of view (compare with FIG. 7A). FIG. 7C illustrates the rotating the 3D cursor 90 degrees for a different viewing angle of the volume of interest (compare with FIG. 7A and FIG. 7B). FIG. 7D illustrates changing the viewing perspectives by rotating it by 90 degrees, such as looking at the cluster of microcalcifications from the side rather than from the front.
(47) FIGS. 8A through 8C illustrate the use of the geo-registered platform and geo-registered focal point pen to inspect the microcalcifications within the 3D cursor. 800 illustrates the geo-registered platform. 801 illustrates a geo-registered point on the platform. 802 illustrates the transmit/receive element. 803 illustrates the inertial measurement unit. 804 illustrates the 3D cursor. 805 illustrates the microcalcifications within the 3D cursor. Note that the microcalcifications are shown as cubes for illustrative purposes only. In actuality, the microcalcifications may take on various shapes, sizes and densities. In FIG. 8A, the hand-held georegistration platform is affixed to the 3D cursor, which enables the mammographer to translate and rotate the georegistration platform and therefore view the 3D cursor and cluster of microcalcifications within the 3D cursor from multiple angles. In FIG. 8B, the hand-held georegistration platform is tilted and the 3D cursor and cluster of microcalcifications within the 3D cursor are likewise tilted. In FIG. 8C, the georegistered focal point pen 806 including the components of the transmit/receive element 807 and the inertial measurement unit 808 and the georegistered point 809 is shown pointing to the microcalcifications 805 within the 3D cursor 804. The georegistered pen can be used to perform functions such as image markup for communication with other physicians or areas of concern.
(48) FIG. 9 illustrates a top down view of the mammographer's desk illustrating several of the tools with position and orientation tracking. 900 illustrates the standard equipment at the radiologists work station including multiple monitors, computer, power supply, mouse keyboard, voice dictation, etc. 901 illustrates the head display unit (HDU), such as an augmented reality, mixed reality or virtual reality system equipped with a transmit/receive element, an inertial measurement unit and a georegistration point. 902 illustrates the real-world image of the georegistered platform that the user can visualize with the HDU. 903 illustrates the virtual image of the 3D cursor displayed on the HDU, which includes the 3D cursor and a cluster of microcalcifications. 904 illustrates the master control platform used to guide the georegistration system. 905 illustrates the mammographer's left hand holding the georegistration platform, which is equipped with a transmit/receive element, an inertial measurement unit and a georegistration point. 906 illustrates the mammographer's right hand holding a focal point pen, which is equipped with a transmit/receive element, an inertial measurement unit and a georegistration point.
(49) FIG. 10 illustrates the conversion of a digital breast tomosynthesis dataset into a single voxelated dataset. 1000 illustrates a first digital breast tomosynthesis image from a first position of the x-ray tube. 1002 illustrates a first digital breast tomosynthesis image from a first position of the x-ray tube. 1004 illustrates a first digital breast tomosynthesis image from a first position of the x-ray tube. 1006 illustrates a first digital breast tomosynthesis image from a first position of the x-ray tube. 1008 illustrates a voxelated dataset. A series of pixels corresponding to calcifications would be detected on a first image 1000. From those pixels, a back projected image would be performed to the x-ray tube position. For each pixel with a calcification density, a set of potential x,y,z coordinates for the true location of the calicifications is yielded (i.e., along a line from the detector location to the x-ray tube). Next, a set of pixels corresponding to calcifications would be detected on a second image 1002. From those pixels, a back projected image would be performed to the x-ray tube position and a second set of potential x,y,z coordinates for the true location of the calcifications is yielded (i.e., along a line from the x-ray detector to the x-ray tube). The x,y,z coordinate of the calcification would be determined by the intersection point from the first tomosynthesis image to the second tomosynthesis image. This process would be repeated and a voxelated dataset formed for each pair of images. Note that multiple voxelated datasets could therefore be formed. Thresholds to plot only the calcifications could be set. Alternatively, thresholds to plot all tissues could be set.
(50) FIGS. 11A through 11D illustrate variable compression during digital breast tomosynthesis. The breast 1100 is shown. The detector 1102 is shown. The compression paddle 1104 is shown. FIG. 11A illustrates a low pressure of compression of the breast 1100. Note the initial contour of the breast 1100. FIG. 11B illustrates a high level of compression of the breast 1100. Note the subsequent contour of the breast 1100. FIG. 11C illustrates an initial configuration of a breast mass 1106. FIG. 11D illustrates a subsequent configuration of a breast mass 1108. Note that different tissues in the body will have different tissue type properties. For example, bone is rigid. Fat is easily deformable. Cancers can be hard. The purpose of doing the compression at two different levels is therefore to see how a lesion changes in its appearance and configuration under the two different conditions. This is accomplished by performing two different DBT examinations under different levels of compression, generating a 3D volume at the first level of compression, generating a 3D volume at the second level of compression and viewing the two datasets to see how the internal structures would change. For example, there might be two adjacent lesions and the radiologist is unsure whether these lesions are connected as a single bilobed mass or whether they are not connected at all. Such a technique can distinguish between these two scenarios. An optimum viewing strategy is to perform segmentation and then watch a segmented structure of interest change over the two (or more) configurations. Alternatively, compression could be performed from side-to-side under two (or more) compression levels. Sometimes there can be many lesions in a breast and it can be difficult for the radiologist to determine which lesion on the top-compression view corresponds to which lesion on the side-compression view. This is where the skin markers may be of benefit. By utilizing skin markers, the internal structures can be better analyzed. For example, there could be four areas in the breast of interest to the radiologist. The radiologist could place a 3D cursor over the first area of concern and then view it under various external compression settings to determine how it changes. Then perform a similar process to the second lesion and so on. An alternative approach to DBT for breast imaging is breast Mill. In this case, a dynamic compression device can be built and synchronized with the breast MRI acquisition such that multiple volumes are obtained. For example, during a first time period at a first compression level, a first volume is built. During a second time period at a second compression level, a second volume is built. And, so on. The volumes can be viewed in a dynamic fashion on an extended reality display. Ideally, at least in some implementations, this would be performed with a fast acquisition MRI. For example, a round mass might stay round even under dramatically different compression levels and this may indicate the hardness of the tumor and serve as an indicator for a first type of tumor (e.g., cancer risk). Alternatively, a round mass might become flattened into a pancake-like shape under a high amount of compression and this may indicate that the tumor is soft and serve as an indicator for a different type of tumor. Dynamic compression devices can be subsequently designed and built to accommodate the process outlined in this patent. Still further, the change in configuration of an anatomic feature can be utilized to help assign a tissue-type property (e.g., hardness). For example, a sphere shaped structure that remains sphere shaped despite a high pressure exerted upon it would be assigned a hard tissue-type property. In contrast, a sphere shaped structure that becomes pancake shaped when the same high pressure is exerted upon it would be assigned a soft tissue-type property. This is further described in U.S. patent application Ser. No. 15/904,092, INTERACTIVE VOXEL MANIPULATION STRATEGIES IN VOLUMETRIC MEDICAL IMAGING ENABLES VIRTUAL MOTION, DEFORMABLE TISSUE, AND VIRTUAL RADIOLOGICAL DISSECTION. Alternatively, this process could be performed in applications other than the breast. For example, a 3D imaging examination of the knee joint could be performed at a first configuration wherein the knee is in a straight 180 degree position. Then, a 3D imaging examination of the knee joint could be performed at a second configuration wherein the knee is slightly flexed to a 135 degrees position. Then, a radiologist wearing an extended reality head display unit can view the anatomic feature of the knee joint under different configurations and analyze changes therein. For example, a small meniscal tear can be obscured in the first 180 degree extended position, but then identified when the knee is in a 135 degree flexed position. MRI coils can be manufactured to accommodate these changes in positions and optimize imaging parameters. In addition to the analysis of motion of joints, this process could be performed to analyze other structures that move in the body. The higher number of volumes would allow an improved ability to assess changes in configuration and would yield a more accurate analysis. For example, this process can be applied to an vascular structure whose configuration changes over the cycles of systole and diastole. For example, this process could be performed on the brain whose configuration changes over the cycles of CSF pulsation. For example, this process could be performed on the trachea or airways whose configuration changes over the cycles of inhalation and exhalation. The human body is mobile; thus, virtually every structure in the human body would be better analyzed during the 3D analysis of motion, changes in configuration and deformation as described in processes outlined in this patent.
(51) FIGS. 12A through 12D illustrate two cases of tumors of varying hardness levels. FIG. 12A illustrates case 1 with a low level of compression. The breast 100, breast mass 1201, detector 1202, compression device 1203 and an initial pressure 1204 (e.g., with units of PSI) is illustrated. Note the configuration of the breast 1200 is somewhat round and the configuration of the mass 1201 is round. FIG. 12B illustrates case 1 with a high level of compression. The breast 1200, breast mass 1201, detector 1202, compression device 1203 and an subsequent higher pressure 1205 (e.g., with units of PSI) is illustrated. Note the configuration of the breast 1200 is now more flattened (compare with FIG. 12A) and the configuration of the mass 1201 is still round. This would indicate that the mass 1201 is hard. A deformability index could be established to indicate the amount of deformation of a given tissue in relation to the amount of pressure. FIG. 12C illustrates case 2 with a low level of compression. The breast 1206, breast mass 1207, detector 1208, compression device 1209 and an initial pressure 1210 (e.g., with units of PSI) is illustrated. Note the configuration of the breast 1206 is somewhat round and the configuration of the mass 1207 is round. FIG. 12D illustrates case 2 with a high level of compression. The breast 1206, breast mass 1207, detector 1208, compression device 1209 and an subsequent higher pressure 1211 (e.g., with units of PSI) is illustrated. Note the configuration of the breast 1206 is now more flattened (compare with FIG. 12C) and the configuration of the mass 1207 is still flattened. This would indicate that the mass 1201 is soft and deformable. A deformability index could be established to indicate the amount of deformation of a given tissue in relation to the amount of pressure. Note the the deformability index of the breast mass 1201 in case 1 would be different from the deformability index of the breast mass 1207 in case 2. Note that in this illustration, two compression levels are illustrated. A 3D breast imaging examination would be performed at each of the two compression levels. Note that additional 3D imaging examinations could be performed at varying compression levels (e.g., 0.5 psi, 1.0 psi, 1.5 psi, 2.0 psi, 2.5 psi, 3.0 psi, 3.5 psi, 4.0 psi, etc.). The volumes will be reconstructed and viewed on an extended reality head display unit.
(52) FIG. 13 illustrates a digital breast tomosynthesis dataset performed with skin markers. The breast 1300, breast mass 1301, skin radioopaque markers 1302, detector 1304, compression device 1305 with adjustable compression 1306 are shown. The 3D cursor 1303 can be viewed on the image processing workstation. The remainder of the elements of a digital breast tomosynthesis machine is not shown. Note that the skin markers can be utilized as landmarks and reference points to improve understanding of how internal structures change. Note that this is especially important with less 3D imaging volumes obtained.
(53) FIG. 14 illustrates a flow diagram for generating a 3D volumetric dataset of microcalcifications from a digital breast tomosynthesis dataset. The flow diagrams do not depict the syntax of any particular programming language. Rather, the flow diagrams illustrate the functional information one of ordinary skill in the art requires to fabricate circuits or to generate computer software to perform the processing required in accordance with the present invention. It should be noted that many routine program elements, such as initialization of loops and variables and the use of temporary variables, are not shown. It will be appreciated by those of ordinary skill in the art that unless otherwise indicated herein, the particular sequence of steps described is illustrative only and can be varied without departing from the spirit of the invention. Thus, unless otherwise stated the steps described below are unordered meaning that, when possible, the steps can be performed in any convenient or desirable order. In Step 1400, a digital breast tomosynthesis examination is performed. In Step 1402, a threshold is set for microcalcifications. Microcalcifications are typically denser than the breast glandular tissue; thus, can be isolated and viewed. In Step 1404, beginning at a first 1-mm focal plane image, determine if there are any microcalcifications as determined by the set threshold and plot only those pixels into an (x,y,z) dataset, such that the x,y value is the location of the pixel on the 1-mm focal plane image and the z-value is the focal plane image. In Step 1406, move to next contiguous 1-mm focal plane image and repeat step 1404 until all 1-mm focal plane images have been examined for calcifications and the (x,y,z) dataset is complete. In Step 1408, perform an iterative 3D viewing process of the 3D volumetric dataset of microcalcifications that has been built.
(54) FIG. 15A illustrates a series of slices with a first threshold setting and a cluster of microcalcifications. In this illustration, digital breast tomosynthesis images are reconstructed to create 1-mm focal plane images. Contiguous 1-mm focal plane images are illustrated. In this example, three threshold settings are applied. The first threshold setting shows four contiguous 1-mm focal plane images 1500, 1502, 1504 and 1506. Assuming 1024 gray scales, this would be a setting, such as all gray scales brighter than 900 are displayed and all grayscales at 899 or less are hidden (i.e., filtered). 1500 illustrates a 1-mm focal plane image with a threshold setting shown such that only a single microcalcification 1501 is displayed. 1502 illustrates a contiguous 1-mm focal plane image to 1500 with a threshold setting shown such that only a single microcalcification 1503 is displayed. 1504 illustrates a contiguous 1-mm focal plane image to 1502 with a threshold setting shown such that only a single microcalcification 1505 is displayed. 1506 illustrates a contiguous 1-mm focal plane image to 1504 with a threshold setting shown such that only a single microcalcification 1507 is displayed. For each calcification, the (x,y) coordinate of the calcification would be recorded along with the z-coordinate (height) of the focal plane image. These coordinates and data units (i.e., grayscale value) of the calcifications would be recorded. Various threshold settings could be performed such that only pixels/voxels that are highly probable (>99%) to represent microcalcifications are shown. These could be given a particular false color in accordance with the methods described in patent application Ser. No. 15/904,092, PROCESSING 3D MEDICAL IMAGES TO ENHANCE VISUALIZATION. Then, pixels/voxels that are probable (95%-99%) to represent microcalcifications could be given an alternative false color in accordance with the methods described in patent application Ser. No. 15/904,092, PROCESSING 3D MEDICAL IMAGES TO ENHANCE VISUALIZATION. And so on. Then, these would be plotted on a 3D dataset. Then, these would be shown on a head display unit (i.e., an Extended Reality display), in the preferred embodiment. Displaying the calcifications on a 2D monitor would be an alternative embodiment.
(55) FIG. 15B illustrates a series of slices with a second threshold setting and a cluster of microcalcifications. The second threshold setting shows four contiguous 1-mm focal plane images 1510, 1512, 1514 and 1516. Assuming 1024 gray scales, this would be a setting, such as all gray scales brighter than 700 are displayed and all grayscales at 699 or less are hidden (i.e., filtered). Note that 1510 illustrates a 1-mm focal plane image with a threshold setting shown such that three microcalcification 1511 is displayed. 1512 illustrates a contiguous 1-mm focal plane image to 1510 with a threshold setting shown such that three microcalcifications 1513 are displayed. 1514 illustrates a contiguous 1-mm focal plane image to 1512 with a threshold setting shown such that three microcalcifications 1515 are displayed. 1516 illustrates a contiguous 1-mm focal plane image to 1514 with a threshold setting shown such that three microcalcification 1517 are displayed. For each calcification, the (x,y) coordinate of the calcification would be recorded along with the z-coordinate (height) of the focal plane image. The finer the focal plane images would result in an improved spatial resolution in the z-direction. Ideally, the spatial resolution would be well below 1 mm in the z-direction. These coordinates and data units (i.e., grayscale value) of the calcifications would be recorded. Then, these would be plotted on a 3D dataset. Then, these would be shown on a head display unit (i.e., an Extended Reality display), in the preferred embodiment.
(56) FIG. 15C illustrates a series of slices with a third threshold setting and a cluster of microcalcifications. The third threshold setting shows four contiguous 1-mm focal plane images 1520, 1522, 1524 and 1526. Assuming 1024 gray scales, this would be a setting, such as all gray scales brighter than 500 are displayed and all grayscales at 499 or less are hidden (i.e., filtered). These numbers are given for illustrative purposes. The actual thresholds would vary in accordance with acquisition technique. Note that 1520 illustrates a 1-mm focal plane image with a threshold setting shown such that three microcalcification 1521 and some breast glandular tissue 1528 are displayed. 1522 illustrates a contiguous 1-mm focal plane image to 1520 with a threshold setting shown such that three microcalcifications 1523 and some breast glandular tissue 1529 are displayed. 1524 illustrates a contiguous 1-mm focal plane image to 1522 with a threshold setting shown such that three microcalcifications 1525 and some breast glandular tissue 1530 are displayed. 1526 illustrates a contiguous 1-mm focal plane image to 1524 with a threshold setting shown such that three microcalcification 1527 and some breast glandular tissue 1531 are displayed. For each calcification, the (x,y) coordinate of the calcification would be recorded along with the z-coordinate (height) of the focal plane image. These coordinates and data units (i.e., grayscale value) of the calcifications would be recorded. Then, these would be plotted on a 3D dataset. Then, these would be shown on a head display unit (i.e., an Extended Reality display), in the preferred embodiment. Note should be made that small calcifications typically only appear on a single 1-mm focal plane image, but are blurred out on the other images; thus, the z-height can be accurately determined. The filtering/threshold settings can and should be changed dynamically over time to optimize bringing in most, if not all, microcalcifications into visualization. In addition, the volumes created should be viewed with rotation, focal point convergence, zooming and false color. False color can be used to communicate properties of the calcifications that might otherwise be difficult to appreciate on grayscale representation.
(57) FIG. 16 illustrates a flow diagram of viewing a 3D volumetric dataset of microcalcifications from a digital breast tomosynthesis dataset. In Step 1600, perform digital breast tomosynthesis examination. In Step 1601, convert dataset into a 3D dataset (e.g., voxelated dataset, etc.) wherein each x,y,z coordinate has a data unit. In Step C 1602, perform an iterative 3D viewing process. In Step 1603, view the initial volume in 3D (e.g., “depth-3-dimensional” D3D viewing). In Step 1604, adjust viewing parameters. For example, set high grayscale threshold, such that only dense microcalcifications are shown and then adjust grayscale threshold (e.g., incrementally lower) such that some additional microcalcifications are shown. If the threshold is brought too low, some glandular tissue would be seen and then the threshold could be raised again. This altering of thresholds was illustrated in FIG. 15. Also, user may perform rotation, zoom, focal point convergence. Also, user may utilize 3D cursor. Also, user may adjust prioritized volume rendering. Also, user may perform dynamic filtering.
(58) FIG. 17A illustrates a cluster of microcalcifications at a first time point. 1700 illustrates a first microcalcification. 1701 illustrates a second microcalcification. 1702 illustrates a third microcalcification. 1703 illustrates a fourth microcalcification.
(59) FIG. 17B illustrates a cluster of microcalcifications at a second time point. 1700 illustrates the same first microcalcification. 1701 illustrates the same second microcalcification. 1702 illustrates the same third microcalcification. 1703 illustrates the same fourth microcalcification. 1704 illustrates a new fifth microcalcification. 1705 illustrates a new sixth microcalcification. Note that the fifth microcalcification 1704 and the sixth microcalcification 1705 are both new and a branching pattern is starting to form.
(60) FIG. 17C illustrates a cluster of microcalcifications at a third time point. 1700 illustrates the same first microcalcification. 1701 illustrates the same second microcalcification. 1702 illustrates the same third microcalcification. 1703 illustrates the same fourth microcalcification. 1704 illustrates a new fifth microcalcification. 1705 illustrates a new sixth microcalcification. Note that the fifth microcalcification 1704 and the sixth microcalcification 1705 are both new and a branching pattern is starting to form. Note that in this scenario, the new calcifications 1704 and 1705 are colored red, to illustrate to the radiologist interval changes. In this illustration, two different time points are illustrated; however, other methods of comparative analysis include the same time point with two different configurations (e.g., CC DBT exam and MLO DBT examination).
