Image processing via a modified segmented structure

Abstract

A method and apparatus of generating a modified segmented structure is disclosed. This modified segmented structure subtracts pixels or voxels in an inward direction from the segmented structure to create a smaller volume or area than the segmented structure. Analysis of the smaller volume can be useful in generating custom volumes, which can then be processed to improve visualization inside the custom volume.

Claims

1. A method comprising: loading an imaging dataset into an image processing computer system; performing an initial segmentation of a structure within the imaging dataset; determining an outer surface of the structure; determining at least one layer of pixels or voxels internal to the outer surface of the structure wherein the at least one layer of pixels or voxels is not conformal with the outer surface of the structure, and wherein a non-conformal shape of said at least one layer of pixels or voxels is determined based on at least one of the group of: said at least one layer of pixels' data units relative to said structure's pixels data units; and said at least one layer of voxels' data units relative to said structure's voxels data units; and subtracting the at least one layer of pixels or voxels internal to the outer surface of the structure to generate a modified segmented structure.

2. The method of claim 1 further comprising wherein a substantially equal layer of pixels or voxels is subtracted from the outer surface of the structure.

3. The method of claim 1 further comprising wherein layers of pixels or voxels are subtracted in a non-uniform manner wherein the non-uniformity is determined by at least one of the group consisting of: pixels or voxels location(s); and, the specific structure that is segmented.

4. The method of claim 1 further comprising wherein after the pixels or voxels internal to the segmented structure are subtracted, displaying said modified segmented structure to a user.

5. The method of claim 1 further comprising displaying a line on said segmented structure to a user to denote the modified segmented structure's surface.

6. The method of claim 5 further comprising wherein the line is shown in at least one of the group comprising: a dotted fashion; a solid fashion; thin weight; medium weight; thick weight; and, color.

7. The method of claim 1 further comprising wherein the modified segmented structure is used to designate a volume for which additional image processing is performed.

8. The method of claim 7 further comprising wherein the additional image processing includes at least one of the group consisting of: radiomics; and, artificial intelligence.

9. The method of claim 8 further comprising wherein the additional image processing is used to determine a dose of a pharmaceutical.

10. The method of claim 9 further comprising wherein the additional image processing comprises visual representation adjustment logic.

11. The method of claim 7 further comprising wherein the additional image processing is performed on the structure.

12. The method of claim 11 further comprising wherein the additional image processing is used to determine a dose of a pharmaceutical.

13. The method of claim 1 further comprising wherein additional non-contiguous pixels or voxels are subtracted from said modified segmented structure to form a further modified segmented structure.

14. The method of claim 1 further comprising wherein a visual appearance of the pixels or voxels in the modified segmented structure are modified independently from the pixels or voxels in the structure.

15. The method of claim 1 further comprising wherein the number of pixels or voxels included in the modified segmented structure is dependent upon at least one of the group consisting of: a patient demographics; a metadata; a type of pathology of clinical concern; a type of pathology in the structure; and, a size of the pathology in the structure.

16. The method of claim 1 further comprising wherein the modified segmented structure at a first time point and the modified segmented structure performed on the structure at a second time point are analyzed to determine interval change.

17. The method of claim 1 further comprises wherein a number of pixels or voxels included in the modified segmented structure can be varied by at least one of the group consisting of: a user input; and, an artificial intelligence input.

18. The method of claim 1 further comprising wherein an annotations is inputted to mark the modified segmented structure.

19. An apparatus comprising: an IO device; and an image processor in communication with the IO device, the image processors comprising a program stored on a computer-readable non-transitory media, the program comprising instructions that perform: loading an imaging dataset; segmenting of a structure within the imaging dataset to define a first boundary wherein the first boundary marks an outer surface of the structure; determining at least one layer of pixels or voxels internal to the first boundary wherein the one layer of pixels or voxels is not conformal with the first boundary, and wherein a non-conformal shape of said at least one layer of pixels or voxels is determined based on at least one of the group of: said at least one layer of pixels' data units relative to said structure's pixels data units; and said at least one layer of voxels' data units relative to said structure's voxels data units; and generating a second boundary wherein the second boundary is generated by moving inward the at least one layer of pixels or voxels from the first boundary.

20. A non-transitory computer readable medium having computer readable code thereon for image processing, the medium comprising: instructions for loading an imaging dataset; instructions for performing an initial segmentation of a structure within the imaging dataset; instructions for determining an outer surface of the structure; instructions for determining at least one layer of pixels or voxels internal to the outer surface of the structure wherein the at least one layer of pixels or voxels is not conformal with the outer surface of the structure, and wherein a non-conformal shape of said at least one layer of pixels or voxels is determined based on at least one of the group of: said at least one layer of pixels' data units relative to said structure's pixels data units; and said at least one layer of voxels' data units relative to said structure's voxels data units; and instructions for subtracting the at least one layer of pixels or voxels internal to the outer surface of the structure to generate a modified segmented structure.

Description

BRIEF DESCRIPTION OF THE FIGURES

(1) The flow diagrams do not depict the syntax of any particular programming language. Rather, the flow diagrams illustrate the functional information one of ordinary skill in the art requires to fabricate circuits or to generate computer software to perform the processing required in accordance with the present invention. It should be noted that many routine program elements, such as initialization of loops and variables and the use of temporary variables, are not shown. It will be appreciated by those of ordinary skill in the art that the particular sequence of steps described is illustrative only and can be varied without departing from the spirit of the invention. Thus, when possible, the steps can be performed in any convenient or desirable order.

(2) FIG. 1 illustrates a D3D image processing system.

(3) FIG. 2 illustrates aspects of voxel adjustments based on radiodensity in greater detail.

(4) FIGS. 3A, 3B, and 3C illustrate adjustment of voxel size, shape, and orientation, respectively.

(5) FIGS. 4A and 4B illustrate feature demarcation.

(6) FIG. 5 illustrates mathematical adjustments of voxels to enhance discrimination between voxels based on grayscale values.

(7) FIG. 6 illustrates aspects of filtering in greater detail.

(8) FIG. 7 illustrates an aspect of segmentation in which a voxel is classified based on the type of tissue of its nearest neighbor voxels.

(9) FIGS. 8A and 8B illustrate aspects of advanced segmentation wherein gaps in a vascular structure are filled.

(10) FIG. 9 illustrates use of controller-entered commands to process and combine three axial CT images of the head using the described image processing techniques.

(11) FIG. 10 is a flow diagram of procedural steps in accordance with aspects of the invention.

(12) FIG. 11 illustrates optimal viewing settings for 2D imaging and for 3D imaging.

(13) FIG. 12 illustrates utilization of range-based grouping as a display settings.

(14) FIG. 13 illustrates generating multiple simultaneous window/level settings for viewing of 3D datasets.

(15) FIG. 14 illustrates challenges faced of visualizing certain ranges without multiple windowing.

(16) FIG. 15 illustrates with overcoming challenges in visualizing certain ranges when using viewing multiple windowing.

(17) FIG. 16 illustrates a method of improving image quality.

(18) FIG. 17 illustrates a method of identifying data unit assurance markers.

(19) FIG. 18 illustrates an example of analyzing density of air outside of the patient's body to correct for artifacts within the patient's body.

(20) FIG. 19A illustrates a CT image of the head with window and level settings optimized for visualization of air.

(21) FIG. 19B illustrates a zoomed in CT image of the head with window and level settings optimized for visualization of air.

(22) FIG. 19C illustrates a plot of voxels within the region of air, which can be used to determine the correction factor applied to voxels within the head.

(23) FIG. 20 illustrates an example of placing phantoms inside of a table associated with a scanner.

(24) FIG. 21 illustrates an example of placing phantoms onto the patient.

(25) FIG. 22 illustrates an example of placing phantoms onto a blanket.

(26) FIG. 23A illustrates a phantoms surrounding a patient in a single, circumferential layer.

(27) FIG. 23B illustrates a phantoms surrounding a patient in a single layer, which is not circumferential.

(28) FIG. 23C illustrates a phantoms surrounding a patient multiple, circumferential layers.

(29) FIG. 23D illustrates a phantoms surrounding a patient in a multiple, non-circumferential layers.

(30) FIG. 24 illustrates multiple measurements in a radial fashion outward from the patient.

(31) FIG. 25A illustrates a region of streak artifact going from scalp soft tissue into the air.

(32) FIG. 25B illustrates the modified dataset wherein the streak artifact is corrected.

(33) FIG. 26 illustrates an x-ray detector attached to a phantom.

(34) FIG. 27 illustrates a flow diagram describing an embodiment of this patent.

(35) FIG. 28A illustrates three representative slices of the spleen with double windowing.

(36) FIG. 28B illustrates three representative slices of the liver with double windowing.

(37) FIG. 29 illustrates a flow diagram describing an embodiment of this patent.

(38) FIG. 30A illustrates a first example of halo windowing.

(39) FIG. 30B illustrates a second example of halo windowing.

(40) FIG. 31 illustrates multiple halos.

(41) FIG. 32 illustrates a method of generating a modified segmented structure.

(42) FIG. 33A illustrates a substantially equal layer of voxels added to the outer surface of the segmented anatomic structure.

(43) FIG. 33B illustrates a CT scan showing a substantially equal layer of voxels added to the outer surface of the segmented anatomic structure.

(44) FIG. 34 illustrates layers of voxels added in a non-uniform manner.

(45) FIG. 35A illustrates an image of the abdomen.

(46) FIG. 35B illustrates subtracting the all tissues except the modified segmented volume of the pancreas.

(47) FIG. 35C illustrates a CT scan showing subtraction of all tissues except the modified segmented volume of the pancreas.

(48) FIG. 36A illustrates the segmented structure.

(49) FIG. 36B illustrates a CT scan showing the pancreas.

(50) FIG. 36C illustrates the segmented structure with a line denoting the outer boundary of the segmented structure.

(51) FIG. 36D illustrates a CT scan showing the pancreas with a line denoting the outer boundary of the segmented structure.

(52) FIG. 36E illustrates the segmented structure and the modified segmented structure with a line denoting the outer boundary of the segmented structure.

(53) FIG. 36F illustrates a CT scan showing the pancreas with a line denoting the outer boundary of the modified segmented structure.

(54) FIG. 37A illustrates a first appearance of the line demarking the outer boundary of the modified segmented structure.

(55) FIG. 37B shows the corresponding line demarking the outer boundary of the modified segmented structure in a CT scan.

(56) FIG. 37C illustrates a second appearance of the line demarking the outer boundary of the modified segmented structure.

(57) FIG. 37D shows the corresponding dotted line demarking the outer boundary of the modified segmented structure in a CT scan.

(58) FIG. 38 illustrates the inclusion of voxels in a non-contiguous manner.

(59) FIG. 39A illustrates a first appearance of the segmented anatomic structure and the modified segmented anatomic structure.

(60) FIG. 39B illustrates a first appearance of the segmented anatomic structure and the modified segmented anatomic structure.

(61) FIG. 40 illustrates examples of factors which can determine the number of voxels included in the modified segmented structure.

(62) FIG. 41A illustrates the a modified segmented structure at a first time point.

(63) FIG. 41B illustrates the a modified segmented structure at a second time point.

(64) FIG. 41C illustrates the comparison of a modified segmented structure at a first time point with the modified segmented structure at a second time point to determine interval change.

(65) FIG. 42 illustrates inputting annotations into the modified segmented structure.

(66) FIG. 43A illustrates a segmented structure (e.g., pancreas).

(67) FIG. 43B illustrates an example analysis performed on the segmented structure.

(68) FIG. 43C illustrates a modified segmented structure (e.g., pancreas with some surrounding voxels added).

(69) FIG. 43D illustrates an example analysis performed on the segmented structure.

(70) FIG. 43E illustrates a volume comprising only the surrounding voxels added, which would represent the difference between FIG. 43A and FIG. 13C.

(71) FIG. 43F illustrates an example analysis performed on the voxels added. In this example, a radiomics analysis is performed on the voxels added.

(72) FIG. 44 illustrates a method of generating an inwardly generated modified segmented structure.

(73) FIG. 45 illustrates an inwardly generated modified segmented structure/boundary.

DETAILED DESCRIPTION

(74) Some aspects, features, and implementations described herein may include machines such as computers, electronic components, optical components, and processes such as computer-implemented steps. It will be apparent to those of ordinary skill in the art that the computer-implemented steps may be stored as computer-executable instructions on a non-transitory computer-readable medium. Furthermore, it will be understood by those of ordinary skill in the art that the computer-executable instructions may be executed on a variety of tangible processor devices. For ease of exposition, not every step, device or component that may be part of a computer or data storage system is described herein. Those of ordinary skill in the art will recognize such steps, devices and components in view of the teachings of the present disclosure and the knowledge generally available to those of ordinary skill in the art. The corresponding machines and processes are therefore enabled and within the scope of the disclosure.

(75) In a D3D imaging system, a radiologist uses a controller to manipulate 3D (three-dimensional) images that may be viewed via special glasses or a VR (virtual reality) headset. The 3D images may be generated from radiological scan data, for example and without limitation from X-ray radiography, CT (computed tomography), PET (positron emission tomography), or MRI (magnetic resonance imaging). There will normally be a linear relationship between density or radiodensity values from the scan data and the grayscale values assigned to corresponding voxels of the 3D images. Advantages of existing examples may include improved depth perception and an improved human machine interface. Still, there are several challenges faced with this approach. First, an area of interest (e.g. tumor) may be in close proximity to structures that are similar in composition/density. Isolating the area of interest for better examination may be difficult. Second, many soft tissues in the body are mobile and deformable, so it can be difficult to achieve the best orientation to properly compare the tumor at multiple time points. Efficiently aligning the orientation to do so may be difficult. Third, certain portions of a tumor can respond to treatment and decrease in size while other portions of a tumor demonstrate increases in size. The pattern of tumor shrinkage has important prognostic implications. Furthermore, composition and complex morphologic features including speculations (spikes extending from the surface), irregular margins and enhancement also have important implications. Consequently, there is a need for a system that facilitates recognition of the subtle, yet important changes in size, shape and margins. Fourth, a patient with metastatic cancer has several areas of interest in different areas of the body. It is difficult and time consuming to find each of the areas of interest at every time point to determine interval change. Consequently, there is a need for a system that enables the observer to do this efficiently.

(76) FIG. 1 illustrates an improved D3D image processing system 100 and various types of controllers and a VR headset 99 that interface with the D3D image processing system. A wide variety of controllers may be utilized, possibly including but not limited to one or more of a keyboard 102, microphone 104 (for voice control), hand held game-type controller 106, joy stick 108, high-end mouse 110, and wand 112. The controllers are used to enter commands that control an application that processes 3D image data and displays that 3D image data. As will be explained in greater detail below, the controllers are used to select, configure and implement image processing techniques including: voxel adjustments based on radiodensity 114; filtering 116; and segmentation 118.

(77) Referring to FIG. 2, an aspect of voxel adjustments based on radiodensity is grayscale and color adjustments. Raw 3D image data may include radiodensity values for locations in a scanned volume. Initially, grayscale and/or color values that are linearly related to radiodensity values may be assigned to voxels of an image corresponding to the 3D image data. Grayscale and/or color values corresponding to selected radiodensity values may then be adjusted for image enhancement. In the illustrated example, relative radiodensity values are used, namely HU (Houndsfeld Units). Initially assigned grayscale values may be linearly related to the HU values. Controller-entered commands enable the radiologist to independently adjust the relationship between HU values and grayscale and/or color values, e.g. from linear to non-linear. This may advantageously enhance presentation of tissue types of interest. In figure line A of the illustrated example, air within the body is initially shown in grayscale as black, whereas bone is shown as white. In line B, false color has been added to a region corresponding to an HU range that represents soft tissue. Selection, configuration, and application occur in response to controller-entered commands. In line C, grayscale values corresponding to HU values above a specified level have been changed to black (or not included in the voxel set displayed). In line D, grayscales values both above and below an HU range have been changed to black (or not included in the voxel set displayed). Tissues and other features may be distinguished based on radiodensity. Consequently, features and tissues may be selected, enhanced, and excluded based on radiodensity.

(78) Although radiodensity is described herein as a basis for selection, it should be understood that a variety of related values, analogs, or proxies could be used instead of, or to represent, radiodensity. For example, and without limitation, tissue density could be used for selection. Further, a grayscale value, or range of values, could be directly selected and adjusted. The term radiodensity as used herein is intended to encompass all related values, analogs, and proxies.

(79) Another aspect of voxel adjustments based on radiodensity is increasing the dynamic range of the displayed voxels. A process to achieve this may involve the following steps: selecting a volume of interest; removing all external tissue; setting a lower bound level of interest and a top bound; removing all tissue external to these bounds; extending the bounds by a factor of a selected factor (e.g., 2 or 10); and interpolating voxel grayscale values between the two bounds. This will have the effect of increasing dynamic range. Humans can only distinguish 7-8 bits of grayscale. This expansion of the upper/lower bounds would distinguish 10 bits or more, thus enabling the possibility of earlier and/or improved detection rate of tumors.

(80) As shown in FIG. 3A, another aspect of voxel adjustments based on radiodensity is changing voxel size via controller-entered commands. For example, voxels having a selected radiodensity, or within a selected range, are adjusted in size by a selected magnitude. This technique may be used to cause a corresponding tissue type to be presented as semi-transparent. In the illustrated example, voxels 300, 302, 304 are reduced in size, yielding corresponding voxels 300, 302, 304, thereby permitting visualization of voxels 306, 308 that are more distant from the viewing point, i.e. deeper in the volume. Consequently, features can be seen that would otherwise have been occluded by the size-adjusted voxels in the near field of view.

(81) Referring to FIG. 3B, another aspect of voxel adjustments based on radiodensity is changing the shape of voxels via controller-entered commands based on radiodensity or some other basis. In the illustrated example, standard cubic voxels 310 having a selected radiodensity, or within a selected range, are adjusted to generate corresponding cylindrical voxels 312. Voxel shape may be selected to allow a smoother presentation of a feature being observed. For example, blood vessels could be better represented by cylindrical voxels as opposed to a column of cubes.

(82) Referring to FIG. 3C, another aspect of voxel adjustments based on radiodensity is changing voxel orientation via controller-entered commands. In the illustrated example voxel orientation is changed from alignment with standard X, Y, Z axes as shown in FIG. 3A, to a slanted coordinate system that more closely aligns with the with the patient's anatomy. For example, voxel orientation may be changed to more closely represent the curvature of body contents to include tissues or surgical devices. In the illustrated example the voxel orientation is adjusted relative to a blood vessel 314 by adjusting axes of the sides or edges 316 of the voxels. Voxels associated with the blood vessel or other tissue types may be selected based on radiodensity or some other basis.

(83) Referring to FIGS. 4A and 4B, another aspect of voxel adjustments based on radiodensity is feature demarcation. In the illustrated example, voxels associated with a vein 400 are demarked with blue lines 402 to enhance visualization. Although a vein is shown as an example, a wide variety of features and tissues may be demarked. Further, a wide variety of colors could be used, for example and without limitation, red lines could be generated to demark arteries. This is one of several potential uses of false color to facilitate medical personnel understanding of complex anatomy. Selection of whether to apply false color and to which types of tissue would be via a controller at the discretion of the medical personnel viewing the images. Voxels associated with the vein may be selected based on radiodensity or some other basis.

(84) Some implementations may include showing the volume subtended by a cursor during review. This would permit, for example, an opportunity for the radiologist to ensure a thorough examination had been conducted and see if any regions had been missed. In some implementations, the techniques could include designating a volume of interest designated by a colored cursor for subsequent review. For medical images captured over time following an injection of any contrast material, color may be used to indicate presence and flow of the contrast material over time. Options include: combining the traces from the multiple images to show the blood vessel structure through which the contrast material moved; providing, at direction of the radiologist, time tags at point(s) of furthest movement of contrast material along each of the blood vessel's paths at each time interval; highlighting by color any connected blood vessels wherein no contrast has flowed as a potential indicator of blockage; and calculating, at direction of the radiologist, the volume of blood by time period for designated blood vessel(s) and display. In some implementations, the techniques could include for cases wherein there have been multiple sets of medical images taken over different time periods and for which a 3D volume of a tissue mass of interest has been recorded, providing the following: registration of the two (or more) volumes using the same metric system; superimposing the volumes and highlight by color (or other means such as flashing) the volumetric differences; and providing a histogram of volume of tissue of interest to quantify changes in size of the different tissue(s) within the cursor at the different times of imaging.

(85) Referring to FIG. 5, another aspect of voxel adjustments based on radiodensity is enhanced voxel discrimination. The illustrated table 500 includes multipliers that enhance discrimination between voxels having near, but different, radiodensity and grayscale values. Voxels with Hounsfeld units of measure 25 (and associated grayscale) are multiplied by 1.2; voxels with Houndsfeld units of 30 (and associated grayscale) are multiplied by 1.5; voxels with Houndsfeld units of 35 (and associated grayscale) are multiplied by 2.0. Any of a wide variety of mathematical adjustments could be used, e.g. the table could be additive instead of multiplicative. Variations may include application of linear or non-linear functions. For example, the linear function (mX+b) could have the variable m (i.e., slope) adjusted from a low slope to a higher slope. The non-linear function could be parabolic (aX.sup.2+bX+c) (or other) where X would be the original grayscale. For other implementations, there could be a step increase. This is one of several ways to increase the ability to visually perceive small differences in radiodensity by adjusting the corresponding grayscale values, thereby improving visualization of the anatomy.

(86) The mathematical adjustments may be temporally adjusted in response to controller-entered commands. The illustrated example includes three temporal multiplier adjustments that are iterated. In some implementations, if filtering results in display of a fraction/percent of the voxels, then the display could alternate display of voxels not currently to be displayed at selected time intervals. In some implementations, color could be added, such as rainbow, to particular ranges to enhance visibility.

(87) FIG. 6 illustrates aspects of filtering in greater detail. In the illustrated example the filter is configured to select ?rd of the voxels for display at a given time. Specifically, every third voxel in a row of nine voxels numbered 1 through 9 is selected. Further, the filter temporally adjusts which ?rd of the voxels is selected for display at time points #1, #2, and #3 such that different sets of voxels are selected for display at different points in time. It may be advantageous to display only a portion of the voxels that represent a particular feature or tissue, e.g., air or fat. In the illustrated example only ?rd of the voxels of a certain radiodensity, and thus grayscale value, are shown at any one time. This option could also alternate which ?rd would be displayed over time. Note that the ?rd would not be a fixed fraction but one that was selectable.

(88) FIG. 7 illustrates an aspect of segmentation in greater detail. In the illustrated example the tissue type of a voxel 700 under consideration is classified based on the tissue type of nearest neighbor voxels in a matrix. Although a 5?5?5 matrix 702 is shown, matrices of other sizes may be used, or adjacent voxels may be considered as nearest neighbors. Moreover, the set or matrix of voxels is not necessarily arranged in a cube. The 124 nearest neighbor voxels in the 5?5?5 matrix 702 are already classified with the same tissue type, so voxel 700 is likewise classified. Selection and configuration of segmentation algorithms would be via a controller at the discretion of the medical personnel viewing the images.

(89) FIGS. 8A and 8B illustrate another aspect of segmentation. In the illustrated example, a gap 800 in a vascular structure is filled. There can be instances where a portion of certain connected tissue is not properly segmented when 2D algorithms are applied to 3D medical images. Vascular structure presents key challenges due to its curved nature within the body; it does not necessarily conform with a 3D grid structure. Advanced segmentation algorithms can be applied to grow the vascular structure from what is shown in FIG. 8A to what is shown in FIG. 8B. Selection of whether to apply advanced segmentation algorithms would be via a controller at the discretion of the medical personnel viewing the images.

(90) FIG. 9 illustrates use of controller-entered commands and image processing logic to generate three axial CT images of the head and combine features extracted therefrom to generate a composite image. Images A, B, and C are generated with any combination of the image processing techniques described above. Bone algorithm image A demonstrates optimal visualization of the calvarium. Grayscale level adjustments based on radiodensity have been configured to optimize visualization of the cancellous bone within the calvarium (window level of 360 and window width of 3400). A side-effect is poor visualization of the scalp and almost no detail of the brain. The brain tissue cannot even be distinguished from the cerebrospinal fluid (CSF). Soft tissue algorithm image B is generated from the same raw image data by grayscale level adjustments based on radiodensity to optimize visualization of the scalp (window level of 40 and window width of 350). Soft tissue algorithm image C is generated from the same raw image data by grayscale level adjustments based on radiodensity to optimize visualization of the brain (window level of 30 and window width of 100).

(91) The features of interest that have been visually optimized in images A, B, and C may be selected and extracted to generate a corresponding set of extracted feature images A, B, and C. The extracted feature images may then be combined to generate a composite image 900. In the illustrated example, each extracted feature image includes unique tissue or tissues within the set of extracted feature images so there is no overlap when the three extracted feature images are combined to generate the composite image that represents all tissues. However, overlapping voxels having different values could be resolved algorithmically. Advantageously, the image processing steps are efficiently implemented in response to controller-entered commands based on a visual/graphical interface, e.g. via the VR headset.

(92) Another aspect of visualization is the capability to combine findings with structured reporting. In some implementations, this could include displaying a list 902 of items which could be digitally marked by the radiologist using the 3D controller. The list of items could be unique to each item on the radiologist review checklist. In some implementations, the list presented on the 3D head display unit or the 2D display. Items which had been digitally marked would be automatically included in the report. In some implementations, a list of frequently used phrases for each item on the radiologist review checklist could be displayed at the direction of the radiologist who could then digitally mark any of the relevant phrases. Phrases which had been digitally marked would be automatically included in the report.

(93) Another aspect of visualization is changing viewing perspective. In some implementations, a selected image, volume, feature, tissue, voxel set, or voxel is automatically re-centered or re-located to enhance radiologist orientation of location within the volume being examined. Examples of auto re-location include, but are not limited to, the following: the viewing perspective could be re-located to an initial viewing point of an image set; and the viewing perspective could be re-located to additional point(s) designated by the radiologist during the course of the examination. In some implementations, the cursor color or shape could correspond to pre-selected conditions, for example, but not limited to: review checklist items; types of tissue of concern; and regions for collaboration between medical personnel.

(94) Another aspect of visualization is the capability to switch between various dimensional views (i.e., change back and forth between viewing 3D volume and 2D slices). In some implementations, selection of the 2D slice could be based on, but not limited to: the center point of the 3D cursor; a point within the 3D volume designated by the radiologist. In some implementations, the 3D controller to scroll through the slices.

(95) FIG. 10 illustrates a method in accordance with aspects of the invention. Procedurally, a volume of tissue within the body may initially be selected for processing in step 101. Raw scan data is retrieved for the selected volume, and one or more processing algorithms may be selected, configured, and applied. For example, one or more voxels adjustments based on radiodensity may be implemented to enhance one or more features of interest and optimize structure visualization as shown in step 103. Further, one or more segmentation algorithms may be selected, configured, and applied to segment tissues as shown in step 105. Further, filters may be selected, configured, and applied as shown in step 107. An image A is then generated based on the processing algorithms as shown in step 109. Steps 103, 105, 107, and 109 may be iterated for the selected volume any number of times to generate more images, e.g. images that enhance different tissues or features within the selected volume. In the illustrated example image B and image C are generated with different processing algorithms and/or configuration settings. The enhanced tissues or features from the images A, B, C are then extracted as shown in step 111, resulting in extracted feature images A, B, and C. The extracted feature images are then combined to generate a composite image as shown in step 113.

(96) FIG. 11 illustrates optimal viewing settings for 2D imaging and for 3D imaging. A list of the optimal viewing settings is generated for each item in an image. In 1100, two examples are provided within an image are shown along with the optimal viewing settings during 2D slice-by-slice viewing per user preference. In the first example, the liver is viewed on a CT scan with conventional settings. The liver is shaded in gray shades and with particular group (e.g., specified range is designed to catch hypervascular tumors, necrotic tumors, etc.) in color with option for voxel manipulation. All other tissues are turned to dark gray shades. In the second example, the breast is viewed during a digital breast tomosynthesis examination. Breast glandular parenchyma in gray shades with particular s (e.g., specified range is designed to catch microcalcifications) shown in red. Additionally, as discussed elsewhere in this disclosure and in U.S. patent application Ser. No. 16/195,251, INTERACTIVE VOXEL MANIPULATION STRATEGIES IN VOLUMETRIC MEDICAL IMAGING ENABLES VIRTUAL MOTION, DEFORMABLE TISSUE, AND VIRTUAL RADIOLOGICAL DISSECTION, the voxels are can be manipulated. This can be utilized to improve visualization. In 1102, a first visual representation adjustment logic is applied to the entire image, such as performing a liver window wherein voxels whose Hounsfield units are less than ?45 are black, voxels whose Hounsfield units are more than +105 are white and voxels whose Hounsfield units are shades of gray. Additionally, this embodiment also enables a second visual representation adjustment logic to be applied to voxels whose range is in between +80 to +105. This group helps the user pick out hypervascular liver metastases whose density is typically in the range of +80 to +105, as shown in red. The preferred embodiment is for the technique to be performed in conjunction with segmentation (e.g., segment the organ and then apply the described visual representation adjustment logic); however, this would not necessarily be required. Additionally, this embodiment also enables a third (or more) visual representation adjustment logic to be applied, such as coloring a group of voxels in the range of +20 to +30 would be in the range of necrotic liver tumors the color green. In 1104, two items within an image are shown along with the optimal viewing settings during 3D extended reality volume-by-volume viewing per user preference. For example, prioritized volume rendering of HU ranges is utilized within the liver and displayed in a dynamic fashion to make more subtle (but dangerous lesions) easier to detect. This is described in more detail in U.S. Provisional Patent Application No. 62/846,770, A method of prioritized volume rendering to improve visualization of prioritized items within a 3D volume. Additionally, the voxels that subtend the liver are divided into groups based on their property (e.g., Hounsfield Unit). For example, assume that voxels that subtend the liver have Hounsfield of 30-60. These can be divided into 3 groups (e.g., upper range of 50-60 HU, a middle range of 40-50 HU, and a lower range of 30-40 HU) wherein at three different time points one of the groups has enhanced visualization (e.g., color schemes) and the other two groups have diminished visualization (e.g., dark gray scale). This process wherein voxels are divided into groups and then visualization enhanced or diminished improves detection of subtle lesions. For example, the liver parenchyma should appear homogeneous on each of the 3 phases. Peering into the liver at each phase can help a certain masses that tend to fall in a band stand out from the rest of the liver parenchyma. All other tissues are made more translucent (e.g., sparse sampling) or are filtered. For example, for prioritized volume rendering is performed wherein voxels with higher priority and be displayed. All other tissues are made more translucent (e.g., via sparse sampling) or are filtered. This processed may be performed in accordance with a checklist.

(97) FIG. 12 illustrates utilization of range-based grouping as a display settings. Certain voxels within the liver having improved visualization through group-wise visual representation adjustment logic. In this embodiment, voxels are first divided into groups and then visual representation adjustment logic is applied wherein at least one group of voxels has a different visual representation adjustment logic as compared to at least one other group of voxels. For example, consider the liver. Assume that the liver is segmented and inside the segmented liver are voxels with Hounsfield Units ranging between 30-60. This embodiment improves visualization and inspection of liver (HU of 30-60) by dividing the voxels into groups (e.g., upper range of 50-60 HU, a middle range of 40-50 HU, and a lower range of 30-40 HU). Then, at three different time points one of the groups has enhanced visualization (e.g., rainbow color, midgray to white shades only, etc.) and the other two groups have diminished visualization (e.g., dark grayscale). Through user inputs, the first group (e.g., upper group with range of 50-60 HU) would be optimally displayed during the first time period and the middle and lower groups would have diminished visualization. In this illustration, the optimized group during the first time period has HU of 50-60 and is assigned yellow shades and the remainder of the groups (HU 30-50) have diminished visualization and are assigned dark gray to black shades. Then, through user inputs (or via automated process) the second group (e.g., middle group with range of 40-50 HU) would be optimally displayed during the second time period and the highest group (e.g., HU 50-60) and lowest group (e.g., HU 30-40) would have diminished visualization. In this illustration, the optimized group during the second time period has HU of 40-50 and is assigned light green shades and the remainder of the groups (HU 30-40 and HU 50-60) have diminished visualization and are assigned dark gray to black shades. Finally, through user inputs (or via an automated process) the third group (e.g., lower group with range of 30-40 HU) would have optimized visualization and the upper (e.g., HU 50-60) and middle groups (e.g., HU 40-50) would have diminished visualization. In this illustration, the optimized group during the third time period has HU of 30-40 and is assigned light purple shades and the remainder of the groups (HU 40-60) have diminished visualization and are assigned dark gray to black shades. This process of changing the appearance could be performed on 2D slices on a radiology monitor. Alternatively, it could be performed on 3D volumes using augmented reality, mixed reality or virtual reality displays and groups selected for diminished visualization would be preferentially filtered. Given that the groups that are filtered change over time (e.g., per user preference), then this filtering would be dynamic. This process wherein voxels are divided into groups and then visualization enhanced or diminished improves detection of subtle lesions. 1202 illustrates a text box, which states range grouping prior to implementing visual representation adjustment logic. 1203 is the liver at a first time point. 1204 is the liver at a second time point. 1205 is the liver at a third time point. The text box 1206 illustrates at time point 1 that the optimized band has Hounsfield Units of 50-60 and is shown in yellow shades and the diminished band has Hounsfield Units of 30-50 and is shown in dark gray to black shades. The text box 1207 illustrates at time point 2 that the optimized band has Hounsfield Units of 40-50 and is shown in light green shades and the diminished band has Hounsfield Units of 30-40 and 50-60 and is shown in dark gray to black shades. The text box 1208 illustrates at time point 3 that the optimized band has Hounsfield Units of 30-40 and is shown in light purple shades and the diminished band has Hounsfield Units of 40-60 and is shown in dark gray to black shades.

(98) FIG. 13 illustrates generating multiple simultaneous window/level settings for viewing of 3D datasets. Step 1300 illustrates performing a first windowing and leveling setting. Step 1301 illustrates performing segmentation of organs. 1302 illustrates generating a list of data that might be normal vs. abnormal. 1303 illustrates applying a first visual representation adjustment logic to standard window/level setting to voxels that are thought to be normal. 1304 illustrates applying second visual representation adjustment logic (false color) that are thought to be abnormal. 1305 illustrates an option to apply additional (third or more) visual representation adjustment logic to additional ranges.

(99) FIG. 14 illustrates challenges faced of visualizing certain ranges without multiple windowing. 1400 is a text box which provides description of the dataset. The following image is a sub-volume of a CT scan of the breast, inside of a volume-subtending 3D cursor. The sub-volume is comprised of approximately 100?100?100 or 1 million voxels. 1401 illustrates an image of the sub-volume. 1404 illustrates a first voxel that blends in with the rest of the gray shades. 1405 illustrates a second voxel that blends in with the rest of the gray shades. 1406 illustrates a third voxel that blends in with the rest of the gray shades. 1407 illustrates a text box, which states an embodiment of this patent is to be able to improve imaging by making some voxels (e.g., voxels with Hounsfield Unit range 70-75) to have a special visibility.

(100) FIG. 15 illustrates with overcoming challenges in visualizing certain ranges when using viewing multiple windowing. 1500 is a text box. Step #1 is to divide the voxels into ranges. The first range will include all voxels with Hounsfield Units between 70 and 75. Assume that there are 3 voxels in this first band. The second range will include all other voxels in the volume. Assuming the 100?100?100 matrix, that would equal 999,997 voxels in this second band. Step #2 is to assign visual representation adjustment logic to the first band of 3 voxels (e.g., color all voxels in this band yellow). Step #3 is to assign a different visual representation adjustment logic to the second band of 999,997 voxels (e.g., color voxels in this band varying shades of gray based on Hounsfield Units and conventional windowing and leveling settings. 1501 shows that the first voxel has been re-assigned a yellow color, from its previous gray-scale appearance. 1502 shows that the second voxel has been re-assigned a yellow color, from its previous gray-scale appearance. 1503 shows that the third voxel has been re-assigned a yellow color, from its previous gray-scale appearance.

(101) FIG. 16 illustrates a method of improving image quality. Step 1600 is to perform imaging examination. Step 1601 is to load an original imaging dataset of a patient wherein each pixel or voxel within the imaging dataset has an associated data unit. Step 1602 is to perform segmentation of the imaging dataset (e.g., segment a phantom, segment the anatomy, segment structures outside of the patient, etc.). Step 1603 is to select a first segmented structure for analysis. Step 1604 is to perform at least one measurement of the data unit(s) within the first segmented structure. Step 1605 is to determine the expected value(s) of the data unit(s) within the first segmented structure. Step 1606 is to determine at least one correction factor based on the difference between the at least one measurement(s) of the data unit(s) within the first segmented structure and the expected value(s) of the data unit(s) within the first segmented structure wherein the corrective factor can be applied to a second segmented structure to cause improved image quality. Step 1607 is to input the at least one corrective factor to modify the data units of at least one of the group comprising the first segmented structure and the second segmented structure in the original imaging dataset to create a modified imaging dataset wherein the modified imaging dataset has improved image quality.

(102) FIG. 17 illustrates a method of identifying data unit assurance markers. This also provides identifying which structure can be analyzed. Step 1700 is to determine which structure to analyze (e.g., below are examples of categories of data unit assurance markers). A first suggested option 1701 is to analyze a structure wherein the data value is known (e.g., phantom). A second suggested option 1702 is to analyze a structure wherein the data value is unknown, but the structure is homogeneous (e.g., air outside of the patient). A third suggested option 1703 is to analyze a structure wherein the data value is unknown and the structure is heterogeneous, but predictable (e.g., foam padding outside of the patient).

(103) FIG. 18 illustrates an example of analyzing density of air outside of the patient's body to correct for artifacts within the patient's body. Step 1800 is to perform a first imaging examination (e.g., CT scan performed at a first time point). Step 1801 is to move to a first slice in the imaging examination. Step 1802 is to segment air inside of the scanner field of view, but outside of human body. Step 1803 is to calculate average Hounsfield Unit of the air inside the scanner. Step 1804 is to determine if there are linear, triangular or trapezoid-like patterns within the air indicative of streak-type artifact (i.e., see if there are hyperattenuating regions or hypoattenuating regions). Step 1805 is to plot the coordinates of each linear, triangular, or trapezoid-like pattern within the air. Step 1806 is to within each linear, triangular or trapezoid-like pattern indicative of streak-type artifact, determine the distribution and trends of data units. Step 1807 is to determine what coordinates within the body would be affected if triangular or trapezoid-like pattern indicative of streak-type artifact were continued into the body. Step 1808 is to quantify (i.e., in Hounsfield units) the effect of the streak artifact of each coordinate in the body. Step 1809 is to alter the data units in the body to correct for the streak artifact. Note that this could be done for many different types of artifact in both CT and MRI. Step 1810 is if all slices have been examined, complete streak-artifact correction process. Step 1811 is if not all slices have been examined, then move to the next slice and return to step 1802. Nested do loops and other types of software strategies can be performed to accomplish these type steps.

(104) FIG. 19A illustrates a CT image of the head with window and level settings optimized for visualization of air. Note that there are groups of high density and low density inside of the segmented air. These correspond to areas of streak artifact.

(105) FIG. 19B illustrates a zoomed in CT image of the head with window and level settings optimized for visualization of air. Note that a trapezoid 1910 is shown in the image to denote the dark group.

(106) FIG. 19C illustrates a plot of voxels within the region of air, which can be used to determine the correction factor applied to voxels within the head. 1901 illustrates a trapezoid containing voxels inside of the head. 1900 illustrates a trapezoid containing voxels in the air outside of the head. Voxel 1902 is shown farthest away from the scalp and has a Hounsfield Unit of ?987. Voxel 1903 is mid-way from the scalp and has a Hounsfield Unit of ?976. Voxel 1904 is closest to the scalp and has a Hounsfield Unit of ?951. Voxel 1905 is in the superficial scalp and has a Hounsfield Unit of ?103. Voxel 1906 is in the brain and has a Hounsfield Unit of 24. A correction factor is applied. Voxel 1902 is corrected to ?1000. Voxel 1903 is corrected to ?1000. Voxel 1904 is corrected to ?1000. Voxel 1905 is corrected to ?90. Voxel 1906 is corrected to 30.

(107) FIG. 20 illustrates an example of placing phantoms inside of a table associated with a scanner. The patient 2000 is shown. The table 2001 is shown. Multiple phantoms 2002 are shown. The type of phantoms used can vary greatly (e.g., designed with varying size, materials, and locations). The design may also vary based on the type of examination and the indication of the examination.

(108) FIG. 21 illustrates an example of placing phantoms onto the patient. The patient 2100 is shown. Multiple phantoms 2101 are shown on the patient. Note that the phantoms could also be inside of the patient, preferably in the form of ingested capsules wherein each capsule contains at least one substance (e.g., water, oils, etc.). This would serve as internal landmarks. Other implantable phantoms (e.g., via surgery or needle placement) could be performed for certain conditions that require precision monitoring (e.g., brain tumor signal). The implantable phantoms may also be placed onto or within a variety of surgical devices (e.g., pacemaker, port-a-cath, etc.).

(109) FIG. 22 illustrates an example of placing phantoms onto a blanket. The blanket 2200 is shown. A first material 2201 is shown in the blanket. A second material 2202 is shown in the blanket. A third material 2203 is shown in the blanket. This could be used to wrap the patient and improve reliability of the data units of a particular examination.

(110) FIG. 23A illustrates a phantoms surrounding a patient in a single, circumferential layer. 2300 is the patient. 2301 is the single, circumferential layer of phantoms. Based on the measured signal in the phantom, corrections to the dataset can be made.

(111) FIG. 23B illustrates a phantoms surrounding a patient in a single layer, which is not circumferential. 2300 is the patient. 2302 is the single, non-circumferential layer of phantoms. Based on the measured signal in the phantom, corrections to the dataset can be made.

(112) FIG. 23C illustrates a phantoms surrounding a patient multiple, circumferential layers. 2300 is the patient. 2303 is the first layer of a circumferential layer of phantoms. 2304 is the second layer of a circumferential layer of phantoms. Based on the measured signal in the phantom, corrections to the dataset can be made.

(113) FIG. 23D illustrates a phantoms surrounding a patient in a multiple, non-circumferential layers. 2303 is the first layer of a circumferential layer of phantoms. 2305 is the first layer of a non-circumferential layer of phantoms. 2306 is the second layer of a non-circumferential layer of phantoms. Based on the measured signal in the phantom, corrections to the dataset can be made. These examples are illustrative only. A wide variety of placement of phantoms could be performed in a scanner for data unit assurance purposes as described in this patent.

(114) FIG. 24 illustrates multiple measurements in a radial fashion outward from the patient. This is an example of the phantom-based data unit correction. 2400 illustrates a cross-section of the patient. 2401 illustrates a first phantom, which contains only water. 2402 illustrates a second phantom, which contains only water. 2403 illustrates a third phantom, which contains only water. 2404 illustrates a fourth phantom, which contains only water. The preferred embodiment is multiple layers in the wrapping material, which allows factoring in differences in the data units over radial distance away from the patient. This provides increased accuracy.

(115) FIG. 25A illustrates a region of streak artifact going from scalp soft tissue into the air. Note that the third row 2500 contains voxels measuring 40, 40, ?990, ?990, and ?990. This third row 2500 illustrates streak artifact.

(116) FIG. 25B illustrates the modified dataset wherein the streak artifact is corrected. Note that the third row 2501 contains voxels measuring 50, 50, ?1000, ?1000, and ?1000. This third row 2500 illustrates correction of the streak artifact.

(117) FIG. 26 illustrates an x-ray detector attached to a phantom. 2600 is the x-ray detector. 2601 is the phantom. Please note that the number of phantoms, type(s) of phantoms and positions of phantoms can vary greatly.

(118) FIG. 27 illustrates a flow diagram describing an embodiment of this patent. Step 2700 is to perform segmentation of the image (e.g., segment liver, segment spleen, segment right kidney, etc.). Step 2701 is to move to an item on the radiology checklist (e.g., via user input or via eye tracking). Example checklists for radiology examinations can be obtained from the American College of Radiology website. For example, a CT scan of the abdomen includes the liver, spleen, gallbladder, pancreas, adrenal glands, kidneys, and so on. Step 2702 is to perform double windowing (e.g., via pre-set window/level settings or via user input). Step 2702 is to adjust windowing and leveling of the segmented checklist item in step 2701 to be analyzed to optimize viewing of the item. Step 2703 is to adjust windowing and leveling such that they are displayed in a manner so as to prevent the user from focusing on them (e.g., show in darker shades of gray) for all items other than the checklist item in step 2701. Step 2704 is to display simultaneously the optimized viewing of the item in 2702 and all items other than the checklist item 2703. Step 2705 is to review segmented item (e.g., visual analysis wherein the radiologist scrolls through the organ and searches for pathology).

(119) FIG. 28A illustrates three representative slices of the spleen with double windowing. The dataset is a CT scan of the chest with IV contrast and contained 181 slices. Slice 2800 is at position 115 of 181. Slice 2801 is at position 125 of 181. Slice 2802 is at position 133 of 181. In this embodiment, each of the organs on the radiologist's checklist have been segmented. In this example, the spleen is shown with double windowing. During viewing of the spleen, a grayscale setting for the spleen is set with a window level of 76 and window width of 419. This was windowed by a user to determine the best possible settings for visualization of the spleen. A grayscale setting for the remainder of the structures in the CT chest dataset is set with a window level of 533 and window width of 2285. On slice 2800, the spleen 2804 is optimized with its window and level setting. The remaining structures 2803 are displayed in a window and level setting, which has been designed to be darkened. In slice 2801, the spleen 2806 is optimized with its window and level setting. The remainder of the structures 2805 are displayed in a window and level setting, which has been designed to be darkened, settings described above. In slice 2802, the spleen 2808 is optimized with its window and level setting. The remainder of the structures 2807 are displayed in a window and level setting, which has been designed to be darkened, settings described above. This is an improvement for the radiologist because it helps the radiologist focus on the spleen more during the time at which the spleen is being examined. This also prevents the radiologist from getting distracted from other bright pixels within the dataset that are not related to the spleen.

(120) FIG. 28B illustrates three representative slices of the liver with double windowing. This is the same CT scan of the chest with IV contrast containing 181 slices as shown in FIG. 28A. Slice 2800 is at position 115 of 181. Slice 2801 is at position 125 of 181. Slice 2802 is at position 133 of 181. In this embodiment, each of the organs on the radiologist's checklist have been segmented. In this example, the liver is shown with double windowing. A grayscale setting for the liver is set with a window level of 117 and a window width of 166. This was windowed by a user to determine the best possible settings for visualization of the liver. During viewing of the liver, a grayscale setting for the remainder of the structures in the CT abdomen/pelvis dataset is set with a window level of 475 and window width of 2618. In slice 2800, the liver 2809 is optimized with its window and level setting. The remainder of the structures 2810 are displayed in a window and level setting, which has been designed to be darkened, settings described above. In slice 2801, the liver 2811 is optimized with its window and level setting. The remainder of the structures 2812 are displayed in a window and level setting, which has been designed to be darkened, settings described above. In slice 2802, the liver 2813 is optimized with its window and level setting. The remainder of the structures 2814 are displayed in a window and level setting, which has been designed to be darkened, settings described above. In some embodiments, more than one organ can be shown in optimal configurations and all other pixels/voxels assigned a window/level setting designed to be darkened. For example, the adrenal glands may both be shown in optimal window/leveling at the same time. A suggested terminology for this would be triple windowing or higher level windowing and leveling.

(121) FIG. 29 illustrates a flow diagram describing an embodiment of this patent. Step 2900 is to perform segmentation of the image (e.g., segment liver, segment spleen, segment right kidney, etc.). Step 2901 is to move to an item on the radiology checklist (e.g., via user input or via eye tracking). Step 2902 is to perform a modified segmentation of the item by methods disclosed in U.S. Patent Application No. 62/963,069. Step 2903 is to perform triple windowing (e.g., via pre-set window/level settings or via user input). In step 2904, for the checklist item in step 2901, adjust windowing and leveling of the segmented item to be analyzed to optimize viewing of the item. In Step 2905, for the modified item in step 2902, adjust windowing and leveling of the segmented item to be analyzed to blend in between steps 2904 and 2906. In Step 2906, for all items other than the checklist item in step 2901 and the modified segmented volume in step 2902, adjust windowing and leveling such that they are displayed in a manner so as to prevent the user from focusing on them (e.g., show in darker shades of gray). Step 2907 is to review segmented item (e.g., visual analysis wherein the radiologist scrolls through the organ and searches for pathology). Step 2908 is to assess whether all of the structures in the examination have been analyzed. If not, the next step is to proceed to step 2902. If yes, then the review is complete 2909. A suggested terminology for this process is called halo windowing and leveling.

(122) FIG. 30A illustrates a first example of halo windowing. 3000 illustrates the liver with a window level grayscale setting for the liver is set with a window level of 117 and a window width of 166. This was windowed by a user to determine the best possible settings for visualization of the liver. 3001 illustrates a modified segmented region with a halo appearance with a window level grayscale setting for the liver halo set with a window level of 33 and a window width of 427. This was windowed by a user to determine the best possible settings for visualization of the tissues immediately adjacent to the liver, which could have relevance to interpretation of the liver. 3002 illustrates all remaining tissues, which have a window level grayscale setting for the remainder of the structures in the CT chest dataset is set with a window level of 475 and window width of 2618. This overall process improves upon the existing art by modifying the images so that the user (e.g., radiologist) focuses on the liver during the liver portion of the examination and is not distracted by other bright voxels.

(123) FIG. 30B illustrates a second example of halo windowing. 3003 illustrates the liver with a window level grayscale setting for the liver is set with a window level of 117 and a window width of 166. This was windowed by a user to determine the best possible settings for visualization of the liver. 3001 illustrates a modified segmented region with a halo appearance with a window level grayscale setting for the liver halo set with a window level of 71 and window width of 357. Note that in FIG. 30A that the window level was 33 and a window width of 427 for the halo. This embodiment shows that the halo can be independently adjusted in its appearance as compared to the organ of interest and the remaining structures in the abdomen. This was windowed by a user to determine the best possible settings for visualization of the tissues immediately adjacent to the liver, which could have relevance to interpretation of the liver. 3005 illustrates all remaining tissues, which have a window level grayscale setting for the remainder of the structures in the CT chest dataset is set with a window level of 475 and window width of 2618. This overall process improves upon the existing art by modifying the images so that the user (e.g., radiologist) focuses on the liver during the liver portion of the examination and is not distracted by other bright voxels. In some embodiments, multiple halos with each halo having a unique window level setting can be performed so as to slowly alter the window level settings in a radial fashion outward from the organ.

(124) FIG. 31 illustrates multiple halos. Each layer of voxels outward would have a slightly more dim appearance. The organ of interest 3100 is given a first window level setting. A first halo 3101 immediately abutting the organ of interest 3000 is illustrated between the first line 3106 and the second line 3107 as a second window level setting. A second halo 3103 immediately abutting the first halo 3101 is illustrated between the second line 3107 and the third line 3108 as a third window level setting. A third halo 3104 immediately abutting the second halo 3103 is illustrated between the third line 3108 and the fourth line 3109 as a fourth window level setting. Finally, a four region of interest abutting the third halo 4310 is illustrated and has a fifth window level setting. Note that any of these regions can can be changed in thickness. For example, a halo could be one voxel thick, two voxels thick, three voxels thick, and so on. It is also important to note that the window level settings inside each individual halo can be adjusted independently from one another.

(125) FIG. 32 illustrates a method of generating a modified segmented structure. Step 100 is loading a three-dimensional imaging dataset. Step 3201 is performing segmentation of a structure within the imaging dataset. Step 3202 is determining the coordinates of a set of voxels that correspond to the outer surface of the segmented structure. Step 3203 is determining at least one layer of voxels external to the outer surface of the segmented structure wherein the one layer of voxels is contiguous with the outer surface of the segmented structure. Step 3204 is adding the at least one layer of voxels external to the outer surface of the segmented structure to generate a modified segmented structure. For example, the surface layer of voxels can be identified. The surface layer of voxels can be identified by going from either the center voxel of the organ in the outward direction and analyzing voxel properties to determine to voxel at the surface. Next, sequentially adds voxels from the outer shell of the structure in a step-wise fashion for a variable number of steps. During each step, additional voxels surrounding the segmented structure tissue would be added.

(126) FIG. 33A illustrates a substantially equal layer of voxels added to the outer surface of the segmented anatomic structure. 3300 illustrates the segmented anatomic structure, which in this case is the pancreas. 3301 illustrates the added layer of voxels to the outer surface of the segmented anatomic structure, which in this case is the fat in the retroperitoneum that surrounds the pancreas. Note that in this example, approximately the same amount of voxels is added on all sides of the segmented structure.

(127) FIG. 33B illustrates a CT scan showing a substantially equal layer of voxels added to the outer surface of the segmented anatomic structure to create a modified segmented anatomic structure. 3302 illustrates a line showing the outer boundary of the modified segmented structure. Note that some of the colon, spleen, vascular structure, retroperitoneal fat, and left adrenal gland are included in the modified segmented anatomic structure.

(128) FIG. 34 illustrates layers of voxels added in a non-uniform manner. 3400 illustrates a descriptive text box for a first example of non-uniform adding of layers. Voxel could be preferentially added to some portions of the periphery of the segmented anatomic structure based on voxel location relative to the segmented anatomic structure (e.g., superior aspect). 3401 illustrates the segmented anatomic structure (e.g., pancreas). 3402 illustrates the layer of voxels added to generate the modified segmented structure. Note that the layer of voxels added is thicker at the top of the segmented anatomic structure than at the bottom of the segmented anatomic structure; therefore, the amount of voxels added to the segmented anatomic structure is non-uniform. 3403 illustrates a descriptive text box for a second example of non-uniform adding of layers. Voxel units could be preferentially added to some portions of the of the periphery of the segmented anatomic structure based on voxel data unit (e.g., up to 4 extra layers of voxels are added if and only if those voxels have Hounsfield Units in the range of 10-20). 3404 illustrates the segmented anatomic structure, which in this case is the pancreas. 3405 illustrates the layer of voxels added to generate the modified segmented structure. 3406 illustrates some additional findings (e.g., fat stranding near the tail of the pancreas). 3407 illustrates an asymmetrically increased amount of voxels added in the region of the additional findings 3406. 3408 illustrates a descriptive text box for a third example of non-uniform adding of layers. Specific anatomic structure segmented (e.g., add a minimum of 10 voxels for the pancreas, but add a minimum of 20 voxels for the kidney). 3409 illustrates a first segmented anatomic structure, which in this case is the pancreas. 3410 illustrates the layer of voxels added to generate the modified segmented structure of the pancreas. 3411 illustrates a first segmented anatomic structure, which in this case is the kidney. 3412 illustrates the layer of voxels added to generate the modified segmented structure of the kidney.

(129) FIG. 35A illustrates an image of the abdomen. 3500 illustrates the segmented structure, which in this case is the pancreas. 3501 voxels added to generate the modified segmented structure. Note that the voxels are shown in light gray for illustrative purposes. 3502 illustrates the liver. 3503 illustrates the stomach. 3504 illustrates the spleen.

(130) FIG. 35B illustrates subtracting the all tissues except the modified segmented volume of the pancreas. 3500 illustrates the segmented structure, which in this case is the pancreas. 3501 voxels added to generate the modified segmented structure.

(131) FIG. 35C illustrates a CT scan showing subtraction of all tissues except the modified segmented volume of the pancreas. 3505 illustrates a line showing the outer boundary of the modified segmented structure. Note that some of the colon, spleen, vascular structure, retroperitoneal fat, and left adrenal gland are included in the modified segmented anatomic structure.

(132) FIG. 36A illustrates the segmented structure. 3600 illustrates the pancreas.

(133) FIG. 36B illustrates a CT scan showing the pancreas. 3601 shows the pancreas.

(134) FIG. 36C illustrates the segmented structure with a line denoting the outer boundary of the segmented structure. 3602 illustrates the pancreas. 3603 illustrates a line denoting the boundary of the pancreas.

(135) FIG. 36D illustrates a CT scan showing the pancreas with a line denoting the outer boundary of the segmented structure. 3604 illustrates a line denoting the outer boundary of the pancreas.

(136) FIG. 36E illustrates the segmented structure and the modified segmented structure with a line denoting the outer boundary of the segmented structure. 3605 illustrates the pancreas. 3606 illustrates the modified segmented structure wherein voxels are added. 3607 illustrates a line denoting the outer boundary of the modified segmented structure.

(137) FIG. 36F illustrates a CT scan showing the pancreas with a line denoting the outer boundary of the modified segmented structure. 3608 illustrates a line denoting the outer boundary of the modified segmented structure.

(138) FIG. 37A illustrates a first appearance of the line demarking the outer boundary of the modified segmented structure. 3700 illustrates the pancreas. 3701 illustrates the layers of voxels added surrounding the pancreas to generate the modified segmented volume. 3702 illustrates a solid black line demarking the outer layer of voxels in the modified segmented volume.

(139) FIG. 37B shows the corresponding line demarking the outer boundary of the modified segmented structure in a CT scan. 3703 illustrates a solid, black line representing the outer boundary of the modified segmented structure in a CT scan.

(140) FIG. 37C illustrates a second appearance of the line demarking the outer boundary of the modified segmented structure. 3704 illustrates the pancreas. 3705 illustrates the layers of voxels added surrounding the pancreas to generate the modified segmented volume. 3706 illustrates a dotted, black line demarking the outer layer of voxels in the modified segmented volume.

(141) FIG. 37D shows the corresponding dotted line demarking the outer boundary of the modified segmented structure in a CT scan. 3707 illustrates a dotted, black line representing the outer boundary of the modified segmented structure in a CT scan. Multiple lines could be shown to denote why certain voxels are included in the modified segmented volume or are not included in the modified segmented volume. A variety of appearances of the lines could be used, which includes, but is not limited to, the following: dotted; solid; dashed; thin weight; medium weight; thick weight; varying colors; flashing; or other appearances.

(142) FIG. 38 illustrates the inclusion of voxels in a non-contiguous manner. 3800 illustrates a text box. Voxel units could be added to generate a non-contiguous halo (e.g., human or AI selection of adjacent item that is relevant to the main anatomic structure of interest). 3801 illustrates the segmented structure. 3802 illustrates the additional layers of voxels added to generate the modified segmented structure. 3803 illustrates a dotted line to demark the outer boundary of the segmented anatomic structure. 3804 illustrates an additional findings which are relevant to the segmented anatomic structure (e.g., fat stranding near the pancreatic tail). 3805 illustrates layers of voxels added to the modified segmented structure. 3806 illustrates dotted lines to demark the outer boundary of the segmented anatomic structure. Note that voxels are included in a non-contiguous manner in this embodiment.

(143) FIG. 39A illustrates a first appearance of the segmented anatomic structure and the modified segmented anatomic structure. In this embodiment, the segmented structure is modified independently from the voxels in the modified segmented structure. 3900 illustrates the segmented structure. 3901 illustrates the layers of voxels added to generate the modified segmented structure. 3902 illustrates a line to demark the outer boundary of the modified segmented anatomic structure.

(144) FIG. 39B illustrates a first appearance of the segmented anatomic structure and the modified segmented anatomic structure. In this embodiment, the segmented structure is modified independently from the voxels in the modified segmented structure. 3900 illustrates the segmented structure, which appears the same as in FIG. 39A. 3901 illustrates the layers of voxels added to generate the modified segmented structure, which appears different from FIG. 39A since it is shown as a lighter shade of gray. 3902 illustrates a line to demark the outer boundary of the modified segmented anatomic structure, which appears the same as in FIG. 39A.

(145) FIG. 40 illustrates examples of factors which can determine the number of voxels included in the modified segmented structure. 4000 is a text box which illustrates patient demographics (e.g., age, gender, etc.). For example, infants have a smaller number of voxels due to small size. 4001 is a text box which illustrates metadata (e.g., BMI, labs, etc.). For example, BMIs>30 can have a larger number of voxels due to large size of retroperitoneal fat. For example, if the amylase and lipase are elevated, can have a larger number of voxels around the pancreas due to higher suspicion for pancreatitis and would not want to miss a fluid collection. 4002 is a text box which illustrates the type of pathology of clinical concern (e.g., infection, tumor, etc.). For example, if a stone is identified in the parotid duct, can have a larger number of voxels to improve detection of obstructive parotitis. 4003 is a text box which illustrates the type of pathology in the segmented structure. For example, if the gallbladder contains gallstones, can have a larger number of voxels to improve detection of cholecystitis.

(146) FIG. 41A illustrates the a modified segmented structure at a first time point. 4100 illustrates a segmented structure (e.g., pancreas). 4101 illustrates the layer of voxels added to create the modified segmented structure. 4102 illustrates a few imaging findings near the tail of the pancreas (e.g., fat stranding indicative of pancreatitis). 4103 illustrates the asymmetric enlargement of the modified segmented volume near the tail of the pancreas.

(147) FIG. 41B illustrates the a modified segmented structure at a second time point. 4104 illustrates a segmented structure (e.g., pancreas). 4105 illustrates the layer of voxels added to create the modified segmented structure. 4103 illustrates the asymmetric enlargement of the modified segmented volume near the tail of the pancreas. Note that the modified segmented structure is specifically designed to be asymmetrically enlarged at the tail of the pancreas to follow up the imaging findings seen on the first time point in FIG. 41A. No abnormalities are seen in the region of the tail of the pancreas indicating that the previously noted findings have resolved.

(148) FIG. 41C illustrates the comparison of a modified segmented structure at a first time point with the modified segmented structure at a second time point to determine interval change. 4107 illustrates the change in the appearance of the pancreas (e.g., the previously noted stranding has resolved).

(149) FIG. 42 illustrates inputting annotations into the modified segmented structure. 4200 illustrates the segmented structure (e.g., pancreas). 4201 illustrates the layers of voxels added surrounding the segmented structure 4200 to generate the modified segmented structure. 4202 illustrates an annotation (e.g., arrow) used to annotate a finding in the modified segmented structure.

(150) FIG. 43A illustrates a segmented structure (e.g., pancreas). 4300 illustrates the segmented structure (e.g., pancreas).

(151) FIG. 43B illustrates an example analysis performed on the segmented structure. In this example, a radiomics analysis is performed on the segmented structure (e.g., pancreas) and a histogram is shown.

(152) FIG. 43C illustrates a modified segmented structure (e.g., pancreas with some surrounding voxels added). 4301 illustrates the modified segmented structure (e.g., pancreas plus the additional layers of voxels added).

(153) FIG. 43D illustrates an example analysis performed on the segmented structure. In this example, a radiomics analysis is performed on the segmented structure (e.g., pancreas) and a histogram is shown. Note that a bimodal distribution is noted because in general the data units (e.g., Hounsfield Units) of the layers of voxels added (e.g., retroperitoneal fat) are different than the segmented structure.

(154) FIG. 43E illustrates a volume comprising only the surrounding voxels added, which would represent the difference between FIG. 43A and FIG. 43C. 4302 illustrates the voxels added.

(155) FIG. 43F illustrates an example analysis performed on the voxels added. In this example, a radiomics analysis is performed on the voxels added. Note that the initial peaks from FIG. 43B are no longer present and the analysis is only on the layers of voxels added (e.g., retroperitoneal fat).

(156) FIG. 44 illustrates a method of generating an inwardly generated modified segmented structure. 4400 illustrates loading a three-dimensional imaging dataset. 4401 illustrates performing segmentation of a structure within the imaging dataset. 4402 illustrates determining the coordinates of a set of voxels that correspond to the outer surface of the segmented structure. 4403 illustrates determining at least one layer of voxels internal to the outer surface of the segmented structure wherein the one layer of voxels is contiguous with the outer surface of the segmented structure and wherein the at least one layer of voxels internal to the outer surface of the segmented structure is conformal to the outer surface of the segmented structure. 4404 illustrates generating a modified segmented boundary wherein the modified segmented boundary is generated by moving inward by the at least one layer of voxels to the outer surface of the segmented structure. 4504 illustrates subtracting (or making transparent) the at least one layer of voxels internal to the outer surface of the segmented structure to generate a modified segmented structure.

(157) FIG. 45 illustrates an inwardly generated modified segmented structure/boundary. 4500 illustrates a segmented structure. 4501 illustrates a layer of voxels internal to the segmented structure. 4502 illustrates a modified segmented structure/boundary. This can be used for additional processing. For example, consider trying to isolate the contents inside the skull. When filtering by data unit (e.g., Hounsfield Units), there is some overlap between the contrast in the blood vessels and the skull so when the filtering occurs both skull and blood vessels are subtracted. So, clip planes have to be used. A current standard process is to use clip planes. A radiologist can place a clip plane at the vertex of the skull, the right side of the skull, the left side of the skull, the front of the skull and the back of the skull. Not only does this take a significant amount of time, it also can clip off some structures of interest, such as the vertex clip plane clipping off the top of the anterior cerebral arteries. Rather than doing this laborious and inadequate effort, a user can use a modified segmented structure/boundary to segment the skull. Then move at least one layer of voxels inward from the skull. This newly formed boundary/structure will preserve even the peripherally most located blood vessels. The newly formed structure can be filtered by data unit and the vessels can be preserved and well visualized.

(158) Several features, aspects, embodiments and implementations have been described. Nevertheless, it will be understood that a wide variety of modifications and combinations may be made without departing from the scope of the inventive concepts described herein. Accordingly, those modifications and combinations are within the scope of the following claims.