Method for data management and machine learning with fine resolution
11232562 · 2022-01-25
Assignee
Inventors
Cpc classification
G16H50/20
PHYSICS
A61B5/0075
HUMAN NECESSITIES
G16H50/30
PHYSICS
G06N7/01
PHYSICS
A61B8/085
HUMAN NECESSITIES
A61B2576/02
HUMAN NECESSITIES
A61B8/5223
HUMAN NECESSITIES
A61B5/055
HUMAN NECESSITIES
A61B6/5217
HUMAN NECESSITIES
A61B5/7275
HUMAN NECESSITIES
International classification
A61B5/055
HUMAN NECESSITIES
Abstract
A method for obtaining a probability in a 3D probability map, includes: obtaining at least one value of at least one parameter for each stop of a 3D moving window, wherein a first, second, third and fourth of the stops are partially overlapped, the first and second stops are shifted from each other by a distance equal to a first dimension of a computation voxel, the first and third stops are shifted from each other by a distance equal to a second dimension of the computation voxel, and the first and fourth stops are shifted from each other by a distance equal to a third dimension of the computation voxel; matching the at least one value to a classifier to obtain a first probability for each stop of the 3D moving window; and calculating a second probability for the computation voxel based on information associated with the first probabilities for the first through fourth stops.
Claims
1. A method for obtaining a probability map for a structure, comprising: providing, by an imaging system, a plurality of computation units each having a first dimension in a first direction, wherein each of the plurality of computation units is a unit of the probability map; moving, by the imaging system, a moving window across the structure in the first direction at a fixed interval of the first dimension to generate a plurality of stops of the moving window for the structure and obtaining, by the imaging system, at least one value of at least one imaging parameter for each of the plurality of stops of the moving window for the structure, wherein a first stop and second stop of the plurality of stops of the moving window are partially overlapped and are shifted from each other in the first direction by a distance substantially equal to the first dimension of the plurality of computation units, wherein the plurality of computation units comprises a first computation unit in the first and second stops; matching, by the imaging system, the at least one value of the at least one imaging parameter to a classifier to obtain a first probability of the event for each stop of the moving window; calculating, by the imaging system, a difference between the first probability of the event for the first stop and the first probability of the event for each of its neighboring stops of the plurality of stops of the moving window partially overlapping the first stop; calculating, by the imaging system, a ratio of each of the differences; determining, by the imaging system, if an absolute value of each of the ratios is less than or equal to a threshold value; defining, by the imaging system, a uniform portion of the probability map covered by the first stop and each of its neighboring stops of the plurality of stops of the moving window partially overlapping the first stop, wherein the second stop and first computation unit are in the uniform portion, and wherein the plurality of computation units comprises a second computation unit in the uniform portion and a third computation unit outside the uniform portion; assigning, by the imaging system, a constant probability of the event for each of the plurality of computation units in the uniform portion, wherein the constant probability of the event is associated with the first probability of the event for the first stop and the first probability of the event for each of its neighboring stops of the plurality of stops of the moving window partially overlapping the first stop; and calculating, by the imaging system, a second probability of the event for the third computation unit based on information associated with the first probability of the event for a third stop of the plurality of stops of the moving window overlapping the second and third computation units and partially overlapping the uniform portion, and the constant probability of the event for the second computation unit.
2. The method of claim 1, wherein each of the plurality of computation units has a second dimension in a second direction, wherein the plurality of stops of the moving window for the structure are further generated by an operating step comprising moving, by the imaging system, the moving window across the structure in the second direction at a fixed interval of the second dimension, wherein the first stop and a fourth stop of the plurality of stops of the moving window are partially overlapped and are shifted from each other in the second direction by a distance substantially equal to the second dimension of the plurality of computation units, wherein the fourth stop is in the uniform portion.
3. The method of claim 2, wherein each of the plurality of computation units has a third dimension in a third direction, wherein the plurality of stops of the moving window for the structure are further generated by an operating step comprising moving, by the imaging system, the moving window across the structure in the third direction at a fixed interval of the third dimension, wherein the first stop and a fifth stop of the plurality of stops of the moving window are partially overlapped and are shifted from each other in the third direction by a distance substantially equal to the third dimension of the plurality of computation units, wherein the fifth stop is in the uniform portion.
4. The method of claim 1, wherein the structure is a biological structure.
5. The method of claim 1, wherein the at least one imaging parameter comprises various magnetic resonance imaging (MRI) parameters.
6. The method of claim 1, wherein the event comprises occurrence of a cancer.
7. The method of claim 1, wherein the classifier comprises a Bayesian classifier.
8. A method for obtaining a computation map for a structure, comprising: providing, by an imaging system, a plurality of computation units each having a first dimension in a first direction, wherein each of the plurality of computation units is a unit of the computation map; moving, by the imaging system, a moving window across the structure in the first direction at a fixed interval of the first dimension to generate a plurality of stops of the moving window for the structure and obtaining, by the imaging system, a first value of an imaging parameter for each of the plurality of stops of the moving window for the structure, wherein a first stop and second stop of the plurality of stops of the moving window are partially overlapped and are shifted from each other in the first direction by a distance substantially equal to the first dimension of the plurality of computation units, wherein the plurality of computation units comprises a first computation unit in the first and second stops; calculating, by the imaging system, a difference between the first value for the first stop and the first value for each of its neighboring stops of the plurality of stops of the moving window partially overlapping the first stop; calculating, by the imaging system, a ratio of each of the differences; determining, by the imaging system, if an absolute value of each of the ratios is less than or equal to a threshold value; defining, by the imaging system, a uniform portion of the computation map covered by the first stop and each of its neighboring stops of the plurality of stops of the moving window partially overlapping the first stop, wherein the second stop and first computation unit are in the uniform portion, and wherein the plurality of computation units comprises a second computation unit in the uniform portion and a third computation unit outside the uniform portion; assigning, by the imaging system, a constant value for each of the plurality of computation units in the uniform portion, wherein the constant value is associated with the first value for the first stop and the first value for each of its neighboring stops of the plurality of stops of the moving window partially overlapping the first stop; and calculating, by the imaging system, a second value for the third computation unit based on information associated with the first value for a third stop of the plurality of stops of the moving window overlapping the second and third computation units and partially overlapping the uniform portion, and the constant value for the second computation unit.
9. The method of claim 8, wherein each of the plurality of computation units has a second dimension in a second direction, wherein the plurality of stops of the moving window for the structure are further generated by an operating step comprising moving, by the imaging system, the moving window across the structure in the second direction at a fixed interval of the second dimension, wherein the first stop and a fourth stop of the plurality of stops of the moving window are partially overlapped and are shifted from each other in the second direction by a distance substantially equal to the second dimension of the plurality of computation units, wherein the fourth stop is in the uniform portion.
10. The method of claim 9, wherein each of the plurality of computation units has a third dimension in a third direction, wherein the plurality of stops of the moving window for the structure are further generated by a step comprising moving, by the imaging system, the moving window across the structure in the third direction at a fixed interval of the third dimension, wherein the first stop and a fifth stop of the plurality of stops of the moving window are partially overlapped and are shifted from each other in the third direction by a distance substantially equal to the third dimension of the plurality of computation units, wherein the fifth stop is in the uniform portion.
11. The method of claim 8, wherein the structure is a biological structure.
12. The method of claim 8, wherein the imaging parameter comprises a magnetic resonance imaging (MRI) parameter.
13. The method of claim 8, wherein the imaging parameter comprises an infrared absorbance parameter.
14. A method for obtaining a computation map for a structure, comprising: providing, by an imaging system, a plurality of computation units each having a first dimension in a first direction, wherein each of the plurality of computation units is a unit of the computation map; moving, by the imaging system, a moving window across the structure in the first direction at a fixed interval of the first dimension to generate a plurality of stops of the moving window for the structure and obtaining, by the imaging system, a first value of an imaging parameter for each of the plurality of stops of the moving window for the structure, wherein a first stop and second stop of the plurality of stops of the moving window are partially overlapped and are shifted from each other in the first direction by a distance substantially equal to the first dimension of the plurality of computation units, wherein the plurality of computation units comprises a first computation unit in the first and second stops; calculating, by the imaging system, a difference between the first value for the first stop and the first value for each of its neighboring stops of the plurality of stops of the moving window partially overlapping the first stop; calculating, by the imaging system, a ratio of each of the differences; determining, by the imaging system, if an absolute value of each of the ratios is less than or equal to a threshold value; defining, by the imaging system, a uniform portion of the computation map covered by the first stop and each of its neighboring stops of the plurality of stops of the moving window partially overlapping the first stop, wherein the second stop and first computation unit are in the uniform portion; and assigning, by the imaging system, a constant value for each of the plurality of computation units in the uniform portion, wherein the constant value is associated with the first value for the first stop and the first value for each of its neighboring stops of the plurality of stops of the moving window partially overlapping the first stop.
15. The method of claim 14, wherein each of the plurality of computation units has a second dimension in a second direction, wherein the plurality of stops of the moving window for the structure are further generated by an operating step comprising moving, by the imaging system, the moving window across the structure in the second direction at a fixed interval of the second dimension, wherein the first stop and a third stop of the plurality of stops of the moving window are partially overlapped and are shifted from each other in the second direction by a distance substantially equal to the second dimension of the plurality of computation units, wherein the third stop is in the uniform portion.
16. The method of claim 15, wherein the first and second directions are substantially perpendicular to each other.
17. The method of claim 15, wherein each of the plurality of computation units has a third dimension in a third direction, wherein the plurality of stops of the moving window for the structure are further generated by an operating step comprising moving, by the imaging system, the moving window across the structure in the third direction at a fixed interval of the third dimension, wherein the first stop and a fourth stop of the plurality of stops of the moving window are partially overlapped and are shifted from each other in the third direction by a distance substantially equal to the third dimension of the plurality of computation units, wherein the fourth stop is in the uniform portion.
18. The method of claim 14, wherein the structure is a biological structure.
19. The method of claim 14, wherein the imaging parameter comprises a magnetic resonance imaging (MRI) parameter.
20. The method of claim 14, wherein the imaging parameter comprises an infrared absorbance parameter.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The drawings disclose illustrative embodiments of the present disclosure. They do not set forth all embodiments. Other embodiments may be used in addition or instead. Details that may be apparent or unnecessary may be omitted to save space or for more effective illustration. Conversely, some embodiments may be practiced without all of the details that are disclosed. When the same reference number or reference indicator appears in different drawings, it may refer to the same or like components or steps.
(2) Aspects of the disclosure may be more fully understood from the following description when read together with the accompanying drawings, which are to be regarded as illustrative in nature, and not as limiting. The drawings are not necessarily to scale, emphasis instead being placed on the principles of the disclosure. In the drawings:
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
(26)
(27)
(28)
(29)
(30)
(31)
(32)
(33)
(34)
(35)
(36)
(37)
(38)
(39)
(40)
(41)
(42)
(43)
(44)
(45)
(46)
(47)
(48)
(49)
(50)
(51)
(52)
(53)
(54)
(55)
(56)
(57)
(58)
(59) While certain embodiments are depicted in the drawings, one skilled in the art will appreciate that the embodiments depicted are illustrative and that variations of those shown, as well as other embodiments described herein, may be envisioned and practiced within the scope of the present disclosure.
DETAILED DESCRIPTION OF THE INVENTION
(60) Illustrative embodiments are now described. Other embodiments may be used in addition or instead. Details that may be apparent or unnecessary may be omitted to save space or for a more effective presentation. Conversely, some embodiments may be practiced without all of the details that are disclosed.
(61) First Aspect: E Operator for Better Resolution of Probabilities of Event in Two-Dimensional Region Via Big-Data Engineering Learning
(62) I-1. Probability Map Derived from Measured Values for Original Pixels of Two-Dimensional Original Map
(63)
(64) Next, in a step S22-2 for big-data engineering learning, a learnt resulting parameter, i.e., a probability CL.sub.m-n of an event, for each stop W.sub.m-n is calculated or obtained by matching the one or the set of values C.sub.m-n of the one or more imaging parameters for said each stop W.sub.m-n of the two-dimensional moving window 2 to a classifier such as Bayesian classifier. The probability CL.sub.m-n of the event for each stop W.sub.m-n of the two-dimensional moving window 2 is independent of an area of said each stop W.sub.m-n.
(65) Next, in a step S22-3 for deconvolution operation (E.sub.d), a probability dl.sub.k-l of the event for each computation pixel P.sub.k-l of the two-dimensional computational map 12 is iteratively updated or calculated, as illustrated in steps ST1-ST11 in following paragraphs in the first aspect, based on one or more of the probabilities CL.sub.m-n of the event for respective one or more of the stops W.sub.m-n each covering said each computation pixel P.sub.k-l, wherein said each computation pixel P.sub.k-l has a smaller area than that of each of the respective one or more of the stops W.sub.m-n of the two-dimensional moving window 2. The probability dl.sub.k-l of the event for each computational pixel P.sub.k-l is independent of an area of said each computational pixel P.sub.k-l.
(66) One aspect of the disclosure provides an algorithm, a method, or an operator, for transformation of data, dataset or information related to original or initial pixels (p.sub.i-j) at respective locations, x.sub.i-j's, of a 2D region to a new data, dataset or information in a final or computation pixels (P.sub.k-l) at related locations X.sub.k-l's, of the same 2D region, wherein i, j, k, l are positive integers, i from 1, 2, . . . , to I; j from 1, 2, . . . , to J; k from 1, 2, . . . , to K; l from l1, 2, . . . , to L. The transformation results in a new set of data, dataset or information of the final or computation pixels with a better resolution and a lower noise as compared to that of the original or initial pixels. K may be different from I and L may be different from J. For a better resolution and a lower noise, the area of each of the final or computation pixels is smaller than that of the original or initial pixels; that is K>I, and L>J. Alternatively, when I=K and J=L, X.sub.k-l can be the same as x.sub.i-j, wherein the noises due to measurement fluctuation in the data, dataset or information of the original or initial pixels are smeared-out. The 2D region may comprise I×J pixels in grids of original or initial pixels, wherein the size and numbers of pixels may be determined by a certain detector or sensor used in obtaining the data, dataset or information related to the original or initial pixels. The 2D region may as well comprise K×L pixels in grids of final or computation pixels, wherein the size and numbers of pixels may be generated for a desired resolution for analysis, diagnosis or a specific application. The data, dataset or information related to the original or initial pixels may be of a certain type, property, category or item (for example, MRI parameters) obtained from a certain detector or sensor. The data, dataset or information related to the final or computation pixels may be of a same type, property, category or item (as that, for example the MRI parameters, of the original or initial pixels) obtained from the transformation or computation. Alternatively, the data, dataset or information related to the original or initial pixels may be, for examples, the IR absorption images for a given range of wavenumbers, the Raman scattering images for a given range of wavenumbers, the fluorescent light images for a given range of wavenumbers, or the ultrasonic images of a human organ. The original or initial pixels have a dimension in one direction (for example, x direction) x.sub.op, and a dimension in the perpendicular direction (for example, y direction) y.sub.op; while the final pixels have a dimension in one direction (for example, x direction) X.sub.fp, and a dimension in the perpendicular direction (for example, y direction) Y.sub.fp. The final pixels may have the same dimensions (size) as that of the original pixels; or with each pixel having a size larger or smaller than the size of original or initial pixels, while both are in the same 2D region. The data, dataset or information in or related to, or describing each of the original or initial pixels (p.sub.i-j) can be a number, multiple numbers, a real number, multiple real numbers, a digitized number (for example a negative integer, 0, or a positive integer), multiple digitized numbers, a 0 or 1, multiple 0's or 1's, a scalar, multiple scalars, a vector, multiple vectors, or a tensor with degree of order 0, 1, 2, . . . , t, where t is an integer.
(67) The disclosed algorithm or operator comprises two operations, a convolution operation (E.sub.c) and the deconvolution operation (E.sub.d). E.sub.c and E.sub.d can be operated separately or together. When combining these two operations together, it is the Engineering operator (E operator), E=E.sub.d E.sub.c. The E operator, as well as the E.sub.c and E.sub.d operators will be described and specifies as follows.
(68) The original data, dataset or information in the original or initial pixels in a given 2D region is transformed to a data, dataset or information in stops of a moving window, with the data, dataset or information of the same type, property, category or item (for example, MRI parameters) as that (for example, a MRI parameters) of the original data, dataset or information in the original or initial pixels. The moving window plays a key role in the E operator or E algorithm. It is defined with some physical, computation, analytical, or statistical purposes for better resolution and lower noise. The size, shape, parameters or format of the moving window may become a default or standard size, shape, parameters or format in collecting, storing, computing, (statistically) analyzing data or information, or engineering learning or machine learning. Usually, the size, shape, parameters or format of the moving window is chosen to enclose at least several original or initial pixels, as well as at least several final or computation pixels. For example, the moving window size and shape can be defined in a 2D MRI slice with a total volume (moving window area times the thickness or height of the MRI slice) is equal to a volume of a biopsy sample; wherein the volume of a biopsy sample may be defined by the averaged volume of biopsy samples taken in the standard biopsy procedure using needles with popular or standard sizes. The moving window area mentioned above is defined as the size, shape, parameters or format of the moving window in the 2D region. The moving window may have a shape of a circle, an elliptic, a square or a rectangle. When the moving widow has a shape of circle, the maximum inscribed square may contain p×p original or initial pixels; or P×P final or computation pixels: wherein p and P are positive numbers, and is greater than or equal to 1. P, in some cases, is chosen to be a positive integer, and is greater than or equal to 2. When the moving widow has a shape of elliptic, the maximum inscribed rectangle may contain p×q original or initial pixels; or P×Q final or computation pixels: where p, q, P and Q are positive numbers, and are greater than or equal to 1. P and Q, in some cases, are chosen to be positive integers, and are greater than or equal to 2. When the moving widow has a shape of square, the square may contain p×p original or initial pixels; or P×P final or computation pixels: where p, and P are positive numbers, and are greater than or equal to 1. P, in some cases, is chosen to be a positive integer, and is greater than or equal to 2. When the moving widow has a shape of rectangle, the rectangle may contain p×q original or initial pixels; or P×Q final or computation pixels: where p, q, P and Q are positive numbers, and greater than or equal to 1. P and Q, in some cases, are chosen to be positive integers, and are greater than or equal to 2. The moving widow are stepping in the same 2D region by a step of X.sub.fp in the x direction and a step of Y.sub.fp in the y direction, and resulting in an array of densely populated and overlapped stops. Each stop overlaps its nearest neighbor stop with a step or shift of X.sub.fp or Y.sub.fp, in the x and y directions, respectively. Each stop in the 2D region comprises a number of original pixels, full or partial. The data, dataset or information for each stop is obtained by averaging over all the pixels enclosed by the stop. For some partially enclosed pixels, the averaging computation over these pixels can be done by weighing the enclosed area proportionally. The averaging can be done by linear averaging, Gaussian averaging or Lorentian averaging. In linear averaging, we assume the data, dataset or information in each stop of moving window is uniform. The above method transforms data, dataset or information in the original or initial pixels to data, dataset or information in stops of moving window; wherein the transform can be called a convolution. The stop of moving window at location X.sub.m-n is defined as W.sub.m-n, wherein m=1, 2, 3, 4, . . . , M, and n=1, 2, 3, 4, . . . , N. The data, dataset or information in or related to each stop of the moving window (W.sub.m-n) can be a number, multiple numbers, a real number, multiple real numbers, a digitized number (for example a negative integer, 0, or a positive integer), multiple digitized numbers, a 0 or 1, multiple 0's or 1's, a scalar, multiple scalars, a vector, multiple vectors, or a tensor with degree of order 0, 1, 2, . . . , t, where t is an integer. Since the moving window is stepping by the size of a final or computation pixel, the number of the stops is counted in the array of final or computation pixels. Assuming each stop of moving comprises P×Q computation pixels. The original matrix M.sub.op comprises I×J pixels and has I×J sets or elements or components of data, dataset or information. The convolution matrix M.sub.cw comprises (K−P+1)×(L−Q+1) stops of moving window, and has (K−P+1)×(L−Q+1) sets or elements or components of data, dataset or information. The E.sub.c operator transforms original matrix M.sub.op (comprising I×J sets or elements of data, dataset or information (for example, MRI parameters) describing or representing each original pixel in the given 2D region) to a convolution matrix M.sub.cw (comprising (K−P+1)×(L−Q+1) sets or elements of averaged data, dataset or information (for example, MRI parameters) describing or representing each stop of moving window in the given 2D region) can be expressed as:
E.sub.c(M.sub.op,W.sub.PQ)=M.sub.cw,
Wherein M.sub.op has dimension or size I×J, the moving window W.sub.PQ has dimension or size P×Q, and M.sub.cw has dimension or size (K−P+1)×(L−Q+1). The M.sub.cw comprise elements of data, dataset, or information of the same type, property, category or item as that of M.sub.op. For example, the elements in both M.sub.cw and M.sub.op are data, dataset or information related to the MRI parameters. Alternatively, the elements in both M.sub.cw and M.sub.op are data, dataset or information related to the IR absorption, Raman scattering, fluorescent light, or ultrasonic imaging.
(69) In this aspect, engineering learning or machine learning is performed using the data, dataset or information related to the moving window, or using the standard size, shape, parameters or format or dimensions of the moving window. The description and specification of the steps, processes and methods related to the convolution operator are the same as in the above. As described and specified above, the convolution operator E.sub.c transforms the original matrix M.sub.op (comprising data, dataset or information (for example, MRI parameters) describing or representing each original or initial pixel in the given 2D region) to a convolution matrix M.sub.cw (comprising averaged data, dataset or information (for example, MRI parameters) describing or representing each stop of moving window in the given 2D region). Through the engineering learning, machine learning or correlation, the data, dataset or information of the elements of M.sub.cw may be transformed to a data, dataset or information in a different type, property, item or category. For example, based on big data (accumulated data of correlated clinical biopsy analysis data and the measured MRI parameters for patients) and using (for example) Bayesian inference, the M.sub.op (elements of MRI parameters) can be transformed or constructed into a matrix of learning window ML.sub.w comprising elements of the probabilities of cancer occurrence. Since the 2D moving window is stepping by the size of a final or computation pixel, the number of the stops is counted in a 2D array of final or computation pixels. Each stop of 2D moving window comprises P×Q final or computation pixels. The original matrix M.sub.op comprises I×J pixels and has I×J sets or elements or components of data, dataset or information. The convolution matrix M.sub.cw and the learning matrix ML.sub.w both comprise (K−P+1)×(L−Q+1) stops of 2D moving window, and has (K−P+1)×(L−Q+1) sets or elements or components of data, dataset or information. The E.sub.c operator transforms original matrix M.sub.op (comprising I×J sets or elements of data, dataset or information (for example, MRI parameters) describing or representing each original pixel in the given 2D region) to a convolution matrix M.sub.cw (comprising (K−P+1)×(L−Q+1) sets or elements of averaged data, dataset or information (for example, MRI parameters) describing or representing each stop of 2D moving window in the given 2D region). The E.sub.1 operator transforms the convolution matrix M.sub.cw (comprising (K−P+1)×(L−Q+1) sets or elements of averaged data, dataset or information (for example, MRI parameters) describing or representing each stop of 2D moving window in the given 2D region) to a learning matrix ML.sub.w (comprising (K−P+1)×(L−Q+1) sets or elements of learned data, dataset or information (for example, the probability of the cancer occurrence) describing or representing each stop of 2D moving window in the given 2D region). The engineering learning operator (or the machine learning operator), E.sub.1, can be expressed as:
E.sub.1(M.sub.cw,W.sub.PQ)=ML.sub.w
(70) wherein the 2D moving window comprises P×Q final or computation pixels with P and Q in the x and y directions, respectively, and the stops W.sub.m-n 's are at locations with m and n final or computation pixels in the given 2D region, wherein m=1, 2, 3, . . . , M, and n=1, 2, 3, . . . , N. The data, dataset or information in or related to, or describing each element of the learning matrix ML.sub.w for the stop W.sub.m-n in the given 2D region is of a different type, property, category or item (for example, the probability of the occurrence of a cancer) as compared to that (for example, MRI parameters) in or related to, or describing each element of the convolution matrix M.sub.cw for the stop W.sub.m-n in the same given 2D region. While the data, dataset or information in or related to, or describing each element of the convolution matrix M.sub.cw for the stop W.sub.m-n in the given 2D region is of a same type, property, category or item (for example, MRI parameters) as compared to that (for example, MRI parameters) in or related to, or describing each element of the original matrix M.sub.op for the original or initial pixel in the same given 2D region. Alternatively, the data, dataset or information related to the original or initial pixels may be, for examples, the IR absorption images for a given range of wavenumbers, the Raman scattering images for a given range of wavenumbers, the fluorescent light images for a given range of wavenumbers, or the ultrasonic images of a human organ. As described and specified in the above, the moving window plays a key role in the engineering learning operator or algorithm (E operator). It is defined with some physical, computation, analytical, or statistical purposes. Furthermore, the size, shape, parameters or format of the moving window is used for the engineering learning or machine learning. The size, shape, parameters or format of the moving window may become a default or standard size or format in collecting, storing, computing, (statistically) analyzing data or information, or engineering learning or machine learning. The methods, algorithms or procedures of engineering learning or machine learning for transforming M.sub.cw to ML.sub.w may be, for example, using (i) statistics, for example, Baysian inference, (ii) connection or association, for example, neuro-computing, (iii) Symbolism: for example, induction or interpretation, (iv) analog, for example, resemblance, (v) evolution, for example, nature processes.
(71) Similar to the deconvolution of M.sub.cw described and specified above, the learning matrix ML.sub.w can be also deconvoluted to obtain a final or computational matrix ML.sub.dp. The size, shape, parameters or format of the final or computation pixels are described and specified as in the above. The deconvolution matrix ML.sub.dp comprises a final or computational data, dataset or information for each final or computation pixel in the given 2D region. The data, dataset or information in or related to, or describing each pixel P.sub.k-l of the final or computation pixels in the given 2D region are of the same type, property, category or item (for example, the probability of the occurrence of a cancer) as that (for example, the probability of the occurrence of a cancer) of the learned data, dataset or information of the elements in ML.sub.w for the stops W.sub.m-n of moving window. The data, dataset or information in or related to, or describing each pixel P.sub.k-l of the final or computation pixels in the given 2D region are of a different type, property, category or item (for example, the probability of the occurrence of a cancer) as that (for example, MRI parameters) of the data, dataset or information of the elements in M.sub.cw for the stops W.sub.m-n of moving window. The data, dataset or information in or related to, or describing each pixel P.sub.k-l of the final or computation pixels in the given 2D region are of a different type, property, category or item (for example, the probability of the occurrence of a cancer) as that (for example, MRI parameters) of the data, dataset or information of the elements in M.sub.op for the original or initial pixels x.sub.i-j. Alternatively, for examples, based on big data (accumulated data of correlated clinical biopsy analysis result or data and the measured IR absorption, Raman scattering data, fluorescent lights, or ultrasonic imaging from the correspondent biopsy samples of patients) and using, for example, Bayesian inference, the M.sub.op (IR absorption, Raman scattering data, fluorescent light intensity, or ultrasonic imaging) can be transformed or constructed into a matrix of learning window ML.sub.w comprising elements of the probabilities of cancer occurrence.
(72) The data, dataset or information in or related to, or describing each pixel P.sub.k-l of the final or computation pixels can be a number, multiple numbers, a real number, multiple real numbers, a digitized number (for example a negative integer, 0, or a positive integer), multiple digitized numbers, a 0 or 1, multiple 0's or 1's, a scalar, multiple scalars, a vector, multiple vectors, or a tensor with degree of order 0, 1, 2, . . . , t, where t is an integer. The deconvolution E.sub.d of the E operator obtains the data, dataset or information for each final or computation pixel by solving a set of linear equations with unknown computation pixel data (dl.sub.k-l's) and known data (CL.sub.m-n's) of stops of the moving windows. The linear equations can be established by equating the data, dataset or information for each moving window stop W.sub.m-n to the data, dataset or information averaged over all the final or computation pixels enclosed by the moving window stop (W.sub.m-n), dl.sub.k-l The averaging can be done by linear averaging, Gaussian averaging or Lorentian averaging of dl.sub.k-l's.
(73)
(74) Wherein dl.sub.k-l's are the data, dataset or information of the final or computation pixels enclosed or within by the stop of the moving window W.sub.m-n, wherein k is from k.sub.1 to k.sub.1+P−1, and l is from l.sub.1 to l.sub.1+Q−1, and m=1, 2, 3, . . . , K−P+1; and n=1, 2, 3, . . . , L−Q+1.
(75) There are (K−P+1)×(L−Q+1) equations with knows (CL.sub.m-n 's), and K×L unknowns (dl.sub.k-l's). The number of unknowns is larger than the number of equations (6) by (PL+KQ-PQ-K-L+P+Q−1). A method to increase number of knows and decrease number of unknowns will be described below by (1) finding uniform or constant data, dataset or information for the final or computation pixels in a region or regions of uniformity or approximately uniformity within the 2D region of interest, and/or (2) finding uniform or constant data, dataset or information for the final or computation pixels in a region or regions of uniformity or approximately uniformity extending from and near or along the boundary of the 2D region of interest. The above method (1) may provide a number of knows (known data for the computation pixels) equal to or larger than the number of (PL+KQ−PQ−K−L+P+Q−1) such that the number (K−P+1)×(L−Q+1) of the equations (6) may be solved. If the moving window comprises 3-by-3 computation pixels, the above method (2) may provide a number of knows (known data for the computation pixels) equal to or larger than the number of [(K+2)(L+2)−(K−P+3)×(L−Q+3)] such that the number (K−P+3)×(L−Q+3) of the equations (6) may be solved. The set of linear equations can be solved by a computer, device, machine, processor, system or tool iteratively. The initial guess of each of the unknowns (the data, dataset or information of final or computation pixels), dl.sub.k-l0, is obtained by averaging over all the stops of covering or enclosing the pixel. The contribution from each enclosing stop calculated by the area ratio of the overlapped area (A′.sub.m-n) to the area of that stop (A.sub.m-n). dl.sub.k-l0 can be obtained using A.sub.m-n, A′.sub.m-n and CL.sub.m-n:
(76)
(77) Wherein stops W.sub.m-n cover or enclose the final or computation pixel P.sub.k-l has stop indices m from m.sub.1 to m.sub.2, and n from n.sub.1 to n.sub.2. In the first iteration, we can calculate and obtain the first data, dataset or information for each stop of the moving window, CL.sub.m-n1's, by using initial guess dl.sub.k-l0's in equation (2). The iteration results in a solution ML.sub.dp(K×L) when the set of computation pixel data or information match the set of learning window data or information with errors or difference smaller than or equal to a specified value or number in the same 2D region. The E.sub.d operator can be expressed as:
E.sub.d(ML.sub.w,W.sub.PQ)=ML.sub.dp
(78) In another aspect of the disclosure, the convolution operator E.sub.c, the learning operator E.sub.1 and the deconvolution operator E.sub.d can be performed in sequence to get the full E operator. The E operator transform the original matrix M.sub.op (comprising elements of data, dataset or information for the I×J original or initial pixels and has I×J sets or elements or components of data or information) to the deconvolution matrix M.sub.dp (comprising elements of data, dataset or information for the K×L pixels and has K×L sets or elements or components of data or information) in the same given 2D region, through the convolution window matrix M.sub.cw (comprising (K−P+1)×(L−Q+1) sets or elements or components of data or information of the convolution window stops) and through the learning window matrix ML.sub.w (comprising (K−P+1)×(L−Q+1) sets or elements or components of data or information of the learning window stops). The E operator can be expressed as
E(M.sub.op(I×J))=E.sub.d(ML.sub.w((K−P+1)×(L−Q+1)))=E.sub.dE.sub.l(M.sub.cw((K−P+1)×(L−Q+1)))=E.sub.dE.sub.lE.sub.c(M.sub.op(I×J))=ML.sub.dp(K×L)
(79) In another aspect of the disclosure, this invention discloses the E operator in the linear algebra. The linear operations, such as addition (+), subtraction (−), multiplication by a scalar (d) or division by a scalar (/), are performed using the data or information of each stop of the moving window, (that is using the elements in the convolution matrix M.sub.cw or the elements in the learning matrix ML.sub.w), instead of using the data or information of the original or initial pixels (that is instead of using the elements in the convolution matrix M.sub.op). The moving window is used as a default or standard size, configuration or format for containing and providing data, dataset or information for analysis, comparison, computing, engineering learning or machine learning.
E(aΣ.sub.sC.sub.sM.sub.s)=M
(80) Where M.sub.s or M is a matrix of the convolution matrix M.sub.cw, or the learning matrix ML.sub.w, and C.sub.s are the real numbers, s is an integer from 1, 2, 3, . . . , S, with S a positive integer.
(81) A method described in the first aspect is performed using MRI detection and diagnosis as an example. The algorithm in the first aspect may be employed to transform, via the engineering learning E, the value sets C.sub.m-n, each having the values for various MRI parameters, for the respective stops W.sub.m-n of the 2D moving window into the computation pixel data dl.sub.k-l, i.e., probabilities of an event, for the respective computation pixels P.sub.k-l.
(82) Alternatively, each combination of computed tomography (CT) parameters, positron emission tomography (PET) parameters, single-photon emission computed tomography (SPECT) parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, and/or bioluminescence optical (BLO) parameters, ultrasound parameters, infrared absorbance parameters, camera-image parameters and/or visible-light-image parameters may also be taken for a value set C.sub.m-n for one of the stops W.sub.m-n of the 2D moving window in the first aspect. The data, dataset or information C.sub.m-n for the stops W.sub.m-n of the 2D moving window in the first aspect may be obtained from detection or analysis instruments, such as camera, microscope (optical or electronic), endoscope, detectors or spectrometer (visible light, fluorescent light, IR, UV or X-ray), ultrasonic machine or system, magnetic resonance imaging (MRI) machine or system, computed tomography (CT) machine or system, positron emission tomography (PET) machine or system, single-photon emission computed tomography (SPECT) machine or system, micro-PET machine or system, micro-SPECT machine or system, Raman spectrometer or system, and/or bioluminescence optical (BLO) machine or system, or other machine for obtaining molecular or structural imaging data.
(83) For further elaboration, an example for MRI detection is mentioned as below:
(84) Computing methods described in the present invention may be performed on any type of image, such as molecular and structural image (e.g., MRI image, CT image, PET image, SPECT image, micro-PET, micro-SPECT, Raman image, or bioluminescence optical (BLO) image), structural image (e.g., CT image or ultrasound image), fluoroscopy image, structure/tissue image, optical image, infrared image, X-ray image, or any combination of these types of images, based on a registered (multi-parametric) image dataset for the image. The registered (multi-parametric) image dataset may include multiple imaging data or parameters obtained from one or more modalities, such as MRI, PET, SPECT, CT, fluoroscopy, ultrasound imaging, BLO imaging, micro-PET, micro-SPECT, Raman imaging, structure/tissue imaging, optical imaging, infrared imaging, and/or X-ray imaging. For a patient, the registered (multi-parametric) image dataset may be created by aligning or registering in space all parameters obtained from different times or from various machines. Methods in first, second and third embodiments of the invention may be performed on a MRI image based on the registered (multi-parametric) image dataset, including, e.g., MRI parameters and/or PET parameters, for the MRI image.
(85) Referring to
(86) Some or all of the subjects for creating the big data database 70 may have been subjected to a treatment such as neoadjuvant chemotherapy or (preoperative) radiation therapy. Alternatively, some or all of the subjects for creating the big data database 70 are not subjected to a treatment such as neoadjuvant chemotherapy or (preoperative) radiation therapy. The imaging parameters in each of the data sets of the big data database 70 may be obtained from different modalities, including two or more of the following: MRI, PET, SPECT, CT, fluoroscopy, ultrasound imaging, BLO imaging, micro-PET, micro-SPECT, and Raman imaging. Accordingly, the imaging parameters in each of the data sets of the big data database 70 may include four or more types of MRI parameters depicted in
(87) In the case of the biopsied tissue obtained by a needle, the biopsied tissue may be long cylinder-shaped with a radius Rn, which is substantially equal to an inner radius of the needle, and a height tT normalized to the thickness T of the MRI slice. In the invention, the volume of the long cylinder-shaped biopsied tissue may be transformed into another shape, which may have a volume the same or about the same as the volume of the long cylinder-shaped biopsied tissue (or Volume of Interest, VOI, which may be π×Rn.sup.2×tT), for easy or meaningful computing purposes, for medical instrumentation purposes, or for clearer final data presentation purposes. For example, the long cylinder of the biopsied tissue with the radius Rn and height tT may be transformed into a planar cylinder to match the MRI slice thickness T. The planar cylinder, for example, may have a height equal to the MRI slice thickness T, a radius Rw equal to the radius Rn multiplied by the square root of the number of the registered images, and a volume the same or about the same as the volume of the biopsied tissue, i.e., VOL The radius Rw of the planner cylinder is used to define the size (e.g., the radius Rm) of a moving window MW in calculating a probability map for a patient (e.g., human). In the invention, the volume of the biopsied tissue, i.e., VOI, for each of the data sets, for example, may be substantially equal to the volume of the moving window MW to be used in calculating probability maps. In other words, the volume of the biopsied tissue, i.e., VOI, defines the size of the moving window MW to be used in calculating probability maps. Statistically, the moving window MW may be determined with the radius Rm, perpendicular to a thickness of the moving window MW, based on the statistical distribution or average of the radii Rw (calculated from multiple VOIs) associated with a subset data (e.g., the following subset data DB-1 or DB-2) from the big data database 70.
(88) The tissue-based information in each of the data sets of the big data database 70 may include (1) a biopsy result, data, information (i.e., pathologist diagnosis, for example cancer or no cancer) for the biopsied tissue, (2) mRNA data or expression patterns, (3) DNA data or mutation patterns (including that obtained from next generation sequencing), (4) ontologies, (5) biopsy related feature size or volume (including the radius Rn of the biopsied tissue, the volume of the biopsied tissue (i.e., VOI), and/or the height tT of the biopsied tissue), and (6) other histological and biomarker findings such as necrosis, apoptosis, percentage of cancer, increased hypoxia, vascular reorganization, and receptor expression levels such as estrogen, progesterone, HER2, and EPGR receptors. For example, regarding the tissue-based information of the big data database 70, each of the data sets may include specific long chain mRNA biomarkers from next generation sequencing that are predictive of metastasis-free survival, such as HOTAIR, RP11-278 L15.2-001, LINC00511-009, AC004231.2-001. The clinical data in each of the data sets of the big data database 70 may include the timing of treatment, demographic data (e.g., age, sex, race, weight, family type, and residence of the subject), and TNM staging depicted in, e.g.,
(89) Data of interest are selected from the big data database 70 into a subset, used to build a classifier CF. The subset from the big data database 70 may be selected for a specific application, such as prostate cancer, breast cancer, breast cancer after neoadjuvant chemotherapy, or prostate cancer after radiation. In the case of the subset selected for prostate cancer, the subset may include data in a tissue-based or biopsy-based subset data DB-1. In the case of the subset selected for breast cancer, the subset may include data in a tissue-based or biopsy-based subset data DB-2. Using suitable methods, such as statistical methods, correlation methods, big data methods, and/or learning and training methods, the classifier CF may be constructed or created based on a first group associated with a first data type or feature (e.g., prostate cancer or breast cancer) in the subset, a second group associated with a second data type or feature (e.g., non-prostate cancer or non-breast cancer) in the subset, and some or all of the variables in the subset associated with the first and second groups. Accordingly, the classifier CF for an event, such as the first data type or feature, may be created based on the subset associated with the event from the big data database 70. The event may be a biopsy-diagnosed tissue characteristic, such as having specific cancerous cells, or occurrence of prostate cancer or breast cancer.
(90) After the database 70 and the classifier CF are created or constructed, a probability map, composed of multiple computation pixels with the same size, is generated or constructed for, e.g., evaluating or determining the health status of a patient (e.g., human subject), the physical condition of an organ or other structure inside the patient's body, or the patient's progress and therapeutic effectiveness by the steps described below. First, an image of the patient is obtained by a device or system, such as MRI system. The image of the patient, for example, may be a molecular image (e.g., MRI image, PET image, SPECT image, micro-PET image, micro-SPECT image, Raman image, or BLO image) or other suitable image (e.g., CT image or ultrasound image). In addition, based on the radius Rm of the moving window MW obtained from the subset, e.g., the subset data DB-1 or DB-2, in the big data database 70, the size of the computation pixel, which becomes the basic unit of the probability map, is defined.
(91) If the moving window MW is circular, the biggest square inscribed in the moving window MW is then defined. Next, the biggest square inscribed in the moving window MW is divided into n.sup.2 small squares, i.e., cubes, each having a width Wsq, where n is an integer, such as 2, 3, 4, 5, 6, or more than 6. The divided squares define the size and shape of the computation pixels in the probability map for the image of the patient. For example, each of the computation pixels of the probability map may be defined as a square, i.e., cube, having the width Wsq and a volume the same or about the same as that of each of the divided squares. The moving window MW may move across the image of the patient at a regular step or interval of a fixed distance, e.g., substantially equal to the width Wsq (i.e., the width of the computation pixels), in the x and y directions. A stop of the moving window MW overlaps the neighboring stop of the moving window MW.
(92) Alternatively, the biggest square inscribed in the moving window MW may be divided into n rectangles each having a width Wrec and a length Lrec, where n is an integer, such as 2, 3, 4, 5, 6, 7, 8, or more than 8. The divided rectangles define the size and shape of the computation pixels in the probability map for the image of the patient. Each of the computation pixels of the probability map, for example, may be a rectangle having the width Wrec, the length Lrec, and a volume the same or about the same as that of each of the divided rectangles. The moving window MW may move across the patient's molecular image at a regular step or interval of a fixed distance, e.g., substantially equal to the width Wrec (i.e., the width of the computation pixels), in the x direction and at a regular step or interval of a fixed distance, e.g., substantially equal to the length Lrec (i.e., the length of the computation pixels), in the y direction. A stop of the moving window MW overlaps the neighboring stop of the moving window MW. In an alternative embodiment, each of the stops of the moving window MW may have a width, length or diameter less than the side length (e.g., the width or length) of the machine-defined original pixels in the image of the patient.
(93) After the size and shape of the computation pixels are obtained or defined, the stepping of the moving window MW and the overlapping between two neighboring stops of the moving window MW can then be determined. Multiple values of specific imaging parameters for each stop of the moving window MW may be obtained from the patient's image and/or different parameter maps (e.g., MRI parameter map(s), PET parameter map(s) and/or CT parameter map(s)) registered to the patient's image. The specific imaging parameters may include two or more of the following: MRI parameters, PET parameters, SPECT parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, BLO parameters, CT parameters, and ultrasound imaging parameters. Each of the specific imaging parameters for each stop of the moving window MW, for example, may have a value calculated based on an average of measured values, for said each of the specific imaging parameters, for machine-defined original pixels of the patient's image inside said each stop of the moving window MW. In the case that some machine-defined original pixels of the patient's image only partially inside that stop of the moving window MW, the average can be weighed by the area proportion. The specific imaging parameters of different modalities may be obtained from registered image sets (or registered value sets or parameter maps), and rigid and non-rigid standard registration techniques may be used to get each section of anatomy into the same exact coordinate location on each of the registered (multi-parametric) image dataset.
(94) A registered (multi-parametric) image dataset may be created for the patient to include multiple registered images (including two or more of the following: MRI slice images, PET images, SPECT images, micro-PET images, micro-SPECT images, Raman images, BLO images, CT images, and ultrasound images) and/or corresponding imaging parameters (including two or more of the following: MRI parameters, PET parameters, SPECT parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, BLO parameters, CT parameters, and/or ultrasound imaging parameters) obtained from various equipment, machines, or devices or from a defined time-point (e.g., specific date) or time range (e.g., within five days after treatment). Each of the imaging parameters in the patient's registered (multi-parametric) image dataset requires alignment or registration. The registration can be done by, for examples, using unique anatomical marks, structures, tissues, geometry, and/or shapes or using mathematical algorithms and computer pattern recognition. The values of the specific imaging parameters for each stop of the moving window MW, for example, may be obtained from the registered (multi-parametric) image dataset for the patient.
(95) Next, the specific imaging parameters for each stop of the moving window MW may be reduced using, e.g., subset selection, aggregation, and dimensionality reduction into a parameter set for said each stop of the moving window MW. In other words, the parameter set includes values for independent imaging parameters. The imaging parameters used in the parameter set may have multiple types, such as two types, more than two types, more than three types, or more than four types, independent from each other or one another, or may have a single type. For example, the imaging parameters used in the parameter set may include (a) MRI parameters and PET parameters, (b) MRI parameters and SPET parameters, (c) MRI parameters and CT parameters, (d) MRI parameters and ultrasound imaging parameters, (e) Raman imaging parameters and CT parameters, (f) Raman imaging parameters and ultrasound imaging parameters, (g) MRI parameters, PET parameters, and ultrasound imaging parameters, or (h) MRI parameters, PET parameters, and CT parameters.
(96) Next, the parameter set for each stop of the moving window MW is matched to the classifier CF to obtain a probability PW or CL.sub.m-n of the event for said each stop of the moving window MW. After the probabilities PWs or CL.sub.m-n of the event for the stops of the moving window MW are obtained, an algorithm is performed based on the probabilities PWs or CL.sub.m-n of the event for the stops of the moving window MW to compute probabilities of the event for the computation pixels, as mentioned in the following steps ST1-ST11. In the step ST1, a first or initial probability PV1 for each of the computation pixels, for example, may be calculated or assumed based on an average of the probabilities PWs or CL.sub.m-n of the event for the stops of the moving window MW overlapping or covering said each of the computation pixels. In the step ST2, a first probability guess PG1 for each stop of the moving window MW is calculated by averaging the first or initial probabilities PV1s (obtained in the step ST1) of all the computation pixels inside said each stop of the moving widow MW. In the step ST3, the first probability guess PG1 for each stop of the moving window MW is compared with the probability PW or CL.sub.m-n of the event for said each stop of the moving window MW by, e.g., subtracting the probability PW or CL.sub.m-n of the event from the first probability guess PG1 so that a first difference DW1 (DW1=PG1−PW) between the first probability guess PG1 and the probability PW or CL.sub.m-n of the event for said each stop of the moving window MW is obtained. In the step ST4, a first comparison is performed to determine whether an absolute value of the first difference DW1 for each stop of the moving window MW is less than or equal to a preset threshold error. If any one of the absolute values of all the first differences DW is is greater than the preset threshold error, the step ST5 continues. If the absolute values of all the first differences DW1s are less than or equal to the preset threshold error, the step ST11 continues. In the step ST5, a first error correction factor (ECF1) for each of the computation pixels is calculated by, e.g., summing error correction contributions from the stops of the moving window MW overlapping or covering said each of the computation pixels. For example, if there are four stops of the moving window MW overlapping or covering one of the computation pixels, each of the error correction contributions to said one of the computation pixels is calculated by obtaining an area ratio of an overlapped area between said one of the computation pixels and a corresponding one of the four stops to an area of the biggest square inscribed in the corresponding one of the four stops, and then multiplying the first difference DW1 for the corresponding one of the four stops by the area ratio. In the step ST6, a second probability PV2 for each of the computation pixels is calculated by subtracting the first error correction factor ECF1 for said each of the computation pixels from the first or initial probability PV1 for said each of the computation pixels (PV2=PV1−ECF1). In the step ST7, a second probability guess PG2 for each stop of the moving window MW is calculated by averaging the second probabilities PV2s (obtained in the step ST6) of all the computation pixels inside said each stop of the moving widow MW. In the step ST8, the second probability guess PG2 for each stop of the moving window MW is compared with the probability PW or CL.sub.m-n of the event for said each stop of the moving window MW by, e.g., subtracting the probability PW or CL.sub.m-n of the event from the second probability guess PG2 so that a second difference DW2 (DW2=PG2−PW) between the second probability guess PG2 and the probability PW or CL.sub.m-n of the event for said each stop of the moving window MW is obtained. In the step S9, a second comparison is performed to determine whether an absolute value of the second difference DW2 for each stop of the moving window MW is less than or equal the preset threshold error. If any one of the absolute values of all the second differences DW2s is greater than the preset threshold error, the step ST10 continues. If the absolute values of all the second differences DW2s are less than or equal to the preset threshold error, the step ST11 continues. In the step ST10, the steps ST5-ST9 are repeated or iterated, using the newly obtained the n.sup.th difference DWn between the n.sup.th probability guess PGn and the probability PW or CL.sub.m-n of the event for each stop of the moving window MW for calculation in the (n+1).sup.th iteration, until an absolute value of the (n+1).sup.th difference DW(n+1) for each stop of the moving window MW is equal to or less than the preset threshold error (Note: PV1, PG1 and DW1 for the first iteration, ECF1, PV2, PG2 and DW2 for the second iteration, and ECF(n−1), PVn, PGn and DWn for the n.sup.th iteration). In the step ST11, the first or initial probabilities PV1s in the first iteration, i.e., the steps ST1-ST4, the second probabilities PV2s in the second iteration, i.e., the steps ST5-ST9, or the (n+1).sup.th probabilities PV(n+1)s in the (n+1).sup.th iteration, i.e., the step ST10, are used to form the probability map. The probabilities of the event for the computation pixels are obtained using the above method, procedure or algorithm, based on the overlapped stops of the moving window MW, to form the probability map of the event for the image (e.g., patient's MRI slice) for the patient having imaging information (e.g., molecular imaging information). The above process is performed to generate the moving window MW across the image in the x and y directions to create a two-dimensional (2D) probability map. In order to obtain a three-dimensional (3D) probability map, the above process may be applied to each of all images of the patient in the z direction perpendicular to the x and y directions.
(97) Description of Subset Data DB-1:
(98) Referring to
(99) The MRI parameters in the columns A-O of the subset data DB-1 are T1 mapping, T2 raw signal, T2 mapping, delta Ktrans (A Ktrans), tau, Dt IVIM, fp IVIM, ADC (high b-values), nADC (high b-values), R*, Ktrans from Tofts Model (TM), Ktrans from Extended Tofts Model (ETM), Ktrans from Shutterspeed Model (SSM), Ve from TM, and Ve from SSM. For more information about the MRI parameters in the subset data DB-1, please refer to
(100) Measured values in the respective columns T, U and V of the subset data DB-1 are Gleason scores associated with the respective prostate biopsy tissues and primary and secondary Gleason grades associated with the Gleason scores;
(101) Referring to
(102) Description of Subset Data DB-2:
(103) Referring to
(104) The MRI parameters in the columns A-O, R, and S of the subset data DB-2 are T1 mapping, T2 raw signal, T2 mapping, delta Ktrans (Δ Ktrans), tau, Dt IVIM, fp IVIM, ADC (high b-values), R*, Ktrans from Tofts Model (TM), Ktrans from Extended Tofts Model (ETM), Ktrans from Shutterspeed Model (SSM), Ve from TM, Ve from SSM, kep from Tofts Model (TM), kep from Shutterspeed Model (SSM), and mean diffusivity (MD) from diffusion tensor imaging (DTI). For more information about the MRI parameters in the subset data DB-2, please refer to
(105) Multiple values in the column AC of the subset data DB-2 may be the diameters of the breast biopsy tissues, and the diameter of each of the breast biopsy tissues may be substantially equal to an inner diameter of a cylinder needle, through which a circular or round hole passes for receiving said each of the breast biopsy tissues. Alternatively, the values in the column AC of the subset data DB-2 may be the widths of the breast biopsy tissues, and the width of each of the breast biopsy tissues may be substantially equal to an inner width of a needle, through which a square or rectangular hole passes for receiving said each of the breast biopsy tissues. The clinical or pathology parameters in the columns AI-AT of the subset data DB-2 are estrogen hormone receptor positive (ER+), progesterone hormone receptor positive (PR+), HER2/neu hormone receptor positive (HER2/neu+), immunohistochemistry subtype, path, BIRADS, Oncotype DX score, primary tumor (T), regional lymph nodes (N), distant metastasis (M), tumor size, and location. For more information about the clinical or pathology parameters in the subset data DB-2, please refer to
(106) Referring to
(107) A similar subset data like the subset data DB-1 or DB-2 may be established from the big data database 70 for generating probability maps for brain cancer, liver cancer, lung cancer, rectal cancer, sarcomas, cervical cancer, or cancer metastasis to any organ such as liver, bone, and brain. In this case, the subset data may include multiple data sets, each of which may include: (1) measured values for MRI parameters (e.g., those in the columns A-O, R, and S of the subset data DB-2) associated with a biopsy tissue (e.g., biopsied brain sample, biopsied liver sample, biopsied lung sample, biopsied rectal sample, biopsied sarcomas sample, or biopsied cervix sample) obtained from a subject (e.g., human); (2) processed parameters (e.g., those in the columns P and Q of the subset data DB-2) associated with the biopsy tissue; (3) a result or pathologist diagnosis of the biopsy tissue, such as cancer, normal tissue, or benign condition; (4) sample characters (e.g., those in the columns S-X of the subset data DB-1) associated with the biopsy tissue; (5) MRI characters (e.g., those in the columns Y, Z and AA of the subset data DB-1) associated with MRI slices registered to respective regions, portions, locations or volumes of the biopsy tissue; (6) a PET parameter (e.g., SUVmax depicted in
(108) Description of Biopsy Tissue, MRI Slices Registered to the Biopsy Tissue, and MRI Parameters for the Biopsy Tissue:
(109) Referring to
(110) The core needle biopsy is a procedure used to determine whether an abnormality or a suspicious area of an organ (e.g., prostate or breast) is a cancer, a normal tissue, or a benign condition or to determine any other tissue characteristic such as mRNA expression, receptor status, and molecular tissue characteristics. With regard to MRI-guided needle biopsy, magnetic resonance (MR) imaging may be used to guide a cylinder needle to the abnormality or the suspicious area so that a piece of tissue, such as the biopsy tissue 90, is removed from the abnormality or the suspicious area by the cylinder needle, and the removed tissue is then sent to be examined by pathology.
(111) During or before the core needle biopsy (e.g., MRI-guided needle biopsy), parallel MRI slices SI.sub.1 through SI.sub.N registered to multiple respective regions, portions, locations or volumes of the tissue 90 may be obtained. The number of the registered MRI slices SI.sub.1-SI.sub.N may range from, equal to or greater than 2 up to, equal to or less than 10. The registered MRI slices SI.sub.1-SI.sub.N may have the same slice thickness T, e.g., ranging from, equal to or greater than 1 millimeter up to, equal to or less than 10 millimeters, and more preferably ranging from, equal to or greater than 3 millimeters up to, equal to or less than 5 millimeters.
(112) Referring to
(113) Regions, i.e., portions, locations or volumes, of interest (ROIs) 94 of the respective MRI slices SI.sub.1-SI.sub.N are registered to and aligned with the respective regions, portions, locations or volumes of the biopsy tissue 90 to determine or calculate values of MRI parameters for the regions, portions, locations or volumes of the biopsy tissue 90. The ROIs 94 of the MRI slices SI.sub.1-SI.sub.N may have the same diameter, substantially equal to the diameter D1 of the biopsy tissue 90, i.e., the inner diameter of the needle for taking the biopsy tissue 90, and may have a total volume covering and substantially equaling the volume of the biopsy tissue 90. As shown in
(114) Taking an example of T1 mapping, in the case of (1) four MRI slices SI.sub.1-SI.sub.4 having four respective regions, portions, locations or volumes registered to respective quarters of the biopsy tissue 90 and (2) the ROI 94 of each of the MRI slices SI.sub.1-SI.sub.4 covering or overlapping the six machine-defined original pixels 96a-96f, values of T1 mapping for the machine-defined original pixels 96a-96f in each of the MRI slices SI.sub.1-SI.sub.4 and the percentages of the areas AI-A6 occupying the ROI 94 of each of the MRI slices SI.sub.1-SI.sub.4 are assumed as shown in
(115) The volume of the long cylinder-shaped biopsied tissue 90 may be transformed into another shape, which may have a volume the same or about the same as the volume of the long cylinder-shaped biopsied tissue 90 (or Volume of Interest (VOI), which may be Rn.sup.2×tT, where Rn is the radius of the biopsied tissue 90, and tT is the height of the biopsied tissue 90), for easy or meaningful computing purposes, for medical instrumentation purposes, or for clearer final data presentation purposes. For example, referring to
(116) Further, each of biopsy tissues provided for pathologist diagnoses in a subset data, e.g., DB-1 or DB-2, of the big data database 70 may have a corresponding planar cylinder 98 with its radius Rw, and data (such as pathologist diagnosis and values of imaging parameters) for said each of the biopsy tissues in the subset data, e.g., DB-1 or DB-2, of the big data database 70 may be considered as those for the corresponding planar cylinder 98. Statistically, the moving window MW may be determined with the radius Rm, perpendicular to a thickness of the moving window MW, based on the statistical distribution or average of the radii Rw of the planar cylinders 98 transformed from the volumes of the biopsy tissues provided for the pathologist diagnoses in the subset data, e.g., DB-1 or DB-2, of the big data database 70. In the invention, each of the biopsy tissues provided for the pathologist diagnoses in the subset data, e.g., DB-1 or DB-2, of the big data database 70, for example, may have a volume, i.e., VOI, substantially equal to the volume of the moving window MW to be used in calculating one or more probability maps. In other words, the volume of the biopsy tissue, i.e., VOI, defines the size (e.g., the radius Rm) of the moving window MW to be used in calculating one or more probability maps.
(117) Each of the prostate biopsy tissues provided for the pathologist diagnoses in the subset data DB-1 may be referred to the illustration of the biopsy tissue 90. In the column W of the subset data DB-1, the diameter of each of the prostate biopsy tissues may be referred to the illustration of the diameter D1 of the biopsy tissue 90. The MRI slices registered to the respective regions, portions, locations or volumes of each of the prostate biopsy tissues provided for the pathologist diagnoses in the subset data DB-1 may be referred to the illustration of the MRI slices SI.sub.1-SI.sub.N registered to the respective regions, portions, locations or volumes of the biopsy tissue 90. The values of the MRI parameters for each of the prostate biopsy tissues, i.e., for each of the corresponding planar cylinders 98, in the respective columns A-O of the subset data DB-1 may be calculated as the values of the MRI parameters for the whole biopsy tissue 90, i.e., for the planar cylinder 98 transformed from the volume of the biopsy tissue 90, are calculated. In the column Z of the subset data DB-1, the MRI slices registered to the respective regions, portions, locations or volumes of each of the prostate biopsy tissues may have the same area resolution, which may be referred to the illustration of the area resolution of the MRI slices SI.sub.1-SI.sub.N registered to the respective regions, portions, locations or volumes of the biopsy tissue 90. In the column AA of the subset data DB-1, the MRI slices registered to the respective regions, portions, locations or volumes of each of the prostate biopsy tissues may have the same slice thickness, which may be referred to the illustration of the slice thickness T of the MRI slices SI.sub.1-SI.sub.N.
(118) In the column S of the subset data DB-1, the percentage of cancer for the whole volume of the prostate biopsy tissue in each of all or some of the data sets may be replaced by the percentage of cancer for a partial volume of the prostate biopsy tissue; a MRI slice is imaged for and registered to the partial volume of the prostate biopsy tissue. In this case, the MRI parameters, in the columns A-O of the subset data DB-1, that are in said each of all or some of the data sets are shown for a ROI of the MRI slice registered to the partial volume of the prostate biopsy tissue. The ROI of the MRI slice covers or overlaps multiple machine-defined original pixels in the MRI slice, and each of the MRI parameters for the ROI of the MRI slice may be calculated by summing measured values of said each of the MRI parameters for the machine-defined original pixels weighed or multiplied by respective percentages of areas, overlapping with the respective machine-defined original pixels in the ROI of the MRI slice, occupying the ROI of the MRI slice. Multiple values for the MRI parameters for the ROI of the MRI slice may be derived from different parameter maps registered to the partial volume of the prostate biopsy tissue. In an alternative example, the values for some of the MRI parameters for the ROI of the MRI slice may be derived from different parameter maps registered to the partial volume of the prostate biopsy tissue, and the values for the others may be derived from the same parameter map registered to the partial volume of the prostate biopsy tissue.
(119) Each of the breast biopsy tissues provided for the pathologist diagnoses in the subset data DB-2 may be referred to the illustration of the biopsy tissue 90. In the column AC of the subset data DB-2, the diameter of each of the breast biopsy tissues may be referred to the illustration of the diameter D1 of the biopsy tissue 90. The MRI slices registered to the respective regions, portions, locations or volumes of each of the breast biopsy tissues provided for the pathologist diagnoses in the subset data DB-2 may be referred to the illustration of the MRI slices SI.sub.1-SI.sub.N registered to the respective regions, portions, locations or volumes of the biopsy tissue 90. The values of the MRI parameters for each of the breast biopsy tissues, i.e., for each of the corresponding planar cylinders 98, in the respective columns A-O, R, and S of the subset data DB-2 may be calculated as the values of the MRI parameters for the whole biopsy tissue 90, i.e., for the planar cylinder 98 transformed from the volume of the biopsy tissue 90, are calculated. In the column AF of the subset data DB-2, the MRI slices registered to the respective regions, portions, locations or volumes of each of the breast biopsy tissues may have the same area resolution, which may be referred to the illustration of the area resolution of the MRI slices SI.sub.1-SI.sub.N registered to the respective regions, portions, locations or volumes of the biopsy tissue 90. In the column AG of the subset data DB-2, the MRI slices registered to the respective regions, portions, locations or volumes of each of the breast biopsy tissues may have the same slice thickness, which may be referred to the illustration of the slice thickness T of the MRI slices SI.sub.1-SI.sub.N.
(120) In the column AB of the subset data DB-2, the percentage of cancer for the whole volume of the breast biopsy tissue in each of all or some of the data sets may be replaced by the percentage of cancer for a partial volume of the breast biopsy tissue; a MRI slice is imaged for and registered to the partial volume of the breast biopsy tissue. In this case, the MRI parameters, in the columns A-O, R, and S of the subset data DB-2, that are in said each of all or some of the data sets are shown for a ROI of the MRI slice registered to the partial volume of the breast biopsy tissue. The ROI of the MRI slice covers or overlaps multiple machine-defined original pixels in the MRI slice, and each of the MRI parameters for the ROI of the MRI slice may be calculated by summing measured values of said each of the MRI parameters for the machine-defined original pixels weighed or multiplied by respective percentages of areas, overlapping with the respective machine-defined original pixels in the ROI of the MRI slice, occupying the ROI of the MRI slice. Multiple values for the MRI parameters for the ROI of the MRI slice may be derived from different parameter maps registered to the partial volume of the breast biopsy tissue. In an alternative example, the values for some of the MRI parameters for the ROI of the MRI slice may be derived from different parameter maps registered to the partial volume of the breast biopsy tissue, and the values for the others may be derived from the same parameter map registered to the partial volume of the breast biopsy tissue.
(121) In an alternative example, the biopsied tissue 90 may be obtained by a needle with a square through hole therein. In this case, the biopsied tissue 90 may have a longitudinal shape with a square-shaped cross-section having a width Wb (which is substantially equal to an inner width of the needle, i.e., the width of the square through hole of the needle) and a height Ht (which is substantially equal to, e.g., the slice thickness T multiplied by the number of the MRI slices SI.sub.1-SI.sub.N). The volume of the biopsied tissue 90 may be transformed into a flat square FS with a width Wf and a thickness or height fT. The flat square FS, having a volume the same or about the same as the volume of the biopsied tissue 90 (or Volume of Interest (VOI), which may be the height Ht multiplied by the square of the width Wb), may be defined by the following formula: Wb.sup.2×M×St=Wf.sup.2×fT, where Wb is the width of the biopsy tissue 90, M is the number of the MRI slices SI.sub.1-SI.sub.N, St is the slice thickness T of the MRI slices SI.sub.1-SI.sub.N, Wf is the width of the flat square FS, and fT is the height or thickness of the flat square FS perpendicular to the width Wf of the flat square FS. In the invention, the height or thickness fT of the flat square FS is substantially equal to the slice thickness T, for example. Accordingly, the flat square FS may have the height or thickness fT equal to the slice thickness T and the width Wf equal to the width Wb multiplied by the square root of the number of the registered MRI slices SI.sub.1-SI.sub.N. In the case of the moving window MW with a square shape, the width Wf of the flat square FS may be used to define the width of the moving window MW in calculating probability maps. Each of the biopsy tissue 90, the flat square FS and the square moving window MW may have a volume at least 2, 3, 5, 10 or 15 times greater than that of each machine-defined original pixel of the MRI slices SI.sub.1-SI.sub.N and than that of each machine-defined original pixel of an MRI image, e.g., 10 from a subject (e.g., patient) depicted in a step S1 of
(122) Description of Area Resolution and Machine-Defined Original Pixels of a Single MRI Slice:
(123) In the invention, an area resolution of a single MRI slice such as single slice MRI image 10 shown in
(124) Description of Moving Window and Probability Map:
(125) Any probability map in the invention may be composed of multiple computation pixels with the same size, which are basic units of the probability map. The size of the computation pixels used to compose the probability map may be defined based on the size of the moving window MW, which is determined or defined based on information data associated with the biopsy tissues provided for the pathologist diagnoses in the subset data, e.g., DB-1 or DB-2, of the big data database 70. The information data, for example, may include the radii Rw of planar cylinders 98 transformed from the volumes of the biopsy tissues. In addition, each of the computation pixels of the probability map may have a volume or size equal to, greater than or less than that of any machine-defined original pixel in a single MRI slice, such as MRI image 10 shown in
(126) The moving window MW may have various shapes, such as a circular shape, a square shape, a rectangular shape, a hexagonal shape, or an octagonal shape. In the invention, referring to
(127) Referring to
(128) The circular moving window 2 in
(129) In an alternative example, referring to
(130) Accordingly, the moving window MW (e.g., the circular moving window 2) may be defined to include four or more non-overlapped grids 6 having the same square shape, the same size or area (e.g., 1 millimeter by 1 millimeter), and the same width Wsq, e.g., equal to, greater than or less than any side length of machine-defined original pixels in a single MRI slice, such as MRI image 10 shown in
(131) Alternatively, the grids 6 may be n rectangles each having a width Wrec and a length Lrec, where n is an integer, such as 2, 3, 4, 5, 6, 7, 8, or more than 8. Based on the size (e.g., the width Wrec and the length Lrec) and shape of the divided rectangles 6, the size and shape of the computation pixels used to compose the probability map may be defined. In other words, each of the computation pixels used to compose the probability map, for example, may be defined as a rectangle with the width Wrec, the length Lrec, and a volume the same or about the same as that of each rectangle 6 based on the radius Rm of the circular moving window 2 and the number of the rectangles 6 in the circular moving window 2, i.e., based on the width Wrec and length Lrec of the rectangles 6 in the circular moving window 2. Accordingly, the moving window MW (e.g., the circular moving window 2) may be defined to include four or more non-overlapped grids 6 having the same rectangle shape, the same size or area, the same width Wrec, e.g., equal to, greater than or less than any side length of machine-defined original pixels in a single MRI slice, such as MRI image 10 shown in
(132) In the case of the moving window MW with a square shape, the square moving window MW may be determined with a width Wsm based on the statistical distribution or average of the widths Wf of flat squares FS obtained from biopsy tissues associated with a subset data of the big data database 70. The square moving window MW may be divided into the aforementioned small grids 6. In this case, each of the computation pixels of the probability map, for example, may be defined as a square with the width Wsq and a volume the same or about the same as that of each square 6 based on the width Wsm of the square moving window MW and the number of the squares 6 in the square moving window MW, i.e., based on the width Wsq of the squares 6 in the square moving window MW. Alternatively, each of the computation pixels of the probability map may be defined as a rectangle with the width Wrec, the length Lrec, and a volume the same or about the same as that of each rectangle 6 based on the width Wsm of the square moving window MW and the number of the rectangles 6 in the square moving window MW, i.e., based on the width Wrec and length Lrec of the rectangles 6 in the square moving window MW.
(133) Description of Classifier CF:
(134) The classifier CF for an event, such as biopsy-diagnosed tissue or tumor characteristic for, e.g., specific cancerous cells or occurrence of prostate cancer or breast cancer, may be created or established based on a subset (e.g., the subset data DB-1 or DB-2 or the aforementioned subset data established for generating the voxelwise or pixelwise probability map of brain cancer, liver cancer, lung cancer, rectal cancer, sarcomas, cervical cancer, or cancer metastasis to any organ such as liver, bone, and brain) obtained from the big data database 70. The subset may have all data associated with the given event from the big data database 70. The classifier CF may be a Bayesian classifier, which may be created by performing the following steps: constructing database, preprocessing parameters, ranking parameters, identifying a training dataset, and determining posterior probabilities for test data.
(135) In the step of constructing database, a first group and a second group may be determined or selected from a tissue-based or biopsy-based subset data, such as the aforementioned subset data, e.g., DB-1 or DB-2, from the big data database 70, and various variables associated with each of the first and second groups are obtained from the tissue-based or biopsy-based subset data. The variables may be MRI parameters in the columns A-O of the subset data DB-1 or the columns A-O, R, and S of the subset data DB-2. Alternatively, the variables may be T1 mapping, T2 raw signal, T2 mapping, delta Ktrans, tau, Dt IVIM, fp IVIM, ADC (high b-values), R*, Ktrans from TM, Ktrans from ETM, Ktrans from SSM, Ve from TM, Ve from ETM, Ve from SSM, and standard PET.
(136) The first group, for example, may be associated with a first data type or feature in a specific column of the subset data DB-1 or DB-2, and the second group may be associated with a second data type or feature in the specific column of the subset data DB-1 or DB-2, wherein the specific column of the subset data DB-1 or DB-2 may be one of the columns R-AR of the subset data DB-1 or one of the columns AA-AX of the subset data DB-2. In a first example, the first data type is associated with prostate cancer in the column R of the subset data DB-1, and the second data type is associated with non-prostate cancer (e.g., normal tissue and benign condition) in the column R of the subset data DB-1. In a second example, the first data type is associated with breast cancer in the column AA of the subset data DB-2, and the second data type is associated with non-breast cancer (e.g., normal tissue and benign condition) in the column AA of the subset data DB-2. In the case of the first group associated with a cancer type (e.g., prostate cancer or breast cancer) and the second group associated with a non-cancer type (e.g., non-prostate cancer or non-breast cancer), the cancer type may include data of interest for a single parameter, such as malignancy, mRNA expression, etc., and the non-cancer type may include normal tissue and benign conditions. The benign conditions may vary based on tissues. For example, the benign conditions for breast tissues may include fibroadenomas, cysts, etc.
(137) In a third example, the first data type is associated with one of Gleason scores 0 through 10, such as Gleason score 5, in the column T of the subset data DB-1, and the second data type is associated with the others of Gleason scores 0 through 10, such as Gleason scores 0 through 4 and 6 through 10, in the column T of the subset data DB-1. In a fourth example, the first data type is associated with two or more of Gleason scores 0 through 10, such as Gleason scores greater than 7, in the column T of the subset data DB-1, and the second data type is associated with the others of Gleason scores 0 through 10, such as Gleason scores equal to and less than 7, in the column T of the subset data DB-1. In a fifth example, the first data type is associated with the percentage of cancer in a specific range from a first percent (e.g., 91 percent) to a second percent (e.g., 100 percent) in the column S of the subset data DB-1, and the second data type is associated with the percentage of cancer beyond the specific range in the column S of the subset data DB-1. In a sixth example, the first data type is associated with a small cell subtype in the column AE of the subset data DB-1, and the second data type is associated with a non-small cell subtype in the column AE of the subset data DB-1. Any event depicted in the invention may be the above-mentioned first data type or feature, occurrence of prostate cancer, occurrence of breast cancer, or a biopsy-diagnosed tissue or tumor characteristic for, e.g., specific cancerous cells.
(138) After the step of constructing database is completed, the step of preprocessing parameters is performed to determine what the variables are conditionally independent. A technique for dimensionality reduction may allow reduction of some of the variables that are conditionally dependent to a single variable. Use of dimensionality reduction preprocessing of data may allow optimal use of all valuable information in datasets. The simplest method for dimensionality reduction may be simple aggregation and averaging of datasets. In one example, aggregation may be used for dynamic contrast-enhanced MRI (DCE-MRI) datasets. Ktrans and Ve measured values from various different pharmacokinetic modeling techniques may be averaged to reduce errors and optimize sensitivity to tissue change.
(139) For the variables, averaging and subtraction may be used to consolidate measured variables. Accordingly, five or more types of parameters may be selected or obtained from the variables. The five or more selected parameters are conditionally independent and may include T1 mapping, T2 mapping, delta Ktrans (obtained by subtracting Ktrans from Tofts Model from Ktrans from Shutterspeed Model), tau, Dt IVIM, fp IVIM, R*, average Ve, and average Ktrans in the respective columns A, C-G, J, P, and Q of the subset data DB-1 or DB-2. Alternatively, the five or more selected parameters may include T1 mapping, T2 mapping, delta Ktrans, tau, fp IVIM, R*, average Ve, average Ktrans, standard PET, and a parameter D obtained by averaging Dt IVIM and ADC (high b-values), wherein the parameter D is conditionally independent of every other selected parameter.
(140) After the step of preprocessing parameters is complete, the step of ranking parameters is performed to determine the optimal ones of the five or more selected parameters for use in classification, e.g., to find the optimal parameters that are most likely to give the highest posterior probabilities, so that a rank list of the five or more selected parameters is obtained. A filtering method, such as t-test, may be to look for an optimal distance between the first group (indicated by GR1) and the second group (indicated by GR2) for every one of the five or more selected parameters, as shown in
(141) Four different criteria may be computed for ranking the five or more selected parameters. The first criterion is the p-value derived from a t-test of the hypothesis that the two features sets, corresponding to the first group and the second group, coming from distributions with equal means. The second criterion is the mutual information (MI) computed between the classes and each of the first and second groups. The last two criteria are derived from the minimum redundancy maximum relevance (mRMR) selection method.
(142) In the step of identifying a training dataset, a training dataset of the first group and the second group is identified based on the rank list after the step of ranking parameters, and thereby the Bayesian classifier may be created based on the training dataset of the first group and the second group. In the step of determining posterior probabilities for test data, the posterior probabilities for the test data may be determined using the Bayesian classifier. Once the Bayesian classifier is created, the test data may be applied to predict posterior probabilities for high resolution probability maps.
(143) In an alternative example, the classifier CF may be a neural network (e.g., probabilistic neural network, single-layer feed forward neural network, multi-layer perception neural network, or radial basis function neural network), a discriminant analysis, a decision tree (e.g., classification and regression tree, quick unbiased and efficient statistical tree, Chi-square automatic interaction detector, C5.0, or random forest decision tree), an adaptive boosting, a K-nearest neighbors algorithm, or a support vector machine. In this case, the classifier CF may be created based on information associated with the various MRI parameters for the ROIs 94 of the MRI slices SI.sub.1-SI.sub.N registered to each of the biopsy tissues depicted in the subset data DB-1 or DB-2.
First Embodiment
(144) After the big data database 70 and the classifier CF are created or constructed, a (voxelwise or pixelwise) probability map (i.e., a decision data map), composed of multiple computation pixels with the same size, for an event (i.e., a decision-making characteristic) may be generated or constructed for, e.g., evaluating or determining the health status of a subject such as healthy individual or patient, the physical condition of an organ or other structure inside the subjects body, or the subjects progress and therapeutic effectiveness by sequentially performing six steps S1 through S6 illustrated in
(145) In the step S2, a desired or anticipated region 11, i.e., target region, is determined on the MRI image 10, and a computation region 12 for the probability map is set in the desired or anticipated region 11, i.e., target region, of the MRI image 10 and defined with the computation pixels based on the size (e.g., the radius Rm) of the moving window 2 and the size and shape of the small grids 6 in the moving window 2 such as the width Wsq of the small squares 6 or the width Wrec and the length Lrec of the small rectangles 6. A side length of the computation region 12 in the x direction, for example, may be calculated by obtaining a first maximum positive integer of a side length of the desired or anticipated region 11, i.e., target region, in the x direction divided by the width Wsq of the small squares 6 in the moving window 2, and multiplying the width Wsq by the first maximum positive integer; a side length of the computation region 12 in the y direction may be calculated by obtaining a second maximum positive integer of a side length of the desired or anticipated region 11, i.e., target region, in the y direction divided by the width Wsq of the small squares 6 in the moving window 2, and multiplying the width Wsq by the second maximum positive integer. Alternatively, a side length of the computation region 12 in the x direction may be calculated by obtaining a first maximum positive integer of a side length of the desired or anticipated region 11, i.e., target region, in the x direction divided by the width Wrec of the small rectangles 6 in the moving window 2, and multiplying the width Wrec by the first maximum positive integer; a side length of the computation region 12 in the y direction may be calculated by obtaining a second maximum positive integer of a side length of the desired or anticipated region 11, i.e., target region, in the y direction divided by the length Lrec of the small rectangles 6 in the moving window 2, and multiplying the length Lrec by the second maximum positive integer. The computation region 12 may cover at least 10, 25, 50, 80, 90 or 95 percent of the FOV of the MRI image 10, which may include the anatomical region of the subject. The computation region 12, for example, may be shaped like a parallelogram such as square or rectangle.
(146) The size and shape of the computation pixels used to compose the probability map, for example, may be defined based on the radius Rm of the moving window 2, wherein the radius Rm is calculated based on, e.g., the statistical distribution or average of the radii Rw of the planar cylinders 98 transformed from the volumes of the prostate biopsy tissues provided for the pathologist diagnoses depicted in the subset data DB-1, as illustrated in the section of description of moving window and probability map. Each of the computation pixels, for example, may be defined as a square with the width Wsq in the case of the moving window 2 defined to include the small squares 6 each having the width Wsq. Alternatively, each of the computation pixels may be defined as a rectangle with the width Wrec and the length Lrec in the case of the moving window 2 defined to include the small rectangles 6 each having the width Wrec and the length Lrec.
(147) A step for abbreviated search functions (such as looking for one or more specific areas of the MRI image 10 where diffusion signals are above a certain signal value) may be performed between the steps S1 and S2, and the computation region 12 may cover the one or more specific areas of the MRI image 10. For clear illustration of the following steps,
(148) For more elaboration, referring to
(149) The specific MRI parameters for each stop of the moving window 2 may include T1 mapping, T2 raw signal, T2 mapping, delta Ktrans, tau, Dt IVIM, fp IVIM, ADC (high b-values), nADC (high b-values), R*, Ktrans from TM, ETM and SSM, and Ve from TM and SSM, which may be referred to the types of the MRI parameters in the columns A-O of the subset data DB-1, respectively. Alternatively, the specific MRI parameters for each stop of the moving window 2 may include four or more of the following: T1 mapping, T2 raw signal, T2 mapping, Ktrans from TM, ETM, and SSM, Ve from TM and SSM, delta Ktrans, tau, ADC (high b-values), nADC (high b-values), Dt IVIM, fp IVIM, and R*. The specific MRI parameters of different modalities may be obtained from registered (multi-parametric) image sets (or the MRI parameter maps in the registered (multi-parametric) image dataset), and rigid and non-rigid standard registration techniques may be used to get each section of anatomy into the same exact coordinate location on each of the registered (multi-parametric) image sets (or on each of the MRI parameter maps).
(150) Referring to
(151) The registered imaging dataset may be created for the subject to include, e.g., multiple registered MRI slice images (including, e.g., MRI image 10) and/or corresponding MRI parameters obtained from various equipment, machines, or devices or from a defined time-point (e.g., specific date) or time range (e.g., within five days after treatment). Each of the MRI parameters in the subjects registered imaging dataset requires alignment or registration. The registration can be done by, for examples, using unique anatomical marks, structures, tissues, geometry, and/or shapes or using mathematical algorithms and computer pattern recognition. The values C.sub.m-n of the specific imaging parameters for each stop W.sub.m-n of the moving window 2, for example, may be obtained from the registered imaging dataset for the subject.
(152) Referring to
(153) In the step S5, in the learning operation (E.sub.1), the value set C.sub.m-n for each stop W.sub.m-n of the moving window 2 from the step S4 (or the values C.sub.m-n of some or all of the specific MRI parameters for each stop W.sub.m-n of the moving window 2 from the step S3) may be matched to a biomarker library or the classifier CF for an event (e.g., the first data type or feature depicted in the section of description of classifier CF, or biopsy-diagnosed tissue characteristic for, e.g., specific cancerous cells or occurrence of prostate or breast cancer) created based on data associated with the event from the subset data DB-1. Accordingly, a probability PW or CL.sub.m-n of the event for each stop W.sub.m-n of the moving window 2 is obtained. In other words, the probability PW or CL.sub.m-n of the event for each stop W.sub.m-n of the moving window 2 may be obtained based on the value set C.sub.m-n (from the step S4) containing the values C.sub.m-n of some or all of the specific MRI parameters (from the step S3) for said each stop W.sub.m-n, of the moving window 2 to match a matching dataset from the established or constructed biomarker library or classifier CF. The biomarker library or classifier CF, for example, may contain population-based information of MRI imaging data and other information such as clinical and demographic data for the event. In the invention, the probability PW or CL.sub.m-n of the event for each stop W.sub.m-n of the moving window 2 is assumed to be that for the square 4 inscribed in said each stop W.sub.m-n of the moving window 2.
(154) In the step S6, an algorithm including steps S11 through S16 depicted in
(155) In the step S12, a probability guess PG for each stop W.sub.m-n of the moving window 2 is calculated by, e.g., averaging the probabilities PVs of the event for all the computation pixels P.sub.k-l inside said each stop W.sub.m-n of the moving widow 2. In the step S13, a difference DW between the probability guess PG and the probability PW of the event for each stop W.sub.m-n of the moving window 2 is calculated by, e.g., subtracting the probability PW of the event for said each stop W.sub.m-n of the moving window 2 from the probability guess PG for said each stop W.sub.m-n of the moving window 2.
(156) In the step S14, an absolute value of the difference DW between the probability guess PG and the probability PW of the event for each stop W.sub.m-n of the moving window 2 is compared with a preset threshold error or value (e.g., 0.001 or 0.0001) to determine whether an error, i.e., the absolute value of the difference DW, between the probability guess PG and the probability PW of the event for each stop W.sub.m-n of the moving window 2 is less than or equal to the preset threshold error or value. If the absolute value of the difference DW for each stop W.sub.m-n of the moving window 2 is determined in the step S14 to be less than or equal to the preset threshold error or value, the step S16 continues. In the step S16, the probabilities PVs or dl.sub.k-l, of the event for the computation pixels P.sub.k-l are determined to be optimal, which are called optimal probabilities hereinafter, and the optimal probabilities dl.sub.k-l of the respective computation pixels P.sub.k-l form the probability map of the event for the MRI image 10 for the subject having imaging information (e.g., MRI imaging information). After the optimal probabilities dl.sub.k-l for the respective computation pixels P.sub.k-l are obtained in the step S16, the algorithm is completed.
(157) If any one of the absolute values of the differences DWs for all the stops W.sub.m-n, of the moving window 2 is determined in the step S14 to be greater than the preset threshold error or value, the step S15 continues. In the step S15, the probability PV or dl.sub.k-l, of the event for each of the computation pixels P.sub.k-l is updated or adjusted by, e.g., subtracting an error correction factor ECF for said each of the computation pixels P.sub.k-l from the probability PV or dl.sub.k-l, of the event for said each of the computation pixels P.sub.k-l. The error correction factor ECF for each of the computation pixels P.sub.k-l is calculated by, e.g., summing error correction contributions from the stops W.sub.m-n of the moving window 2 each having one of its squares 6 covering or overlapping said each of the computation pixels P.sub.k; each of the error correction contributions to said each of the computation pixels P.sub.k-l, for example, may be calculated by multiplying the difference DW for a corresponding one of the stops W.sub.m-n of the moving window 2 by an area ratio of an overlapped area between said each of the computation pixels P.sub.k-l and the corresponding one of the stops W.sub.m-n of the moving window 2 to an area of the square 4 inscribed in the corresponding one of the stops W.sub.m-n of the moving window 2. Alternatively, the error correction factor ECF for each of the computation pixels P.sub.k-l is calculated by, e.g., dividing the sum of the differences DWs for overlapping ones of the stops W.sub.m-n of the moving window 2, each having one of its squares 6 covering or overlapping said each of the computation pixels P.sub.k-l, by the number of all the squares 6 within the moving window 2. After the probabilities PVs or dl.sub.k-l, of the event for the computation pixels P.sub.k-l are updated, the steps S12-S15 are performed repeatedly based on the updated probabilities PVs or dl.sub.k-l, of the event for the computation pixels P.sub.k-l in the step S15, until the absolute value of the difference DW between the probability guess PG and the probability PW or CL.sub.m-n, of the event for each stop W.sub.m-n of the moving window 2 is determined in the step S14 to be less than or equal to the preset threshold error or value.
(158) The steps S12-S14 depicted in
(159) For detailed description of the steps S11-S16, the square 4 inscribed in the moving window 2 with the radius Rm is divided into, e.g., nine small squares 6 each having width Wsq as shown in
(160) Referring to
(161) Referring to
(162) Referring to
(163) Referring to
(164) After the values C.sub.m-n, i.e., C.sub.1-1-C.sub.4-4, of the specific MRI parameters for the sixteen stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2 are obtained, the step S5 is performed for engineering learning or machine learning to obtain the probabilities PWs or CL.sub.m-n, i.e., CL.sub.1-1-CL.sub.4-4, of the event for the respective stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2. The probabilities PWs or CL.sub.m-n, i.e., CL.sub.1-1, CL.sub.2-4, CL.sub.3-1, CL.sub.4-1, CL.sub.1-2, CL.sub.2-2, CL.sub.3-2, CL.sub.4-2, CL.sub.1-3, CL.sub.2-3, CL.sub.3-3, CL.sub.4-3, CL.sub.2-4, CL.sub.3-4, and CL.sub.4-4, of the event for the sixteen stops W.sub.1-1, W.sub.2-1, W.sub.3-1, W.sub.4-1, W.sub.1-2, W.sub.2-2, W.sub.3-2, W.sub.4-2, W.sub.1-3, W.sub.2-3, W.sub.3-3, W.sub.4-3, W.sub.1-4, W.sub.2-4, W.sub.3-4, and W.sub.4-4 of the moving window 2, for example, are assumed to be 0.6055, 0.5628, 0.5366, 0.4361, 0.4982, 0.5534, 0.5521, 0.4227, 0.4618, 0.5132, 0.6214, 0.5810, 0.4371, 0.4698, 0.5774, and 0.5613, respectively. In the example, the sixteen probabilities PWs or CL.sub.m-n, i.e., CL.sub.1-1-CL.sub.4-4, of the event for the sixteen stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2 are assumed to be those for the sixteen squares 4 inscribed in the respective stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2, respectively. In other words, the sixteen probabilities of the event for the sixteen squares 4 inscribed in the sixteen stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2 are 0.6055, 0.5628, 0.5366, 0.4361, 0.4982, 0.5534, 0.5521, 0.4227, 0.4618, 0.5132, 0.6214, 0.5810, 0.4371, 0.4698, 0.5774, and 0.5613, respectively.
(165) Next, the algorithm depicted in
(166) Because the only two stops W.sub.1-1 and W.sub.2-1 of the moving window 2 have the squares W2 and W10 overlapping the computation pixel P.sub.2-1, the probability PV, i.e., dl.sub.2-1, of the event for the computation pixel P.sub.2-1 is assumed to be the average, i.e., 0.5841, of the two probabilities PWs, i.e., CL.sub.1-1 and CL.sub.2-1 equal to 0.6055 and 0.5628 respectively, of the event for the stops W.sub.1-1 and W.sub.2-1 of the moving window 2. Similarly, the probability PV, i.e., dl.sub.5-1, of the event for the computation pixel P.sub.5-1 is assumed to be the average, i.e., 0.4863, of the probabilities PWs, i.e., CL.sub.3-1 and CL.sub.4-1 equal to 0.5366 and 0.4361 respectively, of the event for the stops W.sub.3-1 and W.sub.4-1 of the moving window 2. The probability PV, i.e., dl.sub.1-2, of the event for the computation pixel P.sub.1-2 is assumed to be the average, i.e., 0.5519, of the probabilities PWs, i.e., CL.sub.1-1 and CL.sub.1-2 equal to 0.6055 and 0.4982 respectively, of the event for the stops W.sub.1-1 and W.sub.1-2 of the moving window 2. The probability PV, i.e., dl.sub.6-2, of the event for the computation pixel P.sub.6-2 is assumed to be the average, i.e., 0.4294, of the probabilities PWs, i.e., CL.sub.4-1 and CL.sub.4-2 equal to 0.4361 and 0.4227 respectively, of the event for the stops W.sub.4-1 and W.sub.4-2 of the moving window 2. The probability PV, i.e., dl.sub.1-5, of the event for the computation pixel P.sub.1-5 is assumed to be the average, i.e., 0.4495, of the probabilities PWs, i.e., CL.sub.1-3 and CL.sub.1-4 equal to 0.4618 and 0.4371 respectively, of the event for the stops W.sub.1-3 and W.sub.1-4 of the moving window 2. The probability PV, i.e., dl.sub.6-5, of the event for the computation pixel P.sub.6-5 is assumed to be the average, i.e., 0.5711, of the probabilities PWs, i.e., CL.sub.4-3 and CL.sub.4-4 equal to 0.5810 and 0.5613 respectively, of the event for the stops W.sub.4-3 and W.sub.4-4 of the moving window 2. The probability PV, i.e., dl.sub.2-6, of the event for the computation pixel P.sub.2-6 is assumed to be the average, i.e., 0.4535, of the probabilities PWs, i.e., CL.sub.1-4 and CL.sub.2-4 equal to 0.4371 and 0.4698 respectively, of the event for the stops W.sub.1-4 and P.sub.2-4 of the moving window 2. The probability PV, i.e., dl.sub.5-6, of the event for the computation pixel P.sub.5-6 is assumed to be the average, i.e., 0.5693, of the probabilities PWs, i.e., CL.sub.3-4 and CL.sub.4-4 equal to 0.5774 and 0.5613 respectively, of the event for the stops W.sub.3-4 and W.sub.4-4 of the moving window 2.
(167) Because the only three stops W.sub.1-1, W.sub.2-1 and W.sub.3-1 of the moving window 2 have the squares W3, W11 and W19 overlapping the computation pixel P.sub.3-1, the probability PV, i.e., dl.sub.3-1, of the event for the computation pixel P.sub.3-1 is assumed to be the average, i.e., 0.5683, of the three probabilities PWs, i.e., CL.sub.1-1, CL.sub.2-1 and CL.sub.3-1 equal to 0.6055, 0.5628 and 0.5366 respectively, of the event for the stops W.sub.1-1, W.sub.2-1 and W.sub.3-1 of the moving window 2. Similarly, the probability PV, i.e., dl.sub.4-1, of the event for the computation pixel P.sub.4-1 is assumed to be the average, i.e., 0.5118, of the probabilities PWs, i.e., CL.sub.2-1, CL.sub.3-1 and CL.sub.4-1, of the event for the stops W.sub.2-1, W.sub.3-1 and W.sub.4-1 of the moving window 2. The probability PV, i.e., dl.sub.1-3, of the event for the computation pixel P.sub.1-3 is assumed to be the average, i.e., 0.5219, of the probabilities PWs, i.e., CL.sub.1-1, CL.sub.1-2 and CL.sub.1-3, of the event for the stops W.sub.1-1, W.sub.1-2 and W.sub.1-3 of the moving window 2. The probability PV, i.e., dl.sub.6-3, of the event for the computation pixel P.sub.6-3 is assumed to be the average, i.e., 0.4799, of the probabilities PWs, i.e., CL.sub.4-1, CL.sub.4-2 and CL.sub.4-3, of the event for the stops W.sub.4-1, W.sub.4-2 and W.sub.4-3 of the moving window 2. The probability PV, i.e., dl.sub.1-4, of the event for the computation pixel P.sub.1-4 is assumed to be the average, i.e., 0.4657, of the probabilities PWs, i.e., CL.sub.1-2, CL.sub.1-3 and CL.sub.1-4, of the event for the stops W.sub.1-2, W.sub.1-3 and W.sub.1-4 of the moving window 2. The probability PV, i.e., dl.sub.6-4, of the event for the computation pixel P.sub.6-4 is assumed to be the average, i.e., 0.5216, of the probabilities PWs, i.e., CL.sub.4-2, CL.sub.4-3 and of the event for the stops W.sub.4-2, W.sub.4-3 and W.sub.4-4 of the moving window 2. The probability PV, i.e., dl.sub.3-6, of the event for the computation pixel P.sub.3-6 is assumed to be the average, i.e., 0.4948, of the probabilities PWs, i.e., CL.sub.1-4, CL.sub.2-4 and CL.sub.3-4, of the event for the stops W.sub.1-4, W.sub.2-4 and W.sub.3-4 of the moving window 2. The probability PV, i.e., dl.sub.4-6, of the event for the computation pixel P.sub.4-6 is assumed to be the average, i.e., 0.5362, of the probabilities PWs, i.e., CL.sub.2-4, CL.sub.3-4 and CL.sub.4-4, of the event for the stops W.sub.2-4, W.sub.3-4 and W.sub.4-4 of the moving window 2.
(168) Because the only four stops W.sub.1-1, W.sub.2-1, W.sub.1-2 and W.sub.2-2 of the moving window 2 have the squares W5, W13, W38 and W46 overlapping the computation pixel P.sub.2-2, the probability PV, i.e., dl.sub.2-2, of the event for the computation pixel P.sub.2-2 is assumed to be the average, i.e., 0.5550, of the four probabilities PWs, i.e., CL.sub.1-1, CL.sub.2-1 CL.sub.1-2 and CL.sub.2-2 equal to 0.6055, 0.5628, 0.4982 and 0.5534 respectively, of the event for the stops W.sub.1-1, W.sub.2-1, W.sub.1-2 and W.sub.2-2 of the moving window 2. Similarly, the probability PV, i.e., dl.sub.5-2, of the event for the computation pixel P.sub.5-2 is assumed to be the average, i.e., 0.4869, of the probabilities PWs, i.e., CL.sub.3-1, CL.sub.4-1 CL.sub.3-2 and CL.sub.4-2, of the event for the stops W.sub.3-1, W.sub.4-1, W.sub.3-2 and W.sub.4-2 of the moving window 2. The probability PV, i.e., dl.sub.2-5, of the event for the computation pixel P.sub.2-5 is assumed to be the average, i.e., 0.4705, of the probabilities PWs, i.e., CL.sub.1-3, CL.sub.2-3 CL.sub.1-4 and CL.sub.2-4, of the event for the stops W.sub.1-3, W.sub.2-3, W.sub.1-4 and W.sub.2-4 of the moving window 2. The probability PV, i.e., dl.sub.5-5, of the event for the computation pixel P.sub.5-5 is assumed to be the average, i.e., 0.5852, of the probabilities PWs, i.e., CL.sub.3-3, CL.sub.4-3 CL.sub.3-4 and CL.sub.4-4, of the event for the stops W.sub.3-3, W.sub.4-3, W.sub.3-4 and W.sub.4-4 of the moving window 2.
(169) Because the only six stops W.sub.1-1, W.sub.2-1, W.sub.3-1, W.sub.1-2, W.sub.2-2 and W.sub.3-2 of the moving window 2 have the squares W6, W14, W22, W39, W47 and W55 overlapping the computation pixel P.sub.3-2, the probability PV, i.e., dl.sub.3-2, of the event for the computation pixel P.sub.3-2 is assumed to be the average, i.e., 0.5514, of the six probabilities PWs, i.e., CL.sub.1-1, CL.sub.2-1, CL.sub.3-1, CL.sub.1-2, CL.sub.2-2 and CL.sub.3-2 equal to 0.6055, 0.5628, 0.5366, 0.4982, 0.5534 and 0.5521 respectively, of the event for the stops W.sub.1-1, W.sub.2-1, W.sub.3-1, W.sub.1-2, W.sub.2-2 and W.sub.3-2 of the moving window 2. Similarly, the probability PV, i.e., dl.sub.4-2, of the event for the computation pixel P.sub.4-2 is assumed to be the average, i.e., 0.5106, of the probabilities PWs, i.e., CL.sub.2-1, CL.sub.3-1, CL.sub.4-1, CL.sub.2-2, CL.sub.3-2 and CL.sub.4-2, of the event for the stops W.sub.2-1, W.sub.3-1, W.sub.4-1, W.sub.2-2, W.sub.3-2 and W.sub.4-2 of the moving window 2. The probability PV, i.e., dl.sub.2-3, of the event for the computation pixel P.sub.2-3 is assumed to be the average, i.e., 0.5325, of the probabilities PWs, i.e., CL.sub.1-1, CL.sub.2-1, CL.sub.1-2, CL.sub.2-2, CL.sub.1-3 and CL.sub.2-3, of the event for the stops W.sub.1-1, W.sub.2-1, W.sub.1-2, W.sub.2-2, W.sub.1-3 and W.sub.2-3 of the moving window 2. The probability PV, i.e., dl.sub.5-3, of the event for the computation pixel P.sub.5-3 is assumed to be the average, i.e., 0.5250, of the probabilities PWs, i.e., CL.sub.3-1, CL.sub.4-1, CL.sub.3-2, CL.sub.4-2, CL.sub.3-3 and CL.sub.4-3, of the event for the stops W.sub.3-1, W.sub.4-1, W.sub.3-2, W.sub.4-2, W.sub.3-3 and W.sub.4-3 of the moving window 2. The probability PV, i.e., dl.sub.2-4, of the event for the computation pixel P.sub.2-4 is assumed to be the average, i.e., 0.4889, of the probabilities PWs, i.e., CL.sub.1-2, CL.sub.2-2, CL.sub.1-3, CL.sub.2-3, CL.sub.1-4 and CL.sub.2-4, of the event for the stops W.sub.1-2, W.sub.2-2, P.sub.1-3, W.sub.2-3, W.sub.1-4 and W.sub.2-4 of the moving window 2. The probability PV, i.e., dl.sub.5-4, of the event for the computation pixel P.sub.5-4 is assumed to be the average, i.e., 0.5526, of the probabilities PWs, i.e., CL.sub.3-2, CL.sub.4-2, CL.sub.3-3, CL.sub.4-3, CL.sub.3-4 and CL.sub.4-4, of the event for the stops W.sub.3-2, W.sub.4-2, W.sub.3-3, W.sub.4-3, W.sub.3-4 and W.sub.4-4 of the moving window 2. The probability PV, i.e., dl.sub.3-5, of the event for the computation pixel P.sub.3-5 is assumed to be the average, i.e., 0.5134, of the probabilities PWs, i.e., CL.sub.1-3, CL.sub.2-3, CL.sub.3-3, CL.sub.1-4, CL.sub.2-4 and CL.sub.3-4, of the event for the stops W.sub.1-3, W.sub.2-3, W.sub.3-3, W.sub.1-4, W.sub.2-4 and W.sub.3-4 of the moving window 2. The probability PV, i.e., dl.sub.4-5, of the event for the computation pixel P.sub.4-5 is assumed to be the average, i.e., 0.5540, of the probabilities PWs, i.e., CL.sub.2-3, CL.sub.3-3, CL.sub.4-3, CL.sub.2-4, CL.sub.3-4 and CL.sub.4-4, of the event for the stops W.sub.2-3, W.sub.3-3, W.sub.4-3, W.sub.2-4, W.sub.3-4 and W.sub.4-4 of the moving window 2.
(170) Because the only nine stops W.sub.1-1, W.sub.2-1, W.sub.3-1, W.sub.1-2, W.sub.2-2, W.sub.3-2, W.sub.1-3, W.sub.2-3 and W.sub.3-3 of the moving window 2 have the squares W9, W17, W25, W42, W50, W58, W75, W83 and W91 overlapping the computation pixel P.sub.3-3, the probability PV, i.e., dl.sub.3-3, of the event for the computation pixel P.sub.3-3 is assumed to be the average, i.e., 0.5450, of the nine probabilities PWs, i.e., CL.sub.1-1, CL.sub.2-1, CL.sub.3-1, CL.sub.1-2, CL.sub.2-2, CL.sub.3-2, CL.sub.1-3, CL.sub.2-3 and CL.sub.3-3 equal to 0.6055, 0.5628, 0.5366, 0.4982, 0.5534, 0.5521, 0.4618, 0.5132 and 0.6214 respectively, of the event for the stops W.sub.1-1, W.sub.2-1, W.sub.3-1, W.sub.1-2, W.sub.2-2, W.sub.3-2, W.sub.1-3, W.sub.2-3 and W.sub.3-3 of the moving window 2. Similarly, the probability PV, i.e., dl.sub.4-3, of the event for the computation pixel P.sub.4-3 is assumed to be the average, i.e., 0.5310, of the probabilities PWs, i.e., CL.sub.2-1, CL.sub.3-1, CL.sub.4-1, CL.sub.2-2, CL.sub.3-2, CL.sub.4-2, CL.sub.2-3, CL.sub.3-3 and CL.sub.4-3, of the event for the stops W.sub.2-1, W.sub.3-1, W.sub.4-1, W.sub.2-2, W.sub.3-2, W.sub.4-2, W.sub.2-3, W.sub.3-3 and W.sub.4-3 of the moving window 2. The probability PV, i.e., dl.sub.3-4, of the event for the computation pixel P.sub.3-4 is assumed to be the average, i.e., 0.5205, of the probabilities PWs, i.e., CL.sub.1-2, CL.sub.2-2, CL.sub.3-2, CL.sub.1-3, CL.sub.2-3, CL.sub.3-3, CL.sub.1-4, CL.sub.2-4 and CL.sub.3-4, of the event for the stops W.sub.1-2, W.sub.2-2, W.sub.3-2, W.sub.1-3, W.sub.2-3, W.sub.3-3, W.sub.1-4, W.sub.2-4 and W.sub.3-4 of the moving window 2. The probability PV, i.e., dl.sub.4-4, of the event for the computation pixel P.sub.4-4 is assumed to be the average, i.e., 0.5391, of the probabilities PWs, i.e., CL.sub.2-2, CL.sub.3-2, CL.sub.4-2, CL.sub.2-3, CL.sub.3-3, CL.sub.4-3, CL.sub.2-4, CL.sub.3-4 and CL.sub.4-4, of the event for the stops W.sub.2-2, W.sub.3-2, W.sub.4-2, W.sub.2-3, W.sub.3-3, W.sub.4-3, W.sub.2-4, W.sub.3-4 and W.sub.4-4 of the moving window 2.
(171) After the probabilities PVs or dl.sub.k-l, i.e., dl.sub.1-1-dl.sub.6-6, of the event for the respective computation pixels P.sub.k-l, i.e., P.sub.1-1-P.sub.6-6, are assumed, the step S12 is performed to obtain sixteen probability guesses PGs for the respective stops W.sub.m-n, i.e., W.sub.1-1, W.sub.2-1, W.sub.3-1, W.sub.4-1, W.sub.1-2, W.sub.2-2, W.sub.3-2, W.sub.4-2, W.sub.1-3, W.sub.2-3, W.sub.3-3, W.sub.4-3, W.sub.1-4, W.sub.2-4, W.sub.3-4, and W.sub.4-4, of the moving window 2. The probability guess PG for each of the sixteen stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2 is calculated by averaging nine of the probabilities PVs or dl.sub.k-l, i.e., dl.sub.1-1-dl.sub.6-6, of the event for respective nine of the computation pixels P.sub.k-l, i.e., P.sub.1-1-P.sub.6-6, overlapping or covering the respective nine small squares 6 within the square 4 inscribed in said each of the sixteen stops W.sub.1-1-W.sub.4-4 of the moving window 2. For example, because the nine small squares W1-W9 within the square 4 inscribed in the stop W.sub.1-1 of the moving window 2 overlap or cover the respective computation pixels P.sub.1-1, P.sub.2-1, P.sub.3-1, P.sub.1-2, P.sub.2-2, P.sub.3-2, P.sub.1-3, P.sub.2-3 and P.sub.3-3, the probability guess PG for the stop W.sub.1-1 of the moving window 2 is calculated by averaging the nine probabilities PVs, i.e., dl.sub.1-1, dl.sub.2-1, dl.sub.3-1, dl.sub.1-2, dl.sub.2-2, dl.sub.3-2, dl.sub.1-3, dl.sub.2-3 and dl.sub.3-3 equal to 0.6055, 0.5841, 0.5683, 0.5519, 0.5550, 0.5514, 0.5219, 0.5325 and 0.5450 respectively, of the event for the computation pixels P.sub.1-1, P.sub.2-1, P.sub.3-1, P.sub.1-2, P.sub.2-2, P.sub.3-2, P.sub.1-3, P.sub.2-3 and P.sub.3-3 inside the stop W.sub.1-1 of the moving window 2. Accordingly, the probability guesses PGs for the stops W.sub.m-n, i.e., W.sub.1-1, W.sub.2-1, W.sub.3-1, W.sub.4-1, W.sub.1-2, W.sub.2-2, W.sub.3-2, W.sub.4-2, W.sub.1-3, W.sub.2-3, W.sub.3-3, W.sub.4-3, W.sub.1-4, W.sub.2-4, W.sub.3-4, and W.sub.4-4, of the moving window 2 are 0.5573, 0.5433, 0.5240, 0.4886, 0.5259, 0.5305, 0.5291, 0.5085, 0.5009, 0.5217, 0.5407, 0.5400, 0.4771, 0.5079, 0.5406, and 0.5545, respectively.
(172) After the sixteen probability guesses PGs are obtained or calculated, the step S13 is performed to obtain sixteen differences DWs for the sixteen stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2. Each of the sixteen differences DWs is calculated by, e.g., subtracting the probability PW or CL.sub.m-n i.e., CL.sub.1-1-CL.sub.4-4, of the event for a corresponding one of the sixteen stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2 from the probability guess PG for the corresponding one of the sixteen stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2. For example, the difference DW for the stop W.sub.1-1 of the moving window 2 is calculated by subtracting the probability PW, i.e., CL.sub.1-1 equal to 0.6055, of the event for the stop W.sub.1-1 of the moving window 2 from the probability guess PG, i.e., 0.5573, for the stop W.sub.1-1 of the moving window 2. Accordingly, the differences DWs for the stops i.e., W.sub.1-1, W.sub.2-1, W.sub.3-1, W.sub.4-1, W.sub.1-2, W.sub.2-2, W.sub.3-2, W.sub.4-2, W.sub.1-3, W.sub.2-3, W.sub.3-3, W.sub.4-3, W.sub.1-4, W.sub.2-4, W.sub.3-4, and W.sub.4-4, of the moving window 2 are −0.0482, −0.0194, −0.0126, 0.0525, 0.0276, −0.0230, −0.0230, 0.0858, 0.0391, 0.0085, −0.0807, −0.0410, 0.0400, 0.0380, −0.0368, and −0.0068, respectively.
(173) After the sixteen differences DWs are obtained or calculated, the step S14 is performed to determine whether absolute values of the sixteen differences DWs are less than or equal to a preset threshold value of 0.0001. Because the absolute values of the sixteen differences DWs are greater than the preset threshold value, the step S15 continues in which the probabilities PVs or dl.sub.k-l, i.e., dl.sub.1-1-dl.sub.6-6, of the event for the computation pixels P.sub.k-l, i.e., P.sub.1-1-P.sub.6-6, are updated, as shown in
(174) In the step S15, the updated probability PV or dl.sub.k-l, i.e., updated dl.sub.1-1-dl.sub.6-6, of the event for each of the computation pixels P.sub.k-l, i.e., P.sub.1-1-P.sub.6-6, is calculated by, e.g., subtracting an error correction factor ECF for said each of the computation pixels P.sub.k-l, i.e., P.sub.1-1-P.sub.6-6, from the current probability PV or dl.sub.k-l, i.e., current dl.sub.1-1-dl.sub.6-6, of the event for said each of the computation pixels P.sub.k-l, i.e., P.sub.1-1-P.sub.6-6. The error correction factor ECF for each of the 4 computation pixels P.sub.1-1, P.sub.6-1, P.sub.1-6 and P.sub.6-6 is obtained by, e.g., calculating an error correction contribution only from a corresponding one of the stops W.sub.1-1, W.sub.4-1, W.sub.1-4 and W.sub.4-4 of the moving window 2, which has one of its squares 6 covering or overlapping said each of the 4 computation pixels P.sub.1-1, P.sub.6-1, P.sub.1-6 and P.sub.6-6. For example, because the only stop W.sub.1-1 of the moving window 2 has the small square W1 covering or overlapping the computation pixel P.sub.1-1, the error correction factor ECF, i.e., −0.0054, for the computation pixel P.sub.1-1 is obtained by calculating the error correction contribution only from the stop W.sub.1-1 of the moving window 2. The error correction contribution to the computation pixel P.sub.1-1 from the stop W.sub.1-1 of the moving window 2 is calculated by multiplying the difference DW, i.e., −0.0482, for the stop W.sub.1-1 of the moving window 2 by an area ratio, i.e., 1/9, of an overlapped area between the computation pixel P.sub.1-1 and the stop W.sub.1-1 of the moving window 2 to an area of the square 4 inscribed in the stop W.sub.1-1 of the moving window 2. Accordingly, the updated probability PV, i.e., updated dl.sub.1-1, of the event for the computation pixel P.sub.1-1 is calculated by subtracting the error correction factor ECF, i.e., −0.0054, for the computation pixel P.sub.1-1 from the current probability PV, i.e., current dl.sub.1-1 equal to 0.6055, of the event for the computation pixel P.sub.1-1.
(175) The error correction factor ECF for each of the 32 computation pixels P.sub.2-1-P.sub.5-1, P.sub.1-2-P.sub.6-2, P.sub.1-3-P.sub.6-3, P.sub.1-4-P.sub.6-4, P.sub.1-5-P.sub.6-5 and P.sub.2-6-P.sub.5-6 is calculated by, e.g., summing error correction contributions from overlapping ones of the stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2, each having one of its squares 6 covering or overlapping said each of the 32 computation pixels P.sub.2-1-P.sub.5-1, P.sub.1-2-P.sub.6-2, P.sub.1-3-P.sub.6-3, P.sub.1-4-P.sub.6-4, P.sub.1-5-P.sub.6-5 and P.sub.2-6-P.sub.5-6; each of the error correction contributions to said each of the 32 computation pixels P.sub.2-1-P.sub.5-1, P.sub.1-2-P.sub.6-2, P.sub.1-3-P.sub.6-3, P.sub.1-4-P.sub.6-4, P.sub.1-5-P.sub.6-5 and P.sub.2-6-P.sub.5-6 is calculated by multiplying the difference DW for a corresponding one of the overlapping ones of the stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2 by an area ratio of an overlapped area between said each of the 32 computation pixels P.sub.2-1-P.sub.5-1, P.sub.1-2-P.sub.6-2, P.sub.1-3-P.sub.6-3, P.sub.1-4-P.sub.6-4, P.sub.1-5-P.sub.6-5 and P.sub.2-6-P.sub.5-6 and the corresponding one of the overlapping ones of the stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2 to an area of the square 4 inscribed in the corresponding one of the overlapping ones of the stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2. For example, because the only nine stops W.sub.1-1, W.sub.2-1, W.sub.3-1, W.sub.1-2, W.sub.2-2, W.sub.3-2, W.sub.1-3, W.sub.2-3, and W.sub.3-3 of the moving window 2 have the squares W9, W17, W25, W42, W50, W58, W75, W83 and W91 covering or overlapping the computation pixel P.sub.3-3, the error correction factor ECF, i.e., −0.0146, for the computation pixel P.sub.3-3 is obtained by summing error correction contributions from the respective stops W.sub.1-1, W.sub.2-1, W.sub.3-1, W.sub.1-2, W.sub.2-2, W.sub.3-2, W.sub.1-3, W.sub.2-3, and W.sub.3-3 of the moving window 2. The error correction contribution, i.e., −0.0053, from the stop W.sub.1-1 of the moving window 2 to the computation pixel P.sub.3-3 is calculated by multiplying the difference DW, i.e., −0.0482, for the stop W.sub.1-1 of the moving window 2 by an area ratio, i.e., 1/9, of an overlapped area between the computation pixel P.sub.3-3 and the stop W.sub.1-1 of the moving window 2 to the area of the square 4 inscribed in the stop W.sub.1-1 of the moving window 2. The error correction contribution, i.e., −0.0021, from the stop W.sub.2-1 of the moving window 2 to the computation pixel P.sub.3-3 is calculated by multiplying the difference DW, i.e., −0.0194, for the stop W.sub.2-1 of the moving window 2 by an area ratio, i.e., 1/9, of an overlapped area between the computation pixel P.sub.3-3 and the stop W.sub.2-1 of the moving window 2 to the area of the square 4 inscribed in the stop W.sub.2-1 of the moving window 2. The error correction contribution, i.e., −0.0014, from the stop W.sub.3-1 of the moving window 2 to the computation pixel P.sub.3-3 is calculated by multiplying the difference DW, i.e., −0.0126, for the stop W.sub.3-1 of the moving window 2 by an area ratio, i.e., 1/9, of an overlapped area between the computation pixel P.sub.3-3 and the stop W.sub.3-1 of the moving window 2 to the area of the square 4 inscribed in the stop W.sub.3-1 of the moving window 2. The error correction contribution, i.e., 0.0031, from the stop W.sub.1-2 of the moving window 2 to the computation pixel P.sub.3-3 is calculated by multiplying the difference DW, i.e., 0.0276, for the stop W.sub.1-2 of the moving window 2 by an area ratio, i.e., 1/9, of an overlapped area between the computation pixel P.sub.3-3 and the stop W.sub.1-2 of the moving window 2 to the area of the square 4 inscribed in the stop W.sub.1-2 of the moving window 2. The error correction contribution, i.e., −0.0026, from the stop W.sub.2-2 of the moving window 2 to the computation pixel P.sub.3-3 is calculated by multiplying the difference DW, i.e., −0.0230, for the stop W.sub.2-2 of the moving window 2 by an area ratio, i.e., 1/9, of an overlapped area between the computation pixel P.sub.3-3 and the stop W.sub.2-2 of the moving window 2 to the area of the square 4 inscribed in the stop W.sub.2-2 of the moving window 2. The error correction contribution, i.e., −0.0026, from the stop W.sub.3-2 of the moving window 2 to the computation pixel P.sub.3-3 is calculated by multiplying the difference DW, i.e., −0.0230, for the stop W.sub.3-2 of the moving window 2 by an area ratio, i.e., 1/9, of an overlapped area between the computation pixel P.sub.3-3 and the stop W.sub.3-2 of the moving window 2 to the area of the square 4 inscribed in the stop W.sub.3-2 of the moving window 2. The error correction contribution, i.e., 0.0043, from the stop W.sub.1-3 of the moving window 2 to the computation pixel P.sub.3-3 is calculated by multiplying the difference DW, i.e., 0.0391, for the stop W.sub.1-3 of the moving window 2 by an area ratio, i.e., 1/9, of an overlapped area between the computation pixel P.sub.3-3 and the stop W.sub.1-3 of the moving window 2 to the area of the square 4 inscribed in the stop W.sub.1-3 of the moving window 2. The error correction contribution, i.e., 0.0009, from the stop W.sub.2-3 of the moving window 2 to the computation pixel P.sub.3-3 is calculated by multiplying the difference DW, i.e., 0.0085, for the stop W.sub.2-3 of the moving window 2 by an area ratio, i.e., 1/9, of an overlapped area between the computation pixel P.sub.3-3 and the stop W.sub.2-3 of the moving window 2 to the area of the square 4 inscribed in the stop W.sub.2-3 of the moving window 2. The error correction contribution, i.e., −0.0089, from the stop W.sub.3-3 of the moving window 2 to the computation pixel P.sub.3-3 is calculated by multiplying the difference DW, i.e., −0.0807, for the stop W.sub.3-3 of the moving window 2 by an area ratio, i.e., 1/9, of an overlapped area between the computation pixel P.sub.3-3 and the stop W.sub.3-3 of the moving window 2 to the area of the square 4 inscribed in the stop W.sub.3-3 of the moving window 2. Accordingly, the updated probability PV, i.e., updated dl.sub.3-3, of the event for the computation pixel P.sub.3-3 is calculated by subtracting the error correction factor ECF, i.e., −0.0146, for the computation pixel P.sub.3-3 from the current probability PV, i.e., i.e., current dl.sub.3-3 equal to 0.5450, of the event for the computation pixel P.sub.3-3.
(176) After the updated probabilities PVs or dl.sub.k-l, i.e., updated dl.sub.1-1-dl.sub.6-6, of the event for the computation pixels P.sub.k-l, i.e., P.sub.1-1-P.sub.6-6, are obtained or calculated, the steps S12-S15 are performed repeatedly based on the updated probabilities PVs or dl.sub.k-l, i.e., updated dl.sub.1-1-dl.sub.6-6, of the event for the computation pixels P.sub.k-l, i.e., P.sub.1-1-P.sub.6-6, in the step S15, until the absolute values of the sixteen differences DWs for the sixteen stops W.sub.m-n, i.e., W.sub.1-1-W.sub.4-4, of the moving window 2 are less than or equal to the preset threshold value. Accordingly, the optimal probabilities PVs or dl.sub.k-l, i.e., optimal dl.sub.1-1-dl.sub.6-6, of the event for the computation pixels P.sub.k-l, i.e., P.sub.1-1-P.sub.6-6, as shown in
(177) The above process, including the steps S1-S6, is performed to generate the moving window 2 across the computation regions 12 of the MRI slice 10 along the x and y directions to create a two-dimensional (2D) probability map. In order to obtain a three-dimensional (3D) probability map, the above process, including the steps S1-S6, may be applied to each of all MRI slices (including the MRI slice 10) of the subject arranged in the z direction perpendicular to the x and y directions.
(178) The invention provides a computing method, i.e., the steps S1-S6, to obtain values of the specific MRI parameters for multiple large regions or volumes of the MRI image 10 (i.e., the stops of the moving window 2) each covering multiple machine-defined original pixels p of the MRI image 10 and to obtain a probability map having small regions (i.e., computation pixels P.sub.k-l) with extremely accurate probabilities dl.sub.k-l based on the values C.sub.m-n of the specific MRI parameters for the large regions or volumes (i.e., moving windows W.sub.m-n), which overlaps, of the MRI image 10. Because of calculation for the probabilities CL.sub.m-n based on the large regions or volumes (i.e., moving windows W.sub.m-n) of the MRI image 10, registered or aligned errors between the registered image sets (or registered parameter maps) for different parameters can be compensated.
(179) In the algorithm depicted in
(180) By repeating the stops S1-S6 or the steps S5 and S6 for various events such as occurrence of prostate cancer, occurrence of small cell subtype, and occurrence of Gleason scores greater than 7, multiple probability maps for the various events are obtained or formed. The probability maps, for example, include a prostate cancer probability map shown in
(181) In an alternative embodiment, the subset data DB-1 may further include measured values for a PET parameter (e.g., SUVmax) and a SPECT parameter. In this case, the classifier CF, e.g., Bayesian classifier, for the event (e.g., occurrence of prostate cancer) may be created based on data associated with the event and specific variables, including, e.g., the PET parameter, the SPECT parameter, some or all of the MRI parameters depicted in the section of the description of classifier CF, and the processed parameters of average Ve and average Ktrans, in the subset data DB-1. Next, by using the computing method depicted in
(182) In the invention, the computing method (i.e., the steps S1-S6) depicted in
(183) In the case of the MRI image 10 obtained from the subject (e.g., human patient) that has been given a treatment, such as neoadjuvant chemotherapy or (preoperative) radiation therapy, or has taken or been injected with one or more drugs for a treatment, such as neoadjuvant chemotherapy, the effect of the treatment or the drugs on the subject may be evaluated, identified, or determined by analyzing the probability map(s) for the event(s) depicted in the first embodiment and/or the composite probability image or map depicted in the first embodiment. Accordingly, a method of evaluating, identifying, or determining the effect of the treatment or the drugs on the subject may include the following steps: (a) administering to the subject the treatment or the drugs, (b) after the step (a), obtaining the MRI image 10 from the subject by the MRI system, (c) after the step (b), performing the steps S2-S6 to obtain the probability map(s) for the event(s) depicted in the first embodiment and/or obtaining the composite probability image or map depicted in the first embodiment, and (d) after the step (c), analyzing the probability map(s) for the event(s) and/or the composite probability image or map.
(184) The steps S1-S6 may be employed to generate a probability map of breast cancer. In this case, in the steps S1 and S2, the MRI image 10 shows the breast anatomical structure of the subject as shown in
(185)
(186) After the step S21 or S22 is performed, step S23 is performed. In the step S23, the subject is given the treatment, such as a drug given intravenously or orally. For certain cancers such as prostate cancer, the treatment may be the (preoperative) radiation therapy (or called radiotherapy), a proton beam therapy, a minimally invasive treatment (such as ablation or radiation), or an ablation therapy such as high-intensity focused ultrasound treatment. The (preoperative) radiation therapy for prostate cancer may be performed by a radiotherapy device such as Truebeam or CyberKnife and may use high-energy radiation (e.g., gamma rays) to shrink tumors and kill cancer cells.
(187) In a step S24, after the subject gets or receives the treatment such as an oral or intravenous drug, a second MRI slice image is obtained from the subject by the MRI device or system. The second MRI slice image is composed of multiple machine-defined original pixels p.sub.i-j in its FOV to show the same anatomical region of the subject as the first MRI slice image shows. In a step S25, the steps S2-S6 are performed on the second MRI slice image to generate a second probability map. The first and second probability maps may be generated for an event or data type, such as prostate cancer, breast cancer, one of Gleason scores 0 through 10, two or more of Gleason scores 0 through 10 (e.g., Gleason scores greater than 7), tissue necrosis, or the percentage of cancer in a specific range from a first percent (e.g., 91 percent) to a second percent (e.g., 100 percent). Next, in a step S26, by comparing the first and second probability maps, the effect of the treatment or the drug used in the treatment on the subject may be identified, determined, or evaluated as effective or ineffective. Based on the result from the step S26, a doctor can decide or judge whether the treatment or the drug should be adjusted or changed. The method depicted in the steps S21-S26 can detect responses or progression after the treatment or the drug within less than one week or two weeks, allowing earlier adjustments to the treatment regime.
(188)
(189) After the step S31 or S32 is performed, step S33 is performed. In the step S33, the subject is given the treatment, such as a drug given intravenously or orally. For certain cancers such as prostate cancer, the treatment may be the (preoperative) radiation therapy (or called radiotherapy), a proton beam therapy, a minimally invasive treatment (such as ablation or radiation), or an ablation therapy such as high-intensity focused ultrasound treatment. The (preoperative) radiation therapy for prostate cancer may be performed by a radiotherapy device such as Truebeam or CyberKnife and may use high-energy radiation (e.g., gamma rays) to shrink tumors and kill cancer cells.
(190) In a step S34, after the subject gets or receives the treatment such as an oral or intravenous drug, a second MRI slice image is obtained from the subject by the MRI device or system. The second MRI slice image is composed of multiple machine-defined original pixels in its FOV to show the same anatomical region of the subject as the first MRI slice image shows. In a step S35, the steps S2-S5 are performed on the second MRI slice image to obtain second probabilities CL.sub.m-n of the event or data type for stops W.sub.m-n of the moving window 2 for the computation region 12 of the second MRI slice image. In other words, the second probabilities CL.sub.m-n of the event or data type for the stops W.sub.m-n of the moving window 2 on the second MRI slice image for the subject after the treatment are obtained based on values C.sub.m-n of the specific MRI parameters for the stops W.sub.m-n of the moving window 2 on the second MRI slice image to match the matching dataset from the established classifier CF or biomarker library. The values C.sub.m-n of the specific MRI parameters for the stops W.sub.m-n of the moving window 2 on the second MRI slice image, for example, may be obtained from a registered (multi-parametric) image dataset including, e.g., the second MRI slice image and/or different parameter maps registered to the second MRI slice.
(191) The stops W.sub.m-n of the moving window 2 for the computation region 12 of the first MRI slice may substantially correspond to or may be substantially aligned with or registered to the stops W.sub.m-n of the moving window 2 for the computation region 12 of the second MRI slice, respectively. Each of the stops W.sub.m-n of the moving window 2 for the computation region 12 of the first MRI slice and the registered or aligned one of the stops W.sub.m-n of the moving window 2 for the computation region 12 of the second MRI slice may substantially cover the same anatomical region of the subject.
(192) Next, in a step S36, the first and second probabilities CL.sub.m-n of the event or data type for each aligned or registered pair of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images are subtracted from each other into a corresponding probability change PMC or CCL.sub.m-n for said each aligned or registered pair of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images. For example, for each aligned or registered pair of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images, the probability change PMC may be obtained by subtracting the first probability of the event or data type from the second probability of the event or data type.
(193) In a step S37, the algorithm, including the steps S11-S16, depicted in the step S6 is performed based on the probability changes PMCs or CCL.sub.m-n for the aligned or registered pairs of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images to compute probability changes PVCs or cdl.sub.k-l for respective computation pixels P.sub.k-l used to compose a probability change map for the event or data type, as described below. Referring to
(194) In the step S13, a difference DW between the probability change guess PG and the probability change PMC or CCL.sub.m-n for each aligned or registered pair of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images is calculated by, e.g., subtracting the probability change PMC or CCL.sub.m-n for said each aligned or registered pair of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images from the probability change guess PG for said each aligned or registered pair of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images. In the step S14, an absolute value of the difference DW for each aligned or registered pair of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images is compared with the preset threshold error or value to determine whether an error, i.e., the absolute value of the difference DW, between the probability change guess PG and the probability change PMC or CCL.sub.m-n for each aligned or registered pair of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images is less than or equal to the preset threshold error or value. If the absolute values of the differences DWs for all the aligned or registered pairs of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images are determined in the step S14 to be less than or equal to the preset threshold error or value, the step S16 continues. In the step S16, the probability changes PVCs or cdl.sub.k-l for the computation pixels P.sub.k-l are determined to be optimal, which are called optimal probability changes cdl.sub.k-l hereinafter, and the optimal probability changes cdl.sub.k-l of the computation pixels P.sub.k-l form the probability change map for the event or data type. After the optimal probability changes cdl.sub.k-l for the computation pixels P.sub.k-l are obtained in the step S16, the algorithm is completed.
(195) If any one of the absolute values of the differences DWs for all the aligned or registered pairs of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images is determined in the step S14 to be greater than the preset threshold error or value, the step S15 continues. In the step S15, the probability change PVC, i.e., updated cdl.sub.k-l, for each of the computation pixels P.sub.k-l is updated or adjusted by, e.g., subtracting an error correction factor ECF for said each of the computation pixels P.sub.k-l from the current probability change PVC, i.e., current cdl.sub.k-l, for said each of the computation pixels P.sub.k-l. The error correction factor ECF for each of the computation pixels P.sub.k-l is calculated by, e.g., summing error correction contributions from the aligned or registered pairs, of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images, each having their aligned or registered squares 6 covering or overlapping said each of the computation pixels P.sub.k-l; each of the error correction contributions to said each of the computation pixels P.sub.k-l, for example, may be calculated by multiplying the difference DW for a corresponding one of the aligned or registered pairs of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images by an area ratio of an overlapped area between said each of the computation pixels P.sub.k-l and the corresponding one of the aligned or registered pairs of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images to a common area of the squares 4 inscribed in the corresponding one of the aligned or registered pairs of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images. After the probability changes PVCs or cdl.sub.k-l for the computation pixels P.sub.k-l are updated, the steps S12-S15 are performed repeatedly based on the updated probability changes PVCs, i.e., updated cdl.sub.k-l, for the computation pixels P.sub.k-l in the step S15, until the absolute values of the differences DWs for all the aligned or registered pairs of the stops W.sub.m-n of the moving window 2 on the first and second MRI slice images are determined in the step S14 to be less than or equal to the preset threshold error or value.
(196) The above process uses the moving window 2 in the x and y directions to create a 2D probability change map. In addition, the above process may be applied to multiple MRI slices of the subject registered in the z direction, perpendicular to the x and y directions, to form a 3D probability change map.
(197) In a step S38, by analyzing the probability change map, the effect of the treatment or the drug used in the treatment on the subject may be identified, determined, or evaluated as effective or ineffective. Based on the result from the step S38, a doctor can decide or judge whether the treatment or the drug should be adjusted or changed. The method depicted in the steps S31-S38 can detect responses or progression after the treatment or the drugs within less than one week or two weeks, allowing earlier adjustments to the treatment regime.
(198)
(199) After the step S41 or S42 is performed, a step S43 is performed. In the step S43, the subject is given a treatment such as an oral or intravenous drug. For certain cancers such as prostate cancer, the treatment may be the (preoperative) radiation therapy (or called radiotherapy), a proton beam therapy, or an ablation therapy such as high-intensity focused ultrasound treatment. The (preoperative) radiation therapy for prostate cancer may be performed by a radiotherapy device such as Truebeam or CyberKnife and may use high-energy radiation (e.g., gamma rays) to shrink tumors and kill cancer cells.
(200) In a step S44, after the subject gets or receives the treatment such as an oral or intravenous drug, a second MRI slice image is obtained from the subject by the MRI device or system. The second MRI slice image is composed of multiple machine-defined original pixels p.sub.i-j in its FOV to show the same anatomical region of the subject as the first MRI slice image shows. In a step S45, the steps S2-S6 are performed on the second MRI slice image to generate a second probability map composed of second computation pixels P.sub.k-l. Each of the second computation pixels P.sub.k-l may substantially correspond to or may be substantially aligned with or registered to one of the first computation pixels P.sub.k-l. The first and second probability maps may be generated for an event or data type such as prostate cancer, breast cancer, one of Gleason scores 0 through 10, two or more of Gleason scores 0 through 10 (e.g., Gleason scores greater than 7), tissue necrosis, or the percentage of cancer in a specific range from a first percent (e.g., 91 percent) to a second percent (e.g., 100 percent).
(201) In a step S46, by subtracting a probability dl.sub.k-l for each of the first computation pixels P.sub.k-l from a probability dl.sub.k-l for the corresponding, registered or aligned one of the second computation pixels P.sub.k-l, a corresponding probability change cdl.sub.k-l is obtained or calculated. Accordingly, a probability change map is formed or generated based on the probability changes cdl.sub.k-l. Next, in a step S47, by analyzing the probability change map, the effect of the treatment or the drug used in the treatment on the subject may be identified, determined, or evaluated as effective or ineffective. Based on the result from the step S47, a doctor can decide or judge whether the treatment or the drug should be adjusted or changed. The method depicted in the steps S41-S47 can detect responses or progression after the treatment or the drug within less than one week or two weeks, allowing earlier adjustments to the treatment regime.
(202) 1-2. Probability Map Derived from Measured Values for Stops of Two-Dimensional Moving Window
(203) Alternatively,
(204) One or more of computed tomography (CT) parameters, positron emission tomography (PET) parameters, single-photon emission computed tomography (SPECT) parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, and/or bioluminescence optical (BLO) parameters, ultrasound parameters, camera-image parameters and/or visible-light-image parameters may be measured for the one or the set of values C.sub.m-n for said each stop W.sub.m-n of the two-dimensional moving window. The one or the set of values C.sub.m-n of the one or more imaging parameters for said each stop W.sub.m-n of the two-dimensional moving window may be measured from detection or analysis instruments, such as camera, microscope (optical or electronic), endoscope, detectors or spectrometer (visible light, fluorescent light, IR, UV or X-ray), ultrasonic machine or system, magnetic resonance imaging (MRI) machine or system, computed tomography (CT) machine or system, positron emission tomography (PET) machine or system, single-photon emission computed tomography (SPECT) machine or system, micro-PET machine or system, micro-SPECT machine or system, Raman spectrometer or system, and/or bioluminescence optical (BLO) machine or system, or other machine for obtaining molecular or structural imaging data.
(205) Next, referring to
(206) Second Aspect: E Operator for Better Resolution of Measured Values in Two-dimensional Region
(207) II-1. Computational Map Derived from Measured Values for Original Pixels of Two-dimensional Original Map
(208) Alternatively, the step S22-2 as illustrated in
(209) Next, referring to
(210) For more elaboration, the convolution matrix M.sub.cw as illustrated in the first aspect can be deconvoluted to obtain a final or computational matrix M.sub.dp. The deconvolution matrix M.sub.dp comprises a final or computational data, dataset or information for each final or computation pixel in the given 2D region. The data, dataset or information in or related to, or describing each pixel P.sub.k-l of the final or computation pixels in the given 2D region are of the same type, property, category or item (for example, MRI parameters) as that (for example, a MRI parameters) of the original data, dataset or information in the stops of moving window. The data, dataset or information in or related to, or describing each pixel P.sub.k-l of the final or computation pixels can be a number, multiple numbers, a real number, multiple real numbers, a digitized number (for example a negative integer, 0, or a positive integer), multiple digitized numbers, a 0 or 1, multiple 0's or 1's, a scalar, multiple scalars, a vector, multiple vectors, or a tensor with degree of order 0, 1, 2, . . . , t, where t is an integer. The deconvolution E.sub.d of the E operator obtains the data, dataset or information for each final or computation pixel by solving a set of linear equations with unknown computation pixel data (d.sub.k-l's) and known convolution window data (C.sub.m-n's). The linear equations can be established by equating the data, dataset or information for each convolution window stop W.sub.m-n to the data, dataset or information averaged over all the final or computation pixels enclosed by the convolution window (W.sub.m-n), d.sub.k-l. The averaging can be done by linear averaging, Gaussian averaging or Lorentian averaging of d.sub.k-l's.
(211)
(212) Wherein d.sub.k-l's are the data, dataset or information of the final or computation pixels enclosed or within by the stop of the moving window W.sub.m-n, wherein k is from k.sub.1 to k.sub.1+P−1, and l is from l.sub.1 to l.sub.1Q−1, and m=1, 2, 3, . . . , K−P+1; and n=1, 2, 3, . . . , L−Q+1.
(213) There are (K−P+1)×(L−Q+1) equations with knows (C.sub.m-n 's), and K×L unknowns (d.sub.k-l's). The number of unknowns is larger than the number of equations (5) by (PL+KQ−PQ−K−L+P+Q−1). A method to increase number of knows and decrease number of unknowns will be described below by (1) finding uniform or constant data, dataset or information for the final or computation pixels in a region or regions of uniformity or approximately uniformity with the 2D region of interest, and/or (2) finding uniform or constant data, dataset or information for the final or computation pixels in a region or regions of uniformity or approximately uniformity extending from and out of the boundary of the 2D region of interest. The above method (1) may provide a number of knows (known data for the computation pixels) equal to or larger than the number of (PL+KQ−PQ−K−L+P+Q−1) such that the number (K−P+1)×(L−Q+1) of the equations (5) may be solved. If the moving window comprises 3-by-3 computation pixels, the above method (2) may provide a number of knows (known data for the computation pixels) equal to or larger than the number of [(K+2)(L+2)−(K−P+3)×(L−Q+3)] such that the number (K−P+3)×(L−Q+3) of the equations (5) may be solved. The set of linear equations can be solved by a computer, device, machine, processor, system or tool iteratively. The initial guess of each of the unknowns (the data, dataset or information of final or computation pixels), d.sub.k-l0, is obtained by averaging over all the stops of covering or enclosing the pixel. The contribution from each enclosing stop calculated by the area ratio of the overlapped area (A′.sub.m-n) to the area of that stop (A.sub.m-n). d.sub.k-l0 can be obtained using A.sub.m-n, A′.sub.m-n and C.sub.m-n:
(214)
(215) Wherein stops W.sub.m-n cover or enclose the final or computation pixel P.sub.k-l has stop indices m from m.sub.1 to m.sub.2, and n from n.sub.1 to n.sub.2. In the first iteration, we can calculate and obtain the first data, dataset or information for each stop of the moving window, C.sub.m-n1's, by using initial guess d.sub.k-l0's in equation (1). The iteration results in a solution M.sub.dp(K×L) when the set of computation pixel data or information match the set of convolution window data or information with errors or difference smaller than or equal to a specified value or number in the same 2D region. The E.sub.d operator can be expressed as:
E.sub.d(M.sub.cw,W.sub.PQ)=M.sub.dp
(216) In another aspect of the disclosure, the convolution operator E.sub.c and the deconvolution operator E.sub.d can be performed in sequence to get the full E operator. The E operator transform the original matrix M.sub.op (comprising elements of data, dataset or information for the I×J original or initial pixels and has I×J sets or elements or components of data or information) to the deconvolution matrix M.sub.dp (comprising elements of data, dataset or information for the K×L pixels and has K×L sets or elements or components of data or information) in the same given 2D region, through the convolution window matrix M.sub.cw (comprising (K−P+1)×(L−Q+1) sets or elements or components of data or information in the convolution window stops). The E operator can be expressed as
E(M.sub.op(I×J))=E.sub.d(M.sub.cw((K−P+1)×(L−Q+1)))E.sub.dE.sub.c(M.sub.op(I×J))=M.sub.dp(K×L)
(217) In another aspect of the disclosure, this invention discloses the E operator in the linear algebra. The linear operations, such as addition (+), subtraction (−), multiplication by a scalar (d) or division by a scalar (/), are performed using the data or information of each stop of the moving window, (that is using the elements in the convolution matrix M.sub.cw), instead of using the data or information of the original or initial pixels (that is instead of using the elements in the convolution matrix M.sub.op). The moving window is used as a default or standard size, shape, parameters, configuration or format for containing and providing data, dataset or information for analysis, comparison, computing or engineering.
E(aΣ.sub.sC.sub.sM.sub.s)=M
(218) Where M.sub.s or M is a matrix of the convolution M.sub.cw, and C.sub.s are the real numbers, s is an integer from 1, 2, 3, . . . , S, with S a positive integer.
(219) The convolution operation (E.sub.c) described and specified in the second aspect is similar to the convolution operation (E.sub.c) described in the first aspect using MRI detection and diagnosis as an example. The convolution operation (E.sub.c) in the second aspect may be referred to that as illustrated in the first aspect. The MRI parameters as illustrated in the first aspect may be employed for the values C.sub.m-n for the stops W.sub.m-n of the 2D moving window in the second aspect. A 2D moving window may be applied to a 2D object, e.g., 2D image 10, to obtain one of values C.sub.m-n, of a MRI parameter for example, for each of stops W.sub.m-n of the 2D moving window, as illustrated in the first aspect.
(220) For more elaboration, with regard to the deconvolution operation (E.sub.d) in the step S23-2 in
(221) In the step DC5, an error correction factor (ECF) for each of the computation pixels P.sub.k-l is calculated by summing error correction contributions from the stops W.sub.m-n of the moving window overlapping said each of the computation pixels P.sub.k-l. For a general example, if the moving window has a size of 2-by-2 computation pixels, there may be neighboring four of the stops W.sub.m-n of the moving window overlapping one of the computation pixels P.sub.k-l. The error correction contribution from each of said neighboring four of the stops W.sub.m-n to said one of the computation pixels P.sub.k-l may be calculated by multiplying the difference (obtained from the step DC3) for said each of said neighboring four of the stops W.sub.m-n by a space ratio of an overlapped space between said one of the computation pixels P.sub.k-l and said each of said neighboring eight of the stops W.sub.m-n to a space of the moving window. Next, in a step DC6, one of the initial values d.sub.k-l for each of the computation pixels P.sub.k-l may be updated by subtracting the error correction factor (ECF) (obtained from the step DC5) for said each of the computation pixels P.sub.k-l from the initial value d.sub.k-l for said each of the computation pixels P.sub.k-l. Next, in a step DC7, the guess for each of the stops W.sub.m-n of the moving window may be updated by averaging the updated values d.sub.k-l (obtained from the step DC6) for the computation pixels P.sub.k-l inside said each of the stops W.sub.m-n of the moving window. Next, in a step DC8, one of the updated guesses (obtained from the step DC7) for each of the stops W.sub.m-n of the moving window may be compared with one of the values C.sub.m-n for said each of the stops W.sub.m-n of the moving window by subtracting said one of the values C.sub.m-n for said each of the stops W.sub.m-n from said one of the updated guesses (obtained from the step DC7) for said each of the stops W.sub.m-n to obtain an updated difference between said one of the values C.sub.m-n and said one of the updated guesses. Next, in a step DC9, a determination step may be performed to determine whether the absolute value of the updated difference (obtained from the step DC8) is less than or equal to the preset threshold error. If any of the absolute values of the updated differences (obtained from the step DC8) for the respective stops W.sub.m-n is greater than the preset threshold error, the steps DC5-DC9 continues for another iteration. If the absolute value of the updated difference (obtained from the step DC8) for each of the stops W.sub.m-n is less than the preset threshold error, the step DC10 continues.
(222) In the step DC5 in the another iteration, the error correction factor (ECF) for each of the computation pixels P.sub.k-l may be updated by summing updated error correction contributions from the stops W.sub.m-n of the moving window overlapping said each of the computation pixels P.sub.k-l. For the above general example, the updated error correction contribution from said each of said neighboring eight of the stops W.sub.m-n to said one of the computation pixels P.sub.k-l may be calculated by multiplying the updated difference (obtained from the step DC8 in the last iteration) for said each of said neighboring eight of the stops W.sub.m-n by the space ratio. Next, in the step DC6 in the another iteration, one of the values d.sub.k-l for each of the computation pixels P.sub.k-l may be updated by subtracting the updated error correction factor (ECF) (obtained from the step DC5 in the current iteration) for said each of the computation pixels P.sub.k-l from said one of the last updated values d.sub.k-l (obtained from the step DC6 in the last iteration) for said each of the computation pixels P.sub.k-l. Next, in the step DC7 in the another iteration, the guess for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window may be updated by averaging the updated values d.sub.k-l (obtained from the step DC6 in the current iteration) for the computation pixels P.sub.k-l inside said each of the stops W.sub.m-n of the moving window. Next, in the step DC8 in the another iteration, one of the updated guesses (obtained from the step DC7 in the current iteration) for each of the stops W.sub.m-n of the moving window may be compared with one of the values C.sub.m-n for said each of the stops W.sub.m-n of the moving window by subtracting said one of the values C.sub.m-n for said each of the stops W.sub.m-n from said one of the updated guesses (obtained from the step DC7 in the current iteration) for said each of the stops W.sub.m-n to obtain an updated difference between said one of the values C.sub.m-n and said one of the updated guesses (obtained from the step DC7 in the current iteration). Next, in the step DC9 in the another iteration, a determination step may be performed to determine whether the absolute value of the updated difference (obtained from the DC8 in the current iteration) is less than or equal to the preset threshold error. If any of the absolute values of the updated differences (obtained from the DC8 in the current iteration) for the respective stops W.sub.m-n is greater than the preset threshold error, the above steps DC5-DC9 continues for iteration multiple times until the absolute value of the updated difference (obtained from the DC8 in the current iteration) for each of the stops W.sub.m-n is less than the preset threshold error. If the absolute value of the updated difference (obtained from the DC8 in the current iteration) for each of the stops W.sub.m-n is less than or equal to the preset threshold error, the step DC10 continues.
(223) In the step DC10, one of the updated values d.sub.k-l for each of the computation pixels P.sub.k-l may be determined as an optimal value for said each of the computation pixels P.sub.k-l, which may be constructed for a 2D computational map. In an example for an MRI parameter, each of the widths X.sub.fp and Y.sub.fp of the computation pixels P.sub.k-l in the x and y directions may range from 0.1 to 10 millimeters, and preferably range from 0.5 to 3 millimeters. Alternatively, in an example for an infrared absorbance parameter, each of the widths X.sub.fp and Y.sub.fp of the computation pixels P.sub.k-l in the x and y directions may range from 1 to 20 micrometers, and preferably range from 1 to 5 micrometers.
(224) II-2. Computational Map Derived from Measured Values for Stops of Two-Dimensional Moving Window
(225) Alternatively,
(226) One or more of computed tomography (CT) parameters, positron emission tomography (PET) parameters, single-photon emission computed tomography (SPECT) parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, and/or bioluminescence optical (BLO) parameters, ultrasound parameters, camera-image parameters and/or visible-light-image parameters may be measured for the value C.sub.m-n for said each stop W.sub.m-n of the two-dimensional moving window. The value C.sub.m-n of the imaging parameter for said each stop W.sub.m-n of the two-dimensional moving window may be measured from detection or analysis instruments, such as camera, microscope (optical or electronic), endoscope, detectors or spectrometer (visible light, fluorescent light, IR, UV or X-ray), ultrasonic machine or system, magnetic resonance imaging (MRI) machine or system, computed tomography (CT) machine or system, positron emission tomography (PET) machine or system, single-photon emission computed tomography (SPECT) machine or system, micro-PET machine or system, micro-SPECT machine or system, Raman spectrometer or system, and/or bioluminescence optical (BLO) machine or system, or other machine for obtaining molecular or structural imaging data.
(227) In an example, a biopsy tissue may be fixed on a glass slide and the 2D image for the biopsy tissue may be captured by a camera or microscope. An infrared (IR) detector may generate a two-dimensional moving window to be applied to a two-dimensional target region, e.g., biopsy tissue, to measure a value C.sub.m-n of an IR absorbance parameter at a specific spectrum, for example, for each stop W.sub.m-n of the two-dimensional moving window.
(228) Next, the step S23-2 as illustrated in
(229) Third Aspect: E Operator for Better Resolution of Measured Values in Three-Dimensional Space
(230) III-1. Computational Map Derived from Measured Values for Original Voxels of Three-Dimensional Original Map
(231)
(232) Next, referring to
(233) Another aspect of the disclosure provides an algorithm, a method, or an operator, for transformation of data, dataset or information related to original or initial voxels (p.sub.i-j-g) at respective locations, x.sub.i-j-g's, of a 3D space to a data, dataset or information in a final or computation voxels (P.sub.k-l-h) at related locations X.sub.k-l-h's, of the same 3D space, wherein i, j, g, k, l, h are positive integers, i from 1, 2, . . . , to I; j from 1, 2, . . . , to J; g from 1, 2, . . . , to G; k from 1, 2, . . . , to K; l from 1, 2, . . . , to L; and h from 1, 2, . . . , to H. The transformation results in a new set of data, dataset or information of the final or computation voxels with a better resolution and a lower noise as compared to that of the original or initial voxels. K may be different from I, L may be different from J and H may be different from G. For a better resolution and a lower noise, the volume of each of the final or computation voxels is smaller than that of the original or initial voxels; that is K>I, L>j and H>G. Alternatively, when I=K, J=L and H=G, X.sub.k-l-h can be the same as x.sub.i-jh, wherein the noises due to measurement fluctuation in the data, dataset or information of the original or initial voxels are smeared-out. The 3D space may comprise I×J×G voxels in grids of original or initial voxels, wherein the size and numbers of voxels may be determined by a certain detector or sensor used in obtaining the data, dataset or information related to the original or initial voxels, wherein the original or initial voxels are the same as the measuring voxels in detection using a certain detector or sensor. Alternatively, the size and numbers of voxels may be chosen for forming a 3D space or matrix, wherein the data, dataset or information of the original or initial voxels may be obtained based on the data, dataset or information of the measuring voxels in detection using a certain detector or sensor. For example, the data, dataset or information of the original or initial voxel may be calculated by volume averaging of the data, dataset or information of measuring voxels overlapping the original or initial voxel, full or partial. The 3D space may as well comprise K×L×H voxels in grids of final or computation voxels, wherein the size and numbers of voxels may be generated for a desired resolution for analysis, diagnosis or a specific application. The data, dataset or information related to the original or initial voxels may be of a certain type, property, category or item (for example, MRI parameters) obtained from a certain detector or sensor. The data, dataset or information related to the final or computation voxels may be of a same type, property, category or item (as that, for example the MRI parameters, of the original or initial voxels) obtained from the transformation or computation. Alternatively, the data, dataset or information related to the original or initial voxels may be, for examples, the IR absorption images for a given range of wavenumbers, the Raman scattering images for a given range of wavenumbers, the fluorescent light images for a given range of wavenumbers, or the ultrasonic images of a human organ. The original or initial voxels have a dimension in one direction (for example, x direction) x.sub.op, a dimension in a direction perpendicular to x-direction (for example, y direction) y.sub.op and a dimension in a direction perpendicular to the xy plane (for example, z direction) z.sub.op; while the final voxels have a dimension in one direction (for example, x direction) X.sub.fp, a dimension in a direction perpendicular to x-direction (for example, y direction) Y.sub.fp and a dimension in a direction perpendicular to the xy plane (for example, z direction) Z.sub.fp. The final voxels may have the same dimensions (size) as that of the original voxels; or with each voxel having a size larger or smaller than the size of original or initial voxels, while both are in the same 3D space. The data, dataset or information in or related to, or describing each of the original or initial voxels (p.sub.i-j-g) can be a number, multiple numbers, a real number, multiple real numbers, a digitized number (for example a negative integer, 0, or a positive integer), multiple digitized numbers, a 0 or 1, multiple 0's or 1's, a scalar, multiple scalars, a vector, multiple vectors, or a tensor with degree of order 0, 1, 2, . . . , t, where t is an integer.
(234) The disclosed algorithm or operator comprises two operations, a convolution operation (E.sub.c) and the deconvolution operation (E.sub.d). E.sub.c and E.sub.d can be operated separately or together. When combining these two operations together, it is the Engineering operator (E operator), E×E.sub.d E.sub.c. The E operator, as well as the E.sub.c and E.sub.d operators will be described and specifies as follows.
(235) The original data, dataset or information in the original or initial voxels in a given 3D space is transformed to a data, dataset or information in stops of a 3D moving window, with the data, dataset or information of the same type, property, category or item (for example, MRI parameters) as that (for example, a MRI parameters) of the original data, dataset or information in the original or initial voxels. The 3D moving window plays a key role in the E operator or E algorithm. It is defined with some physical, computation, analytical, or statistical purposes for better resolution and lower noise. The size, volume, shape, parameters or format of the 3D moving window may become a default or standard size, volume, shape, parameters or format in collecting, storing, computing, (statistically) analyzing data or information, or engineering learning or machine learning. Usually, the size, volume, shape, parameters or format of the 3D moving window is chosen to enclose at least several original or initial voxels, as well as at least several final or computation voxels. For example, the 3D moving window size, volume and shape can be defined with a volume (x-dimension of the 3D moving window times y-dimension of the 3D moving window times z-dimension of the 3D moving window) equal to a volume of a biopsy sample; wherein the volume of the biopsy sample may be defined by the averaged volume of biopsy samples taken in the standard biopsy procedure using needles having popular or standard sizes. The 3D moving window volume mentioned above is defined as the size, volume, shape, parameters or format of the 3D moving window in the 3D space. The 3D moving window may have a shape of a sphere, an ellipsoid, a cube or a cuboid. When the 3D moving widow has a shape of sphere, the maximum inscribed cube may contain p×p×p original or initial voxels; or P×P×P final or computation voxels: wherein p and P are positive numbers, and is greater than or equal to 1. P, in some cases, is chosen to be a positive integer, and is greater than or equal to 2. When the 3D moving widow has a shape of ellipsoid, the maximum inscribed cuboid may contain p×q×r original or initial voxels; or P×Q×R final or computation voxels: where p, q, r, P, Q and R are positive numbers, and are greater than or equal to 1. P, Q and R, in some cases, are chosen to be positive integers, and are greater than or equal to 2. When the 3D moving widow has a shape of cube, the cube may contain p×p×p original or initial voxels; or P×P×P final or computation voxels: where p, and P are positive numbers, and are greater than or equal to 1. P, in some cases, is chosen to be a positive integer, and is greater than or equal to 2. When the 3D moving widow has a shape of cuboid, the cuboid may contain p×q×r original or initial voxels; or P×Q×R final or computation voxels: where p, q, r, P, Q and R are positive numbers, and greater than or equal to 1. P, Q and R, in some cases, are chosen to be positive integers, and are greater than or equal to 2. The 3D moving widow are stepping in the same 3D space by a step of X.sub.fp in the x direction, a step of Y.sub.fp in the y direction, and a step of Z.sub.fp in the z direction, and resulting in an array of densely populated and overlapped (3D) stops. Each stop overlaps its nearest neighbor stop with a step or shift of X.sub.fp, Y.sub.fp or Z.sub.fp, in the x, y and z directions, respectively. Each stop in the 3D space comprises a number of original voxels, full or partial. The data, dataset or information for each stop is obtained by averaging over all the voxels enclosed by the stop. For some partially enclosed voxels, the averaging computation over these voxels can be done by weighing the enclosed volume proportionally. The averaging can be done by linear averaging, Gaussian averaging or Lorentian averaging. In linear averaging, we assume the data, dataset or information in each stop of 3D moving window is uniform. The above method transforms data, dataset or information in the original or initial voxels to data, dataset or information in stops of 3D moving window; wherein the transform can be called a convolution. The stop of 3D moving window at location X.sub.m-n-u is defined as W.sub.m-n-u, wherein m=1, 2, 3, 4, . . . , M, n=1, 2, 3, 4, . . . , N, and u=1, 2, 3, 4, . . . , U. The data, dataset or information in or related to each stop (W.sub.m-n-u) of the 3D moving window can be a number, multiple numbers, a real number, multiple real numbers, a digitized number (for example a negative integer, 0, or a positive integer), multiple digitized numbers, a 0 or 1, multiple 0's or 1's, a scalar, multiple scalars, a vector, multiple vectors, or a tensor with degree of order 0, 1, 2, . . . , t, where t is an integer. Since the 3D moving window is stepping by the size of a final or computation voxel, the number of the stops is counted in a 3D array of final or computation voxels. Each stop of moving comprises P×Q×R final or computation voxels. The original matrix M.sub.op comprises I×J×G voxels and has I×J×G sets or elements or components of data, dataset or information. The convolution matrix M.sub.cw comprises (K−P+1)×(L−Q+1)×(H−R+1) stops of moving window, and has (K−P+1)×(L−Q+1)×(H−R+1) sets or elements or components of data, dataset or information. The E.sub.c operator transforms original matrix M.sub.op (comprising I×J×G sets or elements of data, dataset or information (for example, MRI parameters) describing or representing each original voxel in the given 3D space) to a convolution matrix M.sub.cw (comprising (K−P+1)×(L−Q+1)×(H−R+1) sets or elements of averaged data, dataset or information (for example, MRI parameters) describing or representing each stop of 3D moving window in the given 3D space) can be expressed as:
E.sub.c(M.sub.op,W.sub.PQR)=M.sub.cw
Wherein M.sub.op has dimension or size I×J×G, the 3D moving window W.sub.PQR has dimension or size P×Q×R, and M.sub.cw has dimension or size ((K−P+1)×(L−Q+1)×(H−R+1). The M.sub.cw comprises elements of data, dataset, or information of the same type, property, category or item as that of M.sub.op. For example, the elements in both M.sub.cw and M.sub.op are data, dataset or information related to the MRI parameters. Alternatively, the elements in both M.sub.cw and M.sub.op are data, dataset or information related to the IR absorption, Raman scattering, fluorescent light, or ultrasonic imaging.
(236) In another aspect of the disclosure, the convolution matrix M.sub.cw can be deconvoltioned to obtain a final or computational matrix M.sub.dp. The deconvolution matrix M.sub.dp comprises a final or computational data, dataset or information for each final or computation voxel in the given 3D space. The data, dataset or information in or related to, or describing each voxel P.sub.k-l-h of the final or computation voxels in the given 3D space are of the same type, property, category or item (for example, MRI parameters) as that (for example, a MRI parameters) of the original data, dataset or information in the stops of 3D moving window. The data, dataset or information in or related to, or describing each voxel P.sub.k-l-h of the final or computation voxels can be a number, multiple numbers, a real number, multiple real numbers, a digitized number (for example a negative integer, 0, or a positive integer), multiple digitized numbers, a 0 or 1, multiple 0's or 1's, a scalar, multiple scalars, a vector, multiple vectors, or a tensor with degree of order 0, 1, 2, . . . , t, where t is an integer. The deconvolution E.sub.d of the E operator obtains the data, dataset or information for each final or computation voxel by solving a set of linear equations with unknown computation pixel data (d.sub.k-l-h's) and known convolution window data (C.sub.m-n-u's). The linear equations can be established by equating the data, dataset or information for each convolution window stop W.sub.m-n-u to the data, dataset or information averaged over all the final or computation voxels enclosed by the convolution window (W.sub.m-n-u), d.sub.k-l-h. The averaging can be done by linear averaging, Gaussian averaging or Lorentian averaging of d.sub.k-l-h's.
(237)
(238) Wherein d.sub.k-l-h's are the data, dataset or information of the final or computation voxels enclosed or within by the stop W.sub.m-n-u of the 3D moving window, wherein k is from k.sub.1 to k.sub.1+P−1, l is from l.sub.1 to l.sub.1+Q−1, h is from h.sub.1 to h.sub.1+R−1, and m=1, 2, 3, . . . , K−P+1; n=1, 2, 3, . . . , L−Q+1, and u=1, 2, 3, . . . , H−R+1.
(239) There are (K−P+1)×(L−Q+1)×(H−R+1) equations with knows (C.sub.m-n-u's), and K×L×H unknowns (d.sub.k-l-h's). The number of unknowns is larger than the number of equations. A method to increase number of knows and decrease number of unknowns will be described below by (1) finding uniform or constant data, dataset or information for the final or computation voxels in a region or regions of uniformity or approximately uniformity with the 3D space of interest, and/or (2) finding uniform or constant data, dataset or information for the final or computation voxels in a region or regions of uniformity or approximately uniformity extending from and out of the boundary of the 3D space of interest. The set of linear equations can be solved by a computer, device, machine, processor, system or tool iteratively. The initial guess of each of the unknowns (the data, dataset or information of final or computation pixels), d.sub.k-l-h0, is obtained by averaging over all the stops of covering or enclosing the voxel. The contribution from each enclosing stop calculated by the volume ratio of the overlapped volume (V′.sub.m-n-u) to the volume of that stop (V.sub.m-n-u). d.sub.k-l-h0 can be obtained using V.sub.m-n-u, V′.sub.m-n-u and C.sub.m-n-u:
(240)
(241) Wherein stops W.sub.m-n-u's covering or enclosing the final or computation voxel P.sub.k-l-h has stop indices m from m.sub.1 to m.sub.2, n from n.sub.1 to n.sub.2, and u from u.sub.1 to u.sub.2. For examples, if the moving window comprises 8 computation voxels (2×2×2), a given computation voxel will be overlapped by 8 (2.sup.3) window stops; if the moving window comprises 27 computation voxels (3×3×3), a given computation voxel will be overlapped by 27 (3.sup.3) window stops; if the moving window comprises 24 computation voxels (2×3×4), a given computation voxel will be overlapped by 24 (2×3×4) window stops. In the first iteration, we can calculate and obtain the first data, dataset or information for each stop of the 3D moving window, C.sub.m-n-u's, by using initial guess x.sub.k-l-h0's in equation (3). The iteration results in a solution M.sub.dp(K×L×H) when the set of computation voxel data or information match the set of convolution window data or information with errors or difference smaller than or equal to a specified value or number in the same 3D space. The E.sub.d operator can be expressed as:
E.sub.d(M.sub.cw,W.sub.PQR)=M.sub.dp
(242) In another aspect of the disclosure, the convolution operator E.sub.c and the deconvolution operator E.sub.d can be performed in sequence to get the full E operator. The E operator transform the original matrix M.sub.op (comprising elements of data, dataset or information for the I×J×G original or initial voxels and has I×J×G sets or elements or components of data or information) to the deconvolution matrix M.sub.dp (comprising elements of data, dataset or information for the K×L×H voxels and has K×L×H sets or elements or components of data or information) in the same given 3D space, through the convolution window matrix M.sub.cw (comprising (K−P+1)×(L−Q+1)×(H−R+1) sets or elements or components of data or information in the convolution window stops). The E operator can be expressed as
E(M.sub.op(I×J×K))=E.sub.d(M.sub.cw((K−P+1)×(L−Q+1)×(H−R+1)))=E.sub.dE.sub.c(M.sub.op(I×J×I)=M.sub.dp(K×L×H)
(243) In another aspect of the disclosure, this invention discloses the E operator in the linear algebra. The linear operations, such as addition (+), subtraction (−), multiplication by a scalar (d) or division by a scalar (/), are performed using the data or information of each stop of the 3D moving window, (that is using the elements in the convolution matrix M.sub.cw), instead of using the data or information of the original or initial voxels (that is instead of using the elements in the convolution matrix M.sub.op). The 3D moving window is used as a default or standard size, volume, shape, parameters, configuration or format for containing and providing data, dataset or information for analysis, comparison, computing or engineering.
E(aΣ.sub.sC.sub.sM.sub.s)=M
(244) Where M.sub.s or M is a matrix of the convolution M.sub.cw, and C.sub.s are the real numbers, s is an integer from 1, 2, 3, . . . , S, with S a positive integer.
(245) Referring to
(246) The MRI parameters as illustrated in the first aspect may be employed for the values C.sub.m-n-u for the stops W.sub.m-n-u of the 3D moving window in the third aspect.
(247) Alternatively, one or more of computed tomography (CT) parameters, positron emission tomography (PET) parameters, single-photon emission computed tomography (SPECT) parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, and/or bioluminescence optical (BLO) parameters, ultrasound parameters, infrared absorbance parameters, camera-image parameters and/or visible-light-image parameters may also be measured for the value C.sub.m-n-u for said each stop W.sub.m of the three-dimensional moving window in the third aspect. The data, dataset or information C.sub.m-n-u for the stops W.sub.m-n-u of the three-dimensional moving window in the third aspect may be obtained from detection or analysis instruments, such as camera, microscope (optical or electronic), endoscope, detectors or spectrometer (visible light, fluorescent light, IR, UV or X-ray), ultrasonic machine or system, magnetic resonance imaging (MRI) machine or system, computed tomography (CT) machine or system, positron emission tomography (PET) machine or system, single-photon emission computed tomography (SPECT) machine or system, micro-PET machine or system, micro-SPECT machine or system, Raman spectrometer or system, and/or bioluminescence optical (BLO) machine or system, or other machine for obtaining molecular or structural imaging data.
(248) The algorithm in the third aspect may be employed to transform the data, dataset or information C.sub.m-n-u for the stops W.sub.m-n-u of the 3D moving window into the data, dataset or information d.sub.k-l-h for the computation voxel P.sub.k-l-h. The data, dataset or information d.sub.k-l-h may be calculated as mentioned above in the third aspect.
(249) For more elaboration, an example is mentioned as below:
(250) Referring to
(251) Referring to
(252) Referring to
(253) (1) moving step by step with a distance equal to the width X.sub.fp of the cube 106 in the x direction (equal to the width of the computation voxels P.sub.k-l-h in the x direction) from a left side of the 3D image 100 to a right side of the 3D image 100 in a row to obtain one of the values C.sub.m-n-u for each of the stops W.sub.m-n-u of the 3D moving window 102 in the row; for an example, the 3D moving window 102 may move step by step, in a frontmost two of the MRI slices 10 aligned in the z direction for the 3D image 100, with a distance equal to the width X.sub.fp of the cube 106 in the x direction (equal to the width of the computation voxels P.sub.k-l-h in the x direction) from the left side of the 3D image 100 to the right side of the 3D image 100 in the topmost row to obtain one of the values C.sub.1-1-1-C.sub.N-1-1 for each of the stops W.sub.1-1-1-W.sub.N-1-1 of the 3D moving window 102 as seen in
(254) (2) moving to the next row of the 3D image 100 with a distance equal to the width Y.sub.fp of the cube 106 in the y direction (equal to the width of the computation voxels P.sub.k-l-h in the y direction) to repeat the step (1) to obtain one of the values C.sub.m-n-u for each of the stops W.sub.m-n-u of the 3D moving window 102 in the next bottom row, wherein the steps (1) and (2) repeat as seen in
(255) (3) moving to the next combination of the MRI slices 10 aligned in the z direction for the 3D image 100 with a distance equal to the width Z.sub.fp of the cube 106 in the z direction (equal to the width of the computation voxels P.sub.k-l-h in the z direction) to repeat the steps (1) and (2) to obtain one of the values C.sub.m-n-u for each of the stops W.sub.m-n-u of the 3D moving window 102, wherein the steps (1), (2) and (3) repeat until the 3D moving window 102 move to the backmost combination of the MRI slices 10 aligned in the z direction for the 3D image 100 to repeat the steps (1) and (2) in the backmost combination of the MRI slices 10 aligned in the z direction for the 3D image 100; for the example, the 3D moving window 102 may repeat the steps (1) and (2) plane by plane with a distance equal to the width Z.sub.fp of the cube 106 in the z direction (equal to the width of the computation voxels P.sub.k-l-h in the z direction) from the frontmost two of the MRI slices 10 aligned in the z direction for the 3D image 100 to the backmost two of the MRI slices 10 aligned in the z direction for the 3D image 100 to obtain one of the values C.sub.1-1-1-C.sub.N-N-N for each of the stops W.sub.1-1-1-W.sub.N-N-N of the 3D moving window 102 as seen in
(256) For further elaboration, one of the computation voxels P.sub.k-l-h may be in multiple of the stops W.sub.m-n-u of the 3D moving window 102 partially overlapping with each other and one another. In the example, the 3D moving window 102 may have 222 cubes. One of the computation voxels P.sub.k-l-h may be in eight of the stops W.sub.m-n-u of the 3D moving window 102 partially overlapping with one another as seen in
(257) Next, referring to
(258) For more elaboration, with regard to the deconvolution operation (E.sub.d), in a step DC1, one of the initial values d.sub.1-1-1-d.sub.(N+1)-(N+1)-(N+1) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be first calculated or assumed based on an average of the values C.sub.1-1-1-C.sub.N-N-N for the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 overlapping said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1). Next, in a step DC2, a guess for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be calculated by averaging the initial values d.sub.1-1-1-d.sub.(N+1)-(N+1)-(N+1) (obtained from the step DC1) for the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) inside said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102. Next, in a step DC3, one of the guesses (obtained from the step DC2) for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be compared with one of the values C.sub.1-1-1-C.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 by subtracting said one of the values C.sub.1-1-1-C.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N from said one of the guesses (obtained from the step DC2) for said each of the stops W.sub.1-1-1-W.sub.N-N-N to obtain a difference between said one of the values C.sub.1-1-1-C.sub.N-N-N and said one of the guesses (obtained from the step DC2). Next, in a step DC4, a determination step may be performed to determine whether the absolute value of the difference (obtained from the step DC3) is less than or equal to a preset threshold error. If any of the absolute values of the differences (obtained from the step DC3) for the respective stops W.sub.1-1-1-W.sub.N-N-N is greater than the preset threshold error, a step DC5 continues. If the absolute value of the difference (obtained from the step DC3) for each of the stops W.sub.1-1-1-W.sub.N-N-N is less than or equal to the preset threshold error, a step DC10 continues.
(259) In the step DC5, an error correction factor (ECF) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) is calculated by summing error correction contributions from the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 overlapping said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1). For a general example, if the moving window 102 has a size of 2-by-2-by-2 computation voxels, there may be neighboring eight of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 overlapping one of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) The error correction contribution from each of said neighboring eight of the stops W.sub.1-1-1-W.sub.N-N-N to said one of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be calculated by multiplying the difference (obtained from the step DC3) for said each of said neighboring eight of the stops W.sub.1-1-1-W.sub.N-N-N by a space ratio of an overlapped space between said one of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) and said each of said neighboring eight of the stops W.sub.1-1-1-W.sub.N-N-N to a space of the moving window 102. Next, in a step DC6, one of the initial values d.sub.1-1-1-d.sub.(N+1)-(N+1)-(N+1) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be updated by subtracting the error correction factor (ECF) (obtained from the step DC5) for said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) from the initial value d.sub.1-1-1-d.sub.(N+1)-(N+1)-(N+1) for said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1). Next, in a step DC7, the guess for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be updated by averaging the updated values d.sub.1-1-1-d.sub.(N+1)-(N+1)-(N+1) (obtained from the step DC6) for the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) inside said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102. Next, in a step DC8, one of the updated guesses (obtained from the step DC7) for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be compared with one of the values C.sub.1-1-1-C.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 by subtracting said one of the values C.sub.1-1-1-C.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N from said one of the updated guesses (obtained from the step DC7) for said each of the stops W.sub.1-1-1-W.sub.N-N-N to obtain an updated difference between said one of the values C.sub.1-1-1-C.sub.N-N-N and said one of the updated guesses. Next, in a step DC9, a determination step may be performed to determine whether the absolute value of the updated difference (obtained from the step DC8) is less than or equal to the preset threshold error. If any of the absolute values of the updated differences (obtained from the step DC8) for the respective stops W.sub.1-1-1-W.sub.N-N-N is greater than the preset threshold error, the steps DC5-DC9 continues for another iteration. If the absolute value of the updated difference (obtained from the step DC8) for each of the stops W.sub.1-1-1-W.sub.N-N-N is less than the preset threshold error, the step DC10 continues.
(260) In the step DC5 in the another iteration, the error correction factor (ECF) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be updated by summing updated error correction contributions from the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 overlapping said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) For the above general example, the updated error correction contribution from said each of said neighboring eight of the stops W.sub.1-1-1-W.sub.N-N-N to said one of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be calculated by multiplying the updated difference (obtained from the step DC8 in the last iteration) for said each of said neighboring eight of the stops W.sub.1-1-1-W.sub.N-N-N by the space ratio. Next, in the step DC6 in the another iteration, one of the values d.sub.1-1-1-d.sub.(N+1)-(N+1)-(N+1) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be updated by subtracting the updated error correction factor (ECF) (obtained from the step DC5 in the current iteration) for said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) from said one of the last updated values d.sub.1-1-1-d.sub.(N+1)-(N+1)-(N+1) (obtained from the step DC6 in the last iteration) for said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1). Next, in the step DC7 in the another iteration, the guess for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be updated by averaging the updated values d.sub.1-1-1-d.sub.(N+1)-(N+1)-(N+1) (obtained from the step DC6 in the current iteration) for the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) inside said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102. Next, in the step DC8 in the another iteration, one of the updated guesses (obtained from the step DC7 in the current iteration) for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be compared with one of the values C.sub.1-1-1-C.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 by subtracting said one of the values C.sub.1-1-1-C.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N from said one of the updated guesses (obtained from the step DC7 in the current iteration) for said each of the stops W.sub.1-1-1-W.sub.N-N-N to obtain an updated difference between said one of the values C.sub.1-1-1-C.sub.N-N-N and said one of the updated guesses (obtained from the step DC7 in the current iteration). Next, in the step DC9 in the another iteration, a determination step may be performed to determine whether the absolute value of the updated difference (obtained from the DC8 in the current iteration) is less than or equal to the preset threshold error. If any of the absolute values of the updated differences (obtained from the DC8 in the current iteration) for the respective stops W.sub.1-1-1-W.sub.N-N-N is greater than the preset threshold error, the above steps DC5-DC9 continues for iteration multiple times until the absolute value of the updated difference (obtained from the DC8 in the current iteration) for each of the stops W.sub.1-1-1-W.sub.N-N-N is less than the preset threshold error. If the absolute value of the updated difference (obtained from the DC8 in the current iteration) for each of the stops W.sub.1-1-1-W.sub.N-N-N is less than or equal to the preset threshold error, the step DC10 continues.
(261) In the step DC10, one of the updated values d.sub.1-1-1-d.sub.(N+1)-(N+1)-(N+1) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be determined as an optimal value for said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1), which may be constructed for a 3D computational map.
(262) III-2. Computational Map Derived from Measured Values for Stops of Three-Dimensional Moving Window
(263) Alternatively, referring to
(264) One or more of computed tomography (CT) parameters, positron emission tomography (PET) parameters, single-photon emission computed tomography (SPECT) parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, and/or bioluminescence optical (BLO) parameters, ultrasound parameters, camera-image parameters and/or visible-light-image parameters may be measured for the value C.sub.m-n-u for said each stop W.sub.m-n-u of the three-dimensional moving window. The value C.sub.m-n-u of the imaging parameter for said each stop W.sub.m-n-u of the three-dimensional moving window may be measured from detection or analysis instruments, such as camera, microscope (optical or electronic), endoscope, detectors or spectrometer (visible light, fluorescent light, IR, UV or X-ray), ultrasonic machine or system, magnetic resonance imaging (MRI) machine or system, computed tomography (CT) machine or system, positron emission tomography (PET) machine or system, single-photon emission computed tomography (SPECT) machine or system, micro-PET machine or system, micro-SPECT machine or system, Raman spectrometer or system, and/or bioluminescence optical (BLO) machine or system, or other machine for obtaining molecular or structural imaging data.
(265) Next, the step S23-2 as illustrated in
(266) III-3. Summary for Third Aspect
(267) Referring to
(268) In this summary for the third aspect, for the step S23-2, said calculating the second value, e.g. d for the first computation voxel, e.g. P.sub.k-l-h in
(269) In this summary for the third aspect, for the step S23-2, said updating the first assumed value for the first computation voxel, e.g. P.sub.k-l-h in
(270) Fourth Aspect: E Operator for Better Resolution of Probabilities of Event in Three-Dimensional Space Via Big-Data Engineering Learning
(271) IV-1. Probability Map Derived from Measured Values for Original Voxels of Three-Dimensional Original Map
(272) Referring to
(273) Next, referring to
(274) Next, referring to
(275) In this aspect, engineering learning or machine learning is performed using the data, dataset or information related to a 3D moving window, or using the standard size, shape, parameters or format or dimensions of the 3D moving window. The description and specification of the steps, processes and methods related to the convolution operator are the same as in the above. As described and specified above, the convolution operator E.sub.c transforms the original matrix M.sub.op (comprising data, dataset or information (for example, MRI parameters) describing or representing each original or initial voxel in the given 3D space) to a convolution matrix M.sub.cw (comprising averaged data, dataset or information (for example, MRI parameters) describing or representing each stop of 3D moving window in the given 3D space). Through the engineering learning, machine learning or correlation, the data, dataset or information of the elements of M.sub.cw may be transformed to a data, dataset or information in a different type, property, item or category. For example, based on big data (accumulated data of correlated clinical biopsy analysis data and the measured MRI parameters for patients) and using (for example) Bayesian inference, the M.sub.op (elements of MRI parameters) can be transformed or constructed into a matrix of learning window ML.sub.w comprising elements of the probabilities of cancer occurrence. Since the 3D moving window is stepping by the size of a final or computation voxel, the number of the stops is counted in a 3D array of final or computation voxels. Each stop of 3D moving window comprises P×Q×R final or computation voxels. The original matrix M.sub.op comprises I×J×G voxels and has I×J×G sets or elements or components of data, dataset or information. The convolution matrix M.sub.cw and the learning matrix ML.sub.w both comprise (K−P+1)×(L−Q+1)×(H−R+1) stops of 3D moving window, and has (K−P+1)×(L−Q+1)×(H−R+1) sets or elements or components of data, dataset or information. The E.sub.c operator transforms original matrix M.sub.op (comprising I×J×G sets or elements of data, dataset or information (for example, MRI parameters) describing or representing each original voxel in the given 3D space) to a convolution matrix M.sub.cw (comprising (K−P+1)×(L−Q+1)×(H−R+1) sets or elements of averaged data, dataset or information (for example, MRI parameters) describing or representing each stop of 3D moving window in the given 3D space). The E.sub.1 operator transforms the convolution matrix M.sub.cw (comprising (K−P+1)×(L−Q+1)×(H−R+1) sets or elements of averaged data, dataset or information (for example, MRI parameters) describing or representing each stop of 3D moving window in the given 3D space) to a learning matrix ML.sub.w (comprising (K−P+1)×(L−Q+1)×(H−R+1) sets or elements of learned data, dataset or information (for example, the probability of the cancer occurrence) describing or representing each stop of 3D moving window in the given 3D space). The engineering learning operator (or the machine learning operator), E.sub.1, can be expressed as:
E.sub.1(M.sub.cw,W.sub.PQR)=ML.sub.w
(276) wherein the 3D moving window comprises P×Q×R final or computation voxels with P, Q and R in the x, y and z directions, respectively, and the stops W.sub.m-n-u's are at locations with m, n and u final or computation voxels in the given 3D space, wherein m=1, 2, 3, . . . , M, n=1, 2, 3, . . . , N, and u=1, 2, 3, . . . , U. The data, dataset or information in or related to, or describing each element of the learning matrix ML.sub.w for the stop W.sub.m-n-u in the given 3D space is of a different type, property, category or item (for example, the probability of the occurrence of a cancer) as compared to that (for example, MRI parameters) in or related to, or describing each element of the convolution matrix M.sub.cw for the stop W.sub.m-n-u in the same given 3D space. While the data, dataset or information in or related to, or describing each element of the convolution matrix M.sub.cw for the stop W.sub.m-n-u in the given 3D space is of a same type, property, category or item (for example, MRI parameters) as compared to that (for example, MRI parameters) in or related to, or describing each element of the original matrix M.sub.op for the original or initial voxel in the same given 3D space. Alternatively, the data, dataset or information related to the original or initial voxels may be, for examples, the IR absorption images for a given range of wavenumbers, the Raman scattering images for a given range of wavenumbers, the fluorescent light images for a given range of wavenumbers, or the ultrasonic images of a human organ. As described and specified in the above, the 3D moving window plays a key role in the engineering learning operator or algorithm (E operator). It is defined with some physical, computation, analytical, or statistical purposes. Furthermore, the size, volume, shape, parameters or format of the 3D moving window is used for the engineering learning or machine learning. The size, volume, shape, parameters or format of the 3D moving window may become a default or standard size, volume or format in collecting, storing, computing, (statistically) analyzing data or information, or engineering learning or machine learning. The methods, algorithms or procedures of engineering learning or machine learning for transforming M.sub.cw to ML.sub.w may be, for example, using (i) statistics, for example, Baysian inference, (ii) connection or association, for example, neuro-computing, (iii) Symbolism: for example, induction or interpretation, (iv) analog, for example, resemblance, (v) evolution, for example, nature processes.
(277) Similar to the deconvolution of M.sub.cw described and specified above, the learning matrix ML.sub.w can be also deconvoltioned to obtain a final or computational matrix ML.sub.dp. The size, volume, shape, parameters or format of the final or computation voxels are described and specified as in the above. The deconvolution matrix ML.sub.dp comprises a final or computational data, dataset or information for each final or computation voxel in the given 3D space. The data, dataset or information in or related to, or describing each voxel P.sub.k-l-h of the final or computation voxels in the given 3D space are of the same type, property, category or item (for example, the probability of the occurrence of a cancer) as that (for example, the probability of the occurrence of a cancer) of the learned data, dataset or information of the elements in ML.sub.w for the stops W.sub.m-n-u of 3D moving window. The data, dataset or information in or related to, or describing each voxel P.sub.k-l-h of the final or computation voxels in the given 3D space are of a different type, property, category or item (for example, the probability of the occurrence of a cancer) as that (for example, MRI parameters) of the data, dataset or information of the elements in M.sub.cw for the stops W.sub.m-n-u of moving window. The data, dataset or information in or related to, or describing each voxel P.sub.k-l-h of the final or computation voxels in the given 3D space are of a different type, property, category or item (for example, the probability of the occurrence of a cancer) as that (for example, MRI parameters) of the data, dataset or information of the elements in M.sub.op for the original or initial voxels x.sub.i-j-g. Alternatively, for examples, based on big data (accumulated data of correlated clinical biopsy analysis result or data and the measured IR absorption, Raman scattering data, fluorescent lights or ultrasonic imaging from the correspondent biopsy samples of patients) and using, for example, Bayesian inference, the M.sub.op (IR absorption, Raman scattering data, fluorescent lights or ultrasonic imaging) can be transformed or constructed into a matrix of learning window ML.sub.w comprising elements of the probabilities of cancer occurrence.
(278) The data, dataset or information in or related to, or describing each voxel P.sub.k-l-h of the final or computation voxels can be a number, multiple numbers, a real number, multiple real numbers, a digitized number (for example a negative integer, 0, or a positive integer), multiple digitized numbers, a 0 or 1, multiple 0's or 1's, a scalar, multiple scalars, a vector, multiple vectors, or a tensor with degree of order 0, 1, 2, . . . , t, where t is an integer. The deconvolution operator E.sub.d of the E operator obtains the data, dataset or information for each final or computation voxel by solving a set of linear equations with unknown computation voxel data (dl.sub.k-l-h's) and known data (CL.sub.m-n-u's) of stops of the 3D moving windows. The linear equations can be established by equating the data, dataset or information for each stop W.sub.m-n-u of moving window to the data, dataset or information averaged over all dl.sub.k-l-h's of the final or computation voxels enclosed by the stop (W.sub.m-n-u) of the 3D moving window. The averaging can be done by linear averaging, Gaussian averaging or Lorentian averaging of dl.sub.k-l-h's.
(279)
(280) Wherein dl.sub.k-l-h's are the data, dataset or information of the final or computation voxels enclosed by or within the stop W.sub.m-n-u of the 3D moving window, wherein k is from k.sub.1 to k.sub.1+P−1, l is from l.sub.1 to l.sub.1+Q−1, h is from h.sub.1 to h.sub.1+R−1; and m=1, 2, 3, . . . , K−P+1, n=1, 2, 3, . . . , L−Q+1, u=1, 2, 3, . . . , H−R+1.
(281) There are (K−P+1)×(L−Q+1)×(H−R+1) equations with knows (CL.sub.m-n-u's), and K×L×H unknowns (dl.sub.k-l-h's). The number of unknowns is larger than the number of equations. A method to increase number of knows and decrease number of unknowns will be described below by (1) finding uniform or constant data, dataset or information for the final or computation voxels in a region or regions of uniformity or approximately uniformity within the 3D space of interest, and/or (2) finding uniform or constant data, dataset or information for the final or computation voxels in a region or regions of uniformity or approximately uniformity extending from and near or along the boundary of the 3D space of interest. The set of linear equations can be solved by a computer, device, machine, processor, system or tool iteratively. The initial guess of each of the unknowns (the data, dataset or information of final or computation voxels), dl.sub.k-l-h0, is obtained by averaging over all the stops covering or enclosing the voxel. The contribution from each enclosing stop calculated by the a volume ratio of the overlapped area (V′.sub.m-n-u) to the volume of that stop (V.sub.m-n-u). dl.sub.k-l-h0 can be obtained using V.sub.m-n-u, V′.sub.m-n-u and CL.sub.m-n-u:
(282)
(283) Wherein stops W.sub.m-n-u's covering or enclosing the final or computation voxel P.sub.k-l-h have stop indices m from m.sub.1 to m.sub.2, n from n.sub.1 to n.sub.2, and u from u.sub.1 to u.sub.2. In the first iteration, we can calculate and obtain the first data, dataset or information for each stop of the 3D moving window, CL.sub.m-n-u1's, by using initial guess dl.sub.k-l-h0's in equation (4). The iteration results in a solution ML.sub.dp(K×L×H) when the set of computation voxel data or information match the set of learning window data or information with errors or difference smaller than or equal to a specified value or number in the same 3D space. The E.sub.d operator can be expressed as:
E.sub.d(ML.sub.w,W.sub.PQR)=ML.sub.dp
(284) In another aspect of the disclosure, the convolution operator E.sub.c, the learning operator E.sub.1 and the deconvolution operator E.sub.d can be performed in sequence to get the full E operator. The E operator transform the original matrix M.sub.op (comprising elements of data, dataset or information for the I×J×G original or initial voxels and has I×J×G sets or elements or components of data or information) to the deconvolution matrix M.sub.dp (comprising elements of data, dataset or information for the K×L×H voxels and has K×L×H sets or elements or components of data or information) in the same given 3D space, through the convolution window matrix M.sub.cw (comprising (K−P+1)×(L−Q+1)×(H−R+1) sets or elements or components of data or information of the convolution window stops) and through the learning window matrix ML.sub.w (comprising (K−P+1)×(L−Q+1)×(H−R+1) sets or elements or components of data or information of the learning window stops). The E operator can be expressed as
E(M.sub.op(I×J×G))E.sub.d(ML.sub.w((K−P+1)×(L−Q+1)×(H−R+1)))E.sub.dE.sub.l(M.sub.cw((K−P+1)×(L−Q+1)×(H−R+1)))E.sub.dE.sub.lE.sub.c(M.sub.op(I×J×G))ML.sub.dp(K×L×H)
(285) In another aspect of the disclosure, this invention discloses the E operator in the linear algebra. The linear operations, such as addition (+), subtraction (−), multiplication by a scalar (d) or division by a scalar (/), are performed using the data or information of each stop of the 3D moving window, (that is using the elements in the convolution matrix M.sub.cw or the elements in the learning window ML.sub.w), instead of using the data or information of the original or initial voxels (that is instead of using the elements in the convolution matrix M.sub.op). The 3D moving window is used as a default or standard size, volume, configuration or format for containing and providing data, dataset or information for analysis, comparison, computing, engineering learning or machine learning.
E(aΣ.sub.sC.sub.sM.sub.s)=M
(286) Where M.sub.s or M is a matrix of the convolution matrix M.sub.cw, or the learning matrix ML.sub.w, and C.sub.s are the real numbers, s is an integer from 1, 2, 3, . . . , S, with S a positive integer.
(287) Referring to
(288) (1) moving step by step with a distance equal to the width X.sub.fp of the cube 106 in the x direction (equal to the width of the computation voxels P.sub.k-l-h in the x direction) from a left side of the 3D image 100 to a right side of the 3D image 100 in a row to obtain one of the value sets C.sub.m-n-u for each of the stops W.sub.m-n-u of the 3D moving window 102 in the row; for an example, the 3D moving window 102 may move step by step, in a frontmost two of the MRI slices 10 aligned in the z direction for the 3D image 100, with a distance equal to the width X.sub.fp, of the cube 106 in the x direction (equal to the width of the computation voxels P.sub.k-l-h in the x direction) from the left side of the 3D image 100 to the right side of the 3D image 100 in the topmost row to obtain one of the value sets C.sub.1-1-1-C.sub.N-1-1 for each of the stops W.sub.1-1-1-W.sub.N-1-1 of the 3D moving window 102 as seen in
(289) (2) moving to the next row of the 3D image 100 with a distance equal to the width Y.sub.fp of the cube 106 in the y direction (equal to the width of the computation voxels P.sub.k-l-h in the y direction) to repeat the step (1) to obtain one of the value sets C.sub.m-n-u for each of the stops W.sub.m-n-u of the 3D moving window 102 in the next bottom row, wherein the steps (1) and (2) repeat as seen in
(290) (3) moving to the next combination of the MRI slices 10 aligned in the z direction for the 3D image 100 with a distance equal to the width Z.sub.fp of the cube 106 in the z direction (equal to the width of the computation voxels P.sub.k-l-h in the z direction) to repeat the steps (1) and (2) to obtain one of the value sets C.sub.m-n-u for each of the stops W.sub.m-n-u of the 3D moving window 102, wherein the steps (1), (2) and (3) repeat until the 3D moving window 102 move to the backmost combination of the MRI slices 10 aligned in the z direction for the 3D image 100 to repeat the steps (1) and (2) in the backmost combination of the MRI slices 10 aligned in the z direction for the 3D image 100; for the example, the 3D moving window 102 may repeat the steps (1) and (2) plane by plane with a distance equal to the width Z.sub.fp of the cube 106 in the z direction (equal to the width of the computation voxels P.sub.k-l-h in the z direction) from the frontmost two of the MRI slices 10 aligned in the z direction for the 3D image 100 to the backmost two of the MRI slices 10 aligned in the z direction for the 3D image 100 to obtain one of the value sets C.sub.1-1-1-C.sub.N-N-N for each of the stops W.sub.1-1-1-W.sub.N-N-N of the 3D moving window 102 as seen in
(291) Each of the value sets C.sub.1-1-1-C.sub.N-N-N may be a combination of multiple values for various parameters. Each of the value sets C.sub.1-1-1-C.sub.N-N-N having multiple values for various parameters for one of the stops W.sub.1-1-1-W.sub.N-N-N of the 3D moving window 102. In an example for an MRI parameter, each of the widths X.sub.fp, Y.sub.fp, and Z.sub.fp, of the computation voxels P.sub.k-l-h in the x, y and z directions may range from 0.1 to 10 millimeter, and preferably range from 0.5 to 3 millimeters.
(292) The MRI parameters as illustrated in the first aspect may be employed for the values C.sub.m-n-u for the stops W.sub.m-n-u of the 3D moving window in the fourth aspect.
(293) The algorithm in the fourth aspect may be employed to transform, via the engineering learning E.sub.c the value sets C.sub.m-n-u, each having the values for various MRI parameters, for the respective stops W.sub.m-n-u of the 3D moving window into the computation voxel data i.e., probabilities of an event, for the respective computation voxels P.sub.k-l-h.
(294) Alternatively, each combination of computed tomography (CT) parameters, positron emission tomography (PET) parameters, single-photon emission computed tomography (SPECT) parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, and/or bioluminescence optical (BLO) parameters, ultrasound parameters, infrared absorbance parameters, camera-image parameters and/or visible-light-image parameters may also be taken for a value set C.sub.m-n-u for one of the stops W.sub.m-n-u of the 3D moving window in the fourth aspect. The data, dataset or information C.sub.m-n-u for the stops W.sub.m-n-u of the 3D moving window in the fourth aspect may be obtained from detection or analysis instruments, such as camera, microscope (optical or electronic), endoscope, detectors or spectrometer (visible light, fluorescent light, IR, UV or X-ray), ultrasonic machine or system, magnetic resonance imaging (MRI) machine or system, computed tomography (CT) machine or system, positron emission tomography (PET) machine or system, single-photon emission computed tomography (SPECT) machine or system, micro-PET machine or system, micro-SPECT machine or system, Raman spectrometer or system, and/or bioluminescence optical (BLO) machine or system, or other machine for obtaining molecular or structural imaging data. The algorithm in the fourth aspect may be employed to transform, via the engineering learning E, the data, dataset or information C.sub.m-n-u for the stops W.sub.m-n-u of the 3D moving window into the computation voxel data i.e., probability of an event, for the computation voxel P.sub.k-l-n.
(295) Next, referring to
(296) Next, 28, 29, 30A-30C and 31, in a step S22-3 for deconvolution operation (E.sub.d), one of probabilities dl.sub.1-1-1-dl.sub.(N+1)-(N+1)-(N+1) of the event for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be calculated based on the probabilities CL.sub.1-1-1-CL.sub.N-N-N of the event for the respective stops W.sub.1-1-1-W.sub.N-N-N each covering said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) wherein each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) has a smaller volume than that of the three-dimensional moving window 102.
(297) For more elaboration, with regard to the deconvolution operation (E.sub.d), in a step DL1, one of the original probabilities dl.sub.1-1-1-dl.sub.(N+1)-(N+1)-(N+1) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be first calculated or assumed based on an average of the probabilities CL.sub.1-1-1-CL.sub.N-N-N for the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 overlapping said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) Next, in a step DL2, a probability guess for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be calculated by averaging the original probabilities dl.sub.1-1-1-dl.sub.(N+1)-(N+1)-(N+1) (obtained from the step DL1) for the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) inside said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102. Next, in a step DL3, one of the probability guesses (obtained from the step DL2) for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be compared with one of the probabilities CL.sub.1-1-1-CL.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 by subtracting said one of the probabilities CL.sub.1-1-1-CL.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N from said one of the probability guesses (obtained from the step DL2) for said each of the stops W.sub.1-1-1-W.sub.N-N-N to obtain a difference between said one of the probabilities CL.sub.1-1-1-CL.sub.N-N-N and said one of the probability guesses (obtained from the step DL2). Next, in a step DL4, a determination step may be performed to determine whether the absolute value of the difference (obtained from the step DL3) is less than or equal to a preset threshold error. If any of the absolute values of the differences (obtained from the step DL3) for the respective stops W.sub.1-1-1-W.sub.N-N-N is greater than the preset threshold error, a step DL5 continues. If the absolute value of the difference (obtained from the step DL3) for each of the stops W.sub.1-1-1-W.sub.N-N-N is less than or equal to the preset threshold error, a step DL10 continues.
(298) In the step DL5, an error correction factor (ECF) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) is calculated by summing error correction contributions from the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 overlapping said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1). For a general example, if the moving window 102 has a size of 2-by-2-by-2 computation voxels, there may be eight of the neighboring stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 overlapping one of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1). The error correction contribution from each of said neighboring eight of the stops W.sub.1-1-1-W.sub.N-N-N to said one of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be calculated by multiplying the difference (obtained from the step DL3) for said each of said neighboring eight of the stops W.sub.1-1-1-W.sub.N-N-N by a space ratio of an overlapped space between said one of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) and said each of said neighboring eight of the stops W.sub.1-1-1-W.sub.N-N-N to a space of the moving window 102. Next, in a step DL6, one of the original probabilities dl.sub.1-1-1-dl.sub.(N+1)-(N+1)-(N+1) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be updated by subtracting the error correction factor (ECF) (obtained from the step DL5) for said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) from the original probability dl.sub.1-1-1-dl.sub.(N+1)-(N+1)-(N+1) for said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1). Next, in a step DL7, the probability guess for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be updated by averaging the updated probabilities dl.sub.1-1-1-dl.sub.(N+1)-(N+1)-(N+1) (obtained from the step DL6) for the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) inside said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102. Next, in a step DL8, one of the updated probability guesses (obtained from the step DL7) for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be compared with one of the probabilities CL.sub.1-1-1-CL.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 by subtracting said one of the probabilities CL.sub.1-1-1-CL.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N from said one of the updated probability guesses (obtained from the step DL7) for said each of the stops W.sub.1-1-1-W.sub.N-N-N to obtain an updated difference between said one of the probabilities CL.sub.1-1-1-CL.sub.N-N-N and said one of the updated probability guesses (obtained from the step DL7). Next, in a step DL9, a determination step may be performed to determine whether the absolute value of the updated difference (obtained from the step DL8) is less than or equal to the preset threshold error. If any of the absolute values of the updated differences (obtained from the step DL8) for the respective stops W.sub.1-1-1-W.sub.N-N-N is greater than the preset threshold error, the steps DL5-DL9 continues for another iteration. If the absolute value of the updated difference (obtained from the step DL8) for each of the stops W.sub.1-1-1-W.sub.N-N-N is less than or equal to the preset threshold error, the step DL10 continues.
(299) In the step DL5 in the another iteration, the error correction factor (ECF) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be updated by summing updated error correction contributions from the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 overlapping said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) For the above general example, the updated error correction contribution from said each of said neighboring eight of the stops W.sub.1-1-1-W.sub.N-N-N to said one of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be calculated by multiplying the updated difference (obtained from the step DL8 in the last iteration) for said each of said neighboring eight of the stops W.sub.1-1-1-W.sub.N-N-N by the space ratio. Next, in the step DL6 in the another iteration, one of the probabilities dl.sub.1-1-1-dl.sub.(N+1)-(N+1)-(N+1) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be updated by subtracting the updated error correction factor (ECF) (obtained from the step DL5 in the current iteration) for said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) from said one of the last updated probabilities dl.sub.1-1-1-dl.sub.(N+1)-(N+1)-(N+1) (obtained from the step DL6 in the last iteration) for said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1). Next, in the step DL7 in the another iteration, the probability guess for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be updated by averaging the updated probabilities dl.sub.1-1-1-dl.sub.(N+1)-(N+1)-(N+1) (obtained from the step DL6 in the current iteration) for the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) inside said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102. Next, in the step DL8 in the another iteration, one of the updated probability guesses (obtained from the step DL7 in the current iteration) for each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 may be compared with one of the probabilities CL.sub.1-1-1-CL.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N of the moving window 102 by subtracting said one of the probabilities CL.sub.1-1-1-CL.sub.N-N-N for said each of the stops W.sub.1-1-1-W.sub.N-N-N from said one of the updated probability guesses (obtained from the step DL7 in the current iteration) for said each of the stops W.sub.1-1-1-W.sub.N-N-N to obtain an updated difference between said one of the probabilities CL.sub.1-1-1-CL.sub.N-N-N and said one of the updated probability guesses (obtained from the step DL7 in the current iteration). Next, in the step DL9 in the another iteration, a determination step may be performed to determine whether the absolute value of the updated difference (obtained from the DL8 in the current iteration) is less than or equal to the preset threshold error. If any of the absolute values of the updated differences (obtained from the DL8 in the current iteration) for the respective stops is greater than the preset threshold error, the above steps DL5-DL9 continues for iteration multiple times until the absolute value of the updated difference (obtained from the DC8 in the current iteration) for each of the stops W.sub.1-1-1-W.sub.N-N-N is less than or equal to the preset threshold error. If the absolute value of the updated difference (obtained from the DL8 in the current iteration) for each of the stops W.sub.1-1-1-W.sub.N-N-N is less than or equal to the preset threshold error, the step DL10 continues.
(300) In the step DL10, one of the updated probabilities dl.sub.1-1-1-dl.sub.(N+1)-(N+1)-(N+1) for each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1) may be determined as an optimal probability for said each of the computation voxels P.sub.1-1-1-P.sub.(N+1)-(N+1)-(N+1), which may be constructed for a 3D probability map.
(301) An effect of a treatment, such as neoadjuvant chemotherapy or (preoperative) radiation therapy, or a drug for the treatment on a subject (e.g., human or animal) may be evaluated, identified or determined by comparing probabilities for two stops of the 3D moving windows before and after the treatment. Referring to
(302) Referring to
(303) Referring to
(304) The stops W.sub.m-n-u of the 3D moving window 102 for the computation space of the first 3D MRI slice may substantially correspond to or may be substantially aligned with or registered to the stops W.sub.m-n-u of the 3D moving window 102 for the computation space of the second 3D MRI slice, respectively. Each of the stops W.sub.m-n-u of the 3D moving window 102 for the computation space of the first 3D MRI slice and the registered or aligned one of the stops W.sub.m-n-u of the 3D moving window 102 for the computation space of the second 3D MRI slice may cover substantially the same anatomical space of the subject.
(305) Next, referring to
(306) Referring to
(307) The probability change PVC or cdl.sub.k-l-h for each of the computation voxels P.sub.k-l-h is assumed by, e.g., averaging the probability changes PMCs or CCL.sub.m-n-u, of the aligned or registered pairs, of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images, each overlapping or covering said each of the computation voxels P.sub.k-l-h. In the step S12, a probability change guess PG for each aligned or registered pair of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images is calculated by, e.g., averaging the probability changes PVCs or cdl.sub.k-l-h for all the computation voxels P.sub.k-l-h inside said each aligned or registered pair of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images.
(308) In the step S13, a difference DW between the probability change guess PG and the probability change PMC or CCL.sub.m-n-u for each aligned or registered pair of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images is calculated by, e.g., subtracting the probability change PMC or CCL.sub.m-n-u for said each aligned or registered pair of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images from the probability change guess PG for said each aligned or registered pair of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images. In the step S14, an absolute value of the difference DW for each aligned or registered pair of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images is compared with the preset threshold error or value to determine whether an error, i.e., the absolute value of the difference DW, between the probability change guess PG and the probability change PMC or CCL.sub.m-n-u for each aligned or registered pair of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images is less than or equal to the preset threshold error or value. If the absolute values of the differences DWs for all the aligned or registered pairs of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images are determined in the step S14 to be less than or equal to the preset threshold error or value, the step S16 continues. In the step S16, the probability changes PVCs or cdl.sub.k-l-h for the computation voxels P.sub.k-l-h are determined to be optimal, which are called optimal probability changes cdl.sub.k-l-h hereinafter, and the optimal probability changes cdl.sub.k-l-h of the computation voxels P.sub.k-l-h form the 3D probability change map for the event or data type. After the optimal probability changes cdl.sub.k-l-h for the computation voxels P.sub.k-l-h are obtained in the step S16, the algorithm is completed.
(309) If any one of the absolute values of the differences DWs for all the aligned or registered pairs of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images is determined in the step S14 to be greater than the preset threshold error or value, the step S15 continues. In the step S15, the probability change PVC, i.e., updated cdl.sub.k-l-h, for each of the computation voxels P.sub.k-l-h is updated or adjusted by, e.g., subtracting an error correction factor ECF for said each of the computation voxels P.sub.k-l-h from the current probability change PVC, i.e., current cdl.sub.k-l-h, for said each of the computation voxels P.sub.k-l-h. The error correction factor ECF for each of the computation voxels P.sub.k-l-h is calculated by, e.g., summing error correction contributions from the aligned or registered pairs, of the stops W.sub.m-n of the 3D moving window 102 on the first and second 3D MRI slice images, each covering or overlapping said each of the computation voxels P.sub.k-l-h; each of the error correction contributions to said each of the computation voxels P.sub.k-l-h, for example, may be calculated by multiplying the difference DW for a corresponding one of the aligned or registered pairs of the stops W.sub.m-n of the 3D moving window 102 on the first and second 3D MRI slice images by a space ratio of an overlapped space between said each of the computation voxels P.sub.k-l-h and the corresponding one of the aligned or registered pairs of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images to a common space of the corresponding one of the aligned or registered pairs of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images. After the probability changes PVCs or cdl.sub.k-l-h for the computation voxels P.sub.k-l-h are updated, the steps S12-S15 are performed repeatedly based on the updated probability changes PVCs, i.e., updated cdl.sub.k-l-h, for the computation voxels P.sub.k-l-h in the step S15, until the absolute values of the differences DWs for all the aligned or registered pairs of the stops W.sub.m-n-u of the 3D moving window 102 on the first and second 3D MRI slice images are determined in the step S14 to be less than or equal to the preset threshold error or value.
(310) The above process uses the 3D moving window 102 in the x, y and z directions to create a 3D probability change map.
(311) In the step S38, by analyzing the probability change map, the effect of the treatment or the drug used in the treatment on the subject may be identified, determined, or evaluated as effective or ineffective. Based on the result from the step S38, a doctor can decide or judge whether the treatment or the drug should be adjusted or changed. The method depicted in the steps S31-S38 can detect responses or progression after the treatment or the drugs within less than one week or two weeks, allowing earlier adjustments to the treatment regime.
(312) Alternatively, the effect of a treatment, such as neoadjuvant chemotherapy or (preoperative) radiation therapy, or a drug used in the treatment on a subject (e.g., human or animal) may be evaluated, identified, or determined in another way as seen in
(313) After the step S41 or S42 is performed, the step S43 is performed. In the step S43, the subject is given a treatment such as an oral or intravenous drug. For certain cancers such as prostate cancer, the treatment may be the (preoperative) radiation therapy (or called radiotherapy), a proton beam therapy, or an ablation therapy such as high-intensity focused ultrasound treatment. The (preoperative) radiation therapy for prostate cancer may be performed by a radiotherapy device such as Truebeam or CyberKnife and may use high-energy radiation (e.g., gamma rays) to shrink tumors and kill cancer cells.
(314) In the step S44, after the subject gets or receives the treatment such as an oral or intravenous drug, a second 3D MRI slice image is obtained from the subject by the MRI device or system. The second 3D MRI slice image is composed of multiple machine-defined original voxels p.sub.i-j-g in its FOV to show the same anatomical space of the subject as the first 3D MRI slice image shows. In the step S45, the steps S22-1 through S22-3 are performed on the second 3D MRI slice image to generate a second 3D probability map composed of second computation voxels P.sub.k-l-h. Each of the second computation voxels P.sub.k-l-h may substantially correspond to or may be substantially aligned with or registered to one of the first computation voxels P.sub.k-l-h. The first and second 3D probability maps may be generated for an event or data type such as prostate cancer, breast cancer, one of Gleason scores 0 through 10, two or more of Gleason scores 0 through 10 (e.g., Gleason scores greater than 7), tissue necrosis, or the percentage of cancer in a specific range from a first percent (e.g., 91 percent) to a second percent (e.g., 100 percent).
(315) In the step S46, by subtracting a probability dl.sub.k-l-h for each of the first computation voxels P.sub.k-l-h from a probability dl.sub.k-l-h for the corresponding, registered or aligned one of the second computation voxels P.sub.k-l-h, a corresponding probability change cdl.sub.k-l-h is obtained or calculated. Accordingly, a 3D probability change map is formed or generated based on the probability changes cdl.sub.k-l-h. Next, in the step S47, by analyzing the 3D probability change map, the effect of the treatment or the drug used in the treatment on the subject may be identified, determined, or evaluated as effective or ineffective. Based on the result from the step S47, a doctor can decide or judge whether the treatment or the drug should be adjusted or changed. The method depicted in the steps S41-S47 can detect responses or progression after the treatment or the drug within less than one week or two weeks, allowing earlier adjustments to the treatment regime.
(316) IV-2. Probability Map Derived from Measured Values for Stops of Three-Dimensional Moving Window
(317) Referring to
(318) One or more of computed tomography (CT) parameters, positron emission tomography (PET) parameters, single-photon emission computed tomography (SPECT) parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, and/or bioluminescence optical (BLO) parameters, ultrasound parameters, camera-image parameters and/or visible-light-image parameters may be measured for the one or the set of values C.sub.m-n-u for said each stop W.sub.m-n-u of the three-dimensional moving window. The one or the set of values C.sub.m-n-u of the one or more imaging parameters for said each stop W.sub.m-n-u of the three-dimensional moving window may be measured from detection or analysis instruments, such as camera, microscope (optical or electronic), endoscope, detectors or spectrometer (visible light, fluorescent light, IR, UV or X-ray), ultrasonic machine or system, magnetic resonance imaging (MRI) machine or system, computed tomography (CT) machine or system, positron emission tomography (PET) machine or system, single-photon emission computed tomography (SPECT) machine or system, micro-PET machine or system, micro-SPECT machine or system, Raman spectrometer or system, and/or bioluminescence optical (BLO) machine or system, or other machine for obtaining molecular or structural imaging data.
(319) Next, the steps S22-2 and S22-3 as illustrated in
(320) IV-3. Summary for Fourth Aspect
(321) Referring to
(322) In this summary for the fourth aspect, for the step S22-3, said calculating the second probability, e.g. dl.sub.k-l-h, of the event for the first computation voxel, e.g. P.sub.k-l-h in
(323) In this summary for the fourth aspect, for the step S22-3, said updating the first assumed probability of the event for the first computation voxel, e.g. P.sub.k-l-h in
(324) Fifth Aspect: Fixed Value or Probability Set for Computation Pixels or Voxels at Border of Two-Dimensional or Three-Dimensional Computational Map
(325) As mentioned above, in the following equations:
(326)
(327) The number of unknowns, i.e., d.sub.k-l, dl.sub.k-l, d.sub.k-l-h or dl.sub.k-l-h may be larger than the number of equations. The above-mentioned method to increase number of knows and decrease number of unknowns will be described below by (1) finding uniform or constant data, dataset or information for the final or computation pixels or voxels, i.e., d.sub.k-l, dl.sub.k-l, d.sub.k-l-h or dl.sub.k-l-h in a region, space, regions or spaces of uniformity or approximately uniformity within the 2D or 3D image of interest, and/or (2) finding uniform or constant data, dataset or information for the final or computation pixels or voxels, i.e., d.sub.k-l, dl.sub.k-l, d.sub.k-l-h or dl.sub.k-l-h, in a region, space, regions or spaces of uniformity or approximately uniformity extending from and out of the boundary of the 2D or 3D image. The boundary may be a border of a 2D or 3D image for the border of a biopsy sample. In an example, the data or information for an outside region at the border of the 2D or 3D image may be the data or information of a glass holder, which is uniform and approximate uniform and may be used as the background data or information.
(328) V-1. Computational Map Derived from Measured Values for Original Pixels of Two-Dimensional Original Map
(329)
(330) Referring to
(331) Referring to
(332) Alternatively, referring to
(333) For more elaboration, referring to
(334) After the value d.sub.K-L of the imaging parameter for the computation pixel P.sub.K-L is solved, the values d.sub.(K−1)-L and P.sub.K-(L−1) of the imaging parameter for the respective pixels P.sub.(K−1)-L and P.sub.K-(L−1) next to the computation pixel P.sub.K-L at its left and upper sides respectively may be solved. Referring to
(335) Referring to
(336) After the values d.sub.K-L, d.sub.(K−1)-L and d.sub.K-(L−1) of the imaging parameter for the respective pixels P.sub.K-L, P.sub.(K−1)-L and P.sub.K-(L−1) are solved, the unknown values d.sub.(K−2)-L, d.sub.(K−1)-(L−1) and d.sub.K-(L−2) of the imaging parameter for the respective pixels P.sub.(K−2)-L, P.sub.(K−1)-(L−1) and P.sub.K-(L−2) next to the computation pixel P.sub.(K−1)-L or P.sub.K-(L−1) may be solved. Thereby, the unknown values d.sub.k-l of the imaging parameter for the computation pixels P.sub.k-l in the target region 11 may be solved pixel by pixel in leftward or upward direction until the value d.sub.1-1 of the imaging parameter for the computation pixel P.sub.1-1 is solved. Accordingly, all of the values d.sub.1-1-d.sub.K-L of the imaging parameter for the computation pixels P.sub.1-1-P.sub.K-L within the target region 11 may be solved.
(337) V-2. Computational Map Derived from Measured Values for Stops of Two-Dimensional Moving Window
(338) Alternatively,
(339) One or more of computed tomography (CT) parameters, positron emission tomography (PET) parameters, single-photon emission computed tomography (SPECT) parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, and/or bioluminescence optical (BLO) parameters, ultrasound parameters, camera-image parameters and/or visible-light-image parameters may be measured for the value C.sub.1-1-C.sub.(M+2)-(N+2) of the imaging parameter for said each stop W.sub.1-1-W.sub.(M+2)-(N+2) of the two-dimensional moving window. The value C.sub.1-1-C.sub.(M+2)-(N+2) of the imaging parameter for said each stop W.sub.1-1-W.sub.(M+2)-(N+2) of the two-dimensional moving window may be measured from detection or analysis instruments, such as camera, microscope (optical or electronic), endoscope, detectors or spectrometer (visible light, fluorescent light, IR, UV or X-ray), ultrasonic machine or system, magnetic resonance imaging (MRI) machine or system, computed tomography (CT) machine or system, positron emission tomography (PET) machine or system, single-photon emission computed tomography (SPECT) machine or system, micro-PET machine or system, micro-SPECT machine or system, Raman spectrometer or system, and/or bioluminescence optical (BLO) machine or system, or other machine for obtaining molecular or structural imaging data.
(340) In an example, referring to
(341) Next, referring to
(342) V-3. Computational Map Derived from Measured Values for Original Voxels of Three-Dimensional Original Map
(343) Alternatively,
(344) For example, referring to
(345) (1) moving step by step with a distance or shift equal to the width X.sub.fp of the cube 106, i.e., computation voxel, in the x direction from a left side of the three-dimensional original map to a right side of the three-dimensional original map in a row and across the target and outside spaces 100 and 103 to obtain one of the values C.sub.m-n-u of the imaging parameter for each of the stops W.sub.m-n-u of the three-dimensional moving window 102 in the row; for an example, the three-dimensional moving window 102 may move step by step, in frontmost two of the MRI slices 10 aligned in the z direction for the three-dimensional original map, with a distance or shift equal to the width X.sub.fp of the cube 106, i.e., computation voxel, in the x direction from the left side of the three-dimensional original map to the right side of the three-dimensional original map in the topmost row to obtain one of the values C.sub.1-1-1-C.sub.(N+1)-1-1 of the imaging parameter for each of the stops W.sub.1-1-1-W.sub.(N+1)-1-1 of the three-dimensional moving window 102 as seen in
(346) (2) moving to the next row of the three-dimensional original map with a distance or shift equal to the width Y.sub.fp of the cube 106, i.e., computation voxel, in the y direction to repeat the step (1) in a row of the three-dimensional original map to obtain one of the values C.sub.m-n-u of the imaging parameter for each of stops W.sub.m-n-u of the three-dimensional moving window 102 in the next bottom row, wherein the steps (1) and (2) repeat in a plane of the three-dimensional original map until the three-dimensional moving window 102 move to the bottommost row of the three-dimensional original map to repeat the step (1) in the bottommost row of the three-dimensional original map to obtain one of the values C.sub.m-n-u of the imaging parameter for each of the stops W.sub.m-n-u of the three-dimensional moving window 102 in a plane; for the example, the three-dimensional moving window 102 may move to the second topmost row with a distance or shift equal to the width Y.sub.fp of the cube 106, i.e., computation voxel, in the y direction in the frontmost two of the MRI slices 10 aligned in the z direction for the three-dimensional original map to repeat the step (1) in a row of the three-dimensional original map to obtain one of the values C.sub.1-2-1-C.sub.(N+1)-2-1 of the imaging parameter for each of the stops W.sub.1-2-1-W.sub.(N+1)-2-1 of the three-dimensional moving window 102; each of the stops W.sub.1-2-1-W.sub.N-2-1 is in the target space 100, but the stop W.sub.(N+1)-2-1 has a left portion within the target space 100 and a right portion in the outside space 103; the three-dimensional moving window 102 may repeat the step (1) row by row in the frontmost two of the MRI slices 10 aligned in the z direction for the three-dimensional original map until the three-dimensional moving window 102 moves to the bottommost row of the three-dimensional original map to repeat in the step (1) in the bottommost row of the three-dimensional original map to obtain one of the values C.sub.1-1-1-C.sub.(N+1)-(N+1)-1 of the imaging parameter for each of the stops W.sub.1-1-1-W.sub.(N+1)-(N+1)-1 of the three-dimensional moving window 102 as seen in
(347) (3) moving to the next combination of the MRI slices 10 aligned in the z direction for the three-dimensional original map with a distance or shift equal to the width Z.sub.fp of the cube 106, i.e., computation voxel, in the z direction to repeat the steps (1) and (2) in a plane of the three-dimensional original map to obtain one of the values C.sub.m-n-u of the imaging parameter for each of the stops W.sub.m-n-u of the three-dimensional moving window 102; the steps (1), (2) and (3) repeat in a space until the three-dimensional moving window 102 move to the backmost combination of the MRI slices 10 of the three-dimensional original map to repeat the steps (1) and (2) in the backmost combination of the MRI slices 10 aligned in the z direction for the three-dimensional original map; for the example, the three-dimensional moving window 102 may repeat the steps (1) and (2) plane by plane with a distance or shift equal to the width Z.sub.fp of the cube 106 in the z direction from the frontmost two of the MRI slices 10 aligned in the z direction for the three-dimensional original map to the backmost two of the MRI slices 10 aligned in the z direction for the three-dimensional original map to obtain one of the values C.sub.1-1-1-C.sub.(N+1)-(N+1)-(N+1) of the imaging parameter for each of the stops W.sub.1-1-1-W.sub.(N+1)-(N+1)-(N+1) of the three-dimensional moving window 102 as seen in
(348) Referring to
(349) Referring to
(350) Alternatively, one of the values d.sub.1-1-1-d.sub.K-L-H of the imaging parameter for each of the computation voxels P.sub.1-1-1-P.sub.K-L-H within the target space 100 may be solved from the value d.sub.K-L-H of the imaging parameter for the computation voxel P.sub.K-L-H at a corner of the target space 100 among the right-side, bottom-side and backside borders of the target space 100, as described in the following method. At the beginning, the value d.sub.K-L-H of the imaging parameter for the computation voxel P.sub.K-L-H at the corner of the target space 100 among the right-side, bottom-side and backside borders of the target space 100 may be first calculated. Next, the values d.sub.1-1-1-d.sub.(K−1)-L-H, d.sub.K-1-1-d.sub.K-(L−1)-H and d.sub.K-L-1-d.sub.K-L-(H−1) of the imaging parameter for the respective computation voxels P.sub.1-1-1-P.sub.(K−1)-L-H, P.sub.K-1-1-P.sub.K-(L−1)-H and P.sub.K-L-1-P.sub.K-L-(H−1) within the target space 100 may be solved voxel by voxel from one of the values d.sub.k-l-h of the imaging parameter for one of the computation voxels P.sub.k-l-h to another value d.sub.(k−1)-l-h of the imaging parameter for another computation voxel P.sub.(k−1)-l-h shifted from said one of the computation voxels P.sub.k-l-h by the width X.sub.fp of the computation voxels P.sub.k-l-h in the leftward direction, from one of the values d.sub.k-l-h of the imaging parameter for one of the computation voxels P.sub.k-l-h to another value d.sub.k-(l−1)-h of the imaging parameter for another computation voxel P.sub.k-(l−1)-h shifted from said one of the computation voxels P.sub.k-l-h by the width Y.sub.fp of the computation voxels P.sub.k-l-h in the upward direction, or from one of the values d.sub.k-l-h of the imaging parameter for one of the computation voxels P.sub.k-l-h to another value d.sub.k-l-(h−1) of the imaging parameter for another computation voxel P.sub.k-l-(h−1) shifted from said one of the computation voxels P.sub.k-l-h by the width Z.sub.fp, of the computation voxels P.sub.k-l-h in the frontward direction.
(351) V-4. Computational Map Derived from Measured Values for Stops of Three-Dimensional Moving Window
(352) Alternatively, the process as illustrated in
(353) One or more of computed tomography (CT) parameters, positron emission tomography (PET) parameters, single-photon emission computed tomography (SPECT) parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, and/or bioluminescence optical (BLO) parameters, ultrasound parameters, camera-image parameters and/or visible-light-image parameters may be measured for the value C.sub.1-1-1-C.sub.(N+1)-(N+1)-(N+1) of the imaging parameter for said each stop W.sub.1-1-1-W.sub.(N+1)-(N+1)-(N+1) of the three-dimensional moving window. The value C.sub.1-1-1-C.sub.(N+1)-(N+1)-(N+1) of the imaging parameter for said each stop W.sub.1-1-1-W.sub.(N+1)-(N+1)-(N+1) of the three-dimensional moving window may be measured from detection or analysis instruments, such as camera, microscope (optical or electronic), endoscope, detectors or spectrometer (visible light, fluorescent light, IR, UV or X-ray), ultrasonic machine or system, magnetic resonance imaging (MRI) machine or system, computed tomography (CT) machine or system, positron emission tomography (PET) machine or system, single-photon emission computed tomography (SPECT) machine or system, micro-PET machine or system, micro-SPECT machine or system, Raman spectrometer or system, and/or bioluminescence optical (BLO) machine or system, or other machine for obtaining molecular or structural imaging data.
(354) Next, referring to
(355) V-5. Probability Map Derived from Measured Values for Original Pixels of Two-Dimensional Original Map
(356)
(357) Next, referring to
(358) Referring to
(359) Next, referring to
(360) Alternatively, referring to
(361) V-6. Probability Map Derived from Measured Values for Stops of Two-Dimensional Moving Window
(362) Alternatively,
(363) One or more of computed tomography (CT) parameters, positron emission tomography (PET) parameters, single-photon emission computed tomography (SPECT) parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, and/or bioluminescence optical (BLO) parameters, ultrasound parameters, camera-image parameters and/or visible-light-image parameters may be measured for the one or the set of values C.sub.1-1-C.sub.(M+2)-(N+2) for said each stop W.sub.1-1-W.sub.(M+2)-(N+2) of the two-dimensional moving window. The one or the set of values C.sub.1-1-C.sub.(M+2)-(N+2) of the one or more imaging parameters for said each stop W.sub.1-1-W.sub.(M+2)-(N+2) of the two-dimensional moving window may be measured from detection or analysis instruments, such as camera, microscope (optical or electronic), endoscope, detectors or spectrometer (visible light, fluorescent light, IR, UV or X-ray), ultrasonic machine or system, magnetic resonance imaging (MRI) machine or system, computed tomography (CT) machine or system, positron emission tomography (PET) machine or system, single-photon emission computed tomography (SPECT) machine or system, micro-PET machine or system, micro-SPECT machine or system, Raman spectrometer or system, and/or bioluminescence optical (BLO) machine or system, or other machine for obtaining molecular or structural imaging data.
(364) Next, referring to
(365) V-7. Probability Map Derived from Measured Values for Original Voxels of Three-Dimensional Original Map
(366) Alternatively, referring to
(367) Next, referring to
(368) Next, referring to
(369) Referring to
(370) Alternatively, one of the probabilities dl.sub.1-1-1-dl.sub.K-L-H of the event for each of the computation voxels P.sub.1-1-1-P.sub.K-L-H within the target space 100 may be solved from the probability dl.sub.K-L-H of the event for the computation voxel P.sub.K-L-H at a corner of the target space 100 among the right-side, bottom-side and backside borders of the target space 100, as described in the following method. At the beginning, the probability dl.sub.K-L-H of the event for the computation voxel P.sub.K-L-H at the corner of the target space 100 among the right-side, bottom-side and backside borders of the target space 100 may be first calculated. Next, the probabilities dl.sub.1-1-1-dl.sub.(K−1)-L-H, dl.sub.K-1-1-dl.sub.K-(L−1)-H and dl.sub.K-L-1-dl.sub.K-L-(H−1) of the event for the respective computation voxels P.sub.1-1-1-P.sub.(K−1)-L-H, P.sub.K-1-1-P.sub.K-(L−1)-H and P.sub.K-L-1-P.sub.K-L-(H−1) within the target space 100 may be solved voxel by voxel from one of the probabilities dl.sub.k-l-h of the event for one of the computation voxels P.sub.k-l-h to another probability dl.sub.(k−1)-l-h of the event for another computation voxel P.sub.(k−1)-l-h shifted from said one of the computation voxels P.sub.k-l-h by the width X.sub.fp of the computation voxels P.sub.k-l-h in the leftward direction, from one of the probabilities dl.sub.k-l-h of the event for one of the computation voxels P.sub.k-l-h to another probability dl.sub.k-(l−1)-h of the event for another computation voxel P.sub.k-(l−1)-h shifted from said one of the computation voxels P.sub.k-l-h by the width Y.sub.fp of the computation voxels P.sub.k-l-h in the upward direction, or from one of the probabilities dl.sub.k-l-h of the event for one of the computation voxels P.sub.k-l-h to another probability dl.sub.k-l-(h−1) of the event for another computation voxel P.sub.k-l-(h−1) shifted from said one of the computation voxels P.sub.k-l-h by the width Z.sub.fp of the computation voxels P.sub.k-l-h in the frontward direction.
(371) V-8. Probability Map Derived from Measured Values for Stops of Three-Dimensional Moving Window
(372) Referring to
(373) One or more of computed tomography (CT) parameters, positron emission tomography (PET) parameters, single-photon emission computed tomography (SPECT) parameters, micro-PET parameters, micro-SPECT parameters, Raman parameters, and/or bioluminescence optical (BLO) parameters, ultrasound parameters, camera-image parameters and/or visible-light-image parameters may be measured for the one or the set of values C.sub.1-1-1-C.sub.(M+1)-(N+1)-(U+1) for said each stop W.sub.1-1-1-W.sub.(M+1)-(N+1)-(U+1) of the three-dimensional moving window. The one or the set of values C.sub.1-1-1-C.sub.(M+1)-(N+1)-(U+1) of the one or more imaging parameters for said each stop W.sub.1-1-1-W.sub.(M+1)-(N+1)-(U+1) of the three-dimensional moving window may be measured from detection or analysis instruments, such as camera, microscope (optical or electronic), endoscope, detectors or spectrometer (visible light, fluorescent light, IR, UV or X-ray), ultrasonic machine or system, magnetic resonance imaging (MRI) machine or system, computed tomography (CT) machine or system, positron emission tomography (PET) machine or system, single-photon emission computed tomography (SPECT) machine or system, micro-PET machine or system, micro-SPECT machine or system, Raman spectrometer or system, and/or bioluminescence optical (BLO) machine or system, or other machine for obtaining molecular or structural imaging data.
(374) Next, the steps S28-2 through S28-4 as illustrated in
(375) Sixth Aspect: Fixed Value or Probability Set for Computation Pixels or Voxels in Uniform Region or Space of Two-Dimensional or Three-Dimensional Computational Map
(376) VI-1. Computational Map Derived from Measured Values for Original Pixels of Two-Dimensional Original Map
(377)
(378) Referring to
(379)
(380) Next, referring to
(381) Referring to
(382) Next, referring to
(383) Referring to
(384) VI-2. Computational Map Derived from Measured Values for Stops of Two-Dimensional Moving Window
(385) Alternatively,
(386) Next, referring to
(387) VI-3. Computational Map Derived from Measured Values for Original Voxels of Three-Dimensional Original Map
(388) Alternatively,
(389) Next, referring to
(390) Next, referring to
(391) If the absolute value of the ratio of the difference between a value C.sub.m1-n1-u1 of the imaging parameter for a specific stop W.sub.m1-n1-u1 and one of the values C.sub.(m1−1)-(n1−1)-(u1−1)-C.sub.(m+1)-(n1+1)-(u1−1), C.sub.(m1−1)-(n1−1)-u1-C.sub.(m1−1)-(n1+1)-u1, C.sub.(m1+1)-(n1−1)-u1-C.sub.(m1+1)-(n1+1)-u1, C.sub.m1-(n1−1)-u1, C.sub.m1-(n1+1)-u1, C.sub.(m1−1)-(n1+1)-(u1+1)-C.sub.(m1+1)-(n1+1)-(u1+1) of the imaging parameter for each of its neighboring stops W.sub.(m1−1)-(n1−1)-(u1−1)-W.sub.(m1+1)-(n1+1)-(u1−1), W.sub.(m1−1)-(n1−1)-u1-W.sub.(m1−1)-(n1+1)-u1, W.sub.(m1+1)-(n1−1)-u1-W.sub.(m1+1)-(n1+1)-u1, W.sub.m1-(n1−1)-u1, W.sub.m1-(n1+1)-u1, W.sub.(m1−1)-(n1+1)-(u1+1)-W.sub.(m1+1)-(n1+1)-(u1+1) partially overlapping the specific stop W.sub.m1-n1-u1 to the value C.sub.m1-n1-u1 of the imaging parameter for the specific stop W.sub.m1-n1-u1 is determined to be smaller than or equal to the threshold value, the step S31-3 continues to define the three-dimensional computational map with a uniform space 110 therein, wherein the uniform space 110 has a profile defined by a profile of a combination of the specific stop W.sub.m-n-u and each of its neighboring stops W.sub.(m1−1)-(n1−1)-(u1−1)-W.sub.(m1+1)-(n1+1)-(u1−1), W.sub.(m1−1)-(n1−1)-u1-W.sub.(m1−1)-(n1+1)-u1, W.sub.(m1+1)-(n1−1)-u1-W.sub.(m1+1)-(n1+1)-u1, W.sub.m1-(n1−1)-u1, W.sub.m1-(n1+1)-u1, W.sub.(m1−1)-(n1+1)-(u1+1)-W.sub.(m1+1)-(n1+1)-(u1+1) partially overlapping the specific stop W.sub.m1-n1-u1, and to assign or set a constant value of the imaging parameter for a value d.sub.k-l-h of the imaging parameter for each of the computation voxels, e.g. P.sub.k5-l5-h5, in the uniform space 110, wherein the constant value of the imaging parameter is associated with the value C.sub.m1-n1-u1 of the imaging parameter for the specific stop W.sub.m1-n1-u1 and one of the values C.sub.(m1−1)-(n1−1)-(u1−1)-C.sub.(m+1)-(n1+1)-(u1−1), C.sub.(m1−1)-(n1−1)-u1-C.sub.(m1−1)-(n1+1)-u1, C.sub.(m1+1)-(n1−1)-u1-C.sub.(m1+1)-(n1+1)-u1, C.sub.m1-(n1−1)-u1, C.sub.m1-(n1+1)-u1, C.sub.(m1−1)-(n1+1)-(u1+1)-C.sub.(m1+1)-(n1+1)-(u1+1) of the imaging parameter for each of its neighboring stops W.sub.(m1−1)-(n1−1)-(u1−1)-W.sub.(m1+1)-(n1+1)-(u1−1), W.sub.(m1−1)-(n1−1)-u1-W.sub.(m1−1)-(n1+1)-u1, W.sub.(m1+1)-(n1−1)-u1-W.sub.(m1+1)-(n1+1)-u1, W.sub.m1-(n1−1)-u1, W.sub.m1-(n1+1)-u1, W.sub.(m1−1)-(n1+1)-(u1+1)-W.sub.(m1+1)-(n1+1)-(u1+1) partially overlapping the specific stop W.sub.m1-n1-u1, such as an average of the value C.sub.m1-n1-u1 of the imaging parameter for the specific stop W.sub.m1-n1-u1 and one of the values C.sub.(m1−1)-(n1−1)-(u1−1)-C.sub.(m+1)-(n1+1)-(u1−1), C.sub.(m1−1)-(n1−1)-u1-C.sub.(m1−1)-(n1+1)-u1, C.sub.(m1+1)-(n1−1)-u1-C.sub.(m1+1)-(n1+1)-u1, C.sub.m1-(n1−1)-u1, C.sub.m1-(n1+1)-u1, C.sub.(m1−1)-(n1+1)-(u1+1)-C.sub.(m1+1)-(n1+1)-(u1+1) of the imaging parameter for each of its neighboring stops W.sub.(m1−1)-(n1−1)-(u1−1)-W.sub.(m1+1)-(n1+1)-(u1−1), W.sub.(m1−1)-(n1−1)-u1-W.sub.(m1−1)-(n1+1)-u1, W.sub.(m1+1)-(n1−1)-u1-W.sub.(m1+1)-(n1+1)-u1, W.sub.m1-(n1−1)-u1, W.sub.m1-(n1+1)-u1, W.sub.(m1−1)-(n1+1)-(u1+1)-W.sub.(m1+1)-(n1+1)-(u1+1) partially overlapping the specific stop W.sub.m1-n1-u1.
(392) Next, referring to
(393) If none of the uniform space 110 is found in the three-dimensional computational map in the step S31-2, the step S31-5 for the deconvolution operation is performed. In the step S31-5, one of the values d.sub.k-l-h of the imaging parameter for each computation voxel P.sub.k-l-h of the three-dimensional computational map is iteratively updated or calculated, as illustrated in the steps DC1-DC10 in the third aspect, based on one or more of the values C.sub.m-n-u of the imaging parameter for respective one or more of the stops W.sub.m-n-u each covering said each computation voxel P.sub.k-l-h.
(394) VI-4. Computational Map Derived from Measured Values for Stops of Three-Dimensional Moving Window
(395) The process as illustrated in
(396) Next, referring to
(397) VI-5. Summary for Sections VI-3 and VI-4
(398) Following the section III-3 for Summary of Third Aspect, referring to
(399) Furthermore, for the step S32-4, the method includes said calculating the second value, e.g. d.sub.k-l-h, for the first computation voxel, e.g. P.sub.k-l-h in
(400) Furthermore, for the step S31-4, said calculating the second value, e.g. d.sub.k-l-h, for the first computation voxel, e.g. P.sub.k-l-h in
(401) VI-6. Probability Map Derived from Measured Values for Original Pixels of Two-Dimensional Original Map
(402)
(403) Next, referring to
(404) Next, referring to
(405) Next, referring to
(406) Referring to
(407) Next, referring to
(408) Referring to
(409) VI-7. Probability Map Derived from Measured Values for Stops of Two-Dimensional Moving Window
(410) Alternatively,
(411) Next, referring to
(412) VI-8. Probability Map Derived from Measured Values for Original Voxels of Three-Dimensional Original Map
(413) Referring to
(414) Next, referring to
(415) Next, referring to
(416) Next, referring to
(417) Referring to
(418) Next, referring to
(419) Referring to
(420) VI-9. Probability Map Derived from Measured Values for Stops of Three-Dimensional Moving Window
(421) The process as illustrated in
(422) Next, referring to
(423) VI-10. Summary for Sections VI-8 and VI-9
(424) Following the section IV-3 for Summary of Fourth Aspect, referring to
(425) Furthermore, for the step S32-4, the method includes said calculating the second probability, e.g. dl.sub.k-l-h, of the event for the first computation voxel, e.g. P.sub.k-l-h in
(426) Furthermore, for the step S32-4, said calculating the second probability, e.g. dl.sub.k-l-h, of the event for the first computation voxel, e.g. P.sub.k-l-h in
(427) Seventh Aspect: Multiple Measuring Pixels of Two-Dimensional Moving Window
(428) Alternatively,
(429) Referring to
(430)
(431) For example, referring to
(432) Accordingly, in a step S35-1, the two-dimensional moving window 2 may move rightwards step by step in the x direction with a shift equal to the x-direction width X.sub.fp to measure four or four sets of values C.sub.FX1-TY1, C.sub.FX2-TY2, C.sub.FX3-TY3 and C.sub.FX4-TY4 for four respective stops W.sub.FX1-TY1, W.sub.FX2-TY2, W.sub.FX3-TY3 and W.sub.FX4-TY4 of the measuring pixels 20 of a first stop of the two-dimensional moving window 2 for the first step, wherein FX1=FX3=tx*M.sub.mp*m.sub.cp+2, FX2=FX4=tx*M.sub.mp*m.sub.cp+m.sub.cp+2, TY1=TY2=ty*N.sub.mp*n.sub.cp+1 and TY3=TY4=ty*N.sub.mp*n.sub.cp+n.sub.cp+1, wherein tx is the number of jumps of the two-dimensional moving window 2 in the x direction, and ty is the number of jumps of the two-dimensional moving window 2 in the y direction, and to measure four or four sets of values C.sub.SX1-TY1, C.sub.SX2-TY2, C.sub.SX3-TY3 and C.sub.SX4-TY4 for four respective stops W.sub.SX1-TY1, W.sub.SX2-TY2, W.sub.SX3-TY3 and W.sub.SX4-TY4 of the measuring pixels 20 of the second stop of the two-dimensional moving window 2 for the second step, wherein SX1=SX3=tx*M.sub.mp*m.sub.cp+3 and SX2=SX4=tx*M.sub.mp*m.sub.cp+m.sub.cp+.sup.3. Once the two-dimensional moving window 2 moves rightwards step by step in the x direction with a shift equal to the x-direction width X.sub.fp by two steps, the two-dimensional moving window 2 may jump rightwards in the x direction with a shift equal to the x-direction width X.sub.fp times 4 to measure four or four sets of values C.sub.TX1-TY1, C.sub.TX2-TY2, C.sub.TX3-TY3 and C.sub.TX4-TY4 for four respective stops W.sub.TX1-TY1, W.sub.TX2-TY2, W.sub.TX3-TY3 and W.sub.TX4-TY4 of the measuring pixels 20 of a stop of the two-dimensional moving window 2, wherein TX1=TX3=tx*M.sub.mp*m.sub.cp+1 and TX2=TX4=tx*M.sub.mp*m.sub.cp+m.sub.cp+1. The above step S35-1 may repeat until a stop of the two-dimensional moving window 2 has a stop of one of the measuring pixels 20 reaching to a right side of the two-dimensional computational map 12 as seen in
(433) After a stop of the two-dimensional moving window 2 has a stop of one of the measuring pixels 20 reaching to a right side of the two-dimensional computational map 12 in the step S35-1 as seen in
(434) After a stop of the two-dimensional moving window 2 has a stop of one of the measuring pixels 20 reaching to a right side of the two-dimensional computational map 12 in the step S35-3, a step S35-4 continues wherein the two-dimensional moving window 2 may move downwards in a y-direction with a shift equal to the y-direction width Y.sub.fp of the computation pixels P.sub.k-l of the two-dimensional computational map 12 and to the left side of the two-dimensional computational map 2, as seen in
(435) Once the two-dimensional moving window 2 moves downwards row by row in the y direction with a shift equal to the y-direction width Y.sub.fp by three rows, a step S35-6 continues wherein the two-dimensional moving window 2 may jump downwards in the y direction with a shift equal to the y-direction width Y.sub.fp times 3, as seen in
(436) The steps, features, benefits and advantages that have been discussed are merely illustrative. None of them, nor the discussions relating to them, are intended to limit the scope of protection in any way. Numerous other embodiments are also contemplated. These include embodiments that have fewer, additional, and/or different steps, features, benefits and advantages. These also include embodiments in which the steps are arranged and/or ordered differently.