Method and system for determining a phenotype of a neoplasm in a human or animal body

Abstract

The present invention relates to a decision support system and an image analysis method for providing information for enabling determination of a phenotype of a neoplasm in a human or animal body for enabling prognostication, comprising the steps of: receiving, by a processing unit, image data of the neoplasm; and deriving, by the processing unit, a plurality of image feature parameter values from the image data, said image parameter values relating to image features associated with the neoplasm; and deriving, by said processing unit using a signature model, one or more neoplasm signature model values associated with the neoplasm from said image feature parameter values, wherein said signature model includes a functional relation between or characteristic values of said image feature parameter values for deriving said neoplasm signature model values.

Claims

1. An image analysis method for providing information for enabling determination of a phenotype of a neoplasm in a human or animal body for enabling prognostication, comprising the steps of: receiving, by a processing unit, image data of the neoplasm; and deriving, by the processing unit, a plurality of image feature parameter values from the image data, said image parameter values relating to image features associated with the neoplasm; and deriving, by said processing unit using a signature model, one or more neoplasm signature model values associated with the neoplasm from said image feature parameter values, wherein said signature model includes a functional relation between or characteristic values of said image feature parameter values for deriving said neoplasm signature model values therefrom; wherein the image feature parameter values are indicative of image feature parameters, wherein the signature model includes at least all of the image feature parameters from a group comprising: gray-level non-uniformity, and wavelet high-low-high gray-level run-length gray-level non-uniformity.

2. The image analysis method according to claim 1, wherein the signature model further includes at least all of the image feature parameters from a group comprising: statistics energy, and shape compactness.

3. The image analysis method according to claim 1, wherein the method further comprises the steps of: obtaining, by said processing unit from a memory, said signature model comprising one or more signature selector values associated with the image features, wherein the signature selector values indicate whether the associated image features are comprised by the signature model; and multiplying for the at least one signature model, by the processing unit, the image feature parameter values with the associated signature selector values for obtaining said one or more neoplasm signature model values associated with the neoplasm.

4. The image analysis method according to claim 1, further comprising a step of comparing, by the processing unit, the neoplasm signature model values to at least one signature model reference value for the at least one signature model, for associating the neoplasm with the phenotype.

5. The image analysis method according to claim 1, wherein the signature selector values further comprise one or more weighting values associated with the image features comprised by the signature model, for weighing the image features in the signature model.

6. The image analysis method according to claim 1, further comprising a step of calculating, by the processing unit, a neoplasm signature model score as a function of the one or more neoplasm signature model values, wherein the step of comparing includes comparing the neoplasm signature model score with a reference score.

7. The image analysis method according to claim 1, wherein the at least one signature model comprises a plurality of distinct signature models.

8. The image analysis method according to claim 1, wherein the image feature parameters further include at least one element of a group comprising: first-order gray level statistics obtained from image pixels or areas of the image from the image data; second-order gray level statistics obtained from co-occurrence matrices of the image data; run-length gray level statistics, short run emphasis, long run emphasis, run percentage; and shape and size based features.

9. The image analysis method according to claim 2, wherein in accordance with the signature model, the image feature parameter statistics energy has an associated absolute weighting value within a range of 1.0e−20 through 1.0e−5; the image feature parameter shape compactness has an associated absolute weighting value within a range of 1.0e−7through 1.0e−1; the image feature parameter gray-Level non-uniformity has an associated absolute weighting value within a range of 1.0e−9 through 1.0e−1; and the image feature parameter wavelet high-low-high gray-level run-length gray-level non-uniformity has an associated absolute weighting value within a range of 1.0e−9 through 1.0e−1.

10. The image analysis method according to claim 1, wherein the image data is received using an imaging method selected from a group comprising magnetic resonance imaging, computed tomography, positron emission tomography, single-photon emission computed tomography, ultrasonography, thermography, and photo-acoustic imaging.

11. The image analysis method in accordance with claim 1, any of the previous claims, wherein the step of receiving image data comprises the steps of receiving first image data of the neoplasm at a first moment in time and receiving second image data of the neoplasm at a second moment in time, and wherein the steps of deriving the image feature parameter values, obtaining the signature model, and multiplying the image feature parameter values with the associated signature selector values is performed for said first and second image data, further comprising a step of determining a difference between the neoplasm signature model values of the first and second image data.

12. A decision support system arranged for performing an image analysis method for providing information for enabling determination of a phenotype of a neoplasm in a human or animal body for enabling prognostication, said system comprising an input connected to a processing unit for receiving by the processing unit image data of the neoplasm; wherein the processing unit is further arranged for deriving a plurality of image feature parameter values from the received image data, said image parameter values relating to image features associated with the neoplasm, wherein the processing unit is connected to a memory for obtaining therefrom at least one signature model comprising one or more signature selector values associated with the image features, wherein the signature selector values indicate whether the associated image features are comprised by the signature model, and wherein the processing unit is arranged for multiplying for the at least one signature model the image feature parameter values with the associated signature selector values for obtaining one or more neoplasm signature model values associated with the neoplasm; wherein the image feature parameter values are indicative of image feature parameters, wherein the signature model includes at least all of the image feature parameters from a group comprising: statistics energy, shape compactness, gray-level non-uniformity, wavelet high-low-high gray-level run-length gray-level non-uniformity.

13. The decision support system according to claim 12, wherein the processing unit is further arranged for comparing the neoplasm signature model values to at least one signature model reference value for the at least one signature model, for associating the neoplasm with the phenotype, the system further comprising an output for providing an indicator value indicative of a result of said step of associating.

14. The decision support system according to claim 12, wherein the signature selector values further comprise one or more weighting values associated with the image features comprised by the signature model, for weighing the image features in the signature model, and wherein in accordance with the signature model, the image feature parameter statistics energy has an associated absolute weighting value within a range of 1.0e−20through 1.0e−5; the image feature parameter shape compactness has an associated absolute weighting value within a range of 1.0e−7 through 1.0e−1; the image feature parameter gray-level non-uniformity has an associated absolute weighting value within a range of 1.0e−9 through 1.0e−1; and the image feature parameter wavelet high-low-high gray-level run-length gray-level non-uniformity has an associated absolute weighting value within a range of 1.0e−9 through 1.0e−1.

15. A non-transistory computer-readable medium comprising computer-executable instructions which, when run on a computer, are arranged for performing an image analysis method for providing information for enabling determination of a phenotype of a neoplasm in a human or animal body for enabling prognostication, the method comprising the steps of: receiving, by a processing unit, the image data of the neoplasm; and deriving, by the processing unit, a plurality of image feature parameter values from the image data, said image parameter values relating to image features associated with the neoplasm; and deriving, by said processing unit using a signature model, one or more neoplasm signature model values associated with the neoplasm from said image feature parameter values, wherein said signature model includes a functional relation between or characteristic values of said image feature parameter values deriving said neoplasm signature model values therefrom, wherein the image feature parameter values are indicative of image feature parameters, and wherein the signature model includes at least all of the image feature parameters from a group comprising: gray-level non-uniformity, and wavelet high-low-high gray-level run-length gray-level non-uniformity.

16. The image analysis method according to claim 8, wherein the first order gray level statistics are selected from minimum intensity, maximum intensity, mean intensity, intensity range, intensity variance, intensity standard deviation, skewness, kurtosity and entropy.

17. The image analysis method according to claim 8, wherein the second-order gray level statistics are selected from contrast, correlation between neighboring image axels or pixels, energy, homogeneity, inverse difference moment, sum average, sum variance, and sum entropy.

18. The image analysis method according to claim 8, wherein the shape and size based features are selected from perimeter, cross-sectional area, major axis length, maximum diameter, and volume.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The present invention will further be elucidated by means of some specific embodiments thereof, with reference to the enclosed drawings, wherein:

(2) FIG. 1A-1 is a grey level image describing tumor intensity of a first tumor;

(3) FIG. 1A-2 is a histogram of the image of FIG. 1A-1;

(4) FIG. 1A-3 provides an overview of image feature parameter values and corresponding image feature parameters derived from first order grey level statistics of the grey level image of FIG. 1A-1;

(5) FIG. 1B-1 provides a grey level image describing tumor intensity of a second tumor;

(6) FIG. 1B-2 is a histogram of the grey level image of FIG. 1B-1;

(7) FIG. 1B-3 provides an overview of image feature parameter values and associated image feature parameters derived from first order grey level statistics obtained by analyzing the image of FIG. 1B-1;

(8) FIG. 2A-1 illustrates a three dimensional representation of a third tumor;

(9) FIG. 2A-2 provides an overview of image feature parameter values of image feature parameters obtained from shape and/or size analysis of the tumor based on FIG. 2A-1;

(10) FIG. 2B-1 provides a three dimensional representation of a fourth tumor;

(11) FIG. 2B-2 provides an overview of image feature parameter values and associated image feature parameters obtained by shape and/or size analysis based on the image illustrated in FIG. 2B-1;

(12) FIG. 3 is an illustration of a surface contour analysis for obtaining the maximum diameter of a tumor;

(13) FIG. 4A provides an image of a fifth tumor;

(14) FIG. 4B provides an image of a sixth tumor;

(15) FIG. 5 is a schematic illustration of a decision support system in accordance with an embodiment of the present invention;

(16) FIG. 6 is a schematic illustration of an embodiment of an image analysis method in accordance with an embodiment of the present invention;

(17) FIG. 7 is a gray scale ROI image of a tumor from which a gray-level co-occurrence matrix may be determined in accordance with an embodiment of the invention.

DETAILED DESCRIPTION

(18) Before providing a more detailed description of the various image feature parameters which may be derived from image features obtained from imaging data of neoplasms such as tumors, a description will be given herein below with reference to FIGS. 5 and 6 of a decision support system and an image analysis method in accordance with the present invention.

(19) FIG. 5 schematically illustrates a decision support system in accordance with an embodiment of the present invention. In FIG. 5, the decision support system 1 comprises at least an analysis unit 3 which is connected to an imaging system 8. The imaging system 8 may be any suitable imaging system used in medical environments for diagnostic purposes, in particular for visualizing tumors. The imaging system 8 may for example be a magnetic resonance imaging system (MRI), a computer tomography system (CT), a positron emission tomography system (PET), a single photon emission computer tomography system (SPECT), an ultrasonography system, a tomography system, or a photo acoustic imaging system. The imaging system 8 may provide image data directly to the analysis system 3, or may alternatively store the image data in a data repository data system 10 from which it may be obtained by the analysis system 3 at any time required. As will be appreciated, the analysis system 3, the imaging system 8, the data repository system 10, and any output terminal or system 12, may be connected with each other via a data network, or via direct data connections.

(20) As mentioned hereinabove, the analysis system 3 receives imaging data either directly from the imaging system 8 or retrieves it from a data repository system 10 where the image data may be stored. Another possibility is that part of the image data is received directly from the imaging system 8 by analysis unit 3, and another part of imaging data, e.g. imaging data taken from a same tumor at an earlier stage during a treatment of a patient, may be obtained from data repository system 10. As will be appreciated, imaging data may alternatively be obtained from another source or via other means. For example, such data may be obtained from a remote network, from an e-mail server, or from a data storage entity such as a memory stick or an SD card. Performing an analysis in accordance with the present invention on imaging data taken at various stages throughout a treatment process provides information to a medical practitioner that may be used for evaluating the treatment process, and to take necessary action.

(21) The analysis unit 3 comprises a processing unit which receives the image data from input/output ports 6 and 7. The processing unit is arranged for deriving a plurality of image feature parameter values associated with image feature parameters from the image data received. Through this end, the processing unit 4 applies various analysis algorithms, such as statistical analysis algorithms, graphic analysis algorithms and the like. Such algorithms may for example be stored in memory unit 5 within analysis unit 3. The processing unit may further be arranged to obtain one or more signature models from memory unit 5. Each of the obtained signature models comprises signature selector values which determine whether or not specific image feature parameters are included in the respective signature model. Instead of only comprising signature selector values, the signature models may also comprise weighting factors also stored in memory unit 5. Such weighting factors not only determine that a certain image feature parameter is included in the signature model, but also enable to prescribe the importance of a certain image feature parameter in the signature model, e.g. in terms of its predictive value in relation to or in combination with other parameters.

(22) The processing unit 4 is arranged for multiplying each of the image feature parameter values obtained from the imaging data during the step of deriving described herein above, with their associated signature selector values or weighting factors (where applicable) for each of the signature models. This step of multiplication yields the neoplasm signature model values representing the tumor in terms of the respective signature models. These neoplasm signature model values will be used to associate the tumor with a certain phenotype in order to enable prognostication, predict survival expectance, suggest a possible treatment, and other important decision support information to be provided to the user or medical practitioner. For performing the classification of the tumor into a certain phenotype, the neoplasm signature model values are compared to signature model reference values that may for example be stored in the memory unit 5. Such comparison may take any suitable form, and may also include, as will be described for example in relation to FIG. 6, the calculation of a neoplasm signature model score as a function of the neoplasm signature model values calculated herewith. Such a neoplasm signature model score may be compared to a reference score which is also stored in the memory unit 5. The output of the analysis method is provided to an output terminal for example terminal 12. This may be any suitable computer system, display screen, a further analysis unit, a printing system, or a communication system allowing to distribute the relevant information to the user or users of the decision support system.

(23) In FIG. 6, an analysis method in accordance with the present invention is schematically illustrated. To explain the method in relation to a decision support system of the invention, reference is also made to the reference numerals and features of FIG. 5. As will be appreciated, the method and the system are only provided as an example and should not be interpreted limiting. In step 20, image data is received from an imaging system 8 by a processing unit 4. The processing unit 4 in step 22 derives from the image data received, a plurality of image feature parameter values 30, 32, 34, 36, 38, 40, 42, 44, 46, 48 and 50. As will be appreciated the image feature parameter values that should at least be determined in step 22 are dependent on the signature models to be applied. Further on in this document, a detailed description of all the image feature parameters that may be used and may be derived in step 22 will be provided. In FIG. 6, a total of eleven image feature parameter values is illustrated, but the skilled person will appreciate that any other number of image feature parameter values may be derived in this step 22.

(24) The image feature parameter values 30-50 are multiplied by signature selector values 54-74. A signature selector value may for example include a boolean selector (which may have the value 0 or 1 dependent on whether the associated image feature parameter value is to be included in the signature model) and a weighting factor (e.g. a real value between 0 and 1). For example, factor 54 may be a multiplication of signature selector value equal to ‘1’ and the weighting factor equal to 0.83, although these values are just examples. Each of the factors 54, 56, 58, 60, 62, 64, 66, 68, 70, 72 and 74 is set by the processing unit based on a signature model 24 (for example any of the signature models 24a, 24b, or 24c) stored in a memory. In FIG. 6, signature model 24a is applied to the factors 54-74 as indicated by schematic line 27.

(25) The image feature parameter values are multiplied by their associated signature selector values. Image feature parameter value 30 is multiplied by signature selector values 54, image feature parameter value 32 is multiplied by signature selector values 56, image feature parameter value 34 is multiplied by signature selector values 58, image feature parameter value 36 is multiplied by signature selector values 60, image feature parameter value 38 is multiplied by signature selector values 62, image feature parameter value 40 is multiplied by signature selector values 64, image feature parameter value 42 is multiplied by signature selector values 66, image feature parameter value 44 is multiplied by signature selector values 68, image feature parameter value 46 is multiplied by signature selector values 70, image feature parameter value 48 is multiplied by signature selector values 72 and image feature parameter value 50 is multiplied by signature selector values 74. The products of the image feature parameter values and signature selector values are then provided as input to a summing step 78 for calculating a neoplasm signature model score, e.g. by summing all the values obtained such as to calculate a linear combination of the image feature parameter values 30-50 with their associated signature selector values (including weighting factors) 54-74. This score obtained in step 78 may be compared with a reference value from memory 82, and provided to the user of the analysis method in step 80. In case a comparison is made between image data from tumors at various stages during a treatment process, further image data may be obtained from a memory or repository system in step 20 and the analysis method is repeated. Eventually, the results of performing the image analysis method for each of the image data obtained will be compared and presented to the user (not shown).

(26) As will be appreciated, the decision support system of FIG. 5 and the image analysis method of FIG. 6 are embodiments of the present invention, however the invention may be practice otherwise then specifically described with reference to FIGS. 5 and 6.

(27) The present invention uses image feature parameter values obtained from image features derived from image data of a tumor. FIGS. 1A-1 through 1B-3 provide as a first example a number of image feature parameters and their values that may be obtained from first order grey level statistical analysis of an image. In FIG. 1A-1, a grey level image of a tumor is illustrated. The grey level scale is indicated with reference numeral 103 to the right of FIG. 1A-1. Also visible in FIG. 1A-1 is the contour 101 of the tumor to be analyzed. It is to be noted that the contour defining the tumor will usually be determined by a medical practitioner, or any other analysis method or system. The present description assumes this information to be available to the method.

(28) In FIG. 1A-2 a histogram 105 is illustrated which is based on the image data illustrated in FIG. 1A-1. The histogram 105 resembles the tumor image only, i.e. the histogram is based on the pixels of the grey level image FIG. 1A-1 inside the contour 101. All parts of the image outside contour 101 are disregarded from the analysis and is considered to be healthy tissue. The histogram 105 is plotted onto a first access 107 indicating the grey level considered, and a second access 108 resembling the number of pixels occurring with grey level.

(29) FIG. 1B-1 illustrates a second tumor within contour 121, and FIG. 1B-2 illustrates a corresponding histogram 123 associated with this second tumor illustrated in FIG. 1B-1. From a qualitative comparison of the images of FIG. 1A-1 and FIG. 1B-1, one can see a number of characteristic differences between the two tumors. For example, the first tumor within contour 101 appears to be inhomogeneous, while the grey level of the second tumor 121 is more uniform. This difference is for example directly visible in the histograms 105 and 123. Histogram 123 is clearly concentrated around a uniform grey level as a small but sharp peak. Histogram 105 illustrates a broad distribution having a peak at approximately grey level 1050 and a more distributed trail across almost all grey levels below this value. From the histogram of the image of the tumor, relevant information can be quantitatively derived that may also be derived from qualitative examination of the images.

(30) In FIGS. 1A-3 and FIG. 1B-3, an overview is provided from a number of image feature parameter values and associated image feature parameters that may be derived from first order grey level statistical analysis of the images of FIGS. 1A-1 and 1B-1 respectively. These image feature parameters, which will be described with more detail later on in this document, may be used in the various signature models to obtain information that may help the medical practitioner in selecting the correct treatment, determining survival expectancy, and prognostication in general.

(31) FIGS. 2A-1 through 2B-2 provide an example of image feature parameter and image feature parameter values that may be obtained from analysis of shape and size related features, derivable for example from three dimensional (3D) representations of tumors based on imaging data obtained. In FIG. 2A-1 a three dimensional (3D) representation of a third tumor 130 is illustrated. In FIG. 2B-1 a three dimensional (3D) representation of a fourth tumor 135 is illustrated. From qualitative comparison of the two tumors in FIGS. 2A-1 and FIGS. 2B-1, a number of differences may be derived such as a difference in size of the tumor. The fourth tumor 135 is much larger than the third tumor 130, although the third tumor 130 appears to have a much larger surface.

(32) An overview of the image feature parameter values that may be derived from the imaging data in FIGS. 2A-1 and FIG. 2B-1 is provided in FIGS. 2A-2 and 2B-2 respectively. These image feature parameter values for example include the volumes of the tumors, their total surface and their maximum diameter. Besides this, more quantitative information on image feature parameters which may be characteristic for a specific type of tumor growth (phenotype) is derivable from the images. For example, the sphericity provides information on how spherical (i.e. regular) the tumor is. The surface to volume ratio (SVR) expresses how spiky or sharp the tumor is. A maximum diameter represents the maximum distance between the most remote points on the surface of the tumor in the three dimensional representation.

(33) FIG. 3 provides an illustration of a contour analysis from which the maximum diameter of a tumor may be derived. The most remote points in FIG. 3 are at the ultimate ends of the tumor 140, to the left and right side of the plot in FIG. 3. In respect of FIG. 3 it is noted that the points depicted in the plot are voxels lying on the surface of the tumor.

(34) As a further example in FIGS. 4a and 4b, a fifth tumor 143 and a sixth tumor 146 are respectively illustrated. From qualitative observation of the images in FIG. 4a and FIG. 4b, a striking difference is visible in terms of the texture of the tumors illustrated. For example, the sixth tumor 146 in FIG. 4b illustrates a strong variation in color inside the tumor and across its surface. The tumor 143 in FIG. 4a is more homogeneous, being more or less of one color. These differences in texture can be derived from co-occurrence matrices obtained from pixel color analysis of the images of these figures. The concept of co-occurrence matrices will be explained later.

Image Feature Parameter Descriptions

(35) First-order Gray Level Statistics

(36) In this section various image feature parameters are described that can be used to extract and summarize meaningful and reliable information from CT images. We will describe the extraction of image traits that may be used to derive prognostic metrics, and that may be incorporated into signature models of a decision support system, to beneficially support the clinical planning process to modify the patient treatment based on their predicted risk of failure. As appreciated, the objective of the invention is to support (not take over) the decision making process of the medical practitioner with advanced information taken from the images; i.e. image feature data that cannot be objectively assessed by means of qualitative interpretation.

(37) We explore first-order statistics of the image histogram through the commonly used metrics. We denote by I(x,y) as the intensity or gray-level values of the two-dimensional pixel matrix. The formulas used for the first order statistics are as follows:

(38) 1. Minimum
I.sub.min=min{I(x,y)}  (B.1)

(39) 2. Maximum
I.sub.max=max{I(x,y)}  (B.2)

(40) 3. Range
R=max{I(x,y)}−min{I(x,y)}  (B.3)

(41) 4. Mean

(42) $\begin{array}{cc}\mu =\frac{1}{\mathrm{XY}}\underset{x=1}{\overset{X}{.Math.}}\underset{y=1}{\overset{Y}{.Math.}}I\left(x,y\right)& \left(B\mathrm{.4}\right)\end{array}$

(43) 5. Variance

(44) $\begin{array}{cc}{\sigma }^2=\frac{1}{\left(\mathrm{XY}-1\right)}\underset{x=1}{\overset{X}{.Math.}}\underset{y=1}{\overset{Y}{.Math.}}{\left[I\left(x,y\right)-\mu \right]}^2& \left(B\mathrm{.5}\right)\end{array}$

(45) 6. Standard Deviation

(46) $\begin{array}{cc}S={\left(\frac{1}{\mathrm{XY}-1}\underset{i=1}{\overset{\mathrm{XY}}{.Math.}}{\left(x_i-\mu \right)}^2\right)}^{1/2}& \left(B\mathrm{.6}\right)\end{array}$

(47) 7. Skewness

(48) $\begin{array}{cc}\frac{1}{\mathrm{XY}}\underset{x=1}{\overset{X}{.Math.}}\underset{y=1}{\overset{Y}{.Math.}}{\left[\frac{I\left(x,y\right)-\mu }{\sigma }\right]}^3& \left(B\mathrm{.7}\right)\end{array}$

(49) 8. Kurtosis

(50) $\begin{array}{cc}\frac{1}{\mathrm{XY}}\underset{x=1}{\overset{X}{.Math.}}\underset{y=1}{\overset{Y}{.Math.}}\left\{{\left[\frac{I\left(x,y\right)-\mu }{\sigma }\right]}^4\right\}-3& \left(B\mathrm{.8}\right)\end{array}$

(51) 9. Entropy
H=−Σ.sub.i=1.sup.XYP(i).Math.log.sub.2P(i)  (B.9)

(52) In B.9 P(i) is the first order histogram, that is, P(i) is the fraction of pixels with gray level i. The variance (μ.sub.2), skewness (μ.sub.3) and kurtosis (μ.sub.4) are the most frequently used central moments. The variance is a measure of the histogram width, that is, a measure of how much the gray levels differ from the mean. The skewness measures the degree of histogram asymmetry around the mean, and kurtosis is a measure of the histogram sharpness. As a measure of histogram uniformity or randomness we computed the entropy of the image histogram. The closer to a uniform distribution the higher the entropy, or seen in a different way, H would take low values in smooth images where the pixels have the same intensity level.

(53) Second-order Gray Levels Statistics

(54) The features shown above that resulted from the first-order statistics provide information related to the gray-level distribution of the image; however they do not provide any information regarding the relative position of the various gray levels over the image. This information can be extracted from the so called co-occurrence matrices where pixels are considered in pairs and which provide a spatial distribution of the gray level values. The co-occurrence features are based on the second-order joint conditional probability function P(i,j;a,d) of a given image. The ith, jth element of the co-occurrence matrix for a given tumor image represents the number of times that the intensity levels i and j occur in two pixels separated by a distance (d) in the direction (a). The co-occurrence matrix for a pair (d,a) is defined as the N.sub.g×N.sub.g matrix where N.sub.g is the number of intensity levels. The N.sub.g levels were obtained by scaling the gray-level image to a discrete N.sub.g number of gray-level values. The N.sub.g values are normally selected in powers of 2; here we have selected 32 discrete gray-level values which in practice is a sufficient choice for representing the image. Here d was set to a single pixel size and a covered the four available angular directions (horizontal, vertical, diagonal and anti-diagonal). Let for example an image array I(x,y) be:

(55) $\begin{array}{cc}I=\left[\begin{array}{ccccc}3& 5& 8& 10& 8\\ 7& 10& 3& 5& 3\\ 7& 3& 5& 1& 8\\ 2& 6& 7& 1& 2\\ 1& 2& 9& 3& 9\end{array}\right]& \left(B\mathrm{.11}\right)\end{array}$
which corresponds to a 5×5 image. We can assume the number of discrete gray levels is equal to 10. Thus for the image (B.11) and a relative pixel position (1.0°) we obtain:

(56) $\begin{array}{cc}{\mathrm{GLCM}}^0\left(d=1\right)=\left[\begin{array}{cccccccccc}0& 2& 0& 0& 0& 0& 0& 1& 0& 0\\ 0& 0& 0& 0& 0& 1& 0& 0& 1& 0\\ 0& 0& 0& 0& 3& 0& 0& 0& 1& 0\\ 0& 0& 0& 0& 0& 0& 0& 0& 0& 0\\ 1& 0& 1& 0& 0& 0& 0& 1& 0& 0\\ 0& 0& 0& 0& 0& 0& 1& 0& 0& 0\\ 1& 0& 1& 0& 0& 0& 0& 0& 0& 1\\ 0& 0& 0& 0& 0& 0& 0& 0& 0& 1\\ 0& 0& 1& 0& 0& 0& 0& 0& 0& 0\\ 0& 0& 1& 0& 0& 0& 0& 1& 0& 0\end{array}\right]& \left(B\mathrm{.12}\right)\end{array}$

(57) In other words, for each of the intensity pairs, such as (1, 2), we count the number of pixel pairs at relative distance (d=1) and orientation a=0° (horizontal) that take these values. In our case this is 2. There are two instances in the image (B.11) where two, horizontally adjacent pixels have the values 1 and 2. The element (3, 5) in the GLCM is 3 because in the example image there are 3 instances in which two, horizontally adjacent pixels have the values 3 and 5. From the same image (B.11) and (d=1, a=45°) we obtain:

(58) $\begin{array}{cc}{\mathrm{GLCM}}^{45}\left(d=1\right)=\left[\begin{array}{cccccccccc}0& 0& 1& 0& 0& 1& 0& 1& 0& 0\\ 0& 0& 1& 0& 0& 0& 1& 0& 0& 0\\ 0& 1& 1& 0& 0& 0& 0& 0& 0& 1\\ 0& 0& 0& 0& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 1& 0& 0& 1& 0& 0\\ 0& 0& 0& 0& 1& 0& 0& 0& 0& 0\\ 1& 0& 0& 0& 1& 0& 0& 0& 0& 1\\ 0& 0& 0& 0& 0& 0& 0& 0& 0& 0\\ 1& 0& 0& 0& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0& 1& 0& 0\end{array}\right]& \left(B\mathrm{.13}\right)\end{array}$

(59) As illustrative example we will obtain the gray-level co-occurrence matrix from a given tumor image. FIG. 7 provides an example of a given gray scale ROI image (color map changed for better visual inspection) to the left, and a scaled version of the left image to 32 discrete gray levels to the right. In FIG. 7, the image in the left hand side corresponds to a given gray-level ROI image, the color map has been changed to enhance the differences for visual inspection. The image in the right corresponds to the scaled ROI with 32 discrete gray values. The co-occurrence matrices are obtained from the scaled image.

(60) Having defined the probabilities of occurrence of gray levels with respect to relative spatial position we can define the relevant co-occurrence features that have been extracted; in some cases they have a direct physical interpretation with respect to the texture of an image, for example, they quantify coarseness, smoothness, randomness, etc. Others do not have such a property but they still encode highly discriminative texture-related information. Denoting by P(i,j) the normalized co-occurrence matrix, by N.sub.g the number of discrete gray levels of the image, the co-occurrence features relevant for our application are defined as follows:

(61) 10. Contrast

(62) $\begin{array}{cc}\mathrm{Con}=\underset{n=1}{\overset{N_g}{.Math.}}{n}^2\left\{\begin{array}{c}\underset{i=1}{\overset{N_g}{.Math.}}\underset{j=1}{\overset{N_g}{.Math.}}P\left(i,j\right)\\ .Math.i-j.Math.=n\end{array}\right\}& \left(B\mathrm{.14}\right)\end{array}$

(63) This is a measure intensity contrast between a pixel and its neighbor over the entire image, that is, a measure of the local gray level variations. For a constant image this metric is zero. The n.sup.2 dependence weights the big differences more.

(64) 11. Correlation

(65) 0$\begin{array}{cc}\mathrm{Correlation}=\underset{i=1}{\overset{N_g}{.Math.}}\underset{j=1}{\overset{N_g}{.Math.}}\frac{\left(i-{\mu }_i\right)\left(j-{\mu }_j\right)P\left(i,j\right)}{{\sigma }_i{\sigma }_j}& \left(B\mathrm{.15}\right)\end{array}$

(66) This metric measure how correlated is a pixel to its neighbor over the entire image. Correlation takes the values 1 or −1 for perfectly positively or negatively correlated image.

(67) 12. Energy
Energy=Σ.sub.i=1.sup.N.sup.gΣ.sub.j=1.sup.N.sup.g(P(i,j)).sup.2  (B.16)

(68) Energy is the sum of the squared elements of an image and a measure of smoothness. If all pixels are of the same gray level then energy is equal to 1; at the other extreme if we have all possible pairs of gray levels with equal probability, the region is less smooth, with a more uniformly distributed P(i,j) and a lower energy.

(69) 13. Homogeneity

(70) $\begin{array}{cc}\mathrm{Homogeneity}=\underset{i=1}{\overset{N_g}{.Math.}}\underset{j=1}{\overset{N_g}{.Math.}}\frac{P\left(i,j\right)}{1+.Math.i-j.Math.}& \left(B\mathrm{.17}\right)\end{array}$

(71) This feature measures how close is the distribution of elements in the co-occurrence matrix to the diagonal of the co-occurrence matrix. Homogeneity is 1 for a constant image.

(72) 14. Inverse Difference Moment

(73) $\begin{array}{cc}\mathrm{IDM}=\underset{i=1}{\overset{N_g}{.Math.}}\underset{j=1}{\overset{N_g}{.Math.}}\frac{P\left(i,j\right)}{1+{.Math.i-j.Math.}^2}& \left(B\mathrm{.18}\right)\end{array}$

(74) This feature takes high values for images with low contrast due to the (i−j).sup.2 dependence.

(75) 15. Sum Average
SA=Σ.sub.i=2.sup.2N.sup.g[iP.sub.x+y(i)]  (B.19)

(76) In A.18 P.sub.x(i) and P.sub.y(i) are the row and column marginal probabilities, obtained by summing the rows or columns P(i,j).

(77) 16. Sum Variance
SV=Σ.sub.i=2.sup.2N.sup.g[(i−sum average).sup.2P.sub.x+y(i)]  (B.20)

(78) 17. Sum Entropy
SE=−Σ.sub.i=2.sup.2N.sup.g[P.sub.x+y(i)log[P.sub.x+y(i)]]  (B.21)

(79) All the second-order statistics based features are functions of the distance d and the orientation a. Here for the direction d=1, the resulting values for the four directions are averaged. These metrics take into account the local intensity and spatial relationship of pixels over the region and are independent to tumor position, size, orientation and brightness.

(80) Run-length Gray-level Statistics

(81) Additionally we examined gray-level runs derived from run-length matrices (RLM) using a run-length metrics. A gray level run is a set of consecutive pixels having the same gray level value. The length of the run is the number of pixels in the run. Run length features describe textural information related with the number of times each gray level appears by itself, in pairs and so on, in a certain distance and orientation. Taking for example the image

(82) $\begin{array}{cc}I=\left[\begin{array}{ccccc}5& 2& 5& 4& 4\\ 3& 3& 3& 1& 3\\ 2& 1& 1& 1& 3\\ 4& 2& 2& 2& 3\\ 3& 5& 3& 3& 2\end{array}\right]& \left(B\mathrm{.22}\right)\end{array}$
with five possible gray levels. For each of the previously defined angular directions (0°, 45°, 90° and 135°) the corresponding run length matrices are defined. The run length matrix is an N.sub.g×N.sub.r array where N.sub.r is the largest possible run length in the image. For distance (d=1) and orientation (a=0°) we obtain:

(83) $\begin{array}{cc}{Q}_{\mathrm{RL}}\left(0°\right)=\left[\begin{array}{ccccc}1& 0& 1& 0& 0\\ 3& 0& 1& 0& 0\\ 4& 1& 1& 0& 0\\ 1& 1& 0& 0& 0\\ 3& 0& 0& 0& 0\end{array}\right]& \left(B\mathrm{.23}\right)\end{array}$

(84) The element (1,1) of the run length matrix is the number of times that the gray level 1 appears by itself, the second element is the number of times it appears in pairs (zero in the example), and so on. The element (3,3) is the number of times the gray level 3 appears in the image with run length 3. For the diagonal direction we obtain:

(85) $\begin{array}{cc}{Q}_{\mathrm{RL}}\left(45°\right)=\left[\begin{array}{ccccc}2& 1& 0& 0& 0\\ 6& 0& 0& 0& 0\\ 7& 1& 0& 0& 0\\ 3& 0& 0& 0& 0\\ 3& 0& 0& 0& 0\end{array}\right]& \left(B\mathrm{.24}\right)\end{array}$

(86) Denoting by P the total number of pixels of an image, by Q.sub.RL(i,j) the (i,j)-th element of the run length matrix for a specific distance d and a specific angle a and by Nr the number of different runs that occur, based on the definition of the run length matrices, the following rung length features are defined:

(87) 18. Short Run Emphasis

(88) $\begin{array}{cc}\mathrm{SRE}=\frac{\underset{i=1}{\overset{N_g}{.Math.}}\underset{j=1}{\overset{N_r}{.Math.}}\left(Q_{\mathrm{RL}}\left(i,j\right)/j^2\right)}{\underset{i=1}{\overset{N_g}{.Math.}}\underset{j=1}{\overset{N_r}{.Math.}}{Q}_{\mathrm{RL}}\left(i,j\right)}& \left(B\mathrm{.25}\right)\end{array}$

(89) This feature emphasizes small run lengths. The denominator is the number of run lengths in the matrix, for example, 17 in B.23 and 23 in B.24.

(90) 19. Long Run Emphasis

(91) $\begin{array}{cc}\mathrm{LRE}=\frac{\underset{i=1}{\overset{N_g}{.Math.}}\underset{j=1}{\overset{N_r}{.Math.}}\left(Q_{\mathrm{RL}}\left(i,j\right).Math.j^2\right)}{\underset{i=1}{\overset{N_g}{.Math.}}\underset{j=1}{\overset{N_r}{.Math.}}{Q}_{\mathrm{RL}}\left(i,j\right)}& \left(B\mathrm{.26}\right)\end{array}$

(92) In this case long run lengths are emphasized. For smoother images RLE should take larger values while SRE takes larger values with coarser image.

(93) 20. Gray Level Non-uniformity

(94) $\begin{array}{cc}\mathrm{SRE}=\frac{\underset{i=1}{\overset{N_g}{.Math.}}{\left[\underset{j=1}{\overset{N_r}{.Math.}}{Q}_{\mathrm{RL}}\left(i,j\right)\right]}^2}{\underset{i=1}{\overset{N_g}{.Math.}}\underset{j=1}{\overset{N_r}{.Math.}}{Q}_{\mathrm{RL}}\left(i,j\right)}& \left(B\mathrm{.27}\right)\end{array}$

(95) This feature takes small values when the runs are uniformly distributed among the gray levels.

(96) 21. Run Percentage

(97) $\begin{array}{cc}\mathrm{RP}=\frac{\underset{i=1}{\overset{N_g}{.Math.}}\underset{j=1}{\overset{N_r}{.Math.}}{Q}_{\mathrm{RL}}\left(i,j\right)}{P}& \left(B\mathrm{.28}\right)\end{array}$

(98) Run percentage takes high values for coarse images. For each angular direction, the complete set of second-order statistics and run-length features was computed but only the average value was used as feature.

(99) Shape and Size Based Features

(100) We extended the number of extracted image traits by adding measurements of the size and shape of the tumor region. For every two-dimensional image of the tumor in a given CT stack three features are obtained, maximum cross-sectional area, perimeter and major axis length as follows:

(101) 22. Area

(102) We count the number of pixels in the ROI's and the maximum count is denoted as the maximum cross-sectional area.

(103) 23. Perimeter

(104) Is the distance between each adjoining pair of pixels around the border of the region; the total sum of the perimeters for each ROI image is taken as feature.

(105) 24. Major Axis Length

(106) This feature specifies the maximum length in pixels of the major axis of a two-dimensional ROI image.

(107) 25. Volume

(108) The total volume of the tumor is determined by counting the number of pixels in the tumor region and multiplying this value by the voxel size. The voxel size is obtained from the PixelSpacing section of the CT Dicom Header which specifies the size of a voxel in the x, y, and z directions. The result is a value in mm.sup.3. Based on the CT-GTV volume that was described above, 3D representations of the tumor volume have been rendered.

(109) 26. Maximum Diameter

(110) In contrast with the major axis-length which was determined in two-dimensional ROI images, this feature examines the maximum diameter of the tumor region in a three-dimensional space. Firstly, we obtain the coordinates of all the points located at the surface of the tumor region; secondly, the distance between each pair of points in the tumor contour is determined using the following metric called “City Bloc Distance”:
D=|x.sub.1−x.sub.2|+|y.sub.1−y.sub.2|+|z.sub.1−z.sub.2|  (B.29)

(111) The points in the tumor contour whose edges touch are 1 unit apart; points diagonally touching are separated by two units. The two points with the maximum distance are the points at the edges of the maximum diameter. In FIG. 3, as referred to above, a plot of the points in the surface of a given tumor volume is shown; the maximum diameter is calculated among the points in this image.

(112) So far we have described the extraction of image traits regarding the gray level and spatial relationship between pixels in a region, as well as size measurements of the tumor region in two and three-dimensions. Another important issue in the task of patter recognition is the analysis of shape; in this regard the extracted image traits are completed by adding the following three shape-based features:

(113) 27. Surface to Volume Ratio.

(114) This feature is intended to express how spiky or sharp is the tumor volume. A more lobulated tumor volume would result in a higher surface to volume ratio. To calculate this feature first we determine and count the pixels located at the surface of the tumor (e.g. as shown in FIGS. 2A-1 and 2B-1); the resulting number is divided by the sum of all the pixels in the tumor volume.

(115) 28. Sphericity

(116) This is a measure of how spherical or rounded is the shape of the tumor volume. Defined in [16], the sphericity of an object is the ratio of the surface area of a sphere (with the same volume as the given object) to the surface area of the object:

(117) 0$\begin{array}{cc}\Psi =\frac{{{\pi }^{\frac{1}{3}}\left(6V\right)}^{\frac{2}{3}}}{A}& \left(B\mathrm{.30}\right)\end{array}$
Where A and V are the surface area and volume of the tumor respectively as determined for the surface to volume ratio.

(118) 29. Compactness

(119) This is an intrinsic characteristic of the shape of objects that has been widely used in pattern recognition tasks and represents the degree to which a shape is compact. The compactness of a three-dimensional tumor volume is obtained as follows:

(120) $\begin{array}{cc}\mathrm{Comp}=\frac{V}{\sqrt{\pi }{A}^{2/3}}& \left(B\mathrm{.31}\right)\end{array}$

(121) The similarity to a sphere and compactness features are dimensionless numbers and they are independent to scaling and orientation. The feature generation phase of this methodology can be performed in a semi-fully automated fashion since the tumor delineations carried out by the physician are needed by the algorithm. The features enlisted in this appendix will be fed to a classifier as inputs in the learning and recognition phase of the classification task.

Radiomics Signature

(122) The radiomics signature that is used in the method of the present invention is in more detail described below. The signature itself contains the following features: Statistics Energy, Shape Compactness, RLGL Gray Level Nonuniformity, Wavelet HLH RLGL Gray-Level Nonuniformity. These features are described herewith:

(123) 30. Statistics Energy

(124) This feature is described by the following equation:

(125) $\begin{array}{cc}{E}_{\mathrm{tot}}=V_{\mathrm{voxel}}\underset{x=1}{\overset{x}{.Math.}}\underset{y=1}{\overset{y}{.Math.}}\underset{z=1}{\overset{z}{.Math.}}{I\left(x,y,z\right)}^2& \left(B\mathrm{.32}\right)\end{array}$

(126) Where V.sub.voxel is the voxel volume of the three dimensional image. The voxel volume is the product of the pixel spacing in x-direction, the pixel spacing in y-direction and the pixel spacing in z-direction. Total Energy is normalized by the voxel volume.

(127) 31. Shape Compactness

(128) This feature is already described above as parameter 29 (equation B.31 above). Compactness, as the name already states, indicates how compact a 3D shape is. The most compact shape is a perfect sphere.

(129) 32. Gray Level Non-uniformity (GLN)

(130) This feature is described by the following equation:

(131) $\begin{array}{cc}\mathrm{RLN}=\frac{\underset{i=1}{\overset{N_g}{.Math.}}{\left(\underset{j=1}{\overset{N_r}{.Math.}}p\left(i,j|\theta \right)\right)}^2}{\underset{i=1}{\overset{N_g}{.Math.}}\underset{j=1}{\overset{{N_{}}_r}{.Math.}}p\left(i,j|\theta \right)}& B\mathrm{.33}\end{array}$

(132) This gray-level run-length feature quantifies the textural heterogeneity in three dimensions within the tumor volume.

(133) 33. Wavelet HLH RLGL Gray-level Non-uniformity (GLN)

(134) This Gray-Level Run-Length feature is the same as in equation B.33 above, but instead it is applied to the high-low-high filtered wavelet transform of the image data, quantifying the textural heterogeneity in three dimensions within the tumor volume. This parameter is thus obtained by taking the wavelet transform of the image and performing a high-low-high filtering

(135) In the above, V denotes the volume of the tumor, meaning the total number of voxels multiplied by the voxel size of a single voxel. The dimensions of the 3D volume are denoted by X,Y,Z. The total surface area of the tumor is denoted by A.

(136) The gray level co-occurrence matrix is a matrix or distribution that is defined over an image to be the distribution of co-occurring values at a given offset. For the calculation of the Gray Level Non-uniformity (GLN), p denoted the gray level value of the corresponding voxel. The method is applied in all 3D directions. The wavelet transform is a time-frequency-transformation based on a wavelet series. Wavelet series are a representation of a square-integrable (real- or complex-valued) function by a certain orthonormal series generated by a wavelet. This representation is performed on a Hilbert basis defined by orthonormal wavelets. The wavelet transform provides information similar to the short-time-Fourier-transformation, but with additional special properties of the wavelets, which show up at the resolution in time at higher analysis frequencies of the basis function. Wavelet transforms provide the frequency of the signals and the time associated to those frequencies. High-low-high filtering applies to data analysis methods relying on wavelet transforms to detect certain activity patterns or variation patterns in the data; the high-low-high is thereby indicative of the wavelet shape. The high-low-high filter of the wavelet transform is used to calculate feature 33 above. The transform is applied directly on the raw CT image.

(137) The obtained image feature parameter value is of particular prognostic value, and may be used alone or in combination with other features within a signature.

(138) A neoplasm signature model value of an imaged neoplasm obtained using this particular signature model may be obtained as follows. Where no weighting factors would be used, signature model selector values simply take the value ‘1’ in case an image feature is taken along in the signature model, or ‘0’ in case the image feature is ignored. The signature selector values are used here as multipliers, which multiply the corresponding image feature parameter values. Then, the multiplied values may be summed (or alternatively a different functional relation may be applied, e.g. a polynomial, multiplication, or any other suitable relation).

(139) In the present radiomics signature model, the image features indicated above (statistics energy, shape compactness, gray-level non-uniformity, and wavelet HLH RLGL gray-level non-uniformity) are selected. In the more complex model of the present embodiment, preferably, the signature model selector values include weighting factors. Therefore, instead of the selector values ‘1’ and ‘0’, the weights are applied while selecting the image feature parameter values—these weights are thus used as multiplicators for the image feature parameters of the signature model, which may then be summed to obtain signature model values. The corresponding weights are shown in the table below (the weight ranges refer to absolute values):

(140) TABLE-US-00002 Feature Weight Weight Range Statistics Energy  2.42e−11 1.0e−20-1.0e−05 Shape Compactness −5.38e−03 1.0e−07-1.0e−01 RLGL_grayLevelNonuniformity −1.47e−04 1.0e−09-1.0e−01 Wavelet_HLH_rlgl_grayLevelNon-  9.39e−06 1.0e−10-1.0e−02 uniformity

(141) Although a signature that uses the combination of the four image feature parameters is of particular predictive and prognostic value for treatment selection, it has been found that a signature based on only a subset of these features, or including other features, may still provide valuable results. In particular, signatures that include the Wavelet HLH RLGL Gray-Level Non-uniformity (GLN)—the gray level non-uniformity of the high-low-high filtered wavelet transform of the image data, are of particular value. All of these signatures fall within the scope of the present invention as defined by the claims.

(142) In the above description, the invention is described with reference to some specific embodiments thereof. However, it will be appreciated that the present invention may be practiced otherwise than specifically described herein, in relation to these embodiments. Variations and modifications to specific features of the invention may be apparent to the skilled reader, and are intended to fall within the scope of the invention. The scope of the invention is merely restricted by the scope of the appended claims.