QUANTITATIVE IMAGE-BASED DISORDER ANALYSIS FOR EARLY DETECTION OF MELANOMA TYPE FEATURES

Abstract

A method of distinguishing benign and malignant skin conditions includes extracting a numerical value corresponding to an order parameter from an image of skin having a pigmented region. The numerical value of the order parameter may be utilized to assess the likelihood that a skin lesion is benign or malignant. The precise value may also be utilized to assess severity, which may include detecting changes in a skin lesion over time.

Claims

1. A computer-implemented method of distinguishing between benign and malignant skin conditions utilizing an order parameter, the method comprising: utilizing a computer to extract a numerical value corresponding to an order parameter squared (S.sup.2) from image data corresponding to an image of skin, wherein the image data includes at least one pigmented skin lesion having light regions and dark regions, and wherein S.sup.2 comprises a numerical value quantifying a degree of order present in the image data, and wherein the extracted numerical value comprises a ratio of an area of the light regions to a total area that is equal to the sum of an area of the light regions and an area of the dark regions; comparing the extracted numerical value to a predefined malignant value; and determining that the pigmented skin lesion is likely to be malignant if the numerical value of the extracted S.sup.2 is less than or equal to the predefined malignant S.sup.2 value.

2. The method of claim 1, wherein: the extracted numerical value comprises an extracted S.sup.2 value; and including: determining that the pigmented skin lesion is likely to be benign if the extracted S.sup.2 value is greater than or equal to a predefined benign S.sup.2 value.

3. The method of claim 2, wherein: the predefined malignant S.sup.2 value is less than the predefined benign S.sup.2 value.

4. The method of claim 3, including: determining that the malignancy of the pigmented skin lesion is indeterminate if the extracted S.sup.2 value is between the predefined malignant S.sup.2 value and the predefined benign S.sup.2 value.

5. The method of claim 2, wherein: the predefined malignant S.sup.2 value is equal to the predefined benign S.sup.2 value.

6. The method of claim 1, wherein: the predefined malignant value comprises a predefined malignant S.sup.2 value that is determined by utilizing a computer to extract a numerical value of S.sup.2 from a plurality of sets of data corresponding to images of skin including malignant pigmented skin lesions.

7. The method of claim 6, wherein: the predefined malignant S.sup.2 value is equal to or greater than a largest numerical value of S.sup.2 extracted from the corresponding to images of skin including malignant pigmented skin lesions.

8. The method of claim 1, wherein: extracting a numerical value corresponding to S.sup.2 from image data includes selecting a region of interest that includes at least a portion of the at least one pigmented skin lesion.

9. The method of claim 8, wherein: the image data includes at least some skin that is free of skin lesions; and the region of interest does not include skin that is free of skin lesions.

10. The method of claim 8, wherein: the image data comprises a plurality of pixels; extracting a numerical value corresponding to S.sup.2 from the image data includes utilizing a computer to create a pixel intensity histogram of the region of interest.

11. The method of claim 10, wherein: extracting a numerical value corresponding to S.sup.2 from the image data includes utilizing a computer to fit first and second curves to the pixel intensity histogram corresponding to the light and dark regions, respectively.

12. The method of claim 11, wherein: extracting a numerical value corresponding to S.sup.2 from image data includes utilizing a computer to convert the image data to greyscale image data; utilizing a computer to determine a threshold value of the pixel intensity between peaks of the first and second curves; and utilizing a computer to perform a binary threshold on the region of interest using the threshold value to form a set of digital data corresponding to an image having only black and white pixels.

13. The method of claim 8, wherein: the region of interest is selected by creating a curved border around the region of interest.

14. The method of claim 1, including: utilizing a plurality of non-equal predefined malignant values corresponding to increasing probability that a skin lesion is malignant to determine a risk that a specific skin lesion is malignant.

15. The method of claim 1, wherein: the image data comprises a selected one of digital optical image data or image data from a microscope.

16. The method of claim 1, wherein: the skin lesion or region of skin of interest is measured using Raman spectroscopy.

17. The method of claim 8, wherein: selecting a region of interest comprises utilizing a computer to determine a border using an algorithm.

18. The method of claim 17, wherein: the algorithm comprises a morphological filter operation.

19. A method of identifying margins of malignant skin lesions, the method comprising: utilizing a computer to create an S.sup.2 spatial map from image data corresponding to an image of skin by forming binary image data corresponding to a binary image; followed by utilizing a computer to assign each pixel of the binary image data a greyscale value that is equal to the average value of the adjacent pixels in the binary image data; and wherein the S.sup.2 spatial map is configured to be utilized to aid in determining the margins of malignant skin lesions to facilitate removal of the entire malignant lesion without removing an excessive amount of surrounding tissue that is not malignant.

20. A computer-implemented method of distinguishing between benign and malignant skin conditions utilizing a numerical value determined from data corresponding to one or more images of skin, the method comprising: utilizing a computer to extract a numerical value from image data corresponding to a digital image of skin, wherein the digital image corresponding to the image data includes at least one region of concern comprising a potential malignancy having a total area including light regions and dark regions, and wherein the numerical value is determined, based at least in part, on an area of a selected one of the light regions and the dark regions to a total area, wherein the total area is equal to the sum of the areas of the light regions and the areas of the dark regions; estimating the likelihood that the potential malignancy is malignant based, at least in part, on a comparison of the extracted numerical value to one or more predefined numerical malignancy criteria that take into account the likelihood that the potential malignancy is malignant.

21. The method of claim 20, wherein: the numerical value is determined by dividing the area of the light regions by the total area.

22. The method of claim 21, including: determining a threshold brightness value for the digital image; determining the area of the light regions by summing the areas of the pixels having a brightness value above the threshold brightness value.

23. The method of claim 21, wherein: the extracted numerical value comprises an extracted numerical value of an order parameter squared (S.sup.2).

24. The method of claim 23, wherein: the potential malignancy comprises a pigmented skin lesion; and including: determining that the pigmented skin lesion is likely to be benign if the extracted numerical value of S.sup.2 is greater than or equal to a predefined benign S.sup.2 value.

25. The method of claim 24, wherein: the predefined malignant S.sup.2 value is less than the predefined benign S.sup.2 value.

26. The method of claim 25, including: determining that the malignancy of the pigmented skin lesion is indeterminate if the extracted numerical value of S.sup.2 is between the predefined malignant S.sup.2 value and the predefined benign S.sup.2 value.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0013] FIG. 1A is a pixel intensity histogram and image (inset) of a benign skin condition (seborrheic keratosis);

[0014] FIG. 1B is a pixel intensity histogram and image (inset) of a benign skin condition (actinic keratosis);

[0015] FIG. 1C is a pixel intensity histogram and image (inset) of a benign skin condition (melanocytic nevi);

[0016] FIG. 2A is a pixel intensity histogram and image (inset) of a malignant skin condition (basal cell carcinoma);

[0017] FIG. 2B is a pixel intensity histogram and image (inset) of a malignant skin condition (squamous cell carcinoma);

[0018] FIG. 2C is a pixel intensity histogram and image (inset) of a malignant skin condition (melanoma);

[0019] FIG. 3 is a plot of order parameter squared (S.sup.2) value for four separate benign types of skin lesions and three separate malignant types of skin lesions;

[0020] FIG. 4 is a flowchart showing a method of extracting order parameter squared (S.sup.2) from an image;

[0021] FIG. 5 is a graph showing an aspect of the process of FIG. 4; and

[0022] FIG. 6 illustrates an S.sup.2 spatial map created from an image of a skin lesion.

DETAILED DESCRIPTION

[0023] It is to be understood that the items described herein may assume various alternative orientations and step sequences, except where expressly specified to the contrary. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification, are simply exemplary embodiments of the inventive concepts defined in the appended claims. Hence, specific dimensions and other physical characteristics relating to the embodiments disclosed herein are not to be considered as limiting, unless the claims expressly state otherwise.

[0024] Additionally, unless otherwise specified, it is to be understood that discussion of a particular feature or component extending in or along a given direction or the like does not mean that the feature or component follows a straight line or axis in such a direction or that it only extends in such direction or on such a plane without other directional components or deviations, unless otherwise specified.

[0025] One aspect of the present disclosure is a method of extracting an order parameter S, or an order parameter squared (S.sup.2) from images of skin lesions (FIGS. 1A-1C and 2A-2C). The extraction process is discussed in more detail below in connection with FIG. 4. As discussed in more detail below, larger S.sup.2 values generally indicate that a lesion may be benign, whereas lower S.sup.2 values may indicate that a skin lesion is malignant. Thus, the S.sup.2 value extracted from dermatoscopic images may be utilized to determine if a skin lesion is benign or malignant. As used herein, the term “order parameter” broadly refers to S, S.sup.2, or any other suitable parameter or quantity relating to a degree of order present in a physical item and/or an image of a physical item.

[0026] In general, the order parameter S specifies or quantifies the degree of disorder characterizing a specific physical sample in a number of physical systems. In some cases, it is possible to link the order parameter to a physical parameter of interest (such as band gap energy, or critical temperature). Even in cases where such a physical property is not immediately evident, comparison between samples, or quantitative analysis of physical system evolution, can be obtained by comparing order parameter values.

[0027] There are established relationships between the degree of ordering which characterizes a physical system and key system properties. Thus, an appropriate metric for ordering can, in some instances, provide the basis for a detailed understanding of the underlying mechanisms which influence properties, and suggest possible ways to control them. Quantifying system ordering is possible across multiple length scales ranging from the microscopic to the astronomical. Temporal variation of the order parameter yields valuable information regarding system evolution over a range of time scales.

[0028] It is possible to experimentally quantify the degree of disorder in physical systems using a metric such as the Bragg-Williams order parameter (S). For a perfectly ordered system S=1, for a system with complete disorder S=0, and partially ordered systems exhibit a value of S between 0 and 1. Foundational work for obtaining an experimental measurement of S was accomplished via x-ray diffraction measurements on metal binary alloys, such as CuAu and beta-brass (ZnCu). A methodology for extracting S from Raman spectra, reflection high-energy electron diffraction (RHEED), and electron microscopy images has also been developed. These techniques have been applied to heterovalent ternary semiconductors to establish a relationship between disorder and critical system-level properties of the material, specifically the band gap. However, the approach applies to semiconductors in general, including silicon and graphene, and also organic-based polymers as well as biological systems in the context of, for example, viruses and vaccines and skin conditions.

[0029] For the case of an atomic lattice with two elements (A and B) the Bragg-Williams order parameter is defined as S=r.sub.A+r.sub.B−1, where r.sub.A (r.sub.B) is the ratio of A (B) atoms on A (B) lattice sites; in the case of N different elements S=(r.sub.A+r.sub.B+ . . . +r.sub.N−1)/(N−1). However, experimental techniques do not require knowledge apriori of the definition of S, i.e., a methodology for extracting S from experimental techniques, whether x-ray diffraction, RHEED, Raman spectroscopy, or electron microscopy, applies regardless of the number of elements responsible for the disorder. While the full range of S is from 0 to 1, the maximum value achievable in a given system is limited by the compositional stoichiometry, i.e., the perfectly ordered state S=1 is only achievable when there are equal amounts of all constituent elements. For the specific case of two elements, where the composition x is defined as

[00001]$x=\frac{N_A}{N_A+N_B}$

with N.sub.A (N.sub.B) equal to the number of A (B) elements in the system, the maximum S value is S.sub.max=2x for x<0.5 and 2(1−x) for x>0.5; similar constraints can be derived for cases with a higher number of unique elements. Additionally, system-level properties dominated by pair interactions have a linear relationship with S.sup.2. By using a spin modeling technique (each element type is assigned a different spin), in conjunction with cluster expansion theory limited to single and pair-wise interaction terms, it can be shown that P(x, S)=S.sup.2[P(x=0.5, S=1)−P(x, 0)]−P(x, 0). For this reason, and the fact that S.sup.2 is the value often obtained through experimental measurement, the squared order parameter S.sup.2 is discussed herein instead of S.

[0030] FIGS. 1A-1C are pixel intensity histograms (graphs) for benign skin lesions showing disorder-contribution curve fits, order-contribution curve fits, and over-all fits based on a region of interest of a digital dermatoscopic image inset in each graph. As discussed in more detail below, the S.sup.2 value for a lesion is extracted from the region of interest. As discussed in more detail below in connection with FIG. 4, the region of interest may be selected by a user, a computer-implemented algorithm, or a combination thereof. The image (inset) of FIG. 1 shows seborrheic keratosis, and a region of interest that is bounded by the yellow ellipse around the pigmented region of the skin.

[0031] The inset of FIG. 1B is a dermatoscopic image of actinic keratosis. As is shown in the inset, the region of interest in this example is rectangular.

[0032] The inset of FIG. 1C is a dermatoscopic image of melanocytic nevi, and the region of interest comprises an ellipse that is approximately circular.

[0033] FIGS. 2A-2C are pixel intensity (brightness) histograms and insets of corresponding digital dermatoscopic images of malignant skin lesions. It will be understood that a pixel having very low intensity will be darker black in greyscale, and a pixel having high intensity will be very light (white) in greyscale. Specifically, the inset for FIG. 2A is a dermatoscopic image of a basal cell carcinoma, the inset of FIG. 2B is a dermatoscopic image of a squamous cell carcinoma, and the inset of FIG. 2C is a dermatoscopic image of melanoma.

[0034] In general, the region of interest is selected to surround (include) the pigmented region, and to exclude all or most of the adjacent skin that is not pigmented. Thus, the region of interest typically includes a large portion of the pigmented lesion, without including adjacent skin that is not pigmented. The region of interest may be selected by an individual evaluating an image, or by a computer that is configured (e.g. programmed) to determine a boundary around an area of interest.

[0035] The extracted S.sup.2 values for the benign skin conditions of FIGS. 1A-1C are 0.7218, 0.7160, and 0.6245, respectively. The extracted S.sup.2 values for the malignant skin conditions for FIGS. 2A-2C are 0.2766, 0.2263, and 0.3234, respectively. Thus, in the examples of FIGS. 1A-2C and FIGS. 2A-2C, the S.sup.2 values of the benign skin conditions are significantly greater than 0.5, and the extracted S.sup.2 values for malignant skin conditions are significantly less than 0.5.

[0036] The methodology was applied to all images of skin lesions in the HAM10000 dataset, the results of which are shown in FIG. 3. FIG. 3 comprises a plot of extracted S.sup.2 values for four separate benign skin lesions, and three separate malignant types of skin lesions. The number n adjacent to each line is the number of different samples evaluated for that condition. The symbol denotes the average value over n samples, and the bars denote the first standard deviation from the average value. The results shown in FIG. 3 indicate a significant separation in S.sup.2 values between the benign skin lesions (green points) and malignant skin lesions (red points), with all malignant lesions having an S.sup.2 value below 0.5 and all benign lesions having an S.sup.2 value greater than 0.5.

[0037] As discussed above, S=1 corresponds to a perfectly ordered system, and a completely disordered system corresponds to S=0. Not wishing to be bound by any specific theory, it is hypothesized that it is the order parameter associated with discrete skin cells that are actually measured when S.sup.2 is extracted from images of skin lesions, leading to the results described herein.

[0038] With reference to FIG. 4, a process 10 for calculating the threshold value of an image 10 is shown. One or more steps of process 10 may be implemented utilizing a computer. For example, a computer may be configured (programmed) to execute one or more of the steps of process 10. Process 10 starts as shown at 12, and step 14 includes converting the image to greyscale. A region of interest may be selected as shown in step 16. In general, the region of interest may be selected before or after converting an image to greyscale. The region of interest may be selected by an individual inspecting one or more dermatoscopic images, or the region of interest may be selected by a computer algorithm. As shown in FIGS. 1A-1C and FIGS. 2A-2C, the region of interest may have virtually any shape as required to bound a representative portion (region) of the skin lesion.

[0039] The process 10 further includes calculating a pixel intensity histogram of the selected region (see, e.g., FIGS. 1A-1C and 2A-2C). At step 20, an algorithm (e.g., a stochastic funnel algorithm) is used to calculate initial fitting parameters for two skewed Gaussian curves to the pixel intensity (brightness) histogram data. In the examples of FIGS. 1A-1C and 2A-2C, the skewed Gaussian curves are shown as the disordered and ordered curves, and the over-all fit is also shown. Referring again to FIG. 4, at step 22, the least squares method may be used to calculate the Gaussian curve parameters using the initial fitting parameters from step 20.

[0040] At step 24, a root-finding algorithm (e.g., Newton's method) is used to find the intersection between the two Gaussian curves resulting from the curve fit. At step 26, a number of standard deviations that the intersection is away from the curve where the highest center point is calculated. A threshold value is set to the value of the highest center point value minus the floor of that number of standard deviations. As discussed below, the threshold value may be used to determine which pixels are “bright” (white), and which pixels are “dark” (black).

[0041] At step 28, a binary threshold is performed on the region of interest in the image using the threshold calculated in step 26. Pixels having an intensity that is greater than the threshold value are given (assigned) a white (high) intensity value, and pixels having an intensity that is less than the threshold value are given (assigned) a black (low) intensity value. In general, the result of the binary threshold is a black (dark) and white (bright) image (not shown) having white (ordered) regions and black (disordered) regions. At step 30, a numerical value, which may comprise the squared order parameter (S.sup.2) value of the region of interest, is calculated by counting the bright (white) pixels in the thresholded image and dividing this number by the total number of pixels contained with the region of interest. The total number of pixels is equal to the sum of the number of dark (black) pixels and the number of bright (white) pixels. Because the sizes (areas) of each of the pixels are the same, the S.sup.2 value is the ratio of the area of the bright regions to the total area. The method 10 then ends as shown at 32. It will be understood that the numerical value may also comprise the ratio of the area of the dark regions to the total area (i.e. 1−S.sup.2).

[0042] The numerical value (e.g. S.sup.2) determined utilizing the process 10 of FIG. 4 may then be compared to predefined criteria (e.g. S.sup.2) values extracted from images known to be malignant or benign in order to estimate the chances that the region of interest includes a malignant skin condition. For example, a computer may be configured (programmed) to generate a “likely benign” output or signal if the numerical value of S.sup.2 is greater than a predefined value (e.g. 0.4, 0.5, 0.6), and a computer may be configured (programmed) to generate a “likely malignant” output or signal if the numerical value of S.sup.2 is less than the predefined value (e.g. 0.4, 0.5, 0.6). Also, a computer may be configured to generate an “indeterminate signal” is the numerical value of S.sup.2 is between a predefined “likely benign” numerical value (e.g. 0.55) and a predefined “likely malignant” numerical value (e.g. 0.4).

[0043] FIG. 5 shows the process for finding the threshold value 44 from the fitted curves 35 and 36. As discussed above in connection with FIG. 4, a disordered curve 34 and ordered curve 36 may be fitted using, for example, skewed Gaussian curved parameters. The inner section 38 of the curves 34 and 36 may then be determined (e.g., using Newton's method), and the threshold value 44 may then be determined. In the example of FIG. 5, the disordered curve 34 has a center point (peak) 40, and the ordered curve 36 has a center point (peak) 42. In the example of FIG. 5, the threshold value 44 is approximately 140. It will be understood that the present disclosure is not limited to a specific technique or approach for determining the threshold value.

[0044] With further reference to FIG. 6, an S.sup.2 (or S) spatial map 50 (greyscale) may be derived or extracted from an original image 10A. In general, a single (global average) S.sup.2 value can be extracted from an image, or it can be restricted to a smaller region and values can be extracted that are specific to those regions. If S.sup.2 images are extracted at different points in time, the S.sup.2 spatial maps can be compared to see not only changes over time, but more details (e.g., where it is changing). Also, spatial mapping of S.sup.2 for an image of a lesion facilitates determining the margins of malignant lesions, which may be required for surgically removing the lesions.

[0045] The greyscale S.sup.2 spatial map 50 may be created by first determining the S.sup.2 value of the entire original image 10A. Then, a binary image (black and white) is created from the threshold process (e.g. step 26, FIG. 4) of the S.sup.2 analysis to generate the S.sup.2 grey map 50. A box blur or normalization operation is applied to the binary image, which results in each pixel in the resulting image having a value equal to the average value (greyscale) of its neighboring pixels in the input binary image. For example, a black pixel may have a value of 0, and a white pixel may have a value of 255, and a greyscale number may have a value between 0 and 255 as shown by a greyscale 52. Thus, in this example, the value that results from the taking the average value of the neighboring pixels in the binary image, which are all white (255) or black (0), is a greyscale value between 0 and 255. The resulting image from this process is the S.sup.2 map 50. The number of neighboring pixels included in the normalization process may be varied based on the desired resolution of the S.sup.2 map 50 and the overall resolution of the original image.

[0046] It will be understood that the processes described herein comprise a screening tool that may assist in determining if an image includes benign or malignant skin conditions. However, the processes described herein are not intended to be the sole criteria for determining if a skin condition is malignant, which determination will require additional evaluation and testing by medical specialists.

[0047] The examples described above generally relate to skin lesions. However, the process described above may also be utilized to evaluate other malignancies, including other types of tumors besides skin lesions. In general, virtually any image of a potential malignancy may be evaluated according to the process described in connection with FIGS. 4 and 5. It will be understood that the images could comprise virtually any type of image.

[0048] It will be understood by one having ordinary skill in the art that construction of the described device and other components is not limited to any specific material. Other exemplary embodiments of the device disclosed herein may be formed from a wide variety of materials, unless described otherwise herein.

[0049] It is also to be understood that variations and modifications can be made on the aforementioned structures and methods without departing from the concepts of the present disclosure, and further it is to be understood that such concepts are intended to be covered by the following claims unless these claims by their language expressly state otherwise.

[0050] The above description is considered that of the illustrated embodiments only. Modifications of the processes will occur to those skilled in the art and to those who make or use the processes. Therefore, it is understood that the embodiments shown in the drawings and described above are merely for illustrative purposes and not intended to limit the scope of the disclosure, which is defined by the following claims as interpreted according to the principles of patent law, including the Doctrine of Equivalents.