DETERMINATION DEVICE, DETERMINATION METHOD, AND PROGRAM

Abstract

A determination device 1 includes: an acquisition unit 21 that acquires digital color image data 11 of a region of interest in melanonychia of a subject; a calculation unit 23 that calculates an indicator value 12 from variation in RGB values of each pixel of the digital color image data 11; and an output unit 24 that outputs a result of determining that the melanonychia is malignant if the indicator value 12 is higher than a threshold value and determining that the melanonychia is benign if the indicator value 12 is lower than the threshold value.

Claims

1-6. (canceled)

7. A determination device comprising: an acquisition unit that acquires digital color image data of a region of interest in melanonychia of a subject; a calculation unit that calculates an indicator value from variation in RGB values of each pixel of the digital color image data; and an output unit that outputs a result of determining that the melanonychia is malignant if the indicator value is higher than a threshold value and determining that the melanonychia is benign if the indicator value is lower than the threshold value.

8. The determination device according to claim 7, further comprising: a conversion unit that adjusts a color balance of the digital color image data by performing a chromatic adaptation transformation, wherein the acquisition unit calculates the indicator value from the digital color image data after the chromatic adaptation transformation of the region of interest.

9. The determination device according to claim 7, wherein the calculation unit calculates the indicator value from variation of the R value, variation of the G value, and variation of the B value of each pixel.

10. The determination device according to claim 8, wherein the calculation unit calculates the indicator value from variation of the R value, variation of the G value, and variation of the B value of each pixel.

11. The determination device according to claim 7, wherein the calculation unit calculates the indicator value based on: $\mathrm{DI}={\left[\frac{1}{N}\left\{{.Math.}_{i=1}^N{\left(R_i-\stackrel{?}{R}\right)}^2+{.Math.}_{i=1}^N{\left(G_i-\stackrel{?}{G}\right)}^2+{.Math.}_{i=1}^N{\left(B_i-\stackrel{?}{B}\right)}^2\right\}\right]}^{1/2},$ wherein DI corresponds to an indicator value, N corresponds to a number of pixels in digital color image data, R.sub.i, G.sub.i, B.sub.i correspond to values of each of RGB of pixel i, and R, G, B correspond to an average value of values of each of RGB of digital color image data.

12. The determination device according to claim 8, wherein the calculation unit calculates the indicator value based on: $\mathrm{DI}={\left[\frac{1}{N}\left\{{.Math.}_{i=1}^N{\left(R_i-\stackrel{?}{R}\right)}^2+{.Math.}_{i=1}^N{\left(G_i-\stackrel{?}{G}\right)}^2+{.Math.}_{i=1}^N{\left(B_i-\stackrel{?}{B}\right)}^2\right\}\right]}^{1/2},$ wherein DI corresponds to an indicator value, N corresponds to a number of pixels in digital color image data, R.sub.i, G.sub.i, B.sub.i corresponds to values of each of RGB of pixel i, R, G, B corresponds to an average value of values of each of RGB of digital color image data.

13. A determination method comprising: a computer acquiring digital color image data of a region of interest in melanonychia of a subject; the computer calculating an indicator value from variation in RGB values of each pixel of the digital color image data; and the computer outputting a result of determining that the melanonychia is malignant if the indicator value is higher than a threshold value and determining that the melanonychia is benign if the indicator value is lower than the threshold value.

14. A program for causing a computer to function as the determination device according to claim 7.

15. A program for causing a computer to function as the determination device according to claim 8.

16. A program for causing a computer to function as the determination device according to claim 9.

17. A program for causing a computer to function as the determination device according to claim 10.

18. A program for causing a computer to function as the determination device according to claim 11.

19. A program for causing a computer to function as the determination device according to claim 12.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0017] FIG. 1 is a diagram for explaining functional blocks of a determination device according to an embodiment of the present invention.

[0018] FIG. 2 is a diagram for explaining how an image of melanonychia is captured using a dermoscope.

[0019] FIG. 3 shows an example of melanonychia.

[0020] FIG. 4 is a flowchart for explaining determination processing performed by a determination device.

[0021] FIG. 5 is a diagram for explaining a result of determination performed by the determination device according to the embodiment of the present invention.

[0022] FIG. 6 is a diagram for explaining a hardware configuration of a computer used for a determination device.

DESCRIPTION OF EMBODIMENT

[0023] An embodiment of the present invention will be described below with reference to the drawings. In the description of the drawings, the same parts are denoted with the same reference numerals, and the description thereof is omitted.

(Determination Device)

[0024] A determination device 1 according to an embodiment of the present invention shown in FIG. 1 calculates an indicator value based on image data obtained by capturing an image of melanonychia 102 that has occurred on a subject's nail 101 using a dermoscope 2 as shown in FIGS. 2 and 3. The determination device 1 compares the indicator value with a threshold value and determines whether the melanonychia 102 is malignant or benign. The result of the determination performed by the determination device 1 supports a diagnosis of the melanonychia 102 performed by a user such as a physician.

[0025] The determination device 1 is provided with pieces of data of digital color image data 11 and an indicator value 12 and the functions of an acquisition unit 21, a conversion unit 22, a calculation unit 23, and an output unit 24. The pieces of data are stored in a memory 902 or a storage 903. The functions are implemented by a CPU 901.

[0026] The digital color image data 11 is data of a region of interest in the subject's melanonychia 102. The digital color image data 11 is captured by using, for example, the dermoscope 2 with a camera function. The region of interest (ROI) is a portion including the subject's melanonychia 102 or a portion of the subject's melanonychia 102. The digital color image data 11 is data obtained by associating each pixel corresponding to the region of interest with a color value represented by the pixel and digitizing the region of interest. In the embodiment of the present invention, the digital color image data 11 is obtained by associating each pixel in the region with a color value of each of red, green, and blue (RGB).

[0027] The indicator value 12 is calculated from variation in the RGB values of each pixel of the digital color image data 11. The indicator value 12 is an indicator for determining whether the region of interest is benign or malignant. The indicator value is also referred to as a discrimination index (DI) value.

[0028] The acquisition unit 21 acquires the digital color image data 11 of the region of interest in the subject's melanonychia 102. The acquisition unit 21 acquires, from the dermoscope 2, digital-format image data obtained by capturing an image of a subject's nail 101 part. The acquisition unit 21 generates the digital color image data 11 by cutting out the region of interest from the acquired image data. The region of interest may be specified by a user or by a prescribed program.

[0029] The conversion unit 22 adjusts the color balance of the digital color image data 11 by performing a chromatic adaptation transformation and updates the digital color image data 11. The conversion unit 22 adjusts the color balance, and corrects and standardizes the hue of the digital color image data 11.

[0030] Generally, in a dermoscopy inspection, the color balance may differ due to the imaging environment using a dermoscope, the image processing engine used for the dermoscope, and the like. For example, the imaging environment using the dermoscope may be influenced by the spectral distribution of illumination light, which varies depending on the device used for the dermoscopy inspection, or the like. Therefore, the conversion unit 22 eliminates the difference in the color balance caused by the imaging environment and the image processing engine in the inspection by adjusting the color balance. The conversion unit 22 can accurately determine the melanonychia 102 by adjusting the color balance.

[0031] The conversion unit 22 adjusts the hue of each piece of digital color image data such that the appearance of white becomes constant in digital color image data captured by using various devices. The conversion unit 22 standardizes the hue of the digital color image data obtained under various conditions by converting an average chromaticity coordinate of prescribed pixels in the data to a preset reference chromaticity coordinate. The details are disclosed in Patent Literature 4. Patent Literature 4 is incorporated herein.

[0032] The calculation unit 23 calculates the indicator value 12 from the variation in RGB values of each pixel of the digital color image data 11. The calculation unit 23 preferably uses digital color image data 11 of which the hue has been adjusted by the conversion unit 22.

[0033] The calculation unit 23 calculates the indicator value 12 from variation in the R value, variation in the G value, and variation in the B value of each pixel. The indicator value 12 is calculated by normalizing the sum of the standard deviation of each of the R value, the G value, and the B value of each pixel of the digital color image data 11 by the number of pixels.

[0034] The calculation unit 23 calculates the indicator value 12 from formula (1).

[00001]$\begin{array}{cc}\mathrm{DI}={\left[\frac{1}{N}\left\{{.Math.}_{i=1}^N{\left(R_i-\stackrel{?}{R}\right)}^2+{.Math.}_{i=1}^N{\left(G_i-\stackrel{?}{G}\right)}^2+{.Math.}_{i=1}^N{\left(B_i-\stackrel{?}{B}\right)}^2\right\}\right]}^{1/2}& \mathrm{Formula}\left(1\right)\end{array}$ [0035] DI: Indicator value [0036] N: Number of pixels in digital color image data [0037] R.sub.i, G.sub.i, B.sub.i: Values of each of RGB of pixel i [0038] R, G, B: Average value of values of each of RGB of digital color image data

[0039] The calculation unit 23 calculates a higher indicator value 12 as the diversity of the colors in the digital color image data 11 increases. The calculation unit 23 calculates a lower indicator value as the diversity of the colors in the digital color image data 11 decreases.

[0040] The output unit 24 determines that the melanonychia 102 is malignant if the indicator value is higher than the threshold value, and that the melanonychia 102 is benign if the indicator value is lower than the threshold value, and outputs the determination result. When each value of RGB is represented in 8 bits and has a range of 0 to 255, the threshold value is, for example, 40.

[0041] The determination result output by the output unit 24 is not the final determination made by the physician but one element for supporting the physician's determination. For example, if the indicator value 12 is higher than the threshold value, there is a high possibility that the melanonychia 102 is malignant, and therefore the output unit 24 may display a message prompting a biopsy. If the indicator value is lower than the threshold value, there is a high possibility that the melanonychia 102 is benign, and therefore the output unit 24 may display a message such as no biopsy is required or a follow-up observation is advised.

[0042] With reference to FIG. 4, a determination method performed by the determination device 1 according to the embodiment of the present invention will be described. The processing sequence shown in FIG. 4 is an example, and the sequence is not limited thereto.

[0043] In step S1, the determination device 1 acquires the digital color image data 11 of the region of interest of the subject's melanonychia 102. In step S2, the determination device 1 adjusts the color balance of the digital color image data 11 acquired in step S1 by performing the chromatic adaptation transformation and updates the data.

[0044] In step S3, the determination device 1 calculates the indicator value from the variation in RGB values of each pixel of the digital color image data 11 of which the color balance has been adjusted in step S2.

[0045] In step S4, the determination device 1 compares the indicator value calculated in step S3 with the threshold value to determine whether the melanonychia is malignant or benign. If the indicator value is higher than the threshold value, in step S5, the determination device 1 determines that there is a high possibility that the melanonychia 102 imaged as the digital color image data 11 is malignant. If the indicator value is lower than the threshold value, in step S6, the determination device 1 determines that there is a high possibility that the melanonychia 102 imaged as the digital color image data 11 is benign.

[0046] In step S7, the determination device 1 outputs the determination result in step S5 or step S6 to an output device such as a display.

[0047] The results of determination performed by the determination device 1 according to the embodiment of the present invention will be described with reference to FIG. 5. Both of FIGS. 5(a) and 5(b) show receiver operating characteristic (ROC) curves in which the region of interest of the melanonychia 102 is determined. The vertical axis of the ROC curve is a true positive rate and the horizontal axis of the curve is a false positive rate. The area under the curve (AUC) is the area of the portion below the ROC curve. It is shown that the discrimination capability increases as the AUC value approaches 1.

[0048] FIG. 5(a) shows the result of determination by means of the methods disclosed in Patent Literature 1 and Patent Literature 2. FIG. 5(b) shows the result of determination performed by the determination device 1 according to the embodiment of the present invention. The results of determination shown in FIGS. 5(a) and 5(b) are obtained by performing determination for 100 cases of melanonychia. Out of the 100 cases, 80 cases are determined to be benign and 20 cases are determined to be malignant according to the physician's diagnosis. The true positive rate and false positive rate are calculated from the determination results according to each determination method and a correct label according to the physician's diagnosis result.

[0049] In the conventional determination result shown in FIG. 5(a), the AUC is 0.711. Meanwhile, in the determination result according to the embodiment of the present invention shown in FIG. 5(b), the AUC is 0.848, and it can be seen that the result exhibited a higher discrimination capability than that exhibited by the conventional determination result.

[0050] In a determination result using deep learning by means of a convolutional neural network, the AUC of the ROC curve is 0.621 (J. K. Winkler et al., European Journal of Cancer 127 (2020) e21-29). Meanwhile, in the determination result obtained by the determination device 1 according to the embodiment of the present invention, the AUC is 0.848. It can be seen that the determination device 1 has a higher discrimination capability than that exhibited by the determination result obtained by deep learning.

[0051] Further, the sensitivity and the specificity of the detection of malignancy (melanoma) performed by the determination device 1 according to the embodiment of the present invention are 0.85 and 0.79, respectively. In contrast, the average sensitivity of eight dermatologists is 0.66 and the average specificity is 0.98. The average sensitivity of eight non-dermatologists is 0.46 and the average specificity is 0.97. This reveals that, although the specificity is higher in the determination performed by the dermatologists and non-dermatologists, the sensitivity tends to be higher in the determination performed by the determination device 1. The determination device 1 can appropriately determine malignancy and prompt a biopsy for the subject immediately compared with a determination made by a physician.

[0052] When the determination as to whether a biopsy is necessary performed by means of the majority decision of three experts with a wealth of experience in treating nail melanoma is assumed to be the correct label, the determination result by the determination device 1 exhibits a sensitivity of 0.95, a specificity of 0.82, and an AUC of 0.92. The determination device 1 can determine whether the melanonychia is benign or malignant at the same level as a physician who is an expert in nail melanoma.

[0053] The determination device 1 according to the embodiment of the present invention calculates the indicator value from the variation in RGB values of each pixel of the digital color image data 11 obtained by capturing an image of the region of interest of the melanonychia 102 by using the dermoscope 2, and determines whether the melanonychia 102 is benign or malignant. According to the knowledge by the inventors, the essence of malignant melanoma is morphological disorder including color tone, and therefore, the determination device 1 calculates an indicator value considering the diversity of color tones in the region of interest, and thereby can calculate an indicator value suitable for the determination of the melanonychia 102.

[0054] In addition, the determination device 1 can suppress the influence on the indicator value due to differences in imaging devices, persons who capture images, imaging environments, and the like by the conversion unit 22 adjusting the color balance. The indicator value calculated by the determination device 1 has high robustness with respect to input images and enables objective evaluation of the melanonychia 102.

[0055] For the determination device 1 of the present embodiment described above, for example, a general-purpose computer system is used which includes a central processing unit (CPU, processor) 901, a memory 902, a storage 903 (HDD: hard disk drive, SSD: solid state drive), a communication device 904, an input device 905, and an output device 906. In this computer system, each function of the determination device 1 is realized by the CPU 901 executing a program loaded into the memory 902.

[0056] The determination device 1 may be implemented by one computer or a plurality of computers. The determination device 1 may be a virtual machine implemented on a computer.

[0057] The program of the determination device 1 may be stored on a computer-readable recording medium such as an HDD, an SSD, a Universal Serial Bus (USB) memory, a compact disc (CD), or a digital versatile disc (DVD), or may be distributed via a network.

[0058] It should be noted that the present invention is not limited to the above embodiment, and various modifications can be made within the scope of the invention.

REFERENCE SIGNS LIST

[0059] 1 Determination device [0060] 2 Dermoscope [0061] 11 Digital color image data [0062] 12 Indicator value [0063] 21 Acquisition unit [0064] 22 Conversion unit [0065] 23 Calculation unit [0066] 24 Output unit [0067] 901 CPU [0068] 902 Memory [0069] 903 Storage [0070] 904 Communication device [0071] 905 Input device [0072] 906 Output device