PORTABLE SYSTEM FOR IDENTIFYING POTENTIAL CASES OF DIABETIC MACULAR OEDEMA USING IMAGE PROCESSING AND ARTIFICIAL INTELLIGENCE

Abstract

Diabetes is a disease characterized by high levels of blood glucose. It is important to keep diabetes under control to avoid short- and long-term complications. Diabetes can affect vision due to the alterations it produces in the blood vessels of the retina. This is known as Diabetic Retinopathy (DR), which is one of the leading causes of impaired vision in developed countries. One of the complications of diabetic retinopathy is Diabetic Macular Edema (DME), which is the leading cause of vision loss in diabetic patients and can appear at any stage of diabetic retinopathy. This consists of the gradual accumulation of fluid in the macula, the most important area of the retina. The determination of diabetic macular oedema is very important for the retina. The determination of diabetic macular oedema is very important for adequate treatment of this condition. A variety of technological options are used for detecting diabetic retinopathy, although only the most sophisticated detect macular oedema, a complication that appears as a consequence of diabetic retinopathy and is one of the leading causes of blindness. The invention describes a portable system for detecting diabetic macular oedema by capturing a fundus image using a portable ophthalmoscope; said image is sent via wired or wireless means to an embedded system that has an algorithm based on artificial intelligence, which extracts information from the image and processes same to identify the presence of the condition being studied.

Claims

1. A portable system for detecting potential cases of diabetic macular oedema using image processing, characterized in that it comprises: a. For the capturing of the image, a portable ophthalmoscopy device is required that has at least the following technical characteristics: i. A continuous field angle of 45° to 30°+35 D . . . −35 D, ii. A 40 mm lens (patient's eye—objective), at least 4 IR LEDs (each max. 100 mW), iii. A 3.97″ TFT LCD touch screen, 800×480 pixels, 16.7 million colors, anti-reflective film, iv. USB connection and/or Bluetooth-type wireless connection, v. Capture button and power button; this equipment captures the image that is sent to the system for processing and analysis, b. A device with a development card is required in order to process the image, which includes: i. An electrical connector, ii. An LCD screen, iii. An LED power indicator, iv. A power button, v. A wireless Bluetooth-, infrared-, or Zigbee-type module for wireless data reception and one or more USB and/or micro-USB ports for wired data transmission, vi. A microprocessor with at least 1 GB of RAM at 1.4 GHz and a 64-bit processor, c. For the image processing, an algorithm for image processing for the pre-diagnosis of diabetic macular oedema comprises the following steps: i. Reading of the image by wired or wireless means, ii. Conversion of the image to grayscale, iii. Automatic selection of a region through detection of characteristics corresponding to vascular intersections from which the image is cropped; the automatic detection of vascular intersections is performed using a search algorithm based on changes in intensity gradients in the image components and an error minimization process, for which the following steps are performed: 1. Decomposition, scaling, and skeletonization of image components, 2. Search for changes in intensity through gradient changes, and morphological search for branches, 3. Filling the vector of characteristics, 4. Minimization of error, 5. Counting of intersection points iv. Binarization of the image, v. A decision is made as to whether the noise has been eliminated, vi. Removal of noise in the image through morphological operations and automatic identification of objects referred to as noise (objects with an area smaller than 200 pixels), vii. Skeletonization of the noise-free image through morphological operations, identification of vascular intersections and measurement of the distance between them, viii. Rigid body transformation consisting of image scaling, rotation, and translation, ix. Image feature segmentation and extraction, x. Application of the artificial intelligence system: 1. Extraction of statistical characteristics, thereby obtaining: energy, entropy, homogeneity, contrast, brightness, correlation, average RGB, average R, average G, and average B of the image, 2. Classifier that makes use of a neural network, first estimating the thickness of the macula and then classifying the result obtained, 3. Pre-diagnosis of diabetic macular oedema, which is carried out after the estimation of the macular thickness.

2. The portable system for detecting potential cases of diabetic macular oedema by processing according to claim 1, wherein the developed system allows potential cases of diabetic macular oedema to be identified, characterized in that it does not require an optical coherence tomography study to be performed and does not require information to be sent to a diagnostic center and/or any specialist for review, since it can be used by professionals with training in optometry or first-contact health professionals who do not have specialized training in ophthalmology.

3. A method for the portable system for detecting potential cases of diabetic macular oedema, wherein the general steps for the operation of the system include: a. The user takes the portable ophthalmoscopy device and turns it on, b. The portable ophthalmoscopy device is positioned on the patient and captures the fundus image; the image must have a resolution of at least 5 MP, with a size of 1536×1152, in JPG, TIFF format, etc., c. The user sends the fundus image to the embedded system by wired and wireless means; there is also the possibility of storing the image in a USB memory to be inserted into the port having the embedded diagnostic system, d. The embedded system is achieved through image processing and extracts the characteristics thereof for classification, e. Finally, the system displays the information on the presence/absence of the pathology on an LCD screen.

Description

BRIEF DESCRIPTION OF THE FIGURES

[0036] FIG. 1 shows the portable ophthalmoscopy device of the system for identifying potential cases of diabetic macular oedema.

[0037] FIG. 2 shows the device with the development card and the embedded system developed for image processing.

[0038] FIG. 3 shows a schematic diagram of the system for receiving the images and processing them.

[0039] FIG. 4 shows the block diagram of the image processing algorithm that is embedded in the system for receiving the images and processing them.

[0040] FIG. 5 shows the general steps of the operation of the portable ophthalmoscopy system of the system for identifying potential cases of diabetic macular oedema.

[0041] FIG. 6 shows a non-invasive clinical test with the portable ophthalmoscopy system of the system for identifying potential cases of diabetic macular oedema.

[0042] FIG. 7 shows the images used for the baseline of each patient in their study that includes the image of the Optical Coherence Tomograph (OCT), the retinograph and the system that includes the portable ophthalmoscopy device, and a smartphone with the developed algorithm.

[0043] FIG. 8 shows the ROI and ocular vessel intersection points for I.sub.oct in an Optical Coherence Tomograph (OCT) image.

DETAILED DESCRIPTION OF THE INVENTION

[0044] The characteristic details of the portable system for identifying potential cases of diabetic macular oedema by means of image processing and artificial intelligence are clearly elucidated in the following description and in the accompanying illustrative drawings, with same reference signs serving to denote the same parts.

System Components.

[0045] The device has the following important components: [0046] Portable ophthalmoscopy device (101) having at least the following technical characteristics is required in order to capture the image: a continuous field angle of 45° to 30°+35 D . . . −35 D. A 40 mm lens (102) (patient's eye—objective). At least 4 IR LEDs (each max. 100 mW). A 3.97″ TFT LCD touch screen (106), 800×480 pixels, 16.7 million colors, anti-reflective film. A USB connection (104) and wireless connection, capture button (103) and power button (105), which can be equipment similar to the VISUSCOUT100 from the Zeiss company. As shown in FIG. 1, this equipment captures the image that is sent to the embedded system for processing and analysis. [0047] Device with the development card and the embedded system referred to as the image processing device (201) as shown in FIG. 2, which includes an electrical connector (202), an LCD screen (203), a power indicator LED (207), a power button (204), a USB port (205), and a micro-USB port (206) for wired data transmission. [0048] A device with a development card and the embedded system for image processing which is integrated into the image processing device (201) as shown in FIG. 3, which includes the LCD screen (203) and the display port (209), electrical connector (202), wireless module (Bluetooth) (208) for wireless data reception and USB (205) and micro-USB (206) ports for wired data reception, an LED power indicator (207), and the 1.4 GHz, 64-bit processor microprocessor (210) with 1 GB of RAM. [0049] An algorithm for image processing for the pre-diagnosis of diabetic macular oedema shown in FIG. 4 and consisting of the following steps: [0050] Reading of the image (401) [0051] Conversion of the image to grayscale (402) [0052] Let us consider a digital RGB fundus image represented by I.sub.eye(m; n), for demonstration purposes the iExaminer image, where I.sub.eye is a M×N image for m=1, 2, . . . , M and n=1, 2, . . . N, which can be decomposed into color components R.sub.m, n, G.sub.m, n and B.sub.m, n. For each component, P.sub.m, n is a specific pixel that can take a value from 0 to L, where L represents the maximum intensity value; for example, for an 8-bit resolution image, L=255. [0053] Automatic selection of a region through detection of characteristics corresponding to vascular intersections from which the image is cropped (403). Automatic detection of vascular intersections is performed using a search algorithm based on changes in intensity gradients in the image components and an error minimization process, for which the following steps are performed: [0054] Decomposition, scaling, and skeletonization of image components, [0055] Search for changes in intensity through gradient changes, and morphological search for branches, [0056] Filling the vector of characteristics, [0057] Minimization of error, [0058] Counting of intersection points and definition of I.sub.eye, [0059] Once I.sub.eye has been defined, the procedure applied is as follows: [0060] A cropping operation is applied to I.sub.eye in order to generate an image I.sub.roi (i; j), where I.sub.roi⊂I.sub.eye for α≤i≤M and β≤i≤

[00001]$l_{\mathrm{gs}}\left(i,j\right)=\frac{\stackrel{3}{\underset{s=1}{.Math.}}\stackrel{3}{\underset{t=1}{.Math.}}{K}_s\left(s,t\right)l_{\mathrm{rg}}\left(i+s,j+t\right)}{\stackrel{3}{\underset{s=-1}{.Math.}}\stackrel{3}{\underset{t=-1}{.Math.}}{K}_s\left(s.t\right)}$  N, α≥1, β≥1, α, β∈. As a necessary condition, I.sub.roi must contain two intersections of ocular vessels, p.sub.1(x.sub.p1, y.sub.p1) and p.sub.2(x.sub.p2, y.sub.p2), as shown by the yellow box in FIG. 8. [0061] Applying a media filter (404)

I.sub.gm=√{square root over ((K.sub.Gx(I.sub.gs)).sup.2+(K.sub.Gy(I.sub.gs)).sup.2)} [0062] I.sub.ori(i, j) becomes I.sub.mg(i, j), a grayscale representation, [0063] where p.sub.x, y∈[0, L] for reducing the data set. Considering K as a generic 3×3 kernel: [0064] An averaged kernel (K.sub.s) is applied over I.sub.rg in order to reduce the noise using:

[00002]$K=\left[\begin{array}{ccc}{k}_{11}& k_{12}& k_{13}\\ k_{21}& k_{22}& k_{23}\\ k_{31}& k_{32}& k_{33}\end{array}\right]K_s=\frac{1}{9}\left[\begin{array}{ccc}1& 1& 1\\ 1& 1& 1\\ 1& 1& 1\end{array}\right]$ [0065] where I.sub.gs is the averaged image, k.sub.22 is the center of the kernel. [0066] Binarization of the image (405) [0067] During image processing, the gradient is applied using directional kernels, considering K.sub.GX and K.sub.GY as directionals of 3×3 kernels, and the magnitude of the image gradient is obtained using: [0068] I.sub.gm makes it possible to identify the local variation in the illumination and the direction of the changes of maximum intensity of gray for I.sub.gs. [0069] A decision is made as to whether the noise has been eliminated (406) [0070] I.sub.gm is binarized using a dynamic threshold base using Otsu's method for the image histogram in order to generate I.sub.bw. [0071] I.sub.fn is the resulting noise-free image. [0072] Removal of noise in the image through morphological operations and automatic identification of objects referred to as noise (objects with an area smaller than 200 pixels) (407) [0073] Skeletonization of the noise-free image through morphological operations (408), identification of vascular intersections and measurement of the distance between them. [0074] I.sub.fn is skeletonized using mathematical morphological operations. This process eliminates pixels at the boundaries of objects, but does not allow the objects to separate; this process is based on the preservation of the Euler number; in addition, the algorithm stores spatial coordinates of the intersections. [0075] The final image is I.sub.sk [0076] Rigid body transformation consisting of image scaling, rotation, and translation (409) [0077] Image feature segmentation and extraction (410) [0078] Application of the artificial intelligence system: [0079] Extraction of statistical characteristics (411), thereby obtaining: energy, entropy, homogeneity, contrast, brightness, correlation, average RGB, average R, average G, and average B of the image [0080] Classifier that makes use of an artificial neural network (412), first estimating the thickness of the macula and then classifying the result obtained [0081] Pre-diagnosis of diabetic macular oedema (413), which is carried out after the estimation of the macular thickness.

System Features.

[0082] The features of the developed system are as follows: [0083] The developed system allows potential cases of diabetic macular oedema to be identified without having to perform an optical coherence tomography (OCT) study, which is the standard used worldwide for the detection of this pathology. [0084] In turn, the system takes the processing into consideration in the embedded system without the need to send the information to a diagnostic center and/or specialists for review, since it can be used by professionals with training in optometry or first-contact health professionals who do not have specialized training in ophthalmology.

System Operation.

[0085] The general steps for the operation of the portable ophthalmoscopy device of the system for identifying potential cases of diabetic macular oedema are listed below: [0086] 1. The user takes the portable ophthalmoscopy device (101) and turns it on (501). [0087] 2. The portable ophthalmoscopy device (101) is positioned on the patient (502) and captures the fundus image (503). The image must have at least a 5 MP resolution, with a size of 1536×1152, in JPG, TIFF format, etc. [0088] 3. The user sends the fundus image to the embedded system by wired and wireless means (504). There is also the possibility of storing the image in a USB memory to be inserted into the port having the embedded diagnostic system. [0089] 4. The embedded system is achieved through image processing and extracts the characteristics thereof for classification (505). [0090] 5. Finally, the system displays the information on the presence/absence of the pathology on an LCD screen (506).

[0091] Finally, the following technical considerations had to be made for the development of the system: [0092] In generating a capture protocol for the images used in the system, it was sought to establish that the images to be used should have the macular structure centered and be free of reflections and capture the presence of the ocular vascular arch, [0093] It was necessary to analyze/verify whether the characteristics of the images influence the characteristics that it is possible to extract from them. A comparative analysis was carried out between different image compression formats, resolutions, and sizes required by the system in order to extract the characteristics, [0094] In the development of the system, the main problem that had to be solved was the validation of the response of the intelligent system. To do this, non-invasive studies were carried out on patients to verify the relationship between the information present in an image and the response offered by the gold standard for the identification of this pathology: optical coherence tomography.

System Verification.

[0095] In order to ensure the precision of the system, a non-invasive clinical test was carried out in which the fundus images of 76 patients were obtained as shown in FIG. 6, with which the artificial intelligence system was adjusted. The procedures followed were in accordance with the Declaration of Helsinki of 1964, revised in 2004. The complete test was performed on 76 patients (40 without diabetic macular oedema, 10 with diabetic macular oedema, and 26 for a blind test). The inclusion criteria were: [0096] Age from 20 to 60 years, suitable for pupil dilation, with or without diagnosis of diabetic macular oedema, and the willingness to provide a signed informed consent, [0097] The exclusion criteria were patients who were not found suitable for pupil dilation for medical or social reasons, [0098] Pupillary dilation less than 7 mm (clinical pupillary dilation), [0099] History of previous laser therapy or subsequent segment surgery and presence of significant signs of opacity.

[0100] Both eyes were dilated with a topical solution of 1.0% tropicamide and 2.5% phenylephrine. After twenty minutes, the pupils were examined to determine whether the dilation was adequate for the study. Then the Optical Coherence Tomography (OCT) images and retinography are captured. Each acquisition takes 5 minutes. In order to avoid inter-observer and intra-observer variation, only two trained individuals were assigned to perform the tests. These two technicians were trained by expert retinologists from the Centro de Retina Médica y Quirúrgica (CRMQ) during four two-hour sessions to perform ophthalmoscopy and learn the technique of capturing images of the fundus of the eye. Once a full study has been completed at the CRMQ, the database is composed of three images for each patient, as is shown in FIG. 7, this figure including the following: [0101] A) OCT Zeiss Cirrus HD-5000, [0102] B) Zeiss FF450 fundus camera, [0103] C) Portable ophthalmoscopy equipment and a smartphone with the developed algorithm, [0104] D) OCT image, [0105] E) Image from the retinograph, [0106] F) Image from the portable ophthalmoscopy exposure LED device and a smartphone with the developed algorithm.

[0107] The preceding description of the disclosed definitions is provided in order to enable any person skilled in the art to implement or use the present invention. Various modifications to these definitions and/or implementations will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but should be granted the broadest scope consistent with the following claims and the principles and novel features disclosed herein.