Atlas for automatic segmentation of retina layers from OCT images
11481905 · 2022-10-25
Inventors
- Ayman S. El-Baz (Louisville, KY, US)
- Ahmed Soliman (Louisville, KY, US)
- Ahmed Eltanboly (Louisville, KY, US)
- Ahmed Sleman (Louisville, KY, US)
- Robert S. Keynton (Goshen, KY, US)
- Harpal Sandhu (Louisville, KY, US)
- Andrew Switala (Louisville, KY, US)
Cpc classification
G06V10/75
PHYSICS
G06T2207/10101
PHYSICS
G06T7/143
PHYSICS
G06V10/50
PHYSICS
International classification
Abstract
A method for segmentation of a 3-D medical image uses an adaptive patient-specific atlas and an appearance model for 3-D Optical Coherence Tomography (OCT) data. For segmentation of a medical image of a retina, In order to reconstruct the 3-D patient-specific retinal atlas, a 2-D slice of the 3-D image containing the macula mid-area is segmented first. A 2-D shape prior is built using a series of co-aligned training OCT images. The shape prior is then adapted to the first order appearance and second order spatial interaction MGRF model of the image data to be segmented. Once the macula mid-area is segmented into separate retinal layers this initial slice, the segmented layers' labels and their appearances are used to segment the adjacent slices. This step is iterated until the complete 3-D medical image is segmented.
Claims
1. A method for segmenting a medical image comprising: receiving a volumetric medical image comprising a plurality of slices, each slice being adjacent to at least one other slice in the image; selecting an initial slice; segmenting the initial slice based at least in part on a constructed shape model; applying a label to each segmented layer in the initial slice; and segmenting at least one slice adjacent to the initial slice based at least in part on the segmented initial slice wherein segmenting the at least one slice adjacent to the initial slice based at least in part on the segmented initial slice includes, for a pixel in the at least one slice, transforming the pixel to the initial slice, initializing a window, searching within the window for pixels with a corresponding value in the initial slice, and calculating a shape prior probability based on the labels of found pixels with corresponding values, and labeling the pixel in the at least one slice based on the shape prior probability.
2. The method of claim 1, wherein the medical images are retinal images.
3. The method of claim 1, wherein the medical images are optical coherence tomography images.
4. The method of claim 1, wherein the initial slice has two adjacent slices.
5. The method of claim 1, wherein the volumetric medical image is a 3-D medical image and wherein the plurality of slices are a plurality of 2-D medical images.
6. The method of claim 1, wherein segmenting the initial slice includes aligning the initial slice to the constructed shape model.
7. The method of claim 6, wherein segmenting the initial slice further includes applying a joint model to the initial slice subsequent to alignment.
8. The method of claim 1, wherein segmenting the at least one slice adjacent to the initial slice includes aligning the at least one slice to the initial slice.
9. The method of claim 1, wherein each segmented layer corresponds to a retinal layer.
10. The method of claim 1, wherein the pixel in the at least one slice has a reflectivity value and wherein searching within the window for pixels with the corresponding value in the initial slice comprises searching within the window for pixels with corresponding reflectivity values in the initial slice.
11. A method for segmenting a 3-D medical image comprising: receiving a 3-D medical image, the 3-D medical image comprising an array of adjacent 2-D medical images; segmenting an initial 2-D medical image based at least in part on a constructed shape model; and segmenting a 2-D medical image adjacent to the segmented initial 2-D medical image based at least in part on the segmented initial 2-D medical image wherein segmenting the 2-D medical image based at least in part on the previously segmented 2-D medical image includes, for a pixel in the 2-D medical image, determining a value for the pixel, transforming the pixel to the previously segmented 2-D medical image, initializing a window, searching within the window for pixels with a corresponding value in the previously segmented 2-D medical image, calculating a shape prior probability based on labels of found pixels with corresponding values, and labeling the pixel in the 2-D medical image based on the shape prior probability.
12. The method of claim 11, further comprising, after the step of segmenting the 2-D medical image, repeating the step of segmenting the 2-D medical image until all 2-D medical images in the array are segmented.
13. The method of claim 11, wherein the 3-D medical image is a retinal image depicting at least a fovea, and wherein the initial 2-D medical image extends through the fovea.
14. The method of claim 11, wherein segmenting the initial 2-D medical image includes aligning the initial 2-D medical image to the constructed shape model and applying a joint model to the initial 2-D medical image subsequent to alignment.
15. The method of claim 11, wherein the initial 2-D medical image depicts an anatomical feature and wherein the constructed shape model is constructed from a database of images of the anatomical feature.
16. The method of claim 15, wherein the anatomical feature is a fovea.
17. The method of claim 11, wherein the value for the pixel is a reflectivity value for the pixel.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) A better understanding of the present invention will be had upon reference to the following description in conjunction with the accompanying drawings.
(2)
(3)
(4)
(5)
(6)
(7)
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
(8) Referring now to
(9) A. Joint MGRF Based Macula-Centered Foveal Image Segmentation
(10) Let g={g(x):x∈R; g(x)∈Q} and m={l(x):x∈R; l(x)∈L} be a grayscale image taking values from Q, i.e., g:R.fwdarw.Q, with the associated region map taking values from L, i.e., m:R.fwdarw.L, respectively. R denotes a finite arithmetic lattice, Q is a finite set of integer gray values, and L is a set of region labels. An input OCT image, g, co-aligned to the training database, and its map, m, are described with a joint probability model:
P(g,m)=P(g|m)P(m)
which combines a conditional distribution of the images given the map P(g|m), and an unconditional probability distribution of maps P(m)=P.sub.sp(m)P.sub.v(m). Here, P.sub.sp(m) denotes a weighted shape prior, and P.sub.v(m) is a Gibbs probability distribution with potentials V, that specifies a MGRF model of spatially homogeneous maps m.
(11) (1) Shape Model P.sub.sp(m): In order to account for the inhomogeneity of the OCT images, the shape information is incorporated in the segmentation process. The shape model is constructed using OCT scans selected in such a way as to be representative and to capture the biological variability of the whole data set. “Ground truth” segmentations of these scans were delineated under supervision of retina specialists. Using one of the optimal scans as a reference (no tilt, centrally located fovea), the others were co-registered using a thin plate spline (TPS). The shape prior was defined as:
P.sub.sp(m)=n p.sub.sp:y(l)
y∈R
Where, p.sub.sp:y(l) is the pixel-wise probability for label l, and y is the image pixel with gray level g. The same deformations were applied to their respective ground truth segmentations, which were then averaged to produce a probabilistic shape prior of the typical retina, i.e., each location x in the reference space is assigned a prior probability P(m) to lie within each of the 12 layers' classes. The same deformations were applied to their respective ground truth segmentations, then averaged to produce a probabilistic shape prior of the typical retina. The input medical image intended to be segmented (i.e., the initial slice) is first aligned to the shape database. The used alignment approach integrates TPS with the multi-resolution edge tracking method that identifies control points for initializing the alignment process as shown in
(12) (2) Adaptive Model: In order to make the segmentation process adaptive and not biased to only the shape information, a 1st-order intensity model P(g|m) was used for the empirical gray level distribution of the OCT images. The visual appearance of each label of the image is modeled by separating a mixed distribution of pixel intensities into individual components associated with the dominant modes of the mixture. The modes are identified using the LCDG algorithm, which employs positive and negative Gaussian components that is based on a modified version of the classical Expectation Maximization (EM) algorithm. Then, a 2nd-order MGRF model P.sub.v(m) is used to improve the spatial homogeneity of the segmentation. This model was identified using the nearest pixels' 8-neighborhood and analytical bi-valued Gibbs potentials V that depend on the equality of the nearest pair of labels. This MGRF Potts model that accounts for spatial information was incorporated with the shape and intensity information as explained in Section A.
(13) B. Layers Segmentation 3-D Propagation
(14) Referring now to
(15) Referring now to
(16) Algorithm 1: Steps of the Shape Prior Segmentation.
(17) 1) Segment the retinal layers in the midline slice following the procedures in Section A.
(18) 2) For each slice i, i=1 to n I. Use non-rigid registration to align the gray image for the current slice (slice i) with the preceding/succeeding slice (based on the direction) to obtain the deformation fields. II. For each pixel v in slice i (a) Transform v to the neighboring slice domain using the obtained deformation field. (b) Initialize a 2-D window, w, of size N.sub.1i×N.sub.2i centered around the mapped voxel (v.sub.mapped). (c) Search w for pixels with corresponding reflectivity value in the neighboring slice where reflectivity falls within a predefined tolerance ±τ in w. (d) If no pixels are found using Step (c), increase the size of w and repeat step (c) until correspondences are found or the maximum size allowed for w is reached. (e) Calculate the shape probability for each retinal layer at location r based on the found voxels and their labels. End for III. Assign pixel v the label with the highest probability.
(19) End for
(20) For clarification, please note that “slice i” refers to the initial slice in
(21) EXPERIMENTAL RESULTS: Ten 3-D OCT scans of 10 subjects were used to test the accuracy of the disclosed method of retinal layer segmentation and validate the method against manual segmentation. The OCT scans of the subjects were collected using Zeuss Cirrus HD-OCT 5000 with 5 OCT scans per each OCT volume. Subjects' range of age is from 32 to 50 years (mean±SD, 40±10.2 years). Retina expert specialists manually segmented the scans of retinal layers to construct a ground truth segmentation.
(22)
(23) Step 1: Segmentation using joint-MGRF model to obtain the macula mid-slice (slice i, i.e., slice 3 out of 5).
(24) Step 2: 3-D Segmentation propagation using slice i as a patient-specific atlas to segment both slices i−1 and i+1 (i.e., slices 2 and 4).
(25) Step 3: 3-D Segmentation propagation by using slice i−1 (i.e., slice 2) as a patient-specific atlas to segment slice i−2 (i.e., slice 1) and using slice i+1 (i.e., slice 4) as a patient-specific atlas to segment slice i+2 (i.e., slice 5).
(26) As should be readily understood, this process is expandable to any number of slices following the pattern of using a segmented slice as a patient-specific atlas for segmentation of the adjacent slice in the following step. The instant segmentation approach has been found to be reliable for segmentation of challenging diseased cases such as age-related macular degeneration (AMD) and diabetic retinopathy where the layers' anatomy is distorted.
(27) To evaluate the accuracy and robustness of the proposed approach, four commonly used evaluation metrics were used to compare the disclosed segmentation approach with the ground truth and prior work. These metrics are listed below:
(28) (1) Dice Similarity Coefficient (DSC) which is a measure of the ratio of shared segmentation between two images. It can be defined as follows:
(29)
where R and W are the two segmentations to be compared. The higher the DSC value is, the more similar both segmentations are.
(30) (2) 95-percentile bidirectional modified Hausdorff Distance (HD) that measures the maximum distance between 2 matching points for 2 different segmentation techniques. The smaller the distance, the better the segmentation. For two sets of boundary points (X, Y), HD is defined as:
(31)
where the 5% largest distances are removed, then the maximum of HD(X, Y) and HD(Y, X) is determined for each image.
(32) (3) Unsigned Mean Surface Position Error (MSPE) that measures the distance between boundaries in two different segmentations at each point across the images.
(33) (4) Average Volume Difference (AVD) which measures the volume difference between obtained segmentation and the ground truth. A smaller volume difference indicates a better segmentation.
(34) In order to highlight the advantage of the approach disclosed herein, it is compared with a well-established 3-D automated segmentation approach as disclosed in Li, K., et al., “Optimal surface segmentation in volumetric images—a graph-theoretic approach,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 119-134, 2006. However, the approach in Li et al. can only segment 11 of the 12 retinal layers. In order to have a fair comparison, 6th and 7th layers in the 12 retinal layers segmented using the instant approach are merged to make the number of layers equivalent in both techniques.
(35)
(36) TABLE-US-00001 TABLE 1 Comparative segmentation accuracy between this proposed approach and Li et al., evaluated by DSC, HD, MSPE, and AVD. DSC (%) HD (voxels) MSPE (voxels) AVD (%) Metric Pro- Li et Pro- Li et Method Proposed Li et al. Proposed Li et al. posed al. posed al. Layer 82.51 ± 76.32 ± 3.81 ± 8.02 ± 0.16 ± 1.44 ± 15.28 ± 41.17 ± 1 2.91 5.96 0.90 5.54 0.11 0.79 9.73 18.47 Layer 82.81 ± 77.23 ± 5.41 ± 5.65 ± 0.32 ± 0.72 ± 8.98 ± 6.95 ± 2 2.53 4.34 3.40 3.71 0.15 0.50 3.97 4.19 Layer 80.36 ± 77.94 ± 8.19 ± 5.02 ± 0.30 ± 0.75 ± 16.22 ± 6.15 ± 3 3.30 4.19 4.28 3.49 0.14 0.44 15.48 3.29 Layer 80.35 ± 77.80 ± 7.46 ± 4.02 ± 0.46 ± 0.50 ± 8.91 ± 14.39 ± 4 3.30 3.56 3.72 1.77 0.24 0.28 5.94 3.16 Layer 78.93 ± 76.77 ± 15.33 ± 5.92 ± 1.86 ± 1.24 ± 22.41 ± 35.64 ± 5 3.51 3.58 6.07 3.27 0.82 0.44 16.02 14.87 Layer 84.64 ± 81.77 ± 3.33 ± 2.06 ± 0.83 ± 0.23 ± 11.04 ± 5.28 ± 6 3.72 2.15 2.52 0.55 0.37 0.09 5.55 4.18 Layer 84.51 ± 80.89 ± 3.60 ± 6.62 ± 0.55 ± 1.85 ± 20.91 ± 31.60 ± 7 3.74 2.13 4.68 1.18 0.68 0.34 14.48 3.34 Layer 84.42 ± 80.04 ± 3.25 ± 7.34 ± 0.26 ± 2.57 ± 20.25 ± 24.48 ± 8 3.71 2.22 2.65 0.96 0.17 0.58 10.51 14.24 Layer 84.10 ± 79.49 ± 4.20 ± 7.95 ± 1.02 ± 2.48 ± 56.73 ± 36.89 ± 9 4.00 2.26 2.42 1.77 0.78 0.93 63.79 5.03 Layer 83.70 ± 79.08 ± 4.79 ± 14.28 ± 0.94 ± 1.17 ± 26.61 ± 73.24 ± 10 4.11 15.51 1.59 5.59 0.63 0.40 14.44 10.74 Layer 83.31 ± 78.08 ± 3.15 ± 13.13 ± 1.29 ± 2.27 ± 15.92 ± 41.80 ± 11 4.06 15.07 0.29 4.93 1.99 1.27 12.48 15.75 Aver- 82.69 ± 78.67 ± 5.68 ± 7.27 ± 0.73 ± 1.38 ± 20.30 ± 28.87 ± age 3.37 4.72 1.90 2.01 0.21 0.35 11.94 2.93
(37) The above results indicate overall superior performance of the instant method in terms of DSC, HD, MSPE, and AVD as compared to the earlier approach disclosed in Li et al. Using paired t-test for statistical analysis shows a significant advantage of this approach over the earlier approach in terms of all metrics as confirmed by p-values <0.05.
(38) Various aspects of different embodiments of the present disclosure are expressed in paragraphs X1 and X2 as follows:
(39) X1: One embodiment of the present disclosure includes a method for segmenting a medical image comprising: receiving a volumetric medical image comprising a plurality of slices, each slice being adjacent to at least one other slice in the image; selecting an initial slice; segmenting the initial slice based at least in part on a constructed shape model; and segmenting at least one slice adjacent to the initial slice based at least in part on the segmented initial slice.
(40) X2: Another embodiment of the present disclosure includes a method for segmenting a 3-D medical image comprising: receiving a 3-D medical image, the 3-D medical image comprising an array of adjacent 2-D medical images; segmenting an initial 2-D medical image based at least in part on a constructed shape model; and segmenting a 2-D medical image adjacent to the previously segmented 2-D medical image based at least in part on the previously segmented 2-D medical image.
(41) Yet other embodiments include the features described in any of the previous paragraphs X1 or X2 as combined with one or more of the following aspects:
(42) Wherein the medical images are retinal images.
(43) Wherein the medical images are optical coherence tomography images.
(44) Wherein the initial slice has two adjacent slices.
(45) Wherein the volumetric medical image is a 3-D medical image and wherein the plurality of slices are a plurality of 2-D medical images.
(46) Wherein segmenting the initial slice includes aligning the initial slice to the constructed shape model.
(47) Wherein segmenting the initial slice further includes applying a joint model to the initial slice subsequent to alignment.
(48) Wherein segmenting the at least one slice adjacent to the initial slice includes aligning the at least one slice to the initial slice.
(49) Further comprising, after segmenting the initial slice, applying a label to each segmented layer in the initial slice.
(50) Wherein each segmented layer corresponds to a retinal layer.
(51) Wherein segmenting the at least one slice adjacent to the initial slice based at least in part on the segmented initial slice includes, for a pixel in the at least one slice, transforming the pixel to the initial slice, initializing a window, searching within the window for pixels with a corresponding value in the initial slice, and calculating a shape prior probability based on the labels of found pixels with corresponding values, and labeling the pixel in the at least one slice based on the shape prior probability.
(52) Wherein the pixel in the at least one slice has a reflectivity value and wherein searching within the window for pixels with the corresponding value in the initial slice comprises searching within the window for pixels with corresponding reflectivity values in the initial slice.
(53) Further comprising, after the step of segmenting the 2-D medical image, repeating the prior step until all 2-D medical images in the array are segmented.
(54) Wherein the 3-D medical image is a retinal image depicting at least a fovea, and wherein the initial 2-D medical image extends through the fovea.
(55) Wherein segmenting the initial 2-D medical image includes aligning the initial 2-D medical image to the constructed shape model and applying a joint model to the initial 2-D medical image subsequent to alignment.
(56) Wherein the initial 2-D medical image depicts an anatomical feature and wherein the constructed shape model is constructed from a database of images of the anatomical feature.
(57) Wherein the anatomical feature is a fovea.
(58) Wherein segmenting the 2-D medical image based at least in part on the previously segmented 2-D medical image includes, for a pixel in the 2-D medical image, determining a value for the pixel, transforming the pixel to the previously segmented 2-D medical image, initializing a window, searching within the window for pixels with a corresponding value in the previously segmented 2-D medical image, calculating a shape prior probability based on labels of found pixels with corresponding values, and labeling the pixel in the 2-D medical image based on the shape prior probability.
(59) Wherein the value for the pixel is a reflectivity value for the pixel.
(60) The foregoing detailed description is given primarily for clearness of understanding and no unnecessary limitations are to be understood therefrom for modifications can be made by those skilled in the art upon reading this disclosure and may be made without departing from the spirit of the invention. While the disclosed invention has been described primarily in connection with the segmentation of 3-D OCT retinal images, it should be understood that the disclosed segmentation techniques may be usable with segmentation of 3-D medical images obtained using different imaging modalities or depicting different anatomical features.