Adaptive enhancement method for image contrast based on level of detail

Abstract

A level of detail-transformation adaptive enhancement method for image contrast includes: dividing a remote sensing image into a plurality images of different levels of detail, the lowest level of detail defined as L and the highest level of detail defined as H, and gradually transforming an image Image.sub.i of an arbitrary level of detail i between the image Image.sub.H of the highest level of detail H and the image Image.sub.L of the lowest level of detail L from Image.sub.L to Image.sub.H through the following equation: Image.sub.i=R.sub.i×Image.sub.H+(1−R.sub.i)×Image.sub.L. The image Image.sub.H of the highest level of detail H is an image Image.sub.ACE produced with adaptive contrast enhancement processing, or an image produced with a contrast enhancement method such as Gaussian or histogram equalization; the image Image.sub.L of the lowest level of detail L is an image Image.sub.LCE produced by common linear contrast enhancement.

Claims

1. A level of detail-transformation adaptive enhancement method for image contrast, comprising: dividing a remote sensing image into a plurality images of different levels of detail, the lowest level of detail in the plural levels of detail defined as L and the highest level of detail defined as H, and gradually transforming an image Image.sub.i of an arbitrary level of detail i between the image Image.sub.H of the highest level of detail H and the image Image.sub.L of the lowest level of detail L from Image.sub.L to Image.sub.H through the following equation: Image.sub.i=R.sub.i×Image.sub.H+(1−R.sub.i)×Image.sub.L, wherein R.sub.i is the weight of Image.sub.H, wherein when the image Image.sub.H of the highest level of detail H is an image Image.sub.ACE produced by adaptive contrast enhancement processing, and the image Image.sub.L of the lowest level of detail L is an image Image.sub.LCE produced by common linear contrast enhancement, the equation is rewritten as: Image.sub.i=R.sub.i×Image.sub.ACE+(1−R.sub.i)×Image.sub.LCE, wherein the image Image.sub.i of an arbitrary level of detail i is obtained by computing the equation Image.sub.i=R.sub.i×Image.sub.ACE+(1−R.sub.i)×Image.sub.LCE, and then is cut into tiles so that, when the image Image.sub.i of an arbitrary level is browsed on a network platform and switched between different levels of detail, additional partial image enhancement is dispensed with to increase image displaying efficiency of the network platform.

2. The method as claimed in claim 1, wherein the image Image.sub.H of the highest level of detail H is an image Image.sub.ACE produced with adaptive contrast enhancement processing, or an image produced with a contrast enhancement method selected from Gaussian or histogram equalization to reduce limitations imposed by cloud and fog on dynamic range of image contrast and to show Earth's surface features with optimized visual effects.

3. The method as claimed in claim 1, wherein the image Images of the lowest level of detail L is an image Image.sub.LCE produced by common linear contrast enhancement to avoid rather unreal tones as can be found when an entire image of a relatively low level of detail is viewed.

4. The method as claimed in claim 1, wherein the weight Ri of Image.sub.H is derived from linear transformation relation: Ri=(i−L)/(H−L).

5. The method as claimed in claim 1, wherein the weight R.sub.i of Image.sub.H is derived from power transformation relation: R.sub.i=[(i−L)/(H−L)].sup.n, and n is an arbitrary number greater than 0 and smaller than 1.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

(2) FIG. 1 is a flowchart illustrating an embodiment of the level of detail-transformation adaptive enhancement method for image contrast according to the present invention;

(3) FIGS. 2(a)-2(f) are pictures showing images received linear contrast enhancement and the same images received adaptive contrast enhancement;

(4) FIGS. 3(a)-3(f) are pictures showing images of different levels of detail received various contrast enhancement methods;

(5) FIGS. 4(a)-4(d) are pictures showing the result of LOD linear transformation-based adaptive image contrast enhancement;

(6) FIGS. 5(a)-5(f) are pictures showing the result of LOD linear transformation-based adaptive image contrast enhancement; and

(7) FIGS. 6(a)-6(c) are showing the result of LOD linear transformation-based adaptive image contrast enhancement.

DETAILED DESCRIPTION OF THE INVENTION

(8) The detailed description and preferred embodiments of the invention will be set forth in the following content, and provided for people skilled in the art to understand the characteristics of the invention.

(9) Please refer to FIG. 1 for the level of detail-transformation adaptive enhancement method for image contrast according to the present invention.

(10) Although adaptive contrast enhancement (ACE) technique has been employed, additional modification and integration are required for processing a large remote sensing imagery.

(11) Contrast enhancement is a widely-used technique of digital image processing for applications that the subjective quality of images is important for human interpretation. It is also a crucial processing to ensure the quality of visual effect for remote sensing optical images from acquisition to users. Although the general and straightforward approach of linear contrast enhancement (LCE) can provide a clear contrast for most of the land use and land cover (LULC), clouds and hazes that are frequently found on a remote sensing optical image inevitably limit the dynamic range of contrast and deteriorate the quality of visual effect. Unless they can be masked out completely, a small patch of cloud or haze would render the linear contrast enhancement approach invalid.

(12) As demonstrated previously by the present inventor, this problem can be tackled by masking out clouds and applying the adaptive contrast enhancement technique to those regions without cloud masking, and then adding back the cloud mask with a saturated value. The adaptive contrast enhancement technique assigns each pixel to an intensity proportional to its rank within the surrounding neighborhood. Even though the clouds/hazes cannot be masked out completely, the contrast can be stretched well by considering the rank of each pixel. The noise over-enhancement in nearly homogeneous regions can be reduced by setting the size of the overlapped tiles as a fraction of the largest dimensions of the image size, as well as limiting the slope of histogram. This method has proven to be broadly applicable to a wide range of images and to have demonstrated effectiveness.

(13) To begin with, step 91 is performed to divide a remote sensing image into a plurality images of different levels of detail and to define the lowest level of detail (lowest LOD) in the plural levels of detail as L and the highest level of detail (highest LOD) as H. The image Image.sub.i of an arbitrary level of detail i between the image Image.sub.H of the highest level of detail H and the image Image.sub.L of the lowest level of detail L is gradually transformed from Image.sub.L to Image.sub.H through the following equation: Image.sub.i=R.sub.i×Image.sub.H+(1−R.sub.i)×Image.sub.L.

(14) In the following step 92, Image.sub.H is subjected to adaptive contrast enhancement processing to produce an image Image.sub.ACE, or is processed with a contrast enhancement method such as Gaussian or histogram equalization, the goal being to reduce the limitations imposed by cloud and fog on the dynamic range of image contrast and to show the features of the Earth's surface with optimized visual effects. Image.sub.L, on the other hand, is subjected to common linear contrast enhancement to produce an image Image.sub.LCE. Here, common linear contrast enhancement is used instead of a much stronger contrast enhancement method because the latter tends to result in rather unreal tones as can be found when the entire image of a relatively low-LOD is viewed.

(15) When the image Image.sub.H of the highest-LOD H is the image Image.sub.ACE, which is produced by adaptive contrast enhancement processing, and the image Image.sub.L of the lowest-LOD L is the image Image.sub.LCE, which is produced by common linear contrast enhancement, the foregoing equation can be rewritten as: Image.sub.i=R.sub.i×Image.sub.ACE+(1−R.sub.i)×Image.sub.LCE.

(16) In that case, the image Image.sub.i of an arbitrary level of detail i is obtained by computing the equation Image.sub.i=R.sub.i×Image.sub.ACE+(1−R.sub.i)×Image.sub.LCE, and each resulting image is cut into tiles so that, when the processed image is browsed on a network platform and switched between different levels of detail, additional partial image enhancement can be dispensed with to increase the image displaying efficiency of the network platform.

(17) In the last step 93, image contrast enhancement is performed on each image of the to-be-processed level of detail in a progressive manner. The weight R.sub.i of Image.sub.H can be derived from the linear transformation relation R.sub.i=(i−L)/(H−L) (in which case the processing method is referred to as the LOD linear transformation-based adaptive image contrast enhancement method); or from the power transformation relation R.sub.i=[(i−L)/(H−L)].sup.n, where n is an arbitrary number greater than 0 and smaller than 1 (in which case the processing method is referred to as the LOD power transformation-based adaptive image contrast enhancement method); or from another non-linear relation in order to obtain a plurality images of different levels of detail with different degrees of enhancement (in which case the processing method is referred to as the LOD non-linear transformation-based adaptive image contrast enhancement method).

(18) FIG. 2(a) to FIG. 2(f) show a standard Level-1C product of a true natural-color image T51QTF taken by Sentinel-2 on Jun. 15, 2019. More specifically, FIG. 2(a) is a true natural-color image that has received linear contrast enhancement, and FIG. 2(b) is a true natural-color image that has received adaptive contrast enhancement. A comparison between these two images shows that FIG. 2(b) does have better contrast than FIG. 2(a) regardless of cloud coverage or distribution. Parameters that were required to be set for the adaptive contrast enhancement were so determined as to ensure that the relatively dark and uniform pixels of the water area in the red boxes in FIG. 2(a) and FIG. 2(b) or of the vegetation area in the yellow boxes in FIG. 2(a) and FIG. 2(b) would not produce noticeable color spots due to over-enhancement. By examining the two images at full resolution (the highest-LOD), e.g., by comparing FIG. 2(c) and FIG. 2(e), which are the result of linear contrast enhancement, with FIG. 2(d) and FIG. 2(f), which are the result of adaptive contrast enhancement, it can be known that adaptive contrast enhancement is indeed effective in improving visual effects. The side effect of unreal image tones, however, does show when the entire image of a relatively low-LOD of, e.g., FIG. 2(b) is viewed.

(19) As the remote sensing optical images are browsed through the web mapping service, its current level of detail and region are determined on-the-fly from the client end. Since the actions of zoom in and zoom out are equivalent to flip between tiles with different levels of detail, and only the related tiles fall within the current region are required to transmit and display, the remote sensing optical image can be browsed smoothly through the web, no matter how large its original size is. When the image is zooming across different levels of detail, the users are aware of the change of spatial resolution only, rather than the contrast. Because the entire set of pyramid images/tiles is cut/prepared from the same image, their contrasts are all kept the same. This motivates the idea of introducing different levels of enhancement for different levels of detail, namely LOD-based enhancement.

(20) The water area in the red boxes in FIG. 2(a) and FIG. 2(b) is presented in FIG. 3(a) to FIG. 3(f) to facilitate discussion on the enhancement effects of the levels of detail. The image on the left column of each of FIG. 3(a) to FIG. 3(f) is the result of common linear contrast enhancement, whereas the image in the middle column of each of FIG. 3(a) to FIG. 3(f) is the result of processing with the LOD linear transformation-based adaptive image contrast enhancement method. It is worth mentioning that the original image is not only a true natural-color standard Level-1C product, but also a typical optical satellite-remote-sensing image affected by cirrocumulus cloud and smog. The images on the left column of each of FIG. 3(a) to FIG. 3(f), although produced from a true natural-color standard Level-1C product provided directly by Sentinel Hub, demonstrate that the limited dynamic range of common linear contrast enhancement leads to poor visual effects. By contrast, the images in the middle column of each of FIG. 3(a) to FIG. 3(f), i.e., images processed by the LOD transformation-based adaptive image contrast enhancement method, show the water area with increasing clarity as the level of detail rises from FIG. 3(a) to FIG. 3(f). It is noteworthy that the weight of Image.sub.ACE in this example is determined by linear transformation, and that the contrast-enhanced visual effects of levels of detail 9 to levels of detail 11 are not as prominent as those of levels of detail 12 to levels of detail 14. This suggests that the weight of Image.sub.ACE needs to be adjusted for different levels of detail.

(21) To increase the weight of Image.sub.ACE for the relatively low-LOD and to reduce the weight of Image.sub.ACE for the relatively high-LOD, the equation for computing the weight of Image.sub.ACE is modified into power transformation: Ri=[(i−L)/(H−L)].sup.n, where n is an arbitrary number greater than 0 and smaller than 1.

(22) The images on the right column each of FIG. 3(a) to FIG. 3(f) are the result of processing the same true natural-color image with the LOD power transformation-based adaptive image contrast enhancement method, with n being 0.5. More specifically, the images on the right column of each of FIG. 3(a) to FIG. 3(f) are levels of detail 9 to levels of detail 14 respectively. By comparing the right column images (which are the result of power transformation) and the middle column images (which are the result of linear transformation) of FIG. 3(a) to FIG. 3(f), it can be seen that the contrast-enhanced visual effects of levels of detail 9 to levels of detail 11 are significantly improved. This shows that linear transformation is helpful to optical remote-sensing images with zero or low cloud cover percentage, and that power transformation is suitable for optical remote-sensing images that are affected by cirrocumulus and smog. In fact, other forms of transformation, such as logarithmic transformation and root-mean-square transformation, are also applicable, provided that the concept of LOD transformation-based adaptive image contrast enhancement is adhered to.

(23) The LOD-based enhancement approach is written in Interactive Data Language (IDL®) using some of Environment for Visualizing Images (ENVI®) library of built-in routines. It is currently installed on an ordinary PC-based server equipped with an Intel® Core™ i7-4790K (4.0-GHz) CPU (ASUS, Taipei, Taiwan), as well as a regular solid state disk. This new approach has been employed to assist the Soil and Water Conservation Bureau (SWCB) of Taiwan to process various sources of remote sensing optical imagery.

(24) FIG. 4(a) to FIG. 4(d) show a mosaic of cloudless images of Taiwan taken by SPOT-6/7 in 2017. The left-column and right-column images of each of FIG. 4(a) to FIG. 4(d) are respectively the original mosaic and the result of processing with the LOD linear transformation-based adaptive image contrast enhancement method. While the 2017 SPOT-6/7 cloudless mosaic already has quite satisfactory contrast, the effect of applying the LOD linear transformation-based adaptive image contrast enhancement method is still significant. The target area becomes clearer and clearer as the level of detail is gradually switched from level of detail 14 to level of detail 17, i.e., from the right-column image of FIG. 4(a) to that of FIG. 4(d). It can therefore be inferred that the LOD linear transformation-based adaptive image contrast enhancement method can produce noticeable effects on optical remote-sensing images with zero or low cloud cover percentage.

(25) FIG. 5(a) to FIG. 5(f) show images of Taiwan taken by Sentinel-2 on Sep. 3, 2018 (FIG. 5(a) to FIG. 5(c)) and Jan. 11, 2020 (FIG. 5(d) to FIG. 5(f)). The left-column and right-column images of each of FIG. 5(a) to FIG. (f) are respectively a screenshot of the Sentinel Explorer website (https://sentinel2explorer.esri.com/, as browsed on Jan. 16, 2020) and the result of processing with the LOD linear transformation-based adaptive image contrast enhancement method. As the level of detail is sequentially switched from level of detail 7 to levels of detail 9, 10, 11, 13, and 14, i.e., from the right-column image of FIG. 5(a) to that of FIG. 5(f), the target area can be seen more and more clearly. It can therefore be inferred that the LOD linear transformation-based adaptive image contrast enhancement method can produce noticeable effects on optical satellite-remote-sensing images with partial cloud coverage or extensive cirrocumulus coverage.

(26) FIG. 6(a) to FIG. 6(c) show images of Taiwan taken by Landsat-8 on Jul. 29, 2019. The left-column, middle-column, and right-column images of each of FIG. 6(a) to FIG. 6(c) are respectively a screenshot of the USGS LandsatLook Viewer website (https://landsatlook.usgs.gov/viewer.html, as browsed on Jan. 16, 2020), a screenshot of the LandsatExplorer website ((http://landsatexplorer.esri.com/, as browsed on Jan. 16, 2020), and the result of processing with the LOD linear transformation-based adaptive image contrast enhancement method. As the detail of level is sequentially switched from level of detail 10 to levels of detail 12 and 14, i.e., from the right-column image of FIG. 6(a) to that of FIG. 6(c), the target area can be seen more and more clearly. It can therefore be inferred that the LOD linear transformation-based adaptive image contrast enhancement method can produce noticeable effects on optical satellite-remote-sensing images with partial cloud coverage or extensive cirrocumulus coverage.

(27) In summary of the above, the LOD transformation-based adaptive image contrast enhancement method of the present invention can enhance the image contrast of each of a plurality of to-be-processed levels of detail in a progressive manner and use the grey-scale values of the highest-solution target level of detail as the standard of contrast enhancement in order to produce a plurality images of different levels of detail whose degrees of enhancement vary from one to another. With the invention, one who is using Internet map services and wishes to view a map at various magnification ratios will be able to see images of optimal brightness and color saturation, or of great clarity in short, and the objective of the invention is thus achieved.

(28) While the invention has been described in connection with what is considered the most practical and preferred embodiments, it is understood that this invention is not limited to the disclosed embodiments but is intended to cover various arrangements included within the spirit and scope of the broadest interpretation so as to encompass all such modifications and equivalent arrangements.