Abstract: Underwater images normally suffer from absorption and scattering effects of the light due to the oceanic environment. The key challenge in underwater imaging is object recognition due to the turbidity in water. Remotely operated vehicles provide artificial light which illuminates in a non-uniform way, resulting in poor visibility of underwater images. Contrast plays a major role in object recognition. Traditional methods deal with global information of an image and hence often does not achieve a good contrast enhancement. Underwater image characteristics are tentative and often change. Hence, there is a pressing need for adaptive algorithms in this area. Adaptive contrast enhancement algorithms based on the image fuzziness have been proposed for underwater images with varying contrast. Performance metrics like Peak Signal Noise Ratio (PSNR), Contrast to Noise Ratio (CNR), Absolute Mean Brightness Error (AMBE) and Image Enhancement Metric (IEM) are used to evaluate the performance of the proposed algorithm. Fuzzy edge retained amplification method provides enhancement with well-preserved edge information and improved contrast, when compared to the fuzzy amplification method. The proposed algorithm was able to achieve a better contrast for images that had 20% contrast with an AMBE of 36.57, IEM of 9.620, CNR of 11.39.
INTRODUCTION
Underwater image processing has received considerable attention in the recent years due to the increase in the use of Remotely Operated Vehicles (ROV) and Autonomous Underwater Vehicles (AUV) for submarine operations (Patel and Sao, 2014; Garcia et al., 2002). These techniques are applied to survey the ocean floor generally in search for underwater mines, shipwrecks, coral reefs, pipelines, etc., from the underwater environment. Such images obtained suffer from poor visibility due to the propagation properties of the light in water and makes underwater imaging a challenging one. The transmission properties of light such as absorption and scattering causes problems such as limited range, blurring edges and less intensity variation in underwater image. Since natural light is not sufficient for imaging in the sea bed, artificial light sources become necessary. Such artificial light causes additional problems like non-uniform illumination which causes a brightening effect in certain sections of the image and the background being poorly illuminated.
In underwater image processing, the basic physics of light propagation in the water medium comes into extinction. As light enters the water, it is exponentially attenuated and so the visibility distance is limited. Therefore, underwater image processing is necessary (Patel and Sao, 2014). Several research works on underwater image processing has been done in the last few years. Histogram stretching or gray-level transformations do not yield good enhancement results. Traditional techniques often give results only by manual setting of parameters based on image quality. Arnold-Bos et al. (2005) developed an automated denoising framework using existing pre-processing methods for contrast equalization and a robustness criterion to assess the quality of the enhancement methods. Bazeille et al. (2006) developed a novel pre-processing filter algorithm for underwater image enhancement wherein the adjustments were still required for improvement of the algorithm. Iqbal et al. (2007) devised an integrated color model where enhancement was achieved through contrast stretching. Garcia et al. (2002) reviewed the existing techniques like homomorphic filtering, local histogram equalization and conducted experiments on real data, but results were not checked quantitatively.
One cannot achieve reasonable results in dynamic and broad range of non-uniform illumination images by using conventional approaches. Therefore, adaptive algorithms that perform uniform enhancement of dynamic low contrast images are needed (Hasikin and Isa, 2012). Application of Fuzzy set theory has been successful in image enhancement, denoising and pattern recognition (Hasikin and Isa, 2012; Karam et al., 2013). Fuzziness refers to the uncertainty of occurrence of an event and its associated vagueness and/or imprecision. The concept of image contrast on the whole has been interpreted as a qualitative rather than a quantitative measure of an image. The image acquisition process itself often causes vagueness and uncertainty in the acquired image. Imprecise boundaries and intensities of pixels during image digitization become uncertainty or fuzziness in the image. Therefore, application of fuzzy set theory to contrast enhancement results in better enhancement (Kerre and Nachtegael, 2000). In this study, fuzzy based image enhancement techniques were proposed to enhance the quality of underwater images. The remaining of this study is organized as follows: Section II details materials and methods. Section III describes the performance metrics used for analysis. Section IV presents the results on real underwater images. Section V discusses the performance of the proposed methodology. Finally, section VI concludes the study.
MATERIALS AND METHODS
Fuzzy image processing: Fuzzy image processing is the assembly of all approaches in which images, their segments and features are expressed as fuzzy sets. Fuzzy techniques are non-linear and purely based on the knowledge of human developing them. If data are imperfect due to vagueness and ambiguity rather than randomness then, fuzzy logic can provide solutions. When compared to other methods like Histogram equalization, intensity based operations, or feature based methods, a fuzzy based enhancement method will process an image in the so-called membership plane to modify/aggregate the membership values, classify data, or make decisions using fuzzy inference (Pal and Majumder, 1986). The representation and processing depend on the choice of fuzzy method and on the problem to be solved.
Fuzzy image processing has three main stages:
| • | Image fuzzification |
| • | Appropriate modification of membership values and if necessary |
| • | Image defuzzification as in Fig. 1 |
The input image consists of crisp input pixels, which are converted to fuzzy values in the fuzzification procedure, after which the membership values are modified to generate new pixels. Membership function is a property or operation which expresses the fuzziness in a fuzzy set (Kerre and Nachtegael, 2000). Finally, defuzzification is done to convert fuzzy values to crisp output pixels. The modification of membership values makes fuzzy image processing an optimized procedure.
| Fig. 1: | General structure of fuzzy image processing |
Proposed methodology: Fuzzy based method for adaptive enhancement has been proposed. An m×n image with intensity levels in the range {0-255} can be considered as a group of fuzzy singletons expressed in the fuzzy set notation as:
where, μ(pij) or
Fuzzy amplification method: Enhancement of the image cannot be obtained just by uniform darkening or uniform brightening. The proposed method involves a series of processes like defining the membership values, modification of membership values and the generation of new gray levels. Initially, each pixel in the input image was converted to appropriate membership values by normalizing the values.
Fuzzification converts pixels in the range {0-255} to membership values of the range {0-1}. Then, the intensity levels were divided into three regions namely; dark, gray and bright regions.
This classification was done by transforming the pixels with fuzzy values based on multiple thresholds (t1, t2 and t3), instead of a single threshold (Pal and King, 1980). Membership functions were applied over the pixel values based on the threshold, to get the enhanced image.
Then, the membership values were modified by applying amplification over the membership values to get the improved contrast image. Finally, the enhanced version of the image was obtained by re-scaling the membership values to the gray levels. The pseudo code of amplification method is given below.
Pseudocode of amplification method:
Fuzzy edge retained amplification method: The algorithm proposed was unable to achieve better enhancement in images with poor contrast. Liu (2012) proposed a fuzzy enhancement algorithm to improve the contrast of gray scale images wherein membership functions were applied directly to the pixel values. This might result in enhancement for a normal image. But images captured in underwater are often characterized by poor contrast, low illumination, etc. So minimum value and maximum value of a pixel in the image is often not equal to 0 or 255. Before any membership function is applied, the pixels have to be normalized in the range [0, 1]. Hence, the methodology was modified in such a way that it is able to retain edge information. The membership function proposed for the edge retained amplification method is as shown in Eq. 1:
| (1) |
To obtain the crossover point that divides the image into dark and bright regions, the Otsu threshold (Otsu, 1979) was applied and is denoted as t1. Since a threshold is set, the image is divided into dark and bright regions. Fuzzy achieved optimal contrast enhancement by reducing the intensity in darker regions and increasing the intensity in the brighter regions. As the whole function was continuous, this helped in maintaining a membership grade resulting in normalized enhancement. Additionally the tangent function helped in reducing the intensities of darker pixels and increasing the intensities of brighter pixels and thus achieving a balance to get a final enhanced image. The inverse transformation was done along with rescaling to obtain the enhanced version of the image, as shown in Eq. 3:
| (2) |
| (3) |
So, there is a linear transformation of gray levels, which helps in retaining low gray level information. Therefore, this enhancement smoothens the background while preserving edges to obtain good visual enhancement. The pseudocode for edge retained amplification is provided below.
Pseudocode for edge retained amplification:
Performance metrics: The performance of the proposed algorithms were tested using PSNR, CNR, AMBE and IEM. PSNR is defined as the ratio between the maximum signal over the corrupting noise.
| (4) |
where, f (i, j) represents the pixel value in the original image, fʹ(i, j) represents the pixel value in the reconstructed image, pmax is the maximum pixel value and m, n are the dimensions of the image (Bhaskaran and Konstantinides, 1995). Logically, a higher value of PSNR is good because it means that the ratio of Signal to Noise is higher. The CNR is a measure for determining image quality. The CNR is a good metric for describing the signal amplitude relative to the ambient noise in an image. The CNR can be computed using:
| (5) |
where, μf is the mean of the original image, μn is the mean and σn is the standard deviation of the difference between the images. The AMBE is used for determining the amount of brightness preservation. It is defined as the absolute difference between the mean of the original image and the mean of the enhanced image. Generally a higher value of AMBE indicates better contrast improvement.
| (6) |
The IEM measures the sharpness and contrast improvement after enhancement. The IEM can be computed using:
| (7) |
where, the original and enhanced images are divided into k1×k2 non overlapping blocks, k1 = k2 = 3.
RESULTS
In this section, the results of the proposed fuzzy based amplification algorithms are analyzed and compared. The algorithms were implemented and tested using Matlab 2013.
| Fig. 2(a-d): | Visual comparison of fuzzy based enhancement methods for image A, [I-IV] shows enhancement results using Fuzzy amplification, Fuzzy edge based amplification, Fuzzy edge based amplification with number of iterations (k = 2), Fuzzy edge based amplification by applying sliding window technique, (a-d) Shows images with contrast reduced by 0, 20, 50 and 80% |
Data sets used for testing were collected from National Institute of Ocean Technology (NIOT), Chennai and web databases. Experimental results obtained using images from NIOT alone are shown in Fig. 2 and 3. Row 1 of Fig. 2 shows the original image. In order to check the dynamic property of the proposed algorithm, the contrast of the image was varied from 20-80% using Adobe Photoshop CS5 and are shown in rows 2-5 of Fig. 2. A similar operation was also performed on other images as shown in Fig. 3. These images were used to evaluate the proposed algorithms.
The fuzzy based amplification method required manual selection of thresholds. Various thresholds were set and tested for image enhancement. Based on the intensity values of the images, thresholds t1, t2 and t3 with values 0.3, 0.5 and 0.9 were selected for amplification method.
| Fig. 3(a-d): | Visual comparison of fuzzy based enhancement methods for image B, [I-IV] shows enhancement results using Fuzzy amplification, Fuzzy edge based amplification, Fuzzy edge based amplification with number of iterations (k = 2), Fuzzy edge based amplification by applying sliding window technique, (a-d) Shows images with contrast reduced by 0, 20, 50 and 80% |
Membership functions were defined for each threshold selected. This caused the histogram to be stretched to form a histogram equalized image. The results obtained are shown in column 2 of Fig. 2 and 3. Though, amplification method yielded an enhanced image with normal contrast, it was unable to give good results for low contrast images. This is because the pixel values were not spread throughout the histogram, resulting in a single peak. The amplification algorithm should be devised in such a way that the pixel values are stretched evenly in the histogram, along with edge preservation. Trial and error based threshold selection was also a major challenge.
An automatic threshold which classifies dark and bright pixels in a low contrast image was the need and Otsu threshold supplied the need. Hence, for edge retained amplification method, the threshold was computed using Otsu method and threshold values computed for various contrast images are shown in Table 1 and the results are shown in columns 3-5 in Fig. 2 and 3.
Since the edge retained amplification method yielded a better enhancement, it was proposed to apply an iterative approach to check for further enhancement. Therefore, when the number of iterations, k was increased, edges were clear when applying the edge retained amplification method. The results obtained with k = 2 is shown in column 4 of Fig. 2 and 3. When k was further increased, it yielded in a binary image with reduced edges.
Originally a filter of size m×n representing the rows and columns of the image was used. Since our objective is adaptive enhancement, a variation in the filter was also attempted. Usually for multichannel images sliding window technique is applied for filtering or pre-processing. On a trial and error basis, the sliding window of varying sizes from 1-400 were tried for the edge retained amplification method. Amplification method continued to provide the same results as in the case of global filter and hence results are not given. Small window sizes, for example size 30 created an overall boxing effect in the image. This boxing effect reduced when the window size was increased to 400, resulting in a better image quality with an IEM of 7.865 for an image with 20% contrast.
Table 2 shows the PSNR, CNR, AMBE and IEM results of the various methods for the test images. Here, when applying the amplification method a maximum PSNR of 17.03 was obtained for Image A and 22.76 was obtained for Image B. The amplification method did not result in much improvement in case of low contrast images and the image appeared visually blurred even after enhancement. Hence, PSNR appears to be high even for low contrast images when applying amplification method. Similarly, edge retained amplification method provided a maximum PSNR of 20.94 for Image A and 22.89 for Image B. When reducing contrast to a maximum of 80% existing enhancement techniques were unable to provide an enhanced contrast nor retain feature information. In case of low contrast underwater images, the objective was to improve the contrast while retaining the edges. This was achieved by using an edge retained amplification method where in the visual clarity improved and is shown with the help of CNR. A decrease in CNR indicates an improvement in image contrast. Initially, amplification method was able to provide a good CNR of 15.64 for image A. But as the contrast was reduced, amplification method was unable to provide any betterment visually and resulted in an increase in CNR.
| Table 1: | Threshold values for underwater images |
| Table 2: | Comparison of the enhancement methods based on PSNR, CNR, AMBE and IEM, [I-IV] shows enhancement results using fuzzy amplification, fuzzy edge based amplification, fuzzy edge based amplification with the number of iterations, k = 2 and fuzzy edge based amplification by applying the sliding window technique, (a-d) Shows results for images with contrast reduced by 0, 20, 50 and 80% |
| Fig. 4(a-d): | Gradient magnitude histogram (I) and log of gradient magnitude histogram (II) after applying edge retained amplification with iteration, K = 2 for image A, (a-d) Shows the results for images with contrast reduced by 0, 20, 50 and 80% |
Edge retained amplification method gave better results for low contrast images showing a steady decrease in CNR and provided contrast amplification visually also. The IEM is equal to 1 when the input and output images are same (Jaya and Gopikakumari, 2013). As image quality improves an increase in the IEM is seen.
In order to check improved edge retainment in the enhanced images, gradient magnitude histogram and log of gradient magnitude histogram were plotted as shown in Fig. 4. Generally, in histograms an unequalised image has high peaks. A contrast enhanced image has a uniform distribution. These histograms show an increase in gradient values and attenuation of peaks after enhancement indicating contrast enhancement. The performance of the proposed algorithm was further validated by calculating the area under the histogram. The proposed edge retained amplification method showed an increase in area by 34% and 80% for images with original and reduced contrast, respectively.
DISCUSSION
The objective of the proposed work was to enhance the underwater images for accurate object recognition. Though many traditional techniques are available for enhancement they result in loss of edge information and color (Berretti et al., 2016). A fuzzy enhancement technique can solve this vagueness and can retain edge information. In fuzzy enhancement, the pixels in the image are transformed to the fuzzy membership plane with the help of fuzzy membership functions (Cheng and Xu, 2000). Pal and King (1980) proposed an image enhancement technique using fuzzy sets with the help of a single threshold.
When fuzzification was applied on darker pixels the edge information was totally lost (Shiwei et al., 2010). So a fuzzy based enhancement technique which has multiple thresholds (dark, gray and bright) was proposed, thereby even if the edge information is darker, those pixels will not be further reduced and lead to loss of of edge information. Though the proposed algorithm worked for high contrast images they fail to provide enhancement for poor contrast images. An amplification in the contrast was needed and hence a fuzzy edge based amplification method was proposed. This is a variant of the fuzzy enhancement technique proposed by Liu (2012). Here the pixel values are normalized before application of the membership functions so that enhancement occurs in poorly contrasted images also. A poorly contrasted image will have intensity values concentrated in a particular region in the histogram. If the intensity values are concentrated in one particular range and normalization will normalize the values to 0-1. This normalization is necessary as underwater images are often poorly contrasted. The tangent function helped in uniform enhancement of gray levels.
Garcia et al. (2002) used images with non-uniform illumination and images that had a bright spot of light in the image for the experiment. Correcting such images is challenging and enhancement of such images might result in loss of edge information. Our study also included such images and our proposed methodology was able to preserve the image information. Apart from that our study also included images with poor contrast and our methodology was able to achieve significant results in such poor contrast images. The proposed methodology was able to perform equally with the existing methodology in case of images with 80% contrast. It achieved an improvement in AMBE by 12.78, IEM in 5.28 and reduction in noise ratio with 27.3 for images with 20% contrast.
Arnold-Bos et al. (2005) indicated the improvement in the contrast, used gradient magnitude histograms as a quantitative criterion. This was implemented for original and enhanced images as shown in Fig. 4. The histograms have uniform distribution after enhancement indicating contrast improvement and increase in the edge pixels (Bazeille et al., 2006).
CONCLUSION
This study analyses the performance evaluation of fuzzy amplification and fuzzy edge retained amplification on the underwater images. Test data sets were collected from NIOT. Both methods were exhaustively tested by lowering the contrast using Adobe Photoshop CS5. For contrast improvement the number of iterations was also increased (k = 2) resulting in enhanced edges and an upsurge of dark pixels. For adaptive enhancement sliding window technique was applied and results were obtained. In order to test the quantitative factor, PSNR, CNR, AMBE and IEM were computed. The improvement of image quality and edge retainment is shown statistically using gradient magnitude histograms. Test results showed edge retained amplification method performed better over amplification method even for poorly illuminated images. Edge retained amplification method provided adaptive contrast improvement while retaining edges for varied low contrast images.
ACKNOWLEDGMENTS
The authors thank the Naval Research Board (Grant No. NRB-295/SSB/12-13), DRDO, New Delhi, India for their constant support. The authors thank NIOT, Chennai, India for providing the necessary dataset. The authors would like to acknowledge Earth System science Organisation NIOT (ESSO-NIOT) and Ministry of earth sciences for their support. The authors also thank Hindustan Institute of Technology and Science for their continual support.