Abstract: In this study, a new single frame technique for superresolution (SR) based on (symmetric) Daubechies Complex Wavelet Transform (DCWT) is proposed. DCWT is known for numerous advantages including linear phase, directionality and nearly invariant to shift, rotation and scale. The high frequency real component coefficients and all the imaginary component coefficients of the DCWT are bicubic interpolated with an arbitrary factor while the low frequency real component coefficient is substituted with the input image which is bicubic interpolated with half of the interpolation factor. Synthesis is then done with inverse DCWT resulting into a SR output. Standard test images are used in this work for easy comparison. Visual and tabular results confirm the superiority of the proposed technique over conventional and state-of-the-art single frame SR methods.
INTRODUCTION
Current imaging research seeks high resolution images from the acquired Lower Resolution (LR) observation images. Such LR images may have been obtained with noisy hardware, in poor (blurred) visibility conditions or aliased. Superresolution (SR) is a set of image processing (software) techniques aimed to enhance the resolution of an imaging system beyond its sensors and optical limits. Such techniques are presumably inexpensive alternatives compared to imposing High Resolution (HR) unfeasible requirements onto imaging hardware devices or sensors (Park et al., 2003). SR offer more details to the user (human or robot) in various disciplines including: Medical imaging, satellite, texture analysis, surveillance and more.
SR acquisition methods are classified into two, namely; traditional multi-frame method which extracts a HR image signal from a sequence of Low Resolution (LR) frames of the same scene but with slightly different perspectives and aligned at sub-pixel accuracy (Zhang et al., 1999). Single frame method is where a super-resolved image is obtained from a single LR observed image signal. SR from several images is effective, however, if such image frames are identical, then there is no extra information to be collected from them and if the difference between the frames is enormous, then the information from these frames is almost useless for reconstructing a single HR image. Some cases like MRI scan enforce a single observation due to health and economic reasons while in other cases, the LR frames are simply not enough for multi-frame SR. We restrict ourselves to single frame SR to overcome shortcomings of multi-frame SR. Single frame mode limits delay, eliminates the need for buffers and expensive memory operations (Boon et al., 2006). In this study, SR refers to the problem of enlargement/up-sampling of an image while avoiding artifacts and maintaining image sharpness. Unlike other sharpening techniques such as restoration and denoising, SR alters pixel size to increase pixel density and therefore achieve HR. SR may be considered a second generation problem to image restoration (Park et al., 2003).
In this study, a single frame SR method based on interpolation of Daubechies (symmetric) complex wavelet (DCWT) coefficients is proposed. We are not aware of any publication relating DCWT and SR prior to writing this paper. We hope that this work will be a good reference point.
REASONS FOR DAUBECHIES COMPLEX WAVELETS
The Discrete Wavelet Transform (DWT) is a powerful tool for analysis and synthesis of digital images. Its main advantage comes from providing localized information in frequency and time; however, the transform suffers from shift variance, poor directionality and lack phase information (Belzer et al., 1995). Modifications have been made to the DWT such as undecimated wavelet transform, however, despite the increased computational expense, such modifications resolve only some of the issues but not all. Other solutions include the use of complex wavelet transforms. Many complex wavelet transforms have been proposed, such as: Dual tree (Kingsbury, 2001), steerable pyramid, projection based complex wavelet transform; these are shift invariant, provide directionality and phase information. However, Khare and Tiwary (2007), noted that these complex wavelet transforms are implemented with real valued filters similar to the DWT and therefore they are not truly complex wavelet transforms. The real valued filters are characterized by decimations which result into high computational expense and artifacts in resulting images. A promising alternative solution to the issues above is DCWT which implements complex valued filters (Khare and Tiwary, 2007; Lina and Gagnon, 1995; Lina and Mayrand, 1995; Belzer et al., 1995; Lina, 1997). Inspired by the application of DCWT to image compression (Belzer et al., 1995), enhancement and restoration (Lina, 1997) and target tracking (Khare and Tiwary, 2007), the same (DCWT), is proposed for image SR in this study. The transform: (1) is nearly shift invariant, (2) can be made symmetric for easier hardware implementation (Lina and Mayrand, 1995) and handling boundary problems of finite length signals (Zhang et al., 1999), (3) has approximate linear phase properties which preserve the position of details in the filtered signal with lower computation cost (Khare and Tiwary, 2007; Sherlock and Kakad, 2002; Belzer et al., 1995; Zhang et al., 1999), (4) provides good directionality and (5) has limited redundancy.
Most energy (averaging information) in the DCWT is concentrated in the real components while the imaginary components preserve phase and edge information due to hidden Laplacian operator inherited from the symmetric constraint. Indeed, the natural redundancy in DCWT provides dual representation of zero crossings and local extrema (Lina and Gagnon, 1995).
Let
| (1) |
where, bk are scaling coefficients possibly complex valued, such that:
Besides satisfying multiresolution properties, φ (x) (and consequently the
wavelet function ψ (x) in Eq. (2)); has compact supported
in the interval [-N, N+1], is an orthonormal basis on a space of square integrable
functions Vo∈L2 (
The general wavelet function is given by Eq. 2:
| (2) |
where, ak are wavelet coefficients also possibly complex valued. The DCWT exist for all values of N≥2, however, symmetry is only possible when N is even (Lina and Mayrand, 1995).
An image function f (x, y) can be decomposed into a linear combination of translates of the scaling function φ (x,y) at some fixed scale and wavelet function ψ (x) at finer scales as shown in Eq. 3:
| (3) |
where, j0 is lowest resolution level, jmax is the maximum
possible resolution level,
DIGITAL IMAGE OBSERVATION MODEL
The recorded LR observation is assumed to be a down sampled version of the HR image (scene). The two are related by a generative model in Eq. 4 and explained by Fig. 1.
| Fig. 1: | General observation model of imaging system, the case of a single observation |
Let Or represent an ideal un-degraded image of M2x1 pixels sampled at or above Nyquisit rate from a continuous scene which is also assumed to be band-limited and Ob represents the low resolution observed image of N2x1 pixels, then a generative model is given by:
| (4) |
where, is the decimation matrix with size depending on the decimation factor and caused by aliasing, B is the blur matrix and φ is the N2x1 random (Gaussian) noise inherent in any imaging system with zero mean and variance σ2 (Park et al., 2003). The LR observation Ob is aliased, noisy and blurred. This model is valid for our work since the reference image Ob is a single observation which can be generalized to image sequence and higher dimensions.
Given observation Ob SR techniques approximate a HR image Ôr of Or In ideal sense, SR techniques remove noise and blur while recovering an alias-free up-sampled version of Ob.
PROPOSED SUPERRESOLUTION METHOD
This section presents the theory and structure of the proposed SR method. Complex Daubechies wavelet filters can be obtained with the function explained by Sherlock and Kakad (2002). Multiresolution analysis in 2 D signals is accomplished with dyadic quadrature mirror filters which we have implemented in WaveLab v850 originally developed by Donoh and applied for construction of various wavelets (Mallat, 2009).
Proposed procedure: Using the above knowledge, the DCWT was implemented in WaveLab. The DCWT was then applied to decompose input LR image signal Ob into complex sub-band images namely; low frequency component also known as approximation (LL), Horizontal (LH), vertical (HL) and diagonal (HH). Each of these complex coefficients is composed of a real and imaginary image component as shown in Fig. 2.
The input image Ob was bicubic interpolated with an arbitrary interpolation factor Ω/2 and result was then used to replace the low frequency real component coefficient (LL). This act increases the system resolution power (Vandewall et al., 2006). All the other coefficients (real component high frequency and all the imaginary components) coefficients were bicubic interpolated with twice the same arbitrary interpolation factor. The (inverse) IDCWT was then used to reconstruct the HR image. Various interpolation methods were evaluated in our method as well as direct up-sampling of spatial images; bicubic interpolation required more processing time, however, better quality results were produced compared to bilinear and nearest-neighbor interpolation methods as shown in Table 1. This may be due to the algorithm complexity. DCWT with six vanishing moments was chosen for its nice trade-off between image sharpness and noisy results.
| Fig. 2(a-b): | Coefficients (a) Real component, (b) Imaginary component of DCWT |
| Table 1: | Comparison of peak signal to noise ratio for different interpolation methods used to test proposed method |
Fusion of the complex coefficients with IDCWT requires all the four complex sub-band coefficients as shown by the block diagram in Fig. 3.
Input LR images of resolution 128x128 were obtained by applying DWT consecutively twice as done by (Anbarjafari and Damirel, 2010) and the original 512x512 images were used as ground truth for performance evaluation of the proposed technique.
| Fig. 3: | Proposed DCWT based SR method |
A step by step description of the proposed technique is as follows:
Procedure to acquire the input image:
| Step 1: | A selected 512 x512 test image is decomposed with DWT |
| Step 2: | The low frequency coefficient (approximation (LL)) of the DWT is retained while the high frequency coefficients, namely; Horizontal (LH), vertical (HL) and diagonal (HH) are ignored (assumed to be equal to zero). The remaining LL is exactly half (256x256) of the original (512x512) image dimensions due to decimation |
| Step 3: | Repeat steps 1 and 2 using the result of step 2 as input. This leads to a LR image of resolution 128x128 which is used as input to the proposed technique |
Procedure for the proposed technique:
| Step 1: | Decompose the LR (nxm) image with DCWT. The output are decimated complex coefficients of resolution n/2x m/2 |
| Step 2: | Select an arbitrary interpolation (resolution enhancement) factor Ω, for example, Ω = 4 |
| Step 3: | Using the selected interpolation factor Ω, bicubic interpolate the high frequency complex coefficients (LH, HL and HH) together with the imaginary component of the low frequency component LL of the DCWT. The interpolation results are coefficient images of resolution Ωn/2xΩm/2 coefficient images |
| Step 4: | Interpolate the original LR image with half the interpolation factor. The result is a blur image of resolution Ωn/2xm/2 |
| Step 5: | Replace the real component of the low frequency coefficient (LL) of DCWT with the result of step 4 |
| Step 6: | Use the IDCWT to reconstruct the high resolution image. The output is a ΩnxΩm high resolution image in which the high frequency components are more preserved compared to direct interpolation with conventional techniques |
QUALITY OF RE-SAMPLED IMAGE
Interpolation, also known as re-sampling is a method to increase or reduce the number of pixels in a digital image by estimating image values at locations in between pixels. Image quality loss during interpolation based SR is mainly caused by high frequency edge smoothing evident in Fig. 4b. Image quality critically depends on edge preserving. Anbarjafari and Damirel (2010) applied DWT to preserve high frequency edges. Although, the asymmetry in DWT scaling functions does not seem to introduce significant asymmetry in the results, more accurate results can be obtained by using symmetric scaling function in DCWT (Gagnon and Lina, 1994) which is more equipped to preserve such edges and overcome several shortcomings of DWT in section II.
RELATED WORK
Related researches include; the work of Temizel and Vlachos (2005) in which image resolution up-scaling is done with the use of directional cycle spinning and wavelet-domain zero padding (WZP-CS), cycle spinning is a method used against artifacts in wavelet denoising.
| Table 2: | Comparison of peak signal to noise ratio for various conventional and state-of-the art technique |
Cycle spinning aims to approximate shift-invariant statistics by averaging out cyclostationarities introduced by quantization into wavelet coefficients of an image during non-exact estimation of high frequency coefficients. Demirel and Anbarjafari (2010) proposed dual tree complex wavelet transform (DT-CWT) in a technique to enhance satellite image resolution based on interpolation of selected sub-bands of the DT-CWT. The DT-CWT has limited redundancy and is approximately shift invariant compared to the DWT. Demirel and Anbarjafari (2011) used the DWT in conjunction with stationary wavelet transform (DWT+SWT) to enhance edges while up-scaling the image, high frequency sub-band images of the DWT and the input image were interpolated and then stationary wavelet transform was used to enhance edges before inverse transforming the sub-bands with inverse DWT. The same reference shows SWT individually applied for resolution enhancement. These state-of-the art methods have been compared with the proposed method and results have been tabulated in Table 2 showing superiority. Standard test images with various features including Lena, Elaine, mandrill (baboon) and peppers have been used for easy comparison.
RESULTS AND DISCUSSION
In this study, a super-resolved image was acquired from a LR single frame counterpart by interpolation of DCWT coefficients. Figure 4a shows the down-sampled LR input Lena and peppers of 128x128 marked with a square section considered for easy presentation of this study. Figure 4b shows results of bicubic interpolating 128x128 by X4 to get 512x512 for Lena and peppers in spatial domain while Fig. 4c shows an improved quality image of the same using our method. The error (residue) between the ground truth HR original image and the reconstructed image was expressed by subtracting the two images. Figure 5 shows residue (error) between the ground truth and the reconstructed Lena image: (a) with our method, (b) direct spatial interpolation with bicubic interpolation and (c) using DCWT without replacing the high frequency coefficient with the interpolated input. The residue is much smaller in: (a) compared to (b) and (c) hence our method preserves more high-frequency details. Image quality was measure numerically using Peak Signal to Noise Ratio (PSNR) in decibel (dB) which can be calculated by Eq. 5:
| (5) |
where, MSE is the mean square error between Or and Ôr given Eq. 6:
| (6) |
Table 1 compares PSNR values of our method with the various conventional interpolation methods namely; Nearest Neighbor, Bilinear method, Bicubic method. The values conform to visual results in Fig. 4, for example Lena standard image tested in our method using various interpolation methods shows 7.99, 7.82 and 8.39 dB improvement in PSNR over direct interpolation with bicubic, bilinear and nearest neighbor interpolation, respectively. Table 2 compares PSNR from our method with conventional methods together with state-of-the art methods namely; DWT, DT-CWT, DWT+SWT, SWT and WZP-CS. Our method yielded sharper edged images indicated by high PSNR, for example, PSNR of 34.84, 33.45, 24.51 dB for Lena, peppers and baboon images respectively which are higher than conventional and state-of-the art methods. Elain image resulted in a PSNR of 33.49 using our method, this was higher than most methods shown in Table 2, but however, it was 1.52 lower than PSNR of 35.01 given by DWT+SWT method. Our method continued to provide sharper edges even at X8 zoom.
The proposed method implements single frame SR and therefore attempts to resolve the ill-posed problem of lack of sufficient low resolution frames for multi-frame SR.
Visual and numerical results indicate superiority of the proposed technique over conventional and state-of-the art single frame SR techniques. The method reduces noise and blur while up-sampling the images, however, IDCWT introduced aliasing in the final images evident in Fig. 4c, this may be handled by hybrid techniques which combine both local and global processing methods.
| Fig. 4(a-c): | (a) Original 128x128 pixels with box mark, (b) X4 zoom with direct bicubic interpolation and (c) X4 zoom with our method |
Further improvement may be achieved by DCWT at higher decomposition level and application of more complex interpolation technique.
Assuming that individual frames are temporarily independent, methods for enhancing and restoring a gray image in current literature may also apply to higher dimensions for example, color images and sequences (video).
| Fig. 5(a-c): | Residual between ground truth image and superresolved image using: (a) our method, (b) Direct Spatial interpolation and (c) Without replacing LL in our method |
Therefore, it is possible to extend the proposed technique to higher dimension image signals at the expense of complexity and delay. It would not be surprising to produce better results in subsequent applications like remote sensing, target/tumor detection, HD-TV display, compression and texture analysis if our method is implemented in conjunction with such applications.
CONCLUSION
In this study, a new single frame SR method based on interpolation of DCWT coefficients was proposed. DCWT was implemented with Wavelab and used to decompose images into different complex valued sub-band coefficients; the coefficients were selectively interpolated with arbitrary interpolation factor to achieve sub-pixel accuracy and then reconstruction was done with IDCWT resulting into super-resolved images. The method was tested on four standard images, visual and tabular results show superiority of the proposed SR method.
ACKNOWLEDGMENT
Reviewer comments were helpful in improving the presentation of this paper and therefore highly acknowledged. This work was supported by the government of china through the Chinese Scholarship Council (CSC).