Information Technology Journal

Year: 2011 | Volume: 10 | Issue: 11 | Page No.: 2220-2225
DOI: 10.3923/itj.2011.2220.2225

Wavelet-domain Hidden Markov Tree Model Approach to Fusion of Multispectral and Panchromatic Images

Jianjun Yin, Ming Gu, Jiang Wang and Jianqiu Zhang

Abstract: We propose a wavelet-domain Hidden Markov Tree (HMT) model-based multi-spectral and panchromatic images fusion algorithm in this study. Our algorithm exploits the wavelet-domain HMT model learnt from the high-resolution panchromatic image to perform super-resolution operation to the low-resolution multispectral image. In this way, the desired high-resolution multispectral image is obtained. The experimental results showed that the proposed algorithm can produce sharper images as well as retaining good color. Moreover, as a result of the insensitivity of the wavelet coefficients’ statistical information to the noises, our algorithm exhibits stronger robustness to the noises.

Keywords: wavelet, hidden Markov tree model, Image fusion, multispectral image and panchromatic image

INTRODUCTION

The fusion of Panchromatic (PAN) and Multispectral (MS) images is the process of combining the PAN and MS images to produce images characterized by both high spatial and spectral resolutions. With the fusion of different images, we can overcome the limitations of information obtained from individual sources.

It has been noted by Joshi et al. (2006) that the fusion of MS and PAN images is an ill-posed problem. To solve the ill-posed problems, we need regularizations. Accordingly, the model-based fusion approach is proposed. First of all, we compute the statistical prior information from the PAN image. Then, to obtain the high-resolution MS image, we can perform super-resolution to the low-resolution MS image with the prior information. The most important part of the model-based fusion methods is how to obtain the appropriate prior information from the PAN images. Recently, various models have been proposed for the model-based fusion, including auto-regressive model (Joshi et al., 2006), Gaussian Model (Hardie et al., 2004), inhomogeneous Gaussian-Markov model (Joshi and Jalobeanu, 2010) and total variational model (Kumar and Dass, 2009).

However, these models are all computed in the spatial domain. Although, the spatial-domain statistical models generally work well when there is little noise, the performance degenerates considerably when the noise increases. Wavelet-domain information is well-known for its robustness to noise (Eismann and Hardie, 2005) and is therefore, exploited here to obtain a more robust prior model. We propose a MS and PAN images fusion method using wavelet-domain hidden Markov tree model. This model can model both the statistical characteristics of the wavelet coefficients and the relationship across wavelet coefficients hierarchy. It is widely used in image denoising, image super-resolution and compressive sensing (Romberg et al., 2001). It provides useful prior information for model-based image fusion methods. Because of the robustness of the wavelet-domain statistical model, the proposed algorithm is less sensitive to noise. This characteristic will be validated by our experiment in present study.

Wavelet-domain hidden Markov tree model: An image can be decomposed as (Mallat, 1999):

where,

form an orthonormal basis for , while This results in a natural quad-tree structuring of the three subbands. It is known that most of the energy concentrates in a small fraction of wavelet coefficients. This sparsity property is desirable because it enables an image to be represented with a small number of wavelet coefficients. The wavelet coefficients of natural images have two primary properties: non-Gaussianity and persistency. That is why, the wavelet coefficients have peaky, heavy-tailed marginal distributions and large/small values of wavelet coefficients tend to propagate through the scales of the quad-trees. The HMT approximates the non-Gaussian marginal pdf as an independent Gaussian mixture and employs Markov chain to capture the interscale persistence property.

WAVELET-DOMAIN HMT BASED MULTISPECTRAL IMAGE FUSION

Multispectral image observation model: The multispectral image fusion problem can be cast in a high-resolution restoration framework (Joshi et al., 2006), where an appropriate prior model learnt using the high spatial resolution PAN observation is exploited to regularize the solution. Figure 1 shows the block scheme of the model that represents the relationship between the available low-resolution MS-image and its high-resolution version for a single spectral channel. In particular, the observed low-resolution MS images are models as decimated and noisy versions of their high-resolution versions.

Let Z₁ represent the available high-resolution PAN image and represent the lexicographically ordered high-resolution MS images (each N²x1 of pixels) and are the corresponding vectors (M²x1) containing pixels from the low-resolution observations, then we can write:

where, D the decimation matrix of size M²xN² and v_m is the M₂x1 noise vector that assume to be zero-mean i.i.d. process with variance . Although, there are numerous methods to estimate the decimation matrix, in this study, we assume the decimation is fixed in this study, because the estimation of the decimation matrix is not the key point of this study.

An example for a decimation factor of q = 2 and with z_m of the size 16x1, can be written as:

Thus, the low-resolution pixel is the average of high resolution pixels over a neighborhood of q² corrupted with additive noise. The decimation process is represented by the matrix D.

Our problem is now reduced to estimating z_m given y_m, m = 2,3,…, p and z₁.The problem is an ill-posed inverse problem, because the matrix D is not invertible. Therefore, obtaining a robust solution requires a reasonable assumption about the nature of the true image, which is called the prior information. Once the prior model for the true image is chosen, the properties of the model only depend on the model parameters. However, the parameters of the prior model are unknown as the true high resolution MS images are unavailable and have to be estimated. In this study, the parameters are learnt from the available high-resolution PAN observations. In other words, we assume the spatial correlation for the unknown high-resolution MS images can be estimated from the available high-resolution PAN data. In this study, we propose a prior model using wavelet-domain hidden Markov tree.

Multispectral image fusion using wavelet-domain hidden markov tree: According to maximum a posterior principle , can be estimated by maximizing . Using Bayes theorem, we have:

Taking the logarithm of (4), can be obtained via:

As Eq. 2 shows, can be written as:

is characterized by hidden Markov tree model:

where, w_m is the corresponding wavelet coefficient of z_m w_im and s_im are ith wavelet coefficient and state variable of z_m. Due to the complexity of Eq. 7, some simplification will be made. First let consider when s_im = t:

where, is the variance of wavelet coefficient w_im when s_im = t. If s_im = 1 denotes that the wavelet coefficient is small, then. In other words, when s_im = 1, the value of the wavelet coefficients are suppressed. We can simplify Eq. 7 as follows:

Finally, in order to avoid the ringing effect, we regularize the restoration with Gaussian-Markov random field similar to (Joshi and Jalobeanu, 2010), whose energy function is:

where,

denote the value of pixel at coordinate (i, j) of the image z_m, is the parameters learnt from z₁:

Combining Eq. 6, 9 and 10 the optimization formula can be written as:

W is the wavelet transform matrix. Due to the linearity of the wavelet transform Eq. 19 is a quadratic programming problem. Because the size of this system is extremely large, an iterative approach called conjugate gradient algorithm (Shewchuk, 1994) is used to solve this problem. In addition, due to the scale of the system, the direct product the wavelet transform matrix W and decimation matrix D with a vector is too slow and time-consuming. In this study, we employ fast wavelet transform (Mallat, 1999) to calculate the product W of W^T or\with any vector and compute the product of D with a vector with convolution.

EXPERIMENTAL RESULTS

Experimentation on synthetically generated image: Here, we consider the verification of our model for the proposed fusion method by considering the synthetically generated data. To start with, we generate a checkerboard image and treat it as a high-spatial-resolution image. The observed low-resolution image is then formed by decimation and addition of an i.i.d Gaussian noise of the variance 0.0004. The high-resolution image is then estimated from the degraded version. Fig. 2a shows the original high-resolution image. In Fig. 2b, we show the decimated and noisy version of the same image. A decimation factor of two was used and the decimation is obtained via the decimation matrix similar to Eq. 3. Figure 2c shows the estimated high-resolution image using the IGMRF-based approach.

It can be seen that, although, the estimated image exhibits lower noise, there is still considerable artifacts in the center of each box. The MSE between the reconstructed image and the real image is 7.24x0^-5. Figure 2d is the reconstructed image using the proposed algorithm. The fused image is free of ringing effect or noticeable noise. The MSE between the fused image and the real image is 5.0x0^-5. The experiment demonstrates the working of our model and also clarifies that the method is capable of performing image fusion. Moreover, our algorithm is proved to be robust to noise.

Experimental illustration with a single image: The objective of this section is to test the efficacy of the proposed method when we consider a real satellite images as the test image and try to recover this image from its degraded version. We choose a single panchromatic image of the Quickbird image of the Missisipi University. The satellite image is shown in Fig. 3a. The observed image Fig. 3b is obtained via decimation and adding the noise with variance 0.0025. Figure 3c indicates the image obtained by IGMRF-based algorithm. There is noticeable noise in the image. The MSE between the fused image and real image is 0.0012. Figure 3d is the image reconstructed with the proposed algorithm. The image has less noise and has MSE with the real image 7.60x10^-4. However, the image fused with our algorithm lose some detailed information in the baseball court.

In addition, we compare the robustness of the proposed algorithm with IGMRF-based method. Adding noise with variance from 0.0001 to 0.01 to the image, we obtain the relationship between the MSE of the estimated image with the real image and the variance of the noise for both method in Fig. 4.

It can be seen that the proposed algorithm exhibit better robustness to noise than IGMRF-based algorithm does.

Experimentations on quickbird images: Here, we compare the performance of various methods for multispectral fusion. We employ both the subjective and objective evaluation criteria. Each evaluation criteria has their own advantage. The objective evaluation criteria provide normalized and consistent metric for the image fusion performance. However, the human eyes have unparalleled sense of perception to color.

Therefore, this comparison is performed both visually and quantitatively using the following indicators.

Standard Deviation of Difference image (SDD): The SSD is the standard variance of the difference image, relative to the mean of the original image, defined as follows:

The smaller the SSD is, the better the quality of the merged image.

Correlation coefficient (CC): CC is the correlation coefficient between the original and the merged images. It should be as close to 1 as possible:

where, f is the merged image, is its mean value, g is the original high-resolution multispectral image, represents for the mean of the multispectral image.

Difference coefficient (D): Difference coefficient is the difference between the means of the original and the merged image (bias), in radiance as well as its value relative to the mean of the original image (RM). The smaller these differences are, the better the spectral quality of the merged image:

where, fuse is the merged image, ms is the original high-resolution multispectral image, m, n are the width and height of the merged image, respectively.

The high-resolution panchromatic image is shown in Fig. 5a and the low-resolution multispectral image is presented in Fig. 5b. We have performed registration before fusion. The image reconstructed with cubic interpolation is illustrated in Fig. 5c. It can be seen that the merged image is very blurry.

The image merged with IHS-based method in Fig. 5d suffers from significant color distortion. Figure 5e is the image fused with IGMRF model-based approach. Although, the merged image’s color is relatively better, the border of the image has distortion. In addition, the details such as the branch of the river are blurred. In comparison, the image in Fig. 5f, merged with the proposed method has the best quality. Although, the fused image has small distortion at the border, it preserves color and detail information very well. Table 1 compares the quantitative quality metrics of various fusion approaches. It can be seen that the proposed method produces images with the best spectral and spatial quality.

CONCLUSIONS

This study proposes a multispectral image fusion algorithm using wavelet-domain Markov tree model. The wavelet-domain Markov tree model and the IGMRF model learnt from the high-resolution PAN image are used as prior model to regularize the super-resolution restoration of the multispectral image. The experiments show that our algorithm is more robust to additive noise than the algorithm using IGMRF as prior model. Moreover, the fused images are shaper and retain better color than traditional algorithms.

ACKNOWLEDGMENTS

The study reported in this study was funded under the National Natural Science Foundation of China under Grant No. 60872059 and the National Basic Research Program of China (2011CB302900).

REFERENCES