wavelet-based em algorithm for multispectral-image restoration

7
3892 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 47, NO. 11, NOVEMBER 2009 Wavelet-Based EM Algorithm for Multispectral-Image Restoration Arno Duijster, Paul Scheunders, and Steve De Backer Abstract—In this paper, we present a technique for the restora- tion of multispectral images. The presented procedure is based on an expectation–maximization (EM) algorithm, which applies iteratively a deconvolution and a denoising step. The restoration is performed in a multispectral way instead of band-by-band. The deconvolution technique is a generalization of the EM-based grayscale-image restoration and allows for the reconstruction of spatial as well as spectral blurring. The denoising step is per- formed in wavelet domain. To account for interband correlations, a multispectral probability density model for the wavelet coef- ficients is chosen. Rather than using a multinormal model, we opted for a Gaussian scale mixture model, which is a heavy-tailed model. Also in this paper, the framework is extended to include an auxiliary image of the same scene to improve the restoration. Ex- periments on Landsat and AVIRIS multispectral remote-sensing images are conducted. Index Terms—Denoising, expectation–maximization (EM), Gaussian scale mixture (GSM), multispectral images, restoration. I. I NTRODUCTION R EMOTE-SENSING images are subject to degradation caused by different kinds of atmospheric effects and physical limitations of the sensors. The degradation is apparent as blurring, affecting the spatial resolution and noise added on top. The goal of image restoration is to recover the original image. In case of multispectral images, each of the bands is degraded, in general, with a different blurring and noise level. In fact, the blurring need not to be spatial invariant and can also be blurring in the spectral direction. The straightforward way to restore a multispectral image is to restore each band separately. However, this may destroy the spectral information that is contained in the multispectral image. In addition, spectral blurring cannot be treated in this way. Moreover, since the different bands are, in general, highly correlated, a multispectral approach that exploits this correlation is favorable. There exists a vast literature on grayscale-image restoration. Usually, the image degradation is described by a linear space- invariant convolution (blurring) operator and additive Gaussian noise. Inversion of the blurring operator is generally done in Fourier domain, requiring regularization (denoising) to avoid the singularities. Multispectral (multichannel) image restora- Manuscript received October 1, 2008; revised April 16, 2009 and August 7, 2009. First published October 9, 2009; current version published October 28, 2009. This work was supported by the Flemish Interdisciplinary Institute for Broadband Technology (IBBT). The authors are with the Interdisciplinary Institute for Broadband Technol- ogy (IBBT), Vision Lab, Department of Physics, University of Antwerp, 2610 Wilrijk, Belgium (e-mail: [email protected]; [email protected]; [email protected]). Digital Object Identifier 10.1109/TGRS.2009.2031103 tion has been performed as well using similar strategies. A straightforward multiband-restoration approach is to transform the multiband image that spectrally decorrelates the channels and restore the decorrelated images in a single-band fashion [1]. This will only work if no spectral blur is present. Multiband versions of linear methods such as Wiener filtering [2] and least squares restoration [3] have been proposed. In [4], the authors present a generalization of the frequency domain, allowing a generalization of frequency-domain single-band deconvolution techniques. In [5], a Bayesian maximum a posteriori estimation has been proposed. In this paper, the Gibbs priors constrain both spatial and spectral components. Restoration is known to be an ill-posed inverse problem, since the deconvolution requires the inverse of the blurring operator, which may be nearly singular or even not exist, re- sulting in magnification of noise. A disadvantage of the Fourier transform is that it does not efficiently represent image edges. Therefore, only small amounts of regularization are allowed to avoid blurring of the edges in the image. In the wavelet domain, on the other hand, this problem is avoided, since edges are represented by large coefficients which are better retained after regularization. This property makes the wavelet transform suited for image denoising, which has been demon- strated in a countless number of effective denoising schemes [6]. Therefore, for restoration, it is advantageous to separate the deconvolution and the denoising problems. In the recent literature, this strategy has been applied, and several solutions for the deconvolution problem have been formulated. In [7], a technique, referred to as ForWaRD, ap- plies a Wiener deconvolution followed by a wavelet shrink- age. Several iterative deconvolution methods were proposed based on the expectation–maximization (EM) algorithm [8], or generalizations of it [9], and the more recently developed majorization–minimization [10] and TwIST algorithms [11]. The EM algorithm is an iterative procedure developed to max- imize the likelihood function corresponding to the observation model. Each iteration consists of two steps: an expectation step (E-step) solving the deconvolution and a maximization step (M-step) managing the noise. To our knowledge, the problem of multispectral-image restoration using recent strategies has not been handled in the literature. In this paper, we propose a multispectral-image restoration technique by extending the recent restoration strate- gies based on the EM algorithm toward multispectral images. The procedure is constructed as follows. 1) The E-step is formulated by extending the spatial blurring operator toward an operator, allowing for spatial as well as spectral blurring. 0196-2892/$26.00 © 2009 IEEE

Upload: s

Post on 24-Sep-2016

216 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Wavelet-Based EM Algorithm for Multispectral-Image Restoration

3892 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 47, NO. 11, NOVEMBER 2009

Wavelet-Based EM Algorithm forMultispectral-Image Restoration

Arno Duijster, Paul Scheunders, and Steve De Backer

Abstract—In this paper, we present a technique for the restora-tion of multispectral images. The presented procedure is basedon an expectation–maximization (EM) algorithm, which appliesiteratively a deconvolution and a denoising step. The restorationis performed in a multispectral way instead of band-by-band.The deconvolution technique is a generalization of the EM-basedgrayscale-image restoration and allows for the reconstruction ofspatial as well as spectral blurring. The denoising step is per-formed in wavelet domain. To account for interband correlations,a multispectral probability density model for the wavelet coef-ficients is chosen. Rather than using a multinormal model, weopted for a Gaussian scale mixture model, which is a heavy-tailedmodel. Also in this paper, the framework is extended to include anauxiliary image of the same scene to improve the restoration. Ex-periments on Landsat and AVIRIS multispectral remote-sensingimages are conducted.

Index Terms—Denoising, expectation–maximization (EM),Gaussian scale mixture (GSM), multispectral images, restoration.

I. INTRODUCTION

R EMOTE-SENSING images are subject to degradationcaused by different kinds of atmospheric effects and

physical limitations of the sensors. The degradation is apparentas blurring, affecting the spatial resolution and noise added ontop. The goal of image restoration is to recover the originalimage. In case of multispectral images, each of the bands isdegraded, in general, with a different blurring and noise level.In fact, the blurring need not to be spatial invariant and canalso be blurring in the spectral direction. The straightforwardway to restore a multispectral image is to restore each bandseparately. However, this may destroy the spectral informationthat is contained in the multispectral image. In addition, spectralblurring cannot be treated in this way. Moreover, since thedifferent bands are, in general, highly correlated, a multispectralapproach that exploits this correlation is favorable.

There exists a vast literature on grayscale-image restoration.Usually, the image degradation is described by a linear space-invariant convolution (blurring) operator and additive Gaussiannoise. Inversion of the blurring operator is generally done inFourier domain, requiring regularization (denoising) to avoidthe singularities. Multispectral (multichannel) image restora-

Manuscript received October 1, 2008; revised April 16, 2009 and August 7,2009. First published October 9, 2009; current version published October 28,2009. This work was supported by the Flemish Interdisciplinary Institute forBroadband Technology (IBBT).

The authors are with the Interdisciplinary Institute for Broadband Technol-ogy (IBBT), Vision Lab, Department of Physics, University of Antwerp, 2610Wilrijk, Belgium (e-mail: [email protected]; [email protected];[email protected]).

Digital Object Identifier 10.1109/TGRS.2009.2031103

tion has been performed as well using similar strategies. Astraightforward multiband-restoration approach is to transformthe multiband image that spectrally decorrelates the channelsand restore the decorrelated images in a single-band fashion[1]. This will only work if no spectral blur is present. Multibandversions of linear methods such as Wiener filtering [2] and leastsquares restoration [3] have been proposed. In [4], the authorspresent a generalization of the frequency domain, allowing ageneralization of frequency-domain single-band deconvolutiontechniques. In [5], a Bayesian maximum a posteriori estimationhas been proposed. In this paper, the Gibbs priors constrain bothspatial and spectral components.

Restoration is known to be an ill-posed inverse problem,since the deconvolution requires the inverse of the blurringoperator, which may be nearly singular or even not exist, re-sulting in magnification of noise. A disadvantage of the Fouriertransform is that it does not efficiently represent image edges.Therefore, only small amounts of regularization are allowedto avoid blurring of the edges in the image. In the waveletdomain, on the other hand, this problem is avoided, sinceedges are represented by large coefficients which are betterretained after regularization. This property makes the wavelettransform suited for image denoising, which has been demon-strated in a countless number of effective denoising schemes[6]. Therefore, for restoration, it is advantageous to separate thedeconvolution and the denoising problems.

In the recent literature, this strategy has been applied, andseveral solutions for the deconvolution problem have beenformulated. In [7], a technique, referred to as ForWaRD, ap-plies a Wiener deconvolution followed by a wavelet shrink-age. Several iterative deconvolution methods were proposedbased on the expectation–maximization (EM) algorithm [8],or generalizations of it [9], and the more recently developedmajorization–minimization [10] and TwIST algorithms [11].The EM algorithm is an iterative procedure developed to max-imize the likelihood function corresponding to the observationmodel. Each iteration consists of two steps: an expectation step(E-step) solving the deconvolution and a maximization step(M-step) managing the noise.

To our knowledge, the problem of multispectral-imagerestoration using recent strategies has not been handled inthe literature. In this paper, we propose a multispectral-imagerestoration technique by extending the recent restoration strate-gies based on the EM algorithm toward multispectral images.The procedure is constructed as follows.

1) The E-step is formulated by extending the spatial blurringoperator toward an operator, allowing for spatial as wellas spectral blurring.

0196-2892/$26.00 © 2009 IEEE

Page 2: Wavelet-Based EM Algorithm for Multispectral-Image Restoration

DUIJSTER et al.: WAVELET-BASED EM ALGORITHM FOR MULTISPECTRAL-IMAGE RESTORATION 3893

2) In the M-step of the iterative procedure, a state-of-the-artmultispectral-image denoising procedure is introduced.The process of wavelet-based multispectral-image de-noising has been treated recently [12]–[14]. In theseworks, multivariate probability density functions (pdfs)of the images were proposed that account for the correla-tions between the image bands. In particular, heavy-tailedmodels were found to be efficient. These models were ap-plied as priors in a Bayesian framework. We will employthis strategy in our EM-based restoration procedure. Inthe M-step, a Bayesian framework is introduced, includ-ing a multivariate prior model for the pdf of the noise-freeimage. As a prior model, we use a multivariate Gaussianscale mixture (GSM) [15], which describes the distribu-tion of the multispectral wavelet coefficients accurately.

3) Recently, a multispectral-image denoising algorithm hasbeen proposed [16], where, within the same Bayesianframework, extra prior information was included in theform of a coregistered noise-free single-band image. Wewill introduce this extension as well in the M-step of ourrestoration procedure.

The EM-based restoration techniques, as well as thedenoising procedure, all assume that the blurring functionsand the noise variances are known. The proposed multispectralrestoration procedure is validated on real multispectral data,with simulated blurring and noise, allowing for a quantitativevalidation. The presented restoration procedure is compared tosingle-band restoration, where each band of the multispectralimage is restored separately using the EM-restoration techniqueof [8].

The remainder of this paper is organized as follows. In thenext section, the EM restoration algorithm is formulated inthe context of multispectral-image restoration. In Section III,the E-step is explained, while in Section IV, the M-step is dealtwith. Experiments to validate the procedures are conducted inSection V.

II. EM RESTORATION

The observed multicomponent image Y with M pixels andK bands is denoted as follows: Y = [YT

1 ,YT2 , . . . ,YT

M ]T,where Ym = [Y1,m, Y2,m, . . . , YK,m]T expresses the spectralresponse at each spatial position m. Y consists of an unknownsignal S, which is first degraded with a known impulse re-sponse, represented by a linear system H, and then corruptedby some additive noise

Y = HS + N. (1)

Remark that the degradation is not limited to be purelyspatial, which means that blurring in the spectral direction isallowed as well. In addition, in general, the impulse responseis not necessarily spatial or spectral invariant but is allowed tovary from position to position. Usually, the operator is limitedto be spatial invariant to reduce the complexity. We will dolikewise. N is zero-mean white Gaussian noise with covarianceCn, with a pdf denoted by p(N) = φ(0,Cn). The noise isassumed to be translation invariant; the covariance describes itsspectral variation.

We extend the EM procedure of [8] toward multispectralimages. The idea is to perform the deblurring and denoisingin two separate steps. To do so, (1) is decomposed as

Y =HX + N2 (2)

X =S + N1 (3)

where HN1 + N2 = N with p(N1) = φ(0,Cn) and p(N2) =φ(0,Cn − HCnHT ). The spatial invariance of H guaranteesa semipositive-definite covariance for N2. If H would notbe translation invariant, a rescaling is required. In this way,the noise is decomposed into two independent parts. In [8],Figueiredo and Nowak introduce a more general decompositionas αHN1 + N2 = N. The balance between both noise com-ponents is defined by α, with the restriction 0 ≤ α ≤ 1. In theexperimental section in [8], it is stated that α = 1 leads to thebest results (for a more detailed explanation, we refer to [8]).The specific choice of p(N1) guarantees that N1 is Gaussianwhite noise, which makes (3) a simple denoising problem. Inthe first problem (2), this noise component N1 is embeddedin X, so that only N2 remains. We assume that N2 is smallenough to be neglected. As a result, the original problem hasbeen decoupled into a deblurring problem (2) and a denoisingproblem (3).

The EM algorithm is an iterative procedure developed tomaximize the complete-data likelihood function. The goal is tofind an estimation S which maximizes the complete-data pdf

S = arg maxS

p(Y,X) ∝ arg maxS

p(Y,X|S)p(S) (4)

where Y is the observed data and X is missing data.At each iteration k, the EM algorithm contains two steps. In

the first step, the E-step, a likelihood, the so-called Q-function,is estimated as

Q(S, S(k−1)

)= E

[log (p(Y,X|S)p(S)) |Y, S(k−1)

](5)

which depends on the observed image Y and an estimate S ofthe previous iteration. The second step, the M-step, maximizesthis Q-function and calculates a new maximum-likelihoodestimate

S(k) = arg maxS

[Q

(S, S(k−1)

)]. (6)

III. STEP 1: E-STEP

Using Bayes’ rule and the independence of Y on S, when con-ditioned on X, the conditioned pdf from (5) can be written as

log p(Y,X|S) = log [p(Y|X,S)p(X|S)]= log [p(Y|X)p(X|S)]

∝ − 12(S − X)TC−1

n (S − X) (7)

omitting all terms not depending on S.To solve (7), an estimate of X is required. The pdf

p(X|Y, S(k−1)) is easily shown to be ∝ p(Y|X)p(X|S(k−1)).Following (2) and (3), p(Y|X) = φ(Y − HX,Cn −HCnHT) and p(X|S(k−1)) = φ(X − Sk−1,Cn). After some

Page 3: Wavelet-Based EM Algorithm for Multispectral-Image Restoration

3894 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 47, NO. 11, NOVEMBER 2009

calculation, the MAP estimation of this product is obtained asfollows:

X(k) = E

[X|Y, S(k−1)

]

= S(k−1) + HT(Y − HS(k−1)

)(8)

which gives an estimate of the missing data X. In fact, solvingthis equation is solving the deblurring problem of (2). As aninitial value, S(0) = Y can be used.

IV. STEP 2: M-STEP

A new estimate S(k) is then calculated as

S(k) =arg maxS

[−1

2

(S−X(k)

)T

C−1n

(S−X(k)

)−log p(S)

].

(9)

Solving this equation is solving the denoising problem of (3).In this section, we discuss how to solve (9) and, thus, the

denoising problem. This comes down to choosing a proper priorpdf for the signal S.

For denoising of multispectral images, making use of inter-band correlations is essential. An image discontinuity (e.g., anedge, corner, line, point target, etc.) appearing in one imageband is likely to appear in at least some of the remaining bands.Proper use of interband correlations facilitates discriminationbetween noise and image features and even reveals imagedetails that were “hidden” by noise in given bands or discardsfalse structures generated by noise. Thus, the prior pdf shouldbe a multivariate pdf.

The best representation in which to estimate the noise-freesignal is the wavelet transform.

The wavelet transform offers an efficient representation ofspatial discontinuities within each spectral band [17]–[19].It compresses the essential information of an image into arelatively few large coefficients coinciding with the positionsof image discontinuities. Such a representation naturally facil-itates the construction of spatially adaptive denoising methodsthat can smoothen noise without excessive blurring of imagedetails.

The wavelet transform reorganizes image content into alow-resolution approximation and a set of details of differentorientations and different resolution scales. A fast algorithmfor the discrete wavelet transform is an iterative filter bankalgorithm of Mallat [18], where a pair of high- and low-passfilters followed by downsampling by two is iterated on the low-pass output. In the nondecimated wavelet transform that weconsider in this paper, downsampling is excluded, and instead,the filters are upsampled at each decomposition stage.

Because of the linearity of the wavelet transform, (3) holdsfor each subband at each specific resolution scale. In the re-mainder of this paper, the wavelet-transformed signal is denotedby s. Subscripts indicating a specific subband and resolutionscale are omitted, because equivalent processing is applied toall subbands and scales.

To solve the complete problem, a prior distribution p(s) ofthe wavelet coefficients is required. Depending on the choiceof the prior, (9) leads to a specific MAP estimate. A simple

multivariate Gaussian prior model accounts for the multispec-tral covariance, but it assumes that the marginal densities forthe wavelet coefficients are Gaussian distributed. It is wellknown that this assumption is not justified: The marginals aresymmetric and zero-mean but heavier tailed than Gaussians.For grayscale images, different priors were proposed to betterapproximate the marginal densities. Among them are general-ized Laplacian models [8], [20] and GSMs [15]. In this paper,we apply a multivariate GSM.

The GSM prior models the pdf p(s) by a mixture ofGaussians

p(s) =∫

p(s|z)p(z)dz (10)

where p(z) is the mixing density and p(s|z) is a zero-meanGaussian with covariance Cs|z . Using the GSM prior, theadditive-noise model becomes

x = s + n =√

zu + n (11)

where both u and n are zero-mean Gaussians, with covariancesgiven by Cu and Cn, respectively. Then, Cs|z = zCu or, bytaking expectations over z with E(z) = 1, Cs = Cu.

Since it is difficult to calculate the MAP estimate in the caseof a GSM prior, the mmse estimate will be used and is given by(we omit the superscript (k) in the estimation of s and replacex(k) by x)

s =∫

sp(s|x)ds

=

∞∫0

p(z|x)E(s|x, z)dz. (12)

The posterior distribution of z can be obtained, using Bayes’rule, as

p(z|x) =p(x|z)p(z)∫ ∞

0 p(x|α)p(α)dα. (13)

and since the GSM model s conditioned on z is Gaussian, theexpected value within the integral is given by a Wiener estimate

E(s|x, z) = zCu(zCu + Cn)−1x. (14)

Portilla et al. [15] motivate the use of the so-called Jeffrey’sprior for the random multiplier z as p(z) ∝ z−1.

Recently, in [16], a denoising method for multispectralimages was proposed, where an extra noise-free image wasassumed to be available and included as extra prior information.

The use of an auxiliary image was motivated by the researchon image fusion in remote sensing, where images of differentsensors are combined. The correlation between the observedand the extra image was employed to improve the denoising.Since the framework, which is proposed in this paper, is similarto the framework in [16], the extension using an extra imagecan easily be included.

Suppose that the extra image is a noise-free panchromaticimage A with wavelet coefficients a. The mmse estimate (12)

Page 4: Wavelet-Based EM Algorithm for Multispectral-Image Restoration

DUIJSTER et al.: WAVELET-BASED EM ALGORITHM FOR MULTISPECTRAL-IMAGE RESTORATION 3895

can now be written as

s =

∞∫0

p(z|x, a)E(s|x, a, z)dz. (15)

Based on the assumption that the observation and the auxil-iary image are jointly GSM distributed, an expression is derivedin [16] for the first and second terms in the integral

p(z|x, a) =p(x|z, a)p(z)∫ ∞

0 p(x|α, a)p(α)dα(16)

E(s|x, a, z) =Cs|a,z(Cn + Cs|a,z)−1x

+[1 − Cs|a,z(Cn + Cs|a,z)−1

]μs|a. (17)

where

μs|a,z =Cs,aC−1a a

Cs|a,z = z(Cs − Cs,aC−1

a CTs,a

). (18)

For further information about the practical implementationof the determination of (12) and (15), we refer to [15] and [16].

Remark that the mean of the images is preserved. Blurringdoes not change the mean, so the mean of X(k) is exactly thesame as the mean of S(k−1) in (8). During the M-step, only thesubbands with detail coefficients are denoised, while the coarseimage is kept unchanged.

Algorithm 1 Pseudocode of the multispectral EM algorithm1: Set S(0) = Y2: for k = 1 to stop condition do3: Compute X(k) using (8) in Fourier domain4: Decompose X(k) into wavelet subbands5: Compute σn using (20)6: for each subband except low-pass residual do7: Compute Cs using (19)8: if an auxiliary image exists then9: for each z do

10: Compute p(z|x, a) using (16)11: Compute E(s|x, a, z) using (17)12: end for13: Compute s(k) using (15)14: else15: for each z do16: Compute p(z|x) using (13)17: Compute E(s|x, z) using (14)18: end for19: Compute s(k) using (12)20: end if21: end for22: Reconstruct S(k) from subbands and low-pass residual23: end for

V. EXPERIMENTS AND DISCUSSION

A. Implementation

Pseudocode of the proposed method is shown in algorithm 1.For our experiments, it is assumed that the spatial blurring isthe same for each spectral band and that both the spatial and thespectral blurring functions are circulant. The calculation of (8)

Fig. 1. ISNR in function of the number of iterations.

can now easily be done in the Fourier domain using a 3-D fastFourier transform.

For the wavelet decomposition, a four-level decomposition(J = 4) of the nondecimated wavelet transform, as describedin Section IV, is used. For the wavelet filters, we use theDaubechies symmlets family [17] of length six.

To calculate (9), all estimations of the covariance matrix Cs

are obtained from an estimation of x, obtained from (8), in thefollowing way:

Cs = Cx − Cn. (19)

Since Cs is a covariance matrix, it needs to be semipositivedefinite. This is assured by performing an eigenvalue decom-position and clipping possible negative eigenvalues to zero.Throughout this paper, Cx is determined over the entire image:Cx = 〈xT

l xTl 〉l, i.e., it is assumed that the interband correla-

tions are the same for all wavelet coefficients of a subband.The expressions (14) and (17) are dependent on z and should

be recalculated for each z in the integrals of (12) and (15).In [16], these expressions are rewritten in order to simplifytheir dependence on z. For the calculation of the integral overz, we follow [15], where z is sampled logarithmically, whichrequires fewer samples than linear sampling. Optimal resultswere obtained for an interval from −20.5 until 3.5, with a stepsize of two for log z (only 13 samples).

In the proposed technique, the noise variance is required.Instead of assuming that it is known, the noise variance can beestimated as follows: Let for each image component x(1,diag)

denote the wavelet coefficients of the first resolution level anddiagonal orientation. Then, the diagonal elements of the noisecovariance are estimated by the classical median estimator [6]

σ2n =

median(∣∣x(1,diag)

∣∣)0.6745

. (20)

The noise is assumed to be uncorrelated between all spectralbands.

B. Experimental Setup

The first experiment is a validation of the proposed tech-niques. For this, a blurred and noisy image is simulated froma given multispectral Landsat image (with size 512 × 512 × 6).First, spatial-invariant blurring is applied. The first five bands ofthe Landsat image are degraded, while the sixth band is used as

Page 5: Wavelet-Based EM Algorithm for Multispectral-Image Restoration

3896 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 47, NO. 11, NOVEMBER 2009

Fig. 2. Part (120 × 210 × 3) of the restored multispectral Landsat image. (From left to right) Original and the degraded image and three restored images, usingthe original EM algorithm and the proposed method without and with an auxiliary image, respectively.

Fig. 3. Detail from the restored multispectral Landsat image in Fig. 2. (From left to right) Original and the degraded image and three restored images, using theoriginal EM algorithm and the proposed method without and with an auxiliary image, respectively.

the auxiliary image. For each band, a Gaussian blurring kernelis applied with a standard deviation σa = 0.8 pixels sampled ona 5 × 5 window, and Gaussian white noise (σ2

n = 90) is added.Since the operator H is band-specific in this case, the

M-step is the only multicomponent step in the proposedprocedure. We compare the original EM algorithm with ourmultispectral restoration method. Furthermore, the use of theauxiliary image is validated.

Therefore, the three compared restoration methods are asfollows:

1) the original restoration technique of [8] on each of thebands separately;

2) the proposed multispectral algorithm;3) the proposed multispectral algorithm, with an auxiliary

image.

In Fig. 1, the improved signal-to-noise ratios (ISNRs)are shown for this experiment in function of the number ofiterations of the EM algorithm. The ISNR is defined as thePSNR of the restored image minus the PSNR of the degradedimage. Error bars, obtained by repeating the experiment overdifferent noise realizations, are negligible. As shown, themultispectral EM algorithm outperforms the original bandwise

method and leads to a gain of 0.7 dB, while an auxiliary imageincreases the performance with another 0.6 dB. After a fewiterations, the curves drop due to artifacts, mainly caused byneglecting N2 in (2).

In Figs. 2 and 3, the restored images are shown (as a colorcomposite image of bands 1–3). As shown, the restored imageusing the original EM algorithm contains more noise than theother images. Using the proposed algorithm, the amount ofnoise is clearly reduced. The sharpness of the restored images isbetter for the proposed restoration method and is highest whenan auxiliary image is used.

In the second experiment, our algorithm is validated whenthere is also spectral blurring involved. Different image degra-dations are simulated on 80 bands of the AVIRIS image “IndianPine.” For each band, a Gaussian blurring kernel is applied witha standard deviation σa varying from 0 to 1.6 pixels to simulatethe spatial blurring. Furthermore, a Gaussian blurring kernelwith a standard deviation σe between 0 and 6.8 bands is appliedto simulate the spectral blurring. Finally, some Gaussian whitenoise (σ2

n = 10) is added.The E-step (8) is now fully 3-D, because the blurring function

H contains interband blurring. In this restoration experiment,only the first two methods as mentioned in the first experiment

Page 6: Wavelet-Based EM Algorithm for Multispectral-Image Restoration

DUIJSTER et al.: WAVELET-BASED EM ALGORITHM FOR MULTISPECTRAL-IMAGE RESTORATION 3897

TABLE ISAM IN DEGREES AND PSNR IN DECIBELS. THE VALUES FOR THE DEGRADED IMAGE AND THE RESTORED

IMAGES USING THE ORIGINAL METHOD AND THE PROPOSED MULTISPECTRAL METHOD

are compared: the original bandwise restoration versus a multi-spectral restoration.

To compare the results of the restoration, the spectral anglemapper (SAM) is used, which is a measure of the differencebetween two spectra. The SAM is defined as

SAM(A,B) = arccosA · B

‖A‖ ‖B‖ . (21)

In Table I the SAM and the PSNR are shown for thisexperiment. The differences between the two methods are mostclearly visible when the blurring kernels are small. As shown,the multispectral restoration method always has a lower SAMthan the original bandwise method. For larger blurring ker-nels, the differences decrease, but the multispectral restorationmethod always is the best. The PSNR is a comparable measure:A decreasing SAM results in an increasing PSNR.

VI. CONCLUSION

In this paper, a restoration technique for multispectral im-ages has been presented. This multispectral procedure is basedon an iterative EM algorithm, applying alternately a decon-volution and a denoising step. The deconvolution techniqueallows for the reconstruction of spatial as well as spectralblurring. The denoising step is performed in wavelet domain,using a multispectral probability-density model for the waveletcoefficients. Instead of using a simple multinormal model, aheavy-tailed GSM model has been chosen. Furthermore, theframework is extended to include a coregistered auxiliary imageof the same scene to improve the restoration. Experimentson Landsat and AVIRIS images show that a multispectralrestoration method outperforms a bandwise method. The resultsare even better when an auxiliary image of the same scene isincorporated.

REFERENCES

[1] B. R. Hunt and O. Kübler, “Karhunen–Loeve multispectral imagerestoration—Part I: Theory,” IEEE Trans. Acoust., Speech, SignalProcess., vol. ASSP-32, no. 3, pp. 592–600, Jun. 1984.

[2] N. P. Galatsanos and R. T. Chin, “Digital restoration of multichannelimages,” IEEE Trans. Acoust., Speech, Signal Process., vol. 37, no. 3,pp. 415–421, Mar. 1989.

[3] N. P. Galatsanos, A. K. Katsaggelos, R. T. Chin, and A. D. Hillery, “Leastsquares restoration of multichannel images,” IEEE Trans. Signal Process.,vol. 39, no. 10, pp. 2222–2236, Oct. 1991.

[4] A. K. Katsaggelos, K. T. Lay, and N. P. Galatsanos, “A general frame-work for frequency domain multi-channel signal processing,” IEEE Trans.Image Process., vol. 2, no. 3, pp. 417–420, Jul. 1993.

[5] R. R. Schultz and R. L. Stevenson, “Stochastic modeling and estimationof multispectral image data,” IEEE Trans. Image Process., vol. 4, no. 8,pp. 1109–1119, Aug. 1995.

[6] D. L. Donoho and I. M. Johnstone, “Adapting to unknown smoothness viawavelet shrinking,” J. Amer. Stat. Assoc., vol. 90, no. 432, pp. 1200–1224,1995.

[7] R. Neelamani, H. Choi, and R. Baraniuk, “ForWaRD: Fourier-waveletregularized deconvolution for ill-conditioned systems,” IEEE Trans.Image Process., vol. 52, no. 2, pp. 418–433, Feb. 2004.

[8] M. A. T. Figueiredo and R. D. Nowak, “An EM algorithm for wavelet-based image restoration,” IEEE Trans. Image Process., vol. 12, no. 8,pp. 906–916, Aug. 2003.

[9] J. M. Bioucas-Dias, “Bayesian wavelet-based image deconvolution: AGEM algorithm exploiting a class of heavy-tailed priors,” IEEE Trans.Image Process., vol. 15, no. 4, pp. 937–951, Apr. 2006.

[10] M. A. T. Figueiredo, J. M. Bioucas-Dias, and R. D. Nowak,“Majorization–minimization algorithms for wavelet-based image restora-tion,” IEEE Trans. Image Process., vol. 16, no. 12, pp. 2980–2991,Dec. 2007.

[11] J. M. Bioucas-Dias and M. A. T. Figueiredo, “A new TwIST: Two-stepiterative shrinkage/thresholding algorithms for image restoration,” IEEETrans. Image Process., vol. 16, no. 12, pp. 2992–3004, Dec. 2007.

[12] A. Benazza-Benyahia and J.-C. Pesquet, “Building robust wavelet esti-mators for multicomponent images using Steins’ principle,” IEEE Trans.Image Process., vol. 14, no. 11, pp. 1814–1830, Nov. 2005.

[13] A. Pižurica and W. Philips, “Estimating the probability of the presenceof a signal of interest in multiresolution single- and multiband imagedenoising,” IEEE Trans. Image Process., vol. 15, no. 3, pp. 654–665,Mar. 2006.

[14] S. de Backer, A. Pižurica, B. Huysmans, W. Philips, and P. Scheunders,“Denoising of multispectral images using wavelet least-squaresestimators,” Image Vis. Comput., vol. 26, no. 7, pp. 1038–1051,Jul. 2008.

[15] J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, “Imagedenoising using scale mixtures of Gaussians in the wavelet domain,” IEEETrans. Image Process., vol. 12, no. 11, pp. 1338–1351, Nov. 2003.

[16] P. Scheunders and S. De Backer, “Wavelet denoising of multicomponentimages using Gaussian scale mixture models and a noise-free image aspriors,” IEEE Trans. Image Process., vol. 16, no. 7, pp. 1865–1872,Jul. 2007.

[17] I. Daubechies, Ten Lectures on Wavelets. Philadelphia, PA: SIAM, 1992.[18] S. Mallat, “A theory for multiresolution signal decomposition: The

wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 11,no. 7, pp. 674–693, Jul. 1989.

[19] M. Vetterli and J. Kovacevic, Wavelets and Subband Coding.Englewood Cliffs, NJ: Prentice-Hall, 1995.

[20] S. G. Chang, B. Yu, and M. Vetterli, “Adaptive wavelet thresholding forimage denoising and compression,” IEEE Trans. Image Process., vol. 9,no. 9, pp. 1532–1546, Sep. 2000.

Page 7: Wavelet-Based EM Algorithm for Multispectral-Image Restoration

3898 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 47, NO. 11, NOVEMBER 2009

Arno Duijster received the M.Sc. degree in appliedphysics, with work in the field of signal processingand system identification, from Delft University ofTechnology, Delft, The Netherlands, in 2004. He iscurrently working toward the Ph.D. degree in theInterdisciplinary Institute for Broadband Technology(IBBT), Vision Lab, Department of Physics, Univer-sity of Antwerp, Antwerp, Belgium.

His research interests include multispectral-imageanalysis, image restoration, and image fusion.

Paul Scheunders received the M.Sc. degree inphysics and the Ph.D. degree in physics, with work inthe field of statistical mechanics, from the Universityof Antwerp, Antwerp, Belgium, in 1983 and 1990,respectively.

In 1991, he was a Research Associate with theVision Laboratory, Department of Physics, Univer-sity of Antwerp, where he is currently a Lecturer.He has published more than 100 papers in interna-tional journals and proceedings in the field of imageprocessing and pattern recognition. His research in-

terest includes wavelets and multispectral image processing.

Steve De Backer received the B.S. degree inphysics from the University of Hasselt, Diepenbeek,Belgium, in 1994 and the M.S. and Ph.D. degrees inphysics from the University of Antwerp, Antwerp,Belgium, in 1996 and 2002, respectively.

Since then, he has been a Postdoctorate Fellowwith the Vision Lab, Department of Physics, Uni-versity of Antwerp. His research focus is on imagemodeling, pattern recognition, and hyperspectralimage analysis.