introducing dynamic prior knowledge to partially-blurred ......introducing dynamic prior knowledge...

11
Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision & Remote Sensing, Berlin University of Technology Franklinstrasse 28/29, Office FR 3-1, D-10587 Berlin {hzheng, hellwich}@cs.tu-berlin.de Abstract. The paper presents an unsupervised method for partially- blurred image restoration without influencing unblurred regions or objects. Maximum a posteriori estimation of parameters in Bayesian reg- ularization is equal to minimizing energy of a dataset for a given number of classes. To estimate the point spread function (PSF), a parametric model space is introduced to reduce the searching uncertainty for PSF model selection. Simultaneously, PSF self-initializing does not rely on su- pervision or thresholds. In the image domain, a gradient map as a priori knowledge is derived not only for dynamically choosing nonlinear diffu- sion operators but also for segregating blurred and unblurred regions via an extended graph-theoretic method. The cost functions with respect to the image and the PSF are alternately minimized in a convex manner. The algorithm is robust in that it can handle images that are formed in variational environments with different blur and stronger noise. 1 Introduction The challenge of blind image deconvolution (BID) is to uniquely define the opti- mized signals from degraded images with unknown blur information, which is an ill-posed problem in the sense of Hadamard. However, knowledge of the direct model is not sufficient to determine an existing, unique and stable solution, and it is necessary to regularize the solution using some a priori knowledge. Math- ematically, the a priori knowledge is often expressed through a regularization theory [1] which replaces an ill-posed problem by a well-posed problem with an acceptable approximation to the solution. In the real world, CCD and CMOS camera images or medical images are often blurred or partially-blurred in a stationary or non-stationary way. BID of partially-blurred images is to restore blurred regions without influencing un- blurred regions for achieving better visual perception based on the Gestalt the- ory, shown in Fig. 1. To achieve this, blurred regions or objects have to be blur identified, segregated and restored respectively. Different characteristic proper- ties [2,3] (gradient, frequency, entropy, etc.) between blurred and unblurred re- gions or objects endowed with pairwise relationships can be naturally considered as a graph. Thus, we treat BID of partially-blurred images as a combinatorial K. Franke et al. (Eds.): DAGM 2006, LNCS 4174, pp. 111–121, 2006. c Springer-Verlag Berlin Heidelberg 2006

Upload: others

Post on 05-Sep-2020

9 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Introducing Dynamic Prior Knowledge to Partially-Blurred ......Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision

Introducing Dynamic Prior Knowledge to

Partially-Blurred Image Restoration

Hongwei Zheng and Olaf Hellwich

Computer Vision & Remote Sensing, Berlin University of TechnologyFranklinstrasse 28/29, Office FR 3-1, D-10587 Berlin

{hzheng, hellwich}@cs.tu-berlin.de

Abstract. The paper presents an unsupervised method for partially-blurred image restoration without influencing unblurred regions orobjects. Maximum a posteriori estimation of parameters in Bayesian reg-ularization is equal to minimizing energy of a dataset for a given numberof classes. To estimate the point spread function (PSF), a parametricmodel space is introduced to reduce the searching uncertainty for PSFmodel selection. Simultaneously, PSF self-initializing does not rely on su-pervision or thresholds. In the image domain, a gradient map as a prioriknowledge is derived not only for dynamically choosing nonlinear diffu-sion operators but also for segregating blurred and unblurred regions viaan extended graph-theoretic method. The cost functions with respect tothe image and the PSF are alternately minimized in a convex manner.The algorithm is robust in that it can handle images that are formed invariational environments with different blur and stronger noise.

1 Introduction

The challenge of blind image deconvolution (BID) is to uniquely define the opti-mized signals from degraded images with unknown blur information, which is anill-posed problem in the sense of Hadamard. However, knowledge of the directmodel is not sufficient to determine an existing, unique and stable solution, andit is necessary to regularize the solution using some a priori knowledge. Math-ematically, the a priori knowledge is often expressed through a regularizationtheory [1] which replaces an ill-posed problem by a well-posed problem with anacceptable approximation to the solution.

In the real world, CCD and CMOS camera images or medical images areoften blurred or partially-blurred in a stationary or non-stationary way. BIDof partially-blurred images is to restore blurred regions without influencing un-blurred regions for achieving better visual perception based on the Gestalt the-ory, shown in Fig. 1. To achieve this, blurred regions or objects have to be bluridentified, segregated and restored respectively. Different characteristic proper-ties [2,3] (gradient, frequency, entropy, etc.) between blurred and unblurred re-gions or objects endowed with pairwise relationships can be naturally consideredas a graph. Thus, we treat BID of partially-blurred images as a combinatorial

K. Franke et al. (Eds.): DAGM 2006, LNCS 4174, pp. 111–121, 2006.c© Springer-Verlag Berlin Heidelberg 2006

Page 2: Introducing Dynamic Prior Knowledge to Partially-Blurred ......Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision

112 H. Zheng and O. Hellwich

optimization problem including graph partitioning [4,5], blur identification andedge-driven image restoration.

Our work relates to the deterministic edge-preserving image restoration [6,7],and nonlinear filtering techniques incorporated in variational methods [8,9,10].Since the traditional edge-preserving methods have some limitations of noiseprocessing due to the unique diffusion operator and passively edge-preservingprocesses. Likewise, most nonlinear diffusion techniques are studied for the welladapted input data with underlying geometric assumptions [11]. Recently, vari-ational regularization for image restoration [12,13,14] have been investigated.These methods are observed in underutilizing prior information. The initial-ization problem for deterministic optimization is still not progressively solved.Even if a unique solution exists, a proper initialization value is still intractable,e.g., when the cost function is non-convex, convergence to local minima oftenoccurs without proper initialization. [15] have reported that the estimates forthe PSF could vary significantly, depending on the initialization. For restoringpartially-blurred images, it is also not possible to directly apply these methods.Hence, general strategies are still needed for solving the blur problem as well asachieving better visual perception for partially-blurred image restoration.

In this paper, we propose a Bayesian based variational regularization. In Sec-tion 2, it is shown that the Bayesian approach [15,16] provides a structured wayfor introducing prior knowledge from the image domain and the PSF domain.In Section 3 and 4, alternate blur identification and edge-driven image restora-tion are discussed. The proposed blur kernel space including most existing PSFparametric models reduces the searching dimension and uncertainty efficientlyfor PSF self-learning. Thus, the self-learning PSF can be an accurate initial valuefor yielding a unique solution. Alternately, computed edge gradients are derivedas a priori knowledge for choosing image diffusion operators [13,17] in a dy-namic and continuous processing mechanism. An extended bisection-partitioningmethod for identifying and segregating blurred and unblurred regions or objectsis presented in Section 5. Different edge gradients and blur information in differ-ent regions are a priori knowledge to achieve a high perceptual quality segmen-tation according to a clustering criterion [4] without any supervision. Likewise,deconvolution and edge-driven image diffusion improve perceptual quality for re-toring partially-blurred images without influencing unblurred regions or objects.The experimental results are shown in Section 6. Conclusions are presented inSection 7.

2 Bayesian Estimation Based Variational Regularization

An observed image g in the image plane is normally an ideal image f in theobject plane degraded by two unknown factors, including linear space-invariantblur kernel (PSF) h and additive white Gaussian noise n. It can be formulatedin a lexicographic notation, g = h ∗ f +n, where ∗ denotes two-dimensional con-volution. Following a Bayesian paradigm, the ideal image f , the PSF h and theobserved image g fulfill p(f, h|g) = p(g|f, h)p(f, h)/p(g) ∝ p(g|f, h)p(f, h). Based

Page 3: Introducing Dynamic Prior Knowledge to Partially-Blurred ......Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision

Dynamic Prior Knowledge to Partially-Blurred Image Restoration 113

(a) (b) (c)

Fig. 1. (a) Original video. (b) Identified unblurred region. (c) Blurred background.

on this form, our goal is to find the optimal f and h that maximizes the posteriorp(f, h|g). F(f |h, g) = − log{p(g|f, h)p(f)} and F(h|f, g) = − log{p(g|f, h)p(h)}express that the energy cost F is equivalent to the negative log-likelihood of thedata. The priors p(f) and p(h) over the parameters are penalty terms added tothe cost function to minimize the energy cost in a regularization framework forsolving ill-posed problems [1,6,15,16]. To avoid stochastic optimization (longercomputing time)[18,19,20], we solve the optimization problem deterministically[6,7,21] in a convex manner with respect to the image and the PSF. The pro-posed variational double regularized energy functional in a Bayesian frameworkis formulated according to

F(f , h) =∫

Ω

(g − h ∗ f)2dA

︸ ︷︷ ︸fidelityTerm

+λ∫

Ω

φε(x,∇f)dA

︸ ︷︷ ︸imageSmoothing

+β∫

Ω

|∇h|dA

︸ ︷︷ ︸psfSmoothing

+γ∫

Ω

|h − hf |dA

︸ ︷︷ ︸psfLearning

where dA = dxdy. The estimates of the ideal image f and the PSF h are denotedby f and h respectively which can be iteratively alternating minimized (AM) [21].The image smoothing term is a variable exponent, nonlinear diffusion term [13].The PSF smoothing term represents the regularization of blur kernels. The flexi-bility of the last term denotes the PSF learning decision of the best-fit parametricmodel hf . The primary objective of this learning decision approach is to evaluatethe relevance of the parametric structure and integrate the information into thelearning scheme accordingly. It can adjust and incorporate the PSF parametricmodel throughout the process of blur identification and image restoration.

3 Simultaneous Model Selection and Self-initializing PSF

Generally, power spectral densities vary considerably from low frequency uniformregions to medium or high frequency discontinuities and texture regions in a realblurred image. Moreover, most PSFs exist in the form of low-pass filters. To a cer-tain degree, PSF models in numerous real blurred images can be represented asparametric models. Blur identification can be based on these characteristic prop-erties. Compared to general priors, e.g., Gibbs distribution [18,19], smoothness

Page 4: Introducing Dynamic Prior Knowledge to Partially-Blurred ......Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision

114 H. Zheng and O. Hellwich

prior [15] or maximum entropy [18], we define a set of primary parametric blurmodels in a priori model space Θ for model selection based Bayesian PSF estima-tion. It consists of the most typical blur models in Θ = {hi(θ), i = 1, 2, 3, ..., N}.hi(θ) represents the ith PSF parametric model with its defining parameters θ,and N is the number of blur types such as pillbox blur, Gaussian blur, 1D and2D linear motion blur and out-of-focus blur, etc.

Based on the model space, an unsupervised self-initializing PSF learning termcan learn a PSF parametric model according to the following energy functional,

F(h|f , g) = 12

∫Ω

(g − h ∗ f)2dA + β

∫Ω

|∇h|dA + γ

∫Ω

|h − hf |dA (1)

where the likelihood is p(g|f , h) ∝ exp{− 12

∫Ω (g − h ∗ f)2dA}, p(h) is the prior

density. Since an image represents intensity distributions that cannot take nega-tive values, the PSF coefficients are always nonnegative, h(x) ≥ 0. Furthermore,since image formation system normally do not absorb or generate energy, thePSF should fulfill

∑x∈Ω h (x) = 1.0 , x ∈ Ω and Ω ⊂ R2.

A MAP estimator is used to determine the best fit model hi(θ∗) for the es-timated PSF h in resembling the ith parametric model hi(θ) in a multivariateGaussian distribution. The subscript i denotes the index of blur kernel. hi(θ∗) =argmaxθ{(2π)

−LB2 |∑ dd|−1

2 · exp[− 12 (hi(θ) − h)T

∑ −1dd (hi(θ) − h)]}. The mod-

eling error d = hi(θ) − h is assumed to be a zero-mean homogeneous Gaus-sian distributed white noise with covariance matrix

∑dd = σ2

dI independentof image. LB is an assumed support size of blur. The PSF learning likeli-hood is computed based on mahalanobis distance and corresponding modelli(h) = 1

2exp[(hi(θ∗) − h)t∑−1

dd (hi(θ∗) − h)]. A best fit model hi(θ) for h isselected according to the Gaussian distribution and a cluster filter. We use aK-NN rule to find the estimated output blur model hf . hf is obtained fromthe parametric blur models using hf = [l0(h)h +

∑Ci=1 li(h)hi(θ)]/[

∑Ci=1 li(h)],

where l0(h) = 1 − max(li(h)), i = 1, ..., C. The main objective is to assess therelevance of current estimated blur h with respect to parametric PSF models,and to integrate such knowledge progressively into the computation scheme.

If the current blur h is close to the estimated PSF model hf , that meansh belongs to a predefined parametric model. Otherwise, if h differs from hf

significantly, it means that current blur h may not belong to the predefinedpriors. The prior space reduces the uncertainty of model selection and largelyimprove the efficiency for PSF self-initializing in practice.

4 Dynamic Edge-Driven Regularization in Image Domain

During the alternate minimization (AM) with respect to the estimates of thePSF and the image, the previous computed PSF is a priori knowledge for thenext iterative image deconvolution. However, we need diffusion operators to com-pensate and smooth the “ringing” or “staircase” effects for achieving a restored

Page 5: Introducing Dynamic Prior Knowledge to Partially-Blurred ......Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision

Dynamic Prior Knowledge to Partially-Blurred Image Restoration 115

image f with more fidelity and high quality visual perception. In the AM, theimage energy function is minimized according to the following formulation,

F(f |h, g) =12

∫Ω

(g − h ∗ f)2dA+ λ

∫Ω

φε(x,∇f)dA (2)

where the likelihood is p(g|f , h) ∝ exp{− 12

∫Ω

(g − h ∗ f)2dA}. Different frommost passively edge-preserving restoration approaches [6,7,11], the smoothingoperator p(f) ∝ exp{− ∫

Ωφε(x,∇f)dA} as a priori knowledge is extended to a

convex nonlinear diffusion functional with variable exponent [13]. The significantadvantage of this operator is its robustness with respect noise and actively edge-preserving processes in that the chosen diffusion operators oriented dynamicallyand continuously. The optimization for the cost function Eq. (2) is numericallysolved using its associated Euler-Lagrange equation, λdiv(φε(x,∇f)) + (f ∗ h−g) = 0, in Ω × [0, T ]. We indicate with div the divergence operator, and with∇ and Δ respectively the gradient and Laplacian operators, with respect to thespace variables. The Neumann boundary condition ∂f

∂n (x, t) = 0 on ∂Ω × [0, T ]and the initial condition f(x, 0) = f0(x) = g in Ω are used, n is the direc-tion perpendicular to the boundary. Based on variable exponent, linear growthfunction [13] and physical simulation [17], the diffusion operator is computed in,div(φε(x,∇f )) =

|∇f |p(x)−2︸ ︷︷ ︸Coefficient

[(p(x) − 1)Δf︸ ︷︷ ︸IsotropicTerm

+ (2 − p(x))|∇f |div(∇f

|∇f | )︸ ︷︷ ︸CurvatureTerm

+∇p · ∇f log |∇f |︸ ︷︷ ︸HyperbolicTerm

](3)

where p(x) = 1 + 11+k|∇Gσ∗I(x)|2 , when |∇f | < β and p(x) = 1, otherwise. β > 0

is a fixed value. I(x) is the observed image g(x), Gσ(x) = 1σexp(−|x|2

2σ2 ) is aGaussian filter, k > 0, σ > 0 are fixed parameters. The operator div(φε(x,∇f))is discretized with a small positive constant ε based on central differences forthe coefficient and isotropic term, minmod scheme for the curvature term, andupwind finite difference scheme developed by Osher and Sethian for curve evo-lution [17] for the hyperbolic term which can largely improve the signal-to-noiseratio and human visual perception.

The term p(x) ∈ [1, 2] is continuously computed based on the constraints ofedge gradients. In homogeneous regions, the differences of intensity between theneighboring pixels are small, p(x) → 2. The isotropic diffusion operator (Laplace)is used in such regions. In non-homogeneous regions (near discontinuities), theanisotropic diffusion filter is chosen continuously based on the gradient values1 < p(x) < 2. The reason is that the chosen discrete anisotropic operators willhamper the recovery of edges. Simultaneously, the nonlinear diffusion operatorfor piecewise image smoothing is processed during image deconvolution based ona previously estimated PSF. Finally, coupled estimation of PSFs (blur identifica-tion) and images (debluring + denoising + smoothing) are alternately optimizedapplying a stopping criteria. Hence, over-smoothing and under-smoothing neardiscontinuities are avoided on pixel level.

Page 6: Introducing Dynamic Prior Knowledge to Partially-Blurred ......Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision

116 H. Zheng and O. Hellwich

5 Segregation of Blurred and Unblurred Regions

Some significant differences (local scale [3,8], sharpness and contrast [2], andpiecewise smoothing [14]) between pairwise blurred regions and unblurred re-gions can be considered as natural prior knowledge for measuring the low-levelsimilarities for segregation using a global graph-clustering criterion.

We set up the vertices of a graph G = (V, E) into two sets A and B to mini-mize the number of cut edges, i.e., edges with one endpoint in A and the other inB, where V = {vi}n

i=1 are the vertices and E ⊆ {(vi, vj)} are the edges betweenthese vertices. V can correspond to pixels in an image or set of connected pixels.The bisection problem can be formulated as the minimization of a quadraticobjective function by means of the Laplacian matrix L = L(G) of the graph G.Let d(i) denote the degree of a vertex i, i.e., the number of vertices adjacent toi. The Laplacian matrix L can be expressed in terms of two matrices associatedwith a graph as L = D−W in positive semidefinite [22], W = {wij} is the adja-cency matrix of a graph, and D is the n×n diagonal matrix of the degrees of thevertices in G. Let x be an n-vector with component xi = 1 if i ∈ A and xi = −1 ifx ∈ B, then xT Lx = xT Dx−xT Wx =

∑ni=1 dix

2i =

∑(i,j)∈E,i∈A,j∈B (xi − xj)2.

Thus the bisection problem is equivalent to the problem of maximizing similarityof the objects within each cluster, or, find a cut edge through the graph G withminimal weight in the formulation of max(xT Wx) ⇐⇒ min(xT Lx).

Since the bisection-partitioning problem is NP-complete, we need to approx-imate this intractable problem by some relaxing constraints . Likewise, to avoidunnatural bias for partitioning out small sets of points, Shi and Malik [4] pro-posed a new measure of the disassociation between two groups. Instead of look-ing at the value of total edge weight connecting the two partitions, the cutcost is computed as a fraction of the total edge connections to all the nodesin the graph. This disassociation measure is called the normalized cut (Ncut):Ncut(A, B) = cur(A,B)

asso(A,V ) + cut(A,B)asso(A,V ) . A and B are two initial sets. Different from

the weight measurement in [4], we measure the degree of dissimilarity betweenpairwise blurred and unblurred regions based on the special characteristic prop-erties, i.e., stronger difference of edge gradients, pairwise blur and unblur. Theedge weight wij between node i and j as the product of a feature similarity

term and spatial proximity term: wij = exp−‖Q(i)−Q(j)‖2

2σI

∗ exp−‖X(i)−X(j)‖2

2σX

, if(‖X(i) − X(j)‖2) < r, and wij = 0, otherwise. Q(i) = ∇Gσ ∗ I(x) is the edgegradients and large differences between pairwise blurred and unblurred regions.Gσ is a Gaussian filter, I(x) is an input image, X(i) is the spatial location of nodei. The advantage of the suggested algorithm is its directness and simplicity. Thehigh quality segregation of blurred and unblurred regions or objects in Fig. 2 isguided dynamically by computed prior values without any supervision.

6 Experiments and Discussion

Alternate Minimization (AM) of PSF and Image Energy. To avoidthe scale problem between the minimization of the PSF and image via

Page 7: Introducing Dynamic Prior Knowledge to Partially-Blurred ......Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision

Dynamic Prior Knowledge to Partially-Blurred Image Restoration 117

Fig. 2. Left: Identified unblurred foreground walking man from blurred background.Right: Identified blurred foreground walking man from unblurred background.

steepest descent, an AM method [12,21] following the idea of coordinate descentis applied. The AM algorithm decreases complexity. The choice of regularizationparameters is crucial. We use L-curve [23] due to its robustness with respect tocorrelated noise. The global convergence of the algorithm to the local minimaof cost functions can be established by noting the two steps in the algorithm.Since the convergence with respect to the PSF and the image are optimizedalternatively, the flexibility of this proposed algorithm allows us to use con-jugate gradient algorithm for computing the convergence. Conjugate gradientmethods utilize the conjugate direction instead of local gradient to search forthe minima. Therefore, it is faster and also requires less memory when com-pared with the other methods. A meaningful measure called normalized meansquare-error (NMSE) is used to evaluate the performance of the identified blur,NMSE = (

∑x

∑y (h(x, y) − h(x, y))2)1/2/(

∑x

∑y h(x, y)), and the restored

images are measured by peak signal-to-noise ratio (PSNR) in decibels (dB)as PSNR = 10 log10(2552/MSE) with MSE = |Ω|−1

∑xi∈Ω[f(xi) − f(xi)]2,

where f is the noise-free ideal image and f is the restored image.

Denoising and Blind Deconvolution for Noisy and Blurred Images.Firstly, we have studied the importance of diffusion in the regularization basedimage deconvolution, shown in Fig. 3. The second experiments demonstrate theefficiency of the suggested edge-driven diffusion method. From visual perceptionand denoising viewpoint, our unsupervised edge-driven method favorably com-pares to some state-of-the-art methods: the TV [9], a statistic-wavelet method(GSM) [24] and a Markov random field based filter learning method (FoE) [20]using a PIII 1.8GHz PC. In Fig. 4, the structure of the restored fingerprint islargely enhanced than the original image in our method and more recognizablethan the restored image using the GSM method [24]. Fig. 5 shows the advantageof our method, while the TV method [9] has some piecewise constant effectsduring the denoising. Table 1 shows the different properties of different methodsand also shows our method outperforms most of these methods. To achieve sim-ilar results, FoE [20] needs more time. Our method (100 iter.) is faster than theTV method (30 iter.) in that our method does not over-smooth and generate re-dundant image discontinuities. The GSM [24] method is relatively faster due tothe computation in the Fourier domain. However, the GSM is only designed fordenoising. The dual-purpose edge-driven method is not only for denoising butalso for compensating the “ringing” and “staircase” effects and for protectingthe image structure and textures during the image deconvolution.

Page 8: Introducing Dynamic Prior Knowledge to Partially-Blurred ......Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision

118 H. Zheng and O. Hellwich

Fig. 3. Deconvolution with known PSF without using diffusion operator. Left: Blurredimage. Middle: Deconvolution of Gaussian blur. Right: Deconvolution of motion blur.

Fig. 4. Fingerprint denoising. Left: Cropped noisy image, SNR = 8 dB. Middle: GSMmethod[24] PSNR=27.8. dB Right: The suggested method PSNR= 28.6 dB.

Fig. 5. Denoising. Left: Unblurred noisy image, SNR=8dB, size: [256, 256]. Middle: TVmethod, PSNR = 27.1 dB. Right: Edge-driven diffusion, PSNR = 30.2 dB.

Table 1. Denoising performance of different methods on PSNR (dB)

PSNR σ = 17.5, SNR ≈ 8.7 dB, size [512,512] Iter(n) Time(s)

(dB) Lena Barbara Boats House Pepper fingerprint Number Second

Our Met. 32.26 31.25 31.01 31.85 30.61 28.81 100 600 ∼ 650

TV.[9] 31.28 26.33 29.42 31.33 24.57 27.29 30 800 ∼ 820

FoE[20] 32.11 27.65 30.26 32.51 30.42 26.41 1 ∼ 3 × 103 3 ∼ 9 × 103

GSM[24] 32.72 30.12 30.58 32.69 30.78 28.59 100 140 ∼ 180

Page 9: Introducing Dynamic Prior Knowledge to Partially-Blurred ......Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision

Dynamic Prior Knowledge to Partially-Blurred Image Restoration 119

Fig. 6. Deconvolution and denoising. Left: From top to bottom: SNR = 20dB and12dB, size: [256, 256]. Middle: L-R method with known PSF. Right: The suggestedmethod with unknown PSF.

For blind deconvolution, we compare the classical Lucy-Richardson (L-R) de-convolution method with known PSF to the suggested method with unknownPSF. A MRI image is heavily blurred with two level of noise 20 dB and 12dB, shown in the first column of Fig. 6. The noise is amplified during the L-Rdeconvolution with known PSF, shown in the middle column. In the suggestedmethod, the self-initialized PSF is iteratively parametric optimized in the AMalgorithm. Diffusion operators vary with the coefficient p(x) in the interval [1, 2]continuously. The estimated PSF supports the image smoothing coefficients pro-gressively till the best recovered image is reached, shown in the right column.From the restored images, we can observe that the low frequency regions aremore smooth while the fine details of discontinuities (high frequency regions)are preserved during the image deconvolution. The experiment demonstrates theflexibility of Bayesian based double regularization method which can accuratelyidentify the blur and restore images using edge-driven nonlinear image opera-tors. The results also show that the denoising and debluring can be achievedsimultaneously even under the presence of stronger noise and blur.

Identification and segregation of partially-blurred, noisy images or video se-quences have good performance using the suggested method, shown in Fig. 1and Fig. 2. Cluttering blurred or unblurred background does not influence thesegmentation and identification of unblurred or blurred objects. These objectboundaries with different computing weights are grouped into different groupsvia the extended global cluster criterion with blur and edge priors. The seg-mentation results are labeled and color filled following the partitioned regions.The experimental results show that the method yields encouraging results underdifferent kinds and amounts of noise and blur. We applied our method to all

Page 10: Introducing Dynamic Prior Knowledge to Partially-Blurred ......Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision

120 H. Zheng and O. Hellwich

the related images and video data, but due to space limitations we refer theinterested reader to our web-page www.cv.tu-berlin.de/∼hzheng/dagm06.html.

7 Conclusions

This paper validates the hypothesis that the challenging task of nonlinear dif-fusion and BID are tightly coupled in a variational regularized Bayesian esti-mation. Firstly, it provides a statistic self-initializing value in regularization forblur identification. Secondly, it shows a theoretically and experimentally soundway of how local diffusion operators are changed dynamically via a priori knowl-edge of edge gradients. The estimated PSF is the prior knowledge for the nextiteration of image estimation in the alternating minimization, and vice versa. Fi-nally, a graph-theoretical approach is extended to segregate and identify blurredand unblurred regions or objects. The integrated approach also demonstratesthat the mutual supports between natural prior knowledge and low-level imageprocessing have great potential role to improve the results in early vision.

References

1. Tikhonov, A., Arsenin, V.: Solution of Ill-Posed Problems. Wiley, Winston (1977)2. Luxen, M., Forstner, W.: Characterizing image quality: Blind estimation of the

point spread function from a single image. In: PCV02. (2002) 205–2113. Elder, J.H., Zucker, S.W.: Local scale control for edge detection and blur estima-

tion. IEEE Trans. on PAMI 20 (1998) 699–7164. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. on PAMI

8 (2000) 888–9055. Keuchel, J., Schnorr, C., Schellewald, C., Cremers, D.: Binary partitioning, per-

ceptual grouping, and restoration with semidefinite programming. IEEE Trans. onPAMI 25 (2003) 1364–1379

6. Geman, S., Reynolds, G.: Constrained restoration and the recovery of discontinu-ities. IEEE Trans. on PAMI 14 (1995) 932–946

7. Charbonnier, P., Blanc-Feraud, L., Aubert, G., Barlaud, M.: Deterministic edge-preserving regularization in computed imaging. IEEE Tr. I.P. 6 (1997) 298–311

8. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion.IEEE Trans. on PAMI 12 (1990) 629–639

9. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total varition based noise removalalgorithm. Physica D 60 (1992) 259–268

10. Weickert, J.: Coherence-enhancing diffusion filtering. IJCV 31 (1999) 111–12711. Romeny, B.M.: Geometry-Driven Diffusion in Computer Vision. Kluwer Academic

Publishers (1994)12. Bar, L., Sochen, N., Kiryati, N.: Variational pairing of image segmentation and

blind restoration. In Pajdla, T., ed.: ECCV 2004. (LNCS 3022) 166–17713. Chen, Y., Levine, S., Rao, M.: Variable exponent, linear growth functionals in

image restoration. SIAM Journal of Applied Mathematics 66 (2006) 1383–140614. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions

and associated variational problems. Communications on Pure and Applied Math-ematics 42 (1989) 577–684

Page 11: Introducing Dynamic Prior Knowledge to Partially-Blurred ......Introducing Dynamic Prior Knowledge to Partially-Blurred Image Restoration Hongwei Zheng and Olaf Hellwich Computer Vision

Dynamic Prior Knowledge to Partially-Blurred Image Restoration 121

15. Molina, R., Katsaggelos, A., Mateos, J.: Bayesian and regularization methods forhyperparameters estimate in image restoration. IEEE on S.P. 8 (1999) 231–246

16. Bishop, C.M., Tipping, M.E.: Bayesian regression and classification. In: Advancesin Learning Theory: Methods, Models and Applications. (2003) 267–285

17. Osher, S., Sethian, J.A.: Front propagation with curvature dependent speed: algo-rithms based on Hamilton-Jacobi formulations. J. Comp. Phy. 79 (1988) 12–49

18. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distribution and the Bayesianrestoration of images. IEEE Trans. on PAMI 6 (1984) 721–741

19. Zhu, S., Mumford, D.: Prior learning and Gibbs reaction-diffusion. IEEE Trans.on PAMI 19 (1997) 1236–1249

20. Roth, S., Black, M.: Fields of experts: A framework for learning image priors. In:CVPR, San Diego (2005) 860–867

21. Zheng, H., Hellwich, O.: Double regularized Bayesian estimation for blur identifi-cation in video sequences. In Nara., P., ed.: ACCV 2006. (LNCS 3852) 943–952

22. Pothen, A., Simon, H., Liou, K.: Partitioning sparse matrices with eigenvectors ofgraphs. SIAM. J. Matrix Anal. App. 11 (1990) 435–452

23. Hansen, P., O’Leary, D.: The use of the L-curve in the regularization of discreteill-posed problems. SIAM J. Sci. Comput. 14 (1993) 1487–1503

24. Portilla, J., Strela, V., Wainwright, M., Simoncelli, E.: Image denoising using scalemixtures of Gaussians in the wavelet domain. IEEE Trans. on Image Processing12 (2003) 1338–1351