research article an improved adaptive deconvolution...

Research ArticleAn Improved Adaptive Deconvolution Algorithm forSingle Image Deblurring

Hsin-Che Tsai and Jiunn-Lin Wu

Department of Computer Science and Engineering National Chung Hsing University Taichung 40227 Taiwan

Correspondence should be addressed to Jiunn-Lin Wu jlwucsnchuedutw

Received 3 October 2013 Accepted 3 January 2014 Published 19 February 2014

Academic Editor Yi-Hung Liu

Copyright copy 2014 H-C Tsai and J-L Wu This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited

One of the most common defects in digital photography is motion blur caused by camera shake Shift-invariant motion blurcan be modeled as a convolution of the true latent image and a point spread function (PSF) with additive noise The goal ofimage deconvolution is to reconstruct a latent image from a degraded image However ringing is inevitable artifacts arising inthe deconvolution stage To suppress undesirable artifacts regularization based methods have been proposed using natural imagepriors to overcome the ill-posedness of deconvolution problem When the estimated PSF is erroneous to some extent or the PSFsize is large conventional regularization to reduce ringing would lead to loss of image details This paper focuses on the nonblinddeconvolution by adaptive regularization which preserves image details while suppressing ringing artifacts The way is to controlthe regularization weight adaptively according to the image local characteristics We adopt elaborated reference maps that indicatethe edge strength so that textured and smooth regions can be distinguished Then we impose an appropriate constraint on theoptimization process The experimentsrsquo results on both synthesized and real images show that our method can restore latent imagewith much fewer ringing and favors the sharp edges

1 Introduction

Image blurring is one of the prime causes of poor imagequality in digital photography The one main cause of blurryimages is motion blur caused by camera shake If a motionblur is linear shift invariant the blurring process can begenerally modeled as a convolution of the true latent imageand a point spread function (PSF) with additive noise

119861 = 119870 otimes 119868 + 119873 (1)

where 119861 is the degraded image 119868 is the true latent image 119870is the PSF or a motion blur kernel which describes the traceof a sensor 119873 is the additive noise introduced during imageacquisition and otimes denotes the convolution operatorThe goalof image deblurring is to reconstruct a latent image 119868 fromdegraded image 119861

To remove motion blur we need to estimate the PSFand restore a latent image through deconvolution Existingsingle image deblurring methods can be further categorizedinto two classes If both the PSF and the latent image are

unknown the challenging problem is called blind deconvo-lution Although great progresses have been achieved in therecent years [1ndash4] blind case is severely ill-posed problembecause the number of unknowns exceeds the number ofobserved data In contrast to the former if the PSF is assumedto be known or computed in other ways the problem isreduced to estimating the latent image alone This is callednonblind deconvolution However the nonblind case is stillan ill-conditioned problem that has to do with the presenceof image noise Slight mismatches between the PSF usedby the method and the true blurring PSF also lead to poordeblurring results

Unfortunately the deconvolved result usually containsunpleasant artifacts even if the PSF is exactly known or wellestimatedThemain visually disturbing artifact is ringing thatappears around strong edges Because the PSF is often band-limited with a sharp frequency cutoff that will be zero ornear-zero values in its frequency response Thus the directinverse of the PSF causes large amplification of signal andnoise

Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2014 Article ID 658915 11 pageshttpdxdoiorg1011552014658915

2 Mathematical Problems in Engineering

Since the estimated PSF is usually inaccurate and thereal blurred image is also noisy many underconstrainedfactors will affect the amplification of ringing To reduceundesirable artifacts various regularization techniques havebeen proposed using image priors to improve nonblinddeconvolution methods [5ndash10] The most commonly usedpriors are those which encourage the image gradients to aset of derivative filters to follow heavy-tailed distributionHowever strong regularization to reduce severe artifactsdestroys the image details in the deconvolved result Weakregularization preserves image details well but it does notremove artifacts tellingly The challenge of this work is howto balance the details recovery and ringing suppression

In this paper we focus on non-blind deconvolutionwith adaptive regularization that controls the regulariza-tion strength according to the image local characteristicsThis strategy reduces ringing artifacts in a smooth regioneffectively and preserves image details in a textured regionsimultaneously First we estimate the reference maps in thescale space At the coarsest scale we are able to extractreasonably main strong edges Comparing the texture infor-mation of multiscale results we can make the elaboratedreferencemap that differentiates the edge property of texturedregion and smooth region Second regularization strengthis controlled adaptively referring to these maps Then weapply appropriate regularization with hyper-Laplacian priorto image deconvolution so that the sharp edges can beeventually recovered To solve the optimization process weadopt Krishnanrsquos [7] fast algorithm It is performed fast in thefrequency domain using fast Fourier transforms (FFTs) Theexperimental results show that our nonblind deconvolutioncan produce latent image with much fewer ringing andpreserve the sharp edges

2 Regularization Formulation

Assuming that the PSF is known or computed in otherways our method focuses on recovering the sharp imagefrom the blurred image To reduce undesirable artifacts mostadvanced regularization techniques are proposed using theprior of nature image in the gradient domain We now intro-duce the regularization formulation From a probabilisticperspective we seek maximum a posteriori (MAP) estimateof latent image 119868 in Bayesian framework

119901 (119868 | 119861) prop 119901 (119861 | 119868) 119901 (119868) (2)

where 119901(119861 | 119868) represents the likelihood and 119901(119868) denotes thepriors on the latent image Maximizing 119901(119868 | 119861) is equivalentto minimizing the cost minus log119901(119868 | 119861)

minus log119901 (119868 | 119861) = minus log119901 (119861 | 119868) minus log119901 (119868) (3)

The MAP solution of 119868 can be obtained by minimizingthe cost function above We now define these two terms Thelikelihood of a degraded image given the latent image is basedon the common blur model 119873 = 119861 minus 119870 otimes 119868 Assuming thatthe noise is modeled as a set of independent and identicallydistributed (iid) noise random variables for all pixels each

of which follows a Gaussian distribution we can express thelikelihood with Gaussian variance 1205902 as

119901 (119861 | 119868) prop 119890minus(12120590

2)119861minus119870otimes119868

2

(4)

Furthermore recent research in natural image statisticsshows that image gradients follow a heavy-tailed distributionThese distributions are well modeled by a hyper-Laplacianprior and have proven effective priors for deblurring problemIn this paper we utilize the hyper-Laplacian prior to regular-ize the solution and it can be modeled as

119901 (119868) prop 119890minus120572sum2

119896=1|119891119896otimes119868|119902

with 05 le 119902 le 08 (5)

where 119902 is a positive exponent value set in the range of 05 le119902 le 08 as suggested by Krishnan and Fargus [7] In thiswork we unify the use of 23 for 119902 value The 119891

119896denotes the

simple horizontal and vertical first-order derivative filters Itcan also be useful to include second-order derivative filtersor the more sophisticated filters With the Gaussian noiselikelihood and the hyper-Laplacian image prior (3) can berepresented by the following minimization problem

argmin119868

119899

sum

119894=1

((119861 minus 119870 otimes 119868)2

119894+ 120578

2

sum

119896=1

1003816100381610038161003816(119891119896 otimes 119868)119894

1003816100381610038161003816

119902

) (6)

where 119894 is an index running through all pixels 119899 is thewhole pixels in the image and weighting coefficient 120578 =

21205721205902 controls the strength of the regularization term For

simplicity 1198911= [1 minus 1] and 119891

2= [1 minus 1]

119879 are two first-order derivative filters We search for the 119868 which minimizesthe reconstruction error 119861 minus 119870 otimes 119868

2 with the image priorpreferring 119868 to favor the correct sharp explanation

However to analyze the formulation of Equation (6) theabove conventional regularizationrsquos weighting coefficient 120578is applied to all pixels with the same strength When theestimated PSF is inaccurate or the PSF size is large it usuallyadds strong regularization weight for suppressing ringingaround the edges But it also destroys the image details inthe deconvolved result and this is inevitable problem sinceperfect PSF estimation is impossible In addition the weakregularization weight preserves major details well but it doesnot remove artifacts effectively

3 The Proposed Method

To balance the details recovery and ringing suppression themain ideal of our approach is to control the regularizationweight adaptively according to the image local characteristicsThismeans that we need strong regularization for the smoothregions and weak regularization for the sharp edges Ourestimated maps further consider the edge property of imagetextured regions In this chapter we will explain each stepof our algorithm in detail That includes how to estimatereference map and perform the adaptive regularization inthe deconvolution stage Finally we supply some addedimprovement to obtain better results Figure 1 shows theoverall process of our nonblind deconvolution method

Mathematical Problems in Engineering 3

Blurred

Deblurred image

Reference map

Adaptive deconvolution

Stop iteration

Deconvolved

Map estimation

Yes

No

PSFK image B

t = 0

Mref

image I

t = t + 1

+

Figure 1 Overview of our adaptive deconvolution framework

31 Reference Map Estimation Inspired by the local con-straint idea [8 9 11] the reference map is designed forclassifying the smooth regions and different edge regions cor-rectly Referencemap is used for deconvolution with adaptiveregularization and can apply the right image characteristicsinformation to control the local weight of deblurring Infact the classified texture regions also contain the mainstrong edges and some smaller or fainter edges Comparedto the smaller or fainter edges the extracted main edgesare the important visible regions and those need moreweak regularization to preserve the sharp features In addi-tion we should add strong regularization for the classifiedsmooth regions to reduce the most noticeable artifacts sincethe ringing artifacts are usually propagated in the smoothareas

Based on these observations we estimate the referencemaps in the multiscale space We first build a pyramid119861119897119871

119897=0of the full resolution input blurred image 119861 using

bilinear downsampling Our goal is to combine differentedge information from coarse to fine layers So we performthe map estimation approach in each scale and distinguishits different regions At the coarsest scale we are able toextract reasonably main strong edges of the image In thenext finer scale we gradually extract more and more smalledge details Finally all estimated maps from different scaleresults combine to form one elaborated reference map Theestimation is guided to a good map by concentrating onthe main edges of the input image and progressively dealingwith smaller and smoother details But it is difficult to obtain

correct edge information from the blurred image directly sothe map is renewed from every deconvolved result duringeach iteration time

We now describe how to produce and combine the esti-mated reference maps The initial reference map is estimatedfrom the input blurred image Since the locally smooth regionwhich has no edges information is still smooth after blurringthe edge areas of blurred image will be affected Inspired bythis idea we compute the edge strength and use predefinedthreshold to classify different regions The edge strength atlocated pixel 119894 is defined as follows

Eg (119894) =(sumℎisin119882119909

ℎ + sumVisin119882119910

V)

119873total

(7)

where Eg(119894) denotes the edge strength response at pixel 119894 onthe observed image 119882 is the 3 times 3 window whose center islocated on 119894 119882

119909= 119882 otimes [1 minus 1] 119882

119910= 119882 otimes [1 minus 1]

119879 and119873total is the total number of pixels in default window If thecomputed edge strength value is smaller than a predefinedthreshold 119879 which is set to 3 times 10

minus2 in our experiment wewill regard the center pixel 119894 as in smooth region Ω that is119894 isin Ω

119872119897

119894=

1 if Eg (119894) lt 119879

0 otherwise(8)

Since the texture regions which is classified by (7) alsocontain the main strong edges and some smaller edges weaim to further improve the estimated results The improvedextracting method includes the following steps Firstly webuild a pyramid 119861

119897119871

119897=0of the blurred image At each scale

119897 the edge strength Eg(119894) is computed in every level and thenis upsampled It means that this step will produce 119897 numberbinary maps Finally we sum all the edge intensities at thesame pixel location and run through all pixels Now we geta new reference map after normalization process which isshown in Figure 2(b) and the smooth region Ω is shown asthe set of all white pixelsThe strongest edges are indicated bythe most dark pixels and the other gray level pixels mean thesmaller or fainter edges Hence we define this extracting stepusing the multiscale approach of the observed image as

119872ref =sum119871

119897=0119872119897

119871 (9)

where 119872119897 is the binary map at the 119897th level of the images

pyramid by (8) and 119871 denotes the total scale layers forexample 119871 is set to 3 levers in our implementation In (9)the final map119872ref is extracted by summing up all the inten-sity values of those upsampling maps and normalizing theresults

We gradually recover more and more image details by aniterative deconvolution algorithm because it can refine theresult until convergence Obviously the deconvolved imagewith initial adaptive regularization shows much better edgeinformation than does the input blurred image Thus at thefollowing reference map estimation we use the deconvolved


(a) (b)

(c) (d)

Figure 2 Reference map estimation (a) Input blurred image (b) First reference map is estimated from the blurred image It can locallycompute smooth regions and distinguish different edge regions (c)The deconvolved image with the first adaptive regularization (d) Secondreference map is estimated from previous deconvolved result

result from the previous iteration to distinguish differentedge regions elaborately We also adopt the same criterionto extract map Second estimated result is represented inFigure 2(d) The map will be renewed from the earlierdeconvolved result after each iteration

32 Image Deconvolution with Adaptive Regularization Nowwe would like to seek the MAP solution for the latent imagein the deconvolution stage But the limitation of conventionalregularization is that the same value of weighting coefficient 120578is applied to whole pixels To overcome this fault and balancethe image quality our method introduces an improvementthat is to apply appropriate regularization weight accordingto the local characteristics So the adaptive regularization isperformed based on the estimated referencemap during eachiteration time Based on this idea we simply modify (6) asfollows

argmin119868

119899

sum

119894=1

((119861 minus 119870 otimes 119868)2

119894+ 120578119901

2

sum

119896=1

1003816100381610038161003816(119891119896 otimes 119868)119894

1003816100381610038161003816

119902

) (10)

where the weighting coefficient 120578119901is adjusted based on the

estimated reference map It provides a basis to adaptivelysuppress the ringing effects in different regions The effect ofadaptive regularization in deconvolution result is representedin Figure 3 In comparison our result exhibits sharper imagedetail and fewer artifacts

However the use of sparse distributions with 119902 lt 1

makes the optimization problem nonconvex It becomes slow

to solve the approximation Using the half-quadratic split-ting Krishnanrsquos [7] fast algorithm introduces two auxiliaryvariables 120596

1and 120596

2at each pixel to move the (119891

119896otimes 119868)119894terms

outside the | sdot |119902 expressionThus (10) can be converted to thefollowing optimization problem

arg min119868120596

119899

sum

119894=1

((119861 minus 119870 otimes 119868)2

119894

+120573

2((1198911otimes 119868 minus 120596

1)2

119894+ (1198912otimes 119868 minus 120596

2)2

119894)

+ 120578119875(1003816100381610038161003816(1205961)119894

1003816100381610038161003816

119902

+1003816100381610038161003816(1205962)119894

1003816100381610038161003816

119902

))

(11)

where (119891119896otimes 119868 minus 120596

119896)2 term is for constraint of 119891

119896otimes 119868 = 120596

119896

and 120573 is a control parameter that we will vary during theiteration process As 120573 parameter becomes large the solutionof (13) converges into that of (12) This scheme is also calledalternating minimization [12] where we adopt a commontechnique for image restoration Minimizing (13) for a fixed120573 can be performed by alternating two stepsThis means thatwe solve 120596 and 119868 respectively

One subproblem is to solve 1205961and 120596

2 which is called 120596

subproblem First the initial 119868 is set to the input blurred image119861 Given a fixed 119868 finding the optimal120596 can be simplified intothe following optimization problem

argmin120596

(120582119875

2|120596|119902+120573

2(120596 minus ])2) (12)


(a) (b)

(c)

Figure 3 Effect of deconvolutionwith adaptive regularization (a)Thedeconvolved result byKrishnanrsquos [7] algorithmwith the sameweightingvalue (b) Our deconvolved result with different 120578

119901weighting values according to the local characteristics as defined above (c) Close-ups of

two image regions for comparison

where the value ] = (119891119896otimes 119868) For convenient calculation 120578

119901

is replaced with 1205821199012 and it does not affect the performance

Inparticular for the case 119902 = 23 the 120596 satisfying the aboveequation is the analytical solution of the following quarticpolynomial

1205964minus 3]1205963 + 3]21205962 minus ]3120596 +

1205823

119901

271205733= 0 (13)

to find and select the correct roots of the above quarticpolynomial we adopt Krishnanrsquos approach as detailed in [7]They numerically solve (12) for all pixels using a lookup tableto find 120596

Second we briefly describe the 119868 subproblem and itsstraightforward solution Given a fixed value of 120596 fromprevious iteration we aim to obtain the optimal 119868 by the

following optimization problem Equation (11) is modifiedas

argmin119868

((119861 minus 119870 otimes 119868)2

+120573

2((1198911otimes 119868 minus 120596

1)2

+ (1198912otimes 119868 minus 120596

2)2

))

(14)

the 119868 subproblem can be optimized by setting the derivative ofthe cost function to zero So (14) is quadratic in 119868Theoptimal119868 is

(119865119879

11198651+ 119865119879

21198652+2

120573119870119879119870)119868 = 119865

119879

11205961+ 119865119879

21205962+2

120573119870119879119861 (15)

for brevity we set 119865119896119868 = (119891

119896otimes 119868) and119870119868 = (119870otimes 119868) Assuming

circular boundary conditions (15) can apply 2D FFTs whichdiagonalize the convolution matrices 119865

1 1198652 and 119870 helping

us to find optimal 119868 directly

119868 = IFFT(FFT(119865

1)lowast

∘ FFT (1205961) + FFT(119865

2)lowast

∘ FFT (1205962) + (2120573) FFT(119870)lowast ∘ FFT (119861)

FFT(1198651)lowast

∘ FFT (1198651) + FFT(119865

2)lowast


) (16)

where lowast denotes the complex conjugate and ∘ denotescomponent-wise multiplication The FFT(sdot) and IFFT(sdot) rep-resent Fourier and inverse Fourier transforms respectivelySolving (16) requires only three FFTs at each iteration sincesome FFTs operation of 119865

1 1198652 and119870 can be precomputed

The nonconvex optimization problem arises from theuse of a hyper-Laplacian prior with 119902 lt 1 then we

adopt a splitting approach that allows the nonconvexity tobecome separable over pixels We now give the summaryof alternating minimization scheme As described above weminimize (11) by alternating the 120596 and 119868 subproblems beforeincreasing the value of 120573 and repeating At the beginningof each iteration we first compute 120596 given the initial ordeconvolved 119868 by minimizing (14) Then given a fixed value


of 120596 it can minimize (14) to find the optimal 119868 The iterationprocess stops when the parameter 120573 is sufficiently large Inpractice starting with small value 120573 we scale it by an integerpower until it exceeds some fixed values 120573Max

Finally we mention one last improvement detail Inthe blurring process the convolution operator makes useof not only the image inside the field of view (FOV) ofthe given observation but also part of the scene in thearea bordering it The part outside the FOV cannot beavailable to the deconvolution process When any algorithmperforms deconvolution in the Fourier domain this missinginformation would cause artifacts at the image boundariesTo analyze this cause the FFTs assume the periodicity of thedata themissing pixels will be taken from the opposite side ofthe image when FFTs are performed since the data obtainedmay not coincide with the missing ground truth data Thesegive rise to boundary artifacts which poses a difficulty invarious restoration methods

To reduce the visual artifacts caused by the boundaryvalue problem we use Liursquos approach as detailed in [13] Thebasic concept to solve the problem is to expand the blurredimage such that the intensity and gradient are maintainedat the border between the input image and expanded blockThis algorithm suppresses the ripples near the image borderIt does not require the extra assumption of the PSF and canbe adopted by any FFTs based restoration methods

4 Experimental Results

41 Parameters Setting Since we want to suppress the con-trast of ringing in the extracted smooth regions while avoid-ing suppression of sharp edges the regularization strengthshould be large in smooth regions and small in others Wealso concern themain edges of the textured regions and othersmaller or fainter features So we controlled regularizationstrength referring to the different edge property This meansthat we need the most weak regularization for the main edgesregions

Through all the experiments the weighting coefficient120578119901is controlled adaptively referring to estimated maps We

have used a geometric progression for the different regionsof values of 120578

1199011015840 = 120578

119901119903 The rate 119903 is set to 3 that is we

decrease the regularization strength according to the localcharacteristics In our experiments the 120578

119901is set to 1 times 10

minus3

in smooth regions Then we decrease the value to 33 times 10minus4

in next smaller or fainter edges 11 times 10minus4 in the strong

edges and so on The main edges will be applied to theleast value of 120578

119901 and the 120582

1199012 is equal to 120578

119901in (14) The

120578119901also progressively decrease as the number of iterations is

increased It can produce the best results In addition ourmethoduses a hyper-Laplacian priorwith 119902 = 23 To find theoptimal solution parameter 120573 will vary during the iterativeprocessThe 120573 value is varied from 1 to 256 by integer powersof 2radic2 As 120573 is larger than 120573Max = 256 the iterations willstop

Besides for all testing images in our experiments wecovert the RGB color image to the YCbCr color spaceand only the luminance channel is taken into computationFurthermore with the alternating minimization and FFTsoperation those schemes can accelerate total computationaltime In the next two parts we will demonstrate the effective-ness of our approach

42 Synthetic Images The synthesized degradations are gen-erated by convolving the artificial PSF and the sharp imagesIn the first part of the experiments both subjective qualityand objective quality are compared For testing the subjectiveperformance we compare our result against those of threeexisting nonblind image deconvolution methods as shownin Figure 4

Figure 4 shows the comparison of the visual quality forthe simulated image The standard RL method preservesedges well but produces the severe ringing artifacts Moreiteration introduces not only more image details but alsomore ringing Besides the Matlab RL function is per-formed in the frequency domain so it gives rise to severeboundary artifacts Levinrsquos and Krishnanrsquos methods reduceringing effectively due to advanced image priors But theysuppress or blur some details since regularization strengthis applied to all pixels with the same value The IRLSsolution of Levinrsquos algorithm also expands expensive runningtimes Our approach can recover finer image details andthin image structures while successfully suppressing ringingartifacts

In addition for testing the visual objective quality thepeak signal to noise ratio (PSNR) measurement is also usedto evaluate quantitatively the quality of above restored resultGiven signal 119868 the PSNR value of its estimate is defined as

PSNR (dB) = 10 sdot log 2552

(1119898119899)sum119898minus1

119894=0sum119899minus1

119895=0(119868119894119895minus

_119868 119906119895)

2

(17)

where 119898 times 119899 is the size of the input image 119868119894119895is the intensity

value of true clear image at the pixel location (119894 119895) and_119868 119894119895

corresponds to the intensity value of the restored image at thesame location The average PSNR values of above results arelisted in Table 1

43 Real Blurred Images Besides testing the method onsynthetic degradations we also apply our method to real lifeblurred degradationsWe estimate the PSF of these images byXursquos method [4] For the real blurred image only subjectivequality is measured as shown in Figures 5 6 and 7

Figures 5 and 6 show more results of the images from[3] The PSF is estimated by [4] Then the inferred PSF isused as the input of all nonblind methods for comparisonWith estimated PSF which is usually not accurate thedeconvolution methods need to perform robustly The finedetails of the thin branch are compared in Figure 5 It can


(a) (b) (c)

(d) (e)

(f)

Figure 4 Deblurring results of synthetic photo ldquoLenardquo (a) The size of blurred image is 512 times 512 and the estimated PSF size is 33 times 39 (b)The standard Richardson-Lucy method (c) The result of Levinrsquos method [5] with the sparse prior (d) The result of Krishnanrsquos method [7]with the hyper-Laplacian prior (e) The result of our deconvolution with adaptive regularization (f) Close-ups of (a)ndash(e) image regions forcomparison

be verified that our approach shows superior edge preservingability compared to other nonblind methods In Figure 6standard RL produces the most noticeable ringing evenpreserving the sharp edges Our restoration result balancesthe details recovery and ringing suppression Finally thedegraded image of Figure 7 uses a large aperture and thefocused object is blurry due to camera shake In comparisonour deconvolution result exhibits richer and clearer imagestructures than other methods

Finally the computational costs of our proposedmethod and other nonblind image deconvolution methods

considered in the experiments are compared in Table 2 Allcodes are tested in the computer with AMD Phenom IIX4 965 340GHz processor and 40GB RAM The MatlabRL method is performed in the frequency domain with 20iterations For the IRLS scheme we used the implementationof [5] with default parameters Krishnanrsquos method andour proposed method both use hyper-Laplacian prior toregularize the solution and we set the same 120573 value in theexperiment Experimental results show that the proposedadaptive scheme consumes acceptable processing time whilepreserving more details in the deblurred images


(a) (b) (c)

(d) (e)

(f)

Figure 5 Deblurring results of real photo ldquoRed treerdquo (a)The size of blurred image is 454 times 588 and the estimated PSF size is 27 times 27 (b)Thestandard Richardson-Lucy method (c)The result of Levinrsquos method [5] with the sparse prior (d)The result of Krishnanrsquos method [7] (e)Theresult of our deconvolution with adaptive regularization (f) Close-ups of (a)ndash(e) image regions for comparison

5 Conclusions

The image deblurring is a long-standing problem for manyapplications In this paper a high-quality nonblind decon-volution to remove camera motion blur from a single image

has been presented We follow the regularization basedframework using natural image prior to constrain the optimalsolution Our main contribution is an effective scheme forbalancing the image details recovery and ringing suppres-sion We first introduce the reference map indicating the


(a) (b) (c)

(d) (e)

(f)

Figure 6 Deblurring results of real photo ldquoStatuerdquo (a) The size of blurred image is 800 times 800 and the estimated PSF size is 31 times 31 (b) Thestandard Richardson-Lucy method (c)The result of Levinrsquos method [5] with the sparse prior (d)The result of Krishnanrsquos method [7] (e)Theresult of our deconvolution with adaptive regularization (f) Close-ups of (a)ndash(e) image regions for comparison

Table 1 Comparison of average PSNRs (dB)

Image Blurry Richardson-Lucy [16 17] Levin et al 2007 [5] Krishnan and Fergus 2009 [7] OursLena 2399 2969 3106 3123 3176Fruits 2351 2811 2903 3029 3071Airplane 2322 2850 3345 3339 3402

smooth regions and different textured edge regions Thenaccording the image local characteristics we can controlregularization weighting factor adaptively In addition theproposed method is practically considering the complexityby FFTs operations Deconvolved results obtained by our

approach show a noticeable improvement in recoveringvisually pleasing details with fewer ringing

In the future the deblurring topic is still more challengingand exciting We have found that the severe noise PSFestimation errors or large PSF may be propagated and


(a) (b) (c)

(d) (e)

(f)

Figure 7 Deblurring results of real photo ldquoFlowerrdquo (a) The size of blurred image is 533 times 800 and the estimated PSF size is 31 times 31 (b) Thestandard Richardson-Lucy method (c)The result of Levinrsquos method [5] with the sparse prior (d)The result of Krishnanrsquos method [7] (e)Theresult of our deconvolution with adaptive regularization (f) Close-ups of (a)ndash(e) image regions for comparison

amplified the unpleasant artifacts Thus the research direc-tion of future work is how to estimate accurate PSF andpropose the robust blind deblurring model Recently thespatially variant PSF models have also drawn some attentionfor better modeling practical motion blurring operator [1415] This means that the PSFs are not uniform in appearance

for example from slight camera rotation or nonuniformobjects movement It needs to explore the removal of shift-variant blur using a general kernel assumption Moreoverwe are also interested to apply the proposed framework toother restoration problems such as image denoising videodeblurring or surface reconstruction


Table 2 Comparison of the computational costs (Seconds)

Image Richardson-Lucy Levin et al 2007 [5] Krishnan and Fergus 2009 [7] OursLena 1254 32321 782 890Fruits 1420 35486 803 1069Airplane 1298 32811 631 862Red tree 1321 67275 668 1009Statue 1522 106054 893 1169Flower 1427 98914 832 1112

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgment

The authors would like to thank the anonymous review-ers Their comments helped in improving this paper Thisresearch is supported by the National Science CouncilTaiwan under Grants NSC 101-2221-E-005-088 andNSC 102-2221-E-005-083

References

[1] R Fergus B Singh A Hertzmann S T Roweis and W TFreeman ldquoRemoving camera shake from a single photographrdquoACMTransactions on Graphics vol 25 no 3 pp 787ndash794 2006

[2] N Joshi R Szeliski and D J Kriegman ldquoPSF estimationusing sharp edge predictionrdquo in Proceedings of the 26th IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo08) pp 1ndash8 Anchorage Alaska USA June 2008

[3] S Cho and S Lee ldquoFast motion deblurringrdquo ACM Transactionson Graphics vol 28 no 5 article 145 2009

[4] L Xu and J Jia ldquoTwo-phase kernel estimation for robustmotiondeblurringrdquo in Proceedings of the 11th European Conference onComputer Vision (ECCV rsquo10) pp 157ndash170 2010

[5] A Levin R Fergus F Durand and W T Freeman ldquoImage anddepth from a conventional camera with a coded aperturerdquoACMTransactions on Graphics vol 26 no 3 Article ID 12764642007

[6] L Yuan J Sun L Quan and H-Y Shum ldquoProgressive inter-scale and intra-scale non-blind image deconvolutionrdquo ACMTransactions on Graphics vol 27 no 3 article 74 2008

[7] D Krishnan and R Fergus ldquoFast image deconvolution usinghyper-laplacian priorsrdquo in Proceedings of the 23rd AnnualConference onNeural Information Processing Systems (NIPS rsquo09)pp 1033ndash1041 December 2009

[8] Q Shan J Jia and A Agarwala ldquoHigh-quality motion deblur-ring from a single imagerdquo ACM Transactions on Graphics vol27 no 3 article 73 2008

[9] W Yongpan F Huajun X Zhihai L Qi and D ChaoyueldquoAn improved Richardson-Lucy algorithmbased on local priorrdquoOptics and Laser Technology vol 42 no 5 pp 845ndash849 2010

[10] T S Cho N Joshi C L Zitnick S B Kang R Szeliski andW T Freeman ldquoA content-aware image priorrdquo in Proceedingsof the IEEE Computer Society Conference on Computer Visionand Pattern Recognition (CVPR rsquo10) pp 169ndash176 San FranciscoCalif USA June 2010

[11] J-H Lee andY-SHo ldquoHigh-quality non-blind image deconvo-lution with adaptive regularizationrdquo Journal of Visual Commu-nication and Image Representation vol 22 no 7 pp 653ndash6632011

[12] YWang J YangW Yin and Y Zhang ldquoA new alternatingmin-imization algorithm for total variation image reconstructionrdquoSIAM Journal on Imaging Sciences vol 1 no 3 pp 248ndash2722008

[13] R Liu and J Jia ldquoReducing boundary artifacts in image decon-volutionrdquo in Proceedings of the IEEE International Conference onImage Processing (ICIP rsquo08) pp 505ndash508 San Francisco CalifUSA October 2008

[14] A Levin Y Weiss F Durand and W T Freeman ldquoUnder-standing blind deconvolution algorithmsrdquo IEEETransactions onPattern Analysis and Machine Intelligence vol 33 no 12 pp2354ndash2367 2011

[15] Y-WTai P Tan andM S Brown ldquoRichardson-Lucy deblurringfor scenes under a projective motion pathrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1603ndash1618 2011

[16] W H Richardson ldquoBayesian-based iterative method of imagerestorationrdquo Journal of the Optical Society of America vol 62no 1 pp 55ndash59 1972

[17] L Lucy ldquoAn iterative technique for rectification of observeddistributionsrdquo Astronomical Journal vol 79 no 6 pp 745ndash7651974

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of


Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of


Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of


Mathematical PhysicsAdvances in

Complex AnalysisJournal of


OptimizationJournal of


CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of


Operations ResearchAdvances in

Journal of


Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences


The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Algebra

Discrete Dynamics in Nature and Society



Decision SciencesAdvances in

Discrete MathematicsJournal of


Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of


Since the estimated PSF is usually inaccurate and thereal blurred image is also noisy many underconstrainedfactors will affect the amplification of ringing To reduceundesirable artifacts various regularization techniques havebeen proposed using image priors to improve nonblinddeconvolution methods [5ndash10] The most commonly usedpriors are those which encourage the image gradients to aset of derivative filters to follow heavy-tailed distributionHowever strong regularization to reduce severe artifactsdestroys the image details in the deconvolved result Weakregularization preserves image details well but it does notremove artifacts tellingly The challenge of this work is howto balance the details recovery and ringing suppression

In this paper we focus on non-blind deconvolutionwith adaptive regularization that controls the regulariza-tion strength according to the image local characteristicsThis strategy reduces ringing artifacts in a smooth regioneffectively and preserves image details in a textured regionsimultaneously First we estimate the reference maps in thescale space At the coarsest scale we are able to extractreasonably main strong edges Comparing the texture infor-mation of multiscale results we can make the elaboratedreferencemap that differentiates the edge property of texturedregion and smooth region Second regularization strengthis controlled adaptively referring to these maps Then weapply appropriate regularization with hyper-Laplacian priorto image deconvolution so that the sharp edges can beeventually recovered To solve the optimization process weadopt Krishnanrsquos [7] fast algorithm It is performed fast in thefrequency domain using fast Fourier transforms (FFTs) Theexperimental results show that our nonblind deconvolutioncan produce latent image with much fewer ringing andpreserve the sharp edges

2 Regularization Formulation

Assuming that the PSF is known or computed in otherways our method focuses on recovering the sharp imagefrom the blurred image To reduce undesirable artifacts mostadvanced regularization techniques are proposed using theprior of nature image in the gradient domain We now intro-duce the regularization formulation From a probabilisticperspective we seek maximum a posteriori (MAP) estimateof latent image 119868 in Bayesian framework

119901 (119868 | 119861) prop 119901 (119861 | 119868) 119901 (119868) (2)

where 119901(119861 | 119868) represents the likelihood and 119901(119868) denotes thepriors on the latent image Maximizing 119901(119868 | 119861) is equivalentto minimizing the cost minus log119901(119868 | 119861)

minus log119901 (119868 | 119861) = minus log119901 (119861 | 119868) minus log119901 (119868) (3)

The MAP solution of 119868 can be obtained by minimizingthe cost function above We now define these two terms Thelikelihood of a degraded image given the latent image is basedon the common blur model 119873 = 119861 minus 119870 otimes 119868 Assuming thatthe noise is modeled as a set of independent and identicallydistributed (iid) noise random variables for all pixels each

of which follows a Gaussian distribution we can express thelikelihood with Gaussian variance 1205902 as

119901 (119861 | 119868) prop 119890minus(12120590

2)119861minus119870otimes119868

2

(4)

Furthermore recent research in natural image statisticsshows that image gradients follow a heavy-tailed distributionThese distributions are well modeled by a hyper-Laplacianprior and have proven effective priors for deblurring problemIn this paper we utilize the hyper-Laplacian prior to regular-ize the solution and it can be modeled as

119901 (119868) prop 119890minus120572sum2

119896=1|119891119896otimes119868|119902

with 05 le 119902 le 08 (5)

where 119902 is a positive exponent value set in the range of 05 le119902 le 08 as suggested by Krishnan and Fargus [7] In thiswork we unify the use of 23 for 119902 value The 119891

119896denotes the

simple horizontal and vertical first-order derivative filters Itcan also be useful to include second-order derivative filtersor the more sophisticated filters With the Gaussian noiselikelihood and the hyper-Laplacian image prior (3) can berepresented by the following minimization problem

argmin119868

119899

sum

119894=1

((119861 minus 119870 otimes 119868)2

119894+ 120578

2

sum

119896=1

1003816100381610038161003816(119891119896 otimes 119868)119894

1003816100381610038161003816

119902

) (6)

where 119894 is an index running through all pixels 119899 is thewhole pixels in the image and weighting coefficient 120578 =

21205721205902 controls the strength of the regularization term For

simplicity 1198911= [1 minus 1] and 119891

2= [1 minus 1]

119879 are two first-order derivative filters We search for the 119868 which minimizesthe reconstruction error 119861 minus 119870 otimes 119868

2 with the image priorpreferring 119868 to favor the correct sharp explanation

However to analyze the formulation of Equation (6) theabove conventional regularizationrsquos weighting coefficient 120578is applied to all pixels with the same strength When theestimated PSF is inaccurate or the PSF size is large it usuallyadds strong regularization weight for suppressing ringingaround the edges But it also destroys the image details inthe deconvolved result and this is inevitable problem sinceperfect PSF estimation is impossible In addition the weakregularization weight preserves major details well but it doesnot remove artifacts effectively

3 The Proposed Method

To balance the details recovery and ringing suppression themain ideal of our approach is to control the regularizationweight adaptively according to the image local characteristicsThismeans that we need strong regularization for the smoothregions and weak regularization for the sharp edges Ourestimated maps further consider the edge property of imagetextured regions In this chapter we will explain each stepof our algorithm in detail That includes how to estimatereference map and perform the adaptive regularization inthe deconvolution stage Finally we supply some addedimprovement to obtain better results Figure 1 shows theoverall process of our nonblind deconvolution method


Blurred

Deblurred image

Reference map


Stop iteration

Deconvolved

Map estimation

Yes

No

PSFK image B

t = 0

Mref

image I

t = t + 1

+








Eg (119894) =(sumℎisin119882119909

ℎ + sumVisin119882119910

V)

119873total

(7)


119909= 119882 otimes [1 minus 1] 119882

119910= 119882 otimes [1 minus 1]



119872119897

119894=

1 if Eg (119894) lt 119879

0 otherwise(8)


119897119871



119872ref =sum119871

119897=0119872119897

119871 (9)





(a) (b)

(c) (d)




argmin119868

119899

sum

119894=1

((119861 minus 119870 otimes 119868)2

119894+ 120578119901

2

sum

119896=1

1003816100381610038161003816(119891119896 otimes 119868)119894

1003816100381610038161003816

119902

) (10)






1and 120596


119896otimes 119868)119894terms


arg min119868120596

119899

sum

119894=1

((119861 minus 119870 otimes 119868)2

119894

+120573

2((1198911otimes 119868 minus 120596

1)2

119894+ (1198912otimes 119868 minus 120596

2)2

119894)

+ 120578119875(1003816100381610038161003816(1205961)119894

1003816100381610038161003816

119902

+1003816100381610038161003816(1205962)119894

1003816100381610038161003816

119902

))

(11)



119896otimes 119868 = 120596

119896





argmin120596

(120582119875

2|120596|119902+120573

2(120596 minus ])2) (12)


(a) (b)

(c)





119901



1205964minus 3]1205963 + 3]21205962 minus ]3120596 +

1205823

119901

271205733= 0 (13)




argmin119868

((119861 minus 119870 otimes 119868)2

+120573

2((1198911otimes 119868 minus 120596

1)2

+ (1198912otimes 119868 minus 120596

2)2

))

(14)


(119865119879

11198651+ 119865119879

21198652+2

120573119870119879119870)119868 = 119865

119879

11205961+ 119865119879

21205962+2

120573119870119879119861 (15)




1 1198652 and 119870 helping


119868 = IFFT(FFT(119865

1)lowast

∘ FFT (1205961) + FFT(119865

2)lowast


FFT(1198651)lowast

∘ FFT (1198651) + FFT(119865

2)lowast


) (16)













1199011015840 = 120578




minus3




119901 and the 120582

1199012 is equal to 120578

119901in (14) The








(1119898119899)sum119898minus1

119894=0sum119899minus1

119895=0(119868119894119895minus

_119868 119906119895)

2

(17)







(a) (b) (c)

(d) (e)

(f)






(a) (b) (c)

(d) (e)

(f)


5 Conclusions




(a) (b) (c)

(d) (e)

(f)








(a) (b) (c)

(d) (e)

(f)









Acknowledgment


References

























Volume 2014




Journal of











Journal of


Function Spaces






Algebra










Blurred

Deblurred image

Reference map


Stop iteration

Deconvolved

Map estimation

Yes

No

PSFK image B

t = 0

Mref

image I

t = t + 1

+








Eg (119894) =(sumℎisin119882119909

ℎ + sumVisin119882119910

V)

119873total

(7)


119909= 119882 otimes [1 minus 1] 119882

119910= 119882 otimes [1 minus 1]



119872119897

119894=

1 if Eg (119894) lt 119879

0 otherwise(8)


119897119871



119872ref =sum119871

119897=0119872119897

119871 (9)





(a) (b)

(c) (d)




argmin119868

119899

sum

119894=1

((119861 minus 119870 otimes 119868)2

119894+ 120578119901

2

sum

119896=1

1003816100381610038161003816(119891119896 otimes 119868)119894

1003816100381610038161003816

119902

) (10)






1and 120596


119896otimes 119868)119894terms


arg min119868120596

119899

sum

119894=1

((119861 minus 119870 otimes 119868)2

119894

+120573

2((1198911otimes 119868 minus 120596

1)2

119894+ (1198912otimes 119868 minus 120596

2)2

119894)

+ 120578119875(1003816100381610038161003816(1205961)119894

1003816100381610038161003816

119902

+1003816100381610038161003816(1205962)119894

1003816100381610038161003816

119902

))

(11)



119896otimes 119868 = 120596

119896





argmin120596

(120582119875

2|120596|119902+120573

2(120596 minus ])2) (12)


(a) (b)

(c)





119901



1205964minus 3]1205963 + 3]21205962 minus ]3120596 +

1205823

119901

271205733= 0 (13)




argmin119868

((119861 minus 119870 otimes 119868)2

+120573

2((1198911otimes 119868 minus 120596

1)2

+ (1198912otimes 119868 minus 120596

2)2

))

(14)


(119865119879

11198651+ 119865119879

21198652+2

120573119870119879119870)119868 = 119865

119879

11205961+ 119865119879

21205962+2

120573119870119879119861 (15)




1 1198652 and 119870 helping


119868 = IFFT(FFT(119865

1)lowast

∘ FFT (1205961) + FFT(119865

2)lowast


FFT(1198651)lowast

∘ FFT (1198651) + FFT(119865

2)lowast


) (16)













1199011015840 = 120578




minus3




119901 and the 120582

1199012 is equal to 120578

119901in (14) The








(1119898119899)sum119898minus1

119894=0sum119899minus1

119895=0(119868119894119895minus

_119868 119906119895)

2

(17)







(a) (b) (c)

(d) (e)

(f)






(a) (b) (c)

(d) (e)

(f)


5 Conclusions




(a) (b) (c)

(d) (e)

(f)








(a) (b) (c)

(d) (e)

(f)









Acknowledgment


References

























Volume 2014




Journal of











Journal of


Function Spaces






Algebra










(a) (b)

(c) (d)




argmin119868

119899

sum

119894=1

((119861 minus 119870 otimes 119868)2

119894+ 120578119901

2

sum

119896=1

1003816100381610038161003816(119891119896 otimes 119868)119894

1003816100381610038161003816

119902

) (10)






1and 120596


119896otimes 119868)119894terms


arg min119868120596

119899

sum

119894=1

((119861 minus 119870 otimes 119868)2

119894

+120573

2((1198911otimes 119868 minus 120596

1)2

119894+ (1198912otimes 119868 minus 120596

2)2

119894)

+ 120578119875(1003816100381610038161003816(1205961)119894

1003816100381610038161003816

119902

+1003816100381610038161003816(1205962)119894

1003816100381610038161003816

119902

))

(11)



119896otimes 119868 = 120596

119896





argmin120596

(120582119875

2|120596|119902+120573

2(120596 minus ])2) (12)


(a) (b)

(c)





119901



1205964minus 3]1205963 + 3]21205962 minus ]3120596 +

1205823

119901

271205733= 0 (13)




argmin119868

((119861 minus 119870 otimes 119868)2

+120573

2((1198911otimes 119868 minus 120596

1)2

+ (1198912otimes 119868 minus 120596

2)2

))

(14)


(119865119879

11198651+ 119865119879

21198652+2

120573119870119879119870)119868 = 119865

119879

11205961+ 119865119879

21205962+2

120573119870119879119861 (15)




1 1198652 and 119870 helping


119868 = IFFT(FFT(119865

1)lowast

∘ FFT (1205961) + FFT(119865

2)lowast


FFT(1198651)lowast

∘ FFT (1198651) + FFT(119865

2)lowast


) (16)













1199011015840 = 120578




minus3




119901 and the 120582

1199012 is equal to 120578

119901in (14) The








(1119898119899)sum119898minus1

119894=0sum119899minus1

119895=0(119868119894119895minus

_119868 119906119895)

2

(17)







(a) (b) (c)

(d) (e)

(f)






(a) (b) (c)

(d) (e)

(f)


5 Conclusions




(a) (b) (c)

(d) (e)

(f)








(a) (b) (c)

(d) (e)

(f)









Acknowledgment


References

























Volume 2014




Journal of











Journal of


Function Spaces






Algebra










(a) (b)

(c)





119901



1205964minus 3]1205963 + 3]21205962 minus ]3120596 +

1205823

119901

271205733= 0 (13)




argmin119868

((119861 minus 119870 otimes 119868)2

+120573

2((1198911otimes 119868 minus 120596

1)2

+ (1198912otimes 119868 minus 120596

2)2

))

(14)


(119865119879

11198651+ 119865119879

21198652+2

120573119870119879119870)119868 = 119865

119879

11205961+ 119865119879

21205962+2

120573119870119879119861 (15)




1 1198652 and 119870 helping


119868 = IFFT(FFT(119865

1)lowast

∘ FFT (1205961) + FFT(119865

2)lowast


FFT(1198651)lowast

∘ FFT (1198651) + FFT(119865

2)lowast


) (16)













1199011015840 = 120578




minus3




119901 and the 120582

1199012 is equal to 120578

119901in (14) The








(1119898119899)sum119898minus1

119894=0sum119899minus1

119895=0(119868119894119895minus

_119868 119906119895)

2

(17)







(a) (b) (c)

(d) (e)

(f)






(a) (b) (c)

(d) (e)

(f)


5 Conclusions




(a) (b) (c)

(d) (e)

(f)








(a) (b) (c)

(d) (e)

(f)









Acknowledgment


References

























Volume 2014




Journal of











Journal of


Function Spaces






Algebra

















1199011015840 = 120578




minus3




119901 and the 120582

1199012 is equal to 120578

119901in (14) The








(1119898119899)sum119898minus1

119894=0sum119899minus1

119895=0(119868119894119895minus

_119868 119906119895)

2

(17)







(a) (b) (c)

(d) (e)

(f)






(a) (b) (c)

(d) (e)

(f)


5 Conclusions




(a) (b) (c)

(d) (e)

(f)








(a) (b) (c)

(d) (e)

(f)









Acknowledgment


References

























Volume 2014




Journal of











Journal of


Function Spaces






Algebra










(a) (b) (c)

(d) (e)

(f)






(a) (b) (c)

(d) (e)

(f)


5 Conclusions




(a) (b) (c)

(d) (e)

(f)








(a) (b) (c)

(d) (e)

(f)









Acknowledgment


References

























Volume 2014




Journal of











Journal of


Function Spaces






Algebra










(a) (b) (c)

(d) (e)

(f)


5 Conclusions




(a) (b) (c)

(d) (e)

(f)








(a) (b) (c)

(d) (e)

(f)









Acknowledgment


References

























Volume 2014




Journal of











Journal of


Function Spaces






Algebra










(a) (b) (c)

(d) (e)

(f)








(a) (b) (c)

(d) (e)

(f)









Acknowledgment


References

























Volume 2014




Journal of











Journal of


Function Spaces






Algebra










(a) (b) (c)

(d) (e)

(f)









Acknowledgment


References

























Volume 2014




Journal of











Journal of


Function Spaces






Algebra














Acknowledgment


References

























Volume 2014




Journal of











Journal of


Function Spaces






Algebra
















Volume 2014




Journal of











Journal of


Function Spaces






Algebra









research article an improved adaptive deconvolution...

Documents