Filtering and Luminance Correction for Aged Photographs

Alfredo Restrepo (Palacios) and Giovanni Ramponi

IPL - DEEI - University of Trieste, Via Valerio 10, 34127 Trieste, Italy

ABSTRACT

We virtually restore faded black and white photographic prints by the method of decomposing the image intoa smooth component that contains edges and smoothed homogeneous regions, and a rough component that mayinclude grain noise but also fine detail. The decomposition into smooth and rough components is achieved usinga rational filter. Two approaches are considered; in one, the smooth component is histogram-stretched and thengamma corrected before being added back to a homomorphically filtered version of the rough component; in theother the image is initially gamma corrected and shifted towards white. Each approach presents improvementswith respect to the previously separately explored techniques of gamma correction alone, and the stretchingof the smooth component together with the homomorphical filtering of the rough component, alone. Aftercharacterizing the image with the help of the scatter plot of a 2D local statistic of the type (local intensity, localcontrast), namely (local average, local standard deviation), the effects of gamma correction are studied as theeffects on the scatter plot, on the assumption that the quality of the image is related to the distribution of dataon the scatter plot. Also, the correlation coefficient between the local average and the local deviation on the onehand, and the global average of the image play important descriptor roles.

Keywords: Objective quality measure, faded print restoration

1. Introduction

The acquisition and the virtual restoration of photo prints, via the processing of digital copies of the original artwork, is an approach of recognized cultural importance at least for two reasons: it allows the public at large toappreciate collections which would otherwise be practically unaccessible for reasons of costs and of safety of theartistic product; additionally, it makes available an image that effectively represents the original piece, freezingthe deterioration process which, even under a proper conservation environment, often can only be slowed but notstopped. This is particularly true of the restoration of faded antique photographic prints such as the collection inthe historical archive of the Fratelli Alinari Museum, in Florence. For this application, in this paper we combinethe tools of separate processing of the smooth and rough image components, histogram stretching, nonlinearfiltering, and gamma correction in an adaptive scheme; appropriate parameter values are automatically chosen,also based on the computation of objective quality indicators.

The separate role of power-law correction has been previously explored in [1], where the intensity componentof the pixels was gamma corrected, the appropriate value of γ being found automatically for each image. Asreported, the correlation coefficient ρ between the sample mean and the sample deviation for windowed datashows in general negative values when γ = 1 and increases for values of γ > 1. Likewise, the effects of linearhistogram stretching of a certain additive smooth component of the image, together with proper nonlinearfiltering of the corresponding rough image component, have been explored in [10]; a successful filtering techniquefor the virtual restoration of faded prints was developed there. We review in some detail these two techniquesin Section 2.

Section 4 describes the main contribution of the paper, consisting in the combination of the two restorationtechniques previously mentioned; before that, in Section 3, we explore in some detail the characteristics ofscatter plots of the two-dimensional statistic given by the pair (local average, local standard deviation), that

aE-mail:arestrepo@units.it, Telephone: +39.040.5587140 (A. Restrepo is currently on leave from Dpt. Ing. Electr. yElectr., Universidad de los Andes, Bogota, Colombia; arestrep@uniandes.edu.co)cE-mail: ramponi@units.it, Telephone: +39.040.5587853

was found useful in the characterization of the effects of gamma-correction in [10]. In Section 4 we combinethe techniques of histogram stretching, filtering and gamma correction; it is found that it is a very successfultechnique; several examples of the effects of the technique are given. In Section 5, we elaborate further on thesubject of image quality assessment, an important issue in the implementation of automatic and/or adaptiverestoration techniques. The paper is concluded in Section 6.

2. PREVIOUS RESULTS

We briefly describe the separate techniques of the processing of the smooth and the rough image components,and of the power-law correction which have been previously explored for the virtual correction of faded photoprints.

2.a. Separate Correction of Smooth and Rough Components

The restoration technique proposed in10 is mainly a technique for enhancing the luminance component of fadedBW or sepia photo images. Using a rational filter ,11 the image is additively decomposed into a smooth componentthat contains edges as well as smoothed homogeneous regions, and a rough component that includes fine detailand may include grain noise as well. Due to the properties of the rational filter, the smooth component containsedges and filtered homogeneous regions. The smooth component is histogram-stretched before being added backto the rational homomorphically filtered rough component. The main reason for filtering the rough component isto get rid of possible grain noise; however, this step may be skipped if it is desired that the grain characteristicsof the photograph remain.

Values for the maximum MX and the minimum MN of the intensity of the image to be used for the stretchof the histogram are obtained from a median filtered version of the original image; the reason for using a medianfiltered version is that certain outliers such as small stains and thin scratches, which may be nearly black andnearly white respectively, should not be taken into account at this point. The dynamic range of the smoothcomponent is linearly expanded as follows; to each pixel intensity i there corresponds a parameter α such that i= αMN + (1-α)MX. Using the parameter α, the intensity of the pixel is changed to i= αBk + (1-α)Wt whereBk and Wt respectively correspond to black and white. This corresponds to a conventional stretching of thehistogram of the (intensity) image.

The resulting image is the addition of this range-corrected smooth component and a scaled and processedversion of the rough component. The rough component of the image is either homomorphically filtered or scaledwith scale factor 0.5Wt/(MX-MN). The filtering is done in order to attenuate the effects of grain noise is usuallymodeled as multiplicative noise. The logarithm of the rough component is filtered with a rational filter; aftertaking antilog the processed rough version is added back to the processed smooth component.

2.b. Automatic Gamma Correction for Contrast Enhancement

Locally everywhere, faded prints have a large sample average; the sample average being a measure of localintensity. They also have a low sample variance; the sample variance being a measure of local contrast. Asa related fact, faded photo prints have a large global average as well. The application of gamma correctionwith γ > 1 alleviates this extreme situation: it reduces local averages µ allowing for larger local variances σ2.Two-dimensional statistics of the type (location, variation) are found to be useful in the characterization of theeffects of gamma correction and in the automatic choice of a nearly optimal value of γ.

The automatic gamma-correction method for the virtual restoration of faded black and white prints reportedin1 looks for a value of γ that makes the global average intensity nearly 0.5 and nearly uncorrelates the localstandard deviation from local average, for the trimmed subset of sampled windows on the lower 75% percentile,along the average component.

For any sample, there are inherent restrictions between a value of σ and the corresponding possible valuesof µ, and viceversa. For faded black and white prints, the correlation coefficient ρσ,µ between the sample meanand the sample deviation for windowed samples increases with γ, starting at a negative value for γ = 1. For thetrimmed, low-luminance subset of the points in the µ− σ plane, the value of γ for which ρσ,µ = 0 was observed

to be also a value that makes the average of the intensity of the image equal or near to 0.5. For this optimalvalue of γ, the image looks visually correct.

3. ON GAMMA CORRECTION AND σ − µ SCATTER PLOTS

We use scatter plots of (local average, local standard deviation) for the double purpose of giving an indicator ofthe quality of the image and choosing an appropriate value of γ, for the automated enhancement of the contrastin images affected by uniform fading or poor, grayish luminance distribution. It is therefore convenient to explorein some detail this relation between σ − µ scatter plots and gamma correction.

3.a. A Characteristic of the (σ − µ) - Plane

Given a windowed sample of intensity data, coded in the interval [0, 1], 0 corresponding to black and 1 to white,consider the resulting ordered pair: (sample average, sample deviation) which can be seen as an instance of alocal measure of location.vs.dispersion of the image. Such pairs can live only in a certain permitted region of theplane; in fact, the possible positions for the points (σ, µ) are restricted horizontally to abscissae in the interval[0, 1] and are vertically bounded by the horizontal axis below, and above by a certain curve that we derive next.The relative position in the permitted region of the points (σ, µ) gives valuable local information regarding theimage. Note that if µ=0 or µ=1 then necessarily σ= 0 since the sample must be constant in those cases; relatedto this is the fact derived below that the maximum possible value of σ occurs for µ=0.5.

For faded images, the distribution of the points in the permitted region is highly concentrated near the point(1, 0), indicating an image with an overall high intensity and low contrast. The effect of the application ofgamma correction is to cause a migration of the points, according to a certain flow, making the scatter plot moreuniformly distributed on the one hand and also making the overall average near 0.5, on the other; such imageslook better in general.

To get the upper bound curve for the scatter plot (σ, µ), we first derive an alternate expression for the samplevariance. Let n be the window (sample) size and let [x1, x2, ...xn] be the corresponding sample data.

σ2 = 1n

∑ni=1 x2

i - ( 1n

∑ni=1 xi)2

= 1n

∑ni=1 x2

i - 1n2

∑ni=1

∑nj=1 xixj

= nn2

∑ni=1 x2

i - 1n2 (

∑n−1i=1

∑nj=i+1 xixj +

∑ni=2

∑i−1j=1 xixj +

∑ni=1 x2

i )

= n−1n2

∑ni=1 x2

i - 2n2

∑n−1i=1

∑nj=i+1 xixj

= 1n2 [

∑ni=1(n− 1)x2

i −∑n−1

i=1

∑nj=i+1 2xixj ]

= 1n2 [

∑n−1i=1

∑nj=i+1(x

2i + x2

j )−∑n−1

i=1

∑nj=i+1 2xixj ]

= 1n2

∑n−1i=1

∑nj=i+1(x

2i + x2

j − 2xixj)

= 1n2

∑n−1i=1

∑nj=i+1(xi − xj)2

= 1n2

∑1≤i<j≤n(xi − xj)2

This expression determines a function of the variables xi which we want to maximize/minimize, subject tothe restrictions 0 ≤ xi ≤ 1 and 1

n

∑ni=1 xi = µ, for each µ. Next we find the maximum of 1

n2

∑1≤i<j≤n(xi−xj)2

subject to such restrictions, as a function of µ.

First, note that the gradient of the function

f(x) := n2σ2 =∑

1≤i<j≤n(xi − xj)2

is given by

Of(x) = 2[(n− 1)x1 −∑n

j=2 xj , · · · (n− 1)xi −∑

j 6=i xj , · · · (n− 1)xn −∑n−1

j=1 xj ]

Next note that Of(x) is normal to the line

Φ := {x ∈ Rn : x1 = x2 = ... = xn}

since

Of(x).[1, ...1] = 0.

Thus, in the restricted domain, the maximum of f occurs at points farthest from the line Φ. This is achievedas follows. Given∑n

i=1 xi = nµ = k + α;

where k = bnµc and α = nµ − k; with k an integer and α beetwen 0 and 1, the max of f is achieved byletting k of the variables xi be equal to one, another equal to α and the remaining equal to zero.

Proof (Sketch): Consider the n-dimensional cube of Rn given by C := {x ∈ Rn : 0 ≤ xi ≤ 1; i ∈ {1...n}} aswell as the (n-1)-dimensional hyperplane of Rn given by Π := {x ∈ Rn :

∑ni=1 xi = k + α}. It is desired to find

the points on C⋂

Π that are farthest away from Φ, which can be shown to be equivalent to finding the pointson C

⋂Π farthest away from the origin of Rn. This is achieved on the boundary ∂(C

⋂Π) of C

⋂Π. Consider

the n-dimensional ball B of radius√

k + α2 centered at the origin. Clearly, the (n-1)-dimensional sphere S= ∂Bintersects C

⋂Π precisely at the points of the form [1, 1, ... 1, α, 0 ... 0] (with k ones, one α and n−k−1 zeros)

and its permutations; also, the the remaining points of C⋂

Π belong to the interior of B.

Then, the maximum of f is given by max(f) = k(1− α)2 + (n− k − 1)α2 + k(n− k − 1)12 = (n− 1)α2 −2kα + nk − k2

This function (of the maximum of f in the graph of f as a function of µ) is symmetric around µ = 0.5, witha maximum there and roughly increasing for µ ∈ (0, 0.5) although actually piecewise parabolic between eachpair of consecutive values k/n and (k + 1)/n (these parabolic segments include the vertex of the parabola andare initially decreasing and then increasing.) For large n the roughness of the curve decreases and approachesa monotonic curve. See e.g. the right hand side of Fig 3, where linear interpolation has replaced the parabolicsegments.

3.b. Flow of points in the (σ − µ) - Plane under γ correction

We now study the effect of gamma correction on (σ − µ)-plots. Since gamma correction acts at the pixel levelrather than at the window level, it is not surprising that two distinct samples corresponding to the same pointin the (σ − µ)-plane behave differently under the application of gamma correction; nevertheless, it seems thatthe effects are not too different. For example, for n = 4, the samples [0.1, 0.3, 0.4, 0.5] and [0.15, 0.25, 0.35,0.55] both have an average µ = 0.325 and variance σ2= 0.0219 and, when gamma corrected, with γ = 1.1, theybecome

[0.1, 0.3, 0.4, 0.5]1.1=[0.0794, 0.2660, 0.3650, 0.4665]and have µ = 0.2942 and σ2= 0.0204; meanwhile, the other set of samples becomes

[0.15, 0.25, 0.35, 0.55]1.1=[0.1241, 0.2176, 0.3151, 0.5181]which has µ = 0.2937 and σ2= 0.0213;

In both cases, the movements occur on the same directions with respect to the corresponding values for γ=1and of close values.

Before finding a characterization of the differences in changes, for reference purposes, we compute the deriva-tives of µ(γ) := 1

n

∑ni=1 xγ

i and of σ2(γ) := 1n

∑ni=1(x

γi − µ(γ))2, with respect to γ.

ddγ µ(γ) = 1

n

∑ni=1 log(xi)x

γi

where log is natural logarithm; writing γ = 1 + δ we have

= 1n

∑ni=1 log(xi)xix

δi

denoting the ‘entropy’ xlog(x) as ε(x), we have

= 1n

∑ni=1 ε(xi)xδ

i

Similarly,ddγ σ2(γ) = d

dγ1n

∑ni=1(x

γi − µ(γ))2

= ddγ

1n2

∑1≤i<j≤n(xi − xj)2

= 1n2

∑1≤i<j≤n x2δ

i ε(x2i ) + x2δ

j ε(x2j )− 2xδ

i xδjε(xjxj)

As we show below, for γ ≥ 1, for µ ≤ 0.5, µ decreases while for µ ≥0.5, sigma increases. This explainsthe observed flow of points in the σ − µ scatter plots. In Rn, consider the difference [xγ

1 ...xγn] − [x1...xn] =

[xγ1 − x1...x

γn − xn]. We decompose this vector into a component parallel to the line Φ and a component

perpendicular to it; namely,

[xγ1 − x1, ..., x

γn − xn] =

[ 1n

∑ni=1(x

γi − xi), ..., 1

n

∑ni=1(x

γi − xi)] + [xγ

1 − x1 − 1n

∑ni=1(x

γi − xi), ..., xγ

n − xn − 1n

∑ni=1(x

γi − xi)]

Next notice that the set of points [x1...xn] ∈ Rn for which∑n

i=1 xi = M, lie on an (n − 1)-dimensionalhyperplane that is perpendicular to the line Φ and that intersects it at the point [M/n, ..., M/n]. Thus, forγ ≥ 1, the movement from [x1...xn] to [xγ

1 ...xγn] occurs in a direction that decreases µ. It also occurs in a direction

that decreases σ when, broadly speaking, µ ≤ 0.5 and that increases σ when, also broadly, µ ≥ 0.5. The reasonbeing that the flow is towards the line Φ when 1

n

∑ni=1 xγ

i ≤ 0.5 (and therefore 1n

∑ni=1 xi ≤ 0.5), and away from

the line when 1n

∑ni=1 xi ≥ 0.5 (and therefore 1

n

∑ni=1 xγ

i ≥ 0.5.) This is so because the distance from a genericpoint [y1...yn] to the line Φ decreases with the parameter t = 1

n

∑ni=1 yi when t is larger than 0.5, and viceversa,

as can be shown by derivating the distance function from [y1...yn] to the line Φ, with respect to the parameter t.

Locally a faded image typically lives in the high-luminance, low-contrast region of the sigma-mu plane.Largest contrast is only possible at medium intensities and occurs at unusual levels of resolution since the pixelsin the window would have to have half of the pixels white and the other half black. Thus, gamma correction (γlarger than one) has the double effect on points on the scatter plot of decreasing their µ coordinate on the oneside, and of increasing or decreasing their σ coordinate when the value of µ is above or below 0.5, respectively.Tipically for a faded image, the cloud of scatter points starts at the lower right end, then it starts moving leftand upwards, then left and downwards; the cloud rotates and when it has an approximate horizontal oblongshape, the correlation coefficient is zero and the global average is near 0.5.

3.c. A Step Response

Consider a sample of even size 2n consisting of n data of value m−d and n data of value m+d. The correspondingsample average and sample variance are respectively given by m and d2. After gamma correction, the values ofthe data are (m− d)γ and (m + d)γ and the new values of the sample mean µ and sample variance σ2 are givenby

µ = 12 ((m-d)γ + (m+d)γ)

and

σ2 = [(m-d)γ-µ]2 + [(m+d)γ-µ]2

Newton’s binomial formula gives mγ(1- dm )γ =

∑∞k=0

(γk

)(− d

m )k

and

mγ(1+ dm )γ =

∑∞k=0

(γk

)( d

m )k

Therefore

µ = 12mγ(

∑∞k=0

(γk

)(− d

m )k +∑∞

k=0

(γk

)( d

m )k) = mγ∑∞

k=0

(γ2k

)( d

m )2k

The new values of the sample mean and sample variance respectively are mγ(1± dm )γ−µ = mγ(

∑∞k=0

(γ2

)(1±

dm )k −

∑∞k=0

(γ2k

)( d

m )2k) = ±mγ∑∞

k=0

(γ

2k+1

)( d

m )2k+1

and

σ2 =m2γ(∑∞

k=0

(γ

2k+1

)( d

m )2k+1)2

Taking the first term only in each case, we get µ = mγ σ = γmγ−1d.

Thus, we can study the approximate behavior of points in the scatter plots of σ versus µ. A point (m, d)approximately becomes the point (mγ , γmγ−1d) after gamma correction. The difference between the resultingpoint and the original point is then (∆m, ∆d) = (m(m γ−1 − 1), d(γmγ−1 − 1)). The normalized shifts for mean

and deviation are then (mγ−1 − 1) and (γmγ−1 − 1). In the first case, it is always negative, in the second, itmay be positive or negative.

Values of these normalized shifts ∆m/m and ∆d/d are given below for several values of m and γ. The resultsare in agreement with the change of the shape of the scatter plots, for the case of edges of the type considered.

TABLE 1. Normalized Changes of intermediate value andjump ∆m/m//∆d/d of a step edge, with γ correction

γ, m : 0.00 0.25 0.50 0.75 1.001.5 -1//-1 -0.50//-0.250 -0.293//0.061 -0.134//0.299 0//0.52.0 -1//-1 -0.75//-0.500 -0.500//0.000 -0.250//0.500 0//1.03.0 -1//-1 -0.93//-0.812 -0.750//-0.250 -0.437//0.687 0//2.04.0 -1//-1 -0.98//-0.937 -0.875//-0.625 -0.578//0.687 0//3.0

The change of the intermediate value of the edge is always negative: the average decreases, the size of thechange decreasing with γ; regarding the jump value, the change is negative for small values of m (the jumpdecreases) and positive if m > 0.5 (the jump increases.) For m = 0.5, the jump can decrease or increasedepending on the value of gamma.

4. COMBINED RESTORATION TECHNIQUES

The technique for the virtual restoration of faded prints in10 can benefit from the inclusion of a gamma-correctionstep. After considering several possible ways of combining the restoration techniques described in Section 2 forthe correction of faded images, we have chosen the two set ups illustrated in Figs. 1 and 2; in the first onegamma correction is an initial step of the process while in the second one only the smooth component is gammacorrected. In each case an appropriate value of gamma larger than one is chosen, on the assumptions that thecorrelation coefficient depends monotonically on γ and that the quality of the image is a convex function ofthe value of the correlation coefficient ρ; since this type of gamma correction tends to make the image undulydark, the extra ingredient consisting of uniformly shifting the intensity of the pixels towards white, is added.The second approach has conceptual advantages, namely there should not be gamma correction on the roughcomponent; nevertheless the first, simpler approach works well particularly in the cases when the image, inaddition to being faded, has a large dynamic range; see Fig. 7. In the second approach, the image is best whenthe correlation coefficient of yrg (see Fig. 2) is zero; in the first approach, even though there is a dependency ofthe correlation coefficient and of the quality of the image on γ, as indicated before, the optimal value of gammawas chosen by the user and the approach does not seem amenable for its automatization.

RF

MF

shiftygsr

k,h

+ RFlog exp+

–

+

+yg

yd ydf

yout

Cr

Cb

RGBin

csc

Cb

CrRGBout

cscstretch

ygs

yin

powerlaw

Figure 1. Block diagram corresponding to Method 1.

In the first case, the image is initially gamma corrected; the intensity of the pixels is next uniformly shiftedtowards white so that the lightest pixels become white; next the image is filtered as in Section 2.a, with thedifference that the histogram stretching step is applied only after the separately filtered smooth and roughcomponents are added back. In the second approach, the image is first decomposed into smooth and roughcomponents; next the smooth component (only) is gamma corrected and shifted towards white, and the rough

RF

MF

shiftyr

k,h

+ RFlog exp+

–

++

+yrg

yd ydf

yout

Cr

Cb

RGBin

csc

Cb

CrRGBout

cscstretchyrgs

yin

powerlaw

Figure 2. Block diagram corresponding to Method 2.

component is added back using the value of the maximum of a median filtered version of the original image;finally, after the smooth and rough components are added back, the image is histogram-stretched using the valuesof the minimum and the maximum of a median filtered version of this filtered image. Care should be taken sincefor large values of gamma the slope of the function xγ near x = 1 is rather large and the contrast near whitepixels is therefore overemphasized.

Figure 3. Original image Horse Rider (courtesy F.lli Alinari archives, Florence)(left), and corresponding scatter plot(right).

Starting with a value of γ of one, gamma is increased by small steps and the image is processed in oneof the two set ups just described. After checking the quality of the image, it is decided if gamma should beincreased further or if the measure of quality, which depends on γ, is appropriate enough. Alternatively, sincethe computation time is beyond the range of a few seconds for large images, the search for an optimal value ofgamma can be optimized by halving an interval, e.g. γ = 1, 2, 1.5, 1.25, 1.125, etc., guided by the results of theoptimality measure. This presents improvements in the resulting visual quality of the images with respect to thepreviously separately explored techniques described in Section 2.

5. INDICATORS OF IMAGE QUALITY

In addition to a zero correlation between local average and local standard deviation, we have considered otherindicators of image quality. Objective indicators of image quality are necessary for the implementation of

Figure 4. Horse Rider processed with Method 1(left), and corresponding scatter plot (right).

Figure 5. Horse Rider processed with Method 2(left), and corresponding scatter plot (right).

automatic methods such as the one described here. For our purposes, it is assumed that the following indicatorsof visual quality of an image depend continuously on the parameter γ.

i) The global average (defined as the sum of the intensities of all pixels in the image, divided by the numberof pixels) is an indicator of the average lightness of the image. We use as an indicator of image quality a valueof global average of 0.5, which says among other things that the image is neither too light neither too dark.

ii) The visual quality of the image has been found to have a convex dependency on the correlation coefficientbetween local average and local standard deviation, given by

Figure 6. Original image Pantheon (courtesy F.lli Alinari archives, Florence)(left), and corresponding scatter plot (right).

Figure 7. Pantheon processed with Method 1(left), and corresponding scatter plot (right).

ρ(σ, µ) =∑

(σd−σ̄(µd−µ̄))√∑(σd−σ̄)2

∑(µd−µ̄)2

.

In turn, ρ has ben found to have a monotonic dependency on γ. If an image has undergone gamma correctiononly, a value of 0 of ρ is an indicator of best image quality. For a faded image the correlation coefficient starts at anegative value and increases with γ as indicated. In practice this means that at ρ = 0 the cloud of points is nearlyhorizontal in the sigma-mu plane, in turn implying that there is a range of local intensities with different degrees

Figure 8. Pantheon processed with Method 2(left), and corresponding scatter plot (right).

of local contrast. That is, that corresponding to a given contrast there are several possible local illuminances.

iii) The more uniform the covering of the sigma-mu plane is, the better the quality of the image. Which, relatedto ii) implies that as far as possible (taking into account the bounds described in Section 3.a) all combinationsof local contrast and local illumination should appear in the image. Given a grid of squares that partitions theσ − µ cube, a measure of uniform coverage is given by the number of squares for which at least one window ofthe image has a value of the 2D statistic (σ, µ) in the square.

To get a feeling for the information encoded in σ-µ plots, consider a checkerboard pattern image; a patternwith strong blacks and whites corresponds to medium average, large deviation. If the black pixels become alighter gray, the deviation decreases and the average increases; if, on the other hand, the white pixels become adarker gray, both the deviation and the average decrease. In a case of a monotonic dependence between σ andµ, either the higher the local variation the higher (resp. the lower) the intensity and the lower the local contrastthe lower (resp. the higher) the local intensity. In certain cases, for example in relation to Weber’s law, it mightbe argued that it is more important to have large variations when the average intensity is high and thereforethat there should be a positive correlation; however, for faded prints being gamma corrected at least, this is notthe case since the flow of points in σ − µ scatter plots is such that the image obtains an optimal visual qualitywhen the clouds of data are approximately horizontally aligned: in that case the correlation coefficient is zero.Nevertheless, given the characteristics of faded prints, the gamma value corresponding to zero correlation usuallymakes the image dark. Thus, we considered the addition of a luminance shift component.

6. DISCUSSION

In the case of image restoration, image quality refers to likeness to an, usually unrecoverable, original image.The criterion in practice is that of a good visual appearance according to someone’s eye. This by the way, maynot be what the photographer intended, especially in the case of artistic photography and less in the case ofdocumentary photography. Objective quality metrics have the important merit of not needing user interaction,speeding up the process of restoration, an important aspect for the restoration of large collections of pictures.

Objective quality-measures/figures-of-merit may be, in a sense, of two types. A quality measure may saywhich of a set of versions of an image is best; or, it may say on the other hand that, given two possibly very

different images, the quality of one of the images is better than that of the other image. We have consideredhere measures of the first type.

Even accepting Weber’s law as a general description of the response of the human visual system, it is clearthat the cognitive responses will be different when looking at a real world scene, and at a printed photograph ora display with the same scene. In particular, there is not an agreement on what an ideal dependence betweenlocal sample variance and sample mean should be; Oakley and Bu,2 for example, assume the local standarddeviation should be proportional to the pixel mean. Also, from several measurements we have made, we suspectthat the gamma corrrection that some standard images (such as Lena) have undergone was aimed at having anoverall intensity average of 0.5 and a zero correlation coefficient. We will explore this further by estimating thecorrection applied with techniques such as those proposed by Lin and Zhang;3 see also4-.7

We have presented a critical consideration of the use of σ − µ scatter plots as indicators of image qualityand some initial results in the restoration of faded prints using gamma correction and filtering processes thatdecompose an image into smooth and rough components. We are currently working on both aspects. Regardingthe first line of inquiry, we are considering quality measures using order statistics; regarding the second one, weare considering Retinex approaches for the decomposition of the image.

ACKNOWLEDGEMENTS

This work was partially supported by the FIRB project no. RBNE039LLC, 2005-08, and by a grant of theUniversity of Trieste. The original images used in the experiments belong to F.lli Alinari archives, Florence,Italy, who also kindly provided their digitized versions.

REFERENCES1. G. Ramponi. ‘Adaptive contrast improvement for still images and video frames.’ NSIP. 2007.2. J.P. Oakley and H. Bu. Correction of simple contrast loss in color images. IEEE trans. on IP, vol. 16, no. 2,

2007.3. S. Lin and L. Zhang. Determining the radiometric response from a single grayscale image. Proc. Comp. Vis.

Patt. Rec. (CVPR), vol. 2, pp. 66 73, 2005.4. Yoshida, V. Blanz, K. Myszkowski, H.P. Seidel. ‘Testing tone mapping operators with human-perceived

reality’. Journal of Electronic Imaging 16(1), 013004, 2007.5. S. Lin, J. Gu, S. Yamazaki, H.Y. Shum. ‘Radiometric calibration froma single image.’ Procs. of the IEEE

2004 Comp. Soc. Conf. on Comp. Vis. and Patt. Rec. CVPR’04.6. P.E. Debevec and J. Malik. ‘Recovering high dynamic range radiance maps from photographs.’ Proc. ACM

SIGGRAPH, pp. 369-378, 1997.7. J.A.J. Roufs and T.M.J. Goossens. The effect of gamma on perceived image quality. IEEE Int. Display Res.

Conf. pp. 27-31, 1988.8. A. Restrepo, A. Naranjo, M. Valderrama, M. Daniels and R. de la Vega, ‘Designing OSED’s (order statistic

edge detectors)’ Proc. IEEE ICIP 1995, vol II, pp. 173-176, 1995.9. A. Restrepo, G. Hincapie, A. Parra, ‘On the detection of edges using order statistic filters’ Proc. IEEE ICIP

1994, vol I, pp. 308-312, 1994.10. G. Ramponi and V. Bruni. ‘Virtual restoration of faded photographic prints.’ 14th EURASIP Int. Conf.

EUSIPCO-06, 2006.11. G. Ramponi. ‘The rational filter for image smothing.’ IEEE SP Letters, Vol. 3, No. 3, pp63-65, 1996.12. ‘Color Appearance Models.’ Mark D. Fairchild. Wiley, 2005.13. A. Restrepo, C. Alvarado, V. Bruni and G. Ramponi. ‘Objective measures for the evaluation of techniques

for the virtual restoration of faded sepia photographic prints.’ EVA. Florence, 2007.