fast hue and range preserving histogram … · 1 fast hue and range preserving histogram...
TRANSCRIPT
1
Fast Hue and Range Preserving HistogramSpecification: Theory and New Algorithms
for Color Image EnhancementMila Nikolova (Senior Member IEEE), and Gabriele Steidl
Abstract—Color image enhancement is a complex and chal-lenging task in digital imaging with abundant applications.Preserving the hue of the input image is crucial in a wide rangeof situations. We propose simple image enhancement algorithmswhich conserve the hue and preserve the range (gamut) of the R,G, B channels in an optimal way. In our setup, the intensity inputimage is transformed into a target intensity image whose his-togram matches a specified, well-behaved histogram. We derive anew color assignment methodology where the resulting enhancedimage fits the target intensity image. We analyse the obtainedalgorithms in terms of chromaticity improvement and comparethem with the unique and quite popular histogram based hueand range preserving algorithm of Naik and Murthy. Numericaltests confirm our theoretical results and show that our algorithmsperform much better than the Naik-Murthy algorithm. In spit eof their simplicity, they compete with well-established alternativemethods for images where hue-preservation is desired.
Index Terms—color image enhancement, scaling and shiftingmethods, hue preservation, gamut problem, exact histogramspecification, color perception.
I. I NTRODUCTION
This paper assists to the tremendous progress in digitalcolor imaging and display technology. In spite of the importantamount of research, color perception and color appearance arestill open problems. The demand for fast efficient algorithmsimproving the color content of digital images has increaseddramatically. The applications of color image improvementareabundant. They concern for example digital cameras and mo-bile phone cameras, medical imaging, video, post-productionindustry, restoration of old pictures and movies.
Typically, color images are stored and viewed using threecomponents (channels): red (R), green (G) and blue (B). In thispaper we aim to design color image enhancement methodsin the RGB space sharing three important features, namelyhue and range (gamut) preservationand low computationalcomplexity. The hue describes in each area of an image thedominant color ingredient that one really perceives, e.g.,red,orange, magenta, yellow and so on [1], [2]. The hue has thenice property of being invariant under changes of directionand intensity of the incident light [3]. Thus, by preserving
Copyright (c) 2013 IEEE. Personal use of this material is permitted.However, permission to use this material for any other purposes must beobtained from the IEEE by sending a request to [email protected]. Nikolova is with CMLA – CNRS, ENS Cachan, 61 av. President Wilson,94235 Cachan Cedex, France (email: [email protected]). Herwork was supported by the “FMJH Program Gaspard Monge in optimizationand operation research”, and by the support to this program from EDF.
G. Steidl is with the Dept. of Mathematics, University of Kaiserslautern,67663 Kaiserslautern, Germany (email: [email protected]).
Fig. 1. Histogram equalization (HE). Left: Original imageonion (MatlabIPT image credits notice). Middle: HE to each color channel independently.Right: Enhancement in three steps following [2]: RGB to HSI transform, HEof the intensity channel, then HSI to RGB transform. Here 36.1 % of thepixels have values in(255, 443.5].
the hue and enhancing the brightness, the obtained image willappear more colorful. Examples where the hue is modifiedare shown in Fig. 1.The range (gamut) preservationis oftenomitted in works on image enhancement; see, e.g. the recenttextbook [2, p. 80]. Each color channel in a digital imagecan only take a limited number, sayL, of integer values, e.g.,L = 256 for 8-bit coding. If the enhancement method produceslarger or smaller values these are clipped back to the boundaryof [0, L − 1] which also changes the hue. In Fig. 1 right36.1 % of the pixels are clipped back to 255 which yields toomany yellow pixels. Finally, alow computational complexityofalgorithms is particularly important when dealing with “mega-pixel” images taken by commercial cameras, resources inhardware implementations and extensions to video.
Remark 1. Fully automatic color image enhancement faces(at least) two major limits: i) ”The chemical compoundsthat form color receptors vary among the population. Thephysical shapes of the receptors vary among the populationand within the retina. Thus, the color vision among observerswith normal color vision varies significantly.” [1, p. 18]. ii)Image enhancement is always driven by an application: typi-cally the user needsspecificvisual information determined byhis/her purpose. Further subjective criteria are of paramountimportance [2].
Consequently, we do not look forfully automatic imageenhancement algorithms. Here we focus on histogram basedmethods. The selection of a suitable target histogram enablesthe user’s needs to be satisfied. Moreover we wish to conceivefast algorithms. In order to achieve our goals, we proposesimple algorithms composed of two stages:
(a) the intensity channel of the input RGB image is matchedto a specified histogram which gives us the target intensityimage;
(b) the RGB color values are computed based on the target
2
intensity image so that they satisfy the hue and gamutconstraints in an optimal way.
These stages are briefly commented below.Stage(a). Exact histogram specification (HS), also known
as histogram matching, of single-valued (gray-valued) imagesaims to transform an input image to an output image which ex-actly fits a prescribed target histogram. Histogram equalization(HE) is a particular case of HS where the target histogram isuniform. Usually HE leads to unnatural images and should notbe the target of choice. We do not focus on the constructionof target histograms. Instead, we adopt a simple approachinspired by [4]. For digital image HS is an ill posed problem[5]. The clue to ensuring exact HS is to obtain ameaningfultotal strict orderingof all pixels in the input digital image. Weperform exact HS using the algorithm in [32] which currentlyprovides the best pixel ordering in terms of quality and speed.
Stage (b). The extension of histogram methods to colorimagesis a quite complex task. The histogram of a gray-valueimage is 1-D while the histogram of a color image is 3-Dwhich gives rise to an under-determined problem. For instance,applying HE to each color channel independently changes thecolor content (the hue) of the image, see Fig. 1 middle. Furtherit is not easy to produce color images that respect the rangeconstraints; see Fig. 1 right. As a central result of this paper,we propose a general and optimal hue and range preservingcolor assignment methodology.
Related work. Since the inaugural paper [6] providingPDE-based and variational formulationsfor image histogrammodifications, these methods were further expanded to dealwith color image enhancement; see, e.g., [7], [8], [9], [10].These approaches provide flexible tools to incorporate variousknowledge on human visual perceptual phenomena, typicallyin relation with Retinex theory [11]. An automatic colorenhancement (ACE) algorithm for digital images, mimickingsome characteristics of the human visual system, has beenproposed in [12] and refined in [13]. A fast implementationof ACE was developed in [14]. A perceptually inspired vari-ational approach allowing a more flexible control of contrastadjustment and attachment to data was proposed in [7]. Anumerical implementation of the gradient descent techniqueapplied to the corresponding energy functionals coincideswith the equation of the ACE. Some basic requirements for”perceptually inspired” objectives were formulated in [8]andgave rise to successful algorithms [8], [15].
Next we summarize the main approaches viahistogrammodificationof color images following a chronological order.Since the suitably normalized histogram of an image is alsothe empirical probability distribution of its pixel values, astatistical vocabulary is used in many papers. In [16] a 3-Dcolor histogram in the RGB color space was proposed for HE;the resultant images present an excessive brightness for brightpixels, see [17]. A method that preserves both the hue andthe range (gamut) constraints was inaugurated by Naik andMurthy in [18]. Even though this article did not show colorimage applications, it is a state-of-the-art method applied inmany papers; see, e.g., [19], [17]. As to the choice of thecolor space, some methods work directly in the RGB spacewhile others operate in transformed color spaces, e.g., LHS,
HSI, YIQ, HSV, etc., see [5]. When processing is done ina transform color space, coming back to the original RGBspace typically generates a gamut problem, as cautioned in[18]. Beyond the additional numerical cost, a post-processingin RGB is then needed (often realized using [18]). Gray-valuegrouping was tentatively extended to color HE in [20]. In[21], a new definition of the histogram of a color image wasintroduced whose cumulative distribution function (cdf) is theproduct of the marginal cdf’s of each color channel. Thenthe color values are increased / decreased by the same amountiteratively. This work was refined in a later paper [22]. Anotherapproach, developed in [19], is to work in the HSI space wherethe hue and the saturation are equalized and then processedusing probability smoothing. All pixels in the RGB space thatpresent gamut problem are corrected using [18]. A genericbrightness preserving dynamic histogram equalization scheme,composed of five steps, was proposed in [23]. This scheme wasapplied to color images in several ways, including transformsinto other color spaces. The work in [17] demonstrates thatthe methods in [19], [21], [16] based on higher dimensionalhistogram definition, increase the brightness of the imageand cannot fit the prescribed uniform histogram. The mainconclusion is thatonly the 1-D histogram of the intensitychannel can be considered for equalization.The new colorvalues are then computed using the algorithm in [18]. Themethod in [17] was recently improved in [24]. In order to avoidthe excessive contrast enhancement due to HE, a histogrammixing strategy was applied in [25]. There are also manyhistogram based techniques where the enhancement functionisanS-type, or power, or logarithmic transform; see, e.g., [26],[27], [28], [29]. In particular, the approach in [27] is based onmodels for color perception and is automatic. Unfortunately,there are no algorithms nor tests on color images.
Contributions. We propose a general affine model forfast hue and range preserving image enhancement in theRGB space which gives rise to Algorithm 3. Two simple butimportant instances of this algorithm are the Multiplicativealgorithm 4 and the Additive algorithm 5. We show how theoutcome of Algorithm 3 can be faithfully approximated asa convex combination of the images obtained by Algorithm4 and Algorithm 5, which is quite practical. The enhance-ment performances of our algorithms and the Naik-Murthyalgorithm [18] are analyzed in terms of their chromaticityimprovement. In all cases, our algorithms clearly outperformthe algorithm in [18] recently applied to color images in[17]. All numerical tests confirm our theoretical results. Ouralgorithms are simple and fast. They are really efficient whenone wishes to give a better clarity of images (not too alteredby artifacts) while preserving the original color ambience.
Outline. In Section II we sketch our HS method and presentthe Maik-Murthy algorithm [18]. Section III presents ourapproach for color image enhancement. In Section IV weevaluate our algorithms and the algorithm in [18] analyticallyin terms of saturation as well as qualitatively. Section Vpresents numerical results. Conclusions and points for futurework are drawn in Section VI.
The proofs of all statements are given in the Appendix.
3
II. PRELIMINARIES
Let w = (wr, wg, wb) be an RGB image of sizeM × N ,wherewc ∈ {0, . . . , L − 1}, c ∈ {r, g, b} are its red, greenand blue channels, respectively. For8-bit images we haveL =256. We reorder each color channel columnwise into a vectorof size n := MN and address the pixels by the index setIn := {1, · · · , n}.
A. Histogram Specification
The intensityof an RGB imagew is defined by [5]
f(w) :=1
3(wr + wg + wb). (1)
Thenf has3(L−1)+1 different values in13{0, · · · , 3(L−1)}.
Remark 2. Instead of the intensityf we can also work withother convex combinations of RGB values. E.g., we can usethe weights0.299, 0.587 and 0.114 which are in proportionto the human perception of the RGB channels, see [1], [2].
We want to find an intensity imagef with gray values in{0, . . . , L − 1} which has a specified (target) histogramh =(h1, . . . , hL), i.e., h[k] := ♯ {i ∈ In : f [i] = k − 1}, k =1, . . . , L, where ♯ stands for cardinality. Such exact HS canalmost never be achieved for images with a small number ofdifferent values compared to the number of pixels using theclassical statistical method based on the cumulative densityfunction, see [5]. Instead we will apply a procedure based onameaningfulstrict ascending ordering of the pixels inf . Variousordering algorithms for digital images were proposed in theliterature, see e.g. [30], [31], [33]. The method in [32], basedon [33], provides currently the best way in terms of speed andquality to order the pixels in digital images. The basic ideais tominimize a smoothedℓ1−TV functional by simple fixed pointiterations with the original image as initialization. After a fewiterations the approximate minimizer has entries which differ(up to very few outliers) pairwise from each other while theordering of the original gray values is retained. Let∇ denotethe discrete gradient operator (horizontal and vertical forwarddifferences), see [32],∇T its transposed and let
η(t) :=t
α+ |t|and η−1(y) =
αy
1− |y|, (2)
whereα := 0.05 is the default value. Note thatη = θ′ whereθ(t) := |t| −α log(1 + |t|
α), see [32]. Once a strict ordering is
obtained, exact HS is direct. Our HS algorithm reads as:
Algorithm 1 HS using strict ordering [32]
Initialization: u(0) := f , β := 0.1, target histogramh,iteration numberK (defaultK := 5), c0 := 0.———————————————————————–1. Fork = 1, . . . ,K compute
u(k) := f − η−1(β∇Tη(∇u(k−1))
).
2. Obtain the ordering{ij}nj=1 of In from the ascendingsort of the entries ofu(K).3. Fork = 0, . . . , L− 1 setck+1 := ck + hk and
f [ck + 1] = . . . = f [ck+1] = k.
The importance of a meaningful strict ordering for HS isillustrated in Fig. 2 in the context of HE. The Matlab built-infunction histeq does not involve a strict ordering off andthe resulting histogram off is not uniform. This entails someartifacts shown in (a). Such artifacts are not observed in (b)obtained using our Algorithm 1. The colors in Fig. 2(a)-(b)were assigned using Algorithm 2 given in the next subsection.
original (a) histeq histogram of (a) zoom of (a)
0 255
(b) Algorithm 1 histogram of (b) zoom of (b)
0 255
Fig. 2. Illustration of the importance of a meaningful ordering. Top: Originalimage and application of Matlabhisteq. Bottom: Application of Algorithm 1.
B. Hue and Range Preservation
Range preservation is a mandatory constraint for all digitalimaging devices [4]. A transformed versionw of w can becorrectly depicted only if
wc[i] ∈ [0, L− 1] ∀ i ∈ In ∀ c ∈ {r, g, b} , (3)
since no more thanL digits can be displayed. Otherwise,the obtained image is modified according to the visualizationdevice - which is quite an ad-hoc option; see e.g., Fig. 1 right.
The hue of an imagew is given byH(w) = 0 if wr =wg = wb and otherwise by
H(w) :=
{θ if wb ≤ wg,360o − θ if wb > wg,
(4)
where
θ := arccos12 ((wr − wg) + (wr − wb))
((wr − wg)2 + (wr − wb)(wg − wb))1
2
,
see [5]. Note that the denominator ofθ can be rewritten as(12 ((wr − wg)
2 + (wr − wb)2 + (wg − wb)
2))1
2 .
Remark 3. The simplest hue and range preserving methodis to apply the same affine mappingξ(w) := aw + b toall pixels, computinga and b so that the least and thelargest pixels inξ(w) are 0 and L − 1, respectively. Letwmax := max{wc[i]; c ∈ {r, g, b}, i ∈ In} and letwmin :=min{wc[i]; c ∈ {r, g, b}, i ∈ In}. Thenξ(w) given by
ξ(w) := (L − 1)w − wmin
wmax − wmin(5)
is the desired stretching ofw. For example, see Fig. 16, top.
It is easy to see that the hue of the modified imagew isalso preserved if the color values ofeachpixel are modifiedby the same affine transform
wc[i] = a[i]wc[i] + b[i], c ∈ {r, g, b}, (6)
where the constantsa[i] and b[i] have to be chosen for anyi ∈ In. Finding other appropriate hue-preserving transforms is
4
an interesting problem. Fora[i] = 1, model (6) amounts to anadditive transform, usually calledshifting. For b[i] = 0, it is alinear/multiplicativetransform known asscaling. Both scalingand shifting have been introduced in [34], [35]. In general,theresult of (6) fails the range constraint (3).
The gamut problem was examined by Naik and Murthy in[18] in the scaling case fora[i] = f [i]/f [i], wheref is a targetintensity. If f [i]/f [i] > 1, the range constraint (3)might notbe guaranteed. In such a case the authors propose to avoid thepotentialproblem by switching from the RGB color space tothe CMY (Cyan= L − 1−R, Magenta= L − 1−G, Yellow= L−1−B) space and then to transform back into RGB. Thiscorrection step reads for allc ∈ {r, g, b} as
wc[i] = L− 1−L− 1− f [i]
L− 1− f [i](L− 1− wc[i]) if
f [i]
f [i]> 1.
This formula is equivalent towc[i] =L−1−f[i]L−1−f [i](wc[i]−f [i])+
f [i], so that the algorithm in [18] can be formulated as follows:
Algorithm 2 Naik and Murthy [18]
1. Compute the intensityf of w and the target intensityf .2. For i ∈ In compute
(i) wc[i] :=f [i]f [i] wc[i] if f [i]
f [i] ≤ 1
(ii) wc[i] :=L−1−f[i]L−1−f [i](wc[i]− f [i]) + f [i] if f [i]
f [i] > 1
Algorithm 2 is often used to avoid the gamut problem.
III. N EW AFFINE HISTOGRAM SPECIFICATION MODELS
In this section we develop our affine color enhancementmethodology. Given an RGB imagew and a target histogram,we compute its intensityf by (1) and then the target intensityimagef by Algorithm 1. Our next goal is to transformw intoan imagew having the following properties:
(a) Intensity fit: f = 13 (wr + wg + wb).
(b) Hue preservation: the hue ofw andw coincide.(c) Range preservation:0 ≤ wc ≤ L− 1, c ∈ {r, g, b}.
We adopt the hue preserving affine transform (6). Summingup overc in (6) shows that property (a) holds if and only if
f [i] = a[i]f [i] + b[i] ⇔ b[i] = f [i]− a[i]f [i]. (7)
Therefore the affine model (6) obeys (a) if and only if
wc[i] = a[i](wc[i]− f [i]) + f [i], c ∈ {r, g, b}. (8)
Two particular instances of (6) are the following:
− Scaling: For b[i] = 0, model (8) reads as
wc[i] =f [i]
f [i]wc[i], c ∈ {r, g, b}. (9)
− Shifting: For a[i] = 1, model (8) becomes
wc[i] = wc[i]− f [i] + f [i], c ∈ {r, g, b}.
We have to adapt these models so that they preserve the range.We will use for all i ∈ In the magnitudes
M [i] := max{wc[i] : c ∈ {r, g, b}},
m[i] := min{wc[i] : c ∈ {r, g, b}}(10)
and similarlyM [i] for the maximum andm[i] for the mini-mum of the RGB components ofw[i] given in (8).
Remark 4. By the definitions off , m andM we have
0 ≤ m[i] ≤ f [i] ≤ M [i] ≤ L− 1.
Further M [i] = f [i], resp.,m[i] = f [i] if and only ifwr [i] =wg[i] = wb[i], i.e.,w[i] is a gray pixel.
A pixel w[i] has anupper gamutproblem if M [i] > L− 1and alower gamutproblem if m[i] < 0. We will treat thesegamut problems in an optimal way in the following sense:
• Assume that we have an upper gamut problem, i.e.,M [i] >
L − 1 for somei ∈ In. ThenM [i] = wk[i] for somek ∈{r, g, b} and the best correction of this pixels is clearly tochoosea[i] in (8) so thatwk[i] has the closest value in therange, i.e.wk[i] = L− 1, see, e.g., [4]. Equivalently,
L− 1 = a[i](M [i]− f [i]) + f [i]. (11)
From Remark 4 we know that for non gray-valued pixelsM [i]− f [i] > 0, so that
a[i] =L− 1− f [i]
M [i]− f [i]≥ 0.
Thus, for the upper gamut problem, the corrected colorvalues of pixeli are given by
wc[i] =L− 1− f [i]
M [i]− f [i](wc[i]−f [i])+f[i], c ∈ {r, g, b}. (12)
• Assume we have a lower gamut problemm[i] < 0 for somei ∈ In. Let k ∈ {r, g, b} be such thatwk[i] = m[i]. Then theoptimal correction in (8) obeying (c) is to setwk[i] = 0, i.e.,
0 = a[i] (m[i]− f [i]) + f [i]. (13)
By Remark 4f [i]−m[i] > 0 for non gray pixels, so that
a[i] =f [i]
f [i]−m[i]≥ 0.
Hence for the lower gamut problem, the corrected colorvalue ati is given by
wc[i] =f [i]
f [i]−m[i](wc[i]−f [i])+ f[i], c ∈ {r, g, b}. (14)
A. Affine Algorithm with Optimal Range Preservation
Our affine model is a convex combination of the shiftingand scaling models for someλ ∈ [0, 1]:
wc[i] = λf [i]
f [i]wc[i] + (1− λ) (wc[i]− f [i] + f [i]) (15)
= a[i](wc[i]− f [i]) + f [i],
5
where
a[i] := λf [i]
f [i]+ 1− λ (16)
with upper and lower gamut corrections (12) and (14) ifnecessary. Clearly, forλ = 1 we have the scaling model andfor λ = 0 the shifting one. Algorithm 2 corresponds toλ = 1in (15) but the gamut problem is tackled just by thresholdinga[i] at one; this appears to be an important drawback.
The next propositions show that correcting the gamut prob-lems using (12) or (14) does not yield new gamut problems.
Proposition 1. Assume that pixeli ∈ In in (15) has an uppergamut problem. Then its correctionwc[i] in (12) satisfies
0 ≤ wc[i] ≤ L− 1, c ∈ {r, g, b}.
Let us mention that a lower gamut problem can obviouslynot appear for the multiplicative model (9) i.e. forλ = 1.
Proposition 2. Let λ ∈ [0, 1). Assume that pixeli ∈ In in(15) has an lower gamut problem. Then its correctionwc[i]in (14) satisfies
0 ≤ wc[i] ≤ L− 1, c ∈ {r, g, b}.
Using Propositions 1 and 2, the optimal range-preservingapproximation of our affine model (15) can be computed inone iteration where all pixels in the input image are modifiedonly once. The algorithm is described below.
Algorithm 3 Optimal Range-Preserving Enhancement1. Compute the intensityf of w by (1) and the targetintensity f using Algorithm 1 for givenh.2. For i ∈ In computeM [i] andm[i] by (10). If f [i] = 0,then w[i] := 0. Otherwise compute
a[i] := λf [i]
f [i]+ (1− λ),
Gλm[i] := a[i](m[i]− f [i]) + f [i],
GλM [i] := a[i](M [i]− f [i]) + f [i]
and for allc ∈ {r, g, b}:
(i) wc[i] := a[i] (wc[i]− f [i]) + f [i]
if Gλm[i] ≥ 0 and Gλ
M [i] ≤ L− 1 ,
(ii) wc[i] :=L−1−f[i]M [i]−f [i] (wc[i]− f [i]) + f [i]
if GλM [i] > L− 1,
(iii) wc[i] :=f [i]
f [i]−m[i] (wc[i]− f [i]) + f [i]
if Gλm[i] < 0.
Algorithm 3 and the role ofλ is illustrated in Fig. 3. Theimages were computed for Gaussian target histogram withparameters(l, r) = (0.9, 0.1), see (27), Sec. V-A.
B. Multiplicative, Additive Algorithms and their Combinations
For λ ∈ {0, 1} Algorithm 3 yields two simple range pre-serving scaling and shifting algorithms called Multiplicative
Fig. 3. Input imagecouple(top left) and its enhancement by our Algorithm3 for λ = 0, 1
4, 1
2, 3
4, 1. Here the size of the setsU(λ) in (18) in percent
of all image pixels are0.70, 1.24, 2.12, 3.13, 4.17 and the setsL(λ) areempty. All nuances between the very colorful imagew× and the grayishimagew+ can be also obtained by their convex combinations in (17).
and Additive algorithm, respectively. Observing that
G0m[i] = m[i]− f [i] + f [i],
G0M [i] = M [i]− f [i] + f [i],
G1M [i] = f[i]
f [i]M [i]
these algorithms read as follows:
Algorithm 4 Multiplicative Color Enhancement
1. Compute the intensityf of w and the target intensityfusing Algorithm 1.2. For i ∈ In computeM [i] by (10). If f [i] = 0, thenw[i] := 0. Otherwise compute
G1M [i] = f [i]
f [i]M [i]
and for allc ∈ {r, g, b}:
(i) wc[i] :=f[i]f [i] wc[i]
if G1M [i] ≤ L− 1,
(ii) wc[i] :=L−1−f[i]M [i]−f [i] (wc[i]− f [i]) + f [i]
if G1M [i] > L− 1.
Algorithm 5 Additive Color Enhancement
1. Compute the intensityf of w and the target intensityfusing Algorithm 1.2. For i ∈ In computeM [i] andm[i] by (10). If f [i] = 0,then w[i] := 0. Otherwise compute
G0m[i] = m[i]− f [i] + f [i] andG0
M [i] = M [i]− f [i] + f [i]
and for allc ∈ {r, g, b}:
(i) wc[i] := wc[i]− f [i] + f [i]
if G0m[i] ≥ 0 and G0
M [i] ≤ L− 1,
(ii) wc[i] :=L−1−f[i]M [i]−f [i] (wc[i]− f [i]) + f [i]
if G0M [i] > L− 1,
(ii) wc[i] :=f[i]
f [i]−m[i](wc[i]− f [i]) + f [i]
if G0m[i] < 0.
6
Let w× be obtained by the Multiplicative algorithm 4 andw+ by the Additive algorithm 4. For someλ ∈ [0, 1], consider
wc := λw×c + (1 − λ)w+
c ∀c ∈ {r, g, b}. (17)
Since wc is a convex combination ofw× and w+, it obeysall conditions (a)-(c). We want to know ifwc can replace theaffine Algorithm 3. In order to answer this question, we set
U(λ) := {i ∈ In : GλM [i] > L− 1},
L(λ) := {i ∈ In : Gλm[i] < 0}.
(18)
HereU(λ) corresponds to the upper gamut step (ii) andL(λ)– to the lower gamut step (iii) in Algorithm 3.
Proposition 3. The setsU(λ) andL(λ) defined in(18) fulfillL(1) = ∅ and
U(λ1) ⊆ U(λ2), L(λ1) ⊇ L(λ2), 0 ≤ λ1 < λ2 ≤ 1.
In particular, (18) yields
U(1) ={i ∈ In : f [i]
f [i]M [i] > L− 1},
U(0) ={i ∈ In : f [i]− f [i] +M [i] > L− 1
},
L(0) ={i ∈ In : f [i]− f [i] +m[i] < 0
}.
(19)
From Proposition 3 one hasU(0) ⊆ U(1). The notation in(19) enables Algorithms 4 and 5 to be restated as follows:− Multiplicative algorithm(λ = 1)
(i) wc[i] =f [i]f [i] (wc[i]− f [i]) + f [i] if i ∈ In \ U(1),
(ii) wc[i] =L−1−f[i]M [i]−f [i] (wc[i]− f [i]) + f [i] if i ∈ U(1).
(20)
− Additive algorithm(λ = 0)
(i) wc[i] = (wc[i]− f [i]) + f [i] if i ∈ In \ {U(0) ∪ L(0)},
(ii) wc[i] =L−1−f [i]M [i]−f [i] (wc[i]− f [i]) + f [i] if i ∈ U(0),
(iii) wc[i] =f [i]
f [i]−m[i] (wc[i]− f [i]) + f [i] if i ∈ L(0).(21)
The relation betweenwc in (17) and the outcomew of Al-gorithm 3 for the sameλ is described in the next proposition.
Proposition 4. Let w be obtained by Algorithm 3 andwby (17) for the sameλ ∈ [0, 1]. Then it holds fori ∈In \
{U(1) \ U(0) ∪ L(0)
}that wc[i] = wc[i].
The setsU(1), U(0) andL(0) are usually small for reason-able target intensity imagesf (see Table I) andU(1) \ U(0)contains generally much less pixels thanU(1). If we wishto see the enhancement resultsw of Algorithm 3 for variousλ ∈ [0, 1], Proposition 4 justifies to compute insteadw by (17)which is much more practical. Thus, by slidingλ in (17), wecan easily move between the two models.
IV. COMPARISON OF THE ALGORITHMS
A. Saturation Properties
Here we analyze the saturation of images enhanced by ourmethods and by the Naik-Murthy algorithm. Thesaturationofan RGB imagew in the HSI model [5] is defined by
S(w) :=
1−min{wr, wg, wb}
f(w)if f(w) > 0,
0 if f(w) = 0 .(22)
Proposition 5. Let S(w[i]) andS(w[i]) denote the saturationof pixel i in the input imagew and the imagew obtained byour Algorithm 3, respectively. Iff [i] ∈ {m[i],M [i]} we haveS(w[i]) = 0. Otherwise the obtained saturation is given by
(i) S(w[i]) = S(w[i])(λ+ (1− λ) f [i]
f [i]
)
if i ∈ In \ {U(λ) ∪ L(λ)},
(ii) S(w[i]) = S(w[i]) f [i]
f [i]
L−1−f [i]M [i]−f [i] if i ∈ U(λ),
(iii) S(w[i]) = 1 if i ∈ L(λ).
To clarify the comparison, all magnitudes relevant to Al-gorithm 2 (Naik and Murthy) hold the superscript•, thoserelevant to Algorithms 4 (Multiplicative) and 5 (Additive)havethe superscripts× and+, respectively. In particular, we obtain:− Algorithm 4 (Multiplicative)
(i) S(w×[i]) = S(w[i]) if i ∈ In \U(1),
(ii) S(w×[i]) = S(w[i]) f [i]
f [i]
L−1−f [i]M [i]−f [i] if i ∈ U(1).
(23)− Algorithm 5 (Additive)
(i) S(w+[i]) = S(w[i]) f [i]
f[i]if i ∈ In \ (U(0) ∪ L(0)),
(ii) S(w+[i]) = S(w×[i]) if i ∈ U(0),
(iii) S(w+[i]) = 1 if i ∈ L(0).(24)
Let us denote
V :=
{i ∈ In :
f [i]
f [i]> 1
}. (25)
By (19) and Remark 4 we find that ifi ∈ U(1) then f [i]f [i] >
L−1M [i] ≥ 1 and that ifi ∈ L(0), then f [i]
f [i] < 1−m[i]f [i] < 1. Hence
V ⊇ U(1) and L(0) ⊂ In \V . (26)
Using the notation in (25), case (i) in Algorithm 2 (Naik-Murthy) holds for anyi ∈ In \ V and step (ii) holds for anyi ∈ V . The saturation of images enhanced by applying the Naik- Murthy Algorithm 2 is given by the following proposition.
Proposition 6. Let S(w[i]) and S(w •[i]) denote the satura-tion of pixeli in the input imagew and the imagew • obtainedby the Naik-Murthy Algorithm 2, respectively. Then
(i) S(w •[i]) = S(w[i]) if i ∈ In \V ,
(ii) S(w •[i]) = S(w[i]) f [i]
f[i]
L−1−f [i]L−1−f [i] if i ∈ V .
Remark 5. Proposition 3 and(26) show thatV ⊇ U(1) ⊇U(0). Note that all these inclusions are almost always strict;see Table I. E.g., in(26) we findU(1) = V if and only ifM [i] = L− 1 for all i ∈ V .
Using Propositions 5 and 6, the saturation that Algorithms 4,5 and 2 provide can be rigorously compared.
• Let i ∈ In \V . Theni 6∈ U(1) and i 6∈ U(0). Hence
S(w[i]) = S(w×[i]) = S(w •[i]) ≤ S(w+[i]),
7
where the last inequality becomes an equality only forf [i] = f [i]. Beyond this case, only the Additive algorithms 5increases the saturation iff [i] < f [i]. But since the outputintensity is decreased, the perceived colorfulness of the pixelis decreased.
• Let i ∈ V \U(1) = {i ∈ In : 1 < f [i]f [i] ≤ L−1
M [i]}. Theni 6∈ (U(0) ∪ L(0)) and consequently
S(w[i]) = S(w×[i]) > S(w+[i]) > S(w •[i])
and S(w •[i]) decreases faster thanS(w+[i]) when f [i]increases because
S(w •[i])
S(w+[i])=
M [i]− f [i]
L− 1− f [i]< 1.
• Let i ∈ U(1). Then
S(w[i]) > S(w×[i]) ≥ S(w •[i]),
where the equality is reached if and only ifM [i] = L− 1.But for most of the pixels one hasM [i] < L− 1. Further,
S(w[i]) > S(w+[i]) ≥ S(w •[i]),
where the equality holds if and only ifM [i] = L − 1 andi ∈ U(0). So the inequality is strict for most of the pixels.
In all cases, the images enhanced by the Maik-Murthy al-gorithm have the weakest saturation. OnIn \V , where thetarget intensity is less than the input intensity, the Additivealgorithm 5 gives a better saturation than the Multiplicativealgorithm 4. OnV \U(1) the Multiplicative algorithm 4 givesrise to a better saturation than the Additive algorithm 5.
B. Qualitative comparison
We begin with a simple but instructive example where weapply Algorithms 4, 5 and 2 to two different ”images” eachcomposed of one dark and one bright pixel, resp.,
wdark = (25, 48, 32), wbright = (80, 172, 108)
having the same hue but different intensitiesfdark = 35 andfbright = 120. In Figs. 4 and 5, the input pixelsw are shownon the top row, while the next rows detail the results of thealgorithms w.r.t. the target intensityf ∈ {0, . . . , 255} givenon thex-axis. By (19) we see that the pixel belongs toU(1)for f > (L−1)f
M=: fU(1), to U(0) for f > (L − 1) − M +
f =: fU(0), to L(0) for f < f − m =: fL(0) and toV forf > f := fV . The corresponding values for our dark andbright image are given in the following table:
fU(1) fU(0) fL(0) fVwdark 185.9 242 10 35wbright 177.9 203 40 120
Fig. 4 deals with the dark pixelwdark. The Multiplicativealgorithm 4 (i) is applied forf ∈ [0, 185.9]. All color valuesare multiplied byf /f , where f/f > 1 for f > 35 whichyields a clear increase of the distance between all colorchannels. The third row shows a pleasant enhancement of thedark input pixel. By (23) the input saturation is preserved.TheAdditive Algorithm 5 (i) is performed forf ∈ [10, 242], whereall color values are increased by the same amountf−f . Since
w = (25, 48, 32), f = 35, H(w) = 137.3, S(w) = 0.29
Alg. 4 (×) Alg. 5 (+) Alg. 2 (NM)(w×
r , w×
g , w×
b ) (w+r , w+
g , w+b ) (w •
r , w•
g , w•
b )
0 35 186 2550
132.1
169.1
255
10 35 2420
20.7
238255
0 35 2550
24.2946.63
255
w× w+ w •
35 255 35 255 35 255
S(w×) S(w+) S(w •)
0 35 186 2550
0.29
1
10 35 2420
0.29
1
0 35 2550
0.29
1
Fig. 4. Enhancement of a quite dark pixel shown in the first row. Secondand third rows: the output intensityf is on thex-axis and the plots depict theresults of Algorithms 4, 5 and 2. The second row specify the value of eachcolor channel. The third row shows the resulting colorw w.r.t. f and the lastrow plots the saturation of the output pixel as a function off .
the input pixel is quite dark, the valueswc, c ∈ {r, g, b} andf are relatively close to each other. For this reason, all colorchannelsw+
c remain close to each other. On[10, 35) we havef > f so by (24),S(w+) > S(w) andS(w+) continuouslydecreases from 1 toS(w) = 0.29. On [35, 242], S(w+)decreases fromS(w) = 0.29 to 0.04 according toS(w)f/f .This explains why the colors on the third row remain quitedull, compared to Algorithm 4. For Algorithm 2, case (i) holdsonly for f ∈ [0, 35], where the input saturation is preserved.If f > 35, step (ii) is performed andS(w •) decreases muchfaster than in Algorithm 5. As a consequence, on(35, 255]the enhanced colors tend to be nearly equal and the obtainedcolor values are nearly gray, see the third row in the figure.
Fig. 5 shows the performance for the brighter pixelwbright.The Multiplicative algorithm 4 (i) is applied forf ∈ [0, 177.9].The input saturation is preserved. The Additive algorithm 5(i)holds for f ∈ [40, 203]. On [40, 120] the recovered saturationdecreases from 1 toS(w) = 0.33 and on(120, 203] it slowlydecreases to0.6S(w). In Algorithm 2, step (i) holds forf ≤f = 120 where the input saturation is unchanged. Step (ii) isapplied forf ∈ (120, 255] – the interval is not so large as inFig. 4 and the saturation decreases much less fast to zero. Onthe 3rd row one sees that the colors obtained with all the threealgorithms are quite similar.
Remark 6. From Fig. 4, if a dark pixel has a wrong hue (e.g.due to compression or printing artifacts, noise, color cast,etc.), the Multiplicative algorithm 4 can magnify the intensityof this wrong color. If the input image contains a lot of suchpixels, the Additive algorithm 5 can be a better choice.
Our conclusions drawn in Subsection IV-A and our findingsfor one pixel images are confirmed by our tests on the two im-ages,bungalow(underexposed) andflower (slightly lustreless)depicted in Fig. 6 and 7. The distribution off [i]/f [i] for thesetwo images is very different – the first one ranges on[0, 18]and the second one on[0, 1.22]. Roughly speaking,bungalow
8
w = (80, 172, 108), f = 120, H(w) = 137.3, S(w) = 0.33
Alg. 4 (×) Alg. 5 (+) Alg. 2 (NM)(w×
r , w×
g , w×
b ) (w+r , w+
g , w+b ) (w •
r , w•
g , w•
b )
0 120 178 2550
118
159.3
255
0 40 120 203 2550
27.3
89.7
162
190
255
0 120 2550
79.33
107.1
170.6
255
w× w+ w •
120 255 120 255 120 255
S(w×) S(w+) S(w •)
0 120 178 2550
0.33
1
0 40 120 203 2550
0.33
1
0 120 2550
0.33
1
Fig. 5. Enhancement of a quite bright pixel shown in the first row.Arrangement of images as in Fig. 4.
Input intensity histogram Target histogram
0 221 0 255
Input image Alg. 4 (×)
Alg. 5 (+) Alg. 2 (NM)
Fig. 6. Original imagebungalow(660× 1024) and enhanced versions. Forunderexposed images the Naik-Murthy algorithm 2 gives nearly gray-valuedresults. The Multiplicative algorithm 4 gives the most colorful image. TheAdditive algorithm 5 yields color values between those of the multiplicativeand the Naik-Murthy algorithm; it performs better than the last one.
mimics the phenomena explained for Fig. 4 andflower thoserelevant to Fig. 5.
V. NUMERICAL RESULTS
Here we demonstrate the performance of our algorithms torender images where we want to preserve the hue.
A. Target histograms
Our algorithms depend, up to a certain degree, on thechoice of a target histogram for the intensity channel. Varioustarget histograms avoiding the drawbacks of HE have beenproposed in the literature. Some of them leave gaps in thetarget histogram which can yeild artifacts as in Fig. 2, see e.g.[20], [26], others preserve the input brightness which limitsthe enhancement of underexposed images, see, e.g. [23], [28].
Input intensity histogram Target histogram
2 255 0 255
Input image Alg. 4 (×)
Alg. 5 (+) Alg. 2 (NM)
Fig. 7. Original imageflower (300 × 400) and enhanced versions. AllAlgorithms 2, 4 and 5 produce very similar results.
The models proposed in [25] combine the input image his-togram and a uniform histogram using various penalties andparameters. Instead, we adopt a simple and intuitive approach.
A common way for histogram based enhancement is to usethe histogram of a well exposed example image; see, e.g., [4].Commercials in photography and image processing software(e.g., Photoshop) mention that well exposed pictures typicallyhave bell-shaped histograms. Based on these advises we focuson target histograms whose shapes are Gaussian functionshG,with domain[0, L− 1], fixed so that
l := hG(0) ≤ 1, maxx∈[0,L−1]
hG(x) = 1 and r := hG(L−1) < 1 .
A user has to choose two parameters:
• l ∈ (0, 1] which is the desired portion of dark pixels;• r ∈ (0, 1] drawing the desired portion of light pixels.
Note that one cannot choosel = r = 1. Given l ∈ (0, 1] andr ∈ (0, 1], the shapeof the target histogram reads as
hG(x) = exp
(−(x− µ)2
σ
), x ∈ [0, L− 1], for
µ= −(L−1)(ln l−√ln l ln r)
ln r−ln l, σ= (L−1)2(
√− ln l−
√− ln r)2
(ln r−ln l)2 .
(27)
Finally the target histogramh is normalized according to thenumbern of pixels in the image:
h(x) =n hG(x)∑L−1x=0 hG(x)
∀ x ∈ {0, · · · , L− 1}.
Whenever (27) is used, we shall writehG for h.
Remark 7. If the input RGB imagew has no pixel valueson an interval[0, L0] for someL0 ≥ 1 one has to performthe hue-preserving stretching in(5). The target histogram ischosen based on the stretched histogram and the enhancedimage is computed from the stretched image; see Fig. 16.
With this cautionary remark, we can explain how to choosegood target histograms using (27). The input intensity his-togram, after stretching if necessary, is denoted byhf .
9
Remark 8. The choice of the parameters(l, r) to build hG in(27) depends on the input intensity histogramhf and on theenhancement task. E.g.,(l, r) = (1, 0.99) leads to HE.
(i) Function hG can be easily adapted to all images whosehistogram hf is roughly unimodal – see the originalimages in Figs. 6, 7, 9, 10, 11, 13, 14 and 16.If the pixel values are mainly in the middle of the interval[0, L− 1] and decay at the ends (see Figs. 7, 9 and 16),a good enhancement can be done withl ' r ∈ [0.1, 0.2].Whenhf rapidly decays towardsL−1, one should chooser ∈ (0, 0.1] (see Figs. 6, 10, 11, 13 and 16). The strongerthis decay, the smaller the value ofr should be selected(e.g. in Fig. 11,r = 10−4.)A too larger should entail artifacts typical for HE.If most of the pixel have small values (underexposedimages, see Figs. 6, 10, 11 and 13), it is reasonable toselectl ∈ [0.8, 1]. The higher the concentration near0,the larger the value ofl / 1 should be taken.
(ii) For images with important very dark and very brightareas, function(27) should not work well. Then a goodoption is to take a mixed target histogram
hmix :=1
2(hf + hG) (28)
where the parameters(l, r) for hG are selected followingthe rules in (i). For example, see Figs. 8, 12 and 15.
This remark is illustrated in Fig. 8.
B. Enhancement Tests
We present some results from a large series of test imageswith the goal to improve the visual quality. The enhancedimage should seemnatural and an observer should not suspectthat it is a post-processing result.
We compare our Multiplicative algorithm 4 (×) with theNaik-Murthey algorithm 2 (NM) and the Additive algorithm5 (+). The HS in these algorithms is done by Algorithm 1.The histogram of the original intensity image and the targethistograms are depicted beneath the images. The percentageof pixels having a gamut problem in Alg. 4 (×), Alg. 5 (+)and Alg. 2 (NM) is contained in Table I. Further we providecomparison results with
• the fast implementation of ACE by [14] available onlineat http://demo.ipol.im/demo/g_ace/, and
• the perceptual color enhancement through variationalmethods in [7] and [8].
ACE has one main parameter, the enhancement strengthαwhose default valueα = 0.5 is often a good choice. For thetwo perceptual enhancement methods [7] and [8], the authorsgave us their codes and helped us to tune the parameters.
We present the results for images with different defects. Theparameter values for all methods are given in the captions, aswell as the image credits. The original images in Figs. 6, 7,8, 10, 11, 13, 12 and 16 are photos taken by the authors whowanted to improve them. For all these images, we do not have“ground truth”. For Fig. 17 we shot an underexposed and abetter “example” image which enabled us to compare with theperceptual histogram-based method in [10].
(a) Original image (b) Alg. 4 (×) hG
0 255 0 255
(c) Alg. 4 (×) hmix (d) Perceptual [8]
0 255
Fig. 8. (a) Imageclub (1800×3200). Enhancement using our Multiplicativealgorithm with: (b) hG for (l, r) = (1, 0.2); (c) hmix by (28) for hG as in(b). (d) Enhancement by the variational method in [8].
(a) Original image (b) Alg. 5 (+) (c) Perceptual [8]
12 247 0 255
Fig. 9. (a) Imageislanda (courtesy of P. Greenspun) of size294×293, andplot of hf . Enhancement results: (b) Additive algorithm withhG for (l, r) =(0.1, 0.1), see (27); (c) Perceptual variational method [8] with Michelson’scontrast function and default parameters (courtesy of the authors of [8]).
Fig. 8 illustrates Remark 8. The imageclub in (a) is under-exposed and its histogramhf does not obey Remark 8(i). Theresult in (b) is obtained with a Gaussian target histogramhG
for parameters following (i). In (c) we use a mixed targethistogramhmix as proposed in (28). This image better showsthe ambience of the club. The perceptual method [8] in (d)gives a colder color palette.
Two tests with the imageislandaare shown in Fig. 9. Theoriginal in (a) is a rather light image. The Multiplicative,the Additive and the NM algorithms with Gaussian targethistograms produce quite similar results, which confirms ourdiscussion in Subsection IV-B. Only the issue of our Additivealgorithm is shown in (b). The perceptual method [8] and theACE perform similarly and give a nice, different color contentof the image. We depict the result by [8] in (c).
The photoboy-on-stonesin Fig. 10(a) was taken in a verysunny day. Due to camera corrections, the picture appearsunderexposed. Our Multiplicative algorithm gives a realisticresult shown in (b). In particular, observe the reflectance ofthe sunlight on the stones. The NM algorithm yields a grayishresult (c). The Additive algorithm (not shown) gives a slightlybetter enhancement than the NM algorithm which confirms
10
(a) Original image (b) Alg.4 (×)
0 239 0 255
(c) Alg.2 (NM) (d) Perceptual [7]
(e) Perceptual [8] (f) ACE [14]
Fig. 10. (a) Imageboy-on-stones(800× 800) and plot ofhf . Enhancement
results: (b) Multiplicative algorithm withhG for (l, r) = (0.8, 0.2); (c) Naik-Murthy algorithm with the samehG; (d) Perceptual variational method [7]with data-fitting parameterγ = 0.2; (e) Perceptual variational method [8]with a symmetric contrast function and slope parameter10; (f) ACE, α = 5.
our findings in Subsection IV-B. The perceptional variationalalgorithms [7] in (d) and [8] in (e), as well as the ACE in (f),shift the colors towards blue; observe the stones.
The cathedralphoto in Fig. 11(a) is much too dark. Theresult of our Multiplicative algorithm in (b) is quite colorful.The same algorithm for the mixed target histogram (28) givesin (c) a darker and still colorful image. The results of [7] in(d)and [8] in (c) have a darker color palette. The NM algorithmproduces a nearly gray value image shown in (f). All resultsgive a different atmosphere.
The photo taken inJericoacoara, Fig. 12(a), has very darkand also some quite clear areas; seehf in (c). By Remark 8(ii),we use a mixed target histogramhmix. The original has lotsof JPEG artifacts so we prefer our Additive algorithm (seeRemark 6). The result in (b) is convincing and the details inthe dark are clarified. For the ACE in (d) we use a smallenhancement,α = 3, in order to limit the false color shift.
The orchid image in Fig. 13(a) has a bad flashlight effect.This artifact is removed by all tested methods and the back-ground of the scene is clear. Our Multiplicative algorithm gives
(a) Original image (b) Alg.4(×), hG (c) Alg.4(×), hmix
0 255 0 255 0 255
(d) Perceptual [7] (e) Perceptual [8] (f) Alg.2 (NM)
Fig. 11. (a) Imagecathedral(768×1024) and plot ofhf . Enhancement results:
(b) Multiplicative algorithm with hG for (l, r) = (1, 10−4); (c) The samealgorithm for hmix in Remark 8(ii) withhG in (b); (d) Perceptual variationalmethod [7], default parameters; (e) Perceptual variational method [8] withMichelson’s contrast function; (f) NM algorithm withhG in (b).
(a) Original image (b) Alg. 5 (+), hmix
(c) Histograms (d) ACE [14]
0 248
0 255
hf
hmix
Fig. 12. (a) ImageJericoacoara(886 × 1181). Enhancement results: (b)Additive algorithm with mixed target histogramhmix for (l, r) = (1, 0.1);(c) Histograms of the input and the target intensities; (d) ACE for α = 3.
a realistic colorful result, see (b). As in Fig. 6, our Additivealgorithm produces a rather pale image (c) while the issue ofthe NM algorithm in (d) is too gray. The images obtained by[7] in (e) and by the ACE in (f) exhibit color shifts (see thegreen leaves on the right and the grass on the bottom left).
The frog image in Fig. 14(a) has an intensity histogrambetween (i) and (ii) in Remark 8. Indeed, both recipes gavesimilarly good results. The result withhG is shown in (b). Forthe ACE in (c) we select a smallα = 3 limit the color shift.
Theferrari image in Fig. 15(a) has very dark and very brightareas. Using Remark 8 (ii), we take a mixed target histogram.Our Multiplicative algorithm gives a realistic image shownin(b) with vivid colors that fit the typical red of the brand. Thevariational methods [7] in (c) and [8] in (d) outperform theACE (result not shown).
The imagefields in Fig. 16(a) was taken trough an aircraftporthole. It has no pixels with values in[0, 109]. By Remark
11
(a) Original image (b) Alg. 4 (×)
0 255 0 255
(c) Alg.5 (+) (d) Alg. 2 (NM)
(e) Perceptual by [7] (f) ACE [14]
Fig. 13. (a) Imageorchid (768×1024) with a bad flashlight effect. Enhance-ment results: (b) Multiplicative algorithm withhG for (l, r) = (1, 0.1); (c)Additive algorithm for the same target histogram; (d) NM algorithm [18] forthe same target histogram; (e) Perceptual variational method [7] for γ = 0.2;(f) ACE [14], default parameter.
(a) Original image (b) Alg. 4 (×) (c) ACE [14]
0 255 0 255
Fig. 14. (a) Imagefrog, 332 × 300 (credits: John D. Willson, USGSAmphibian Research and Monitoring Initiative). Enhancement results: (b)Multiplicative algorithm withhG for (l, r) = (0.4, 0.1); (c) ACE for α = 3.
3, we use in (b) the global hue-preserving stretching. Itsfurther enhancements using our Multiplicative and Additivealgorithms give visually the same results, so only the first oneis shown in (c). For the ACE in (d) we takeα = 8 in order toobtain an enhancement strength similar to (c). The JPEG blueartifacts are stronger in (d) compared to (c).
Fig. 17 (a) and (b) show an underexposed and a betterexposed “example” image of the same sceneflag. In [10] theauthors propose an algorithm for the color transfer betweenimages (usually of different scenes). Using this algorithmwetransferred the colors from the example image (b) to theunderexposed one (a). The result in (c) is close to the example
(a) Original image (b) Alg. 4 (×), hmix
0 254 0 255
(c) Perceptual [7] (d) Perceptual [8]
Fig. 15. (a) Imageferrari (courtesy of P. Greenspun) of size235×240 andhistogram of its intensity channel.. Enhancement results:(b) Multiplicativealgorithm with mixed target histogram for(l, r) = (1, 0.1); (c) Perceptualvariational method [7] with default parameters (courtesy of the authors); (d)Perceptual variational method [8] with Michelson’s contrast function anddefault parameters (courtesy of the authors).
(a) Original image (b) Stretch
110 254 9 241
(c) Alg. 4 (×) (d) ACE [14]
0 255
Fig. 16. (a) Imagefields (512 × 512) and histogram of its intensity.Enhancement results: (b) Global affine stretching (Remark 3) and intensityhistogram; (c) Multiplicative algorithm applied to (b) with Gaussian targethistogram for(l, r) = (0.1, 0.1); (d) ACE for α = 8 applied to (a).
12
image ♯U(1) ♯U(0) ♯L(0) ♯V
bungalowhG 1.90 0.94 0 98.15islandahG 0.89 0.85 0.94 2.93club hG 0.98 0.23 0.67 94.61club hmix 0.33 0.15 1.39 92.61boy-on-stones (hG) 4.76 2.40 0.54 94.15cathedral (hG) 1.04 0.00 1.38 95.71Jericoacoara (hmix) 5.98 5.71 0.40 68.26orchid (hG) 0.21 0.10 1.85 87.93frog (hG) 0.09 0.08 6.79 80.04ferrari (hmix) 4.40 4.22 0.30 60.7fields (hG) 4.61 2.99 2.06 89.16
TABLE IPercentage of pixels requiring an upper or lower gamut correction. For ourAlgorithms 4 (×) and 5 (+) the numbers♯U(1), resp., ♯U(0), ♯L(0) are
very small in all tests; this is not the case for♯V in the NM algorithm.
(a) Input (b) Example (c) Alg. [10] (d) Alg.4 (×)
0 163 0 213
zoom of (a) zoom of (b) zoom of (c) zoom of (d)
Fig. 17. (a) Input underexposed imageflag (1500× 1125) and (b) examplebetter-exposedflag together with the histograms of their intensities. Enhance-ment results: (c) the color transfer method [10] and (d) our Multiplicativealgorithm with the example target intensity histogram in (b).
image. We have applied our Multiplicative algorithm to image(a) using the intensity histogram of (b). The result, shown in(d), is less dull than the example (b). The third row in Fig. 17depicts a zoom into the area with the raven (right middle). Oneobserves that the bird is fused with part of the background in(c), whereas it is distinguishable in (b) and (d).
VI. CONCLUSIONS AND FUTURE WORK
This work provides the first comprehensive and rigorouspresentation of the wide family of histogram specificationbased affine color assignment models. We have proposed afast hue and range preserving algorithm. We analyzed theperformances of this algorithm and two of its importantinstances as well as the gamut preserving method in [18].
Many open questions have been raised that we want toanswer in our future research. Since our algorithms are fast,extensions to video should be envisaged. We are aware ofthe broad literature on color enhancement taking both globaland local neighborhood of pixels into account see, e.g., [6],[7], [8], [13], [36]. It will be interesting to incorporate such
information into our framework. Moreover, we want to takeinto account other important properties of the human visualsystem.
VII. A PPENDIX
Proof of Proposition1: A pixel i ∈ In in (15) has an uppergamut problem if
λf [i] + (1− λ)f [i]
f [i]M [i] + (1− λ)(f [i]− f [i]) > L− 1. (29)
The upper bound follows from the choice ofwc[i], c ∈{r, g, b} in (12). We focus the lower bound. First we seethat (29) impliesf [i] ≥ f [i], since in casef [i] < f [i] wewould get by replacingf [i] by f [i] in the denominator of thequotient and the second summand in (29) the contradictionM [i] > L − 1. SinceM [i]− f [i] > 0 the lower bound holdstrue if and only if
(L− 1− f [i])(wc[i]− f [i]) + f [i](M [i] − f [i]) ≥ 0,
(L− 1− f [i])wc[i]− (L− 1)f [i] + f [i]M [i] ≥ 0
which is clearly fulfilled if
f [i]M [i] − (L− 1)f [i] ≥ 0. (30)
By (29) we have
f [i]M [i] − (L− 1)f [i]
> f [i]M [i] −(λf [i] + (1− λ)f [i]
)(M [i]− f [i]) − f [i]f [i]
=(f [i]−
(λf [i] + (1− λ)f [i]
))(M [i] − f [i]).
By f [i] ≥ f [i] the first factor on the right-hand side is≥ 0the second one is> 0. Thus, (30) is satisfied. �
Proof of Proposition2: A pixel i ∈ In in (15) has a lowergamut problem if
λf [i] + (1− λ)f [i]
f [i]m[i] + (1− λ)(f [i]− f [i]) < 0. (31)
The lower bound is clear from the construction ofwc[i] in(14). We show the upper bound. Developing (14) yields
wc[i] =f [i]wc[i]− f [i] f [i] + f [i] f [i]− f [i] m[i]
f [i]−m[i]
=f [i]
f [i] −m[i](wc[i]−m[i]), c ∈ {r, g, b}. (32)
Using (31), one has
f [i]λm[i] + (1− λ)f [i]
f [i]+ (1− λ)(m[i]− f [i]) < 0
and hence
f [i] <(1− λ)f [i]
λm[i] + (1− λ)f [i](f [i]−m[i]) < f [i]−m[i].
Thus, since0 < wc[i]−m[i] < L− 1, we obtain finally
f [i]
f [i]−m[i](wc[i]−m[i]) < L− 1.
Proof of Proposition3: Let 0 ≤ λ1 < λ2 ≤ 1. In the uppergamut case we always havef [i] > f [i]. Thenλ1f [i] + (1 −λ1)f [i] < λ2f [i] + (1 − λ2)f [i], henceGλ1
M [i] ≤ Gλ2
M [i] andU(λ1) ⊆ U(λ2). Similarly, since f [i] < f [i] in the lowergamut case,λ1f [i] + (1− λ1)f [i] > λ2f [i] + (1− λ2)f [i] so
13
thatGλ1
m [i] ≥ Gλ2
M [i] and henceL(λ1) ⊇ L(λ2). �
Proof of Proposition4: Let i ∈ In \ {U(1) ∪ L(0)}. Thenw×
c [i] is given by (20)(i) and sinceU(0) ⊆ U(1), the valuew+
c [i] is given by (21)(i). Consequently,
wc[i] = λf [i]
f [i](wc[i]− f [i]) + (1− λ)(wc[i]− f [i]) + f [i] = wc[i],
where the last equality follows byU(λ) ⊆ U(1) andAlgorithm 3(i).
Let i ∈ U(0). Then w+c [i] is given by (21)(ii) and since
U(0) ⊆ U(1), w×c [i] is given by (20)(ii). We havew+
c [i] =w×
c [i] which shows byU(0) ⊆ U(λ) that wc[i] = w[i]. �
Proof of Proposition5: By (22) we haveS(w[i]) = 1−m[i]f [i] , or
equivalently,f [i]−m[i] = f [i]S(w[i]). Algorithm 3 computes
wc[i] = d[i](wc[i]− f [i]) + f [i], (33)
for
d[i] =
λf[i]f [i]
+ (1− λ) if i ∈ In \ {U(λ) ∪ L(λ)},L−1−f [i]M[i]−f [i]
if i ∈ U(λ),f [i]
f [i]S(w[i])if i ∈ L(λ).
Sinced[i] > 0 in all cases, we have
S(w[i]) = 1−1
f [i]
(d[i]
(min{wr[i]wg [i], wb[i]} − f [i]
)+ f [i]
)
= 1−f [i]
f [i]d[i]
min{wr[i], wg [i], wb[i]}
f [i]+
f [i]
f [i]d[i]− 1
= S(w[i])f [i]
f [i]d[i] . (34)
Inserting the above valuesd[i] into (34) finishes the proof.�
Proof of Proposition6: The casef [i]f [i] ≤ 1 follows just as a
special case of Proposition 5(i) forλ = 1. Let f [i] > f [i].Then case (ii) in Algorithm 2 can be rewritten as (33) for
d[i] := L−1−f[i]L−1−f [i] . Combining this with (34) we are done.�
Acknowledgement: The authors thank M. Bertalmio, E.Provenzi and N. Papadakis for providing their codes corre-sponding to [7], [8] and [10] and for the discussion. We want tothank the anonymous reviewers for their constructive remarks.
REFERENCES
[1] R. S. Berns,Billmeyer and Saltzman’s Principles of Color Technology,Willey & Sons. Roy S., Ca, 3 edition, 2000.
[2] C. Solomon and T. Breckon,Fundamentals of Digital Image Processing.A Practical Approach with Examples in Matlab, John Wiley & Sons,UK, 1st edition, 2011.
[3] T. Gevers and A. Smeulders, “Color-based object recognition”, PatternRecognition, vol. 32, 1999.
[4] A. C. Bovik, Handbook of Image and Video Processing (Communica-tions, Networking and Multimedia), Academic Press, Orlando, 2005.
[5] R. Gonzalez and R. Woods,Digital Image Processing, NJ: Pentice-Hall,3rd edition, 2007.
[6] G. Sapiro, and V. Caselles, “Histogram Modification via DifferentialEquations”,J. Differ. Equations, vol. 135, no. 2, pp. 238–268, 1997.
[7] M. Bertalmıo, V. Caselles, E. Provenzi, and A. Rizzi, “Perceptual colorcorrection through variational techniques”,IEEE Trans. Image Process.,vol. 16, no. 4, pp. 1058–1072, 2007.
[8] R. Palma-Amestoy, E. Provenzi, M. Bertalmıo and V. Caselles, “A per-ceptually inspired variational framework for color enhancement”, IEEETrans. Pattern Anal. Mach. Intell., vol. 31, no. 3, pp. 458–474, 2009.
[9] E. Provenzi,Perceptual color correction: A variational perspective, pp.109–119, Lect. Notes Comput. Sc. 5646. Springer, 2009.
[10] N. Papadakis, E. Provenzi, and V. Caselles, “A variational model forhistogram transfer of color images”,IEEE Trans. Image Process., vol.30, no. 6, pp. 1682–1695, 2011.
[11] E. H. Land, “The Retinex Theory of Color Vision”,Scientific American,vol. 237, no. 6, 1977.
[12] C. Gatta, A. Rizzi, and D. Marini, “ACE:”, inAn automatic colorequalization algorithm. Proceedings: Conference Color in GraphicsImage and Vision (CGIV), vol. 5, 2002, pp. 316–320.
[13] A. Rizzi, C. Gatta, and D. Marini, “A new algorithm for unsupervisedglobal and local color correction”,Pattern Recognit. Lett, vol. 124,2003.
[14] P. Getreuer, “Automatic color enhancement (ACE) and its fast imple-mentation”, Image Processing On Line, DOI: 10.5201/ipol.2012.g-ace,vol. 2012, 2012.
[15] M. Bertalmıo, V. Caselles, and E. Provenzi, “Issues About RetinexTheory and Contrast Enhancement”,Int. J. Comput. Vis., vol. 83, pp.101–119, 2009.
[16] P. E. Trahanias and A. Venetsanopoulos, “Color image enhancementthrough 3-D histogram equalization”, inProc. 15th IAPR Int. Conf.Pattern Recognit, 1997, vol. 1, pp. 545–548.
[17] J. H. Han, S. Yang, and B. U. Lee, “A novel 3-D color histogramequalization method with uniform 1-D gray scale histogram”, IEEETrans. Image Process., vol. 20, no. 2, pp. 506–512, Feb. 2011.
[18] S. F. Naik and C. A. Murthy, “Hue-preserving color imageenhancementwithout gamut problem”,IEEE Trans. Image Process., vol. 12, no. 12,pp. 1591–1598, 2003.
[19] N. Bassiou and C. Kotropoulos, “Color image histogram equalizationby absolute discounting back-off”,Computer Vision and Image Under-standing, vol. 107, 2007.
[20] Z. Y. Chen, B. R. Abidi, D. L. Page, and M. A. Abidi, “Gray-LevelGrouping (GLG): An automatic method for optimized image contrastenhancementpart i: The basic method”,IEEE Trans. Image Process.,vol. 15, no. 8, pp. 2290–2302, 2006.
[21] D. Menotti, L. Najman, A. d. Araujo, and J. Facon, “A fast hue-preserving histogram equalization method for color image enhancementusing a bayesian framework”, inProc. 14th Int.Workshop Syst., SignalImage Process. (IWSSIP), 2007, pp. 414–417.
[22] D. Menotti, L. Najman, J. Facon, and A. Albuquerque, “Histogramequalization methods for color image contrast enhancement”, Int. J.Comput. Sci. & Inform. Technology, vol. 4, no. 5, pp. 243–259, 2012.
[23] N. Kong and H. Ibrahim, “Color image enhancement using brightnesspreserving dynamic histogram equalization”,IEEE Trans. Consum.Electron., vol. 54, no. 4, pp. 1962–1967, 2008.
[24] M. Nikolova, “A fast algorithm for exact histogram specification. Simpleextension to colour images”, Lect. Notes Comput. Sc. 7893, Springer,2013, pp. 174–185.
[25] T. Arici, S. Dikbas, and Y. Altunbasak, “A histogram modificationframework and its application for image contrast enhancement”, IEEETrans. Image Process., vol. 18, no. 9, pp. 1921–1935, 2009.
[26] Q. Wang and R. K. Ward, “Fast image/video contrast enhancementbased on weighted thresholded histogram equalization”,IEEE Trans.Consum. Electron., vol. 53, no. 2, pp. 757–764, 2007.
[27] D. Sen and P. Sankar, “Automatic exact histogram specification forcontrast enhancement and visual system based quantitativeevaluation”,IEEE Trans. Image Process., vol. 20, no. 5, pp. 1211–1220, 2011.
[28] G. Thomas, “A modified version of van-cittert’s iterative deconvolutionprocedure”,IEEE Trans. Acoust. Speech, Signal Process., vol. ASSP-29,pp. 938–939, 1981.
[29] C.-L. Chien and D.-C. Tseng, “Color image enhancement with exactHSI color model”, Int. J. of Innovative Computing, Information andControl, vol. 7, no. 12, pp. 6691–6710, 2011.
[30] D. Coltuc, P. Bolon, and J.-M. Chassery, “Exact histogram specifica-tion”, IEEE Trans. Image Process., vol. 15, no. 6, pp. 1143–1152, 2006.
[31] Y. Wan and D. Shi, “Joint exact histogram specification and imageenhancement through the wavelet transform”,IEEE Trans. ImageProcess., vol. 16, no. 9, pp. 2245–2250, 2007.
[32] M. Nikolova and G. Steidl, “Fast sorting algorithm for exact histogramspecification”, Preprint hal-00870501, 2013.
[33] M. Nikolova, Y.-W. Wen, and R. Chan, “Exact histogram specificationfor digital images using a variational approach”,J. Math. Imaging andVision, vol. 46, no. 3, pp. 309–325, 2013.
[34] C. C. Yang and J. J. Rodriguez, “Efficient luminance and saturationprocessing techniques for bypassing color coordinate transformations”,in Proc. IEEE Int. Conf. on Systems, Man, and Cybernetics, 1995, vol. 1,pp. 667–672.
14
[35] C. C. Yang and J. J. Rodriguez, “Saturation clipping in the lhs andyiq color spaces”, inProc. IS&T/SPIE Int. Symp. Electronic Imaging:Science & Technology Color Imaging: Device-Independent Color, ColorHard Copy, and Graphic Arts, 1996, vol. 1.
[36] J.-Y. Kim and L.S. Kim, “An advanced contrast enhancement usingpartially overlapped sub-block histogram equalization”,IEEE Trans.Circ. Sys. Video, vol. 11, no. 4, pp. 475–484, 2001.
Gabriele Steidl received her PhD and Habilitation in Math-ematics from the University of Rostock, Germany, in 1988 and1991, respectively. From 1993 to 1996, she held a position asassistant professor of Applied Mathematics at the Universityof Darmstadt, Germany. From 1996 to 2010, she was fullprofessor at the Department of Mathematics and ComputerScience, University of Mannheim, Germany. Since 2011, she isProfessor at the Department of Mathematics at the Universityof Kaiserslautern, Germany. She was a visiting researcher atthe University of Debrecen (Hungary), the University of Zrich(Switzerland), the Banach Center Warsaw (Poland), the ENSCachan and the University Paris Est (France). Her researchinterests include applied and computational harmonic analysisand convex analysis with applications in image processing.
Mila Nikolova received the Ph.D. degree from the Univer-site de Paris-Sud, France, in 1995. She got a Habilitation to di-rect research in 2006. Currently, she is Research Director withthe National Center for Scientific Research (CNRS), France.She performs herduty as a full-time researcher at CMLA(Centre de Mathematiques et de Leurs Applications), De-partment of Mathematics, ENS Cachan, France. Her researchinterests are in mathematical Image and signal reconstruction,Inverse problems, Regularization and variational methodsandthe properties of their solutions, Scientific computing.