a histogram modification framework and its application for ......– comparison of weighted...

School of Electrical Engineering and Computer Science Kyungpook National Univ.

A Histogram Modification Framework and Its Application for Image Contrast Enhancement

IEEE Transactions on Image Processing,Vol. 18, No. 9, 2009

Tarik Arici, Salih Dikbas, and Yucel Altunbasak

Presented by Jung Yul Choi

Abstract

General framework based on histogram equalization for image contrast enhancement– Optimization problem that minimizes a cost function

Conventional histogram equalization– Effective technique for contrast enhancement– Resulte of excessive contrast enhancement

Adjusting level of contrast enhancement– Introducing specifically designed penalty terms

2 / 37

Introduction

Contrast enhancement– Occurring poor contrast of image and video

• No reveal all the details in captured scene• Washed-out and unnatural look

– Target of contrast enhancement• Eliminating these problems

– Obtaining more visually-pleasing or informative image or both

Broadly categorizing contrast enhancement techniques– Direct methods and indirect methods

3 / 37

– Direct methods• Defining contrast measure and trying to improve it

– Indirect methods• Improving the contrast through exploiting the under-

utilized regions of dynamic range • Most methods in the literature• Dividing into several subgroups

1. Decomposing image into high and low frequency signals2. Histogram modification techniques3. Transform-based techniques

4 / 37

– Global contrast enhancement (GCE)• Use of single mapping derived from the image• Impossible enhancing local contrast• One of most popular GCE techniques

– Histogram equalization» Effective technique to transform narrow histogram

by spreading gray-level clusters in histogram» Excessively enhanced output image for some

applications

– Local contrast enhancement (LCE)• Use of neighborhood of each pixel to obtain local

mapping function• More computationally complex than GCE

5 / 37

Histogram modification techniques– Obtaining through modification on HE

• Bi-histogram equalization– Reducing mean brightness change

• Dualistic sub-image histogram equalization (DSIHE)– Using median intensity instead of mean intensity

• Becoming problem when histogram has spikes

– One method to deal with histogram spikes• Histogram low-pass filtering and modifying cumulation

function of histogram– Still sensitive to problem created by histogram spikes– LCE methods

6 / 37

– Recent method proposed by Wang and Ward (2007)

• Modifying image histogram by weighting and thresholding– Before histogram equalization

– Gray-level grouping (GLG)• Grouping histogram bins and redistributing groups

iteratively– Robust to histogram spikes– Mainly designed for still images

7 / 37

Aforementioned techniques– Performing well on some images– Creating problems

• When sequence of images is enhanced• When the histogram has spikes• When natural looking enhanced image is strictly required

Goal in this paper– Obtaining visually pleasing enhancement method

• Low computational complexity• Easily implementing on FPGAs or ASICs• Working well with both video and still images

8 / 37

Contrast Enhancement

Enhancement mapping function– Histogram-based methods

• Obtaining image with histogram of uniform distribution

– Mapping function in discrete form

where B is number of bits used to represent the pixel values,p[j] is normalized histogram, and n∈[0, 2B-1]

• As uniform as possible, no exactly uniform– Because of the discrete nature of the pixel intensities

0[ ] (2 1) [ ] 0.5

nB

jT n p j

=

= − +

∑ (1)

9 / 37

– Black stretching and white stretching• Making dark pixels darker, while bright pixels brighter• Linear black and white stretching

where b is maximum gray-level to be stretched to black,w is minimum gray-level to be stretched to white,g[n] is any function mapping intensities in between, andsb, sw are black and white stretching factors both of which are less than one

,[ ] [ ],

( ) ,

b

w

n s n bT n n g n b n w

w n w s w n

× ≤= × < < + − × ≤

(2)

10 / 37

Histogram Modification

Fully exploit available dynamic range on HE– Creating uniformly distributed output histogram

• Using cumulated histogram as its mapping function

– One problem with HE• Large backward-difference values of T[n]

– Unusually large T[n]- T[n-1]

• Modifying input histogram– Then accumulating histogram

• Already uniform input distribution– Mapping function is T[n]=n

» Identically mapping input to output

11 / 37

– Lessening level of enhancement• Altering input histogram

– Modified histogram » Closer to uniformly distributed histogram u

• Solution of bi-criteria optimization problem– Fining modified histogram – Making residual

where and , andλ varies over [0, ∞).

h

h

− ih h

ih

( )minλ − + −ih h h u

256 1, , , R ×∈ih h h u

(3)

12 / 37

Adjustable histogram equalization– Obtaining analytical solution to (3)

• Using squared sum of the Euclidean norm

• Quadratic optimization problem

• Solution of (5)

( )2 2

2 2arg minλ= − + −i

hh h h h u

arg min ( ) ( )λ( ) ( )T T = − − + − − i ih

h h h h h h u h u

λ 1λ1λ1λ1λ+ = = + + + +

ii

h uh h u

(4)

(5)

(6)

13 / 37

– Example image and enhanced images• Using modified histogram equalization

Fig. 1. Modified histogram equalization results using (6) for image Door. (a) Original image, (b) enhanced image using (6) with λ = 0, (c) enhanced image using (6) with λ = 1, (d) enhanced image using (6) with λ = 2.

Fig. 2. The mappings and histograms for Fig. 1.

14 / 37

– Problem of having large slope• Arising from spikes in the input histogram• Observing sensitivity to spikes

– Because l2 norm heavily penalizes large

• Using l1 norm instead of l2 norm in (4)

• Transforming into quadratic programming problem

where

• Another way to deal with the histogram spikes– Using one more penalty term to measure the smoothness

of

[ *]T n

( )2

1 2arg minλ= − + −i

hh h h h u

arg minλ( ) ( )T T = + − − h

h t 1 h u h u

( )− ≤ − ≤it h h t256 1, ,R ×∈t 1

h

(7)

15 / 37

Histogram smoothing– Measuring histogram smoothness

• Backward-difference of histogram, h[i]-h[i-1]

1 1 0 0 0 00 1 1 0 0 0

0 0 0 1 1 00 0 0 0 1 1

− − = − −

D

(255×256)

( )2 2 2

2 2 2minλγ − + − +ih h h u Dh (8)

16 / 37

• Solution of (8)

• Low-pass filtering operation on averaged histogram

• Existence of computational complex

( ) 11 (1λ)γ T −− = + +S I D D

2γ(1λ) 2γ0 02γ4γ(1λ) 2γ00 2γ4γ(1λ) 2γ

+ + − − + + − = − + + −

S

( ) 1(1λ)γ(λ) T −

= + + +ih I D D h u (9)

(10)

17 / 37

• Performance of histogram smoothing

Fig. 3. Histogram smoothing results using (9) for image Palermo. (a) Original image,(b) enhanced image using (9) with γ=0 and λ=1, (c) enhanced image using (9) with γ=0 and λ=3, (d) enhanced image using (9) with γ=1000 and λ=1.

Fig. 4. The mappings for the enhanced images given in Fig. 3.

18 / 37

Weighted histogram approximation– Large number of pixels of exactly same gray-level

• Due to large smooth areas in the image• Average local variance of all the pixels

– Using to weight approximation error, h - hi

where W∈R256X256 is diagonal error weight matrix.

– Solution of (11)

» Computationally simpler than (9)

( )min ( ) ( )λ( ) ( )T T− − + − −i ih h W h h h u h u

1(λ) (λ) −= + +ih W I Wh u

(11)

(12)

19 / 37

– Comparison of weighted histogram approximation and histogram smoothing

Fig. 5. Comparison results of histogram smoothing and weighted histogram approximation for image Palermo. (a) Histogram smoothing using(9) with γ=1000 and λ=1, (b) weighted approximation using (12) with λ=1000.

Fig. 6. Mappings for the enhanced images given in Fig. 5.

20 / 37

Black and White (B&W) stretching– Decreasing histogram bin length

• [0, b] and [w, 255]

– Incorporating B&W stretching into histogram modification

• Adding additional penalty term to (5)

where IB is diagonal matrix.

• Solution to minimization problem

( )min ( ) ( )λ( ) ( )T T T Bα− − + − − +i ih h h h h u h u h I h

( ) 1(1λ) (λ) Bα

−= + + +ih I I h u

( , ) 1 for {[0, ] [ , 255]}B i i i b w= ∈ ∪I

(13)

(14)

21 / 37

– Comparison of histogram smoothing with and without B&W

Fig. 7. (a) Original image, (b) enhanced image using (9) with γ=1000 and λ=1, (c) enhanced image using (14) with γ=1000, λ=1, and α=5, (d) mappings for the two enhanced images in (b) and (c).

22 / 37

Low complexity histogram modification algorithm

Histogram computation– High complex computation

• Because of histogram spike problem

– Simple way instead of complex computation• Using conditional probability of pixel

where denotes probability of pixel having gray-level igiven event C

– Robust noise

• Obtaining by counting only those pixels

[ ] [ | ]h i p i= C

[ | ]p i C

[ | ]p i C

23 / 37

Adjusting the level of enhancement– GCE histogram modification algorithm

User controlled parameter

1λ1λ1λ = + + +

ih h u

Measuring input contrast[ ] [ | ]h i p i= C

24 / 37

11λ+Limitation of very low slope

B&W stretching

Modification of histogram

25 / 37

Results and discussion

Assessment of image enhancement– Hard task

• Absence of any accepted objective criterion

– Proposed method of quantitative measures• Absolute mean brightness error (AMBE)

– Absolute difference between input and output mean

• Discrete entropy (H)– Measuring content of image

• Measure of enhancement (EME)– Dividing image into blocks– Finding measure Min and Max intensity values in each block,

and averaging them

26 / 37

Subjective assessment– Gray-scale images

Fig. 8. Results for image Beach. (a) Original image, (b) enhanced image obtained using HE, (c) enhanced image obtained using WTHE, (d) enhanced image obtained using the proposed algorithm.

Fig. 12. (a) Solid line indicates the HE mapping, red dashed line indicates the WTHE mapping, blue dash-dotted line indicates the proposed method, and the dottedline indicates the no change mapping.

27 / 37


Fig. 12. (b) Solid line indicates the HE mapping, red dashed line indicates the WTHE mapping, blue dash-dotted line indicates the proposed method, and the dottedline indicates the no change mapping.

28 / 37


Fig. 12. (c) Solid line indicates the HE mapping, red dashed line indicates the WTHE mapping, blue dash-dotted line indicates the proposed method, and the dottedline indicates the no change mapping.

29 / 37

– Color images


Fig. 12. (b) Solid line indicates the HE mapping, red dashed line indicates the WTHE mapping, blue dash-dotted line indicates the proposed method, and the dottedline indicates the no change mapping.

30 / 37

Fig. 13. Results for image Hats. (a) Original image, (b) Enhanced image obtained using HE, (c) Enhanced image obtained using WTHE, (d) Enhanced image obtained using the proposed algorithm.

31 / 37

Fig. 14. Results for image Window. (a) Original image, (b) Enhanced image obtained using HE, (c) Enhanced image obtained using WTHE, (d) Enhanced image obtained using the proposed algorithm.

32 / 37

Fig. 14. Results for image Island. (a) Original image, (b) Enhanced image obtained using HE, (c) Enhanced image obtained using WTHE, (d) Enhanced image obtained using the proposed algorithm.

33 / 37

Fig. 14. Results for image Face. (a) Original image, (b) Enhanced image obtained using HE, (c) Enhanced image obtained using WTHE, (d) Enhanced image obtained using the proposed algorithm.

34 / 37

Objective assessment

Table. 1. Quantitative measurement results. AMBE denotes the absolute mean brightness error, H denotes the discrete entropy, and EME denotes the measure of enhancement

35 / 37

Complexity comparison– Analyzing time complexities of HE, WTHE, and

proposed algorithm for M x N image

Algorithm Histogram computation

Mapping function

Obtaining result image Total

HE O(MN) O(2B) O(MN) O(2MN + 2B)

WTHE O(MN) O(2B) + O(2B) O(MN) O(2MN + 2B+1)

Proposed algorithm O(MN) O(2B) + O(2B) O(MN) O(2MN + 2B+1)

36 / 37

Conclusion

Low-complexity algorithm– Suitable for video display applications

Improving contrast of image and video– No introducing visual artifacts – No decreasing visual quality of image

No introducing flickering for video applications

37 / 37

Norm– Assigning length or size of vectors in vector space– Euclidean norm

– Taxicab norm or Manhattan norm

– p-norm or lp norm

– l2 norm

2 2 21 2

Tnx x x= + + + =x x x

11

n

ii

x=

=∑x

1

1

ppn

ipi

x=

= ∑x

1 12 22 2

21 1

n nT

i ii i

x x= =

= = = ∑ ∑x x x

Absolute Mean Brightness Error (AMBE)

– Gray level of input image

– Computing statistical expectation

where Xm is mean brightness of input image.

where XG = ( X0 + XL-1 ) / 2

1 0 0 1( ) 1/ ( )L Lp x X X for X x X− −= − ≤ ≤

( ) ( )AMBE E E= −X Y

{ }12

( ) ( | ) ( ) ( | ) ( )( | ) ( | )

m m m m

m m

E E X Pr X E X Pr XE X E X

= ≤ ≤ + > >

= ≤ + >

Y Y X X Y X XY X Y X

0

1

( | ) ( ) / 2( | ) ( ) / 2

m m

m m L

E X X XE X X X −

≤ = +> = +

Y XY X

( ) ( ) / 2m GE X X= +Y

39 / 37

Measure of enhancement (EME)Let image x(m, n) be split into k1k2 blocks wk, l(i, j) of sizes l1xl2, and let α, β, and γ are fixed enhancement parameters.

where and respectively are the minimum and maximum of image x(m, n) inside block wk,l, after processing block by Φ transform based enhancement algorithm.Function is sign function.

1 2

2 1

1 2

, , , ,{ }

max; ,, , , ,

1 11 2 min; ,

maxχ( ( ))

( )1( ) 20log( )

k k

wk kk l

k k wl k k l

EME EME

IEME

k k I

α β λ

α β λ

Φ∈ Φ

= =

= Φ

ΦΦ =

Φ∑∑

max; , ( )wk lI Φ min; , ( )w

k lI Φ

χ

40 / 37

41 / 37

a histogram modification framework and its application for ......– comparison of weighted...

Documents