1

Jayanta MukhopadhyayDepartment of Computer Science & Engineering

Indian Institute of Technology, Kharagpur, 721302, India

jay@cse.iitkgp.ernet.in

Sanjit K. MitraMing Hsieh Dept. of Electrical Engineering

University of Southern California

Los Angeles, CA 90089, USA

skmitra@usc.edu

COLOR ENHANCEMENT IN THE COMPRESSED DOMAIN

2

Image Enhancement: An Example

(a) Original (b) Enhanced

3

Enhancement Factors

• Dynamic Range of Intensity Values. Limited number of bit-planes.

• Varying illumination over the image space.

• Brightness

• Contrast

• Color

4

Enhancement Techniques

• Spatial Domain Techniques.

Work with the pixels values. Histogram Stretching, Pixel Mapping, Image Sharpening

• Compressed Domain Techniques. Works with DCT coefficients

5

Let x (m,n), 0 m,n N-1 be a 2-D image of size NxN. Its N-point DCT C(l,k), 0 l,k N-1 is defined as:

DCT: Definition

1 1

0 0

2 (2 1) (2 1)( , ) ( ) ( ) ( , ) cos cos

2 2

N N

m n

n l n kC l k l k x m n

N N N

otherwise 1

0for 2

1)(

pp

6

Motivations

Computation with reduced storage. Avoid overhead of entropy decoding

and encoding. Exploit spectral factorization for

improving the quality of result and speed of computation.

7

Existing Approaches

• Alpha Rooting Aghaglzadeh and Ersoy (1992), Opt.Engg• Multi Contrast Enhancement Tang, Peli and Acton (2003), IEEE SPL

A spectral band is defined with Yijs such that i+j=n; 0<n<15

Contrast measure (Hn)= Cumulative Energy Sum till nth band in original image ----------------------------------------------------------------------- Cumulative Energy Sum till nth band in enhanced image Each coefficient in a band is scaled by : Yij = λ Hn. Yij, , i+j=n

8

Existing Approaches

• Multi-Contrast Enhancement with Dynamic Range Compression (S. Lee (2007), IEEE CSVT)

Modification of DC coefficients and AC coefficients (following similar strategy of multi-contrast enhancement).

Normalized DC coefficients (x) are modified as follows:

9

Proposed Approach

• Adjust background illumination. Use DC coefficients of the Y component.

• Preserve Local Contrast. Scale AC coefficients of the Y component appropriately.

• Preserve Colors. Preserve Color Vectors in the DCT domain. DCT coefficients of Cb and Cr components.

10

Let μ and σ denote the mean and standard deviation of

an image. Contrast ζ of an image is defined here as: .

Contrast : Definition

Weber Law:

where is the difference in luminance between a stimulus and its surround, and L is the luminance of the surround

L

L

L

11

Theorem on Contrast Preservation in the DCT Domain

Let d be the scale factor for the DC coefficient and a a be the scale factor for the AC coefficients of a DCT block Y. The processed DCT block Ye is given by:

( , ), 0( , ),( , ) { d

a

Y i j i je Y i j otherwiseY i j

The contrast of the processed image then becomes a / d times of the contrast of the original image.

In this algorithm d = a = for preservation of the contrast.

12

Preservation of Colours in the DCT Domain

Let U and V be the DCT coefficients of the Cb and Cr components, respectively. If the luminance component Y of an image is uniformly scaled by a factor , the colors of the processed image with Ye , Ue and Ve are preserved by the following operations:

( , )( ( 128)) 128, 0

( , ),( , ) {U i j

N i jN

e U i j otherwiseU i j

( , )( ( 128)) 128, 0

( , ),( , ) {V i j

N i jN

e V i j otherwiseV i j

13

Enhancement by Scaling Coefficients

• Find the scale factor by mapping the DC coefficient with a monotonically increasing function.

• Apply scaling to all other coefficients in all the

components.

• For blocks having greater details, apply block

decomposition and re-composition strategy.

14

Mapping functions for adjusting the local background illumination

( (0,0))

(0,0)

f Y

Y

(TW)

Mitra and Yu , CVGIP’87

(DRC)

Lee, CSVT’07

(SF)

De, TENCON’89

15

Monotonic Mapping Functions

16

Scaling only DC coefficients

17

Scaling both DC and AC coefficients

18

Preservation of Contrast and Color

original

19

Block Decompos.

Smaller DCT blocks

8x8 block

Apply CES on smaller

blocks

Block Composition

Enhanced Block

Enhancement of Blocks with more details

20

Removal of Blocking Artifacts

original

21

Some Results

original AR MCE

MCEDRC TW-CES-BLK MSR

22

Enhancement near Edges

AR MCE MCEDRC

TW-CES-BLK DRC-CES-BLK SF-CES-BLK

23

Some Results

original AR MCE

MCEDRC TW-CES-BLK MSR

24

Enhancement near edges

AR MCE MCEDRC

TW-CES-BLK DRC-CES-BLK SF-CES-BLK

25

Some Results

original AR MCE

MCEDRC TW-CES-BLK MSR

26

Enhancement near edges

AR MCE MCEDRC

TW-CES-BLK DRC-CES-BLK SF-CES-BLK

27

Metrics for Comparison

2

2 2

4 xy

x y

x yQM

x y

Wang and Bovic (SPL, 2002)

JPEG Quality Metric (JPQM) Wang and Bovic (ICIP,2002)

2 2 2 20.3CM Susstrunk and Winkler (SPIE, 2004)

R G

2

R GB

28

Other Approaches

• Alpha Rooting (AR) : Aghaglzadeh and Ersoy (1992), Opt.Engg.

• Multi-Contrast Enhancement (MCE): Tang, Peli and Acton (2003), IEEE SPL

• Multi-Contrast Enhancement with Dynamic Range Compression (MCEDRC):

S. Lee (2007), IEEE CSVT

• Contrast Enhancement by Scaling (CES): Proposed work

• Multi-Scale Retinex (MSR) (a reference spatial domain technique):

Jobson, Rahman and Woodell (1997), IEEE IP

29

Average Performance Measures

Techniques JPQM CEF Y-QM

Cb-QM

Cr-QM

AR 8.58 0.97 0.80 0.67 0.67

MCE 7.00 0.94 0.76 0.67 0.67

MCEDRC 7.92 0.97 0.86 0.67 0.67

TW-CES-BLK

7.79 1.50 0.90 0.82 0.81

DRC-CES-BLK

8.16 1.18 0.86 0.76 0.76

SF-CES-BLK

8.13 1.25 0.89 0.78 0.77

30

Computational Complexities

Techniques

Per Pixel Operations

AR 1E + 1M

MCE 2.19M+1.97A

MCEDRC 0.03E+3.97M+2A

TW-CES 0.02E+4.02M+1.05A

DRC-CES 0.05E+4M+1.08A

SF-CES 0.03E+4.02M+1.06A

MSR 18E+1866378M+8156703A

aE+bM+cA implies a Exponentiation, b Multiplication and c Addition operations.

31

Iterative Enhancement

original

Iteration no.=1

Iteration no.=2

Iteration no.=3

Iteration no.=4

32

Conclusion

• A novel algorithm for enhancement of color images in the block DCT domain.

• It takes care of adjustment of back-ground illumination, local contrast, as well as preservation of colours.

• Comparative study shows the superiority of the proposed scheme compared to some of the existing schemes.

33

References

• [1] J. Mukherjee and S.K.Mitra, Enhancement of Color Images by

Scaling the DCT coefficients, IEEE trans. on Image Processing, vol. 17, no. 10, Oct., pp. 1783-1794, 2008. • [2] J. Mukherjee and S.K. Mitra,     Color Enhancement in the Compressed Domain, Int. Conf. on Image Proc. (ICIP-2008), San Diego, USA, Oct. 12 -15, pp. 3144-3147, 2008 .  

34

Thanks!