Algorithms for Multidimensional Color TransformDesign

Xuan Mo

iPAL Group Meeting

September 10, 2010(Monga et al., ICASSP 2010)

Outline

Background of Color Transform

Uniform lattices

Non-uniform lattices

Challenges

Sort-Select-Damp

Efficient High Quality Color Transformation

Joint optimization

Future work

09/10/2010 iPAL Group Meeting 2

Background of Color TransformThe goal of color transformation: keep color properties duringtransformation

Figure: Color changed between different devices

For example

Images in device-dependent color space =⇒Device-independent space.

09/10/2010 iPAL Group Meeting 3

Look Up Table

Look-up table is a kind of sparse mapping to implementmultidimensional transform.

In many cases, transformations are complex nonlinear functions,making impractical to process large images in real time.

Precompute the transform for all inputs and store correspondingoutputs in look up table(LUT)

Full-Resolution LUT

Perfectly accurate transformationBut, impractical because constraints of storageCMYK −→ CIELAB need 16GB of storage

Sparse LUTs are employed, multidimensional interpolation for inputis used.

09/10/2010 iPAL Group Meeting 4

Uniform lattice structure

Figure: A uniform lattice in 3 dimensions

uniform lattice

Easy to select nodes

But, big interpolation error when the LUT sparse

09/10/2010 iPAL Group Meeting 5

Non-uniform lattice structure

Figure: uniform and non-uniform lattice1

Non-uniform lattice

Node selection is a problem

will reduce the error

1Monga et al., ICASSP 2010

09/10/2010 iPAL Group Meeting 6

Challenges

Challenging

Computational and storage cost

Minimize the transformation error

Trade-off between cost and accuracy

Two main research topics

Optimizing node location

Optimizing node value

09/10/2010 iPAL Group Meeting 7

Node location

Suggesting more sample points in high curvature area(Amidrod et al., 2002)

Efficient interpolation models to determine output value(Bala et al., 2003)

Sort-Select-Damp strategy for LUT lattice design(Monga et al., 2008)

An efficient high quality color transformation(Tastl et al., 2009)

09/10/2010 iPAL Group Meeting 8

Node value

Less investigation on this part

Optimize the output value at node location so as to minimize erroris also important

Iterative technique for refining color correction lookuptables(Balasubramanian., 1995)

Combine these two problems

Joint Optimization for node locations and node values(Monga et al., 2010)

09/10/2010 iPAL Group Meeting 9

Sort-Select-Damp2

Based on non-uniformed node locations

Truly optimal node selection is a NP problem, computationalinfeasible

Pursue sub-optimize problem and practical algorithm

Significance function qualifies the importance of input variable.

Select node lie at maxima of significance function namely minimumthe cost measure

2Monga et al., CIC 2008

09/10/2010 iPAL Group Meeting 10

Problem Formulation

Cost measure:

E(SN ) =

∫x∈S

p(x)||g(x)− g(x, SN )||2d(x)

Finding global optimal a Np-class problem, So seek local minima.

Make a intelligent choice of SN so as to minimize the E(SN )

09/10/2010 iPAL Group Meeting 11

Preprocessing

Define a significance function that bring

The factors of input image distributionFunction curvature

S(x) = αp(x) + (1− α)‖det(Hessian(g(x)))‖,∀α ∈ [0, 1]

p(x) show the distribution of input

Hessian(g(x)) as a measure of function curvature

α trades off these two factors.

09/10/2010 iPAL Group Meeting 12

Sort-Select-Damp procedure

1 Sort the nodes in decreasing order of significance.

2 Select the node with maximum significance.

3 Damp significance function to ensure no subsequent node lie tooclose.

Considering the trade-off between significance function andpositioning of the nodes, the Dampening function is:

d(x,y) = (1− e−β||x−y||2), β > 0

09/10/2010 iPAL Group Meeting 13

Dampening function

09/10/2010 iPAL Group Meeting 14

Sort-Select-Damp Procedure

Figure: Node selection procedure3

3Monga et al., CIC 2008

09/10/2010 iPAL Group Meeting 15

Post-processing

Avoid node close to each other in the projections onto the individualLUT axes

Enforcing priority-based minimum node-spacing

Node addition via median-cut placement

09/10/2010 iPAL Group Meeting 16

SSD Results

Uniform lattice V.S. this method:

J = 20logEuniformEadaptive

LUT Lattice Design Gain in dB over uniform node spacingProbability Density Based (α = 1) 18 dB

Curvature Based (α = 0) 10.3 dBBased on joint significance function (α = 0.3) 21 dB

Table: Gain in dB over uniform node placement

Error of this method is one-tenth of uniform node-spacing

09/10/2010 iPAL Group Meeting 17

Efficient High Quality Color Transformation4

In 1D case:

Error =

∫ x2

x1

|fs(x)− fsub(x)|dx

In 2D

Error =

∫ y2

y1

∫ x2

x1

|fs(x, y)− fsub(x, y)|dxdy

In 3D

Error =

∫ z2

z1

∫ y2

y1

∫ x2

x1

|fs(x, y, z)− fsub(x, y, z)|dxdydz

4Tastl et al., CIC 2009

09/10/2010 iPAL Group Meeting 18

Approximation error

09/10/2010 iPAL Group Meeting 19

Joint Optimization

Present a principled algorithm approach to combine the merits ofnode locations and node values

Two significant contributions:

1 Formulate a cost function that exhibits separable convexity in itsarguments which enables an efficient alternating convex optimizationalgorithm

2 Split into a primary and an auxiliary optimization, which greatlyimproves the quality of solution than individually optimizing just thenode locations or output values.

09/10/2010 iPAL Group Meeting 20

Objective function

Minimize the cost function:

C(χnd,ynd) =

t∑i=1

(yi − f(xi, χnd,ynd))

Wheref(x, χnd,ynd) =

∑xndj ∈N(xi)

wjyndj

Or can be written as:

(χndopt,yndopt) = arg min

(χnd,ynd)||ytr −Wχndynd||2

09/10/2010 iPAL Group Meeting 21

Separable ConvexityC(χnd,ynd) is not jointly convex as a function of χnd and ynd

After simplicity, the cost function exhibits separable convexity inWχnd and ynd

Figure: Illustration of simplex interpolation in 2-D carried out over a set ofnodes. The training point is denoted by ”x”.

09/10/2010 iPAL Group Meeting 22

Algorithm SOLVE

09/10/2010 iPAL Group Meeting 23

Algorithm RESOLVE

09/10/2010 iPAL Group Meeting 24

Algorithm RESOLVE

09/10/2010 iPAL Group Meeting 25

Result

09/10/2010 iPAL Group Meeting 26

Future Work

Seek for the direct expression of Wχnd

Prove convergence of the iterative algorithm

Determine the value of K, ε

Find the termination condition of each iteration

09/10/2010 iPAL Group Meeting 27

Review

Challenging: trade-off between complexity and quality ofapproximation

Two main research topics:

Optimizing node locationsOptimizing node values

Method:

Sort-Select-DampEfficient High Quality Color TransformationJoint Optimization of Node Locations and Output Values

The Joint method combine two research topics, perform promisingresult.

09/10/2010 iPAL Group Meeting 28

Acknowledgments

Prof. Vishal Monga, Penn State

Umamahesh Srinivas, iPAL

09/10/2010 iPAL Group Meeting 29