Automatic Image Rescaling Preserving Design Intention

Research Update

Prasad Gabbur

Goal

Simple scaling

Original:

Scaling that preserves the design intention of the objects:

Approach

• Split input image into background (bg) and foreground (fg) layers

• Scale the bg and fg layers separately

Background

Foreground

Background layer scaling

• Scale and shift elements to fit new page

• Ignore aspect ratio

• Classification of background elements

Area Horizontal

(Top, Bottom)

Vertical

(Left, right)

Background layer scaling• Scale and shift of background elements

Scale

Scale + Shift

Scale

+Shift

Sx > Sy Sy > Sx

Area

Horizontal

Vertical

To fit new page

To fit new page width

To fit new page height

Foreground layer scaling

• Scale and shift elements to fit new page

• Preserve aspect ratio

• Classification of foreground elements

Corner

(TL, TR, BL, BR)

Horizontal

(Top, Bottom)

Vertical

(Left, Right)

Foreground layer scaling

Scale

+ Shift

Scale

+ Shift

Scale

+ Shift

Corner

Horizontal

Vertical

Sx > Sy Sy > Sx

Foreground layer scaling

• Scaling preserves aspect ratio (Scale factor = min (Sx, Sy))• Shifting preserves distance ratio (dL/dR = const, dT/dB = const)

dL_old dR_old

dB_old

dT_olddT_new

dB_new

dL_new dR_new

Original Sx > Sy

(dL_new / dR_new)

= (dL_old / dR_old)

Sy > Sx

(dT_new / dB_new)

= (dT_old / dB_old)

HorizontalVertical

Shift of horizontal elements

Shift of vertical elements

Element extraction

• Elements are connected regions in the foreground or background layer

Multilayer image

Background layer(s)

Foreground layer(s)

Extract

alpha

channel

Label

connected

components

Connected component labeling

• Group together spatially connected pixels as a single component

• Each component is assigned a unique integer label

4-connectivity 8-connectivity

Connected component labeling• Region coloring algorithm (4-connected) [Ballard & Brown, 1982]

– Each pixel (Xc) in the image is scanned with the following mask:

XcXl

Xu

new_label = 1

If (Xu Є background & Xl Є foreground), then

label (Xc) = label (Xl)

Else if (Xu Є foreground & Xl Є background), then

label (Xc) = label (Xu)

Else if (Xu Є foreground & Xl Є foreground), then

label (Xc) = min ( label (Xl), label (Xu) )

Else

label (Xc) = new_label

new_label = new_label + 1

Connected component labeling

• Basic region coloring algorithm is slow• Requires multiple passes through the image• A faster version is realized with the help of a

custom data structure

• An array of the above data type can store information about all connected components

• Only one pass through the image is necessary

Connected component labeling

• A single raster scan of the image gives rise to following structure

……………………

Image with two connected components

Data structure at the end of a single image scan

Connected component labeling• Links in the array can be visualized as a tree structure

• Nodes in the tree are equivalent labels of a connected component

1 1 2

3 4

11 12 5 10

6 7 8 9

A fictitious connected component

Connected component labeling

• Trees with different configurations are possible depending on region complexity

• All branches merge at the bottom

One branch Two branches Four branches

Connected component labeling

• Resolving label equivalences

• All the equivalent labels are assigned the least value among them by stepping through the tree

1

1

1

1

1

1

1

1

1

1

1

1

1

2

3

4

5

6

7

8

9

10

11

12

Resolve

Connected component labeling

1 1 2

3 4

11 12 5 10

6 7 8 9

• Resolving label equivalences

1 1 1

1 1

1 1 1 1

1 1 1 1

Resolve

Elements

• Each connected component in the background or foreground layer is an element

• Geometric properties (bounding box center and limits) are computed as part of the labeling process

• Elements are classified based on the geometric properties

Background layer Foreground layer

Scaling issues*

• Scale up– Sparse distribution of pixels in the output

image– Bilinear interpolation to fill in pixel values

• Scale down– Aliasing due to sub-sampling– Low pass filtering before sub-sampling

* Thanks to Jian Fan, HP Labs.

Results• Background layer

Original

Labeled

Scaled (Sx >S y)

Scaled (Sy >S x)

Results• Foreground layer

Original

Labeled

Scaled (Sx >S y)

Scaled (Sy >S x)

Next

• Stitch together the foreground and background layers

• Work on an XML design for input

Thank you!