Compression-based Texture Merging

“Unsupervised Segmentation of Natural Images via Lossy Data Compression”

Allen Y. Yang, John Wright, Shankar Sastry, Yi Ma

Segmentation cues

Color Edge Contour Texture

Filter bank Color value stacks

Filter bank

Response to a 2D-filter bank

Color value stacks

A w×w window of each of the three L*a*b channels around each pixel is convoluted with a Gaussian and then all channels are stacked into a single vector v.

Two assumptions about natural images

1. The distribution of texture features in a natural image is (approximately) a mixture of Gaussians that can be degenerate and of different dimensions, one for each image segment.

2. At any given quantization scale, the optimal segmentation is the one that gives the most compressed representation of the image features, as measured by the number of binary bits needed to encode all the features.

Lossy Compression

X YC

010011101010011……

rate-distortion function Memoryless (Independent) Gaussian Source

The total number of bits needed to encode the data set V, including bits needed to represent the codebook and mean

Upper bound of the total number of bits needed to code V drawn from a mixture of Gaussians

A greedy scheme - pairwise steepest descent

As a greedy descent scheme, the algorithm does not guarantee to always find the globally optimal segmentation for any given (V, ε2). In our experience, the main factor affecting the global convergence of the algorithm appears to be the density of the samples relative to the distortion ε2.

Image Segmentation via Lossy Compression

Superpixels Region adjacency graph (RAG)

Superpixels

In order to group edge pixels appropriately, we preprocess an image with a low-level segmentation based on local cues such as color and edges. That is, we oversegment the image into (usually several hundred) small, homogeneous regions, known asSuperpixels.

Such low-level segmentation can be effectively computed using K-Means or Normalized-Cuts (NCuts)

Region adjacency graph (RAG) In order to enforce that the resulting segmentation consists o

f connected segments, we impose an additional spatial constraint that two segments Si and Sj can be merged together only if they are adjacent in the 2D image.

We represent the RAG using an adjacency list G{i} for each segment Si.

Choosing the Distortion

ε=0.001 ε=0.02 ε=0.05

Heuristically select the scale by stipulating that feature distributions in adjacent regions must be sufficiently dissimilar, i.e.

the distance between the means of the adjacent segments must be larger than a preselected threshold γ