Graph Cuts

Marc Niethammer

Segmentation by Graph-CutsA way to compute solutions to the optimization problems we looked at before.

Example: Binary Segmentation

Images from ECCV Tutorial, Kumar/Kohli

Need to partition the picture into foreground and background.

Segmentation

Graph G=(V,E)

Images from ECCV Tutorial, Kumar/Kohli

But there is much more to graph cuts than binary segmentation (as we will see later)

Approach: Interpret the image as a graph,where pixels are connected to its neighbors.

Goal: Cut the graph into pieces to obtain the desired image partition.Assign a label to every pixel.

Segmentation

Optimization Problem:

Looking at it on a pixel-by-pixel basis (where f is the labeling):

Problem: We cannot try all possible pairings for the labels f. Need an efficient algorithm to solve this problem.

Maximum Flow and Minimum Cut

Solution: Borrow algorithms for network flows and use them on images.

Preview: Graph structure for binary labeling

Represent the image such that pixels are connected to neighbors (pairwise interaction cost) as well as to the source and a target vertices (data cost).

Images: Boykov

The cut that separates source (s) from sink (t) gives the segmentation.

Max-Flow/Min-Cut Theorem

For any network having a single origin and a single destination node,the maximum possible flow from origin to destination equals theminimum cut value for all the cuts in the network.

If we want to find the minimum cut, we can compute the maximum flowand look for the cut(s) that separate origin and destination by cuttingthrough bottlenecks of the network.

Example: Ford & Fulkerson

The Ford and Fulkerson algorithm is one of many possibilities to computethe maximal flow.

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Example: Ford & Fulkerson

Cost of min-cut is 4. Divides the nodes (pixels) into two groups.Which equals the maximum flow.

Example: Ford & Fulkerson

This is where everything flows. Flow is conserved at nodes.

Look at the difference between initial and final capacities.Ignore negative capacities.

Why to update backward flow?

Image: John Chinneck

Can undo flows. Final flow does not include the central edge.

Image Denoising

This approach allows us (amongst many things) to compute binary segmentations.Image: Boykov

Original Noisy Reconstructed

Multi-Label Case

Image: Boykov

Multi-Label Case

Image: Boykov

Graph Cut Properties

• Combinatorial optimization for a restricted class of energies• Globally optimal (for certain problem classes)• Discrete formulation with combinatorial optimization

(compare with continuous variational approaches)

Problem: Large memory consumption, in particular in 3D.

Metrication Errors

Minimum cost cut (standard 4-neighborhoods)

Minimum length geodesic contour (image-based Riemannian metric)

Images: Boykov

Can choose different weighted neighborhood to reduce metrication errors.Also, continuous maximal flows exist.

Interactive Segmentations

Images: Boykov

Continuous Maximum FlowContinuous version of maximum flow [Appleton].Results in a PDE and can be solved as such.

Energy to be minimized:

Play the usual trick of introducing the auxiliary variable p, which we maximize for:

Now solve this by gradient descent.

Continuous Maximum Flow

The variation is

The gradient descent scheme becomes

Segmentation with Continuous Max-Flow

Images: Unger

Thresholding

Continuous max-flowwith seeds.

Bone Segmentation

Also works nicely in 3D.