MINCUTS

SI/EECS 767Yang LiuApr 2, 2010

INTRODUCTION

A minimum cut is the smallest cut that will disconnect a graph into two disjoint subsets.

Application: Graph partitioning Data clustering Graph-based machine learning

BACKGROUND KNOWLEDGE

Cut A cut C = (S,T) is a partition of V of a graph G =

(V, E). An s-t cut C = (S,T) of a network N = (V, E) is a cut

of N such that s∈S and t∈T, where s and t are the source and the sink of N respectively.

The cut-set of a cut C = (S,T) is the set {(u,v)∈E | u∈S, v∈T}.

The size of a cut C = (S,T) is the number of edges in the cut-set. If the edges are weighted, the value of the cut is the sum of the weights.

(http://en.wikipedia.org/wiki/Cut_(graph_theory))

BACKGROUND KNOWLEDGE

Minimum cut A cut is minimum if the size of the cut is not larger

than the size of any other cut.

Max-flow-min-cut theorem The maximum flow between two vertices is always equal

to the size of the minimum cut times the capacity of a single pipe.

Also applies to weighted networks in which individual pipes can have different capacities.

BACKGROUND KNOWLEDGE

Max-flow min-cut theorem is very useful because there are simple computer algorithms that can calculate maximum flows quite quickly (in polynomial time) for any given networks.

We can use these same algorithms to quickly calculate the size of a cut set.

THE AUGMENTING PATH ALGORITHM

Basic idea: First find a path from source s to sink t

using the breadth-first search; Then find another path from s to t among

the remaining edges and repeat this procedure until no more paths can be found.

s t

A SIMPLE FIX

Allow fluid to flow simultaneously both ways down an edge in the network.

Mark Newman’s text book (preprint version)

Graph Clustering and Minimum Cut Tress

(Flake et al 2004)

INTRODUCTION

Clustering data into disjoint groups Data sets can be represented as

weighted graphs Nodes = entities to be clustered Edges = a similarity measure between

entities Present a new clustering algorithm

based on maximum flow. (in particular minimum cut tree)

MINIMUM CUT TREE

Also known as Gomory–Hu tree A weighted tree that consists of edges

representing all pairs minimum s-t cut in the graph

For every undirected graph, there always exists a min-cut tree.

See [Gomory and Hu 61] for detail and the algorithm for calculating min-cut trees.

CUT CLUSTERING ALGORITHM

CHOOSING

α→0, the trivial cut ({t}, V) α→∞, n trivial clusters, all singletons The exact value of α depends on the structure

of G and the distribution of the weights over the edges.

The algorithm finds all clusters either in increasing or decreasing order, we can stop the algorithm as soon as a desired cluster has been found.

α

HIERARCHICAL CUT-CLUSTERING ALGORITHM

Once a clustering is produced, contract the clusters into single nodes and apply the same algorithm to the resulting graph.

When contracting a set of nodes, they get replaced by a single new node; possible loops get deleted and parallel edges are combined into a single edge with weight equal to the sum of their weights.

break if ((clusters returned are of desired number and size)

or (clustering failed to create nontrivial clusters))

EXPERIMENTAL RESULTS

CiteSeer Citation network (documents as nodes,

citations as edges)Low level high level

CONCLUSION

Minimum cut trees, based on expanded graphs, provide a means for producing quality clusterings and for extracting heavily connected components.

A single parameter, α, can be used as a strict bound on the expansion of the clustering while simultaneously serving to bound the intercluster weight as well.

Bipartite Graph Partitioning and Data Clustering

(Zha et al 2001)

INTRODUCTION

Bipartite graph Two kinds of vertices One representing the original vertices and the

other representing the groups to which they belong

Examples: terms and documents, authors and authors of an article

Adapt undirected graphs criteria for bipartite graph partitioning and therefore solve the bi-clustering problem.

BIPARTITE GRAPH PARTITIONING

Bipartite graph G(X, Y, W) In the context of document clustering

X represents the set of terms Y represents the set of documents W = (wij) represents term frequency of i in

document j.

Tends to produce unbalanced clusters

The problem becomes following optimization problem

Computational complexity: general linear in the number of documents to be clustered

EXPERIMENTS

20 news groups

Learning from Labeled and Unlabeled Data using

Graph Mincuts

(Blum & Chawla 2001)

INTRODUCTION

Many application domains suffer from not having enough labeled training data for learning.

Large amounts of unlabeled examples How unlabeled data can be used to aid

classification

THE GRAPH MINCUT LEARNING ALGORITHM

A set L of labeled examples A set U of unlabeled examples Binary classification

L+ to denote the set of positive examples L- to denote the set of negtive examples

Construct a weighted graph G = (V, E), where V = L∪U∪{v+, v-}, e ∈ E is a weight w(e).

v+, v-: classification vertices;

other vertices: example vertices; w(v, v+) = ∞ for all v ∈L+ and w(v, v-) = ∞ for all

v ∈L- The edge between example vertices are

assigned weights based on some relationship (similarity/distance) between the examples

Determine a minimum (v+, v-) cut for the graph, i.e. the minimum total weight set of edges whose removal disconnects v+ and v-. (using a max-flow algorithm in which v+ is the source, v- is the sink)

Assign a positive label to all unlabeled examples in the set V+ and a negative label to all unlabeled examples in the set V-.

*edges between examples which are similar to each other should be given a high weight

POTENTIAL PROBLEM

If there are few labeled examples, it can cause mincut to assign the unlabeled examples to one class or the other

If the graph is too sparse, it could have a number of disconnected components

Therefore it is important to use a proper weighting function

EXPERIMENTAL ANALYSIS

Datasets: UCI, 2000 The mincut algorithm has many degrees of freedom

in terms of how the edge weights are defined. Mincut-3: each example is connected to its nearest

labeled example and two other nearest examples overall Mincut- δ: if too nodes are closer than δ , they are

connected Mincut- δ0: max δ which graph has a cut of value 0 Mincut- δ1/2: the size of the largest connected component in

the graph is half the number of datapoints Mincut- δopt: the values of δ that corresponds to the least

classification error in hindsight

REVIEW

The basic idea of this algorithm is to build a graph on all the data with edges between examples that are sufficiently similar

then to partition the graph into a positive and a negative set in a way that (a) agrees with the labeled data (b) cuts as few edges as possible

Semi-supervised Learning using Randomized

Mincuts

(Blum et al 2004)

INTRODUCTION

The drawbacks of the graph mincut approach: A graph may have many minimum cuts and

the mincut algorithm produces just one, typically the “leftmost” one using standard network flow algorithms.

Produced based on joint labeling rather than per-node probabilities.

Can be improved by averaging over many small cuts.

BASIC IDEA

Repeatedly add artificial random noise to the edge weights

Solve for the minimum cut in the resulting graphs

Output a fractional label for each example corresponding to the fraction of the time it was on one side or the other

RANDOMIZED MINCUTS WITH SANITY CHECK

Given a graph G, produce a collection of cuts by repeatedly adding random noise to the edge weights and then solving for the minimum cut in the perturbed graph.

Sanity check: remove those that are highly unbalanced (any cut with less than 5% of the vertices on one side in this paper)

Predict based on a majority vote

EXAMPLE

Overcome some of the limitations of the plain mincut algorithm.

Consider a graph which simply consists of a line with a positively labeled node at one end and a negatively labeled node at the other end with the rest being unlabeled. Plain mincut: the cut will be the leftmost or right

most one Randomized mincut: end up using the middle of

the line with confidence that increases linearly out to the endpoints

UNIFORM DISTRIBUTION OF MINIMUM CUTS

GRAPH DESIGN CRITERIA

The graph should be either be connected or at least have the property that a small number of connected components cover nearly all the examples.

Good to create a graph that at least has some small balanced cuts.

TWO GRAPH CONSTRUCTION METHODS MST: simply construct a minimum

spanning tree on the entire dataset δ-MST: connect two points with an edge

if they are within a radius δ. Then veiw the components produced as super nodes and connect them via an MST.

EXPERIMENTAL ANALYSIS

Handwritten digits 20 newsgroups Various UCI datasets

CONCLUSTION

Improve performance when the number of labeled examples is small

Providing a confidence score for accuracy-coverage curves.

A Sentimental Education: Sentiment Analysis using

Subjectivity Summarization based on

Minimum Cuts

(Pang & Lee 2004)

INTRODUCTION

machine-learning method that applies text-categorization techniques to determine the sentiment polarity—positive (“thumbs up”) or negative (“thumbs down”)

Previous approaches focused on selecting indicative lexical features

Their approach: Label the sentences as either subjective or

objective Apply a standard machine-learning

classifier to the resulting extract.

METHODS

SUBJECTIVITY DETECTION

n items x1, . . . , xn to divide into two classes C1 and C2

• Individual scores indj(xi): non-negative estimates of each xi’s preference for being in Cj based on just the features of xi alone;

• Association scores assoc(xi, xk): non-negative estimates of how important it is that xi and xk be in the same class.

Minimize the partition cost

:

Build an undirected graph G with vertices {v1, . . . , vn, s, t}; the last two are, respectively, the source and sink.

Add n edges (s, vi), each with weight ind1(xi), and n edges (vi, t), each with weight ind2(xi).

Finally, add edges (vi, vk), each with weight assoc(xi, xk).

EVALUATION FRAMEWORK

Classifying movie reviews as either positive and negative

The correct label can be extracted automatically from rating information (number of stars)

CONSTRUCTION OF THE GRAPH

The source s and sink t correspond to the class of subjective and objective sentences

Each internal node vi corresponds to the document’s ith sentence si

Set the ind1(si) to : Naive Bayes’ estimate of the

probability that sentence s is subjective

EXPERIMENTAL RESUTLS

NB as a subjectivity detector in conjunction with a NB document-level polarity 86.4% accuracy VS 82.8% without extraction

SVM: 87.15% VS 86.4% Sentences labeled as objective as input: 71% for NB and 67% for SVMs Taking just the N most subjective sentences:

5 most subjective sentences is almost as informative as the Full review while containing only about 22% of the source words.

CONCLUSION

Subjectivity detection can compress reviews into shorter extracts still retain polarity information

Minimum-cut frame work results in the development of efficient algorithm for sentiment analysis

Questions?