5

Maximizing submodular functionsMinimizing convex functions:

Polynomial time solvable!Minimizing submodular functions:

Polynomial time solvable!

Maximizing convex functions:

NP hard!Maximizing submodular functions:

NP hard!

But can get approximation guarantees

6

Example: Set cover

Node predictsvalues of positionswith some radius

SERVER

LAB

KITCHEN

COPYELEC

PHONEQUIET

STORAGE

CONFERENCE

OFFICEOFFICE

For A µ V: z(A) = “area covered by sensors placed at A”

Formally: W finite set, collection of n subsets Si

µ WFor A µ V={1,…,n} define z(A) = |i2 A Si|

Want to cover floorplan with discsPlace sensorsin building Possible

locations V

7

Set cover is submodular

SERVER

LAB

KITCHEN

COPYELEC

PHONEQUIET

STORAGE

CONFERENCE

OFFICEOFFICE

SERVER

LAB

KITCHEN

COPYELEC

PHONEQUIET

STORAGE

CONFERENCE

OFFICEOFFICE

S1 S2

S1 S2

S3

S4 S’

S’

A={S1,S2}

B = {S1,S2,S3,S4}

z(A[{S’})-z(A)

z(B[{S’})-z(B)

¸

8

Example: Feature selection• Given random variables Y, X1, … Xn

• Want to predict Y from subset XA = (Xi1,…,Xik

)

Want k most informative features:

A* = argmax IG(XA; Y) s.t. |A| · k

where IG(XA; Y) = H(Y) - H(Y | XA)

Y“Sick”

X1

“Fever”X2

“Rash”X3

“Male”

Naïve BayesModel

Uncertaintybefore knowing XA

Uncertaintyafter knowing XA

9

Example: Submodularity of info-gain

Y1,…,Ym, X1, …, Xn discrete RVs

z(A) = IG(Y; XA) = H(Y)-H(Y | XA)• z(A) is always monotonic• However, NOT always submodular

Theorem [Krause & Guestrin UAI’ 05]If Xi are all conditionally independent given Y,then z(A) is submodular!

Y1

X1

Y2

X2

Y3

X4X3

Hence, greedy algorithm works!

In fact, NO algorithm can do better than (1-1/e) approximation!

10

• People sit a lot• Activity recognition in

assistive technologies• Seating pressure as

user interface

Equipped with 1 sensor per cm2!

Costs $16,000!

Can we get similar accuracy with fewer,

cheaper sensors?

Leanforward

SlouchLeanleft

82% accuracy on 10 postures! [Tan et al]

Building a Sensing Chair [Mutlu, Krause, Forlizzi, Guestrin, Hodgins UIST ‘07]

11

How to place sensors on a chair?• Sensor readings at locations V as random variables• Predict posture Y using probabilistic model P(Y,V)• Pick sensor locations A* µ V to minimize entropy:

Possible locations V

Accuracy CostBefore 82% $16,000 After 79% $100

Placed sensors, did a user study:

Similar accuracy at <1% of the cost!

12

Bounds on optimal solution[Krause et al., J Wat Res Mgt ’08]

Submodularity gives data-dependent bounds on the performance of any algorithm

Sens

ing

qual

ity z(

A)Hi

gher

is b

ette

r

Water networks

data

0 5 10 15 200

0.2

0.4

0.6

0.8

1

1.2

1.4Offline

(Nemhauser)bound Data-dependent

bound

Greedysolution

Number of sensors placed

Summary (1)

• Minimization of submodular functions– Submodularity and convexity– Submodular Polyhedron– Symmetric submodular functions

Summary (2)• Pseudo-boolean functions

– Representation (polynomial, posiform, tableau, graph cut)

– Reduction to quadratic polynomial– Necessary and sufficient conditions for submodularity– Minimization of quadratic and cubic submodular

functions via graph cuts– Lower bound via roof duality

• LP via posiform representation• LP via linear relaxation• Max flow via symmetric graph construction

Further reading

• Combinatorial algorithms for submodular (and bisubmodular) function minimization

• More algorithms/bounds for maximizing submodular functions

• Linear and semidefinite relaxations• Matroids, greedoids, intersection of

matroids, polymatroids and more• Generalized roof duality