Download - Constraint Optimization
![Page 1: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/1.jpg)
Constraint Optimization 1
Constraint Optimization
Presentation by Nathan StenderChapter 13 of Constraint Processing by Rina Dechter
3/25/2013
![Page 2: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/2.jpg)
Constraint Optimization 2
Motivation
3/25/2013
• Real-life problems often have both hard and soft constraints– Hard constraints must be satisfied– Soft constraints would like to be satisfied
• Goal is to – optimize soft constraints – while satisfying hard constraints
• Applications– Planning– Scheduling– Design
![Page 3: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/3.jpg)
Constraint Optimization 3
Example: Combinatorial Auction
• Bidders place a bid on a set of items
• Hard constraints: no two bids sharing an item can be selected
• Soft constraints: cost of selecting each bid
• Auctioneer must select bids to maximize revenue
3/25/2013
![Page 4: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/4.jpg)
Constraint Optimization 4
Outline
3/25/2013
• Motivation• Constraint Optimization and Cost Networks Section 13.1
• Branch-and-Bound Search Section 13.2
• Bucket Elimination for Optimization Section 13.3
• Mini-bucket Elimination Section 13.4
• Search with Mini-bucket Heuristics Section 13.5
• Summary
![Page 5: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/5.jpg)
Constraint Optimization 5
Constraint Optimization Problem
• COP = Constraint network + a cost function
• Global cost function– Real-valued functional components: F1,…,Fl– F1,…,Fl defined over scopes: Q1,…,Ql, Qj ∈X– Assignment: ā = (a1,…, an), ai in the domain of xi
3/25/2013
![Page 6: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/6.jpg)
Constraint Optimization 6
Example: Combinatorial Auction• Boolean variables bi for bids
– Each bid has an associated cost ri– bi = 1 if bid is selected, bi =v0 if bid is rejected
• Hard constraint Rij between bi and bj – If they share an item: (bi=1,bi=1)∉Rij
• Soft constraint F(bi) – F(bi)=ri if bi =1, F(bi)=0 if bi =0
• Goal is to maximize:3/25/2013
Rijbi bj cost0 0 01 0 ri0 1 rj
![Page 7: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/7.jpg)
Constraint Optimization 7
Example: Combinatorial Auction
Consider the following set of bids:
3/25/2013
b1b2 b3
b4 b5Optimal solution: b1 = 0, b2 = 1, b3 = 1, b4 = 0, b5 = 0
b1 = {1,2,3,4} Db1 = {0,1} r1 = 8b2 = {2,3,6} Db2 = {0,1} r2 = 6b3 = {1,4,5} Db3 = {0,1} r3 = 5b4 = {2,8} Db4 = {0,1} r4 = 2b5 = {5,6} Db5 = {0,1} r5 = 2
R12, R13, R14, R24, R25, R35
![Page 8: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/8.jpg)
Constraint Optimization 8
• 4-tuple:– (X,D,Ch) is a constraint network
– Cs = {FQ1,…, FQl} is a set of cost components
– Ch and Cs also called hard and soft constraints–
• Graph representation– Variables are nodes– Arcs connect variables in the same hard or soft constraint
3/25/2013
Cost Network
![Page 9: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/9.jpg)
Constraint Optimization 9
Solving COP as a Series of CSPs
• It is always possible to a solve COP as a series of CSPs
• Add a single hard constraint:
• Increase the cost-bound:
• Eventually the cost-bound so high no solution exists, the previous solution is optimal
3/25/2013
![Page 10: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/10.jpg)
Constraint Optimization 10
Solving COP as a Series of CSPs
• With cost Cj = 1, we have 4 solutions• C2 = 11, 1 solution• C1 = 12, 0 solutions• Backtrack to previous solution, which is optimal
• The cost of solving so many CSPs is prohibitive• Instead, we use search & inference
3/25/2013
![Page 11: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/11.jpg)
Constraint Optimization 11
Outline
3/25/2013
• Motivation• Constraint Optimization and Cost Networks Section 13.1
• Branch-and-Bound Search Section 13.2
• Bucket Elimination for Optimization Section 13.3
• Mini-bucket Elimination Section 13.4
• Search with Mini-bucket Heuristics Section 13.5
• Summary
![Page 12: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/12.jpg)
Constraint Optimization 12
Branch-and-Bound Search
• Extends backtracking search– Traverses hard constraint search tree with DFS– Keeps track of two bounds using cost function
• Bounds– Lower bound : cost of best solution so far– Upper bound using bounding evaluation function– For partial assignment, :– If , backtrack
3/25/2013
![Page 13: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/13.jpg)
Constraint Optimization 13
Example: Combinatorial Auction
3/25/2013
1
0
0
1
0
0
1
0
1
0
0
0
23101712101081515151311119
0
10 23
17
12
10
10 8
15
15
15
13
11
9
10
r1 = 8r2 = 6r3 = 5r4 = 2r5 = 2R12, R13, R14, R24, R25, R35
b1 b2 b3 b4 b5
![Page 14: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/14.jpg)
Constraint Optimization 14
Russian Doll Search• Run n successive BnB searches
– Each search involves a single additional variable– First subproblem includes just nth variable– The (n-i+1)th subproblem involves xi,…,xn
• Bounds are augmented by previous searches
• Previous solutions can also heuristically guide search
• Extra pruning power is often worth extra cost of doing n searches
3/25/2013
![Page 15: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/15.jpg)
Constraint Optimization 15
Example: Combinatorial Auction
3/25/2013
1
0
0
0
1
0
1
0
0
0
0
8+11=19
8+7=15
8+4=12
8+2=10
11
11
11
11
11
7
10
Russian Doll: Previous searches:
Over b2,b3,b4,b5:b2=1,b3=1,b4=0,b5=0 11Cost:
Initial lower bound given by previous optimal solution
Over b3,b4,b5:b3=1,b4=0,b5=1 7Over b4,b5:b4=1,b5=1 4Over b5:b5=1 2Optimal solutions from previous searches estimate upper bound
r1 = 8, r2 = 6, r3 = 5, r4 = 2, r5 = 2
![Page 16: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/16.jpg)
Constraint Optimization 16
Example: Combinatorial Auction
3/25/2013
1
0
0
0
1
0
1
0
0
0
0
19
15
12
10
11
11
11
11
11
7
10
Russian Doll:
10
1
0
0
1
0
0
1
0
1
0
0
0
0
23
17
12
10
10 8
15
15
15
13
11
9
10
Original:
b1 b2 b3 b4 b5
![Page 17: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/17.jpg)
Constraint Optimization 17
Improving Branch-and-Bound• Substantial work on BnB has been done for integer
programs by OR community
• Primary way to improve BnB is to improve accuracy of bounding function
• Constraint community extends same ideas to general constraints and cost functions
• Generate bounding functions using Bucket Elimination
3/25/2013
![Page 18: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/18.jpg)
Constraint Optimization 18
Outline
3/25/2013
• Motivation• Constraint Optimization and Cost Networks Section 13.1
• Branch-and-Bound Search Section 13.2
• Bucket Elimination for Optimization Section 13.3
• Mini-bucket Elimination Section 13.4
• Search with Mini-bucket Heuristics Section 13.5
• Summary
![Page 19: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/19.jpg)
Constraint Optimization 19
Example: All soft constraints
Consider the following cost network:
3/25/2013
ab c
f
g
d
and ordering: a,c,b,f,d,g
g
d
f
b
c
a
![Page 20: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/20.jpg)
Constraint Optimization 20
Example: All soft constraints
3/25/2013
a
c
b
f
d
g
![Page 21: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/21.jpg)
Constraint Optimization 21
Derivation of Bucket Elimination
3/25/2013
Cost function:
Move each function into lowest (leftmost) bucket:
Maximize functions in highest bucket for bucket variable and move to new lowest bucket:
Repeat:
![Page 22: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/22.jpg)
Constraint Optimization 22
Bucket Elimination for Optimization• Inference algorithm for COP
• Analogous to normal bucket-elimination– Each bucket holds a set of cost functions– Join operation replaced with summation– Projection replaced with maximization (or min)
• Forward direction generates an augmented cost network representation
• Backward direction generates optimal solution in linear time
3/25/2013
![Page 23: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/23.jpg)
Constraint Optimization 23
Joining Soft Constraints
Joining is replaced by summation:
3/25/2013
x1 x2 Cost
0 0 4
0 1 5
1 0 3
1 1 1
x2 x3 Cost
0 0 3
0 1 2
1 0 1
1 1 0
x1 x2 x3 Cost
0 0 0 7
0 0 1 6
0 1 0 6
0 1 1
…
5
![Page 24: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/24.jpg)
Constraint Optimization 24
Projecting Soft Constraints
Projection is replaced by maximization:
3/25/2013
x1 x2 x3 Cost
0 0 0 7
0 0 1 6
0 1 0 6
0 1 1 5
…
x1 x3 Cost
0 0 7
0 1
…
6
![Page 25: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/25.jpg)
Constraint Optimization 25
Bucket Elimination for Opt: Properties
• Can incorporate hard constraints by adding back join and project operations
• It is always possible to treat hard constraints as cost functions, but not optimal– Hard constraints have stronger properties that can only be exploited if
expressed explicitly– Hard constraints permit use of local consistency enforcing
• Complexity is based on induced width– For r hard and soft constraints and ordering d with induced width w*(d)– Time and space: O(r•expw*(d)+1)
3/25/2013
![Page 26: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/26.jpg)
26
Example: Combinatorial Auction
3/25/2013Constraint Optimization
b1
b2 b3
b4 b5
Opt
![Page 27: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/27.jpg)
Constraint Optimization 27
R41b1
b4
0 00 11 0
r(b4)b4
C0 01 2
R42b2
b4
0 00 11 0
R124b1
b2
b4
0 0 00 0 10 1 01 0 01 1 0
h4(b1,b2)b1
b2
C0 0 20 1 01 0 01 1 0
h4(b1,b2,b4)b1
b2
b4
C0 0 0 00 0 1 20 1 0 01 0 0 01 1 0 0
Example: Combinatorial Auction
3/25/2013
R21, R42, r(b4) h4(b1 ,b2)
Join hard constraintsSum soft constraints (cost)Maximize over bucket variable Opt
b1
b2 b3
b4 b5
![Page 28: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/28.jpg)
Constraint Optimization 28
h3(b1,b3,b5)b1
b3
b5
C0 0 0 00 0 1 00 1 0 51 0 0 01 0 1 0
h3(b1,b5)b1
b5
C0 0 50 1 01 0 01 1 0
R135b1
b3
b5
0 0 00 0 10 1 01 0 01 0 1
r(b3)b3
C0 01 5
R31b1
b3
0 00 11 0
R35b3
b5
0 00 11 0
Example: Combinatorial Auction
3/25/2013
R31, R35, r(b3) h3(b1 ,b5)
Join hard constraintsSum soft constraints (cost)Maximize over bucket variable Opt
b1
b2 b3
b4 b5
![Page 29: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/29.jpg)
Constraint Optimization 29
R125b1
b2
b5
0 0 00 0 10 1 01 0 01 0 1
h2(b1, b2,b5)b1
b2
b5
C0 0 0 20 0 1 20 1 0 61 0 0 01 0 1 0
h4(b1,b2)b1
b2
C0 0 20 1 01 0 01 1 0
h2(b1,b5)b1
b5
C0 0 60 1 21 0 01 1 0r(b2)b
2C
0 01 6
R12b1
b2
0 00 11 0
R25b2
b5
0 00 11 0
Example: Combinatorial Auction
3/25/2013
R12, R25, r(b2), h4(b1 ,b2) h2(b1 ,b5)
Join hard constraintsSum soft constraints (cost)Maximize over bucket variable Opt
b1
b2 b3
b4 b5
![Page 30: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/30.jpg)
Constraint Optimization 30
h5(b1)b1
C0 111 2
h5(b1,b5)b1
b5
C0 0 110 1 41 0 01 1 2
r(b5)b5
C0 01 2
h2(b1,b5)b1
b5
C0 0 60 1 21 0 01 1 0
h3(b1,b5)b1
b5
C0 0 50 1 01 0 01 1 0
Example: Combinatorial Auction
3/25/2013
r(b5), h3(b1 ,b5), h2(b1 ,b5) h5(b1)
Sum soft constraints (cost)Maximize over bucket variable Opt
b1
b2 b3
b4 b5
![Page 31: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/31.jpg)
Constraint Optimization 31
Opt11
h1(b1)b1
C0 111 10
r(b1)b5
C0 01 8
h5(b1)b1
C0 111 2
Example: Combinatorial Auction
3/25/2013
r(b1), h5(b1 ) Opt
Sum soft constraints (cost)Maximize over bucket variable Opt
b1
b2 b3
b4 b5
![Page 32: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/32.jpg)
Constraint Optimization 32
h1(b1)b1
C0 111 10
h5(b1,b5)b1
b5
C0 0 110 1 41 0 01 1 2
h2(b1, b2,b5)b1
b2
b5
C0 0 0 20 0 1 20 1 0 61 0 0 01 0 1 0
h3(b1,b3,b5)b1
b3
b5
C0 0 0 00 0 1 00 1 0 51 0 0 01 0 1 0
h4(b1,b2,b4)b1
b2
b4
C0 0 0 00 0 1 20 1 0 01 0 0 01 1 0 0
Example: Combinatorial Auction
3/25/2013
Go in the reverse direction to generate a solution:
b1=0 b5=0 b2=1 b3=1 b4=0
Cost of optimal solution = 0 + 0 + 6 + 5 + 0 = 11
![Page 33: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/33.jpg)
Constraint Optimization 33
Outline
3/25/2013
• Motivation• Constraint Optimization and Cost Networks Section 13.1
• Branch-and-Bound Search Section 13.2
• Bucket Elimination for Optimization Section 13.3
• Mini-bucket Elimination Section 13.4
• Search with Mini-bucket Heuristics Section 13.5
• Summary
![Page 34: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/34.jpg)
Constraint Optimization 34
Example: All Soft Constraints
3/25/2013
Bucket: Mini-bucket (i=3):
Upper Bound(constraints are ignored, thus, the bound is optimistic)
![Page 35: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/35.jpg)
Constraint Optimization 35
Mini-bucket Elimination
• BE is exponential in separator size (space)
• Instead use mini-bucket elimination– Bucket are partitioned to restrict arity of projected
constraints to at most i (variables)– Result generates an estimation of optimal solution– Size of mini-buckets limits complexity and accuracy of
estimation– Selection of partition will also affect results, so a good
heuristic is needed
3/25/2013
![Page 36: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/36.jpg)
Constraint Optimization 36
Example: Combinatorial Auction
3/25/2013
b1
b2 b3
b4 b5
Upper Bound
![Page 37: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/37.jpg)
Constraint Optimization 37
h4(b2,b4)b2
b4
C0 0 00 1 21 0 0
h4(b2)b2
C0 2
1 0
r(b4)b4
C0 01 2
R42b2
b4
0 00 11 0
Example: Combinatorial Auction
3/25/2013
R42, r(b4) h4(b2)
Sum soft constraints (cost)Maximize over bucket variable Bound
b1
b2 b3
b4 b5
![Page 38: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/38.jpg)
Constraint Optimization 38
h3(b3,b5)b3
b5
C0 0 0
0 1 0
1 0 5
h3(b5)b5
C0 5
1 0
r(b3)b3
C0 01 5
R35b3
b5
0 00 11 0
Example: Combinatorial Auction
3/25/2013
R35, r(b3) h3(b5)
Sum soft constraints (cost)Maximize over bucket variable Bound
b1
b2 b3
b4 b5
![Page 39: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/39.jpg)
Constraint Optimization 39
h2(b2,b5)b2
b5
C0 0 2
0 1 2
1 0 6
h2(b5)b5
C0 6
1 2
r(b2)b2
C0 01 6
R25b2
b5
0 00 11 0
h4(b2)b2
C0 21 0
Example: Combinatorial Auction
3/25/2013
R25, r(b2), h4(b2) h2(b5)
Sum soft constraints (cost)Maximize over bucket variable Bound
b1
b2 b3
b4 b5
![Page 40: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/40.jpg)
Constraint Optimization 40
h5(b5)b5
C0 111 2
h511r(b5)b5
C0 01 2
h3(b5)b5
C0 51 0
h2(b5)b5
C0 61 0
Example: Combinatorial Auction
3/25/2013
r(b2), h3(b5) , h2(b5) h5
Sum soft constraints (cost)Maximize over bucket variable Bound
b1
b2 b3
b4 b5
![Page 41: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/41.jpg)
Constraint Optimization 41
Bound19h1(b1)b1 C0 111 19
r(b1)b1
C0 01 8
h511
Example: Combinatorial Auction
3/25/2013
r(b1), h5 Bound
Sum soft constraints (cost)Maximize over bucket variable Bound
b1
b2 b3
b4 b5
![Page 42: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/42.jpg)
Constraint Optimization 42
h5(b5)b5 C0 11
1 2
h2(b2,b5)b2
b5
C0 0 2
0 1 2
1 0 6
h3(b3,b5)b3
b5
C0 0 0
0 1 0
1 0 5
h4(b2,b4)b2
b4
C0 0 0
0 1 2
1 0 0
h1(b1)b1 C0 11
1 19
Example: Combinatorial Auction
3/25/2013
Go in the forward direction to generate a solution (lower bound):
b1=1 b5=0 b2=0 b3=0 b4=0
b1
b2 b3
b4 b5
Cost of solution/lower bound: 8
Interval bound on optimal solution: [8,19]
![Page 43: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/43.jpg)
Constraint Optimization 43
Mini-bucket Elimination: Properties• Partitioning buckets amounts to moving maximization inside summation:
• Going up the elimination ordering (backward direction)– Maximization over mini-buckets gives an upper bound
• Going down the instantiation ordering (forward direction)– Finding a solution provides a lower bound
• Complexity depends on i, assuming i < w*– If : Time and space complexity = O(exp(i)) – Otherwise: Time = O(r•exp(i)), Space = O(r•exp(i-1)), where r is the number of
cost functions
3/25/2013
![Page 44: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/44.jpg)
Constraint Optimization 44
Outline
3/25/2013
• Motivation• Constraint Optimization and Cost Networks Section 13.1
• Branch-and-Bound Search Section 13.2
• Bucket Elimination for Optimization Section 13.3
• Mini-bucket Elimination Section 13.4
• Search with Mini-bucket Heuristics Section 13.5
• Summary
![Page 45: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/45.jpg)
Constraint Optimization 45
Search with Mini-Bucket Heuristics
• Mini-bucket allows us to bound error, however we may need the optimal solution
• We can improve search with estimations from mini-buckets– Search using BnB, use mini-bucket to improve accuracy of
bounding function– At each step, estimate the cost of optimal solution given a
partial assignment using mini-bucket– Assuming a static variable ordering, buckets can be computed
before search
3/25/2013
![Page 46: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/46.jpg)
Constraint Optimization 46
8fmb (b1=0,b5=0,b2=1,b3=0) = 6+0+h4(b2) = 6+0+0 = 6 fmb (b1=0,b5=0,b2=0) = 0+h4(b2) +h3(b5) = 0+2+5 = 7 fmb (b1=1,b5=0) = 8+0+h2(b5) +h3(b5) = 8+0+6+5 = 19 fmb (b1=0) = 0+h5 = 0+11 = 11 fmb (b1=0) = 0+h5 = 0+11 = 11 fmb (b1=0,b5=0) = 0+h2(b5) +h3(b5) = 0+6+5 = 11 fmb (b1=0,b5=1) = r5+h2(b5) +h3(b5) = 2+2+0 = 4 fmb (b1=0,b5=0,b2=1) = r2+h4(b2) +h3(b5) = 6+0+5 = 11 fmb (b1=0,b5=0,b2=1,b3=1) = 6+r3+h4(b2) = 6+5+0 = 11 fmb (b1=1) = r1+h5 = 8+11 = 19 fmb (b1=1,b5=1) = 8+r5+h2(b5) +h3(b5) = 8+2+0+0 = 10
Example: Combinatorial Auction
3/25/2013
1
1
0
0
0
0
0
0
0
0
0
0
0
0
11
1
810
10
10
10
8
8
1911
1119
4
11 6
1
11 1
h4(b2)b2
C0 2
1 0
h3(b5)b5
C0 5
1 0
h2(b5)b5
C0 6
1 0
7
b1
b2 b3
b4 b5h511
Bound
L=1011 Lower bound is initialized to lower bound found during mini-bucket
![Page 47: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/47.jpg)
Constraint Optimization 47
Mini-bucket Heuristics Properties
3/25/2013
Definition 13.1: Given an ordered set of augmented buckets generated by the mini-bucket algorithm, the bounding function is the sum of all the new functions that are generated in buckets p+1 through n and reside in buckets 1 through p. That is:
Theorem 13.5 (paraphrased): • The mini-bucket heuristic never under-estimates the cost of the optimal
extension of a given partial assignment
• The heuristic is monotonic, as search goes deeper the bound can only decrease, meaning it is more accurate.
![Page 48: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/48.jpg)
Constraint Optimization 48
Example: Combinatorial Auction
3/25/2013
In this case the bounding functions perform similarly, but mini-bucket is more accurate!
First-choice Mini-bucket
1
1
0
0
0
0
0
0
0
0
0
0
0
7
11
0
10
12
17
23
15
2113
1523
15
9
0
13 8
1
13 1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
7
11
1
810
10
10
10
8
8
1911
1119
4
11 6
1
11 1
10
![Page 49: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/49.jpg)
Constraint Optimization 49
Outline
3/25/2013
• Motivation• Constraint Optimization and Cost Networks Section 13.1
• Branch-and-Bound Search Section 13.2
• Bucket Elimination for Optimization Section 13.3
• Mini-bucket Elimination Section 13.4
• Search with Mini-bucket Heuristics Section 13.5
• Summary
![Page 50: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/50.jpg)
Constraint Optimization 50
Summary• Constraint optimization problems consist of a constraint network and a global cost
function.
• A COP can be solved as a CSP, but search and inference methods more efficient
• Branch-and-bound search is a simple method which extends backtrack search with a bounding function
• Bucket Elimination is an inference method which can generate COP solutions in a backtrack-free manner
• Mini-bucket Elimination partitions buckets into subsets in order to reduce complexity, while still generating a bound for a COP
• BnB can be augmented with a mini-bucket based heuristic allowing inference and search methods to be combined.
3/25/2013
![Page 51: Constraint Optimization](https://reader033.vdocuments.mx/reader033/viewer/2022051118/56816030550346895dcf4c6e/html5/thumbnails/51.jpg)
Constraint Optimization 51
Thank You!
• Any Questions?
3/25/2013