![Page 1: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/1.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 1
Problem Solving with ConstraintsCSCE496/896, Fall2011:
www.cse.unl.edu/~choueiry/F11-496-896
Berthe Y. Choueiry (Shu-we-ri)Avery Hall, Room [email protected]
Intelligent Backtracking Algorithms
![Page 2: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/2.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 2
Reading• Required reading
– Hybrid Algorithms for the Constraint Satisfaction Problem [Prosser, CI 93]
• Recommended reading– Chapters 5 and 6 of Dechter’s textbook– Tsang, Chapter 5
• Available upon request– Notes of Fahiem Bacchus: Chapter 2, Section 2.4
![Page 3: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/3.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 3
Outline• Review of terminology of search • Hybrid backtracking algorithms
![Page 4: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/4.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 4
Backtrack search (BT)
Var 1 v1 v2
S
• Variable/value ordering
• Variable instantiation
• (Current) path
• Current variable
• Past variables
• Future variables
• Shallow/deep levels /nodes
• Search space / search tree
• Back-checking
• Backtracking
![Page 5: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/5.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 5
Outline• Review of terminology of search • Hybrid backtracking algorithms
– Vanilla: BT– Improving back steps: {BJ, CBJ} – Improving forward step: {BM, FC}
![Page 6: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/6.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 6
Two main mechanisms in BT1. Backtracking:
• To recover from dead-ends • To go back
2. Consistency checking: • To expand consistent paths• To move forward
![Page 7: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/7.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 7
BacktrackingTo recover from dead-ends
1. Chronological (BT)2. Intelligent
• Backjumping (BJ)• Conflict directed backjumping (CBJ)• With learning algorithms (Dechter Chapt 6.4)• Etc.
![Page 8: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/8.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 8
Consistency checkingTo expand consistent paths1. Back-checking: against past variables
• Backmarking (BM)2. Look-ahead: against future variables
• Forward checking (FC) (partial look-ahead)• Directional Arc-Consistency (DAC) (partial
look-ahead)• Maintaining Arc-Consistency (MAC) (full
look-ahead)
![Page 9: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/9.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 9
Hybrid algorithmsBacktracking + checking = new hybrids
BT BJ CBJBM BMJ BM-CBJFC FC-BJ FC-CBJ
Evaluation:
• Empirical: Prosser 93. 450 instances of Zebra
• Theoretical: Kondrak & Van Beek 95
![Page 10: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/10.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 10
Notations (in Prosser’s paper)
• Variables: Vi, i in [1, n]• Domain: Di = {vi1, vi2, …,viMi}• Constraint between Vi and Vj: Ci,j
• Constraint graph: G• Arcs of G: Arc(G)• Instantiation order (static or dynamic)• Language primitives: list, push, pushnew,
remove, set-difference, union, max-list
![Page 11: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/11.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 11
Main data structures• v: a (1xn) array to store assignments
– v[i] gives the value assigned to ith variable – v[0]: pseudo variable (root of tree), backtracking to
v[0] indicates insolvability• domain[i]: a (1xn) array to store the original
domains of variables• current-domain[i]: a (1xn) array to store the
current domains of variables– Upon backtracking, current-domain[i] of future
variables must be refreshed• check(i,j): a function that checks whether the
values assigned to v[i] and v[j] are consistent
![Page 12: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/12.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 12
Generic search: bcssp1. Procedure bcssp (n, status)2. Begin3. consistent true4. status unknown5. i 16. While status = unknown
7. Do Begin8. If consistent
9. Then i label (i, consistent)10. Else i unlabel (i, consistent)
11. If i > n12. Then status “solution”13. Else If i=0 then status “impossible”14. End
15. End
• Forward move: x-label
• Backward move: x-unlabel
• Parameters: i: current variable,consistent: Boolean
• Return: i: new current variable
![Page 13: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/13.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 13
Chronological backtracking (BT)• Uses bt-label and bt-unlabel• bt-label:
– When v[i] is assigned a value from current-domain[i], we perform back-checking against past variables (check(i,k))
– If back-checking succeeds, bt-label returns i+1– If back-checking fails, we remove the assigned value from
current-domain[i], assign the next value in current-domain[i], etc.– If no other value exists, consistent nil (bt-unlabel will be called)
• bt-unlabel– Current level is set to i-1 (notation for current variable: v[h])– For all future variables j: current-domain[j] domain[j]– If domain[h] is not empty, consistent true (bt-label will be called)– Note: for all past variables g, current-domain[g] domain[g]
![Page 14: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/14.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 14
BT-label1. Function bt-label(i,consistent): INTEGER2. BEGIN3. consistent false4. For v[i] each element of current-domain[i] while not consistent
5. Do Begin6. consistent true7. For h 1 to (i-1) While consistent8. Do consistent check(i,h)9. If not consistent10. Then current-domain[i] remove(v[i], current-domain[i])11. End
12. If consistent then return(i+1) ELSE return(i)13. END
Terminates:
• consistent=true, return i+1
• consistent=false, current-domain[i]=nil, returns i
![Page 15: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/15.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 15
BT-unlabel1. FUNCTION bt-unlabel(i,consistent):INTEGER2. BEGIN
3. h i -14. current-domain[i] domain[i]5. current-domain[h] remove(v[h],current-domain[h])6. consistent current-domain[h] nil7. return(h)
8. END • Is called when consistent=false and current-domain[i]=nil
• Selects vh to backtrack to
• (Uninstantiates all variables between vh and vi)
• Uninstantiates v[h]: removes v[h] from current-domain [h]:
• Sets consistent to true if current-domain[h] 0
• Returns h
![Page 16: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/16.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 16
Example: BT (the dumbest example ever)
{1,2,3,4,5}
{1,2,3,4,5}
{1,2,3,4,5}
{1,2,3,4,5}
{1,2,3,4,5}
V2
V1
V3
V4
V5
CV3,V4={(V3=1,V4=3)}
CV2,V5={(V2=5,V5=1),(V2=5,V5=4)}
-
v[1]
v[2]
v[3]
v[4]
v[5]
v[0]
1
1
1
1
21 3 4
2 3 4 5
etc…
![Page 17: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/17.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 17
Outline• Review of terminology of search • Hybrid backtracking algorithms
– Vanilla: BT– Improving back steps: BJ, CBJ – Improving forward step: BM, FC
![Page 18: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/18.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 18
Danger of BT: thrashing• BT assumes that the instantiation of v[i]
was prevented by a bad choice at (i-1). • It tries to change the assignment of v[i-1]• When this assumption is wrong, we suffer
from thrashing (exploring ‘barren’ parts of solution space)
• Backjumping (BT) tries to avoid that– Jumps to the reason of failure – Then proceeds as BT
![Page 19: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/19.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 19
Backjumping (BJ)• Tries to reduce thrashing by saving some
backtracking effort• When v[i] is instantiated, BJ remembers
v[h], the deepest node of past variables that v[i] has checked against.
• Uses: max-check[i], global, initialized to 0• At level i, when check(i,h) succeeds
max-check[i] max(max-check[i], h)• If current-domain[h] is getting empty,
simple chronological backtracking is performed from h– BJ jumps then steps!
12
3
0
23
1
i
h-1h-1
h
h
h-2
0
0
0
Current variablePast variable
![Page 20: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/20.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 20
BJ: label/unlabel• bj-label: same as bt-label, but updates max-
check[i]• bj-unlabel, same as bt-unlabel but
– Backtracks to h = max-check[i]– Resets max-check[j] 0 for j in [h+1,i]
Important: max-check is the deepest level we checked against, could have been success or could have been failure
12
3
0
23
1
i
h-1h-1
h
h
h-2
0
0
0
![Page 21: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/21.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 21
Example: BJ
2
{1,2,3,4,5}
{1,2,3,4,5}
{1,2,3,4,5}
{1,2,3,4,5}
{1,2,3,4,5}
V2
V1
V3
V4
V5
CV2,V4={(V2=1,V4=3)}
CV1,V5={(V1=1,V5=2)}
CV2,V5={(V2=5,V5=1)}
-
v[1]
v[2]
v[3]
v[4]
v[5]
v[0] = 0
1
1
1
1
21 3 4
2 3 4 5
Max-check[1] = 0
Max-check[2] = 1
max-check[4] = 3
max-check[5] = 2
V4=1, fails for V2, mc=2V4=2, fails for V2, mc=2 V4=3, succeeds
V5=1, fails for V1, mc=1V5=2, fails for V2, mc=2V5=3, fails for V1V5=4, fails for V1V5=5, fails for V1
![Page 22: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/22.jpg)
Problem Solving with Constraints
Intelligent Backtracking AlgorithmsBacktracking 22
Conflict-directed backjumping (CBJ)
• Backjumping– jumps from v[i] to v[h], – but then, it steps back from v[h] to v[h-1]
• CBJ improves on BJ– Jumps from v[i] to v[h]– And jumps back again, across conflicts
involving both v[i] and v[h]– To maintain completeness, we jump back to
the level of deepest conflict
![Page 23: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/23.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 23
CBJ: data structure
• Maintains a conflict set: conf-set• conf-set[i] are first initialized to {0}• At any point, conf-set[i] is a subset of
past variables that are in conflict with i
{0}
{0}
{0}
{0}{0}{0}
conf-set[g]
conf-set[h]
conf-set[i]
01
2
g
h-1h
i
conf-set
![Page 24: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/24.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 24
CBJ: conflict-set
{x}
{3}
{1, g, h}
{0}{0}{0}
conf-set[g]
conf-set[h]
conf-set[i]
123g
h-1h
Current variable i
Pas
t var
iabl
es
{3,1, g}
{x, 3,1}
• When a check(i,h) failsconf-set[i] conf-set[i] {h}
• When current-domain[i] empty
1. Jumps to deepest past variable h in conf-set[i]
2. Updates conf-set[h] conf-set[h] (conf-set[i] \{h})
• Primitive form of learning (while searching)
![Page 25: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/25.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 25
Example CBJ{1,2,3,4,5}
{1,2,3,4,5}
{1,2,3,4,5}
{1,2,3,4,5}
{1,2,3,4,5}
V2
V1
V3
V4
V5
{(V1=1, V6=3)}
-
v[1]
v[2]
v[3]
v[4]
v[6]
v[0] = 0
1
1
1
1
21 3
2 3 4 5
conf-set[1] = {0}
conf-set[2] = {0}
conf-set[3] = {0}
{(V4=5, V6=3)}
{(V2=1, V4=3), (V2=4, V4=5)}
conf-set[6] = {1}
{1,2,3,4,5}V6
{(V1=1, V5=3)}
conf-set[4] = {2}
v[5] 21 3
conf-set[6] = {1}conf-set[6] = {1,4}
conf-set[6] = {1,4}conf-set[6] = {1,4}
conf-set[4] = {1, 2}
conf-set[5] = {1}
![Page 26: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/26.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms
CBJ for finding all solutions• After finding a solution, if we jump from this last
variable, then we may miss some solutions and lose completeness
• Two solutions, proposed by Chris Thiel (S08)1. Using conflict sets2. Using cbf of Kondrak, a clear pseudo-code
• Rationale by Rahul Purandare (S08)– We cannot skip any variable without chronologically
backtracking to it at least once – In fact, exactly once
26
![Page 27: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/27.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms
CBJ/All solutions without cbf• When a solution is found, force the last variable,
N, to conflict with everything before it– conf-set[N] {1, 2, ..., N-1}.
• This operation, in turn, forces some chronological backtracking as the conf-sets are propagated backward
27
![Page 28: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/28.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms
CBJ/All solutions with cbf• Kondrak proposed to fix the problem using
cbf (flag), a 1xn vector i, cbf[i] 0– When you find a solution, i, cbf[i] 1
• In unlabel – if (cbf[i]=1)
• Then h i-1; cbf[i] 0• Else h max-list (conf-set[i])
28
![Page 29: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/29.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 29
Backtracking: summary• Chronological backtracking
– Steps back to previous level– No extra data structures required
• Backjumping– Jumps to deepest checked-against variable, then
steps back– Uses array of integers: max-check[i]
• Conflict-directed backjumping– Jumps across deepest conflicting variables– Uses array of sets: conf-set[i]
![Page 30: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/30.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 30
Outline• Review of terminology of search • Hybrid backtracking algorithms
– Vanilla: BT– Improving back steps: BJ, CBJ – Improving forward step: BM, FC
![Page 31: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/31.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 31
Backmarking: goal• Tries to reduce amount of consistency
checking• Situation:
– v[i] about to be re-assigned k– v[i]k was checked against v[h]g– v[h] has not been modified
v[h] = g
v[i] kk
![Page 32: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/32.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 32
BM: motivation• Two situations
1. Either (v[i]=k,v[h]=g) has failed it will fail again2. Or, (v[i]=k,v[h]=g) was founded consistent it will remain consistent
v[h] = g
v[i] kk
v[h] = g
v[i] kk
• In either case, back-checking effort against v[h] can be saved!
![Page 33: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/33.jpg)
Problem Solving with Constraints
33
Data structures for BM: 2 arrays
0 0 0 0 0 0 0 0 0
00
0
0
Num
ber o
f var
iabl
es
n
max domain size m
Num
ber o
f var
iabl
es
n
• maximum checking level: mcl (n x m)• Minimum backup level: mbl (n x 1)
![Page 34: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/34.jpg)
Problem Solving with Constraints
34
Maximum checking level
0 0 0 0 0 0 0 0 0000
0
Num
ber o
f var
iabl
es
nmax domain size m
• mcl[i,k] stores the deepest variable that v[i]k checked against
• mcl[i,k] is a finer version of max-check[i]
![Page 35: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/35.jpg)
Problem Solving with Constraints
35
Minimum backup level
Num
ber o
f var
iabl
es
n
• mbl[i] gives the shallowest past variable whose value has changed since v[i] was the current variable
• BM (and all its hybrid) do not allow dynamic variable ordering
![Page 36: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/36.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 36
When mcl[i,k]=mbl[i]=j
v[i] kk
v[j]
mbl[i] = j
BM is aware that• The deepest variable that (v[i] k)
checked against is v[j]• Values of variables in the past of
v[j] (h<j) have not changed
So• We do need to check (v[i] k)
against the values of the variables between v[j] and v[i]
• We do not need to check (v[i] k) against the values of the variables in the past of v[j]
![Page 37: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/37.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 37
Type a savings
v[h]
v[i] kk
v[j]
mcl[i,k]=h mcl[i,k] < mbl[i]=j
When mcl[i,k] < mbl[i], do not check v[i] k because it will fail
![Page 38: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/38.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 38
Type b savings
h
v[i] kk
v[j]
v[g]
mcl[i,k]=g
mbl[i] = j
mcl[i,k]mbl[i]
When mcl[i,k] mbl[i], do not check (i,h<j) because they will succeed
![Page 39: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/39.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 39
Hybrids of BM• mcl can be used to allow backjumping in
BJ• Mixing BJ & BM yields BMJ
– avoids redundant consistency checking (types a+b savings) and
– reduces the number of nodes visited during search (by jumping)
• Mixing BM & CBJ yields BM-CBJ
![Page 40: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/40.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 40
Problem of BM and its hybrids: warning
v[m]v[m]
v[g]
v[h]
v[i]
v[g]
v[h]
v[i]
v[m]
v[g]
v[h]
v[i]
v[h]
v[f]
• Backjumping from v[i]:– v[i] backjumps up to v[g]
• Backmarking of v[h]:– When reconsidering v[h], v[h] will
be checked against all f [m,g) – effort could be saved
• Phenomenon will worsen with CBJ
• Problem fixed by Kondrak & van Beek 95
BMJ enjoys only some of the advantages of BMAssume: mbl[h] = m and max-check[i]=max(mcl[i,x])=g
![Page 41: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/41.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 41
Forward checking (FC)• Looking ahead: from current variable, consider all future
variables and clear from their domains the values that are not consistent with current partial solution
• FC makes more work at every instantiation, but will expand fewer nodes
• When FC moves forward, the values in current-domain of future variables are all compatible with past assignment, thus saving backchecking
• FC may “wipe out” the domain of a future variable (aka, domain annihilation) and thus discover conflicts early on. FC then backtracks chronologically
• Goal of FC is to fail early (avoid expanding fruitless subtrees)
![Page 42: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/42.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 42
FC: data structures
v[i]
v[k]
v[l]
v[n]
v[m]
v[j]
• When v[i] is instantiated, current-domain[j] are filtered for all j connected to i and I < j n
• reduction[j] store sets of values remove from current-domain[j] by some variable before v[j]
reductions[j] = {{a, b}, {c, d, e}, {f, g, h}}
• future-fc[i]: subset of the future variables that v[i] checks against (redundant)
future-fc[i] = {k, j, n}• past-fc[i]: past variables that checked against v[i]
• All these sets are treated like stacks
![Page 43: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/43.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 43
Forward Checking: functions• check-forward• undo-reductions• update-current-domain• fc-label• fc-unlabel
![Page 44: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/44.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 44
FC: functions• check-forward(i,j) is called when instantiating v[i]
– It performs Revise(j,i)– Returns false if current-domain[j] is empty, true
otherwise– Values removed from current-domain[j] are pushed,
as a set, into reductions[j]• These values will be popped back if we have to
backtrack over v[i] (undo-reductions)
![Page 45: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/45.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 45
FC: functions• update-current-domain
– current-domain[i] domain[i] \ reductions[i] – actually, we have to iterate over reductions, which is a
set of sets• fc-label
– Attempts to instantiate current-variable– Then filters domains of all future variables (push into
reductions)– Whenever current-domain of a future variable is wiped-
out: • v[i] is un-instantiated and • domain filtering is undone (pop reductions)
![Page 46: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/46.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 46
Hybrids of FC• FC suffers from thrashing: it is based on BT• FC-BJ:
– max-check is integrated in fc-bj-label and fc-bj-unlabel– Enjoys advantages of FC and BJ… but suffers
malady of BJ (first jumps, then steps back)• FC-CBJ:
– Best algorithm so far– fc-cbj-label and fc-cbj-unlabel
![Page 47: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/47.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 47
Consistency checking: summary• Chronological backtracking
– Uses back-checking– No extra data structures
• Backmarking– Uses mcl and mbl– Two types of consistency-checking savings
• Forward-checking– Works more at every instantiation, but expands fewer
subtrees– Uses: reductions[i], future-fc[i], past-fc[i]
![Page 48: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/48.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 48
Experiments• Empirical evaluations on Zebra
– Representative of design/scheduling problems– 25 variables, 122 binary constraints– Permutation of variable ordering yields new search
spaces– Variable ordering: different bandwidth/induced width
of graph• 450 problem instances were generated• Each algorithm was applied to each instance
Experiments were carried out under static variable ordering
![Page 49: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/49.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 49
Analysis of experimentsAlgorithms compared with respect to:1. Number of consistency checks (average)
FC-CBJ FC-BJ BM-CBJ FC CBJ BMJ BM BJ BT≼ ≼ ≼ ≼ ≼ ≼ ≼ ≼• Number of nodes visited (average)
FC-CBJ FC-BJ FC BM-CBJ BMJ=BJ BM=BT≼ ≼ ≼ ≼ ≼• CPU time (average)
FC-CBJ FC-BJ FC BM-CBJ CBJ BMJ BJ BT BM≼ ≼ ≼ ≼ ≼ ≼ ≼ ≼
FC-CBJ apparently the champion
![Page 50: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/50.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms 50
Additional developments• Other backtracking algorithms exist:
– Graph-based backjumping (GBJ), etc. [Dechter]
– Pseudo-trees [Freuder 85]
• Other look-ahead techniques exist– DAC, MAC, etc.
• More empirical evaluations– over randomly generated problems
• Theoretical comparisons [Kondrak & van Beek IJCAI’95]
![Page 51: Problem Solving with Constraints Intelligent Backtracking Algorithms 1 Problem Solving with Constraints CSCE496/896, Fall2011: choueiry/F11-496-896](https://reader035.vdocuments.mx/reader035/viewer/2022062523/5a4d1ad57f8b9ab059972b89/html5/thumbnails/51.jpg)
Problem Solving with Constraints
Intelligent Backtracking Algorithms
Implementing BT-based algorithms
• Preprocessing– Enforce NC, do not include in #CC (e.g., Zebra)– Normalize all constraints (fapp01-0200-0)– Check for empty relations (bqwh-15-106-0_ext)
• Interrupt as soon as you detect domain wipe out• Dynamic variable ordering
– Apply domino effect
51