constraint satisfaction problems, pt 2

1

CONSTRAINT SATISFACTION PROBLEMS, PT 2Instructor: Kris Hauserhttp://cs.indiana.edu/~hauserk

2

CONSTRAINT PROPAGATION … … is the process of determining how the

constraints and the possible values of one variable affect the possible values of other variables

It is an important form of “least-commitment” reasoning

2

3

FORWARD CHECKINGWhenever a pair (Xv) is added to assignment A do: For each variable Y not in A do:

For every constraint C relating Y to the variables in A do:

Remove all values from Y’s domain that do not satisfy C

n = number of variables d = size of initial domains s = maximum number of constraints

involving a given variable (s n-1) Forward checking takes O(nsd) time

4

FORWARD CHECKING IN MAP COLORING

WA NT Q NSW V SA TRGB RGB RGB RGB RGB RGB RGBR RGB RGB RGB RGB RGB RGBR GB G RGB RGB GB RGBR B G RB B B RGB

Empty set: the current assignment {(WA R), (Q G), (V B)}does not lead to a solution

4

5


TWA

NT

SA

Q

NSWV

Contradiction that forward checking did not detect


5

6


TWA

NT

SA

Q

NSWV

Contradiction that forward checking did not detect


Detecting this contradiction requires a more powerful constraint propagation technique

6

7

CONSTRAINT PROPAGATION FOR BINARY CONSTRAINTSREMOVE-VALUES(X,Y)1. removed false2. For every value v in the domain of Y do

– If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied thena. Remove v from Y‘s domainb. removed true

3. Return removed

8

CONSTRAINT PROPAGATION FOR BINARY CONSTRAINTSAC3 1. Initialize queue Q with all variables (not yet

instantiated)2. While Q do

a. X Remove(Q)b. For every (not yet instantiated) variable Y related to

X by a (binary) constraint do– If REMOVE-VALUES(X,Y) then

i. If Y’s domain = then exitii. Insert(Y,Q)

9

EDGE LABELINGWe consider an image of a scene composed of polyhedral objects such that each vertex is the endpoint of exactly three edges

9

10

EDGE LABELING An “edge extractor” has accurately extracted all the visible edges in the image. The problem is to label each edge as convex (+), concave (-), or occluding () such that the complete labeling is physically possible

10

11

Convex edges

Concave edgesOccludin

gedges

11

12

+- -

++

++

+

The arrow isoriented suchthat the object is on the right ofthe occluding edge

12

13

ONE POSSIBLE EDGE LABELING

+

++

+

+

+

+

+

++

--

13

14

JUNCTION TYPES

ForkL

T

Y

14

15

JUNCTION LABEL SETS

+ + --

-- - + +

++ ++

+

--

--

-+

(Waltz, 1975; Mackworth, 1977) 15

16

EDGE LABELING AS A CSP A variable is associated with each junction The domain of a variable is the label set

associated with the junction type Constraints: The values assigned to two

adjacent junctions must give the same label to the joining edge

16

17

AC3 APPLIED TO EDGE LABELINGQ = (X1, X2, X3, ...)

X1

X5

X3

X8

X12

X2

X4

17

18+ -

+-

+- -++

X1

X5

Q = (X1, ...)

18

AC3 APPLIED TO EDGE LABELING

19+ -

+-

+- -++

X1

X5

Q = (X1, ...)

19

20+ -

+-

+- -++

X5

Q = (X5, ...)

20

21

+

++

+---

-- -

+

Q = (X5, ...)

X5

X3

21

22

+

++

+---

-- -

+

Q = (X5, ...)

X5

X3

22

23

+

++

+---

-- -

+

Q = (X3, ...)

X3

23

24

++

+

+

+

-- - + +

++

Q = (X3, ...)

X3

X8

+

24

25

++

+

+

+

-- - + +

++

Q = (X3, ...)

X3

X8

+

25

26

++

+

+

+

-- - + +

++

Q = (X8, ...)

X8

+

26

27

+

+

- -++

+ + --+

X12

X8

Q = (X8, ...)

27

28

COMPLEXITY ANALYSIS OF AC3 n = number of variables d = size of initial domains s = maximum number of

constraints involving a given variable (s n-1)

Each variables is inserted in Q up to d times

REMOVE-VALUES takes O(d2) time

AC3 takes O(ndsd2) = O(nsd3) time

Usually more expensive than forward checking

AC3 1. Initialize queue Q with all variables (not

yet instantiated)2. While Q do

a. X Remove(Q)b. For every (not yet instantiated) variable Y

related to X by a (binary) constraint do– If REMOVE-VALUES(X,Y) then

i. If Y’s domain = then exitii. Insert(Y,Q)

REMOVE-VALUES(X,Y)1. removed false2. For every value v in the domain of Y do

– If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then

a. Remove v from Y‘s domainb. removed true

3. Return removed

29

IS AC3 ALL THAT WE NEED? No !! AC3 can’t detect all contradictions among

binary constraintsX

Z

YXY

XZ YZ

{1, 2}

{1, 2}{1, 2}

29

30


binary constraintsX

Z

YXY

XZ YZ

{1, 2}

{1, 2}{1, 2}

30




3. Return removed

31


binary constraintsX

Z

YXY

XZ YZ

{1, 2}

{1, 2}{1, 2}

31




3. Return removed

REMOVE-VALUES(X,Y,Z)1. removed false2. For every value w in the domain of Z do

– If there is no pair (u,v) of values in the domains of X and Y verifying the constraint on (X,Y) such that the constraints on (X,Z) and (Y,Z) are satisfied thena. Remove w from Z‘s domainb. removed true

3. Return removed

32


binary constraints

Not all constraints are binary

X

Z

YXY

XZ YZ

{1, 2}

{1, 2}{1, 2}

32

33

TRADEOFF Generalizing the constraint propagation

algorithm increases its time complexity

Tradeoff between time spent in backtracking search and time spent in constraint propagation

A good tradeoff when all or most constraints are binary is often to combine backtracking with forward checking and/or AC3 (with REMOVE-VALUES for two variables)

33

34

MODIFIED BACKTRACKING ALGORITHM WITH AC3

CSP-BACKTRACKING(A, var-domains)1. If assignment A is complete then return A2. Run AC3 and update var-domains accordingly3. If a variable has an empty domain then return

failure4. X select a variable not in A5. D select an ordering on the domain of X6. For each value v in D do

a. Add (Xv) to Ab. var-domains forward checking(var-domains, X, v, A)c. If no variable has an empty domain then

(i) result CSP-BACKTRACKING(A, var-domains)(ii) If result failure then return result

d. Remove (Xv) from A7. Return failure

35

A COMPLETE EXAMPLE:4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

1) The modified backtracking algorithm starts by calling AC3, which removes no value

35

36

4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

2) The backtracking algorithm then selects a variable and a value for this variable. No heuristic helps in this selection. X1 and the value 1 are arbitrarily selected

36

37

4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

3) The algorithm performs forward checking, which eliminates 2 values in each other variable’s domain

38

4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

4) The algorithm calls AC338

39

4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

4) The algorithm calls AC3, which eliminates 3 from the domain of X2

X2 = 3 isincompatiblewith any of the remaining valuesof X3

39




3. Return removed

40

4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

4) The algorithm calls AC3, which eliminates 3 from the domain of X2, and 2 from the domain of X3 40

41

4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

4) The algorithm calls AC3, which eliminates 3 from the domain of X2, and 2 from the domain of X3, and 4 from the domain of X3

41

42

4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

5) The domain of X3 is empty backtracking42

43

4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

6) The algorithm removes 1 from X1’s domain and assign 2 to X1 43

44

4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

7) The algorithm performs forward checking44

45

4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

8) The algorithm calls AC345

46

4-QUEENS PROBLEM

1

32

4

32 41X1

{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

8) The algorithm calls AC3, which reduces the domains of X3 and X4 to a single value

46

47

EXPLOITING THE STRUCTURE OF CSP If the constraint graph contains several

components, then solve one independent CSP per component

TWA

NT

SA

Q

NSWV

47

48

EXPLOITING THE STRUCTURE OF CSP If the constraint graph is a tree, then :1. Order the variables from the

root to the leaves (X1, X2, …, Xn)

2. For j = n, n-1, …, 2 callREMOVE-VALUES(Xj, Xi) where Xi is the parent of Xj

3. Assign any valid value to X1

4. For j = 2, …, n doAssign any value to Xj consistent with the

value assigned to its parent Xi

X

Y Z

U V

W (X, Y, Z, U, V, W)

48

49

EXPLOITING THE STRUCTURE OF CSP Whenever a variable is assigned a value by

the backtracking algorithm, propagate this value and remove the variable from the constraint graph

WA

NT

SA

Q

NSWV

49

50

EXPLOITING THE STRUCTURE OF CSP Whenever a variable is assigned a value by

the backtracking algorithm, propagate this value and remove the variable from the constraint graph

WA

NTQ

NSWV

If the graph becomes a tree, then proceed as shown in previousslide

50

51

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4

RepresentationVariables Xij for i, j in {1,..,9}Domains {1,…,9}Constraints:XijXik, for jk

Xij Xkj, for ik

XijXmn, for (i,j), (m,n) in same cell

52

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


Xij Xkj, for ik


53

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


Xij Xkj, for ik


Can we detect this using constraint propagation?

54

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


Xij Xkj, for ik


Must detect 9-way interactions

55

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4

Representation 2VariablesRi

j in {1,…,9}Ci

j in {1,…,9}Xi

j in {(1,1),…,(3,3)}

Constraints:???

R18=3

R22=8

R32=6

R37=9

…

C15=4

C12=9 C2

8=4 C38=1

C33=8

C39=5

…

56

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


j in {1,…,9}Ci

j in {1,…,9}Xi

j in {(1,1),…,(3,3)}

Constraints:???

R18=3

R22=8

R32=6

R37=9

…

C15=4

C12=9 C2

8=4 C38=1

C33=8

C39=5

…

X18 = (1,3)

57

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


j in {1,…,9}Ci

j in {1,…,9}Xi

j in {(1,1),…,(3,3)}

Constraints:???

R18=3

R22=8

R32=6

R37=9

…

C15=4

C12=9 C2

8=4 C38=1

C33=8

C39=5

…

X18 = (1,3) X2

2 = (3,3)

58

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


j in {1,…,9}Ci

j in {1,…,9}Xi

j in {(1,1),…,(3,3)}

Constraints:???

R18=3

R22=8

R32=6

R37=9

…

C15=4

C12=9 C2

8=4 C38=1

C33=8

C39=5

…

X18 = (1,3) X2

2 = (3,3) X32 = (2,3)

X37 = (3,3)

59

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


j in {1,…,9}Ci

j in {1,…,9}Xi

j in {(1,1),…,(3,3)}

Constraints:Ri

j=k Ckj=I

Similar constraints between X’s and R’s, X’s and C’sRi

j Rik for j k

Cij Ci

k for j k

Xij Xi

k for j k

R18=3

R22=8

R32=6

R37=9

…

C15=4

C12=9 C2

8=4 C38=1

C33=8

C39=5

…

X18 = (1,3) X2

2 = (3,3) X32 = (2,3)

X37 = (3,3)

60

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


j in {1,…,9}Ci

j in {1,…,9}Xi

j in {(1,1),…,(3,3)}

Constraints:Ri

j=k Ckj=I


j Rik for j k

Cij Ci

k for j k

Xij Xi

k for j k

C12 = 9

R12=

{1-9}

61

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


j in {1,…,9}Ci

j in {1,…,9}Xi

j in {(1,1),…,(3,3)}

Constraints:Ri

j=k Ckj=i


j Rik for j k

Cij Ci

k for j k

Xij Xi

k for j k

C22 = {1-9}C1

2 = 9

R12=

{2-9}

62

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


j in {1,…,9}Ci

j in {1,…,9}Xi

j in {(1,1),…,(3,3)}

Constraints:Ri

j=k Ckj=I


j Rik for j k

Cij Ci

k for j k

Xij Xi

k for j k

C12 = 9

R12=

{2-9}

X22 = (3,3)

X32 = (2,2)

63

8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


j in {1,…,9}Ci

j in {1,…,9}Xi

j in {(1,1),…,(3,3)}

Constraints:Ri

j=k Ckj=I


j Rik for j k

Cij Ci

k for j k

Xij Xi

k for j k

C12 = 9

R12=2

X22 = (3,3)

X32 = (2,2)

64

2 8

5 8

9

3

2

2

9

5

6

9

2

7

7

4


j in {1,…,9}Ci

j in {1,…,9}Xi

j in {(1,1),…,(3,3)}

Constraints:Ri

j=k Ckj=I


j Rik for j k

Cij Ci

k for j k

Xij Xi

k for j k

C12 = 9

R12=2

X22 = (3,3)

X32 = (2,2)

65

LOCAL SEARCH FOR CSPS Init: Make an arbitrary assignment Repeat: Modify some variable to reduce # of

violated constraints

66

BOOLEAN SATISFIABILITY PROBLEMS

Highly successful local search algorithms WalkSAT

See R&N 7.3

p constraints of form ui* uj* uk*= 1 where u* is either u or u

n variables ui, …, un

67

OBSERVATIONS… If a CSP has few constraints, local

search solves it quicklyRandom starting assignment not too far

from a solutionMillion-queens puzzles solved in < 1min, c.

1990 If a CSP has many constraints, local

search solves it quicklyConstraints “guide” solver to a solution (if

one exists)

68

HARD SUDOKU’S

1 . . | . . . | . . 2. 9 . | 4 . . | . 5 .. . 6 | . . . | 7 . .------+-------+------. 5 . | 9 . 3 | . . .. . . | . 7 . | . . .. . . | 8 5 . | . 4 .------+-------+------7 . . | . . . | 6 . .. 3 . | . . 9 | . 8 .. . 2 | . . . | . . 1

Human solvers: Lot of logic (deep constraint propagation)

Computer solvers: Lot of backtracking

69

HARD & EASY 3-SAT PROBLEMS Let R = # of constraints / # of

variables

As n , the fraction of hard problems reduces to 0

70

RECAP Constraint propagation, AC3 Taking advantage of CSP structure Local search for CSPs

71

NEXT CLASS Intro to uncertainty R&N 4.3-4, 13.1-2

constraint satisfaction problems, pt 2

Documents

variable y

domain of y doif

queue q

instantiatedwhile q

domain of x

ys domain

constraint c

edge labeling17q