1 state-space search 159.741 state-space search motivation and background 159741 uninformed search,...

1

159.741 STATE-SPACE SEARCH STATE-SPACE SEARCH

Motivation and BackgroundMotivation and Background

159741159741

Uninformed Search, Informed Search, Any-Path Search, Optimal Search

Source of contents: MIT OpenCourseWare

2

SEARCHSEARCHBackground and MotivationBackground and Motivation

General Idea: SearchSearch allows exploring alternatives. allows exploring alternatives.

44

• Background• Uninformed vs. Informed Searches• Any Path vs. Optimal Path• Implementation and Performance

Topics for Discussion:

These algorithms provide the conceptual backbone of almost every approach to the systematic exploration of alternatives.

3


Consider the maze shown in the figure below. S is the initial state and G is the goal state. Dark squares are blocked cells. All squares, except for the start and goal states, are labeled, with row and column numbers. For example, n54 is the cell in the fifth row, fourth column. ..

44

Sequence of exploration of moves: N, E, S, W. Diagonal moves are not permitted, nor are moves into or across a black square

4


Trees and GraphsTrees and Graphs

44

A

B

C

B is parent of CC is a child of BA is an ancestor of CC is descendant of A

rootNode

(vertex)Link

(edge)

Terminal (leaf)

No loops are allowed!

5


Directed Graphs Directed Graphs (one-way streets)(one-way streets)

44

Undirected Graphs Undirected Graphs (two-way streets)(two-way streets)

Loops are allowed, node can have multiple parents

6

SEARCHSEARCHExamples of GraphsExamples of Graphs

Planning actions (graph of possible states of the world)Planning actions (graph of possible states of the world)

44

A B C

A B

C

A B

C

A

B

C

A

B

C

Put C on A

Put C on B

Put B on C

Put A on C

Here, the nodes denote descriptions of the state of the world

Path = “plan of actions”

7

SEARCHSEARCHGraph Search as Tree SearchGraph Search as Tree Search

• Trees are directed graphs without cycles, Trees are directed graphs without cycles, and with nodes having <= 1 parentand with nodes having <= 1 parent

44

1. Replacing undirected links by 2 directed links2. Avoiding loops in path (or better yet, keeping

track of visited nodes globally)

• We can turn graph search problems graph search problems into tree search problems tree search problems by:

8

SEARCHSEARCHGraph Search as Tree SearchGraph Search as Tree Search

44

C

A D G

• We can turn graph search problems graph search problems (from S to G) into tree search problems tree search problems by:

1. Replacing undirected links by 2 directed links2. Avoiding loops in path (or keeping track of visited nodes globally)

S

B

S

DD

C G C G

A B

GC

Note that nodes may still get duplicated.

9

SEARCHSEARCHTerminologiesTerminologies

44

StateState – used to refer to the vertices of the underlying graph that is being searched; that is, states in the problem domain(e.g. a city, an arrangement of blocks or the arrangement of parts in a puzzle)

Search NodeSearch Node – refers to the vertices of the search tree which is being generated by the search algorithm. Each node refers to a state of the world; Many nodes may refer to the same state. Importantly, a node implicitly represents a path (from the start state of the search to the state associated with the node). Because search nodes are part of a search tree, they have a unique ancestor node (except for the root node).

10

159.302159.302 STATE-SPACE SEARCHSTATE-SPACE SEARCH

Any-Path SearchAny-Path Search

33

Depth-First Search, Breadth-First Search , Best-First Search

11

SEARCHSEARCHClasses of SearchClasses of Search

33

Class Name Operation

Any Path UninformedDepth-First Systematic exploration of the

whole tree until a goal is foundBreadth-First

Any Path Informed Best-First

Uses heuristic measure of goodness of a state

(e.g. estimated distance to goal)

Optimal UninformedUniform-Cost Uses path-length measure.

Finds “shortest” path.

Optimal InformedA*

Uses path “length” measure and heuristic. Finds “shortest” path.

12

SEARCHSEARCHSimple Search AlgorithmSimple Search Algorithm

A Search Node is a path from some state X to the start state A Search Node is a path from some state X to the start state e.g. (X B A S)e.g. (X B A S)

33

1. Initialise Q with search node (S) as only entry; set Visited = (S).

The state of a search node is the most recent state of the pathThe state of a search node is the most recent state of the path e.g. Xe.g. X

Let Q be a list of search nodesLet Q be a list of search nodes e.g. (X B A S) (C B A S) …)e.g. (X B A S) (C B A S) …)

Let Let SS be the start state. be the start state.2. If Q is empty, fail. Else, pick some search node N from Q.

3. If state (N) is a goal, return N (we’ve reached the goal).

4. (Otherwise) Remove N from Q.

5. Find all the descendants of state (N) not in Visited and create all the one-step extensions of N to each descendant.

6. Add the extended paths to Q; add children of state (N) to Visited.

7. Go to Step 2.

13



44




Let Let SS be the start state. be the start state.2. If Q is empty, fail. Else, pick some search node N from Q.





7. Go to Step 2.

Critical Decisions:

Step 2: picking N from Q.Step 6: adding extensions of N

to Q.

Guarantees that each state is reached only once.

14

SEARCHSEARCHImplementing the Search StrategiesImplementing the Search Strategies

44

1. Pick first element of Q.

Depth-First

2. Add path extensions to front of Q.

15


44


Breadth-First

2. Add path extensions to end of Q.


Depth-First


16


2. If Q is empty, fail. Else, pick some search node N from Q.





7. Go to Step 2.

This algorithm stops (in Step 3) when state (N) = G, or, in general, when state (N) satisfies the goal test.

We could have performed this test in Step 6 as each extended path is added to Q. This would catch termination earlier and be perfectly correct for the searches we have covered so far.

However, performing the test in Step 6 will be incorrect for the optimal searches. We have chosen to leave the test in Step 3 to maintain uniformity with these future searches.

SEARCHSEARCHTesting for the goalTesting for the goal

17

SEARCHSEARCHTerminologyTerminology

44

Visited – a state M is first visited when a path to M first gets added to Q.In general, a state is said to have been visited if it has ever shown up in a search node in Q. The intuition is that we have briefly “visited” “visited” them to place them on Q, but we have not yet examined them carefully.

18

SEARCHSEARCHTerminologyTerminology

44

Visited – a state M is first visited when a path to M first gets added to Q.In general, a state is said to have been visited if it has ever shown up in a search node in Q. The intuition is that we have briefly “visited” them to place them on Q, but we have not yet examined them carefully.

Expanded – a state M is expanded when it is the state of a search node that is pulled-off of Q. At that point, the descendants of M are visited and the path that led to M is extended to the eligible descendants. In principle, a state may be expanded multiple times. We sometimes refer to the search node that led to M (instead of M itself) as being expanded. However, once a node is expanded we are done with it; we will not need to expand it again. In fact, we discard it from Q.

19

SEARCHSEARCHVisited StatesVisited States

44

Keeping track of visited states generally improves time efficiency when searching graphs, without affecting correctness. Note, however, that substantial additional space may be required to keep track of visited states.

If all we want to do is find a path from the start to the goal, there is no advantage to adding a search node whose state is already the state of another search node.

Any state reachable from the node the second time would have been reachable from that node the first time.

Note that, when using Visited, each state will only ever have at most one path to it (search node) in Q.

We’ll have to revisit this issue when we look at optimal searching

20


44

Although we speak of a Visited list, this is never the preferred implementation

If the graph states are known ahead of time as an explicit set, then space is allocated in the state itself to keep a mark; which makes both adding to Visited and checking if a state is Visited a constant time operation

Alternatively, as is more common in AI, if the states are generated on the fly, then a hash table hash table may be used for efficient detection of previously visited states.

Note that, in any case, the incremental space cost of a Visited list will be proportional to the number of states – which can be very high in some problems.

21

SEARCHSEARCHHash functionHash function

Keys Hash function Hash Values

(GFCAS)

(GJIHDBS)

(GKCS)

(CAS)

00

01

02

03

.

.

.

.

.

.

Search Nodes

Fixed length

A hash function could be used to map the large data sets of variable length, called keyskeys to hash values that are of a fixed length. The hash values may serve as indexes to an associative array.

However, hash values may have duplicates; therefore, may cause collision between keys.

22


44

Heuristic – the word generally refers to a “rule of thumb”, something that may be helpful in some cases but not always. Generally held to be in contrast to “guaranteed” or “optimal”

Heuristic function – in search terms, a function that computes a value for a state (but does not depend on any path to that state) that may be helpful in guiding the search.

Estimated distance to goal – this type of heuristic function depends on the state of the goal. The classic example is straight-line distance used as an estimate for actual distance in a road network. This type of information can help increase the efficiency of a search.

23


44


Breadth-First

2. Add path extensions to end of Q.


Depth-First (backtracking search)


1. Pick best (measured heuristic value of state) element of Q.

Best-First (greedy search)

2. Add path extensions anywhere in Q (it may be efficient to keep the Q ordered in some way so as to make it easier to find the “best” element.

Heuristic functions are applied in the hope of completing the

search quicker or finding a relatively good goal state. It does not guarantee finding the

“best” path though.

24

SEARCHSEARCHImplementation Issues: Finding the best nodeImplementation Issues: Finding the best node

44

There are many possible approaches to findingfinding the best node in QQ.

• Scanning Q to find lowest value• Sorting Q and picking the first element• Keeping the Q sorted by doing “sorted” insertions• Keeping Q as a priority queuepriority queue

Which of these is best will depend among other things on how many children the nodes have on average. We will look at this in more detail later.

http://www.cplusplus.com/reference/stl/priority_queue/

25

Hand-simulationHand-simulation

Let’s look at some simulation (hand-computations) of the algorithms first before we characterize them more fully.

Go to Slide 49, Simulations of the AlgorithmsSlide 49, Simulations of the Algorithms

26

SEARCHSEARCHWorst Case Running TimeWorst Case Running Time

44

The number of states in the search space may be exponential in some “depthdepth” parameter,e.g. number of actions in a plan, number of moves in a game.

Tree represents the search spaced is depthb is branching factorNumber of States in the tree = bd < (bd+1 – 1)/(b-1) < bd+1

d=0

d=1

d=2

d=3

b=2

Max Time is proportional to the Max. number of Nodes visited.

27

SEARCHSEARCHWorst Case Running TimeWorst Case Running Time

44

In the worst case, all the searches, with or without visited list may have to visit each state at least once.

d is depthb is branching factorNumber of States in the tree = bd < (bd+1 – 1)/(b-1) < bd+1

d=0

d=1

d=2

d=3

b=2

Max Time is proportional to the Max. number of Nodes visited.

So, all searches will have worst case running times that are at least proportional to the total number of states and therefore exponential in the “depth” parameter.

28

SEARCHSEARCHWorst Case SpaceWorst Case Space

44

Depth-first Search: maximum Q size (always less than bd): (b-1)d ≈ bd

d=0

d=1

d=2

d=3

b=2

Dominant Factor: Max Q SizeMax Q Size = Max (#visited – #expanded).

visited

expanded

The QQ holds the number of unexpanded “siblings” of the nodes along the path that we are currently considering.The space requirements are linear linear in d.

29

SEARCHSEARCHWorst Case SpaceWorst Case Space

44

Depth-first maximum Q size: (b-1)d ≈ bd

d=0

d=1

d=2

d=3

Max Q Size = Max (#visited – #expanded).

visited

expanded

b=2

Breadth-first max. Q size: bd

30

SEARCHSEARCHCost and Performance of Any-Path MethodsCost and Performance of Any-Path Methods

44

Search Method

Worst Time Worst Space

Fewest states?

Guaranteed to find a path?

Depth-First bd+1 bd NoYes*

Breadth-Firstbd+1 bd Yes

Yes

Best-First bbd+1 d+1 **** bd No Yes*

Searching a tree with branching factor b and depth d (without using a Visited Listwithout using a Visited List)

*If there are no indefinitely long paths in the search space**Best-First needs more time to locate the best node in Q.

Worst case time is proportional to the number of nodes added to Q.Worst case space is proportional to maximal length of Q.

Considering that all states are unique

31

SEARCHSEARCHCost and Performance of Any-Path MethodsCost and Performance of Any-Path Methods

44

Search Method

Worst Time Worst Space

Fewest states?

Guaranteed to find a path?

Depth-First bd+1 bd bd+1 NoYes*

Breadth-Firstbd+1 bd bd+1 Yes

Yes

Best-First bd+1 ** bd bd+1 No Yes*

Searching a tree with branching factor b and depth d (with Visited Listwith Visited List)

*If there are no indefinitely long paths in the search space**Best-First needs more time to locate the best node in Q.

Worst case time is proportional to the number of nodes added to Q.Worst case space is proportional to maximal length of Q and Visited and Visited listlist..

Considering that all states are unique

32

SEARCHSEARCHStates vs. PathStates vs. Path

44

d=0

d=1

d=2

d=3

b=2

Using a Visited list helps prevent loops; that is, no path visits a state more than once. However, using the Visited list for very large spaces may not be appropriate as the space requirements would be prohibitive.

Using a Visited list, the worst-case time performance is limited by the number of states in the search space rather than the number of paths through the nodes in the space (which may be exponentially larger than the number of states).

Using the Visited List is a way of spending space to limit the worst case running time.

33

SEARCHSEARCHSpace (the final frontier)Space (the final frontier)

44

In large search problems, memory is often the limiting factor.

• Imagine searching a tree with branching factor 88 and depth 1010. Assume a node requires just 8 bytes of storage. Then, Breadth-First search might require up to:

(23)10 * 23 = 233 bytes = 8,000 Mbytes = 8 Gbytes

34

SEARCHSEARCHSpace (the final frontier)Space (the final frontier)

44

In large search problems, memory is often the limiting factor.

• Imagine searching a tree with branching factor 8 and depth 10. Assume a node requires just 8 bytes of storage. Then, Breadth-First search might require up to:

(23)10 * 23 = 233 bytes = 8,000 Mbytes = 8 Gbytes

One strategy is to trade time for memory. For example, we can emulate Breadth-First search by repeated applications of Depth-First search, each up to a preset depth limit. This is called Progressive Progressive Deepening Search Deepening Search (PDS)(PDS):

1. C=12. Do DFS to max. depth C. If path is found, return it.3. Otherwise, increment C and go to Step 2.

35

SEARCHSEARCHProgressive Deepening SearchProgressive Deepening Search

44

Best of Both Worlds

Depth-First Search (DFS) Depth-First Search (DFS) has small space requirements (linear in depth), but has major problems:

• DFS can run forever in search space with infinite length paths• DFS does not guarantee finding the shallowest goal

36


44

Depth-First Search (DFS) has small space requirements (linear in depth), but has major problems:


Breadth-First Search (BFS) Breadth-First Search (BFS) guarantees finding the shallowest goal states, even in the presence of infinite paths, but has one great problem:

• BFS requires a great deal of space (exponential in depth)

Best of Both Worlds

37


44

Best of Both Worlds

Depth-First Search (DFS) has small space requirements (linear in depth), but has major problems:


Breadth-First Search (BFS) guarantees finding the shallowest goal states, even in the presence of infinite paths, but has one great problem:

• BFS requires a great deal of space (exponential in depth)

Progressive Deepening SearchProgressive Deepening Search (PDS) (PDS) has the advantages of both DFS and BFS.

• PDS has small space requirements (linear in depth)• PDS guarantees finding the shallowest goal

38


44

Best of Both Worlds

Isn’t Progressive Deepening SearchProgressive Deepening Search (PDS) (PDS) too expensive?

• PDS looks at the same nodes over and over again…

39


44

Best of Both Worlds

Isn’t Progressive Deepening Search (PDS) too expensive?

• In small graphs?• In exponential trees, time is dominated by deepest search

40


44

Isn’t Progressive Deepening Search (PDS) too expensive?

• In small graphs?• In exponential trees, time is dominated by deepest search

For example, if branching factor is 2, then the number of nodes at depth d (also the worst case running time at depth d) is 2d while the total number of nodes in all previous levels is (2d – 1), so the difference between looking at the whole tree versus only the deepest level is at worst a factor of 2 factor of 2 in performance.

b=2

2d-1

2d

41


44

Ratio of the work done by PDS to that done by a single DFS

Compare the ratio of average timeratio of average time spent on PDSPDS with average time spent on a single DFSDFS with the full depth tree:

(Average time for PDS)/(Average time for DFS) ≈ (b+1)/(b-1)(b+1)/(b-1)

b ratio

2 3

3 2

5 1.5

25 1.08

100 1.02

42


44

Ratio of the work done by PDS to that done by a single DFS

Compare the ratio of average time spent on PDS with average time spent on a single DFS with the full depth tree:

(Average time for PDS)/(Average time for DFS) ≈ (b+1)/(b-1)

b ratio

2 3

3 2

5 1.5

25 1.08

100 1.02

Progressive Deepening is an effective strategy for difficult searches.

43

159.302159.302 STATE-SPACE SEARCHSTATE-SPACE SEARCH

Any-Path Search ExamplesAny-Path Search Examples

55

Depth-First, Breadth-First, Best-First Search Algorithms

44

SEARCHSEARCHSimulations by handSimulations by hand

44

Using the following graph, simulate: • Depth-First with Visited List• Depth-First without Visited List• Breadth-First with Visited List• Breadth-First without Visited List• Best-First with Visited List

C

A D G

S

B

45

SEARCHSEARCHDepth-First with Visited ListDepth-First with Visited List

44

Pick firstfirst element of Q; Add path extensions to frontfront of Q.

Step Q Visited

1 (S) S

2 (AS)(BS) A, B, S

3 (CAS)(DAS)(BS) C, D, B, A, S

4 (DAS)(BS) C, D, B, A, S

5 (GDAS)(BS) G,C,D,B,A,S

Sequence of State Expansions: S-A-C-D-G

C

A D G

S

B

In DFS – nodes are pulled off the queue and inserted into the queue using a stack.

46

SEARCHSEARCH Depth-First Depth-First withoutwithout Visited List Visited List

44

Pick first element of Q; Add path extensions to front of Q.

Step Q1 (S)

2 (AS)(BS)

3 (CAS)(DAS)(BS)

4 (DAS)(BS)

5 (CDAS)(GDAS)(BS)

6 (GDAS)(BS)

Do NOT extend a path to a state if the resulting path would have a loop.

If we have visited a state along a particular path, we do not re-visit that state again in any extensions of the path

Sequence of State Expansions: S-A-C-D-C-G

C

A D G

S

B

47

SEARCHSEARCHBreadth-First with Visited ListBreadth-First with Visited List

44

Pick firstfirst element of Q; Add path extensions to ENDEND of Q.

Step Q Visited

1 (S) S

2 (AS)(BS) A, B, S

3 (BS)(CAS)(DAS) C, D, B, A, S

4 (CAS)(DAS)(GBS) G,C, D, B, A, S

5 (DAS)(GBS) G,C,D,B,A,S

6 (GBS) G,C,D,B,A,S

Sequence of State Expansions: S-A-B-C-D-G

C

A D G

S

B

48

SEARCHSEARCH Breadth-First Breadth-First withoutwithout Visited List Visited List

44

Pick firstfirst element of Q; Add path extensions to ENDEND of Q.

Step Q1 (S)

2 (AS)(BS)

3 (BS)(CAS)(DAS)

4 (CAS)(DAS)(DBS)(GBS)

5 (DAS)(DBS)(GBS)

6 (DBS)(GBS)(CDAS)(GDAS)

7 (GBS)(CDAS)(GDAS)(CDBS)(GDBS)

Sequence of State Expansions: S-A-B-C-D-D-G

C

A D G

S

B

49

SEARCHSEARCH Best-First with Visited ListBest-First with Visited List

44

Pick “best”“best” element of Q (by heuristic value); Add path extensions to anywhereanywhere in Q.

Step Q Visited1 (10 S) S

2 (2 AS)(3 BS) A, B, S

3 (1 CAS)(3 BS)(4 DAS) C, D, B, A, S

4 (3 BS)(4 DAS) C, D, B, A, S

5 (0 GBS)(4 DAS) G, C, D, B, A, S Heuristic Values:S=10, A=2, B=3, C=1, D=4, G=0

Sequence of State Expansions: S-A-C-B-G

C

A D G

S

B

For our convention in class, add the path extensions to

the frontfront of the Queue.

50

Go to Slide 31, Worst case scenariosSlide 31, Worst case scenarios

51

159.302 STATE-SPACE SEARCHSTATE-SPACE SEARCH

Optimal SearchOptimal Search

55

Optimal Uninformed Search: Uniform-Cost Search

52


33


Any Path Uninformed

Depth-First Systematic exploration of the whole tree until a goal is foundBreadth-First




Optimal Uninformed Uniform-CostUses path “length” measure.


53



33




Let Let SS be the start state. be the start state.






7. Go to Step 2.This is what we have seen previously. Critical Decisions:

Step 2: picking N from Q.Step 6: adding extensions of N to Q.

54



33










7. Go to Step 2.Do NOT use Visited Visited

ListList for OptimalOptimal Searching! Critical Decisions:

Step 2: picking N from Q.Step 6: adding extensions of N to Q.

55

SEARCHSEARCHOptimal SearchOptimal Search

33

• For the Any-Path algorithms, the Visited List would not cause us to fail to find a path when one existed, since the path to a state did not matter.

Why can’t we use a Visited List in connection with optimal searching? In

the earlier searches, the use of the Visited List guaranteed that we would not do extra work by re-visiting or re-

expanding states. It did not cause any failures then (except possibly of

intuition)

• However, the Visited List in connection with Optimal searches can cause us to miss the best path.

Why not a Visited List?

56

SEARCHSEARCHWhy not a Visited List?Why not a Visited List?

33


Clearly, the shortest path (as determined by sum of link

costs) to G is (S A D G) and an optimal search had better

find it.


A

D G

S

2

1

4

• The shortest path from S to G is (S A D G).

57

SEARCHSEARCHWhy not a Visited List?Why not a Visited List?

33



A

D G

S

2

1

4

• The shortest path from S to G is (S A D G).

• But on extending (S), A and D would be added to the Visited List and so (S A) would not be extended to (S A D)

58

SEARCHSEARCHImplementing Optimal Search StrategiesImplementing Optimal Search Strategies

33

• Pick the best (measured by path length) element of Q.

Uniform Cost

• Add path extensions anywhere in Q.



59


33

• Like Best-First Search except that it uses the “total length (cost)”“total length (cost)” of a path instead of a heuristic value for the state.

Uniform Cost

• Each link has a “length”“length” or “cost”“cost” (which is always greater than 0)

• We want “shortest”“shortest” or “least cost”“least cost” path

C

A D G

S

B

22

22 44

55

3322

5511

60


33


Uniform Cost



C

A D G

S

B

22

22 44

55

3322

5511

Total path cost:

(S A C) 4

61


33


Uniform Cost



C

A D G

S

B

22

22 44

55

3322

5511

Total path cost:

(S A C) 4(S B D G) 8

62


33


Uniform Cost



C

A D G

S

B

22

22 44

55

3322

5511

Total path cost:

(S A C) 4(S B D G) 8(S A D C) 9

63

SEARCHSEARCHUniform CostUniform Cost

44

Note: the underlined paths are chosen for extension.

Step Q1 (0 S)

2 (2 AS)(5 BS)

3 (4 CAS)(6 DAS)(5 BS)

4 (6 DAS)(5 BS)

5 (6 DBS)(10 GBS)(6 DAS)

6 (8 GDBS)(9 CDBS)(10 GBS)(6 DAS)

7 (8 GDAS)(9 CDAS)(8 GDBS)(9 CDBS) (10 GBS)

Sequence of State Expansions: S-A-C-B-D-D-G

C

A D G

S

B

22

22 44

55

3322

5511

Pick the bestbest (by path lengthby path length) element of Q; Add path extensions anywhereanywhere in Q.

The most recent path is added at the front of Qfront of Q (order is used to settle ties).

64

SEARCHSEARCHWhy not stop on first visiting a goal?Why not stop on first visiting a goal?

33

• When doing Uniform Cost, it is not correct to stop the search when the first path to a goal is generated; that is, when a node whose state is a goal is added to Q.

Uniform Cost

65


33


Uniform Cost

• We must wait until such a path is pulled off the Q and tested in Step 3. It is only at this point that we are sure it is the shortest path to a goal since there are no other shorter paths that remain unexpanded.

66


33


Uniform Cost


• This contrasts with the non-optimal searches where the choice of where to test for a goal was a matter of convenience and efficiency, not correctness.

67


33


Uniform Cost


• This contrasts with the non-optimal searches where the choice of where to test for a goal was a matter of convenience and efficiency, not correctness.

• In the previous example, a path to G was generated at Step 5, but it was a different, shorter, path at Step 7 that was accepted.

68


44

Step Q

1 (0 S)

2 (2 AS)(5 BS)

3 (4 CAS)(6 DAS)(5 BS)

4 (6 DAS)(5 BS)

5 (6 DBS)(10 GBS)(6 DAS)


7 (8 GDAS)(9 CDAS)(8 GDBS)(9 CDBS)(10 GBS)

C

A D G

S

B

22

22 44

55

3322

5511

Uniform Cost enumerates paths in order of total path cost!

S

D C

C G

D G

C G

A B22

66

55

99 88

4466 1010

99 88

69


44

Step Q

1 (0 S)

2 (2 AS)(5 BS)

3 (4 CAS)(6 DAS)(5 BS)

4 (6 DAS)(5 BS)

5 (6 DBS)(10 GBS)(6 DAS)


7 (8 GDAS)(9 CDAS)(8 GDBS)(9 CDBS) (10 GBS)

C

A D G

S

B

22

22 44

55

3322

5511

Uniform Cost enumerates paths in order of total path cost!

S

D C

C G

D G

C G

A B22

66

55

99 88

4466 1010

99 88

Sequence of State Expansions: S – A – C – B – D – D - G

The sequence of path extensions corresponds precisely to path-length order, so it is not surprising we found the shortest path.

0 2 4 5 6 6 8 0 2 4 5 6 6 8

70



55

Optimal Informed Search: A* Search

71


33


Any Path Uninformed

Depth-First Systematic exploration of the whole tree until a goal is foundBreadth-First




Optimal Uninformed Uniform-CostUses path “length” measure.


Optimal Informed A*

Uses path “length” measure and heuristic.


72

SEARCHSEARCHGoal DirectionGoal Direction

33

• UC is really trying to identify the shortest path to every state in the graph in order. It has no particular bias to finding a path to a goal early in the search.

• We can introduce such a bias by means of heuristic function h(N)h(N), which is an estimate (h)(h) of the distance from a state to the goal.

• Instead of enumerating paths in order of just length (gg), enumerate paths in terms of ff = estimated total path lengthestimated total path length = g + hg + h.

• An estimate that always underestimates the real path length to the goal is called admissibleadmissible. For example, an estimate of 00 is admissibleadmissible (but useless). Straight line distance is admissible estimate for path length in Euclidean space.

• Use of an admissible estimate guarantees that UCUC will still find the shortestshortest path.

• UCUC with an admissible estimatewith an admissible estimate is known as A* SearchA* Search.

73

SEARCHSEARCHStraight-Line EstimateStraight-Line Estimate

33

S

A

B

G

D

E

C

• Simple graph embedded in Euclidean space

• States as city locations• Length of the links are

proportional to the driving distances between cities

In Euclidean space

74


33

S

A

B

G

D

E

C

• Simple graph embedded in Euclidean space

• States as city locations• Length of the links are

proportional to the driving distances between cities

• We can use the straight-line (airline) distances (dotted red dotted red lineslines) as an underestimate of the actual driving distance between any city and the goal.

75


33

S

A

B

G

D

E

C

• Imagine that BB and GG could be separated by a big mountain or an island

• Here we can see that the straight-line estimate between BB and GG is very badvery bad.

• The actual driving distance is much longer than the straight-line underestimate.

76

SEARCHSEARCHWhy use estimate of goal distance?Why use estimate of goal distance?

33

• Order in which UC looks at states.

• AA and BB are of the same distance from start SS, so will be looked at before any longer paths.

• No “bias” towards goal.

SA B G

• Assume states are points in the Euclidean space

77

SEARCHSEARCHWhy use estimate of goal distance?Why use estimate of goal distance?

33

SA

• Order of examination using distance from SS + estimate of distance to GG.

• Note: “bias”“bias” towards the goal; • The points away from GG look worse

• Order in which UC looks at states. AA and BB are of the same distance from start SS, so will be looked at before any longer paths. No “biasbias” towards goal.

• Assume states are points in the Euclidean space

B G

78

SEARCHSEARCHA*A*

44

C

A D G

S

B

22

22 44

55

3322

5511

Step Q

1 (0 S)

2 (4 AS)(8 BS)

3 (5 CAS)(7 DAS)(8 BS)

4 (7 DAS)(8 BS)

5 (8 GDAS)(10 CDAS)(8 BS)

Sequence of State Expansions: S – A – C – D - G

Heuristic Values:A=2, B=3, C=1, D=1, S=0, G=0

• Pick bestbest (by path length path length + heuristicheuristic) element of Q, Add path extensions anywhereanywhere in Q.



79

SEARCHSEARCHNot all heuristics are admissibleNot all heuristics are admissible

44

C

A D G

S

B

22

22 44

55

3322

5511

Heuristic values: A=2, B=3, C=1, D=4, S=10, G=0

Given the link lengths in the figure, is the table of heuristic values that we used in our earlier Best-First example an admissibleadmissible heuristicheuristic?

80

SEARCHSEARCHNot all heuristics are admissibleNot all heuristics are admissible

44

C

A D G

S

B

22

22 44

55

3322

5511

Heuristic values: A=2, B=3, C=1, D=4, S=10, G=0

Given the link lengths in the figure, is the table of heuristic values that we used in our earlier Best-First example an admissibleadmissible heuristicheuristic?

No!No!

• A is ok• B is ok• C is ok• DD is too big, needs to be <= 2• S is too big, can always use 0 for start

81



55

A*A* and Expanded ListExpanded List

82

SEARCHSEARCHStates vs. PathsStates vs. Paths

44

We have ignored the issue of revisiting states in our discussion of Uniform-Cost Search and A*. We indicated that we could not use the Visited List and still preserve optimality, but can we use something else that will keep the worst-case cost of a search proportional to the number of states in a graph rather than to the number of non-looping paths?

d=0

d=1

d=2

d=3

b=2

83

SEARCHSEARCHDynamic Programming Optimality PrincipleDynamic Programming Optimality Principle

44

SS

XX

GG

Given that path length is additive, the shortest path from S to G via a state X is made up of the shortest path shortest path from from SS to to XX and the shortest path shortest path from from XX to to GG. This is the “dynamic programming optimality principle”.

and the EXPANDED LIST

84


44

SS

XX

GG

Given that path length is additive, the shortest path from S to G via a state X is made up of the shortest path from S to X and the shortest path from X to G. This is the “dynamic programming optimality principle”.


This means that we only need to keep the single best path single best path from S to any state X; If we find a new path to a state already in Q, discard the longer onediscard the longer one.

85


44



This means that we only need to keep the single best path from S to any state X; If we find a new path to a state already in Q, discard the longer one.

Note that the first time UCUC pulls a search node off of Q whose state is X, this path is the shortest path from S to X. This follows from the fact that UC expands nodes in order of actual path length.

86


44




Note that the first time UC pulls a search node off of Q whose state is X, this path is the shortest path from S to X. This follows from the fact that UC expands nodes in order of actual path length.

So, once we expand one path to state X, we don’t need to consider (extend) any other paths to X. We can keep a list of these states, call it Expanded ListExpanded List. If the state of the search node we pull off of Q is in the Expanded List, we discard the node. When we use the Expanded List this way, we call it “strictstrict”

87


44




Note that the first time UC pulls a search node off of Q whose state is X, this path is the shortest path from S to X. This follows from the fact that UC expands nodes in order of actual path length.

So, once we expand one path to state X, we don’t need to consider (extend) any other paths to X. We can keep a list of these states, call it Expanded List. If the state of the search node we pull off of Q is in the Expanded List, we discard the node. When we use the Expanded List this way, we call it “strict”

Note that UC without this is still correct, but inefficient for searching graphs.

88

SEARCHSEARCHSimple Simple OptimalOptimal Search Algorithm: Search Algorithm: Uniform CostUniform Cost


33

1. Initialise Q with search node (S) as only entry.







5. -

6. Find all the children of state(N) and create all the one-step extensions of N to each descendant.

7. Add all the extended paths to Q.

8. Go to Step 2.

89

SEARCHSEARCHSimple Simple OptimalOptimal Search Algorithm: Search Algorithm: Uniform Cost Uniform Cost + + Strict Expanded ListStrict Expanded ListA Search Node is a path from some state X to the start state A Search Node is a path from some state X to the start state e.g. (X B A S)e.g. (X B A S)

33

1. Initialise Q with search node (S) as only entry; set Expanded = ()set Expanded = ()







5. If state (N) in Expanded, go to Step 2; otherwise, add state (N) If state (N) in Expanded, go to Step 2; otherwise, add state (N) to Expanded List.to Expanded List.

6. Find all the children of state(N) (Not in Expanded)(Not in Expanded) and create all the one-step extensions of N to each descendant.

7. Add all the extended paths to Q. If descendant state already in Add all the extended paths to Q. If descendant state already in Q, keep only shorter path to state in Q.Q, keep only shorter path to state in Q.

Take note that we need to add some

precautionary measures when adding extended

paths to Q.

8. Go to Step 2.

In effect, discard that state

90

SEARCHSEARCHUniform Cost (with Strict Expanded List)Uniform Cost (with Strict Expanded List)

44

C

A D G

S

B

22

22 44

55

3322

5511

Step Q Expanded

1 (0 S)

2 (2 AS)(5 BS) S

3 (4 CAS)(6 DAS)(5 BS) S, A

4 (6 DAS)(5 BS) S, A, C

5 (6 DBS)(10 GBS)(6 DAS) S, A, C, B

6 (8 GDAS)(9 CDAS)(10 GBS) S, A, C, B, D

Sequence of State Expansions: S – A – C – B – D - G

Removed because C was expanded already

Removed because there’s a new shorter path to G

Removed because D is already in Q (our convention: take element at the front of the Q if a path leading to the same state is in Q).

91

SEARCHSEARCHA* (without Expanded List)A* (without Expanded List)

44

Let g(N)g(N) be the path cost of n, where n is a search tree node, i.e. a partial path.


Let h(N)h(N) be h(state(N)), the heuristic estimate of the remaining path length to the goal from state(N).

Let f(N)f(N) = g(N) + h(state(N)) = g(N) + h(state(N)) be the total estimated path cost of a node, i.e. the estimate of a path to a goal that starts with the path given by N.

A* A* picks the node with lowest lowest ff value value to expand.

92


44

Let g(N) be the path cost of n, where n is a search tree node, i.e. a partial path.

Let h(N) be h(state(N)), the heuristic estimate of the remaining path length to the goal from state(N).

Let f(N) = g(N) + h(state(N)) be the total estimated path cost of a node, i.e. the estimate of a path to a goal that starts with the path given by N.

A* picks the node with lowest f value to expand.

A* A* (without Expanded List) and with admissible heuristic is guaranteed to find optimal paths – those with smallest path cost.

93

SEARCHSEARCHA* and the Strict Expanded ListA* and the Strict Expanded List

44

The Strict Expanded ListStrict Expanded List (also known as a Closed ListClosed List) is commonly used in implementations of A* but, to guarantee finding optimal paths, this implementation requires a stronger conditionstronger condition for a heuristic than simply being an underestimate.

94


44

The Strict Expanded List (also known as a Closed List) is commonly used in implementations of A* but, to guarantee finding optimal paths, this implementation requires a stronger condition for a heuristic than simply being an underestimate.

Here’s a counterexample: The heuristic values listed below are all underestimates but A* using an Expanded List will not find the optimal path.

The misleading estimate at B throws the algorithm off; C is expanded before the optimal path to it is found.

A C

G

S

B

11 11

22

100100

22Heuristic Values:

A=100, B=1, C=90, S=90, G=0

S=0 in the orig. slide

95


44

Here’s a counterexample: The heuristic values listed below are all underestimates but A* using an Expanded List will not find the optimal path.

The misleading estimate at BB throws the algorithm offthrows the algorithm off; C is expanded before the optimal path to it is found.

A C

G

S

B

11 11

22

100100

22

Heuristic Values:A=100, B=1, C=90, S=90, G=0

Step Q Expanded List

1 (0 S)

2 (3 BS)(101 AS) S

3 (94 CBS) (101 AS) B, S

4 (101 AS)(104 GCBS) C, B, S

5 (104 GCBS) A, C, B, S

Underlined paths are chosen for extensionS=0 in the orig. slide

96

SEARCHSEARCHConsistencyConsistency

44

To enable implementing A* using the strict Expanded List, HH needs to satisfy the following consistency (also known as monotonicity) conditions

1. h(Si) = 00, if ni is a goal

2. h(Si) - h(Sj) <=<= c(Si, Sj), if nj a child of ni

97

SEARCHSEARCHConsistencyConsistency

44

To enable implementing A* using the strict Expanded List, H needs to satisfy the following consistency (also known as monotonicity) conditions


2. h(Si) - h(Sj) <= c(Si, Sj), if nj a child of ni

That is, the heuristic cost in moving from one entry to the next cannot decrease by more than the arc cost between the states. This is a kind of triangle inequality triangle inequality This condition is a highly desirable property of a heuristic function and often simply assumed (more on this later).

ni

nj

goal

h(Si)

h(Sj)

C(Si, Sj)

98

SEARCHSEARCHConsistency Consistency Violation!Violation!

44

A simple example of violation of consistency:


2. h(Si) - h(Sj) <= c(Si, Sj), if nj a child of ni

ni

nj

goal

h(Si)=100

h(Sj)=10

C(Si, Sj)=10

In the example, 100-10 > 10

If you believe that the goalgoal is 100 units from nnii, then moving 10 units to nnj j

should not bring you to a distance of 10 units from the goal.

99


44

Let g(N)g(N) be the path cost of n, where n is a search tree node, i.e. a partial path.

Let h(N)h(N) be h(state(N))h(state(N)), the heuristic estimate of the remaining path length to the goal from state(N).

Let f(N) = g(N) + h(state(N))f(N) = g(N) + h(state(N)) be the total estimated path cost of a node, i.e. the estimate of a path to a goal that starts with the path given by N.

A* A* picks the node with lowest f value to expand.

A* (A* (withoutwithout Expanded List) Expanded List) and with admissible heuristic with admissible heuristic is guaranteed to find optimal paths – those with smallest path cost.

This is true even if heuristic is NOT consistent.

100

SEARCHSEARCHA* (A* (withoutwithout the Strict Expanded List) the Strict Expanded List)

44

Note that the heuristic is admissible but NOT consistent.

A C

G

S

B

11 11

22

100100

22


Step Q

1 (90 S)

2 (3 BS)(101 AS)

3 (94 CBS) (101 AS)

4 (101 AS)(104 GCBS)

5 (92 CAS)(104 GCBS)

6 (102 GCAS)(104 GCBS)

Underlined paths are chosen for extension.This heuristic is not consistent but it is an underestimate and that is all that is

needed for A* without an Expanded List to guarantee optimality.

101

SEARCHSEARCHA* (with Strict Expanded List)A* (with Strict Expanded List)

44

Just like Uniform Cost Search.

When a node N is expanded, if state (N) is in Expanded List, discard N, else add state (N) to Expanded List.

If some node in Q has the same state as some descendant of N, keep only node with smaller f, which will also correspond to smaller g.

For A* (with Strict Expanded List) to be guaranteed to find the optimal path, the heuristic must be consistent.

102

SEARCHSEARCHA* (with Strict Expanded List)A* (with Strict Expanded List)

44

Note that the heuristic is admissible and consistent.

A C

G

S

B

11 11

22

100100

22


Step Q Expanded List

1 (90 S)

2 (90 BS)(101 AS) S

3 (101 AS)(104 CBS) S, B

4 (102 CAS)(104 CBS) S, B,A

5 (102 GCAS) S, B, A, C

Underlined paths are chosen for extension.

If we modify the heuristic in the example we have been considering so that it is consistent, as we have done here by increasing the value of h(B), then A* (with the Expanded List) will work.

103

SEARCHSEARCHDealing with inconsistent heuristicDealing with inconsistent heuristic

44

What can we do if we have an inconsistent heuristic but we still want optimal paths?

104


44


Modify A* so that it detects and corrects when inconsistency has led us astray.

105


44



Assume we are adding node1 to QQ and node2 is present in Expanded List

with node1.state = node2.state.

106


44



Assume we are adding node1 to Q and node2 is present in Expanded List


Strict:• Do NOT add nodenode11 to Q.

107


44



Assume we are adding node1 to Q and node2 is present in Expanded List


Strict:• Do NOT add node1 to Q.

Non-Strict Expanded List:

• If (node1.path_length < node2.path_length), then

1. Delete node2 from Expanded List2. Add node1 to Q.

108

SEARCHSEARCHWorst Case ComplexityWorst Case Complexity

44

The number of states in the search space may be exponential in some “depth” parameter, e.g. number of actions in a plan, number of moves in a game.

109


44


All the searches, with or without Visited or Expanded Lists, may have to visit (or expand) each state in the worst case.

110


44



So, all searches will have worst case complexities that are at least proportional to the number of states and therefore exponential in the “depth” parameter.

111


44



So, all searches will have worst case complexities that are at least proportional to proportional to the number of statesthe number of states and therefore exponential in the “depth” parameter.

This is the bottom-line irreducible worst case cost of systematic searches.

112


44



So, all searches will have worst case complexities that are at least proportional to the number of states and therefore exponential in the “depth” parameter.

This is the bottom-line irreducible worst case cost of systematic searches.

Without memory of what states have been visited (expanded), searches can do (much) worse than visit every state.

113


44

• A state space with NN states may give rise to a search tree that has a number of nodes that is exponential in NN, as in this example.

d=0

d=1

d=2

d=3

b=2

A state space with NN statesstates can generate a tree with 22NN nodes nodes!

114


44

• A state space with N states may give rise to a search tree that has a number of nodes that is exponential in N, as in this example.

d=0

d=1

d=2

d=3

b=2

A state space with NN statesstates can generate a tree with 22NN nodesnodes!

• Searches without a Visited (Expanded) List may, in the worst case, visit (expand) every node in the search tree.

• Searches with Strict Visited (Expanded Lists) will visit (expand) each state onlyonly once once

115

SEARCHSEARCHOptimality and Worst Case ComplexityOptimality and Worst Case Complexity

44

Algorithm Heuristic

Expanded List

Optimality Guaranteed?

Worst Case & Expansions

Uniform Uniform CostCost

None Strict Yes N

A*A* Admissible None Yes >N

A*A* Consistent Strict Yes N

A*A* Admissible Strict No N

A*A* Admissible Non-Strict Yes >N

1 state-space search 159.741 state-space search motivation and background 159741 uninformed search,...

Documents

search nodes

search node s

searchgraph search

tree search problems

tree search trees

graph search problems

shortest path

start state