comp6721 ai search

Artificial Intelligence: State Space Search

1

State Space Search

� Russell & Norvig – chap. 3 & section 4.1.1

� Many slides from: robotics.stanford.edu/~latombe/cs121/2003/home.htm

Motivation

moebius puzzle

Play the 8-puzzle online

Moebius puzzle

2

Play the 8-puzzle online

2

Rubik’s cube Tetris

Today

� State Space Representation

� State Space Search� Uninformed search

� Breadth-first and Depth-first

� Depth-limited Search

� Iterative DeepeningSearch

Reasoning

Learning

Agent

Robotics

Perception

3

� Iterative Deepening

� Informed search � Hill climbing

� Best-First

� A*

� Summary

Knowledge

rep.Planning

Natural

language... Expert

Systems

Constraint

satisfaction

Example: 8-Puzzle

1

2

3 4

5 6

7

8 1 2 3

4 5 6

7 8

4

Initial state Goal state

State: Any arrangement of 8 numbered tiles and an empty tile on a 3x3 board

(n2-1)-puzzle

2

3 4 7

8

5 6 7 8

4321

....

5

15 6 12

15

11

14

10

13

9

....8-puzzle

15-puzzle

15-Puzzle

Invented in 1874 by Noyes Palmer Chapman … but Sam Loyd claimed he invented it!

6

15-Puzzle

Sam Loyd even offered $1,000 of his own money to the first person who would solve the following problem:

43214321

7

12

14

11

15

10

13

9

5 6 7 8

4321

12

15

11

14

10

13

9

5 6 7 8

4321

?

But no one ever won the prize…

8

State Space

� Many AI problems, can be expressed in terms of going from an initial state to a goal state� Ex: to solve a puzzle, to drive from home to Concordia…

� Often, there is no direct way to find a solution to a problem

9

problem

� but we can list the possibilities and search through them

� Brute force search: � generate and search all possibilities (but inefficient)

� Heuristic search: � only try the possibilities that you think (based on your current best guess) are more likely to lead to good solutions

State Space� Problem is represented by:

1. Initial State� starting state

� ex. unsolved puzzle, being at home

2. Set of operators� actions responsible for transition between states

3. Goal test functionApplied to a state to determine if it is a goal state

10

� Applied to a state to determine if it is a goal state

� ex. solved puzzle, being at Concordia

4. Path cost function� Assigns a cost to a path to tell if a path is preferable to

another

� Search space: the set of all states that can be reached from the initial state by any sequence of action

� Search algorithm: how the search space is visited

Example: The 8-puzzle

1

2

3 4

5 6

7

8 1 2 3

4 5 6

7 8

Initial state Goal state

11

Set of operators: blank moves up, blank moves down, blank moves left, blank moves right

Goal test function:state matches the goal state

Path cost function:each movement costs 1so the path cost is the length of the path (the number of moves)

source: G. Luger (2005)

8-Puzzle: Successor Function

1

2

3 4

5 6

78

12

1

2

3 4

5 6

7

8

1

2

3 4

5

6

78

1

2

3 4

5 6

78

Search is about the exploration of alternatives

State Space Representation

� State space S� Successor function:

x∈S → SUCCESSORS(x)

S1

3 2

13

x∈S → SUCCESSORS(x) � Initial state s0� Goal states: x∈S → GOAL?(x) =T or F

� Arc cost

3

State Graph

� Each state is represented by a distinct node

� An arc (or edge)

14

� An arc (or edge) connects a node s to a node s’ if s’ ∈ SUCCESSOR(s)

� The state graph may contain more than one connected component

State Space for the 8-puzzle

15source: G. Luger (2005)

How big is the state space of the (n2-1)-puzzle?

� Nb of states:� 8-puzzle --> 9! = 362,880 states� 15-puzzle --> 16! ~ 2.09 x 1013 states24-puzzle --> 25! ~ 1025 states

16

� 24-puzzle --> 25! ~ 1025 states

� At 100 millions states/sec:� 8-puzzle --> 0.036 sec� 15-puzzle --> ~ 55 hours� 24-puzzle --> > 109 years

Searching the State Space

� It is often not feasible (or too expensive) to build a complete representation of the state

17

of the state graph

Just to make sure we’re clear…

18

14

7

5

2

63

8

Initial state

64

7

1

5

2

8

3

Goal state

Problems with Graph Search

� Several paths to same nodes� Need to select best path

according to needs of problem

� Cycles in path can prevent termination

Blind search without cycle

19

� Blind search without cycle check will never terminate (ex. see graph on the top)

� There is no cycle problems with trees


State Space as a Search Tree

20

Search tree

State Search of Travelling Salesperson

� Salesperson has � to visit 5 cities

� must return home afterwards

� Goal: find shortest path for travel� minimize cost and/or time of

travel

� Nodes represent cities

21

� Nodes represent cities

� Weighted arcs represent cost of travel

� Simplification: salesperson lives in city A and will return there

� Goal is to go from A to A with minimal cost


Search of TSP problem

22

Each arc is marked with the total weight of all paths from the start node (A) to its endpoint (A)


Today

� State Space Representation� State Space Search

� Uninformed search� Breadth-first and Depth-first� Depth-limited Search � Iterative Deepening

23


� Informed search � Hill climbing� Best-First� A*

� Summary

Uninformed VS Informed Search

� Uninformed search � We systematically explore the alternatives� aka: systematic/exhaustive/blind/brute force search� Breadth-first� Depth-first� Uniform-cost� Depth-limited searchIterative deepening search

24

Depth-limited search� Iterative deepening search� Bidirectional search � …

� Informed search (heuristic search)� We try to choose smartly� Hill climbing� Best-First� A*� …

Today



25



� Summary

Breadth-first vs Depth-first Search

� Determine order for examining states� Depth-first:

� visit successors before siblings

� Breadth-first:

26

� visit siblings before successors (ie. visit level-by-level

Data Structures� In all search strategies, you need:

� open list (aka the frontier)

� lists generated states not yet expanded

� order of states controls order of search

� closed list (aka the explored set)

� stores all the nodes that have already been visited (to avoid cycles).

27

cycles).

� ex:

Closed = [A, B, C, D, E]

Open = [K, L, F, G, H, I, J]


Data Structures

PARENT-NODE

1

2

3 4

5 6

7

8STATE

REPRESENTATION

� To trace back the entire path of the solution after the search, each node is the lists contain:

28

Depth = length of path from root to node

15 6

BOOKKEEPING

5Path-Cost

5Depth

RightAction

...

CHILDREN

Generic Search Algorithm 1. Initialize the open list with the initial node so (top node)

2. Initialize the closed list to empty

3. Repeat

a) If the open list is empty, then exit with failure.

b) Else take the first node s from the open list.

c) If s is a goal state, exit with success. Extract the path from s to s

29

from s to sod) Insert s in the closed list (s has been visited /expanded)

e) Insert the successors of s in the open list in a certain order if they are not already in the closed/open lists (to avoid cycles)

Notes:

• The order of the nodes in the open list depends on the search strategy

� DFS and BFS differ only in the way they order nodes in the open list:� DFS uses a stack:

� nodes are added on the top of the list.

DFS and BFS

•

30

� BFS uses a queue: � Nodes are added at the end of the list.

Breadth-First Search

a)

b)

c)

31

c)

d)

e)


Breadth-First Search Example

� BFS: (open is a queue)


Snapshot of BFS� Search graph at

iteration 6 of breadth-first search

� States on open and closed are highlighted

33

and closed are highlighted


BFS of the 8-puzzleShows order in which states were removed from open

34

Goal


Function Depth-First Search

a)

b)c)

35

d)

e)


Trace of Depth-First Search


Snapshot of DFS

� Search graph at iteration 6 of depth-first search

� States on open

37

� States on open and closed are highlighted


Depth-first search of the 8-puzzle with a depth bound of 5

DFS on the 8-puzzle

38

Goal


� Breadth-first:� Complete: always finds a solution if it exists

� Optimal: always finds shortest path

But:

� inefficient if branching factor B is very high

� memory requirements high -- exponential space for states required: Bn

Depth-first:

Depth-first vs. Breadth-first

39

� Depth-first:� Not complete (ex. may get stuck in an infinite branch)

� Not optimal (will not find the shortest path)

But:

� Requires less memory -- only memory for states of one path needed: B××××n

� May find the solution without examining much of the search space

� Efficient if solution path is known to be long

Depth-first vs. Breadth-first

� If you want the shortest path --> breadth first

� If memory is a problem --> depth first

40

� Many variants of breadth & depth first:� Depth-limited Search

� Iterative deepening search

� Uniform-cost search

� Bidirectional search

Today



41



� Summary

Depth-Limited Search

Compromise for DFS :� Do depth-first but with depth cutoff k (depth at which nodes are not expanded)

Three possible outcomes:

42

� Three possible outcomes:� Solution� Failure (no solution)� Cutoff (no solution within cutoff)

Today


� Uninformed search� Breath-first and Depth-first� Depth-limited Search � Iterative Deepening

43



� Summary

Compromise between BFS and DFS:

� use depth-first search, but

� with a maximum depth before going to next level

� Repeats depth first search with gradually increasing

Iterative Deepening

44

� Repeats depth first search with gradually increasing depth limits

� Requires little memory (fundamentally, it’s a depth first)

� Finds the shortest path (limited depth)

� Preferred search method when there is a large search space and the depth of the solution is unknown

Iterative Deepening: Example

45source: Russel & Norvig (2003)

Today



49



� Summary

Search of TSP problem

50

Each arc is marked with the total weight of all paths from the start node (A) to its endpoint (A)


Size of Search Space

� Complexity: (N - 1)! with N the number of cities� interesting problem size: 50 cities� simple exhaustive search becomes intractable

� Several techniques can reduce complexity� --> heuristic search

51

� --> heuristic search � ex: Nearest neighbor heuristic

� go to closest unvisited city� highly efficient since only onepath is tried

� might fail (see next example)� similar to hill climbing

Nearest Neighbour on TSP

� nearest neighbour path in bold

� this path (A, E, D, B, C, A) has a cost of 550

� but it is not the shortest path

52

� but it is not the shortest path

� the high cost of (C, A) defeated the heuristic “nearest neighbour”

� Most of the time, it is not feasible to do an exhaustive search, search space is too large

� so far, all search algorithms have been uninformed (general search)

� so need an informed/heuristic search

Informed Search (aka heuristic search)

53

� Idea: � choose "best" next node to expand� according to a selection function (heuristic)

� But: heuristic might fail

Heuristic - Heureka!� Heuristic:

� a rule of thumb, a good bet

� but has no guarantee to be correct whatsoever!

� Heuristic search:� A technique that improves the efficiency of search, possibly sacrificing on completeness

54

possibly sacrificing on completeness

� Focus on paths that seem most promising according to some function

� Need an evaluation function (heuristic function) to score a node in the search tree

� Heuristic function h(n) = an approximation to a function that gives the true evaluation of the node n

Why do we need heuristics?

1. A problem may not have an exact solution � ex: ambiguity in the problem statement or the

data

2. The computational cost of finding the

55

2. The computational cost of finding the exact solution is prohibitive

� ex: search space is too large

� ex: state space representation of all possible moves in chess = 10120

� 1075 = nb of molecules in the universe

� 1026 = nb of nanoseconds since the “big bang”

Today



� Breath-first and Depth-first


56

Depth-limited Search



� Best-First

� A*

� Summary

Example: Hill Climbing with Blocks World

� Operators:� pickUp(Block)

� putOnTable(Block)

� stack(Block1,Block2)

57

� Heuristic:� 0pt if a block is sitting where it is supposed to sit

� +1pt if a block is NOT sitting where it is supposed to sit

� so lower h(n) is better� h(initial) = 2

� h(goal) = 0

source: Rich & Knight, Artificial Intelligence, McGraw-Hill College 1991.


pickUp(A)+putOnTable(A)

h(n) = 1

h(n) = 2

58

pickUp(H)+stack(H,A)

pickUp(A)+stack(A,H) pickUp(H)+putOnTable(H)

h(n) = 2h(n) = 2h(n) = 2

h(n) = 1

Hill Climbing� General hill climbing strategy:

� as soon as you find a position that is better than the current one, select it.

� Is a local search

� Does not maintain a list of next nodes to visit (an open list)� Similar to climbing a mountain in the fog with amnesia …

always go higher than where you are now, help!

59

always go higher than where you are now,

but never go back…

� Steepest ascent hill climbing:� instead of moving to the first position that

is better than the current one

� pick the best position out of all the next possible moves

help!


pickUp(A)+putOnTable(A)

h(n) = 1

h(n) = 2

hill-climbing will stop,because all successors arehigher than parent… -->local minimum

60

pickUp(H)+stack(H,A)

pickUp(A)+stack(A,H) pickUp(H)+putOnTable(H)

h(n) = 2h(n) = 2h(n) = 2

h(n) = 1local minimum

Don’t be confused… a lower h(n) is better…

So with lower h(n), hill-climbing should really be called deep-diving ☺

Problems with Hill Climbing

� Foothills (or local maxima) � reached a local maximum, not the global maximum

� a state that is better than all its neighbors but is not better than some other states farther away.

� at a local maximum, all moves appear to make things worse.

� ex: 8-puzzle: we may need to move tiles temporarily out of goal

61

� ex: 8-puzzle: we may need to move tiles temporarily out of goal position in order to place another tile in goal position

� ex: TSP: "nearest neighbour" heuristic


Problems with Hill Climbing

� Plateau� a flat area of the search space in which the next states

have the same value.

� it is not possible to determine the best direction in which to move by making local comparisons.

can cause the algorithm to stop (or wander aimlessly).

62

� can cause the algorithm to stop (or wander aimlessly).


Some Solutions to Hill-Climbing � Random-restart hill-climbing

� random initial states are generated

� run each until it halts or makes no significant progress.

� the best result is then chosen.

63

� keep going even if the best successor has the same value ascurrent node� works well on a "shoulder"

� but could lead to infinite loop

on a plateau

source: Rich & Knight, Artificial Intelligence, McGraw-Hill College 1991. & Russel & Norvig (2003)

Today



64



� Summary

Best-First Search� problem with hill-climbing:

� one move is selected and all others are forgotten.

� solution to hill-climbing: � use "open" as a priority queue

� this is called best-first search

65

� Best-first search:� Insert nodes in open list so that the nodes are sorted in

best (ascending/descending) h(n)

� Always choose the next node to visit to be the one with the best h(n) -- regardless of where it is in the search space

Best-First: Example

Lower h(n) is better

66


Notes on Best-first

� If you have a good evaluation function,

best-first can find the solution very quickly

� The first solution may not be the best,

67

� The first solution may not be the best,

but there is a good chance of finding it quickly

� It is an exhaustive search …

� will eventually try all possible paths

Best-First Search: Example (1)

1

68

Lower h(n) is better

2 3

4

5 6


Best-First Search: Example (2)


Designing Heuristics

� Heuristic evaluation functions are highlydependent on the search domain

� In general: the more informed a heuristic is,the better the search performance

Bad heuristics lead to frequent backtracking

70

� Bad heuristics lead to frequent backtracking

Example: 8-Puzzle – Heuristic 1

� h1: Simplest heuristic� Hamming distance : count

number of tiles out of placewhen compared with goal

5 8 1 2 3

71

� h1(n) = 6� does not consider the

distance tiles have to be moved


14

7

2

63

STATE n

64

7

5

8

Goal state

Example: 8-Puzzle – Heuristic 2� h2: Better heuristic

� Manhattan distance: sum up all the distances by which tiles are out of place

14

7

5

2

63

8

64

7

1

5

2

8

3

72

� h2(n) = 2+3+0+1+3+0+3+1 = 13


7 63

STATE n

7 8

Goal state

Example: 8-Puzzle – Heuristic 3

� h3: Even Better� sum of permutation

inversions� See next slide…


h3(N) = sum of permutation inversions

14

7

5

2

63

8

STATE n

5 8 4 2 1 7 3 6

74

� For each numbered tile, count how many tiles on its right should be on its left in the goal state.

� h3(n) = n5 + n8 + n4 + n2 + n1 + n7 + n3 + n6= 4 + 6 + 3 + 1 + 0 + 2 + 0 + 0 = 16

� BTW:

� If this number is even, the puzzle is solvable.

� If this number is odd, the puzzle is not solvable.

64

7

1

5

2

8

3

Goal state

Remember this slide...

75

h3(n) = n1 + n2 + n3 + n4 + … + n13 + n14 + n15= 0 + 0 + 0 + 0 + … + 0 + 1 + 0 = 1

� the puzzle is not solvable.

� h1(n) = misplaced numbered tiles = 6

Heuristics for the 8-Puzzle

14

7

5

2

63

8

STATE n

64

7

1

5

2

8

3

Goal state

76

= 6

� h2(n) = Manhattan distance = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 = 13

� h3(n) = sum of permutation inversions= n5 + n8 + n4 + n2 + n1 + n7 + n3 + n6= 4 + 6 + 3 + 1 + 0 + 2 + 0 + 0 = 16

Algorithm A

� Heuristics might be wrong:� so search could continue down a wrong path

� Solution:� Maintain depth count, i.e., give preference to shorter paths

Modified evaluation function f:

77

� Modified evaluation function f:

f(n) = g(n) + h(n)� f(n) estimate of total cost along path through n

� g(n) actual cost of path from start to node n

� h(n) estimate of cost to reach goal from node n

Heuristic f(n) applied to 8-puzzle


State Space for 8-puzzle


State Space for 8-puzzle (con’t)


Today



82



� Summary

Evaluating Heuristics

� Admissibility:� “optimistic”

� never overestimates the cost of reaching the goal

� guarantees to find the shortest path to the goal, if it exists

� e.g.: breadth-first is admissible -- it uses f(n) = g(n) + 0

Monotonicity:

83

� Monotonicity:� “local admissibility”

� guarantees to find the shortest path to each state encountered in the search

� Informedness:� measure for the “quality” of a heuristic

� usually, the more informed, the better

Admissibility

� Evaluation function f(n) = g(n) + h(n) for node n:

� g(n) cost so far to reach n (eg. depth of n in graph)

� h(n) heuristic estimate of distance from n to goal

� f(n) estimate for total cost of solution path through n

84

� Now consider f*(n) = g*(n) + h*(n):

� g*(n) cost of shortest path from start to node n

� h*(n) actual cost of shortest path from n to goal

� f*(n) actual cost of optimal path from start to goal

through n

Algorithm A*

� Problem: f* is usually not available

� if g(n) ≥ g*(n) for all n� best-first used with such a g(n) is called “algorithm A”

� if h(n) ≤ h*(n) for all n

85

� if h(n) ≤ h*(n) for all n� i.e. h(n) never overestimates the true cost from n to a goal

� algorithm A used with such an h(n) is called A*

� an A* algorithm is admissible (guarantees to find the shortest path to the goal)

Example: 8-Puzzle

1 2 3

4 5 6

7 8

12

3

4

5

67

8

n goal

86

n goal

� h1(n) = Hamming distance = number of misplaced tiles = 6 --> admissible

� h2(n) = Manhattan distance = 13 --> admissible

Monotonicity (aka consistent)

� An admissible heuristics may temporarily reach non-goal

states along a suboptimal path

� A heuristic is monotonic if it always finds the minimal path to each state the 1st time it is encountered

� h is monotonic if for every node n and every successor n’ of n:

� h(n) ≤≤≤≤ c(n,n') + h(n')

87

� h(n) ≤≤≤≤ c(n,n') + h(n')

� i.e. f(n) is non-decreasing along any path

Informedness� Intuition: number of misplaced tiles is less informed than

Manhattan distance

� For two A* (ie. admissible) heuristics h1 and h2� if h1(n) ≤ h2(n), for all states n

� then h2 is more informed than h1� h1(n) ≤ h2(n) ≤ h*(n)

88

1 2

� More informed heuristics search smaller space to find optimal solution

� However, you need to consider the computational cost of evaluating the heuristic…

� The time spent computing heuristics must be recovered by a better search

Today



89



� Summary

SummarySearch Uses

h(n) ?Open is a

Breadth-first No Queue

Depth-first No Stack

Depth-limited No Stack

Iterative Deepening No Stack

Hill climbing Yes none

90

Hill climbing Yes none

Best-First Yes Priority queue sorted by h(n)

A* Yes Priority queue sorted by f(n) f(n) = g(n) + h(n)

Today



� Breadth-first and Depth-first



91



� Best-First

� A*

� Summary

comp6721 ai search

Documents