Informed Search

Next time: Search Application

Reading: Machine Translation paper under Links

Username and password will be mailed to class

2

Questions on the HW?

3

Agenda

A* example Hill climbing

Example: n-Queens Online search

Depth first Example: maze

4

A* Search OPEN = start node; CLOSED = empty While OPEN is not empty do

Remove leftmost state from OPEN, call it X

If X = goal state, return success Put X on CLOSED

SUCCESSORS = Successor function (X) Remove any successors on OPEN or CLOSED Compute f(n)= g(n) + h(n) Put remaining successors on either end of OPEN Sort nodes on OPEN by value of heuristic function

End while

5

6

Heuristics for other problems

Problems

Shortest path from one city to another

Challenge: Is there an admissable heuristic for sodoku?

7

Romania with step costs in km

8

Local Search Algorithms

Operate using a single current state

Move only to neighbors of the state

Paths followed by search are not retained

Iterative improvement Keep a single current state and try to improve

it

9

Advantages to local search

Use very little memory – usually a constant amount

Can often find reasonable solutions in large or infinite state spaces (e.g., continuous)

Unsuitable for systematic search

Useful for pure optimatization problems Find the best state according to an objective

function Traveling salesman

10

11

12

Hill-climbing aka gradient ascent/descent; steepest ascent Function Hill-Climbing(problem)

Returns a local maximum Inputs: problem Local variables: current (a node)

neighbor (a node)

Current <- make-node(initial-state[problem]) Loop do

Neighbor <- highest valued successor of current If value(neighbor) <- value(current) return state(current) current <- neighbor

End.

13

What we think hill-climbinglooks like

What we learn hill-climbing isUsually like

14

15

Problems for hill climbing

When the higher the heuristic function the better: maxima (objective fns); when the lower the function the better: minima (cost fns)

Local maxima: A local maximum is a peak that is higher than each of its neighboring states, but lower than the global maximum

Ridges: a sequence of local maxima Plateaux: an area of the state space landscape

where the evaluation function is flat

16

Hill-climbing search: 8-queens problem

A local minimum with h = 1

17

Some solutions

Stochastic hill-climbing Chose at random from among the uphill moves

First-choice hill climbing Generates successors randomly until one is

generated that is better than current state Simulated annealing

Generate a random move. Accept if improvement. Otherwise accept with continually decreasing probability.

Local beam search Keep track of k states rather than just 1

18

Another point of view

Wrong peak has been climbed for over 22 years According to the official Nepalese peak list the highest Trekking Peak, Mera Peak -6654m- has never been climbed. Despite having been a popular climb for over 22 years, all expeditions attacked a peak that has now been revealed as the wrong one. Both new Nepalese maps and the official peak list confirm this new information. A first attempt to climb the new peak was made in May 2000, but the peak remains unclimbed. You can read more in Rock and Ice Super Guide 104 and Climber UK Sept 2000.http://www.abc.net.au/science/k2/moments/s1086384.htm

19

Avoiding climbing the wrong peak

Random-restart hill climbing Keep restarting from randomly generated

initial states, stopping when goal is found

20

ROOMBA

21

Online Search

Agent operates by interleaving computation and action

No time for thinking The agent only knows

Actions (s) The step-cost function c(s,a,s’) Goal-test (s)

Cannot access the successors of a state without trying all actions

22

Assumptions

Agent recognizes a state it has seen before

Actions are deterministic

Competitive ratio: Compare cost that agent actually travels with cost of the actual shortest path

23

What properties of search are desirable?

Will A* work?

Expand nodes in a local order Depth first Variant of greedy search

Difference from offline search: Agent must physically backtrack Record states to which agent can backtrack

and has not yet explored

24

Depth-first

OPEN = start node; CLOSED = empty While OPEN is not empty do

Remove leftmost state from OPEN, call it X

If X = goal state, return success Put X on CLOSED

SUCCESSORS = Successor function (X) Remove any successors on OPEN or CLOSED Put remaining successors on left end of OPEN

End while

25

Online DFS - setup

Inputs: s’, a percept that identifies the current state

Static: result, a table indexed by action and state,

initially empty unexplored: a table that lists, for each visited

state, the actions not yet tried unbacktracked: a table that lists, for each

visited state, the backtracks not yet tried s,a: the previous state and action, initially

null

26

Online DFS – the algorithm If Goal-test(s’) then return stop If s’ is a new state then unexplored[s’] actions(s’) If s is not null then do

result[a,s]s’, result[reverse(a),s’)-< s Add s to the front of unbacktracked[s’]

If unexplored[s’] is empty Then return stop Else a action b such that

result[b,s’]=pop(unbacktracked[s’]) Else apop(unexplored[s’]) s s’ Return a

27

Learning Real Time A* (LRTA*)

Augment hill-climbing with memory

Store current best estimate of cost from node to goal: H(s)

Initially, H(s) = h(s)

Update H(s) through experience

Estimated cost to reach the goal through neighbor s’ H(s) = c(s,a,s’)+ H(s’)

28

End of Class Questions