a general introduction to artificial intelligence

A General Introduction to A General Introduction to Artificial IntelligenceArtificial Intelligence

OutlineOutline Best-first search.Best-first search. Greedy best-first searchGreedy best-first search AA** search. search. Characteristics of HeuristicsCharacteristics of Heuristics Variations of A*Variations of A*

Memory Bounded A*.Memory Bounded A*. Online (Real Time) A*.Online (Real Time) A*.

Local search algorithmsLocal search algorithms Hill-climbing search.Hill-climbing search. Simulated annealing search.Simulated annealing search. Gradient descent search.Gradient descent search.

Review: Tree SearchReview: Tree Search

A search strategy is defined by picking A search strategy is defined by picking the the order of node expansionorder of node expansion

Best-First SearchBest-First Search

Idea: use an Idea: use an evaluation functionevaluation function f(n) f(n) for each for each nodenode estimate of "desirability"estimate of "desirability" Expand most desirable unexpanded node.Expand most desirable unexpanded node.

ImplementationImplementation::Order the nodes in fringe in decreasing order of Order the nodes in fringe in decreasing order of desirability.desirability.

Special cases:Special cases: greedy best-first searchgreedy best-first search AA** search. search.

Best-First SearchBest-First Search

Romania with step costs in Romania with step costs in kmkm

Greedy Best-First SearchGreedy Best-First Search

Greedy Best-First SearchGreedy Best-First Search

Evaluation function Evaluation function f(n) = h(n) f(n) = h(n) ((hheuristic)euristic)

= estimate of cost from = estimate of cost from nn to to goal.goal. e.g., e.g., hhSLDSLD(n)(n) = straight-line distance = straight-line distance

from from nn to Bucharest. to Bucharest. Greedy best-first search expands the Greedy best-first search expands the

node that node that appearsappears to be closest to to be closest to goal.goal.

Greedy Best-First Search Greedy Best-First Search exampleexample

Greedy Best-First Search Greedy Best-First Search exampleexample

Greedy Best-First Search Greedy Best-First Search exampleexample

Greedy Best-First Search Greedy Best-First Search exampleexample

Properties of Greedy BFSProperties of Greedy BFS

Complete?Complete? No – can get stuck in No – can get stuck in loops, e.g., Iasi loops, e.g., Iasi Neamt Neamt Iasi Iasi Neamt Neamt

Time?Time? O(bO(bmm)), but a good heuristic can , but a good heuristic can give dramatic improvement.give dramatic improvement.

Space?Space? O(bO(bmm) ) -- keeps all nodes in -- keeps all nodes in memory.memory.

Optimal?Optimal? No. No.

AA** Search Search

Idea: avoid expanding paths that are Idea: avoid expanding paths that are already expensive.already expensive.

Evaluation function Evaluation function f(n) = g(n) + h(n)f(n) = g(n) + h(n).. g(n) g(n) = cost so far to reach = cost so far to reach nn h(n)h(n) = estimated cost from = estimated cost from nn to goal to goal f(n) f(n) = estimated total cost of path = estimated total cost of path

through through nn to goal. to goal.

AA** Search Search

AA** Search Example Search Example

AA** Search Example Search Example

AA** Search Example Search Example

AA** Search Example Search Example

AA** Search Example Search Example

AA** Search Example Search Example

Admissible HeuristicsAdmissible Heuristics

A heuristic A heuristic h(n)h(n) is is admissibleadmissible if for every if for every node node nn,,

h(n) h(n) ≤≤ h h**(n), (n), where where hh**(n)(n) is the is the true true cost to cost to reach the goal state from reach the goal state from nn..

An admissible heuristic An admissible heuristic never overestimatesnever overestimates the cost to reach the goal, i.e., it is the cost to reach the goal, i.e., it is optimisticoptimistic

Example: Example: hhSLDSLD(n) (n) (never overestimates the (never overestimates the actual road distance)actual road distance)

TheoremTheorem: If : If h(n) h(n) is admissible, Ais admissible, A** using using TREE-SEARCHTREE-SEARCH is optimal. is optimal.

Optimality of AOptimality of A** (proof) (proof) Suppose some suboptimal goal Suppose some suboptimal goal GG22 has been generated and is in has been generated and is in

the fringe. Let the fringe. Let nn be an unexpanded node in the fringe such that be an unexpanded node in the fringe such that n n is on a shortest path to an optimal goal is on a shortest path to an optimal goal GG..

f(Gf(G22) = g(G) = g(G22)) since since hh(G(G22) = 0 ) = 0 g(Gg(G22) > g(G) ) > g(G) since Gsince G22 is suboptimal is suboptimal f(G) = g(G)f(G) = g(G) since since hh(G) = 0 (G) = 0 f(Gf(G22) > f(G)) > f(G) from above from above

Optimality of AOptimality of A** (proof) (proof) Suppose some suboptimal goal Suppose some suboptimal goal GG22 has been generated and is has been generated and is

in the fringe. Let in the fringe. Let nn be an unexpanded node in the fringe such be an unexpanded node in the fringe such that that n n is on a shortest path to an optimal goal is on a shortest path to an optimal goal GG..

f(Gf(G22)) > f(G) > f(G) from above from above h(n)h(n) ≤≤ h^*(n) h^*(n) since h is admissiblesince h is admissible g(n) + h(n)g(n) + h(n) ≤≤ g(n) + h g(n) + h**(n) (n) f(n) f(n) ≤≤ f(G) f(G)

Hence Hence f(Gf(G22) > f(n)) > f(n), and A, and A** will never select G will never select G22 for expansion. for expansion.

Consistent HeuristicsConsistent Heuristics A heuristic is A heuristic is consistentconsistent if for every node if for every node nn, every successor , every successor n'n'

of of nn generated by any action generated by any action aa, ,

h(n) h(n) ≤≤ c(n,a,n') + h(n') c(n,a,n') + h(n')

If If hh is consistent, we have is consistent, we havef(n') f(n') = g(n') + h(n') = g(n') + h(n') = g(n) + c(n,a,n') + h(n') = g(n) + c(n,a,n') + h(n') ≥ ≥ g(n) + h(n) g(n) + h(n) = f(n)= f(n) i.e., i.e., f(n)f(n) is non-decreasing along any path. is non-decreasing along any path. TheoremTheorem: If : If h(n)h(n) is consistent, A is consistent, A** using using GRAPH-SEARCHGRAPH-SEARCH is is

optimaloptimal

Optimality of AOptimality of A**

AA** expands nodes in order of increasing expands nodes in order of increasing ff value value

Gradually adds "Gradually adds "ff-contours" of nodes -contours" of nodes Contour Contour ii has all nodes with has all nodes with f=ff=fii, where , where ffii < f < fi+1i+1

Properties of A*Properties of A*

Complete?Complete? Yes (unless there are Yes (unless there are infinitely many nodes with f infinitely many nodes with f ≤≤ f(G) f(G) ))

Time?Time? Exponential Exponential Space?Space? Keeps all nodes in memory Keeps all nodes in memory Optimal?Optimal? Yes Yes

Admissible HeuristicsAdmissible Heuristics

E.g., for the 8-puzzle:E.g., for the 8-puzzle: hh11(n) (n) = number of misplaced tiles= number of misplaced tiles hh22(n) (n) = total Manhattan distance= total Manhattan distance(i.e., no. of squares from desired location of each tile)(i.e., no. of squares from desired location of each tile)

hh11(S) = ? (S) = ? hh22(S) = ?(S) = ?

Admissible HeuristicsAdmissible Heuristics

E.g., for the 8-puzzle:E.g., for the 8-puzzle: hh11(n) (n) = number of misplaced tiles= number of misplaced tiles hh22(n) (n) = total Manhattan distance= total Manhattan distance(i.e., no. of squares from desired location of each tile)(i.e., no. of squares from desired location of each tile)

hh11(S) = ?(S) = ? 8 8 hh22(S) = ?(S) = ? 3+1+2+2+2+3+3+2 = 18 3+1+2+2+2+3+3+2 = 18

DominanceDominance If If hh22(n) (n) ≥≥ h h11(n)(n) for all for all nn (both admissible) (both admissible) then then hh22 dominatesdominates hh11 hh22 is better for searchis better for search

Typical search costs (average number of nodes Typical search costs (average number of nodes expanded):expanded):

d=12d=12 IDS = 3,644,035 nodesIDS = 3,644,035 nodesAA**(h(h11) = 227 nodes ) = 227 nodes AA**(h(h22) = 73 nodes ) = 73 nodes

d=24 d=24 IDS = too many nodesIDS = too many nodesAA**(h(h11) = 39,135 nodes ) = 39,135 nodes AA**(h(h22) = 1,641 nodes ) = 1,641 nodes

Relaxed ProblemsRelaxed Problems

A problem with fewer restrictions on the A problem with fewer restrictions on the actions is called a actions is called a relaxed problemrelaxed problem

The cost of an optimal solution to a The cost of an optimal solution to a relaxed problem is an admissible heuristic relaxed problem is an admissible heuristic for the original problemfor the original problem

If the rules of the 8-puzzle are relaxed so If the rules of the 8-puzzle are relaxed so that a tile can move that a tile can move anywhereanywhere, then , then hh11(n) (n) gives the shortest solutiongives the shortest solution

If the rules are relaxed so that a tile can If the rules are relaxed so that a tile can move to move to any adjacent square,any adjacent square, then then hh22(n) (n) gives the shortest solutiongives the shortest solution

Variations of A*Variations of A*

Limitation of A* is memory consuming.Limitation of A* is memory consuming. The main idea is to mimic depth-first search, The main idea is to mimic depth-first search,

iterative deepening search (using linear iterative deepening search (using linear space and branch and bound technique).space and branch and bound technique).

Iterative Deepening A*Iterative Deepening A* (IDA): use f=g+h as (IDA): use f=g+h as the cutoff value. More recent improvements: the cutoff value. More recent improvements: Recursive Best First SearchRecursive Best First Search (RBFS) and (RBFS) and Memory-Bounded A* (MA*), Simplified Memory-Bounded A* (MA*), Simplified Memory Bounded A*. Memory Bounded A*.

Iterated Deepening A*Iterated Deepening A*

Recursive Best First SearchRecursive Best First Search

Recursive Best First SearchRecursive Best First Search

Recursive Best First SearchRecursive Best First Search

Recursive Best First SearchRecursive Best First Search

Recursive Best First SearchRecursive Best First Search

Online Search AlgorithmsOnline Search Algorithms In many problem the requirement for search is In many problem the requirement for search is

online and real time. For such problems, it can only online and real time. For such problems, it can only be solved by agent action executions.be solved by agent action executions.

A online search agent just know the following:A online search agent just know the following: Actions(s) - a list of possible action at node s.Actions(s) - a list of possible action at node s. C(s,a,s') - Step cost function. Cannot be used C(s,a,s') - Step cost function. Cannot be used

until agent know that s' is an outcome.until agent know that s' is an outcome. Goal-Test(s) - to test is s is a goalGoal-Test(s) - to test is s is a goal

Performance measure: Performance measure: Compression RatioCompression Ratio -the ratio -the ratio of cost of actual path taken by agent and the of cost of actual path taken by agent and the optimal path if the agent know the search space optimal path if the agent know the search space beforehandbeforehand

Online Search AlgorithmsOnline Search Algorithms

Online Search AlgorithmsOnline Search Algorithms

Local Search AlgorithmsLocal Search Algorithms

In many optimization problems, the In many optimization problems, the pathpath to to the goal is irrelevant; the goal state itself is the goal is irrelevant; the goal state itself is the solutionthe solution

State space = set of "complete" State space = set of "complete" configurationsconfigurations

Find configuration satisfying constraints, Find configuration satisfying constraints, e.g., n-queense.g., n-queens

In such cases, we can use In such cases, we can use local search local search algorithmsalgorithms

keep a single "current" state, try to improve keep a single "current" state, try to improve itit

Example: Example: nn-queens-queens

Put Put nn queens on an queens on an n n × × nn board with board with no two queens on the same row, no two queens on the same row, column, or diagonal.column, or diagonal.

Hill-Climbing SearchHill-Climbing Search

Hill-Climbing SearchHill-Climbing Search Problem: depending on initial state, Problem: depending on initial state,

can get stuck in local maxima.can get stuck in local maxima.

Example: 8-queens problemExample: 8-queens problem

hh = number of pairs of queens that are attacking = number of pairs of queens that are attacking each other, either directly or indirectly each other, either directly or indirectly

h = 17h = 17 for the above state for the above state

Example: 8-queens problemExample: 8-queens problem

A local minimum with A local minimum with h = 1h = 1

Local Search in Continuous Local Search in Continuous Domains - Gradient DescentDomains - Gradient Descent

xk 1+ xk kpk+=

x k xk 1+ x k– kpk= =

pk - search direction

k - search step size (learning factor)

Or

xk

x k 1+

kpk

Gradient Descent AlgorithmGradient Descent Algorithm

F x k 1+ F xk Choose the next step so that:

F xk 1+ F xk x k+ F xk gkT x k+=

With small step size we can approximate F(x):

g k F x x xk=

Where

g kTx k kg k

Tpk 0=

To make the function value decrease:

pk g– k=

Steepest Descent Direction:

x k 1+ xk kg k–=

ExampleExampleF x x1

22 x1x 2

2x22x1+ + +=

x00.5

0.5=

F x x1F x

x2F x

2x1 2x2 1+ +

2x1 4x2+= = g0 F x

x x0=

3

3= =

0.1=

x1 x0 g0– 0.5

0.50.1 3

3– 0.2

0.2= = =

x2 x1 g1– 0.2

0.20.1 1.8

1.2– 0.02

0.08= = =

Convergence of AlgorithmConvergence of Algorithm

-2 -1 0 1 2-2

-1

0

1

2

Simulated Annealing Simulated Annealing SearchSearch

Idea: escape local maxima by allowing Idea: escape local maxima by allowing some "bad" moves but some "bad" moves but gradually decreasegradually decrease their frequencytheir frequency

Properties of SA SearchProperties of SA Search

One can prove: If One can prove: If TT decreases slowly decreases slowly enough, then simulated annealing search enough, then simulated annealing search will find a global optimum with probability will find a global optimum with probability approaching 1.approaching 1.

Widely used in VLSI layout, airline Widely used in VLSI layout, airline scheduling, etc.scheduling, etc.

Local Beam SearchLocal Beam Search

Keep track of Keep track of kk states rather than just one states rather than just one

Start with Start with kk randomly generated states randomly generated states

At each iteration, all the successors of all At each iteration, all the successors of all kk states are generatedstates are generated

If any one is a goal state, stop; else select If any one is a goal state, stop; else select the the kk best successors from the complete best successors from the complete list and repeat.list and repeat.

Further ReadingFurther Reading

Text book : chapter 4.Text book : chapter 4. J. Pearl, J. Pearl, HeuristicsHeuristics, Addison-Wesley , Addison-Wesley

Publishing, 1984.Publishing, 1984. E. Aarts and J. Korst, E. Aarts and J. Korst, Simulated Annealing Simulated Annealing

and Boltzman Machinesand Boltzman Machines, Addison-Wesley , Addison-Wesley Publishing, 1987.Publishing, 1987.

E. Aarts and J. Lenstra, E. Aarts and J. Lenstra, Local Search in Local Search in Combinatorial OptimizationCombinatorial Optimization, Princeton , Princeton University Press, 2003. University Press, 2003.