ku nlp heuristic search30 4.1.3 heuristic search and expert systems (1) q an interesting approach to...

Heuristic Search 1

KU NLPKU NLP

4.1.3 Heuristic Search and Expert Systems (1)

An interesting approach to implementing heuristics is the use of confidence measures by expert systems to weigh the results of a rule.

Expert systems employ confidence measures to select the conclusions with the highest likelihood of success.

Games are ideal vehicles for exploring the design and behavior of heuristic search algorithm.1. Search spaces are large enough to require heuristic pruning2. Most games are complex enough to suggest a rich variety of

heuristic evaluations for comparison and analysis.3. Games generally do not involve complex representational issues.4. Because each node of the state space has a common

representation(e.g. a board description), a single heuristic may be applied throughout the search space.

Heuristic Search 2

KU NLPKU NLP

4.1.3 Heuristic Search and Expert Systems (2)

More realistic problems(such as those found in expert

systems applications, planning, intelligent control,

and machine learning) complicate the implementation

and analysis of heuristic search by requiring multiple

heuristics to deal with different situations in the

problem space. A single heuristic may not apply to each state in these domains.

Instead, situation specific problem-solving heuristics are encoded in the syntax and content of individual problem solving operators.

Heuristic Search 3

KU NLPKU NLP

Example 4.1.1 The Financial Advisor (1)

So far, the knowledge base has been treated as a set of logical implications, whose conclusions are either true or false, depending on the truth value of the premises.savings_account(adequate)income(adequate) investment(stocks)

In reality, it is possible that such an individual may prefer the added security of a combination strategy or even that of placing all investment money in savings The problem solver should try to account for this uncertainty. Expert systems have attached a numeric weight (confidence

measure or certainty factor) to the conclusion of each rule. Savings_account(adequate) income(adequate)

investment(stocks) with confidence=0.8

Heuristic Search 4

KU NLPKU NLP

Example 4.1.1 The Financial Advisor (2)

Savings_account(adequate) income(adequate) investment(combination) with confidence=0.5

Savings_account(adequate) income(adequate) investment(savings) with confidence=0.1

Heuristic search algorithms can use certainty factors in a number of ways

The results of all applicable rules could be produced with varying certainties placed on multiple conclusions.

The program might return only the result with the strongest confidence value.

Heuristic Search 5

KU NLPKU NLP

4.3 Using Heuristics in Games (1)

Games are good application area for heuristic

algorithm.

Tow-person games are more complicated than

simple puzzles. Hostility

maximize own advantage while minimize opponent’s opportunity of win.

Unpredictable opponent different knowledge of games.

Credit assignment is difficult.

Heuristic Search 6

KU NLPKU NLP

4.3 Using Heuristics in Games (2)

Various games Checker

Samuel’s program had an interesting learning component which allowed its performance to improve with experience. Ultimately, the program was able to beat its author.

Evaluate all states at a level with a evaluation polynomial. (C1*piece advantage + C2*advancement + C3*center control +

C4*fork treat + C5*mobility …)

If the evaluation polynomial led to a losing series of moves, the program adjusted its coefficients to improve performance.

Limitations No notion of global strategy, it may lead the program into local maxima. The learning component of the program is vulnerable to inconsistencies

in the opponent play. Go : Go is a very difficult game to play by machine since the

average branching factor of the game tree is very high. Backgammon : a backgammon program must choose its moves

with incomplete information about what may happen. Othello, Chess: achieved world-championship level.

Heuristic Search 7

KU NLPKU NLP

4.3.1 Minimax Procedure on Exhaustively Searchable Graphs (1)

If a game whose state space is small enough to be ex

haustively searched, then the problem is systematical

ly searching the space of possible moves and counter

moves by the opponent. A game of “nim” (Fig 4.13, p145, tp37)

To predict opponent’s behavior, we assume that our opponent us

es the same knowledge of the state space as we use.

MAX represents the player trying to win, or to maximi

ze the advantage. MIN is the opponent who attempts t

o minimize MAX’s score.

Heuristic Search 8

KU NLPKU NLP


Heuristic Search 9

KU NLPKU NLP


In implementing minimax, label each level MIN or MAX in the search space according to whose move it is at that point in the game (Fig 4.14, p146) If the parent state is a MAX node, give it the maximum value

among its children. If the parent is a MIN node, give the minimum value of its chi

ldren. The value assigned to each state indicates the value of the b

est state that this player can hope to achieve.

Heuristic Search 10

KU NLPKU NLP


Heuristic Search 11

KU NLPKU NLP

4.3.2 Minimaxing to Fixed Ply Depth

In applying MINIMAX to complicated games, the state space is searched to a predefined number of levels(N-move look-ahead) The value propagated back to the root node is simply the heuristic

value of the best state that can be reached in N moves.

Many game heuristics measure the advantage of one player over another. simple heuristic take the difference in the number of pieces

belonging to MAX and MIN, and try to maximize the difference. more sophisticated strategy might assign different values to the

pieces (e.g. queen, pawn, king, checker) or location on the board.

Game playing programs typically look ahead a fixed ply depth. The states on that ply are measured heuristically and the values are propagated back up the graph using MINIMAX.

Heuristic Search 12

KU NLPKU NLP

Minimax Search (1)

One-ply search

two-ply search

C

A

B D(8) (3) (-2)

(9)

C

A

B D

E F G H I J K

(-6) (0) (0) (-2) (-4)(-3)

Heuristic Search 13

KU NLPKU NLP

Minimax Search (2)

Two-ply search (continued)

C

A

B D

E F G H I J K

(9) (-6) (0) (0) (-2) (-4)(-3)

(-4)(-2)(-6)

(-2)Maximizing ply

Minimizing ply

Heuristic Search 14

KU NLPKU NLP

Minimax Search (3)

Heuristic Search 15

KU NLPKU NLP

Minimax Procedure (1)

When a heuristic applied with a limited look-ahead, it is possible that the depth of the look-ahead may not detect that a heuristically promising path leads to a bad situation later in the game. Horizon effect: selection of the state may cause the entire game

to be lost. Selective deepening of search : searching several plies deeper

from states that look exceptionally good.

The evaluations that take place very deep in the space can be biased by their very depth.

A desirable heuristic can be one that attempts to measure the conflict in the game. A heuristic applied to tic-tac-toe (Fig 4.16, tp45)

Heuristic Search 16

KU NLPKU NLP

MinimaxProcedure (2)

Heuristic Search 17

KU NLPKU NLP


Heuristic Search 18

KU NLPKU NLP


Heuristic Search 19

KU NLPKU NLP

4.3.3 Alpha-Beta Procedure (1)

Minimax procedure is a depth-first process, its efficie

ncy can be improved by using branch-and-bound tec

hniques in which partial solutions that are clearly wor

se than known solutions can be abandoned early.

It requires the maintenance of two threshold values (a

lpha and beta) one representing a lower bound on the value that a maximizing no

de may be assigned one representing an upper bound on the value that a minimizing n

ode may be assigned.

Heuristic Search 20

KU NLPKU NLP


At maximizing levels, rule out a move early if it

becomes clear that its value will be less than the

current threshold.

At minimizing levels, search will be terminated if

values that are greater than the current threshold

are discovered.

Heuristic Search 21

KU NLPKU NLP


An Alpha Cutoff

Maximizing ply

Minimizing ply

A

B C

D E F G

(3)

(3)

(3) (5) (-5)

(-5)

Heuristic Search 22

KU NLPKU NLP


Alpha and Beta Cutoffs

Maximizing ply

Minimizing ply

Maximizing ply

Minimizing ply

A

B C

D E F G

(3)

(3)

(3) (5)(0) H

I J M N

K L

(0)

(0)

(5)

Heuristic Search 23

KU NLPKU NLP


Alpha and Beta Cutoffs (continued)A

B C

D E F G

(5)

(5)

(3) (5)(5) H

I J M N

K L

(0)

(0)

(5)

(7)(4)

(7)

Maximizing ply

Minimizing ply

Maximizing ply

Minimizing ply

Heuristic Search 24

KU NLPKU NLP


ku nlp heuristic search30 4.1.3 heuristic search and expert systems (1) q an interesting approach to...

Documents