ai_all

04.02.2012 AI 1 1

Artificial Intelligence: introduction

Lecturer: Lars MehnenTechnikum-Wien

04.02.2012 AI 1 2

Practical stuff

Course server: http://

Mail: [email protected]

Textbook: S. Russell and P. Norvig Artificial

Intelligence: A Modern Approach Prentice Hall, 2003, Second Edition

Exam: written at end of semester (50%)

04.02.2012 AI 1 3

Course overview

What is AI. Problem solving. Uncertain knowledge and reasoning. Planning.

04.02.2012 AI 1 4

Practical stuff

Excercices:Agents

WumpusProblem solving

Evolutionary algorithmsKnowledge and

reasoningExpert systemFuzzy logic

PlanningAgents

04.02.2012 AI 1 5

What is Artificial Intelligence

Creative extension of philosophy:Understand and BUILD intelligent entities

Origin after WWII Highly interdisciplinary Currently consist of huge variety of subfields

04.02.2012 AI 1 6

What is Artificial Intelligence

Different definitions due to different criteriaTwo dimensions:Thought processes/reasoning vs. behavior/actionSuccess according to human standards vs. success

according to an ideal concept of intelligence: rationality.

Systems that think like humans Systems that think rationally

Systems that act like humans Systems that act rationally

04.02.2012 AI 1 7

Systems that act like humans

When does a system behave intelligently? Turing (1950) Computing Machinery and Intelligence Operational test of intelligence: imitation game

Test still relevant now, yet might be the wrong question. Requires the collaboration of major components of AI: knowledge,

reasoning, language understanding, learning, …

04.02.2012 AI 1 8

Systems that act like humans

Alan Turing, the enigma

Problem with Turing test: not reproducible, constructive or amenable to mathematical analysis.

04.02.2012 AI 1 9

Systems that think like humans

How do humans think?Requires scientific theories of internal brain activities

(cognitive model): Level of abstraction? (knowledge or circuitry?) Validation?

• Predicting and testing human behavior• Identification from neurological data

Cognitive Science vs. Cognitive neuroscience. Both approaches are now distinct from AI Share that the available theories do not explain anything

resembling human intelligence.Three fields share a principal direction.

04.02.2012 AI 1 10

Systems that think rationally

Capturing the laws of thoughtAristotle: What are ‘correct’ argument and thought

processes?Correctness depends on irrefutability of reasoning

processes.This study initiated the field of logic.The logicist tradition in AI hopes to create intelligent systems

using logic programming. Problems:Not all intelligence is mediated by logic behaviorWhat is the purpose of thinking? What thought should one

have?

04.02.2012 AI 1 11

Systems that act rationally

Rational behavior: “doing the right thing”The “Right thing” is that what is expected to maximize

goal achievement given the available information. Can include thinking, yet in service of rational

action.Action without thinking: e.g. reflexes.

04.02.2012 AI 1 12

Systems that act rationally

Two advantages over previous approaches: More general than law of thoughts approach More amenable to scientific development.

Yet rationality is only applicable in ideal environments. Moreover rationality is not a very good model of reality.

04.02.2012 AI 1 13

Rational agents

An agent is an entity that perceives and acts This course is about designing rational agents

An agent is a function from percept histories to actions: For any given class of environments and task we seek the agent (or

class of agents) with the best performance. Problem: computational limitations make perfect rationality

unachievable.

04.02.2012 AI 1 14

Foundations of AI

Different fields have contributed to AI in the form of ideas,viewpoints and techniques. Philosophy: Logic, reasoning, mind as a physical system, foundations of

learning, language and rationality. Mathematics: Formal representation and proof algorithms, computation,

(un)decidability, (in)tractability, probability. Psychology: adaptation, phenomena of perception and motor control. Economics: formal theory of rational decisions, game theory. Linguistics: knowledge represetatio, grammar. Neuroscience: physical substrate for mental activities. Control theory: homeostatic systems, stability, optimal agent design.

04.02.2012 AI 1 15

A brief history

What happened after WWII?1943: Warren Mc Culloch and Walter Pitts: a model of

artificial boolean neurons to perform computations.First steps toward connectionist computation and learning

(Hebbian learning).Marvin Minsky and Dann Edmonds (1951) constructed the

first neural network computer

1950: Alan Turing’s “Computing Machinery and Intelligence”First complete vision of AI.

04.02.2012 AI 1 16

A brief history (2)

The birth of AI (1956) Darmouth Workshop bringing together top minds on automata theory,

neural nets and the study of intelligence. Allen Newell and Herbert Simon: The logic theorist (first nonnumerical thinking

program used for theorem proving) For the next 20 years the field was dominated by these participants.

Great expectations (1952-1969) Newell and Simon introduced the General Problem Solver.

• Imitation of human problem-solving Arthur Samuel (1952-)investigated game playing (checkers ) with great

success. John McCarthy(1958-) :

• Inventor of Lisp (second-oldest high-level language)• Logic oriented, Advice Taker (separation between knowledge and reasoning)

04.02.2012 AI 1 17

A brief history (3)

The birth of AI (1956) Great expectations continued ..

Marvin Minsky (1958 -)• Introduction of microworlds that appear to require intelligence to solve: e.g.

blocks-world.• Anti-logic orientation, society of the mind.

Collapse in AI research (1966 - 1973) Progress was slower than expected.

Unrealistic predictions. Some systems lacked scalability.

Combinatorial explosion in search. Fundamental limitations on techniques and representations.

Minsky and Papert (1969) Perceptrons.

04.02.2012 AI 1 18

A brief history (4)

AI revival through knowledge-based systems (1969-1970)General-purpose vs. domain specific

E.g. the DENDRAL project (Buchanan et al. 1969)• First successful knowledge intensive system.

Expert systems MYCIN to diagnose blood infections (Feigenbaum et al.)

• Introduction of uncertainty in reasoning. Increase in knowledge representation research.

Logic, frames, semantic nets, …

04.02.2012 AI 1 19

A brief history (5)

AI becomes an industry (1980 - present) R1 at DEC (McDermott, 1982) Fifth generation project in Japan (1981) American response …

Puts an end to the AI winter.

Connectionist revival (1986 - present) Parallel distributed processing (RumelHart and McClelland,

1986); backprop.

04.02.2012 AI 1 20

A brief history (6)

AI becomes a science (1987 - present)Neats vs. scruffies.

In speech recognition: hidden markov models In neural networks In uncertain reasoning and expert systems: Bayesian network

formalism…

The emergence of intelligent agents (1995 - present)The whole agent problem:

“How does an agent act/behave embedded in real environments with continuous sensory inputs”

04/02/2012 AI 1 1

Artificial Intelligence: Intelligent Agents

Lecturer: Lars MehnenTechnikum Wien

04/02/2012 AI 1 2

Outline

Agents and environments.The vacuum-cleaner world

The concept of rational behavior. Environments. Agent structure.

04/02/2012 AI 1 3

Agents and environments

Agents include human, robots, softbots, thermostats, etc.

The agent function maps percept sequence to actions

An agent can perceive its own actions, but not always it effects.

04/02/2012 AI 1 4

Agents and environments

The agent function will internally be represented by the agent program.

The agent program runs on the physical architecture to produce f.

04/02/2012 AI 1 5

The vacuum-cleaner world

Environment: square A and B Percepts: [location and content] e.g. [A, Dirty] Actions: left, right, suck, and no-op

04/02/2012 AI 1 6


Percept sequence Action

[A,Clean] Right

[A, Dirty] Suck

[B, Clean] Left

[B, Dirty] Suck

[A, Clean],[A, Clean] Right

[A, Clean],[A, Dirty] Suck

… …

04/02/2012 AI 1 7


function REFLEX-VACUUM-AGENT ([location, status]) return an actionif status == Dirty then return Suckelse if location == A then return Rightelse if location == B then return Left

What is the right function? Can it be implemented in a small agent program?

04/02/2012 AI 1 8

The concept of rationality

A rational agent is one that does the right thing. Every entry in the table is filled out correctly.

What is the right thing? Approximation: the most successful agent. Measure of success?

Performance measure should be objective E.g. the amount of dirt cleaned within a certain time. E.g. how clean the floor is. …

Performance measure according to what is wanted in the environment instead of how the agents should behave.

04/02/2012 AI 1 9

Rationality

What is rational at a given time depends on four things: Performance measure, Prior environment knowledge, Actions, Percept sequence to date (sensors).

DEF: A rational agent chooses whichever action maximizes the expected value of the performance measure given the percept sequence to date and prior environment knowledge.

04/02/2012 AI 1 10

Rationality

Rationality ≠ omniscience An omniscient agent knows the actual outcome of its

actions.

Rationality ≠ perfection Rationality maximizes expected performance, while

perfection maximizes actual performance.

04/02/2012 AI 1 11

Rationality

The proposed definition requires:

Information gathering/explorationTo maximize future rewards

Learn from perceptsExtending prior knowledge

Agent autonomyCompensate for incorrect prior knowledge

04/02/2012 AI 1 12

Environments

To design a rational agent we must specify its task environment.

PEAS description of the environment:Performance EnvironmentActuatorsSensors

04/02/2012 AI 1 13

Environments

E.g. Fully automated taxi:

PEAS description of the environment:• Performance

Safety, destination, profits, legality, comfort• Environment

Streets/freeways, other traffic, pedestrians, weather,, …• Actuators

Steering, accelerating, brake, horn, speaker/display,…• Sensors

Video, sonar, speedometer, engine sensors, keyboard, GPS, …

04/02/2012 AI 1 14

Environment types

Solitaire Backgammon Internet shopping Taxi

Observable??

Deterministic??

Episodic??

Static??

Discrete??

Single-agent??

04/02/2012 AI 1 15

Environment types


Observable??

Deterministic??

Episodic??

Static??

Discrete??

Single-agent??

Fully vs. partially observable: an environment is full observable when the sensors can detect all aspects that are relevant to the choice of action.

Presenter

Presentation Notes

Volledige observeerbare omgevingen zijn handig omdat de agent geen Interne staat moet bijhouden over de omgeving Partieel observeerbare omgevingen door noise of niet goed afgestelde sensoren of ontbrekende informatie

04/02/2012 AI 1 16

Environment types


Observable?? FULL FULL PARTIAL PARTIAL

Deterministic??

Episodic??

Static??

Discrete??

Single-agent??

Fully vs. partially observable: an environment is full observable when the sensors can detect all aspects that are relevant to the choice of action.

04/02/2012 AI 1 17

Environment types



Deterministic??

Episodic??

Static??

Discrete??

Single-agent??

Deterministic vs. stochastic: if the next environment state is completelydetermined by the current state the executed action then the environment is

deterministic.

Presenter

Presentation Notes

Als de omgeving gedeeltelijk observeerbaar is kan deze stochastisch lijken. Dus het is beter om te kijken of de omgebing determinitisch of stochastisch is vanuit het standpunt van de agent. Als de omgeving deterministisch is behalve de acties van de andere agenten dan is de omgeving strategisch.

04/02/2012 AI 1 18

Environment types



Deterministic?? YES NO YES NO

Episodic??

Static??

Discrete??

Single-agent??

Deterministic vs. stochastic: if the next environment state is completelydetermined by the current state the executed action then the environment is

deterministic.

04/02/2012 AI 1 19

Environment types




Episodic??

Static??

Discrete??

Single-agent??

Episodic vs. sequential: In an episodic environment the agent’s experiencecan be divided into atomic steps where the agents perceives and then performs

A single action. The choice of action depends only on the episode itself

04/02/2012 AI 1 20

Environment types




Episodic?? NO NO NO NO

Static??

Discrete??

Single-agent??

Episodic vs. sequential: In an episodic environment the agent’s experiencecan be divided into atomic steps where the agents perceives and then performs

A single action. The choice of action depends only on the episode itself

04/02/2012 AI 1 21

Environment types





Static??

Discrete??

Single-agent??

Static vs. dynamic: If the environment can change while the agent is choosingan action, the environment is dynamic. Semi-dynamic if the agent’s performance

changes even when the environment remains the same.

Presenter

Presentation Notes

If the agent performance score changes ahen time passes, the environment is semi-dynamic.

04/02/2012 AI 1 22

Environment types





Static?? YES YES SEMI NO

Discrete??

Single-agent??

Static vs. dynamic: If the environment can change while the agent is choosingan action, the environment is dynamic. Semi-dynamic if the agent’s performance

changes even when the environment remains the same.

Presenter

Presentation Notes


04/02/2012 AI 1 23

Environment types






Discrete??

Single-agent??

Discrete vs. continuous: This distinction can be applied to the state of the environment, the way time is handled and to the percepts/actions of the agent.

Presenter

Presentation Notes


04/02/2012 AI 1 24

Environment types






Discrete?? YES YES YES NO

Single-agent??

Discrete vs. continuous: This distinction can be applied to the state of the environment, the way time is handled and to the percepts/actions of the agent.

Presenter

Presentation Notes


04/02/2012 AI 1 25

Environment types







Single-agent??

Single vs. multi-agent: Does the environment contain other agents whoare also maximizing some performance measure that depends on the

current agent’s actions?

Presenter

Presentation Notes


04/02/2012 AI 1 26

Environment types







Single-agent?? YES NO NO NO

Single vs. multi-agent: Does the environment contain other agents whoare also maximizing some performance measure that depends on the

current agent’s actions?

Presenter

Presentation Notes


04/02/2012 AI 1 27

Environment types

The simplest environment isFully observable, deterministic, episodic, static, discrete

and single-agent.

Most real situations are:Partially observable, stochastic, sequential, dynamic,

continuous and multi-agent.

04/02/2012 AI 1 28

Agent types

How does the inside of the agent work?Agent = architecture + program

All agents have the same skeleton: Input = current perceptsOutput = actionProgram= manipulates input to produce output

Note difference with agent function (page 3-4)

04/02/2012 AI 1 29

Agent types

Function TABLE-DRIVEN_AGENT(percept) returns an action

static: percepts, a sequence initially emptytable, a table of actions, indexed by percept sequence

append percept to the end of perceptsaction ← LOOKUP(percepts, table)return action

This approach is doomed to failure!

04/02/2012 AI 1 30

Agent types

Four basic kind of agent programs will be discussed:Simple reflex agentsModel-based reflex agentsGoal-based agentsUtility-based agents

All these can be turned into learning agents.

04/02/2012 AI 1 31

Agent types; simple reflex

Select action on the basis of only the current percept. E.g. the vacuum-agent

Large reduction in possible percept/action situations(next page).

Implemented through condition-action rules If dirty then suck

04/02/2012 AI 1 32


function REFLEX-VACUUM-AGENT ([location, status]) return an actionif status == Dirty then return Suckelse if location == A then return Rightelse if location == B then return Left

04/02/2012 AI 1 33

Agent types; simple reflex

function SIMPLE-REFLEX-AGENT(percept) returns an action

static: rules, a set of condition-action rules

state ← INTERPRET-INPUT(percept)rule ← RULE-MATCH(state, rule)action ← RULE-ACTION[rule]return action

Will only work if the environment is fully observable otherwise infinite loops may occur.

04/02/2012 AI 1 34

Agent types; reflex and state

To tackle partially observableenvironments. Maintain internal state

Over time update state using world knowledge How does the world change. How do actions affect world.⇒ Model of World

04/02/2012 AI 1 35

Agent types; reflex and state

function REFLEX-AGENT-WITH-STATE(percept) returns an action

static: rules, a set of condition-action rulesstate, a description of the current world stateaction, the most recent action.

state ← UPDATE-STATE(state, action, percept)rule ← RULE-MATCH(state, rule)action ← RULE-ACTION[rule]return action

04/02/2012 AI 1 36

Agent types; goal-based

The agent needs a goal to know which situations are desirable. Things become difficult when long

sequences of actions are required to find the goal.

Typically investigated in search and planning research.

Major difference: future is taken into account

Is more flexible since knowledge is represented explicitly and can be manipulated.

04/02/2012 AI 1 37

Agent types; utility-based

Certain goals can be reached in different ways. Some are better, have a higher

utility. Utility function maps a (sequence of)

state(s) onto a real number. Improves on goals:

Selecting between conflicting goals Select appropriately between

several goals based on likelihood of success.

04/02/2012 AI 1 38

Agent types; learning

All previous agent-programs describe methods for selecting actions. Yet it does not explain the origin

of these programs. Learning mechanisms can be

used to perform this task. Teach them instead of

instructing them. Advantage is the robustness of

the program toward initially unknown environments.

04/02/2012 AI 1 39

Agent types; learning

Learning element: introduce improvements in performance element. Critic provides feedback on agents

performance based on fixed performance standard.

Performance element: selecting actions based on percepts. Corresponds to the previous agent

programs Problem generator: suggests actions

that will lead to new and informative experiences. Exploration vs. exploitation

Artificial Intelligence: problem solving and search


04/02/2012 AI 1 2

Outline

Problem-solving agentsA kind of goal-based agent

Problem typesSingle state (fully observable)Search with partial information

Problem formulationExample problems

Basic search algorithmsUninformed

04/02/2012 AI 1 3

Example: Romania

04/02/2012 AI 1 4

Example: Romania

On holiday in Romania; currently in Arad Flight leaves tomorrow from Bucharest

Formulate goal Be in Bucharest

Formulate problem States: various cities Actions: drive between cities

Find solution Sequence of cities; e.g. Arad, Sibiu, Fagaras, Bucharest, …

04/02/2012 AI 1 5

Problem-solving agent

Four general steps in problem solving:Goal formulation

What are the successful world statesProblem formulation

What actions and states to consider given the goalSearch

Determine the possible sequence of actions that lead to the states of known values and then choosing the best sequence.

Execute Give the solution perform the actions.

04/02/2012 AI 1 6

Problem-solving agent

function SIMPLE-PROBLEM-SOLVING-AGENT(percept) return an actionstatic: seq, an action sequence

state, some description of the current world stategoal, a goalproblem, a problem formulation

state ← UPDATE-STATE(state, percept)if seq is empty then

goal ← FORMULATE-GOAL(state)problem ← FORMULATE-PROBLEM(state,goal)seq ← SEARCH(problem)

action ← FIRST(seq)seq ← REST(seq)return action

04/02/2012 AI 1 7

Problem types

Deterministic, fully observable ⇒ single state problem Agent knows exactly which state it will be in; solution is a sequence.

Partial knowledge of states and actions: Non-observable ⇒ sensorless or conformant problem

Agent may have no idea where it is; solution (if any) is a sequence. Nondeterministic and/or partially observable ⇒ contingency problem

Percepts provide new information about current state; solution is a tree or policy; often interleave search and execution.

Unknown state space ⇒ exploration problem (“online”) When states and actions of the environment are unknown.

04/02/2012 AI 1 8

Example: vacuum world

Single state, start in #5. Solution??

04/02/2012 AI 1 9


Single state, start in #5. Solution?? [Right, Suck]

04/02/2012 AI 1 10


Single state, start in #5. Solution?? [Right, Suck]

Sensorless: start in {1,2,3,4,5,6,7,8} e.g Right goes to {2,4,6,8}. Solution??

Contingency: start in {1,3}. (assume Murphy’s law, Suck can dirty a clean carpet and local sensing: [location,dirt] only. Solution??

04/02/2012 AI 1 11

Problem formulation

A problem is defined by: An initial state, e.g. Arad Successor function S(X)= set of action-state pairs

e.g. S(Arad)={<Arad → Zerind, Zerind>,…}intial state + successor function = state space Goal test, can be

Explicit, e.g. x=‘at bucharest’ Implicit, e.g. checkmate(x)

Path cost (additive) e.g. sum of distances, number of actions executed, … c(x,a,y) is the step cost, assumed to be >= 0

A solution is a sequence of actions from initial to goal state.Optimal solution has the lowest path cost.

04/02/2012 AI 1 12

Selecting a state space

Real world is absurdly complex.State space must be abstracted for problem solving.

(Abstract) state = set of real states. (Abstract) action = complex combination of real actions.

e.g. Arad →Zerind represents a complex set of possible routes, detours, rest stops, etc.

The abstraction is valid if the path between two states is reflected in the real world.

(Abstract) solution = set of real paths that are solutions in the real world. Each abstract action should be “easier” than the real problem.

04/02/2012 AI 1 13


States?? Initial state?? Actions?? Goal test?? Path cost??

04/02/2012 AI 1 14


States?? two locations with or without dirt: 2 x 22=8 states. Initial state?? Any state can be initial Actions?? {Left, Right, Suck} Goal test?? Check whether squares are clean. Path cost?? Number of actions to reach goal.

04/02/2012 AI 1 15

Example: 8-puzzle


04/02/2012 AI 1 16

Example: 8-puzzle

States?? Integer location of each tile Initial state?? Any state can be initial Actions?? {Left, Right, Up, Down} Goal test?? Check whether goal configuration is reached Path cost?? Number of actions to reach goal

04/02/2012 AI 1 17

Example: 8-queens problem


04/02/2012 AI 1 18


Incremental formulation vs. complete-state formulation States?? Initial state?? Actions?? Goal test?? Path cost??

04/02/2012 AI 1 19


Incremental formulation States?? Any arrangement of 0 to 8 queens on the board Initial state?? No queens Actions?? Add queen in empty square Goal test?? 8 queens on board and none attacked Path cost?? None

3 x 1014 possible sequences to investigate

04/02/2012 AI 1 20


Incremental formulation (alternative) States?? n (0≤ n≤ 8) queens on the board, one per column in the n leftmost columns

with no queen attacking another. Actions?? Add queen in leftmost empty column such that is not attacking other

queens2057 possible sequences to investigate; Yet makes no difference when n=100

04/02/2012 AI 1 21

Example: robot assembly


04/02/2012 AI 1 22

Example: robot assembly

States?? Real-valued coordinates of robot joint angles; parts of the object to be assembled.

Initial state?? Any arm position and object configuration. Actions?? Continuous motion of robot joints Goal test?? Complete assembly (without robot) Path cost?? Time to execute

04/02/2012 AI 1 23

Basic search algorithms

How do we find the solutions of previous problems? Search the state space (remember complexity of space depends on

state representation)

Here: search through explicit tree generation ROOT= initial state. Nodes and leafs generated through successor function.

In general search generates a graph (same state through multiple paths)

04/02/2012 AI 1 24

Simple tree search example

function TREE-SEARCH(problem, strategy) return a solution or failureInitialize search tree to the initial state of the problemdo

if no candidates for expansion then return failurechoose leaf node for expansion according to strategyif node contains goal state then return solutionelse expand the node and add resulting nodes to the search tree

enddo

04/02/2012 AI 1 25




enddo

04/02/2012 AI 1 26




enddo

←Determines searchprocess!!

04/02/2012 AI 1 27

State space vs. search tree

A state is a (representation of) a physical configuration A node is a data structure belong to a search tree

A node has a parent, children, … and ncludes path cost, depth, … Here node= <state, parent-node, action, path-cost, depth> FRINGE= contains generated nodes which are not yet expanded.

White nodes with black outline

04/02/2012 AI 1 28

Tree search algorithm

function TREE-SEARCH(problem,fringe) return a solution or failurefringe ← INSERT(MAKE-NODE(INITIAL-STATE[problem]), fringe)loop do

if EMPTY?(fringe) then return failurenode ← REMOVE-FIRST(fringe)if GOAL-TEST[problem] applied to STATE[node] succeeds

then return SOLUTION(node)fringe ← INSERT-ALL(EXPAND(node, problem), fringe)

04/02/2012 AI 1 29

Tree search algorithm (2)

function EXPAND(node,problem) return a set of nodessuccessors ← the empty setfor each <action, result> in SUCCESSOR-FN[problem](STATE[node]) do

s ← a new NODESTATE[s] ← resultPARENT-NODE[s] ← nodeACTION[s] ← actionPATH-COST[s] ← PATH-COST[node] + STEP-COST(node, action,s)DEPTH[s] ← DEPTH[node]+1add s to successors

return successors

04/02/2012 AI 1 30

Search strategies

A strategy is defined by picking the order of node expansion. Problem-solving performance is measured in four ways:

Completeness; Does it always find a solution if one exists? Optimality; Does it always find the least-cost solution? Time Complexity; Number of nodes generated/expanded? Space Complexity; Number of nodes stored in memory during search?

Time and space complexity are measured in terms of problem difficulty defined by: b - maximum branching factor of the search tree d - depth of the least-cost solution m - maximum depth of the state space (may be ∞)

04/02/2012 AI 1 31

Uninformed search strategies

(a.k.a. blind search) = use only information available in problem definition. When strategies can determine whether one non-goal state is better

than another → informed search. Categories defined by expansion algorithm:

Breadth-first search Uniform-cost search Depth-first search Depth-limited search Iterative deepening search. Bidirectional search

04/02/2012 AI 1 32

BF-search, an example

Expand shallowest unexpanded node Implementation: fringe is a FIFO queue

A

04/02/2012 AI 1 33



A

B C

04/02/2012 AI 1 34



A

B C

D E

04/02/2012 AI 1 35



A

B C

D E F G

04/02/2012 AI 1 36

BF-search; evaluation

Completeness:Does it always find a solution if one exists?YES If shallowest goal node is at some finite depth dCondition: If b is finite

• (maximum num. Of succ. nodes is finite)

04/02/2012 AI 1 37


Completeness: YES (if b is finite)

Time complexity: Assume a state space where every state has b successors.

root has b successors, each node at the next level has again b successors (total b2), … Assume solution is at depth d Worst case; expand all but the last node at depth d Total numb. of nodes generated:

b + b2 + b3 + ...+ bd + (bd +1 − b) = O(bd +1)

04/02/2012 AI 1 38



Time complexity: Total numb. of nodes generated:

Space complexity: Idem if each node is retained in memory

b + b2 + b3 + ...+ bd + (bd +1 − b) = O(bd +1)

04/02/2012 AI 1 39



Time complexity: Total numb. of nodes generated:

Space complexity: Idem if each node is retained in memory

Optimality: Does it always find the least-cost solution? In general YES

unless actions have different cost.

b + b2 + b3 + ...+ bd + (bd +1 − b) = O(bd +1)

04/02/2012 AI 1 40


Two lessons: Memory requirements are a bigger problem than its execution time. Exponential complexity search problems cannot be solved by uninformed search

methods for any but the smallest instances.

DEPTH2 NODES TIME MEMORY

2 1100 0.11 seconds 1 megabyte

4 111100 11 seconds 106 megabytes

6 107 19 minutes 10 gigabytes

8 109 31 hours 1 terabyte

10 1011 129 days 101 terabytes

12 1013 35 years 10 petabytes

14 1015 3523 years 1 exabyte

04/02/2012 AI 1 41

Uniform-cost search

Extension of BF-search:Expand node with lowest path cost

Implementation: fringe = queue ordered by path cost.

UC-search is the same as BF-search when all step-costs are equal.

04/02/2012 AI 1 42

Uniform-cost search

Completeness: YES, if step-cost > ε (smal positive constant)

Time complexity: Assume C* the cost of the optimal solution.

Assume that every action costs at least ε Worst-case:

Space complexity: Idem to time complexity

Optimality: nodes expanded in order of increasing path cost. YES, if complete.

O(bC*/ε )

04/02/2012 AI 1 43

DF-search, an example

Expand deepest unexpanded node Implementation: fringe is a LIFO queue (=stack)

A

04/02/2012 AI 1 44



A

B C

04/02/2012 AI 1 45



A

B C

D E

04/02/2012 AI 1 46



A

B C

D E

H I

04/02/2012 AI 1 47



A

B C

D E

H I

04/02/2012 AI 1 48



AB C

D E

H I

04/02/2012 AI 1 49



AB C

D E

H I J K

04/02/2012 AI 1 50



A

B C

D E

H I J K

04/02/2012 AI 1 51



A

B C

D E

H I J K

04/02/2012 AI 1 52



A

B C

D E

H I J K

F G

04/02/2012 AI 1 53



A

B C

D E

H I J K

F G

L M

04/02/2012 AI 1 54



A

B C

D E

H I J K

F G

L M

04/02/2012 AI 1 55

DF-search; evaluation

Completeness;Does it always find a solution if one exists?NOunless search space is finite and no loops are

possible.

04/02/2012 AI 1 56


Completeness;NO unless search space is finite.

Time complexity;Terrible if m is much larger than d (depth of optimal solution)But if many solutions, then faster than BF-search

O(bm )

04/02/2012 AI 1 57



Time complexity; Space complexity;Backtracking search uses even less memory

One successor instead of all b.

O(bm +1)

O(bm )

04/02/2012 AI 1 58



Time complexity; Space complexity; Optimallity; NoSame issues as completenessAssume node J and C contain goal states

O(bm +1)

O(bm )

04/02/2012 AI 1 59

Depth-limited search

Is DF-search with depth limit l. i.e. nodes at depth l have no successors. Problem knowledge can be used

Solves the infinite-path problem. If l < d then incompleteness results. If l > d then not optimal. Time complexity: Space complexity:

O(bl )

O(bl)

04/02/2012 AI 1 60

Depth-limited algorithm

function DEPTH-LIMITED-SEARCH(problem,limit) return a solution or failure/cutoffreturn RECURSIVE-DLS(MAKE-NODE(INITIAL-STATE[problem]),problem,limit)

function RECURSIVE-DLS(node, problem, limit) return a solution or failure/cutoffcutoff_occurred? ← falseif GOAL-TEST[problem](STATE[node]) then return SOLUTION(node)else if DEPTH[node] == limit then return cutoffelse for each successor in EXPAND(node, problem) do

result ← RECURSIVE-DLS(successor, problem, limit)if result == cutoff then cutoff_occurred? ← trueelse if result ≠ failure then return result

if cutoff_occurred? then return cutoff else return failure

04/02/2012 AI 1 61

Iterative deepening search

What?A general strategy to find best depth limit l.Goals is found at depth d, the depth of the shallowest

goal-node.Often used in combination with DF-search

Combines benefits of DF- en BF-search

04/02/2012 AI 1 62

Iterative deepening search

function ITERATIVE_DEEPENING_SEARCH(problem) return a solution or failure

inputs: problem

for depth ← 0 to ∞ doresult ← DEPTH-LIMITED_SEARCH(problem, depth)if result ≠ cuttoff then return result

04/02/2012 AI 1 63

ID-search, example

Limit=0

04/02/2012 AI 1 64

ID-search, example

Limit=1

04/02/2012 AI 1 65

ID-search, example

Limit=2

04/02/2012 AI 1 66

ID-search, example

Limit=3

04/02/2012 AI 1 67

ID search, evaluation

Completeness:YES (no infinite paths)

04/02/2012 AI 1 68


Completeness: YES (no infinite paths)

Time complexity: Algorithm seems costly due to repeated generation of certain states. Node generation:

level d: once level d-1: 2 level d-2: 3 … level 2: d-1 level 1: d

N(IDS) = (d)b + (d −1)b2 + ...+ (1)bd

N(BFS) = b + b2 + ...+ bd + (bd +1 − b)

O(bd )

N(IDS) = 50 + 400 + 3000 + 20000 +100000 =123450N(BFS) =10 +100 +1000 +10000 +100000 + 999990 =1111100

Num. Comparison for b=10 and d=5 solution at far right

04/02/2012 AI 1 69



Time complexity: Space complexity:Cfr. depth-first search

O(bd )

O(bd)

04/02/2012 AI 1 70



Time complexity: Space complexity: Optimality: YES if step cost is 1.Can be extended to iterative lengthening search

Same idea as uniform-cost search Increases overhead.

O(bd )

O(bd)

04/02/2012 AI 1 71

Bidirectional search

Two simultaneous searches from start an goal. Motivation:

Check whether the node belongs to the other fringe before expansion. Space complexity is the most significant weakness. Complete and optimal if both searches are BF.

bd / 2 + bd / 2 ≠ bd

04/02/2012 AI 1 72

How to search backwards?

The predecessor of each node should be efficiently computable.When actions are easily reversible.

04/02/2012 AI 1 73

Summary of algorithms

Criterion Breadth-First Uniform-cost Depth-First Depth-limited Iterative deepening

Bidirectional search

Complete? YES* YES* NO YES, if l ≥ d

YES YES*

Time bd+1 bC*/e bm bl bd bd/2

Space bd+1 bC*/e bm bl bd bd/2

Optimal? YES* YES* NO NO YES YES

04/02/2012 AI 1 74

Repeated states

Failure to detect repeated states can turn a solvable problems into unsolvable ones.

04/02/2012 AI 1 75

Graph search algorithm

Closed list stores all expanded nodes

function GRAPH-SEARCH(problem,fringe) return a solution or failureclosed ← an empty setfringe ← INSERT(MAKE-NODE(INITIAL-STATE[problem]), fringe)loop do


then return SOLUTION(node)if STATE[node] is not in closed then

add STATE[node] to closedfringe ← INSERT-ALL(EXPAND(node, problem), fringe)

04/02/2012 AI 1 76

Graph search, evaluation

Optimality: GRAPH-SEARCH discard newly discovered paths.

This may result in a sub-optimal solution YET: when uniform-cost search or BF-search with constant step cost

Time and space complexity, proportional to the size of the state space

(may be much smaller than O(bd)).

DF- and ID-search with closed list no longer has linear space requirements since all nodes are stored in closed list!!

04/02/2012 AI 1 77

Search with partial information

Previous assumption:Environment is fully observableEnvironment is deterministic Agent knows the effects of its actions

What if knowledge of states or actions is incomplete?

04/02/2012 AI 1 78

Search with partial information

Partial knowledge of states and actions: sensorless or conformant problem

Agent may have no idea where it is; solution (if any) is a sequence. contingency problem

Percepts provide new information about current state; solution is a tree or policy; often interleave search and execution.

If uncertainty is caused by actions of another agent: adversarial problem

exploration problemWhen states and actions of the environment are unknown.

04/02/2012 AI 1 79

Conformant problems

start in {1,2,3,4,5,6,7,8} e.g Right goes to {2,4,6,8}. Solution?? [Right, Suck, Left,Suck]

When the world is not fully observable: reason about a set of states that migth be reached=belief state

04/02/2012 AI 1 80

Conformant problems

Search space of belief states Solution = belief state with all

members goal states. If S states then 2S belief states. Murphy’s law:

Suck can dirty a clear square.

04/02/2012 AI 1 81

Belief state of vacuum-world

04/02/2012 AI 1 82

Contingency problems Contingency, start in {1,3}. Murphy’s law, Suck can dirty a clean carpet. Local sensing: dirt, location only.

Percept = [L,Dirty] ={1,3} [Suck] = {5,7} [Right] ={6,8} [Suck] in {6}={8} (Success) BUT [Suck] in {8} = failure

Solution?? Belief-state: no fixed action sequence

guarantees solution Relax requirement:

[Suck, Right, if [R,dirty] then Suck] Select actions based on contingencies arising

during execution.

Artificial intelligence: informed search


4 februari 2012 AI 1 2

Outline

Informed = use problem-specific knowledge Which search strategies?

Best-first search and its variants Heuristic functions?

How to invent them Local search and optimization

Hill climbing, local beam search, genetic algorithms,… Local search in continuous spaces Online search agents


Previously: tree-searchfunction TREE-SEARCH(problem,fringe) return a solution or failure

fringe ← INSERT(MAKE-NODE(INITIAL-STATE[problem]), fringe)loop do


then return SOLUTION(node)fringe ← INSERT-ALL(EXPAND(node, problem), fringe)

A strategy is defined by picking the order of node expansion


Best-first search

General approach of informed search: Best-first search: node is selected for expansion based on an

evaluation function f(n) Idea: evaluation function measures distance to the goal.

Choose node which appears best Implementation: fringe is queue sorted in decreasing order of desirability. Special cases: greedy search, A* search


A heuristic function

[dictionary]“A rule of thumb, simplification, or educated guess that reduces or limits the search for solutions in domains that are difficult and poorly understood.” h(n) = estimated cost of the cheapest path from node n to goal

node. If n is goal then h(n)=0

More information later.


Romania with step costs in km

hSLD=straight-line distance heuristic.

hSLD can NOT be computed from the problem description itself

In this example f(n)=h(n) Expand node that is closest to

goal

= Greedy best-first search


Greedy search example

Assume that we want to use greedy search to solve the problem of travelling from Arad to Bucharest.

The initial state=Arad

Arad (366)



The first expansion step produces: Sibiu, Timisoara and Zerind

Greedy best-first will select Sibiu.

Arad

Sibiu(253) Timisoara(329)

Zerind(374)



If Sibiu is expanded we get: Arad, Fagaras, Oradea and Rimnicu Vilcea

Greedy best-first search will select: Fagaras

Arad

Sibiu

Arad(366) Fagaras

(176)Oradea(380)

Rimnicu Vilcea(193)



If Fagaras is expanded we get: Sibiu and Bucharest

Goal reached !! Yet not optimal (see Arad, Sibiu, Rimnicu Vilcea, Pitesti)

Arad

Sibiu

Fagaras

Sibiu(253)

Bucharest(0)


Greedy search, evaluation

Completeness: NO (cfr. DF-search) Check on repeated states Minimizing h(n) can result in false starts, e.g. Iasi to Fagaras.



Completeness: NO (cfr. DF-search) Time complexity?

Cfr. Worst-case DF-search(with m is maximum depth of search space) Good heuristic can give dramatic improvement.

O(bm )



Completeness: NO (cfr. DF-search) Time complexity: Space complexity:Keeps all nodes in memory

O(bm )

O(bm )



Completeness: NO (cfr. DF-search) Time complexity: Space complexity: Optimality? NOSame as DF-search

O(bm )

O(bm )


A* search

Best-known form of best-first search. Idea: avoid expanding paths that are already expensive. Evaluation function f(n)=g(n) + h(n)

g(n) the cost (so far) to reach the node. h(n) estimated cost to get from the node to the goal. f(n) estimated total cost of path through n to goal.


A* search

A* search uses an admissible heuristic A heuristic is admissible if it never overestimates the cost to

reach the goalAre optimistic

Formally: 1. h(n) <= h*(n) where h*(n) is the true cost from n2. h(n) >= 0 so h(G)=0 for any goal G.

e.g. hSLD(n) never overestimates the actual road distance


Romania example


A* search example

Find Bucharest starting at Arad f(Arad) = c(??,Arad)+h(Arad)=0+366=366


A* search example

Expand Arrad and determine f(n) for each node f(Sibiu)=c(Arad,Sibiu)+h(Sibiu)=140+253=393 f(Timisoara)=c(Arad,Timisoara)+h(Timisoara)=118+329=447 f(Zerind)=c(Arad,Zerind)+h(Zerind)=75+374=449

Best choice is Sibiu


A* search example

Expand Sibiu and determine f(n) for each node f(Arad)=c(Sibiu,Arad)+h(Arad)=280+366=646 f(Fagaras)=c(Sibiu,Fagaras)+h(Fagaras)=239+179=415 f(Oradea)=c(Sibiu,Oradea)+h(Oradea)=291+380=671 f(Rimnicu Vilcea)=c(Sibiu,Rimnicu Vilcea)+

h(Rimnicu Vilcea)=220+192=413 Best choice is Rimnicu Vilcea


A* search example

Expand Rimnicu Vilcea and determine f(n) for each node f(Craiova)=c(Rimnicu Vilcea, Craiova)+h(Craiova)=360+160=526 f(Pitesti)=c(Rimnicu Vilcea, Pitesti)+h(Pitesti)=317+100=417 f(Sibiu)=c(Rimnicu Vilcea,Sibiu)+h(Sibiu)=300+253=553

Best choice is Fagaras


A* search example

Expand Fagaras and determine f(n) for each node f(Sibiu)=c(Fagaras, Sibiu)+h(Sibiu)=338+253=591 f(Bucharest)=c(Fagaras,Bucharest)+h(Bucharest)=450+0=450

Best choice is Pitesti !!!


A* search example

Expand Pitesti and determine f(n) for each node f(Bucharest)=c(Pitesti,Bucharest)+h(Bucharest)=418+0=418

Best choice is Bucharest !!! Optimal solution (only if h(n) is admissable)

Note values along optimal path !!


Optimality of A*(standard proof)

Suppose suboptimal goal G2 in the queue. Let n be an unexpanded node on a shortest to optimal goal G.

f(G2 ) = g(G2 ) since h(G2 )=0> g(G) since G2 is suboptimal>= f(n) since h is admissible

Since f(G2) > f(n), A* will never select G2 for expansion


BUT … graph search

Discards new paths to repeated state.Previous proof breaks down

Solution:Add extra bookkeeping i.e. remove more expsive of two

paths.Ensure that optimal path to any repeated state is always

first followed. Extra requirement on h(n): consistency (monotonicity)


Consistency

A heuristic is consistent if

If h is consistent, we have

i.e. f(n) is nondecreasing along any path.

h(n) ≤ c(n,a,n') + h(n')

f (n') = g(n') + h(n')= g(n) + c(n,a,n') + h(n')≥ g(n) + h(n)≥ f (n)


Optimality of A*(more usefull) A* expands nodes in order of increasing f value Contours can be drawn in state space

Uniform-cost search adds circles.

F-contours are graduallyAdded: 1) nodes with f(n)<C*2) Some nodes on the goalContour (f(n)=C*).

Contour I has allNodes with f=fi, wherefi < fi+1.


A* search, evaluation

Completeness: YESSince bands of increasing f are addedUnless there are infinitly many nodes with f<f(G)



Completeness: YES Time complexity:Number of nodes expanded is still exponential in the length of

the solution.



Completeness: YES Time complexity: (exponential with path length) Space complexity:

It keeps all generated nodes in memory Hence space is the major problem not time



Completeness: YES Time complexity: (exponential with path length) Space complexity:(all nodes are stored) Optimality: YES

Cannot expand fi+1 until fi is finished. A* expands all nodes with f(n)< C* A* expands some nodes with f(n)=C* A* expands no nodes with f(n)>C*

Also optimally efficient (not including ties)


Memory-bounded heuristic search

Some solutions to A* space problems (maintain completeness and optimality) Iterative-deepening A* (IDA*)

Here cutoff information is the f-cost (g+h) instead of depthRecursive best-first search(RBFS)

Recursive algorithm that attempts to mimic standard best-first search with linear space.

(simple) Memory-bounded A* ((S)MA*) Drop the worst-leaf node when memory is full


Recursive best-first search

function RECURSIVE-BEST-FIRST-SEARCH(problem) return a solution or failurereturn RFBS(problem,MAKE-NODE(INITIAL-STATE[problem]),∞)

function RFBS( problem, node, f_limit) return a solution or failure and a new f-cost limitif GOAL-TEST[problem](STATE[node]) then return nodesuccessors ← EXPAND(node, problem)if successors is empty then return failure, ∞for each s in successors do

f [s] ← max(g(s) + h(s), f [node])repeat

best ← the lowest f-value node in successorsif f [best] > f_limit then return failure, f [best]alternative ← the second lowest f-value among successorsresult, f [best] ← RBFS(problem, best, min(f_limit, alternative))if result ≠ failure then return result


Recursive best-first search

Keeps track of the f-value of the best-alternative path available. If current f-values exceeds this alternative f-value than

backtrack to alternative path.Upon backtracking change f-value to best f-value of its

children.Re-expansion of this result is thus still possible.


Recursive best-first search, ex.

Path until Rumnicu Vilcea is already expanded Above node; f-limit for every recursive call is shown on top. Below node: f(n) The path is followed until Pitesti which has a f-value worse than the f-limit.



Unwind recursion and store best f-value for current best leaf Pitestiresult, f [best] ← RBFS(problem, best, min(f_limit, alternative))

best is now Fagaras. Call RBFS for new best best value is now 450



Unwind recursion and store best f-value for current best leaf Fagarasresult, f [best] ← RBFS(problem, best, min(f_limit, alternative))

best is now Rimnicu Viclea (again). Call RBFS for new best Subtree is again expanded. Best alternative subtree is now through Timisoara.

Solution is found since because 447 > 417.


RBFS evaluation

RBFS is a bit more efficient than IDA* Still excessive node generation (mind changes)

Like A*, optimal if h(n) is admissible Space complexity is O(bd).

IDA* retains only one single number (the current f-cost limit) Time complexity difficult to characterize

Depends on accuracy if h(n) and how often best path changes. IDA* en RBFS suffer from too little memory.


(simplified) memory-bounded A*

Use all available memory. I.e. expand best leafs until available memory is full When full, SMA* drops worst leaf node (highest f-value) Like RFBS backup forgotten node to its parent

What if all leafs have the same f-value? Same node could be selected for expansion and deletion. SMA* solves this by expanding newest best leaf and deleting oldest worst

leaf. SMA* is complete if solution is reachable, optimal if optimal solution is

reachable.


Learning to search better

All previous algorithms use fixed strategies. Agents can learn to improve their search by exploiting the meta-level

state space. Each meta-level state is a internal (computational) state of a

program that is searching in the object-level state space. In A* such a state consists of the current search tree

A meta-level learning algorithm from experiences at the meta-level.


Heuristic functions

E.g for the 8-puzzle Avg. solution cost is about 22 steps (branching factor +/- 3) Exhaustive search to depth 22: 3.1 x 1010 states. A good heuristic function can reduce the search process.


Heuristic functions

E.g for the 8-puzzle knows two commonly used heuristics h1 = the number of misplaced tiles

h1(s)=8 h2 = the sum of the distances of the tiles from their goal positions (manhattan

distance). h2(s)=3+1+2+2+2+3+3+2=18


Heuristic quality

Effective branching factor b* Is the branching factor that a uniform tree of depth d would

have in order to contain N+1 nodes.

Measure is fairly constant for sufficiently hard problems. Can thus provide a good guide to the heuristic’s overall usefulness. A good value of b* is 1.

N +1=1+ b*+(b*)2 + ...+ (b*)d


Heuristic quality and dominance

1200 random problems with solution lengths from 2 to 24.

If h2(n) >= h1(n) for all n (both admissible)then h2 dominates h1 and is better for search


Inventing admissible heuristics

Admissible heuristics can be derived from the exact solution cost of a relaxed version of the problem: Relaxed 8-puzzle for h1 : a tile can move anywhere

As a result, h1(n) gives the shortest solution

Relaxed 8-puzzle for h2 : a tile can move to any adjacent square.As a result, h2(n) gives the shortest solution.

The optimal solution cost of a relaxed problem is no greater than the optimal solution cost of the real problem.

ABSolver found a usefull heuristic for the rubic cube.


Inventing admissible heuristics Admissible heuristics can also be derived from the solution cost of a subproblem of a

given problem. This cost is a lower bound on the cost of the real problem. Pattern databases store the exact solution to for every possible subproblem instance.

The complete heuristic is constructed using the patterns in the DB


Inventing admissible heuristics

Another way to find an admissible heuristic is through learning from experience:Experience = solving lots of 8-puzzlesAn inductive learning algorithm can be used to predict costs

for other states that arise during search.


Local search and optimization

Previously: systematic exploration of search space. Path to goal is solution to problem

YET, for some problems path is irrelevant. E.g 8-queens

Different algorithms can be used Local search



Local search= use single current state and move to neighboring states.

Advantages: Use very little memory Find often reasonable solutions in large or infinite state spaces.

Are also useful for pure optimization problems. Find best state according to some objective function. e.g. survival of the fittest as a metaphor for optimization.


Hill-climbing search

“is a loop that continuously moves in the direction of increasing value” It terminates when a peak is reached.

Hill climbing does not look ahead of the immediate neighbors of the current state.

Hill-climbing chooses randomly among the set of best successors, if there is more than one.

Hill-climbing a.k.a. greedy local search


Hill-climbing search

function HILL-CLIMBING( problem) return a state that is a local maximuminput: problem, a problemlocal variables: current, a node.

neighbor, a node.

current ← MAKE-NODE(INITIAL-STATE[problem])loop do

neighbor ← a highest valued successor of currentif VALUE [neighbor] ≤ VALUE[current] then return STATE[current]current ← neighbor


Hill-climbing example

8-queens problem (complete-state formulation). Successor function: move a single queen to another square in the

same column. Heuristic function h(n): the number of pairs of queens that are

attacking each other (directly or indirectly).


Hill-climbing example

a) shows a state of h=17 and the h-value for each possible successor.b) A local minimum in the 8-queens state space (h=1).

a) b)


Drawbacks

Ridge = sequence of local maxima difficult for greedy algorithms to navigate Plateaux = an area of the state space where the evaluation function is flat. Gets stuck 86% of the time.


Hill-climbing variations

Stochastic hill-climbingRandom selection among the uphill moves.The selection probability can vary with the steepness of the

uphill move. First-choice hill-climbing cfr. stochastic hill climbing by generating successors

randomly until a better one is found. Random-restart hill-climbingTries to avoid getting stuck in local maxima.


Simulated annealing

Escape local maxima by allowing “bad” moves. Idea: but gradually decrease their size and frequency.

Origin; metallurgical annealing Bouncing ball analogy:

Shaking hard (= high temperature). Shaking less (= lower the temperature).

If T decreases slowly enough, best state is reached. Applied for VLSI layout, airline scheduling, etc.


Simulated annealingfunction SIMULATED-ANNEALING( problem, schedule) return a solution state

input: problem, a problemschedule, a mapping from time to temperature

local variables: current, a node.next, a node.T, a “temperature” controlling the probability of downward steps

current ← MAKE-NODE(INITIAL-STATE[problem])for t ← 1 to ∞ do

T ← schedule[t]if T = 0 then return currentnext ← a randomly selected successor of current∆E ← VALUE[next] - VALUE[current]if ∆E > 0 then current ← next else current ← next only with probability e∆E /T


Local beam search

Keep track of k states instead of one Initially: k random states Next: determine all successors of k states If any of successors is goal → finished Else select k best from successors and repeat.

Major difference with random-restart search Information is shared among k search threads.

Can suffer from lack of diversity. Stochastic variant: choose k successors at proportionally to state

success.


Genetic algorithms

Variant of local beam search with sexual recombination.


Genetic algorithmfunction GENETIC_ALGORITHM( population, FITNESS-FN) return an individual

input: population, a set of individualsFITNESS-FN, a function which determines the quality of the individual

repeatnew_population ← empty setloop for i from 1 to SIZE(population) do

x ← RANDOM_SELECTION(population, FITNESS_FN)y ← RANDOM_SELECTION(population, FITNESS_FN)

child ← REPRODUCE(x,y)if (small random probability) then child ← MUTATE(child )add child to new_population

population ← new_populationuntil some individual is fit enough or enough time has elapsedreturn the best individual


Exploration problems

Until now all algorithms were offline. Offline= solution is determined before executing it. Online = interleaving computation and action

Online search is necessary for dynamic and semi-dynamic environments It is impossible to take into account all possible contingencies.

Used for exploration problems: Unknown states and actions. e.g. any robot in a new environment, a newborn baby,…


Online search problems

Agent knowledge: ACTION(s): list of allowed actions in state s C(s,a,s’): step-cost function (! After s’ is determined) GOAL-TEST(s)

An agent can recognize previous states. Actions are deterministic. Access to admissible heuristic h(s)

e.g. manhattan distance


Online search problems

Objective: reach goal with minimal cost Cost = total cost of travelled path Competitive ratio=comparison of cost with cost of the solution path if

search space is known. Can be infinite in case of the agent

accidentally reaches dead ends


The adversary argument

Assume an adversary who can construct the state space while the agent explores it Visited states S and A. What next?

Fails in one of the state spaces No algorithm can avoid dead ends in all state spaces.


Online search agents

The agent maintains a map of the environment.Updated based on percept input.This map is used to decide next action.

Note difference with e.g. A*An online version can only expand the node it is physically in

(local order)


Online DF-searchfunction ONLINE_DFS-AGENT(s’) return an action

input: s’, a percept identifying current statestatic: result, a table indexed by action and state, initially empty

unexplored, a table that lists for each visited state, the action not yet triedunbacktracked, a table that lists for each visited state, the backtrack not yet trieds,a, the previous state and action, initially null

if GOAL-TEST(s’) then return stopif s’ is a new state then unexplored[s’] ← ACTIONS(s’)if s is not null then do

result[a,s] ← s’add s to the front of unbackedtracked[s’]

if unexplored[s’] is empty thenif unbacktracked[s’] is empty then return stopelse a ← an action b such that result[b, s’]=POP(unbacktracked[s’])

else a ← POP(unexplored[s’])s ← s’return a


Online DF-search, example

Assume maze problem on 3x3 grid.

s’ = (1,1) is initial state Result, unexplored (UX),

unbacktracked (UB), … are empty

S,a are also empty


Online DF-search, example GOAL-TEST((,1,1))?

S not = G thus false (1,1) a new state?

True ACTION((1,1)) -> UX[(1,1)]

{RIGHT,UP}

s is null? True (initially)

UX[(1,1)] empty? False

POP(UX[(1,1)])->a A=UP

s = (1,1) Return a

S’=(1,1)



GOAL-TEST((2,1))? S not = G thus false

(2,1) a new state? True ACTION((2,1)) -> UX[(2,1)]

{DOWN}

s is null? false (s=(1,1)) result[UP,(1,1)] <- (2,1) UB[(2,1)]={(1,1)}


A=DOWN, s=(2,1) return A

S

S’=(2,1)


Online DF-search, example GOAL-TEST((1,1))?


false s is null?

false (s=(2,1)) result[DOWN,(2,1)] <- (1,1) UB[(1,1)]={(2,1)}


A=RIGHT, s=(1,1) return A

S

S’=(1,1)



GOAL-TEST((1,2))? S not = G thus false

(1,2) a new state? True, UX[(1,2)]={RIGHT,UP,LEFT}

s is null? false (s=(1,1)) result[RIGHT,(1,1)] <- (1,2) UB[(1,2)]={(1,1)}


A=LEFT, s=(1,2) return AS

S’=(1,2)


Online DF-search, example GOAL-TEST((1,1))?


false s is null?

false (s=(1,2)) result[LEFT,(1,2)] <- (1,1) UB[(1,1)]={(1,2),(2,1)}

UX[(1,1)] empty? True UB[(1,1)] empty? False

A= b for b in result[b,(1,1)]=(1,2) B=RIGHT

A=RIGHT, s=(1,1) …S

S’=(1,1)


Online DF-search Worst case each node is visited twice. An agent can go on a long walk even

when it is close to the solution. An online iterative deepening

approach solves this problem. Online DF-search works only when

actions are reversible.


Online local search Hill-climbing is already online

One state is stored. Bad performancd due to local maxima

Random restarts impossible. Solution: Random walk introduces exploration (can produce exponentially many

steps)


Online local search

Solution 2: Add memory to hill climber Store current best estimate H(s) of cost to reach goal H(s) is initially the heuristic estimate h(s) Afterward updated with experience (see below)

Learning real-time A* (LRTA*)


Learning real-time A*function LRTA*-COST(s,a,s’,H) return an cost estimate

if s’ is undefined the return h(s)else return c(s,a,s’) + H[s’]

function LRTA*-AGENT(s’) return an actioninput: s’, a percept identifying current statestatic: result, a table indexed by action and state, initially empty

H, a table of cost estimates indexed by state, initially emptys,a, the previous state and action, initially null

if GOAL-TEST(s’) then return stopif s’ is a new state (not in H) then H[s’] ← h(s’)unless s is null

result[a,s] ← s’H[s] ← MIN LRTA*-COST(s,b,result[b,s],H)

b ∈ ACTIONS(s)

a ← an action b in ACTIONS(s’) that minimizes LRTA*-COST(s’,b,result[b,s’],H)s ← s’return a

Artificial Intelligence: Constraint Satisfaction problems



Outline

CSP?Backtracking for CSPLocal search for CSPsProblem structure and decomposition


Constraint satisfaction problems

What is a CSP?

Finite set of variables V1, V2, …, Vn Finite set of variables C1, C2, …, Cm Non-emtpy domain of possible values for each variable

DV1, DV2, … DVn Each constraint Ci limits the values that variables can take, e.g., V1 ≠ V2

A state is defined as an assignment of values to some or all variables.

Consistent assignment: assignment does not violate the constraints.


Constraint satisfaction problems

An assignment is complete when every value is mentioned. A solution to a CSP is a complete assignment that satisfies all

constraints. Some CSPs require a solution that maximizes an objective function. Applications: Scheduling the time of observations on the Hubble

Space Telescope, Floor planning, Map coloring, Cryptography


CSP example: map coloring

Variables: WA, NT, Q, NSW, V, SA, T Domains: Di={red,green,blue} Constraints:adjacent regions must have different colors.

E.g. WA ≠ NT (if the language allows this) E.g. (WA,NT) ≠ {(red,green),(red,blue),(green,red),…}


CSP example: map coloring

Solutions are assignments satisfying all constraints, e.g.{WA=red,NT=green,Q=red,NSW=green,V=red,SA=blue,T=green}


Constraint graph

CSP benefits Standard representation pattern Generic goal and successor functions Generic heuristics (no domain specific

expertise).

Constraint graph = nodes are variables, edges show constraints.

Graph can be used to simplify search.e.g. Tasmania is an independent sub-problem.


Varieties of CSPs

Discrete variables Finite domains; size d ⇒O(dn) complete assignments.

E.g. Boolean CSPs, include. Boolean satisfiability (NP-complete).

Infinite domains (integers, strings, etc.) E.g. job scheduling, variables are start/end days for each job Need a constraint language e.g StartJob1 +5 ≤ StartJob3. Linear constraints solvable, nonlinear undecidable.

Continuous variables e.g. start/end times for Hubble Telescope observations. Linear constraints solvable but time costy.


Varieties of constraints

Unary constraints involve a single variable. e.g. SA ≠ green

Binary constraints involve pairs of variables.

e.g. SA ≠ WA Higher-order constraints involve 3 or more variables.

e.g. cryptharithmetic column constraints. Preference (soft constraints) e.g. red is better than green often

representable by a cost for each variable assignment → constrained optimization problems.


CSP as a standard search problem

A CSP can easily expressed as a standard search problem.

Incremental formulation Initial State: the empty assignment {}.Successor function: Assign value to unassigned variable

provided that there is not conflict.Goal test: the current assignment is complete.Path cost: as constant cost for every step.


CSP as a standard search problem

This is the same for all CSP’s !!! Solution is found at depth n (if there are n variables).

Hence depth first search can be used. Path is irrelevant, so complete state representation can also be

used. Branching factor b at the top level is nd. b=(n-l)d at depth l, hence n!dn leaves (only dn complete

assignments).


Commutativity

CSPs are commutative.The order of any given set of actions has no effect on

the outcome.Example: choose colors for Australian territories one at

a time [WA=red then NT=green] same as [NT=green then

WA=red]All CSP search algorithms consider a single variable

assignment at a time ⇒ there are dn leaves.


Backtracking search

Cfr. Depth-first search Chooses values for one variable at a time and

backtracks when a variable has no legal values left to assign.

Uninformed algorithmNo good general performance


Backtracking search

function BACKTRACKING-SEARCH(csp) return a solution or failurereturn RECURSIVE-BACKTRACKING({} , csp)

function RECURSIVE-BACKTRACKING(assignment, csp) return a solution or failureif assignment is complete then return assignmentvar ← SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assignment,csp)for each value in ORDER-DOMAIN-VALUES(var, assignment, csp) do

if value is consistent with assignment according to CONSTRAINTS[csp] thenadd {var=value} to assignment result ← RRECURSIVE-BACTRACKING(assignment, csp)if result ≠ failure then return resultremove {var=value} from assignment

return failure


Backtracking example


Improving backtracking efficiency

Previous improvements → introduce heuristicsGeneral-purpose methods can give huge gains

in speed:Which variable should be assigned next? In what order should its values be tried?Can we detect inevitable failure early?Can we take advantage of problem structure?


Minimum remaining values

var ← SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assignment,csp)

A.k.a. most constrained variable heuristic Rule: choose variable with the fewest legal moves Which variable shall we try first?


Degree heuristic

Use degree heuristic Rule: select variable that is involved in the largest number of constraints on other

unassigned variables. Degree heuristic is very useful as a tie breaker. In what order should its values be tried?


Least constraining value

Least constraining value heuristic Rule: given a variable choose the least constraing value i.e. the one that

leaves the maximum flexibility for subsequent variable assignments.


Forward checking

Can we detect inevitable failure early? And avoid it later?

Forward checking idea: keep track of remaining legal values for unassigned variables.

Terminate search when any variable has no legal values.


Forward checking

Assign {WA=red} Effects on other variables connected by constraints with WA

NT can no longer be red SA can no longer be red


Forward checking

Assign {Q=green} Effects on other variables connected by constraints with WA

NT can no longer be green NSW can no longer be green SA can no longer be green

MRV heuristic will automatically select NT and SA next, why?


Forward checking

If V is assigned blue Effects on other variables connected by constraints with WA

SA is empty NSW can no longer be blue

FC has detected that partial assignment is inconsistent with the constraints and backtracking can occur.


Example: 4-Queens Problem

1

3

2

4

32 41

X1{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

[4-Queens slides copied from B.J. Dorr CMSC 421 course on AI]



1

3

2

4

32 41

X1{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}



1

3

2

4

32 41

X1{1,2,3,4}

X3{ ,2, ,4}

X4{ ,2,3, }

X2{ , ,3,4}



1

3

2

4

32 41

X1{1,2,3,4}

X3{ , , , }

X4{ ,2,3, }

X2{ , ,3,4}



1

3

2

4

32 41

X1{ ,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}



1

3

2

4

32 41

X1{ ,2,3,4}

X3{1, ,3, }

X4{1, ,3,4}

X2{ , , ,4}



1

3

2

4

32 41

X1{ ,2,3,4}

X3{1, , , }

X4{1, ,3, }

X2{ , , ,4}



1

3

2

4

32 41

X1{ ,2,3,4}

X3{1, , , }

X4{ , ,3, }

X2{ , , ,4}


Constraint propagation

Solving CSPs with combination of heuristics plus forward checking is more efficient than either approach alone.

FC checking propagates information from assigned to unassigned variables but does not provide detection for all failures. NT and SA cannot be blue!

Constraint propagation repeatedly enforces constraints locally


Arc consistency

X → Y is consistent ifffor every value x of X there is some allowed y

SA → NSW is consistent iffSA=blue and NSW=red


Arc consistency

X → Y is consistent ifffor every value x of X there is some allowed y

NSW → SA is consistent iffNSW=red and SA=blueNSW=blue and SA=???

Arc can be made consistent by removing blue from NSW


Arc consistency

Arc can be made consistent by removing blue from NSW RECHECK neighbours !!

Remove red from V


Arc consistency

Arc can be made consistent by removing blue from NSW RECHECK neighbours !!

Remove red from V Arc consistency detects failure earlier than FC Can be run as a preprocessor or after each assignment.

Repeated until no inconsistency remains


Arc consistency algorithm

function AC-3(csp) return the CSP, possibly with reduced domainsinputs: csp, a binary csp with variables {X1, X2, …, Xn}local variables: queue, a queue of arcs initially the arcs in csp

while queue is not empty do(Xi, Xj) ← REMOVE-FIRST(queue)if REMOVE-INCONSISTENT-VALUES(Xi, Xj) then

for each Xk in NEIGHBORS[Xi ] doadd (Xi, Xj) to queue

function REMOVE-INCONSISTENT-VALUES(Xi, Xj) return true iff we remove a valueremoved ← falsefor each x in DOMAIN[Xi] do

if no value y in DOMAIN[Xi] allows (x,y) to satisfy the constraints between Xi and Xj

then delete x from DOMAIN[Xi]; removed ← truereturn removed


K-consistency

Arc consistency does not detect all inconsistencies: Partial assignment {WA=red, NSW=red} is inconsistent.

Stronger forms of propagation can be defined using the notion of k-consistency.

A CSP is k-consistent if for any set of k-1 variables and for any consistent assignment to those variables, a consistent value can always be assigned to any kth variable. E.g. 1-consistency or node-consistency E.g. 2-consistency or arc-consistency E.g. 3-consistency or path-consistency


K-consistency A graph is strongly k-consistent if

It is k-consistent and Is also (k-1) consistent, (k-2) consistent, … all the way down to 1-

consistent.

This is ideal since a solution can be found in time O(nd) instead of O(n2d3)

YET we didn’t win already: any algorithm for establishing n-consistency must take time exponential in n, in the worst case.


Further improvements

Checking special constraints Checking Alldif(…) constraint

E.g. {WA=red, NSW=red}

Checking Atmost(…) constraint Bounds propagation for larger value domains

Intelligent backtracking Standard form is chronological backtracking i.e. try different value for

preceding variable. More intelligent, backtrack to conflict set.

Set of variables that caused the failure or set of previously assigned variables that are connected to X by constraints.

Backjumping moves back to most recent element of the conflict set. Forward checking can be used to determine conflict set.


Local search for CSP

Use complete-state representation For CSPs

allow states with unsatisfied constraints operators reassign variable values

Variable selection: randomly select any conflicted variable Value selection: min-conflicts heuristic

Select new value that results in a minimum number of conflicts with the other variables


Local search for CSP

function MIN-CONFLICTS(csp, max_steps) return solution or failureinputs: csp, a constraint satisfaction problem

max_steps, the number of steps allowed before giving up

current ← an initial complete assignment for cspfor i = 1 to max_steps do

if current is a solution for csp then return currentvar ← a randomly chosen, conflicted variable from VARIABLES[csp]value ← the value v for var that minimize CONFLICTS(var,v,current,csp)set var = value in current

return failure


Min-conflicts example 1

Use of min-conflicts heuristic in hill-climbing.h=5 h=3 h=1


Min-conflicts example 2

A two-step solution for an 8-queens problem using min-conflicts heuristic. At each stage a queen is chosen for reassignment in its column. The algorithm moves the queen to the min-conflict square breaking ties randomly.


Advantages of local search

The runtime of min-conflicts is roughly independent of problem size.Solving the millions-queen problem in roughly 50 steps.

Local search can be used in an online setting.Backtrack search requires more time


Problem structure

How can the problem structure help to find a solution quickly? sub-problem identification is important:

Coloring Tasmania and mainland are independent sub-problems Identifiable as connected components of constrained graph.

Improves performance


Problem structure

Suppose each problem has c variables out of a total of n. Worst case solution cost is O(n/c dc), i.e. linear in n

Instead of O(d n), exponential in n E.g. n= 80, c= 20, d=2

280 = 4 billion years at 1 million nodes/sec. 4 * 220= .4 second at 1 million nodes/sec


Tree-structured CSPs

Theorem: if the constraint graph has no loops then CSP can be solved in O(nd 2) time

Compare difference with general CSP, where worst case is O(d n)


Tree-structured CSPs

In most cases sub-problems of a CSP are connected as a tree Any tree-structured CSP can be solved in time linear in the number of variables.

Choose a variable as root, order variables from root to leaves such that every node’s parent precedes it in the ordering. (label var from X1 to Xn)

For j from n down to 2, apply REMOVE-INCONSISTENT-VALUES(Parent(Xj),Xj) For j from 1 to n assign Xj consistently with Parent(Xj )


Nearly tree-structured CSPs

Can more general constraint graphs be reduced to trees? Two approaches:

Remove certain nodes Collapse certain nodes



Idea: assign values to some variables so that the remaining variables form a tree. Assume that we assign {SA=x} ← cycle cutset

And remove any values from the other variables that are inconsistent. The selected value for SA could be the wrong one so we have to try all of them



This approach is worthwhile if cycle cutset is small. Finding the smallest cycle cutset is NP-hard

Approximation algorithms exist This approach is called cutset conditioning.



Tree decomposition of the constraint graph in a set of connected sub-problems.

Each sub-problem is solved independently

Resulting solutions are combined. Necessary requirements:

Every variable appears in ar least one of the sub-problems.

If two variables are connected in the original problem, they must appear together in at least one sub-problem.

If a variable appears in two sub-problems, it must appear in each node on the path.


CSP and sudoku

Variabes? 81

Domains? 1-9

Constraints?There may not be 2

numbers in the same row and column and in a 3x3 group box.


Summary

CSPs are a special kind of problem: states defined by values of a fixed set of variables, goal test defined by constraints on variable values

Backtracking=depth-first search with one variable assigned per node Variable ordering and value selection heuristics help significantly Forward checking prevents assignments that lead to failure. Constraint propagation does additional work to constrain values and

detect inconsistencies. The CSP representation allows analysis of problem structure. Tree structured CSPs can be solved in linear time. Iterative min-conflicts is usually effective in practice.

Artificial Intelligence: game playing



Outline

What are games? Optimal decisions in gamesWhich strategy leads to success?

α-β pruning Games of imperfect information Games that include an element of chance


What are and why study games?

Games are a form of multi-agent environment What do other agents do and how do they affect our success? Cooperative vs. competitive multi-agent environments. Competitive multi-agent environments give rise to adversarial search

Why study games? Fun; historically entertaining Interesting subject of study because they are hard problems Easy to represent and agents restricted to small number of actions


Relation of Games to Search

Search – no adversary Solution is (heuristic) method for finding goal Heuristics and CSP techniques can find optimal solution Evaluation function: estimate of cost from start to goal through given node Examples: path planning, scheduling activities

Games – adversary Solution is strategy (strategy specifies move for every possible opponent reply). Time limits force an approximate solution Evaluation function: evaluate “goodness” of game position Examples: chess, checkers, Othello, backgammon


Types of Games


Game setup

Two players: MAX and MIN MAX moves first and they take turns until the game is over.

Winner gets award, looser gets penalty. Games as search:

Initial state: e.g. board configuration of chess (TICTACTOE) Successor function: list of (move,state) pairs specifying legal moves. Terminal test: Is the game finished? Utility function: Gives numerical value of terminal states. E.g. win (+1), loose

(-1) and draw (0) in tic-tac-toe (next) MAX uses search tree to determine next move.


Partial Game Tree for Tic-Tac-Toe


Optimal strategies

Find the contingent strategy for MAX assuming an infallible MIN opponent.

Assumption: Both players play optimally !! Given a game tree, the optimal strategy can be determined

by using the minimax value of each node:

MINIMAX-VALUE(n)=UTILITY(n) If n is a terminalmaxs ∈ successors(n) MINIMAX-VALUE(s) If n is a max nodemins ∈ successors(n) MINIMAX-VALUE(s) If n is a min node


Two-Ply Game Tree


Two-Ply Game Tree

The minimax decision

Minimax maximizes the worst-case outcome for max.


What if MIN does not play optimally?

Definition of optimal play for MAX assumes MIN plays optimally: maximizes worst-case outcome for MAX.

But if MIN does not play optimally, MAX will do even better. [proven.]


Minimax Algorithmfunction MINIMAX-DECISION(state) returns an action

inputs: state, current state in gamev←MAX-VALUE(state)return the action in SUCCESSORS(state) with value v

function MIN-VALUE(state) returns a utility valueif TERMINAL-TEST(state) then return UTILITY(state)v ← ∞for a,s in SUCCESSORS(state) do

v ← MIN(v,MAX-VALUE(s))return v

function MAX-VALUE(state) returns a utility valueif TERMINAL-TEST(state) then return UTILITY(state)v ← ∞for a,s in SUCCESSORS(state) do

v ← MAX(v,MIN-VALUE(s))return v


Properties of Minimax

Criterion Minimax

Time O(bm)

Space O(bm)


Multiplayer games

Games allow more than two players Single minimax values become vectors


Problem of minimax search

Number of games states is exponential to the number of moves.Solution: Do not examine every node ==> Alpha-beta pruningRemove branches that do not influence final decision

Revisit example …


Alpha-Beta Example

[-∞, +∞]

[-∞,+∞]

Range of possible values

Do DF-search until first leaf


Alpha-Beta Example (continued)

[-∞,3]

[-∞,+∞]



[-∞,3]

[-∞,+∞]



[3,+∞]

[3,3]



[-∞,2]

[3,+∞]

[3,3]

This node is worse for MAX



[-∞,2]

[3,14]

[3,3] [-∞,14]

,



[−∞,2]

[3,5]

[3,3] [-∞,5]

,



[2,2][−∞,2]

[3,3]

[3,3]



[2,2][-∞,2]

[3,3]

[3,3]


Alpha-Beta Algorithmfunction ALPHA-BETA-SEARCH(state) returns an action

inputs: state, current state in gamev←MAX-VALUE(state, - ∞ , +∞)return the action in SUCCESSORS(state) with value v

function MAX-VALUE(state,α , β) returns a utility valueif TERMINAL-TEST(state) then return UTILITY(state)v ← - ∞for a,s in SUCCESSORS(state) do

v ← MAX(v,MIN-VALUE(s, α , β))if v ≥ β then return vα ← MAX(α ,v)

return v


Alpha-Beta Algorithm

function MIN-VALUE(state, α , β) returns a utility valueif TERMINAL-TEST(state) then return UTILITY(state)v ← + ∞for a,s in SUCCESSORS(state) do

v ← MIN(v,MAX-VALUE(s, α , β))if v ≤ α then return vβ ← MIN(β ,v)

return v


General alpha-beta pruning

Consider a node n somewhere in the tree

If player has a better choice at Parent node of n Or any choice point further up

n will never be reached in actual play.

Hence when enough is known about n, it can be pruned.


Final Comments about Alpha-Beta Pruning

Pruning does not affect final results Entire subtrees can be pruned. Good move ordering improves effectiveness of pruning With “perfect ordering,” time complexity is O(bm/2)

Branching factor of sqrt(b) !! Alpha-beta pruning can look twice as far as minimax in the same amount of

time Repeated states are again possible.

Store them in memory = transposition table


Games of imperfect information

Minimax and alpha-beta pruning require too much leaf-node evaluations.

May be impractical within a reasonable amount of time. SHANNON (1950):

Cut off search earlier (replace TERMINAL-TEST by CUTOFF-TEST)

Apply heuristic evaluation function EVAL (replacing utility function of alpha-beta)


Cutting off search

Change: if TERMINAL-TEST(state) then return UTILITY(state)

into if CUTOFF-TEST(state,depth) then return EVAL(state)

Introduces a fixed-depth limit depth Is selected so that the amount of time will not exceed what the rules of

the game allow. When cuttoff occurs, the evaluation is performed.


Heuristic EVAL

Idea: produce an estimate of the expected utility of the game from a given position.

Performance depends on quality of EVAL. Requirements:

EVAL should order terminal-nodes in the same way as UTILITY. Computation may not take too long. For non-terminal states the EVAL should be strongly correlated with

the actual chance of winning. Only useful for quiescent (no wild swings in value in near future)

states


Heuristic EVAL example

Eval(s) = w1 f1(s) + w2 f2(s) + … + wnfn(s)


Heuristic EVAL example

Eval(s) = w1 f1(s) + w2 f2(s) + … + wnfn(s)

Addition assumes independence


Heuristic difficulties

Heuristic counts pieces won


Horizon effectFixed depth search thinks it can avoidthe queening move


Games that include chance

Possible moves (5-10,5-11), (5-11,19-24),(5-10,10-16) and (5-11,11-16)



Possible moves (5-10,5-11), (5-11,19-24),(5-10,10-16) and (5-11,11-16) [1,1], [6,6] chance 1/36, all other chance 1/18

chance nodes



[1,1], [6,6] chance 1/36, all other chance 1/18 Can not calculate definite minimax value, only expected value


Expected minimax valueEXPECTED-MINIMAX-VALUE(n)=

UTILITY(n) If n is a terminalmaxs ∈ successors(n) MINIMAX-VALUE(s) If n is a max nodemins ∈ successors(n) MINIMAX-VALUE(s) If n is a max node∑s ∈ successors(n) P(s) . EXPECTEDMINIMAX(s) If n is a chance node

These equations can be backed-up recursively all the way to the root of the game tree.


Position evaluation with chance nodes

Left, A1 wins Right A2 wins Outcome of evaluation function may not change when values are scaled differently. Behavior is preserved only by a positive linear transformation of EVAL.


Discussion

Examine section on state-of-the-art games yourself Minimax assumes right tree is better than left, yet …

Return probability distribution over possible values Yet expensive calculation


Discussion

Utility of node expansion Only expand those nodes which lead to significanlty better moves

Both suggestions require meta-reasoning


Summary

Games are fun (and dangerous) They illustrate several important points about AIPerfection is unattainable -> approximationGood idea what to think aboutUncertainty constrains the assignment of values to states

Games are to AI as grand prix racing is to automobile design.

Artificial Intelligence: Logic agents



“Thinking Rationally”

Computational models of human “thought” processes

Computational models of human behavior

Computational systems that “think” rationally

Computational systems that behave rationally


Logical Agents

Reflex agents find their way from Arad to Bucharest by dumb luck Chess program calculates legal moves of its king, but doesn’t know

that no piece can be on 2 different squares at the same time Logic (Knowledge-Based) agents combine general knowledge with

current percepts to infer hidden aspects of current state prior to selecting actions Crucial in partially observable environments


Outline

Knowledge-based agents Wumpus world Logic in general Propositional and first-order logic Inference, validity, equivalence and satisfiabilityReasoning patterns

Resolution Forward/backward chaining


Knowledge Base

Knowledge Base : set of sentences represented in a knowledge representation language and represents assertions about the world.

Inference rule: when one ASKs questions of the KB, the answer should follow from what has been TELLed to the KB previously.

tell ask


Generic KB-Based Agent


Abilities KB agent

Agent must be able to:Represent states and actions,Incorporate new perceptsUpdate internal representation of the worldDeduce hidden properties of the worldDeduce appropriate actions


Description level

The KB agent is similar to agents with internal state Agents can be described at different levelsKnowledge level

What they know, regardless of the actual implementation. (Declarative description)

Implementation level Data structures in KB and algorithms that manipulate them e.g

propositional logic and resolution.


A Typical Wumpus World

Wumpus


Wumpus World PEAS Description


Wumpus World Characterization

Observable? Deterministic? Episodic? Static? Discrete? Single-agent?



Observable? No, only local perception Deterministic? Episodic? Static? Discrete? Single-agent?



Observable? No, only local perception Deterministic? Yes, outcome exactly specified Episodic? Static? Discrete? Single-agent?



Observable? No, only local perception Deterministic? Yes, outcome exactly specified Episodic? No, sequential at the level of actions Static? Discrete? Single-agent?



Observable? No, only local perception Deterministic? Yes, outcome exactly specified Episodic? No, sequential at the level of actions Static? Yes, Wumpus and pits do not move Discrete? Single-agent?



Observable? No, only local perception Deterministic? Yes, outcome exactly specified Episodic? No, sequential at the level of actions Static? Yes, Wumpus and pits do not move Discrete? Yes Single-agent?



Observable? No, only local perception Deterministic? Yes, outcome exactly specified Episodic? No, sequential at the level of actions Static? Yes, Wumpus and pits do not move Discrete? Yes Single-agent? Yes, Wumpus is essentially a

natural feature.


Exploring the Wumpus World

[1,1] The KB initially contains the rules of the environment. The first percept is [none, none,none,none,none], move to safe cell e.g. 2,1

[2,1] breeze which indicates that there is a pit in [2,2] or [3,1], return to [1,1] to try next safe cell



[1,2] Stench in cell which means that wumpus is in [1,3] or [2,2]YET … not in [1,1]YET … not in [2,2] or stench would have been detected in [2,1]THUS … wumpus is in [1,3]THUS [2,2] is safe because of lack of breeze in [1,2]THUS pit in [1,3]move to next safe cell [2,2]



[2,2] move to [2,3][2,3] detect glitter , smell, breeze

THUS pick up goldTHUS pit in [3,3] or [2,4]


What is a logic?

A formal languageSyntax – what expressions are legal (well-formed)Semantics – what legal expressions mean

in logic the truth of each sentence with respect to each possible world.

E.g the language of arithmeticX+2 >= y is a sentence, x2+y is not a sentenceX+2 >= y is true in a world where x=7 and y =1X+2 >= y is false in a world where x=0 and y =6


Entailment

One thing follows from anotherKB |= α

KB entails sentence α if and only if α is true in worlds where KB is true.

Ε.g. x+y=4 entails 4=x+y Entailment is a relationship between sentences that is based on

semantics.


Models

Logicians typically think in terms of models, which are formally structured worlds with respect to which truth can be evaluated.

m is a model of a sentence α if α is true in m M(α) is the set of all models of α


Wumpus world model


Logical inference

The notion of entailment can be used for logic inference. Model checking (see wumpus example): enumerate all possible models and

check whether α is true. If an algorithm only derives entailed sentences it is called sound or truth

preserving. Otherwise it just makes things up.

i is sound if whenever KB |-i α it is also true that KB|= α Completeness : the algorithm can derive any sentence that is entailed.

i is complete if whenever KB |= α it is also true that KB|-i α


Schematic perspective

If KB is true in the real world, then any sentence α derivedFrom KB by a sound inference procedure is also true in the real world.

Artificial Intelli-gence: logic agents


February 4, 2012 2

Standard Logical Equivalences

February 4, 2012 3

Terminology

A sentence is valid if its truth value is t in all interpretationsValid sentences: true, ¬ false, P ∨¬ P

A sentence is satisfiable if its truth value is t in at least one interpretationSatisfiable sentences: P, true, ¬ P

A sentence is unsatisfiable iff its truth value is fin all interpretationsUnsatisfiable sentences: P ∧ ¬ P, false, ¬ true

February 4, 2012 4

Inference

KB ⊢i αSoundness: Inference procedure i is

sound if whenever KB ⊢ i α, it is also true that KB ⊧ αCompleteness: Inference procedure i is

complete if whenever KB ⊧ α, it is also true that KB ⊢ i α

February 4, 2012 5

Rules of Inference

α ⊢ β α

β Valid Rules of Inference:

Modus Ponens And-Elimination And-Introduction Or-Introduction Double Negation Unit Resolution Resolution

February 4, 2012 6

Searching for Proofs

Finding proofs is exactly like finding solutions to search problems.

Can search forward (forward chaining) to derive goal or search backward (backward chaining) from the goal.

Searching for proofs is not more efficient than enumerating models, but in many practical cases, it’s more efficient because we can ignore irrelevant propositions

February 4, 2012 7

Full Resolution Rule Revisited

Start with Unit Resolution Inference Rule:

Full Resolution Rule is a generalization of this rule:

For clauses of length two:

February 4, 2012 8

Resolution: Complete Inference ProcedureAny complete search algorithm, applying

only the resolution rule, can derive any conclusion entailed by any knowledge base in propositional logic.Refutation completeness: Resolution can

always be used to either confirm or refute a sentence, but it cannot be used to enumerate true sentences.

February 4, 2012 9

Horn Clauses

Real World KB’s are often a conjunction of Horn clausesHorn clause =proposition symbol; or(conjunction of symbols) ⇒ symbol

February 4, 2012 10

Forward Chaining

Fire any rule whose premises are satisfied in the KB.Add its conclusion to the KB until query is

found.

February 4, 2012 11

Backward Chaining

Motivation: Need goal-directed reasoning in order to keep from getting overwhelmed with irrelevant consequencesMain idea:Work backwards from query q To prove q: Check if q is known alreadyProve by backward chaining all premises of some

rule concluding q

February 4, 2012 12

Forward Chaining vs. Backward ChainingFC is data-driven—it may do lots of work

irrelevant to the goalBC is goal-driven—appropriate for

problem-solving

Artificial Intelligence: Agents and Propositional Logic.



Propositional Logic


Wumpus world logic


Truth tables for inference

Enumerate the models and check that α is true in every modelIn which KB is true.


Inference by enumeration

Depth-first enumeration of all models is sound and complete

For n symbols, time complexity is O(2n), space complexity is O(n).


Logical equivalence

Two sentences are logically equivalent iff true in same set of models or α ≡ ß iff α |= ß and ß |= α.


Validity and satisfiability

A sentence is valid if it is true in all models, e.g., True, A ∨¬A, A ⇒ A, (A ∧ (A ⇒ B)) ⇒ B

Validity is connected to inference via the Deduction Theorem: KB |=α if and only if (KB ⇒ α) is valid

A sentence is satisfiable if it is true in some model e.g., A ∨ B, C

A sentence is unsatisfiable if it is true in no models e.g., A∧¬A

Satisfiability is connected to inference via the following: KB |= α if and only if (KB ∧ ¬α ) is unsatisfiable Remember proof by contradiction.


Inference rules in PL

Modens Ponens

And-elimination: from a conjuction any conjunction can be inferred:

All logical equivalences can be used as inference rules.

α ⇒ β,αβ

α ∧βα

α ⇔ β(α ⇒ β) ∧ (β ⇒ α)


Example

Assume R1 through R5:

How can we prove ¬P1,2?

¬P1,1,B1,1 ⇒ P1,1 ∨ P2,1( ),B2,1 ⇒ P1,1 ∨ P2,2 ∨ P3,1( ),¬B1,1,B2,1

R6 : B1,1 ⇒ P1,2 ∨ P2,1( )( )∧ P1,2 ∨ P2,1( )⇒ B1,1( )R7 : P1,2 ∨ P2,1( )⇒ B1,1

R8 :¬B1,1 ⇒ ¬ P1,2 ∨ P2,1( )R9 :¬ P1,2 ∨ P2,1( )R10 :¬P1,2 ∧¬P2,1

Biconditional elim.

And elim.

Contraposition

Modens ponens

Morgan’s rule


Searching for proofs

Finding proofs is exactly like finding solutions to search problems. Search can be done forward (forward chaining) to derive goal or backward

(backward chaining) from the goal. Searching for proofs is not more efficient than enumerating models, but in many

practical cases, it is more efficient because we can ignore irrelevant properties. Monotonicity: the set of entailed sentences can only increase as information is

added to the knowledge base.

for any sentenceα andβ : if KB |= α then KB ∧β |= α


Proof methods

Proof methods divide into (roughly) two kinds:Application of inference rules

Legitimate (sound) generation of new sentences from old Proof = a sequence of inference rule application can use inference

rules as operators in a standard search algorithm Typically require transformation of sentences into a normal form

Model checking truth table enumeration (always exponential in n) improved backtracking, e.g., Davis--Putnam-Logemann-Loveland

(DPLL) heuristic search in model space (sound but incomplete)

e.g., min-conflicts-like hill-climbing algorithms


Resolution

Start with Unit Resolution Inference Rule:

Full Resolution Rule is a generalization of this rule:

For clauses of length two:


Resolution in Wumpus world

At some point we can derive the absence of a pit in square 2,2:

Now after biconditional elimination of R3 followed by a modens ponens with R5:

Resolution :

¬P1,1,B1,1 ⇒ P1,1 ∨ P2,1( ),B2,1 ⇒ P1,1 ∨ P2,2 ∨ P3,1( ),¬B1,1,B2,1,¬P2,2,¬P3,1

R15 : P1,1 ∨ P2,2 ∨ P3,1( )

P1,1 ∨ P2,2 ∨ P3,1 ¬P2,2

P1,1 ∨ P3,1


Resolution

Uses CNF (Conjunctive normal form) Conjunction of disjunctions of literals (clauses)

The resolution rule is sound: Only entailed sentences are derived

Resolution is complete in the sense that it can always be used to either confirm or refute a sentence (it can not be used to enumerate true sentences.)


Conversion to CNFB1,1 ⇔ (P1,2 ∨ P2,1)

• Eliminate ⇔, replacing α ⇔ ß with (α ⇒ ß)∧(ß ⇒ α).• (B1,1 ⇒ (P1,2 ∨ P2,1)) ∧ ((P1,2 ∨ P2,1) ⇒ B1,1)

Eliminate ⇒, replacing α ⇒ ß with ¬ α ∨ ß. (¬B1,1 ∨ P1,2 ∨ P2,1) ∧ (¬(P1,2 ∨ P2,1) ∨ B1,1)

Move ¬ inwards using de Morgan's rules and double-negation: (¬B1,1 ∨ P1,2 ∨ P2,1) ∧ ((¬P1,2 ∨ ¬P2,1) ∨ B1,1)

Apply distributivity law (∧ over ∨) and flatten: (¬B1,1 ∨ P1,2 ∨ P2,1) ∧ (¬P1,2 ∨ B1,1) ∧ (¬P2,1 ∨ B1,1)


Resolution algorithm

Proof by contradiction, i.e., show KB∧¬ α unsatisfiable


Resolution algorithm

First KB∧¬ α is converted into CNF Then apply resolution rule to resulting clauses. The process continues until:There are no new clauses that can be added

Hence α does not ential ß

Two clauses resolve to entail the empty clause. Hence α does ential ß


Resolution example

KB = (B1,1 ⇔ (P1,2∨ P2,1)) ∧¬ B1,1 = ¬P1,2


Forward and backward chaining

The completeness of resolution makes it a very important inference model.

Real-world knowledge only requires a restricted form of clauses: Horn clauses = disjunction of literals with at most one positive literal Three important properties

Can be written as an implication

Inference through forward chaining and backward chaining. Deciding entailment can be done in a time linear size of the knowledge base.

¬L1,1 ∨¬Breeze ∨ B1,1( )→ L1,1 ∨ Breeze( )⇒ B1,1


Forward chaining

Idea: fire any rule whose premises are satisfied in the KB, add its conclusion to the KB, until query is found


Forward chaining algorithm

Forward chaining is sound and complete for Horn KB


Forward chaining example


Proof of completeness

FC derives every atomic sentence that is entailed by KB1. FC reaches a fixed point where no new atomic sentences are

derived.2. Consider the final state as a model m, assigning true/false to

symbols.3. Every clause in the original KB is true in m

a1 ∧ … ∧ ak ⇒ b4. Hence m is a model of KB5. If KB |= q, q is true in every model of KB, including m


Backward chaining

Idea: work backwards from the query q:to prove q by BC,

check if q is known already, orprove by BC all premises of some rule concluding q

Avoid loops: check if new subgoal is already on the goal stack

Avoid repeated work: check if new subgoal1. has already been proved true, or2. has already failed


Backward chaining example


Forward vs. backward chaining

FC is data-driven, automatic, unconscious processing, e.g., object recognition, routine decisions

May do lots of work that is irrelevant to the goal

BC is goal-driven, appropriate for problem-solving, e.g., Where are my keys? How do I get into a PhD program?

Complexity of BC can be much less than linear in size of KB


Effective propositional inference

Two families of efficient algorithms for propositional inference based on model checking:

Are used for checking satisfiability Complete backtracking search algorithms

DPLL algorithm (Davis, Putnam, Logemann, Loveland) Improves TT-Entails? Algorithm.

Incomplete local search algorithms WalkSAT algorithm


The DPLL algorithm Determine if an input propositional logic sentence (in CNF) is satisfiable. Improvements over truth table enumeration:

1. Early terminationA clause is true if any literal is true. A sentence is false if any clause is false.E.g. (¬B1,1 ∨ P1,2 ∨ P2,1) ∧ (¬P1,2 ∨ B1,1) ∧ (¬P2,1 ∨ B1,1)

2. Pure symbol heuristicPure symbol: always appears with the same "sign" in all clauses. e.g., In the three clauses (A ∨ ¬B), (¬B ∨ ¬C), (C ∨ A), A and B are pure, C is impure. Assign a pure symbol so that their literals are true.

3. Unit clause heuristicUnit clause: only one literal in the clause or only one literal which has not yet received a

value. The only literal in a unit clause must be true. First do this assignments before continuing with the rest (unit propagation!).


The DPLL algorithm


The WalkSAT algorithm

Incomplete, local search algorithm. Evaluation function: The min-conflict heuristic of minimizing the

number of unsatisfied clauses. Steps are taken in the space of complete assignments, flipping the

truth value of one variable at a time. Balance between greediness and randomness.

To avoid local minima


The WalkSAT algorithm


Hard satisfiability problems

Underconstrained problems are easy: e.g n-queens in CSP. In SAT: e.g.,(¬D ∨ ¬B ∨ C) ∧ (B ∨ ¬A ∨ ¬C) ∧ (¬C ∨ ¬B ∨ E) ∧(E ∨ ¬D ∨ B) ∧ (B ∨ E ∨ ¬C)

Increase in complexity by keeping the number of symbols fixed and increasing the amount of clauses.

m = number of clauses n = number of symbols

Hard problems seem to cluster near m/n = 4.3 (critical point)



Median runtime for 100 satisfiable random 3-CNF sentences, n = 50


Inference-based agents in the wumpus world

A wumpus-world agent using propositional logic ( a knowledge base about the physics of the W-world):

¬P1,1¬W1,1

Bx,y ⇔ (Px,y+1 ∨ Px,y-1 ∨ Px+1,y ∨ Px-1,y) Sx,y ⇔ (Wx,y+1 ∨ Wx,y-1 ∨ Wx+1,y ∨ Wx-1,y)

W1,1 ∨ W1,2 ∨ … ∨ W4,4 (at least one wumpus)¬W1,1 ∨ ¬W1,2 (at most one wumpus)¬W1,1 ∨ ¬W1,3…⇒ 64 distinct proposition symbols, 155 sentences⇒ A fringe square is provably safe if the sentence

is entailed by the knowledge base.

¬Pi, j ∧¬Wi, j( )


KB contains "physics" sentences for every single square With all consequences for large KB

Better would be to have just two sentences for breezes and stenches for all squares. Impossible for propositional logic.

Simplification in agent: location info is not in KB!! For every time t and every location [x,y],

Ltx,y ∧ FacingRightt ∧ Forwardt ⇒ Lt

x+1,y

PROBLEM: Rapid proliferation of clauses.

Expressiveness limitation of propositional logic


Summary Logical agents apply inference to a knowledge base to derive new information and

make decisions. Basic concepts of logic:

syntax: formal structure of sentences

semantics: truth of sentences wrt models

entailment: necessary truth of one sentence given another

inference: deriving sentences from other sentences

soundness: derivations produce only entailed sentences

completeness: derivations can produce all entailed sentences

Wumpus world requires the ability to represent partial and negated information, reason by cases, etc.

Resolution is complete for propositional logicForward, backward chaining are linear-time, complete for Horn clauses

Propositional logic lacks expressive power

Artificial Intelligence: Agents and First-Order

Logic



Outline

Why FOL? Syntax and semantics of FOL Using FOLWumpus world in FOL Knowledge engineering in FOL


Pros and cons of propositional logic

Propositional logic is declarative Propositional logic allows partial/disjunctive/negated information

(unlike most data structures and databases) Propositional logic is compositional:

meaning of B1,1 ∧ P1,2 is derived from meaning of B1,1 and of P1,2

Meaning in propositional logic is context-independent (unlike natural language, where meaning depends on context)

Propositional logic has very limited expressive power (unlike natural language) E.g., cannot say "pits cause breezes in adjacent squares“

except by writing one sentence for each square


First-order logic

Whereas propositional logic assumes the world contains facts,

first-order logic (like natural language) assumes the world containsObjects: people, houses, numbers, colors, baseball games,

wars, …Relations: red, round, prime, brother of, bigger than, part of,

comes between, …Functions: father of, best friend, one more than, plus, …


Logics in General

Ontological Commitment: What exists in the world — TRUTH PL : facts hold or do not hold. FL : objects with relatios between them that hold or do not hold

Epistemoligical Commitment: What an agent believes about facts —BELIEF


Syntax of FOL: Basic elements

Constants KingJohn, 2, NUS,... Predicates Brother, >,... Functions Sqrt, LeftLegOf,... Variables x, y, a, b,... Connectives ¬, ⇒, ∧, ∨, ⇔ Equality = Quantifiers ∀, ∃


Atomic sentences

Atomic sentence = predicate (term1,...,termn) or term1 = term2

Term = function (term1,...,termn) or constant or variable

E.g., Brother(KingJohn,RichardTheLionheart) > (Length(LeftLegOf(Richard)), Length(LeftLegOf(KingJohn)))


Complex sentences

Complex sentences are made from atomic sentences using connectives

¬S, S1 ∧ S2, S1 ∨ S2, S1 ⇒ S2, S1 ⇔S2,

E.g. Sibling(KingJohn,Richard) ⇒Sibling(Richard,KingJohn)>(1,2) ∨ ≤ (1,2)>(1,2) ∧ ¬ >(1,2)


Truth in first-order logic

Sentences are true with respect to a model and an interpretation

Model contains objects (domain elements) and relations among them

Interpretation specifies referents for

constant symbols → objectspredicate symbols → relationsfunction symbols → functional relations

An atomic sentence predicate(term1,...,termn) is trueiff the objects referred to by term1,...,termnare in the relation referred to by predicate.


Models for FOL: Example


Models for FOL

We can enumerate the models for a given KB vocabulary:

Computing entailment by enumerating the models will not be easy !!


Quantifiers

Allows us to express properties of collections of objects instead of enumerating objects by name

Universal: “for all” ∀ Existential: “there exists” ∃


Universal quantification

∀<variables> <sentence>

Everyone at VUB is smart:∀x At(x,VUB) ⇒ Smart(x)

∀x P is true in a model m iff P is true with x being each possible object in the model

Roughly speaking, equivalent to the conjunction of instantiations of P

At(KingJohn,VUB) ⇒ Smart(KingJohn) ∧At(Richard,VUB) ⇒ Smart(Richard) ∧At(VUB,VUB) ⇒ Smart(VUB) ∧ ...


A common mistake to avoid

Typically, ⇒ is the main connective with ∀A universally quantifier is also equivalent to a set of

implications over all objects Common mistake: using ∧ as the main

connective with ∀:∀x At(x, VUB) ∧ Smart(x)means “Everyone is at VUB and everyone is smart”


Existential quantification

∃<variables> <sentence>

Someone at VUB is smart:∃x At(x, VUB) ∧ Smart(x)

∃x P is true in a model m iff P is true with x being some possible object in the model

Roughly speaking, equivalent to the disjunction of instantiations of P

At(KingJohn,VUB) ∧ Smart(KingJohn) ∨ At(Richard,VUB) ∧ Smart(Richard) ∨ At(VUB, VUB) ∧ Smart(VUB) ∨ ...


Another common mistake to avoid

Typically, ∧ is the main connective with ∃

Common mistake: using ⇒ as the main connective with ∃:

∃x At(x, VUB) ⇒ Smart(x)is true even if there is anyone who is not at VUB!


Properties of quantifiers

∀x ∀y is the same as ∀y ∀x∃x ∃y is the same as ∃y ∃x

∃x ∀y is not the same as ∀y ∃x∃x ∀y Loves(x,y)

“There is a person who loves everyone in the world”∀y ∃x Loves(x,y)

“Everyone in the world is loved by at least one person”

Quantifier duality: each can be expressed using the other∀x Likes(x,IceCream) ¬∃x ¬Likes(x,IceCream)∃x Likes(x,Broccoli) ¬∀x ¬Likes(x,Broccoli)


Equality

term1 = term2 is true under a given interpretation if and only if term1 and term2 refer to the same object

E.g., definition of Sibling in terms of Parent:∀x,y Sibling(x,y) ⇔ [¬(x = y) ∧ ∃m,f ¬ (m = f) ∧ Parent(m,x) ∧

Parent(f,x) ∧ Parent(m,y) ∧ Parent(f,y)]


Interacting with FOL KBs Suppose a wumpus-world agent is using an FOL KB and perceives a smell

and a breeze (but no glitter) at t=5:

Tell(KB,Percept([Smell,Breeze,None],5)) (= assertion)Ask(KB,∃a BestAction(a,5))(=queries)

I.e., does the KB entail some best action at t=5?

Answer: Yes, {a/Shoot} ← substitution (binding list) Given a sentence S and a substitution α, Sα denotes the result of plugging α into S; e.g.,

S = Smarter(x,y)α = {x/Hillary,y/Bill}Sα = Smarter(Hillary,Bill)

Ask(KB,S) returns some/all α such that KB |= Sα.


Using FOL

The kinship domain: Brothers are siblings

∀x,y Brother(x,y) ⇔ Sibling(x,y) One's mother is one's female parent

∀m,c Mother(c) = m ⇔ (Female(m) ∧ Parent(m,c)) “Sibling” is symmetric

∀x,y Sibling(x,y) ⇔ Sibling(y,x) A first cousin is a child of a parent’s sibling

∀x,y FirstCousin(x,y) ⇔ ∃p,ps Parent(p,x) ∧ Sibling(ps,p) ∧Parent(ps,y)


Using FOL

The set domain: ∀s Set(s) ⇔ (s = {} ) ∨ (∃x,s2 Set(s2) ∧ s = {x|s2}) ¬∃x,s {x|s} = {} ∀x,s x ∈ s ⇔ s = {x|s} ∀x,s x ∈ s ⇔ [ ∃y,s2} (s = {y|s2} ∧ (x = y ∨ x ∈ s2))] ∀s1,s2 s1 ⊆ s2 ⇔ (∀x x ∈ s1 ⇒ x ∈ s2) ∀s1,s2 (s1 = s2) ⇔ (s1 ⊆ s2 ∧ s2 ⊆ s1) ∀x,s1,s2 x ∈ (s1 ∩ s2) ⇔ (x ∈ s1 ∧ x ∈ s2) ∀x,s1,s2 x ∈ (s1 ∪ s2) ⇔ (x ∈ s1 ∨ x ∈ s2)


FOL Version of Wumpus World

Typical percept sentence:Percept([Stench,Breeze,Glitter,None,None],5)

Actions:Turn(Right), Turn(Left), Forward, Shoot, Grab, Release, Climb

To determine best action, construct query:∀ a BestAction(a,5)

ASK solves this and returns {a/Grab} And TELL about the action.


Knowledge base for the wumpus world

Perception ∀b,g,t Percept([Smell,b,g],t) ⇒ Smelt(t) ∀s,b,t Percept([s,b,Glitter],t) ⇒ Glitter(t)

Reflex ∀t Glitter(t) ⇒ BestAction(Grab,t)

Reflex with internal state ∀t Glitter(t) ∧¬Holding(Gold,t) ⇒ BestAction(Grab,t)

Holding(Gold,t) can not be observed: keep track of change.All synchronic sentences!


Deducing hidden properties

Environment definition:∀x,y,a,b Adjacent([x,y],[a,b]) ⇔

[a,b] ∈ {[x+1,y], [x-1,y],[x,y+1],[x,y-1]} Properties of locationsß:

∀s,t At(Agent,s,t) ∧ Smelt(t) ⇒ Smelly(s)∀s,t At(Agent,s,t) ∧ Breeze(t) ⇒ Breezy(s)

Squares are breezy near a pit:Diagnostic rule---infer cause from effect

∀s Breezy(s) ⇔ ∃ r Adjacent(r,s) ∧ Pit(r)Causal rule---infer effect from cause (model based reasoning)

∀r Pit(r) ⇒ [∀s Adjacent(r,s) ⇒ Breezy(s)]


Knowledge engineering in FOL

Identify the task (what will the KB be used for) Assemble the relevant knowledge

Knowledge acquisition Decide on a vocabulary of predicates, functions, and constants

Translate domain-level knowledge into logic-level names. Encode general knowledge about the domain

define axioms Encode a description of the specific problem instance Pose queries to the inference procedure and get answers Debug the knowledge base


The electronic circuits domain

One-bit full adder



Identify the task Does the circuit actually add properly? (circuit verification)

Assemble the relevant knowledge Composed of wires and gates; Types of gates (AND, OR, XOR, NOT) Connections between terminals Irrelevant: size, shape, color, cost of gates

Decide on a vocabulary Alternatives:

Type(X1) = XORType(X1, XOR)XOR(X1)



4. Encode general knowledge of the domain ∀t1,t2 Connected(t1, t2) ⇒ Signal(t1) = Signal(t2) ∀t Signal(t) = 1 ∨ Signal(t) = 0 1 ≠ 0 ∀t1,t2 Connected(t1, t2) ⇒ Connected(t2, t1) ∀g Type(g) = OR ⇒ Signal(Out(1,g)) = 1 ⇔ ∃n Signal(In(n,g)) = 1 ∀g Type(g) = AND ⇒ Signal(Out(1,g)) = 0 ⇔ ∃n Signal(In(n,g)) = 0 ∀g Type(g) = XOR ⇒ Signal(Out(1,g)) = 1 ⇔ Signal(In(1,g)) ≠

Signal(In(2,g)) ∀g Type(g) = NOT ⇒ Signal(Out(1,g)) ≠ Signal(In(1,g))



5. Encode the specific problem instanceType(X1) = XOR Type(X2) = XORType(A1) = AND Type(A2) = ANDType(O1) = OR

Connected(Out(1,X1),In(1,X2)) Connected(In(1,C1),In(1,X1))Connected(Out(1,X1),In(2,A2)) Connected(In(1,C1),In(1,A1))Connected(Out(1,A2),In(1,O1)) Connected(In(2,C1),In(2,X1))Connected(Out(1,A1),In(2,O1)) Connected(In(2,C1),In(2,A1))Connected(Out(1,X2),Out(1,C1)) Connected(In(3,C1),In(2,X2))Connected(Out(1,O1),Out(2,C1)) Connected(In(3,C1),In(1,A2))



6. Pose queries to the inference procedureWhat are the possible sets of values of all the terminals for the

adder circuit? ∃i1,i2,i3,o1,o2 Signal(In(1,C_1)) = i1 ∧ Signal(In(2,C1)) = i2 ∧Signal(In(3,C1)) = i3 ∧ Signal(Out(1,C1)) = o1 ∧ Signal(Out(2,C1)) = o2

7. Debug the knowledge baseMay have omitted assertions like 1 ≠ 0


Summary

First-order logic:objects and relations are semantic primitivessyntax: constants, functions, predicates, equality,

quantifiers.

Increased expressive power: sufficient to define wumpus world

Artificial intelligence: Inference in first-order logic



Outline

Reducing first-order inference to propositional inference

UnificationGeneralized Modus Ponens Forward chaining Backward chaining Resolution


FOL to PL

First order inference can be done by converting the knowledge base to PL and using propositional inference.How to convert universal quantifiers?Replace variable by ground term.

How to convert existential quantifiers?Skolemization.


Universal instantiation (UI)

Every instantiation of a universally quantified sentence is entailed by it:∀v α

Subst({v/g}, α)

for any variable v and ground term g (Subst(x,y) = substitution of y by x)

E.g., ∀x King(x) ∧ Greedy(x) ⇒ Evil(x) yields:King(John) ∧ Greedy(John) ⇒ Evil(John)King(Richard) ∧ Greedy(Richard) ⇒ Evil(Richard)King(Father(John)) ∧ Greedy(Father(John)) ⇒ Evil(Father(John))...


Existential instantiation (EI) For any sentence α, variable v, and constant symbol k that does not appear

elsewhere in the knowledge base:∃v α

Subst({v/k}, α)

E.g., ∃x Crown(x) ∧ OnHead(x,John) yields:

Crown(C1) ∧ OnHead(C1,John)

provided C1 is a new constant symbol, called a Skolem constant


EI versus UI

UI can be applied several times to add new sentences; the new KB is logically equivalent to the old.

EI can be applied once to replace the existential sentence; the new KB is not equivalent to the old but is satisfiable if the old KB was satisfiable.


Reduction to propositional inference

Suppose the KB contains just the following:∀x King(x) ∧ Greedy(x) ⇒ Evil(x)King(John)Greedy(John)Brother(Richard,John)

Instantiating the universal sentence in all possible ways, we have:King(John) ∧ Greedy(John) ⇒ Evil(John)King(Richard) ∧ Greedy(Richard) ⇒ Evil(Richard)King(John)Greedy(John)Brother(Richard,John)

The new KB is propositionalized: proposition symbols areJohn, Richard and also King(John), Greedy(John), Evil(John), King(Richard), etc.


Reduction contd.

CLAIM: A ground sentence is entailed by a new KB iff entailed by the original KB.

CLAIM: Every FOL KB can be propositionalized so as to preserve entailment

IDEA: propositionalize KB and query, apply resolution, return result

PROBLEM: with function symbols, there are infinitely many ground terms,e.g., Father(Father(Father(John)))


Reduction contd. THEOREM: Herbrand (1930). If a sentence α is entailed by an FOL KB, it is

entailed by a finite subset of the propositionalized KB

IDEA: For n = 0 to ∞ do create a propositional KB by instantiating with depth-n terms see if α is entailed by this KB

PROBLEM: works if α is entailed, loops if α is not entailed THEOREM: Turing (1936), Church (1936) Entailment for FOL is semi decidable

algorithms exist that say yes to every entailed sentence, but no algorithm exists that also says no to every non-entailed sentence.


Problems with propositionalization

Propositionalization seems to generate lots of irrelevant sentences. E.g., from:

∀x King(x) ∧ Greedy(x) ⇒ Evil(x)King(John)∀y Greedy(y)Brother(Richard,John)

It seems obvious that Evil(John), but propositionalization produces lots of facts such as Greedy(Richard) that are irrelevant. With p k-ary predicates and n constants, there are p·nk instantiations!


Lifting and Unification

Instead of translating the knowledge base to PL, we can redefine the inference rules into FOL. Lifting; they only make those substitutions that are required to

allow particular inferences to proceed.E.g. generalised Modus Ponens

To intriduce substitutions different logical expressions have to be look identical

Unification


Unification We can get the inference immediately if we can find a substitution α such that King(x) and

Greedy(x) match King(John) and Greedy(y)

α = {x/John,y/John} works

Unify(α ,β) = θ if αθ = βθp q θKnows(John,x) Knows(John,Jane) Knows(John,x) Knows(y,OJ) Knows(John,x) Knows(y,Mother(y))Knows(John,x) Knows(x,OJ)

Standardizing apart eliminates overlap of variables, e.g., Knows(z17,OJ)





Unify(α ,β) = θ if αθ = βθp q θKnows(John,x) Knows(John,Jane) {x/Jane}Knows(John,x) Knows(y,OJ) Knows(John,x) Knows(y,Mother(y))Knows(John,x) Knows(x,OJ)






Unify(α ,β) = θ if αθ = βθp q θKnows(John,x) Knows(John,Jane) {x/Jane}Knows(John,x) Knows(y,OJ) {x/OJ,y/John}Knows(John,x) Knows(y,Mother(y))Knows(John,x) Knows(x,OJ)






Unify(α ,β) = θ if αθ = βθp q θKnows(John,x) Knows(John,Jane) {x/Jane}Knows(John,x) Knows(y,OJ) {x/OJ,y/John}Knows(John,x) Knows(y,Mother(y)) {y/John,x/Mother(John)}}Knows(John,x) Knows(x,OJ)






Unify(α ,β) = θ if αθ = βθp q θKnows(John,x) Knows(John,Jane) {x/Jane}Knows(John,x) Knows(y,OJ) {x/OJ,y/John}Knows(John,x) Knows(y,Mother(y)) {y/John,x/Mother(John)}}Knows(John,x) Knows(x,OJ) {fail}



Unification

To unify Knows(John,x) and Knows(y,z),α = {y/John, x/z } or α = {y/John, x/John, z/John}

The first unifier is more general than the second.

There is a single most general unifier (MGU) that is unique up to renaming of variables.

MGU = { y/John, x/z }


The unification algorithm


Generalized Modus Ponens (GMP)

p1', p2', … , pn', ( p1 ∧ p2 ∧ … ∧ pn ⇒q)

qθp1' is King(John) p1 is King(x) p2' is Greedy(y) p2 is Greedy(x) θ is {x/John,y/John} q is Evil(x) qθ is Evil(John)

GMP used with KB of definite clauses (exactly one positive literal). All variables assumed universally quantified.

where pi'θ = piθ for all i


Soundness of GMP

Need to show that p1', …, pn', (p1 ∧ … ∧ pn ⇒ q) |= qθ

provided that pi'θ = piθ for all I

LEMMA: For any sentence p, we have p |= pθ by UI

1. (p1 ∧ … ∧ pn ⇒ q) |= (p1 ∧ … ∧ pn ⇒ q)θ = (p1θ ∧ … ∧ pnθ ⇒ qθ)2. p1', …, pn' |= p1' ∧ … ∧ pn' |= p1'θ ∧ … ∧ pn'θ3. From 1 and 2, qθ follows by ordinary Modus Ponens.


Example knowledge base

The law says that it is a crime for an American to sell weapons to hostile nations. The country Nono, an enemy of America, has some missiles, and all of its missiles were sold to it by Colonel West, who is American.

Prove that Col. West is a criminal


Example knowledge base contd.

... it is a crime for an American to sell weapons to hostile nations:



... it is a crime for an American to sell weapons to hostile nations:American(x) ∧ Weapon(y) ∧ Sells(x,y,z) ∧ Hostile(z) ⇒ Criminal(x)




Nono … has some missiles




Nono … has some missiles, i.e., ∃x Owns(Nono,x) ∧ Missile(x):Owns(Nono,M1) and Missile(M1)





… all of its missiles were sold to it by Colonel West





… all of its missiles were sold to it by Colonel WestMissile(x) ∧ Owns(Nono,x) ⇒ Sells(West,x,Nono)






Missiles are weapons:






Missiles are weapons:Missile(x) ⇒ Weapon(x)







An enemy of America counts as "hostile“:







An enemy of America counts as "hostile“:Enemy(x,America) ⇒ Hostile(x)








West, who is American …








West, who is American …American(West)









The country Nono, an enemy of America …









The country Nono, an enemy of America …Enemy(Nono,America)


Forward chaining algorithm


Properties of forward chaining

Sound and complete for first-order definite clauses. Cfr. Propositional logic proof.

Datalog = first-order definite clauses + no functions (e.g. crime KB) FC terminates for Datalog in finite number of iterations

May not terminate in general DF clauses with functions if α is not entailed This is unavoidable: entailment with definite clauses is semidecidable


Efficiency of forward chaining

Incremental forward chaining: no need to match a rule on iteration k if a premise wasn't added on iteration k-1 match each rule whose premise contains a newly added positive literal.

Matching itself can be expensive: Database indexing allows O(1) retrieval of known facts

e.g., query Missile(x) retrieves Missile(M1)

Matching conjunctive premises against known facts is NP-hard. (Pattern matching)

Forward chaining is widely used in deductive databases


Hard matching example

Colorable() is inferred iff the CSP has a solution CSPs include 3SAT as a special case, hence matching is NP-hard

Diff(wa,nt) ∧ Diff(wa,sa) ∧ Diff(nt,q) ∧Diff(nt,sa) ∧ Diff(q,nsw) ∧ Diff(q,sa) ∧Diff(nsw,v) ∧ Diff(nsw,sa) ∧ Diff(v,sa) ⇒Colorable()

Diff(Red,Blue) Diff (Red,Green) Diff(Green,Red) Diff(Green,Blue) Diff(Blue,Red) Diff(Blue,Green)


Backward chaining algorithm

SUBST(COMPOSE(α1, α2), p) = SUBST(α2, SUBST(α1, p))


Properties of backward chaining

Depth-first recursive proof search: space is linear in size of proof. Incomplete due to infinite loops

fix by checking current goal against every goal on stack Inefficient due to repeated subgoals (both success and failure)

fix using caching of previous results (extra space!!) Widely used for logic programming


Logic programming

Logic programming Identify problem Assemble information <coffee break> Encode info in KB Encode problem instances

as facts Ask queries Find false facts.

Procedural programming Identify problem Assemble information Figure out solution Program solution Encode problem instance as

data Apply program to data Debug procedural errors

Should be easier to debug Capital(NY, US) than x=x+2


Logic programming: Prolog

BASIS: backward chaining with Horn clauses + bells & whistlesWidely used in Europe, Japan (basis of 5th Generation project)Compilation techniques ⇒ 60 million LIPS

Program = set of clauses = head :- literal1, … literaln.criminal(X) :- american(X), weapon(Y), sells(X,Y,Z), hostile(Z).

Efficient unification and retrieval of matching clauses. Depth-first, left-to-right backward chaining Built-in predicates for arithmetic etc., e.g., X is Y*Z+3 Built-in predicates that have side effects (e.g., input and output predicates, assert/retract

predicates) Closed-world assumption ("negation as failure")

e.g., given alive(X) :- not dead(X). alive(joe) succeeds if dead(joe) fails


Prolog

Appending two lists to produce a third:append([],Y,Y).

append([X|L],Y,[X|Z]) :- append(L,Y,Z).

query: append(A,B,[1,2]) ?

answers: A=[] B=[1,2]A=[1] B=[2]

A=[1,2] B=[]


Resolution: brief summary Full first-order version:

l1 ∨ ··· ∨ lk, m1 ∨ ··· ∨ mn

(l1 ∨ ··· ∨ li-1 ∨ li+1 ∨ ··· ∨ lk ∨ m1 ∨ ··· ∨ mj-1 ∨ mj+1 ∨ ··· ∨ mn)θ

where Unify(li, ¬mj) = θ. The two clauses are assumed to be standardized apart so that they share no variables. For example,

¬Rich(x) ∨ Unhappy(x) Rich(Ken)

Unhappy(Ken)with θ = {x/Ken}

Apply resolution steps to CNF(KB ∧ ¬α); complete for FOL


Conversion to CNF Everyone who loves all animals is loved by someone:

∀x [∀y Animal(y) ⇒ Loves(x,y)] ⇒ [∃y Loves(y,x)]

Eliminate biconditionals and implications∀x [¬∀y ¬Animal(y) ∨ Loves(x,y)] ∨ [∃y Loves(y,x)]

Move ¬ inwards: ¬∀x p ≡ ∃x ¬p, ¬ ∃x p ≡ ∀x ¬p∀x [∃y ¬(¬Animal(y) ∨ Loves(x,y))] ∨ [∃y Loves(y,x)] ∀x [∃y ¬¬Animal(y) ∧ ¬Loves(x,y)] ∨ [∃y Loves(y,x)]∀x [∃y Animal(y) ∧ ¬Loves(x,y)] ∨ [∃y Loves(y,x)]


Conversion to CNF contd. Standardize variables: each quantifier should use a different one:

∀x [∃y Animal(y) ∧ ¬Loves(x,y)] ∨ [∃z Loves(z,x)]

Skolemize: a more general form of existential instantiation.Each existential variable is replaced by a Skolem function of the enclosing universally quantified variables:∀x [Animal(F(x)) ∧ ¬Loves(x,F(x))] ∨ Loves(G(x),x)

Drop universal quantifiers:[Animal(F(x)) ∧ ¬Loves(x,F(x))] ∨ Loves(G(x),x)

Distribute ∨ over ∧ :[Animal(F(x)) ∨ Loves(G(x),x)] ∧ [¬Loves(x,F(x)) ∨ Loves(G(x),x)]


Resolution proof: definite clauses

Artificial Intelligence: knowledge representation


February 4, 2012 Mehnen 2

Outline

Ontological engineering Categories and objects Actions, situations and eventsMental events and mental objects The internet shopping world Reasoning systems for categories Reasoning with default information Truth maintenance systems


Ontological engineering

How to create more general and flexible representations.Concepts like actions, time, physical object and beliefsOperates on a bigger scale than K.E.

Define general framework of conceptsUpper ontology

Limitations of logic representationRed, green and yellow tomatoes: exceptions and

uncertainty


The upper ontology of the world


Difference with special-purpose ontologies A general-purpose ontology should be

applicable in more or less any special-purpose domain.Add domain-specific axioms

In any sufficiently demanding domain different areas of knowledge need to be unified.Reasoning and problem solving could involve several

areas simultaneouslyWhat do we need to express?

Categories, Measures, Composite objects, Time, Space, Change, Events, Processes, Physical Objects, Substances, Mental Objects, Beliefs


Categories and objects

KR requires the organisation of objects into categories Interaction at the level of the object Reasoning at the level of categories

Categories play a role in predictions about objects Based on perceived properties

Categories can be represented in two ways by FOL Predicates: apple(x) Reification of categories into objects: apples

Category = set of its members


Category organizationRelation = inheritance:All instance of food are edible, fruit is a

subclass of food and apples is a subclass of fruit then an apple is edible.

Defines a taxonomy


FOL and categories

An object is a member of a categoryMemberOf(BB12,Basketballs)

A category is a subclass of another category SubsetOf(Basketballs,Balls)

All members of a category have some properties ∀ x (MemberOf(x,Basketballs) ⇒ Round(x))

All members of a category can be recognized by some properties ∀ x (Orange(x) ∧ Round(x) ∧ Diameter(x)=9.5in ∧

MemberOf(x,Balls) ⇒ MemberOf(x,BasketBalls)) A category as a whole has some properties

MemberOf(Dogs,DomesticatedSpecies)


Relations between categories

Two or more categories are disjoint if they have no members in common: Disjoint(s)⇔(∀ c1,c2 c1 ∈ s ∧ c2 ∈ s ∧ c1 ≠ c2 ⇒ Intersection(c1,c2)

={}) Example; Disjoint({animals, vegetables})

A set of categories s constitutes an exhaustive decomposition of a category c if all members of the set c are covered by categories in s: E.D.(s,c) ⇔ (∀ i i ∈ c ⇒ ∃ c2 c2 ∈ s ∧ i ∈ c2)

Example: ExhaustiveDecomposition({Americans, Canadian, Mexicans},NorthAmericans).


Relations between categories

A partition is a disjoint exhaustive decomposition:Partition(s,c) ⇔ Disjoint(s) ∧ E.D.(s,c)Example: Partition({Males,Females},Persons).

Is ({Americans,Canadian, Mexicans},NorthAmericans) a partition?

Categories can be defined by providing necessary and sufficient conditions for membership∀ x Bachelor(x) ⇔ Male(x) ∧ Adult(x) ∧ Unmarried(x)


Natural kindsMany categories have no clear-cut definitions

(chair, bush, book). Tomatoes: sometimes green, red, yellow, black.

Mostly round. One solution: category Typical(Tomatoes). ∀ x, x ∈ Typical(Tomatoes) ⇒ Red(x) ∧ Spherical(x).We can write down useful facts about categories without

providing exact definitions.What about “bachelor”? Quine challenged the

utility of the notion of strict definition. We might question a statement such as “the Pope is a bachelor”.


Physical composition

One object may be part of another: PartOf(Bucharest,Romania) PartOf(Romania,EasternEurope) PartOf(EasternEurope,Europe)

The PartOf predicate is transitive (and irreflexive), so we can infer that PartOf(Bucharest,Europe)

More generally: ∀ x PartOf(x,x) ∀ x,y,z PartOf(x,y) ∧ PartOf(y,z) ⇒ PartOf(x,z)

Often characterized by structural relations among parts. E.g. Biped(a) ⇒


Measurements

Objects have height, mass, cost, .... Values that we assign to these are measures

Combine Unit functions with a number: Length(L1) = Inches(1.5) = Centimeters(3.81).

Conversion between units:∀ i Centimeters(2.54 x i)=Inches(i).

Some measures have no scale: Beauty, Difficulty, etc. Most important aspect of measures: is that they are

orderable.Don't care about the actual numbers. (An apple can

have deliciousness .9 or .1.)


Actions, events and situations

• Reasoning about outcome of actions is central to KB-agent.

• How can we keep track of location in FOL?• Remember the multiple

copies in PL.• Representing time by

situations (states resulting from the execution of actions).• Situation calculus



Situation calculus:Actions are logical termsSituations are logical

terms consiting of The initial situation I All situations resulting

from the action on I (=Result(a,s))

Fluent are functions and predicates that vary from one situation to the next. E.g. ¬Holding(G1, S0)

Eternal predicates are also allowed E.g. Gold(G1)



Results of action sequences are determined by the individual actions.

Projection task: an SC agent should be able to deduce the outcome of a sequence of actions.

Planning task: find a sequence that achieves a desirable effect


Describing change

Simples Situation calculus requires two axioms to describe change:Possibility axiom: when is it possible to do the action

At(Agent,x,s) ∧ Adjacent(x,y) ⇒ Poss(Go(x,y),s)Effect axiom: describe changes due to action

Poss(Go(x,y),s) ⇒ At(Agent,y,Result(Go(x,y),s))

What stays the same?Frame problem: how to represent all things that stay the

same?Frame axiom: describe non-changes due to actions

At(o,x,s) ∧ (o ≠ Agent) ∧ ¬Holding(o,s) ⇒At(o,x,Result(Go(y,z),s))


Representational frame problem

If there are F fluents and A actions then we need AF frame axioms to describe other objects are stationary unless they are held.We write down the effect of each actions

Solution; describe how each fluent changes over timeSuccessor-state axiom:

Pos(a,s) ⇒ (At(Agent,y,Result(a,s)) ⇔ (a = Go(x,y)) ∨(At(Agent,y,s) ∧ a ≠ Go(y,z))

Note that next state is completely specified by current state.

Each action effect is mentioned only once.


Other problems

How to deal with secondary (implicit) effects? If the agent is carrying the gold and the agent moves

then the gold moves too.Ramification problem

How to decide EFFICIENTLY whether fluents hold in the future? Inferential frame problem.

Extensions:Event calculus (when actions have a duration)Process categories


Mental events and objects

So far, KB agents can have beliefs and deduce new beliefs

What about knowledge about beliefs? What about knowledge about the inference proces?Requires a model of the mental objects in someone’s

head and the processes that manipulate these objects. Relationships between agents and mental

objects: believes, knows, wants, …Believes(Lois,Flies(Superman)) with Flies(Superman)

being a function … a candidate for a mental object (reification).

Agent can now reason about the beliefs of agents.


The internet shopping world

A Knowledge Engineering example An agent that helps a buyer to find product offers

on the internet. IN = product description (precise or ¬precise)OUT = list of webpages that offer the product for sale.

Environment = WWW Percepts = web pages (character strings)Extracting useful information required.


The internet shopping world

Find relevant product offersRelevantOffer(page,url,query) ⇔ Relevant(page, url, query) ∧

Offer(page)Write axioms to define Offer(x)Find relevant pages: Relevant(x,y,z) ?Start from an initial set of stores.What is a relevant category?What are relevant connected pages?

Require rich category vocabulary.Synonymy and ambiguity

How to retrieve pages: GetPage(url)?Procedural attachment

Compare offers (information extraction).


Reasoning systems for categories

How to organise and reason with categories?Semantic networksVisualize knowledge-baseEfficient algorithms for category membership

inferenceDescription logicsFormal language for constructing and combining

category definitionsEfficient algorithms to decide subset and superset

relationships between categories.


Semantic Networks

Logic vs. semantic networksMany variationsAll represent individual objects, categories of objects

and relationships among objects. Allows for inheritance reasoningFemale persons inherit all properties from person.Cfr. OO programming.

Inference of inverse linksSisterOf vs. HasSister


Semantic network example


Semantic networks

DrawbacksLinks can only assert binary relationsCan be resolved by reification of the proposition

as an eventRepresentation of default valuesEnforced by the inheritance mechanism.


Description logics

Are designed to describe defintions and properties about categoriesA formalization of semantic networks

Principal inference task is Subsumption: checking if one category is the subset of

another by comparing their definitionsClassification: checking whether an object belongs to a

category.Consistency: whether the category membership criteria

are logically satisfiable.


Truth maintenance systems

Many of the inferences have default status rather than being absolutely certain Inferred facts can be wrong and need to be retracted =

BELIEF REVISION.Assume KB contains sentence P and we want to

execute TELL(KB, ¬P)To avoid contradiction: RETRACT(KB,P)But what about sentences inferred from P?

Truth maintenance systems are designed to handle these complications.

Artificial Intelligence: Planning



Planning

The Planning problemPlanning with State-space

searchPartial-order planningPlanning graphsPlanning with propositional logicAnalysis of planning

approaches


What is Planning

Generate sequences of actions to perform tasks and achieve objectives. States, actions and goals

Search for solution over abstract space of plans. Classical planning environment: fully observable,

deterministic, finite, static and discrete. Assists humans in practical applications

design and manufacturingmilitary operations games space exploration


Difficulty of real world problems

Assume a problem-solving agentusing some search method …Which actions are relevant?Exhaustive search vs. backward search

What is a good heuristic functions?Good estimate of the cost of the state?Problem-dependent vs, -independent

How to decompose the problem?Most real-world problems are nearly

decomposable.


Planning language

What is a good language?Expressive enough to describe a wide variety

of problems.Restrictive enough to allow efficient

algorithms to operate on it.Planning algorithm should be able to take

advantage of the logical structure of the problem.

STRIPS and ADL


General language features

Representation of statesDecompose the world in logical conditions and represent

a state as a conjunction of positive literals. Propositional literals: Poor ∧ UnknownFO-literals (grounded and function-free): At(Plane1, Melbourne) ∧

At(Plane2, Sydney)

Closed world assumption Representation of goalsPartially specified state and represented as a conjunction

of positive ground literalsA goal is satisfied if the state contains all literals in goal.


General language features

Representations of actionsAction = PRECOND + EFFECT

Action(Fly(p,from, to),PRECOND: At(p,from) ∧ Plane(p) ∧ Airport(from) ∧ Airport(to)EFFECT: ¬AT(p,from) ∧ At(p,to))

= action schema (p, from, to need to be instantiated)Action name and parameter listPrecondition (conj. of function-free literals)Effect (conj of function-free literals and P is True and not P is

false)

Add-list vs delete-list in Effect


Language semantics?

How do actions affect states?An action is applicable in any state that satisfies the

precondition.For FO action schema applicability involves a

substitution θ for the variables in the PRECOND.At(P1,JFK) ∧ At(P2,SFO) ∧ Plane(P1) ∧ Plane(P2) ∧

Airport(JFK) ∧ Airport(SFO)Satisfies : At(p,from) ∧ Plane(p) ∧ Airport(from) ∧ Airport(to)With θ ={p/P1,from/JFK,to/SFO}Thus the action is applicable.


Language semantics?

The result of executing action a in state s is the state s’ s’ is same as s except

Any positive literal P in the effect of a is added to s’Any negative literal ¬P is removed from s’EFFECT: ¬AT(p,from) ∧ At(p,to):At(P1,SFO) ∧ At(P2,SFO) ∧ Plane(P1) ∧ Plane(P2) ∧ Airport(JFK) ∧

Airport(SFO)

STRIPS assumption: (avoids representational frame problem)

every literal NOT in the effect remains unchanged


Expressiveness and extensions

STRIPS is simplified Important limit: function-free literals

Allows for propositional representation Function symbols lead to infinitely many states and actions

Recent extension:Action Description language (ADL)Action(Fly(p:Plane, from: Airport, to: Airport),

PRECOND: At(p,from) ∧ (from ≠ to)EFFECT: ¬At(p,from) ∧ At(p,to))

Standardization : Planning domain definition language (PDDL)


Example: air cargo transport

Init(At(C1, SFO) ∧ At(C2,JFK) ∧ At(P1,SFO) ∧ At(P2,JFK) ∧ Cargo(C1) ∧Cargo(C2) ∧ Plane(P1) ∧ Plane(P2) ∧ Airport(JFK) ∧ Airport(SFO))

Goal(At(C1,JFK) ∧ At(C2,SFO))Action(Load(c,p,a)

PRECOND: At(c,a) ∧At(p,a) ∧Cargo(c) ∧Plane(p) ∧Airport(a)EFFECT: ¬At(c,a) ∧In(c,p))

Action(Unload(c,p,a)PRECOND: In(c,p) ∧At(p,a) ∧Cargo(c) ∧Plane(p) ∧Airport(a)EFFECT: At(c,a) ∧ ¬In(c,p))

Action(Fly(p,from,to)PRECOND: At(p,from) ∧Plane(p) ∧Airport(from) ∧Airport(to)EFFECT: ¬ At(p,from) ∧ At(p,to))

[Load(C1,P1,SFO), Fly(P1,SFO,JFK), Load(C2,P2,JFK), Fly(P2,JFK,SFO)]


Example: Spare tire problemInit(At(Flat, Axle) ∧ At(Spare,trunk))Goal(At(Spare,Axle))Action(Remove(Spare,Trunk)

PRECOND: At(Spare,Trunk)EFFECT: ¬At(Spare,Trunk) ∧ At(Spare,Ground))

Action(Remove(Flat,Axle)PRECOND: At(Flat,Axle)EFFECT: ¬At(Flat,Axle) ∧ At(Flat,Ground))

Action(PutOn(Spare,Axle)PRECOND: At(Spare,Groundp) ∧¬At(Flat,Axle)EFFECT: At(Spare,Axle) ∧ ¬At(Spare,Ground))

Action(LeaveOvernightPRECOND:EFFECT: ¬ At(Spare,Ground) ∧ ¬ At(Spare,Axle) ∧ ¬ At(Spare,trunk) ∧ ¬At(Flat,Ground) ∧ ¬ At(Flat,Axle) )

This example goes beyond STRIPS: negative literal in pre-condition (ADL description)


Example: Blocks world

Init(On(A, Table) ∧ On(B,Table) ∧ On(C,Table) ∧ Block(A) ∧ Block(B) ∧Block(C) ∧ Clear(A) ∧ Clear(B) ∧ Clear(C))

Goal(On(A,B) ∧ On(B,C))Action(Move(b,x,y)

PRECOND: On(b,x) ∧ Clear(b) ∧ Clear(y) ∧ Block(b) ∧ (b≠ x) ∧ (b≠ y) ∧ (x≠y) EFFECT: On(b,y) ∧ Clear(x) ∧ ¬ On(b,x) ∧ ¬ Clear(y))

Action(MoveToTable(b,x)PRECOND: On(b,x) ∧ Clear(b) ∧ Block(b) ∧ (b≠ x) EFFECT: On(b,Table) ∧ Clear(x) ∧ ¬ On(b,x))

Spurious actions are possible: Move(B,C,C)


Planning with state-space search

Both forward and backward search possible Progression plannersforward state-space searchConsider the effect of all possible actions in a given

state Regression planners backward state-space searchTo achieve a goal, what must have been true in the

previous state.


Progression and regression


Progression algorithm

Formulation as state-space search problem: Initial state = initial state of the planning problem

Literals not appearing are false Actions = those whose preconditions are satisfied

Add positive effects, delete negative Goal test = does the state satisfy the goal Step cost = each action costs 1

No functions … any graph search that is complete is a complete planning algorithm. E.g. A*

Inefficient: (1) irrelevant action problem (2) good heuristic required for efficient search


Regression algorithm

How to determine predecessors?What are the states from which applying a given action leads to

the goal?Goal state = At(C1, B) ∧ At(C2, B) ∧ … ∧ At(C20, B)Relevant action for first conjunct: Unload(C1,p,B)Works only if pre-conditions are satisfied.Previous state= In(C1, p) ∧ At(p, B) ∧ At(C2, B) ∧ … ∧ At(C20, B)Subgoal At(C1,B) should not be present in this state.

Actions must not undo desired literals (consistent) Main advantage: only relevant actions are considered.

Often much lower branching factor than forward search.


Regression algorithm

General process for predecessor constructionGive a goal description GLet A be an action that is relevant and consistentThe predecessors is as follows:

Any positive effects of A that appear in G are deleted.Each precondition literal of A is added , unless it already

appears.

Any standard search algorithm can be added to perform the search.

Termination when predecessor satisfied by initial state.In FO case, satisfaction might require a substitution.


Heuristics for state-space search

Neither progression or regression are very efficient without a good heuristic.How many actions are needed to achieve the goal?Exact solution is NP hard, find a good estimate

Two approaches to find admissible heuristic:The optimal solution to the relaxed problem.

Remove all preconditions from actions

The subgoal independence assumption:The cost of solving a conjunction of subgoals is approximated

by the sum of the costs of solving the subproblems independently.


Partial-order planning

Progression and regression planning are totally ordered plan search forms.They cannot take advantage of problem

decomposition.Decisions must be made on how to sequence

actions on all the subproblems

Least commitment strategy:Delay choice during search


Shoe example

Goal(RightShoeOn ∧ LeftShoeOn)Init()Action(RightShoe, PRECOND: RightSockOn

EFFECT: RightShoeOn)Action(RightSock,PRECOND:

EFFECT: RightSockOn)Action(LeftShoe, PRECOND: LeftSockOn

EFFECT: LeftShoeOn)Action(LeftSock, PRECOND:

EFFECT: LeftSockOn)

Planner: combine two action sequences (1)leftsock, leftshoe (2)rightsock, rightshoe


Partial-order planning(POP) Any planning algorithm that can place two actions into a

plan without which comes first is a PO plan.


POP as a search problem States are (mostly unfinished) plans.

The empty plan contains only start and finish actions. Each plan has 4 components:

A set of actions (steps of the plan) A set of ordering constraints: A < B (A before B)

Cycles represent contradictions. A set of causal links

The plan may not be extended by adding a new action C that conflicts with the causal link. (if the effect of C is ¬p and if C could come after A and before B)

A set of open preconditions. If precondition is not achieved by action in the plan.

A p → B


Example of final plan

Actions={Rightsock, Rightshoe, Leftsock, Leftshoe, Start, Finish}

Orderings={Rightsock < Rightshoe; Leftsock < Leftshoe}

Links={Rightsock->Rightsockon -> Rightshoe, Leftsock->Leftsockon-> Leftshoe, Rightshoe->Rightshoeon->Finish, …}

Open preconditions={}


POP as a search problem

A plan is consistent iff there are no cycles in the ordering constraints and no conflicts with the causal links.

A consistent plan with no open preconditions is a solution.

A partial order plan is executed by repeatedly choosing any of the possible next actions.This flexibility is a benefit in non-cooperative

environments.


Solving POPAssume propositional planning problems:

The initial plan contains Start and Finish, the ordering constraint Start < Finish, no causal links, all the preconditions in Finish are open.Successor function :picks one open precondition p on an action B andgenerates a successor plan for every possible

consistent way of choosing action A that achieves p.

Test goal


Enforcing consistency

When generating successor plan:The causal link A->p->B and the ordering constraint A <

B is added to the plan. If A is new also add start < A and A < B to the plan

Resolve conflicts between new causal link and all existing actions

Resolve conflicts between action A (if new) and all existing causal links.


Process summary

Operators on partial plansAdd link from existing plan to open precondition.Add a step to fulfill an open condition.Order one step w.r.t another to remove possible

conflictsGradually move from incomplete/vague plans to

complete/correct plans Backtrack if an open condition is unachievable

or if a conflict is irresolvable.


Example: Spare tire problem

Init(At(Flat, Axle) ∧ At(Spare,trunk))Goal(At(Spare,Axle))Action(Remove(Spare,Trunk)



Action(PutOn(Spare,Axle)PRECOND: At(Spare,Groundp) ∧¬At(Flat,Axle)EFFECT: At(Spare,Axle) ∧ ¬Ar(Spare,Ground))

Action(LeaveOvernightPRECOND:EFFECT: ¬ At(Spare,Ground) ∧ ¬ At(Spare,Axle) ∧ ¬ At(Spare,trunk) ∧ ¬At(Flat,Ground) ∧ ¬ At(Flat,Axle) )


Solving the problem

Initial plan: Start with EFFECTS and Finish with PRECOND.


Solving the problem

Initial plan: Start with EFFECTS and Finish with PRECOND. Pick an open precondition: At(Spare, Axle) Only PutOn(Spare, Axle) is applicable Add causal link: Add constraint : PutOn(Spare, Axle) < Finish

PutOn(Spare,Axle) At(Spare,Axle ) → Finish


Solving the problem

Pick an open precondition: At(Spare, Ground) Only Remove(Spare, Trunk) is applicable Add causal link: Add constraint : Remove(Spare, Trunk) < PutOn(Spare,Axle)

Re move(Spare,Trunk) At(Spare,Ground ) → PutOn(Spare,Axle)



Solving the problem

Pick an open precondition: ¬At(Flat, Axle) LeaveOverNight is applicable conflict: LeaveOverNight also has the effect ¬ At(Spare,Ground)

To resolve, add constraint : LeaveOverNight < Remove(Spare, Trunk)



Solving the problem

Pick an open precondition: At(Spare, Ground) LeaveOverNight is applicable conflict:

To resolve, add constraint : LeaveOverNight < Remove(Spare, Trunk) Add causal link:

LeaveOverNight ¬At(Spare,Ground ) → PutOn(Spare,Axle)


Solving the problem

Pick an open precondition: At(Spare, Trunk) Only Start is applicable Add causal link: Conflict: of causal link with effect At(Spare,Trunk) in LeaveOverNight

No re-ordering solution possible.

backtrack

Start At(Spare,Trunk ) → Re move(Spare,Trunk)


Solving the problem

Remove LeaveOverNight, Remove(Spare, Trunk) and causal links Repeat step with Remove(Spare,Trunk) Add also RemoveFlatAxle and finish


Some details …

What happens when a first-order representation that includes variables is used?Complicates the process of detecting and resolving

conflicts.Can be resolved by introducing inequality constraint.

CSP’s most-constrained-variable constraint can be used for planning algorithms to select a PRECOND.


Planning graphs

Used to achieve better heuristic estimates. A solution can also directly extracted using GRAPHPLAN.

Consists of a sequence of levels that correspond to time steps in the plan. Level 0 is the initial state. Each level consists of a set of literals and a set of actions.

Literals = all those that could be true at that time step, depending upon the actions executed at the preceding time step.

Actions = all those actions that could have their preconditions satisfied at that time step, depending on which of the literals actually hold.


Planning graphs

“Could”? Records only a restricted subset of possible negative

interactions among actions.

They work only for propositional problems. Example:

Init(Have(Cake))Goal(Have(Cake) ∧ Eaten(Cake))Action(Eat(Cake), PRECOND: Have(Cake)

EFFECT: ¬Have(Cake) ∧ Eaten(Cake))Action(Bake(Cake), PRECOND: ¬ Have(Cake)

EFFECT: Have(Cake))


Cake example

Start at level S0 and determine action level A0 and next level S1. A0 >> all actions whose preconditions are satisfied in the previous level. Connect precond and effect of actions S0 --> S1 Inaction is represented by persistence actions.

Level A0 contains the actions that could occur Conflicts between actions are represented by mutex links


Cake example

Level S1 contains all literals that could result from picking any subset of actions in A0 Conflicts between literals that can not occur together (as a consequence of the

selection action) are represented by mutex links. S1 defines multiple states and the mutex links are the constraints that define this

set of states. Continue until two consecutive levels are identical: leveled off

Or contain the same amount of literals (explanation follows later)


Cake example

A mutex relation holds between two actions when: Inconsistent effects: one action negates the effect of another. Interference: one of the effects of one action is the negation of a precondition of the other. Competing needs: one of the preconditions of one action is mutually exclusive with the

precondition of the other. A mutex relation holds between two literals when (inconsistent support):

If one is the negation of the other OR if each possible action pair that could achieve the literals is mutex.


PG and heuristic estimation

PG’s provide information about the problem A literal that does not appear in the final level of the graph

cannot be achieved by any plan. Useful for backward search (cost = inf).

Level of appearance can be used as cost estimate of achieving any goal literals = level cost.

Small problem: several actions can occur Restrict to one action using serial PG (add mutex links between

every pair of actions, except persistence actions). Cost of a conjunction of goals? Max-level, sum-level and set-

level heuristics.PG is a relaxed problem.


The GRAPHPLAN Algorithm

How to extract a solution directly from the PG

function GRAPHPLAN(problem) return solution or failuregraph ← INITIAL-PLANNING-GRAPH(problem)goals ← GOALS[problem]loop do

if goals all non-mutex in last level of graph then dosolution ← EXTRACT-SOLUTION(graph, goals, LENGTH(graph))if solution ≠ failure then return solutionelse if NO-SOLUTION-POSSIBLE(graph) then return failure

graph ← EXPAND-GRAPH(graph, problem)


Example: Spare tire problemInit(At(Flat, Axle) ∧ At(Spare,trunk))Goal(At(Spare,Axle))Action(Remove(Spare,Trunk)



Action(PutOn(Spare,Axle)PRECOND: At(Spare,Groundp) ∧¬At(Flat,Axle)EFFECT: At(Spare,Axle) ∧ ¬At(Spare,Ground))

Action(LeaveOvernightPRECOND:EFFECT: ¬ At(Spare,Ground) ∧ ¬ At(Spare,Axle) ∧ ¬ At(Spare,trunk) ∧ ¬ At(Flat,Ground) ∧ ¬At(Flat,Axle) )

This example goes beyond STRIPS: negative literal in pre-condition (ADL description)


GRAPHPLAN example

Initially the plan consist of 5 literals from the initial state and the CWA literals (S0).

Add actions whose preconditions are satisfied by EXPAND-GRAPH (A0) Also add persistence actions and mutex relations. Add the effects at level S1 Repeat until goal is in level Si


GRAPHPLAN example

EXPAND-GRAPH also looks for mutex relations Inconsistent effects

E.g. Remove(Spare, Trunk) and LeaveOverNight due to At(Spare,Ground) and not At(Spare, Ground) Interference

E.g. Remove(Flat, Axle) and LeaveOverNight At(Flat, Axle) as PRECOND and not At(Flat,Axle) as EFFECT

Competing needs E.g. PutOn(Spare,Axle) and Remove(Flat, Axle) due to At(Flat.Axle) and not At(Flat, Axle)

Inconsistent support E.g. in S2, At(Spare,Axle) and At(Flat,Axle)


GRAPHPLAN example

In S2, the goal literals exist and are not mutex with any other Solution might exist and EXTRACT-SOLUTION will try to find it

EXTRACT-SOLUTION can use Boolean CSP to solve the problem or a search process: Initial state = last level of PG and goal goals of planning problem Actions = select any set of non-conflicting actions that cover the goals in the state Goal = reach level S0 such that all goals are satisfied Cost = 1 for each action.


GRAPHPLAN example

Termination? YES PG are monotonically increasing or decreasing:

Literals increase monotonically Actions increase monotonically Mutexes decrease monotonically

Because of these properties and because there is a finite number of actions and literals, every PG will eventually level off !


Planning with propositional logic

Planning can be done by proving theorem in situation calculus. Here: test the satisfiability of a logical sentence:

Sentence contains propositions for every action occurrence. A model will assign true to the actions that are part of the correct plan

and false to the others An assignment that corresponds to an incorrect plan will not be a model

because of inconsistency with the assertion that the goal is true. If the planning is unsolvable the sentence will be unsatisfiable.

initial state ∧ all possible action descriptions ∧ goal


SATPLAN algorithm

function SATPLAN(problem, Tmax) return solution or failureinputs: problem, a planning problem

Tmax, an upper limit to the plan lengthfor T= 0 to Tmax do

cnf, mapping ← TRANSLATE-TO_SAT(problem, T)assignment ← SAT-SOLVER(cnf)

if assignment is not null then return EXTRACT-SOLUTION(assignment, mapping)

return failure


cnf, mapping ← TRANSLATE-TO_SAT(problem, T)

Distinct propositions for assertions about each time step.Superscripts denote the time step

At(P1,SFO)0 ∧ At(P2,JFK)0

No CWA thus specify which propositions are not true¬At(P1,SFO)0 ∧ ¬At(P2,JFK)0

Unknown propositions are left unspecified. The goal is associated with a particular time-stepBut which one?



How to determine the time step where the goal will be reached?Start at T=0

Assert At(P1,SFO)0 ∧ At(P2,JFK)0

Failure .. Try T=1 Assert At(P1,SFO)1 ∧ At(P2,JFK)1

…

Repeat this until some minimal path length is reached. Termination is ensured by Tmax



How to encode actions into PL?Propositional versions of successor-state axioms

At(P1,JFK)1 ⇔(At(P1,JFK)0 ∧ ¬(Fly(P1,JFK,SFO)0 ∧At(P1,JFK)0))∨ (Fly(P1,SFO,JFK)0 ∧ At(P1,SFO)0)

Such an axiom is required for each plane, airport and time step

If more airports add another way to travel than additional disjuncts are required

Once all these axioms are in place, the satisfiability algorithm can start to find a plan.


assignment ← SAT-SOLVER(cnf)

Multiple models can be found They are NOT satisfactory: (for T=1)

Fly(P1,SFO,JFK)0 ∧ Fly(P1,JFK,SFO)0 ∧ Fly(P2,JFK.SFO)0

The second action is infeasibleYet the plan IS a model of the sentence

Avoiding illegal actions: pre-condition axiomsFly(P1,SFO,JFK)0 ⇒ At(P1,JFK)

Exactly one model now satisfies all the axioms where the goal is achieved at T=1.

initial state ∧ all possible action descriptions ∧ goal1


assignment ← SAT-SOLVER(cnf)

A plane can fly at two destinations at once They are NOT satisfactory: (for T=1)

Fly(P1,SFO,JFK)0 ∧ Fly(P2,JFK,SFO)0 ∧ Fly(P2,JFK.LAX)0

The second action is infeasibleYet the plan allows spurious relations

Avoid spurious solutions: action-exclusion axioms¬(Fly(P2,JFK,SFO)0 ∧ Fly(P2,JFK,LAX))

Prevents simultaneous actions Lost of flexibility since plan becomes totally ordered : no

actions are allowed to occur at the same time. Restrict exclusion to preconditions


Analysis of planning approach

Planning is an area of great interest within AISearch for solutionConstructively prove a existence of solution

Biggest problem is the combinatorial explosion in states.

Efficient methods are under researchE.g. divide-and-conquer

Artificial Intelligence: Planning in the real world



Outline

Time, schedules and resourcesHierarchical task network

planningNon-deterministic domainsConditional planningExecution monitoring and

replanningContinuous planning

Multi-agent planning


Time, schedules and resources

Until know: what actions to do

Real-world: + actions occur at certain moments in time. + actions have a beginning and an end. + actions take a certain amount of time.

Job-shop scheduling: Complete a set of jobs, each of which consists of a sequence of

actions,Where each action has a given duration and might require

resources. Determine a schedule that minimizes the total time required to

complete all jobs (respecting resource constraints).


Car construction example

Init(Chassis(C1) ∧ Chassis(C2) ∧ Engine(E1,C1,30) ∧Engine(E1,C2,60) ∧ Wheels(W1,C1,30) ∧ Wheels(W2,C2,15))

Goal(Done(C1) ∧ Done(C2))Action(AddEngine(e,c,m)

PRECOND: Engine(e,c,d) ∧ Chassis(c) ∧ ¬EngineIn(c)EFFECT: EngineIn(c) ∧ Duration(d))

Action(AddWheels(w,c)PRECOND: Wheels(w,c,d) ∧ Chassis(c)EFFECT: WheelsOn(c) ∧ Duration(d))

Action(Inspect(c)PRECOND: EngineIn(c) ∧ WheelsOn(c) ∧ Chassis(c)EFFECT: Done(c) ∧ Duration(10))


Solution found by POPSlack of 15

critical path


Planning vs. scheduling

How does the problem differ from a standard planning problem?

When does an action start and when does it end? So next to order (planning) duration is also considered

Duration(d)

Critical path method is used to determine start and end times: Path = linear sequence from start to end Critical path = path with longest total duration

Determines the duration of the entire plan Critical path should be executed without delay


ES and LS

Earliest possible (ES) and latest possible (LS) start times.

LS-ES = slack of an action for all actions determines the schedule for the entire

problem.ES(Start) = 0ES(B)=maxA<B ES(A) + Duration(A)LS(Finish)=ES(Finish)LS(A) = minA<B LS(B) -Duration(A)

Complexity is O(Nb) (given a PO)


Scheduling with resources

Resource constraints = required material or objects to perform taskReusable resourcesA resource that is occupied during an action but becomes

available when the action is finished.Require extension of action syntax:

Resource:R(k)• k units of resource are required by the action.• Is a pre-requisite before the action can be performed.• Resource can not be used for k time units by other.


Car example with resourcesInit(Chassis(C1) ∧ Chassis(C2) ∧ Engine(E1,C1,30) ∧ Engine(E1,C2,60) ∧

Wheels(W1,C1,30) ∧ Wheels(W2,C2,15) ∧ EngineHoists(1) ∧ WheelStations(1) ∧Inspectors(2))

Goal(Done(C1) ∧ Done(C2))Action(AddEngine(e,c,m)

PRECOND: Engine(e,c,d) ∧ Chassis(c) ∧ ¬EngineIn(c)EFFECT: EngineIn(c) ∧ Duration(d),RESOURCE: EngineHoists(1))

Action(AddWheels(w,c)PRECOND: Wheels(w,c,d) ∧ Chassis(c)EFFECT: WheelsOn(c) ∧ Duration(d)RESOURCE: WheelStations(1))

Action(Inspect(c)PRECOND: EngineIn(c) ∧ WheelsOn(c) ∧ Chassis(c)EFFECT: Done(c) ∧ Duration(10)RESOURCE: Inspectors(1))

aggregation


Car example with resources


Scheduling with resources

Aggregation = group individual objects into quantities when the objects are undistinguishable with respect to their purpose.Reduces complexity

Resource constraints make scheduling problems more complicated.Additional interactions among actions

Heuristic: minimum slack algorithmSelect an action with all pre-decessors scheduled and

with the least slack for the earliest possible start.


Hierarchical task network planning

Reduce complexity ⇒ hierarchical decomposition At each level of the hierarchy a computational task is reduced to

a small number of activities at the next lower level. The computational cost of arranging these activities is low.

Hierarchical task network (HTN) planning uses a refinement of actions through decomposition. e.g. building a house = getting a permit + hiring a contractor +

doing the construction + paying the contractor. Refined until only primitive actions remain.

Pure and hybrid HTN planning.


Representation decomposition

General descriptions are stored in plan library.Each method = Decompos(a,d); a= action and d= PO

plan. See buildhouse example Start action supplies all preconditions of actions

not supplied by other actions.=external preconditions

Finish action has all effects of actions not present in other actions=external effects

Primary effects (used to achieve goal) vs. secondary effects


Buildhouse example

External precond External effects


Buildhouse example

Action(Buyland, PRECOND: Money, EFFECT: Land ∧ ¬Money)Action(GetLoan, PRECOND: Goodcredit, EFFECT: Money ∧ Mortgage)Action(BuildHouse, PRECOND: Land, EFFECT: House)Action(GetPermit, PRECOND: LAnd, EFFECT: Permit)Action(HireBuilder, EFFECT: Contract)Action(Construction, PRECOND: Permit ∧ Contract, EFFECT: HouseBuilt ∧ ¬Permit),Action(PayBuilder, PRECOND: Money ∧ HouseBuilt, EFFECT: ¬Money ∧ House ∧ ¬

¬Contract),Decompose(BuildHouse,

Plan ::STEPS{ S1: GetPermit, S2:HireBuilder, S3:Construction, S4 PayBuilder}ORDERINGS: {Start < S1 < S3< S4<Finish, Start<S2<S3},LINKS

Start Land → S1,Start Money → S4,S1 Permit → S3,S2 Contract → S3,

S3 HouseBuilt → S4,S4 house → Finish,S4 ¬Money → Finish


Properties of decomposition

Should be correct implementation of action a Correct if plan d is complete and consistent PO plan for the

problem of achieving the effects of a given the preconditions of a.

A decomposition is not necessarily unique. Performs information hiding:

STRIPS action description of higher-level action hides some preconditions and effects

Ignore all internal effects of decomposition Does not specify the intervals inside the activity during which

preconditions and effects must hold.

Information hiding is essential to HTN planning.


Recapitulation of POP (1)

Assume propositional planning problems:

The initial plan contains Start and Finish, the ordering constraint Start < Finish, no causal links, all the preconditions in Finish are open.

Successor function :picks one open precondition p on an action B andgenerates a successor plan for every possible

consistent way of choosing action A that achieves p.

Test goal


Recapitulation of POP (2)

When generating successor plan:The causal link A--p->B and the ordering constrain

A < B is added to the plan.If A is new also add start < A and A < B to the plan

Resolve conflicts between new causal link and all existing actions

Resolve conflicts between action A (if new) and all existing causal links.


Adapting POP to HTN planning

Remember POP?Modify the successor function: apply decomposition to

current plan

NEW Successor function:Select non-primitive action a’ in PFor any Decompose(a’,d’) method in library where a and

a’ unify with substitution θReplace a’ with d’ = subst(θ,d)


POP+HTN examplea’


POP+HTN example

d

a’


How to hook up d in a’?

Remove action a’ from P and replace with dθ For each step s in d’ select an action that will play the role of s (either new s or

existing s’ from P) Possibility of subtask sharing

Connect ordering steps for a’ to the steps in d’ Put all constraints so that constraints of the form

B < a’ are maintained. Watch out for too strict orderings !

Connect the causal links If B -p-> a’ is a causal link in P, replace it by a set of causal links from B to all

steps in d’ with preconditions p that were supplied by the start step Idem for a’ -p-> C


What about HTN?

Additional modification to POP are necessary BAD news: pure HTN planning is undecidable due to

recursive decomposition actions.Walk=make one step and walk

Resolve problems by Rule out recursion. Bound the length of relevant solutions, Hybridize HTN with POP

Yet HTN can be efficient (see motivations in book)


The Gift of magi


Non-deterministic domains

So far: fully observable, static and deterministic domains. Agent can plan first and then execute plan with eyes closed

Uncertain environment: incomplete (partially observable and/or nondeterministic) and incorrect (differences between world and model) information Use percepts Adapt plan when necessary

Degree of uncertainty defined by indeterminacy Bounded: actions can have unpredictable effects, yet can be listed in action

description axioms. Unbounded: preconditions and effects unknown or to large to enumerate.


Handling indeterminacy

Sensorless planning (conformant planning) Find plan that achieves goal in all possible circumstances

(regardless of initial state and action effects). Conditional planning (Contingency planning)

Construct conditional plan with different branches for possible contingencies.

Execution monitoring and replanning While constructing plan judge whether plan requires revision.

Continuous planning Planning active for a life time: adapt to changed circumstances and

reformulate goals if necessary.


Sensorless planning


Abstract example

Initial state = <chair,table, cans of paint, unknown colors>, goal state=<color(table) = color(chair)>

Sensorless planning (conformant planning) Open any can of paint and apply it to both chair and table.

Conditional planning (Contingency planning) Sense color of table and chair, if they are the same then finish else sense labels

paint if color(label) =color(Furniture) then apply color to othe piece else apply color to both

Execution monitoring and replanning Same as conditional and can fix errors (missed spots)

Continuous planning Can revise goal when we want to first eat before painting the table and the chair.


Conditional planning

Deal with uncertainty by checking the environment to see what is really happening.

Used in fully observable and nondeterministic environments: The outcome of an action is unknown.Conditional steps will check the state of the environment.How to construct a conditional plan?


Example, the vacuum-world



Actions: left, right, suck Propositions to define states: AtL, AtR, CleanL, CleanR How to include indeterminism?

Actions can have more than one effect E.g. moving left sometimes fails

Action(Left, PRECOND: AtR, EFFECT: AtL)Becomes : Action(Left, PRECOND: AtR, EFFECT: AtL∨AtR)

Actions can have conditional effectsAction(Left, PRECOND:AtR, EFFECT: AtL∨(AtL∧when cleanL: ¬cleanL)

Both disjunctive and conditional



Conditional plans require conditional steps: If <test> then plan_A else plan_B

if AtL∧CleanL then Right else Suck

Plans become trees Games against nature:

Find conditional plans that work regardless of which action outcomes actually occur.

Assume vacuum-worldInitial state = AtR ∧ CleanL ∧ CleanRDouble murphy: possibility of desposit dirt when moving to other square and

possibility of despositing dirt when action is Suck.


Game treeState node

chance node


Solution of games against N.

Solution is a subtree that Has a goal node at every leaf Specifies one action at each of its state nodes Includes every outcome branch at each of the chance nodes.

In previous example:[Left, if AtL ∧ CleanL ∧ CleanR then [] else Suck]

For exact solutions: use minimax algorithm with 2 modifications: Max and Min nodes become OR and AND nodes Algorithm returns conditional plan instead of single move


And-Or-search algorithm

function AND-OR-GRAPH-SEARCH(problem) returns a conditional plan or failurereturn OR-SEARCH(INITIAL-STATE[problem], problem, [])

function AND-SEARCH(state_set, problem, path) returns a conditional plan or failurefor each si in state_set do

plani ← OR-SEARCH(si, problem,path )if plan = failure then return failure

return [ if s1 then plan1 else if s2 then plan2 else … if sn-1 then plann-1 else plann]

function OR-SEARCH(state, problem, path) returns a conditional plan or failureif GOAL-TEST[problem](state) then return the empty planif state is on path then return failurefor action,state_set in SUCCESSORS[problem](state) do

plan ← AND-SEARCH(state_set, problem, [state | plan] )if plan ≠ failure then return [action | plan]

return failure



How does it deal with cycles?When a state that already is on the path appears, return

failureNo non-cyclic solution

Ensures algorithm termination The algorithm does not check whether some state is already on

some other path from the root.



Sometimes only a cyclic solution exists e.g. tripple murphy: sometimes the move is not performed

[Left, if CleanL then [] else Suck] is not a solution Use label to repeat parts of plan (but infinite loops)

[L1: Left, if AtR then L1 else if CleanL then [] else Suck]


CP and partially observable env.

Fully observable: conditional tests can ask any question and get an answer

Partially observable??? The agent has limited information about the environment. Modeled by a state-set = belief states E.g. assume vacuum agent which can not sense presence or

absence of dirt in other squares than the one it is on. + alternative murphy: dirt can be left behind when moving to other square. Solution in fully observable world: keep moving left and right, sucking dirt

whenever it appears until both squares are clean and I’m in square left.


PO: alternate double murphy


Belief states

Representation?Sets of full state descriptions

{(AtR∧CleanR∧CleanL) ∨ (AtR∧CleanR∧¬CleanL)}Logical sentences that capture the set of possible

worlds in the belief state (OWA)AtR ∧ CleanR

Knowledge propositions describing the agent’s knowledge (CWA)

K(AtR) ∧ K(CleanR)


Belief states

Choice 2 and 3 are equivalent (let’s continue with 3) Symbols can appear in three ways in three ways: positive, negative or

unknown: 3n possible belief states for n proposition symbols. YET, set of belief sets is a power set of the phyiscal states which is much

larger than 3n

Hence 3 is restricted as representationAny scheme capable of representing every possible belief state will require O(2n) bit to

represent each one in the worst case. The current scheme only requires O(n)


Sensing in Cond. Planning

How does it work?Automatic sensing

At every time step the agent gets all available perceptsActive sensing

Percepts are obtained through the execution of specific sensory actions.

checkDirt and checkLocation

Given the representation and the sensing, action descriptions can now be formulated.


Monitoring and replanning

Execution monitoring: check whether everything is going as planned.Unbounded indeterminancy: some unanticipated

circumstances will arise.A necessity in realistic environments.

Kinds of monitoring:Action monitoring: verify whether the next action will work.Plan monitoring: verify the entire remaining plan.


Monitoring and replanning

When something unexpected happens: replanTo avoid too much time on planning try to repair the old plan.

Can be applied in both fully and partially observable environments, and to a variety of planning representations.


Replanning-agentfunction REPLANNING-AGENT(percept) returns an action

static: KB, a knowledge base (+ action descriptions)plan, a plan initially []whole_plan, a plan initially []goal, a goal

TELL(KB, MAKE-PERCEPT-SENTENCE(percept,t))current ← STATE-DESCRIPTION(KB,t)if plan = [] then return the empty plan

whole_plan ← plan ← PLANNER(current, goal, KB)if PRECONDITIONS(FIRST(plan)) not currently true in KB then

candidates ← SORT(whole_plan,ordered by distance to current) find state s in candidates such that

failure ≠ repair ← PLANNER(current, s, KB)continuation ← the tail of whole_plan starting at swhole_plan ← plan ← APPEND(repair, continuation)

return POP(plan)


Repair example


Repair example: paintingInit(Color(Chair, Blue) ∧Color(Table,Green) ∧ ContainsColor(BC,Blue) ∧ PaintCan(BC) ∧

ContainsColor(RC,Red) ∧ PaintCan(RC)) Goal(Color(Chair,x) ∧ Color(Table,x))Action(Paint(object, color)

PRECOND: HavePaint(color)EFFECT: Color(object, color))

Action(Open(can)PRECOND: PaintCan(can) ∧ ContainsColor(can,color)EFFECT: HavePaint(color))

[Start; Open(BC); Paint(Table,Blue), Finish]


Repair example: painting

Suppose that the agent now perceives that the colors of table and chair are different Figure out point in whole_plan to aim for

Current state is identical as the precondition before Paint Repair action sequence to get there.

Repair =[] and plan=[Paint, Finish] Continue performing this new plan

Will loop until table and chair are perceived as the same. Action monitoring can lead to less intelligent behavior

Assume the red is selected and there is not enough paint to apply to both chair and table.

Improved by doing plan monitoring


Plan monitoring

Check the preconditions for success of the entireplan.Except those which are achieved by another step in the

plan.Execution of doomed plan is cut of earlier.

Limitation of replanning agent: It can not formulate new goals or accept new goals in

addition to the current one


Continuous planning.

Agent persists indefinitely in an environmentPhases of goal formulation, planning and acting

Execution monitoring + planner as one continuous process

Example:Blocks worldAssume a fully observable environmentAssume partially ordered plan


Block world example

Initial state (a) Action(Move(x,y),

PRECOND: Clear(x) ∧ Clear(y) ∧ On(x,z)EFFECT: On(x,y) ∧ Clear(z) ∧ ¬On(x,z) ∧ ¬Clear(y)

The agent first need to formulate a goal: On(C,D) ∧ On(D,B) Plan is created incrementally, return NoOp and check percepts


Block world example

Assume that percepts don’t change and this plan is constructed Ordering constraint between Move(D,B) and Move(C,D) Start is label of current state during planning. Before the agent can execute the plan, nature intervenes:

D is moved onto B


Block world example

Start contains now On(D,B) Agent perceives: Clear(B) and On(D,G) are no longer true

Update model of current state (start) Causal links from Start to Move(D,B) (Clear(B) and On(D,G)) no longer valid. Remove causal relations and two PRECOND of Move(D,B) are open Replace action and causal links to Finish by connecting Start to Finish.


Block world example

Extending: whenever a causal link can be supplied by a previous step All redundant steps (Move(D,B) and its causal links) are removed from the

plan Execute new plan, perform action Move(C,D)

This removes the step from the plan

Extending causal link


Block world example

Execute new plan, perform action Move(C,D) Assume agent is clumsy and drops C on A

No plan but still an open PRECOND Determine new plan for open condition Again Move(C,D)


Block world example

Similar to POP On each iteration find plan-flaw and fix it Possible flaws: Missing goal, Open precondition, Causal conflict,

Unsupported link, Redundant action, Unexecuted action, unnecessary historical goal


Multi-agent planning

So far we only discussed single-agent environments. Other agents can simply be added to the model of the world:

Poor performance since agents are not indifferent ot other agents’ intentions

In general two types of multi-agent environments: Cooperative Competitive


Cooperation: Joint goals and plans

Multi-planning problem: assume double tennis example where agents want to return ball.

Agents(A,B)Init(At(A,[Left,Baseline])∧ At(B,[Right, Net]) ∧ Approaching(Ball,[Right, Baseline]) ∧ PArtner(A,B) ∧ Partner(B,A)) Goal(Returned(Ball) ∧ At(agent,[x,Net]))Action(Hit(agent, Ball)

PRECOND: Approaching(Ball,[x,y]) ∧ At(agent,[x,y]) ∧ Partner(agent, partner) ∧ ¬At(partner,[x,y])EFFECT: Returned(Ball))

Action(Go(agent,[x,y])PRECOND: At(agent,[a,b])EFFECT: At(agent,[x,y]) ∧ ¬ At(agent,[a,b]))


Cooperation: Joint goals and plans

A solution is a joint-plan consisting of actions for both agents.

Example:A: [Go(A,[Right, Baseline]), Hit(A,Ball)]B: [NoOp(B), NoOp(B)]

OrA: [Go(A,[Left, net), NoOp(A)]B: [Go(B,[Right, Baseline]), Hit(B, Ball)]

Coordination is required to reach same joint plan


Multi-body planning

Planning problem faced by a single centralized agent that can dictate action to each of several physical entities.

Hence not truly multi-agent Important: synchronization of actions

Assume for simplicity that every action takes one time step and at each point in the joint plan the actions are performed simultaneously

[<Go(A,[Left,Net]), Go(B,[Right,Baseline]>;<NoOp(A), Hit(B, Ball)>]

Planning can be performed using POP applied to the set of all possible joint actions. Size of this set???


Multi-body planning

Alternative to set of all joint actions: add extra concurrency lines to action description Concurrent action

Action(Hit(A, Ball)CONCURRENT: ¬Hit(B,Ball) PRECOND: Approaching(Ball,[x,y]) ∧ At(A,[x,y]) EFFECT: Returned(Ball))

Required actions (carrying object by two agents)Action(Carry(A, cooler, here, there)

CONCURRENT: Carry(B,cooler, here there) PRECOND: …)

Planner similar to POP with some small changes in possible ordering relations


Coordination mechanisms

To ensure agreement on joint plan: use convention. Convention = a constraint on the selection of joint plans (beyond the

constraint that the joint plan must work if the agents adopt it).e.g. stick to your court or one player stays at the net.

Conventions which are widely adopted= social laws e.g. language. Can be domain-specific or independent. Could arise through evolutionary process (flocking behavior).


Flocking example

Three rules:Separation:

Steer away from neighbors when you get too closeCohesion

Steer toward the average position of neighbors Alignment

Steer toward average orientation (heading) of neighbors

Flock exhibits emergent behavior of flying as a pseudo-rigid body.


Coordination mechanisms

In the absence of conventions: Communicatione.g. Mine! Or Yours! in tennis example

The burden of arriving at a succesfull joint plan can be placed onAgent designer (agents are reactive, no explicit models of

other agents)Agent (agents are deliberative, model of other agents

required)


Competitive environments

Agents can have conflicting utilities e.g. zero-sum games like chess

The agent must: Recognize that there are other agents Compute some of the other agents plans Compute how the other agents interact with its own plan Decide on the best action in view of these interactions.

Model of other agent is required YET, no commitment to joint action plan.

17.05.11 AI NN 1

Artificial Intelligence: Neural Networks


17.05.11 AI NN 2

Neural NetworksMcCulloch & Pitts (1943) are generally

recognised as the designers of the first neural network

Many of their ideas still used today (e.g. many simple units combine to give increased computational power and the idea of a threshold)

17.05.11 AI NN 3

Neural Networks

Hebb (1949) developed the first learning rule (on the premise that if two neurons were active at the same time the strength between them should be increased)

17.05.11 AI NN 4

Neural Networks During the 50’s and 60’s many

researchers worked on the perceptron amidst great excitement.

1969 saw the death of neural network research for about 15 years – Minsky & Papert

Only in the mid 80’s (Parker and LeCun) was interest revived (in fact Werbos discovered algorithm in 1974)

17.05.11 AI NN 5

Neural Networks

17.05.11 AI NN 6

Neural Networks

We are born with about 100 billion neurons

A neuron may connect to as many as 100,000 other neurons

17.05.11 AI NN 7

Neural Networks Signals “move” via electrochemical signals

The synapses release a chemical transmitter – the sum of which can cause a threshold to be reached – causing the neuron to “fire”

Synapses can be inhibitory or excitatory

17.05.11 AI NN 8

The First Neural Neural Networks

McCulloch and Pitts produced the first neural network in 1943

Many of the principles can still be seen in neural networks of today

17.05.11 AI NN 9


-1

2

2X1

X2

X3

Y

The activation of a neuron is binary. That is, the neuron either fires (activation of one) or does not fire (activation of zero).

17.05.11 AI NN 10


-1

2

2X1

X2

X3

Y

For the network shown here the activation function for unit Y is

f(y_in) = 1, if y_in >= θ else 0

where y_in is the total input signal receivedθ is the threshold for Y

17.05.11 AI NN 11


-1

2

2X1

X2

X3

Y

Neurons in a McCulloch-Pitts network are connected by directed, weighted paths

17.05.11 AI NN 12


-1

2

2X1

X2

X3

Y

If the weight on a path is positive the path is excitatory, otherwise it is inhibitory

17.05.11 AI NN 13


-1

2

2X1

X2

X3

Y

All excitatory connections into a particular neuron have the same weight, although different weighted connections can be input to different neurons

17.05.11 AI NN 14


-1

2

2X1

X2

X3

Y

Each neuron has a fixed threshold. If the net input into the neuron is greater than or equal to the threshold, the neuron fires

17.05.11 AI NN 15


-1

2

2X1

X2

X3

Y

The threshold is set such that any non-zero inhibitory input will prevent the neuron from firing

17.05.11 AI NN 16


-1

2

2X1

X2

X3

Y

It takes one time step for a signal to pass over one connection.

17.05.11 AI NN 17


AND Function

1

1X1

X2

Y

ANDX1 X2 Y

1 1 11 0 00 1 00 0 0

Threshold(Y) = 2

17.05.11 AI NN 18


AND FunctionOR Function

2

2X1

X2

Y

ORX1 X2 Y

1 1 11 0 10 1 10 0 0

Threshold(Y) = 2

17.05.11 AI NN 19


AND NOT Function

-1

2X1

X2

Y

ANDNOTX1 X2 Y

1 1 01 0 10 1 00 0 0

Threshold(Y) = 2

17.05.11 AI NN 20


XOR Function

2

2

2

2

-1

-1

Z1

Z2

Y

X1

X2

XORX1 X2 Y

1 1 01 0 10 1 10 0 0

X1 XOR X2 = (X1 AND NOT X2) OR (X2 AND NOT X1)

17.05.11 AI NN 21

The First Neural Neural NetworksIf we touch something cold we perceive heat

If we keep touching something cold we will perceive cold

If we touch something hot we will perceive heat

17.05.11 AI NN 22

The First Neural Neural NetworksTo model this we will assume that time is discrete

If cold is applied for one time step then heat will be perceived

If a cold stimulus is applied for two time steps then cold will be perceived

If heat is applied then we should perceive heat

17.05.11 AI NN 23


X1

X2

Z1

Z2

Y1

Y2

Heat

Cold

2

2

2

12

-1

1

Hot

Cold

17.05.11 AI NN 24


X1

X2

Z1

Z2

Y1

Y2

Heat

Cold

2

2

2

12

-1

1

Hot

Cold

• It takes time for the stimulus (applied at X1 and X2) to make its way to Y1 and Y2where we perceive either heat or cold

• At t(0), we apply a stimulus to X1 and X2

• At t(1) we can update Z1, Z2 and Y1

• At t(2) we can perceive a stimulus at Y2

• At t(2+n) the network is fully functional

17.05.11 AI NN 25

The First Neural Neural NetworksWe want the system to perceive cold if a cold stimulus is applied for two time steps

Y2(t) = X2(t – 2) AND X2(t – 1)

X2(t – 2) X2( t – 1) Y2(t)1 1 11 0 00 1 00 0 0

17.05.11 AI NN 26

The First Neural Neural NetworksWe want the system to perceive heat if either a hot stimulus is applied or a cold stimulus is applied (for one time step) and then removed

Y1(t) = [ X1(t – 1) ] OR [ X2(t – 3) AND NOT X2(t – 2) ]

X2(t – 3) X2(t – 2) AND NOT X1(t – 1) OR1 1 0 1 11 0 1 1 10 1 0 1 10 0 0 1 11 1 0 0 01 0 1 0 10 1 0 0 00 0 0 0 0

17.05.11 AI NN 27

The First Neural Neural NetworksThe network shows

Y1(t) = X1(t – 1) OR Z1(t – 1)

Z1(t – 1) = Z2( t – 2) AND NOT X2(t – 2)

Z2(t – 2) = X2(t – 3)

Substituting, we get

Y1(t) = [ X1(t – 1) ] OR [ X2(t – 3) AND NOT X2(t – 2) ]

which is the same as our original requirements

17.05.11 AI NN 28

Modelling a Neuron

aj :Activation value of unit j wj,I :Weight on the link from unit j to unit i inI :Weighted sum of inputs to unit i aI :Activation value of unit i g :Activation function

in i=∑ jW j , i a j

17.05.11 AI NN 29

Activation Functions

Stept(x)= 1 if x >= t, else 0

Sign(x) = +1 if x >= 0, else –1

Sigmoid(x) = 1/(1+e-x)

17.05.11 AI NN 30

Simple Networks

AND OR NOTInput 1 0 0 1 1 0 0 1 1 0 1Input 2 0 1 0 1 0 1 0 1Output 0 0 0 1 0 1 1 1 1 0

17.05.11 AI NN 31

Simple Networks

t = 0.0

y

x

W = 1.5

W = 1

-1

17.05.11 AI NN 32

Perceptron• Synonym for Single-

Layer, Feed-Forward Network

• First Studied in the 50’s

• Other networks were known about but the perceptron was the only one capable of learning and thus all research was concentrated in this area

17.05.11 AI NN 33

Perceptron• A single weight only

affects one output so we can restrict our investigations to a model as shown on the right

• Notation can be simpler, i.e.

O= Step 0∑ j WjIj

17.05.11 AI NN 34

What can perceptrons represent?

AND XORInput 1 0 0 1 1 0 0 1 1Input 2 0 1 0 1 0 1 0 1Output 0 0 0 1 0 1 1 0

17.05.11 AI NN 35


0,0

0,1

1,0

1,1

0,0

0,1

1,0

1,1

AND XOR

• Functions which can be separated in this way are called Linearly Separable

• Only linearly Separable functions can be represented by a perceptron

17.05.11 AI NN 36


Linear Separability is also possible in more than 3 dimensions –but it is harder to visualise

17.05.11 AI NN 37

Training a perceptron

ANDInput 1 0 0 1 1Input 2 0 1 0 1Output 0 0 0 1

Aim

17.05.11 AI NN 38

Training a perceptrons

t = 0.0

y

x

-1W = 0.3

W = -0.4

W = 0.5

I1 I2 I3 Summation Output-1 0 0 (-1*0.3) + (0*0.5) + (0*-0.4) = -0.3 0-1 0 1 (-1*0.3) + (0*0.5) + (1*-0.4) = -0.7 0-1 1 0 (-1*0.3) + (1*0.5) + (0*-0.4) = 0.2 1-1 1 1 (-1*0.3) + (1*0.5) + (1*-0.4) = -0.2 0

17.05.11 AI NN 39

LearningWhile epoch produces an error

Present network with next inputs from epoch

Err = T – OIf Err <> 0 then

Wj = Wj + LR * Ij * ErrEnd If

End While

17.05.11 AI NN 40

LearningWhile epoch produces an error

Present network with next inputs from epoch

Err = T – OIf Err <> 0 then


End While

Epoch : Presentation of the entire training set to the neural network.In the case of the AND function an epoch consists of four sets of inputs being presented to the network (i.e. [0,0], [0,1], [1,0], [1,1])

17.05.11 AI NN 41

Learning

Training Value, T : When we are training a network we not only present it with the input but also with a value that we require the network to produce. For example, if we present the network with [1,1] for the AND function the training value will be 1

While epoch produces an errorPresent network with next inputs from

epoch Err = T – OIf Err <> 0 then


End While

17.05.11 AI NN 42

Learning

Error, Err : The error value is the amount by which the value output by the network differs from the training value. For example, if we required the network to output 0 and it output a 1, then Err = -1




End While

17.05.11 AI NN 43

Learning

Output from Neuron, O : The output value from the neuron

Ij : Inputs being presented to the neuron

Wj : Weight from input neuron (Ij) to the output neuron

LR : The learning rate. This dictates how quickly the network converges. It is set by a matter of experimentation. It is typically 0.1




End While

17.05.11 AI NN 44

Learning

0,0

0,1

1,0

1,1I1

I2

After First Epoch

0,0

0,1

1,0

1,1I1

I2

At Convergence

NoteI1 point = W0/W1I2 point = W0/W2

17.05.11 AI NN 45

Biological Neural Nets

Pigeons as art experts (Watanabe et al.1995)

Experiment: Pigeon in Skinner box Present paintings of two different artists (e.g. Chagall

/ Van Gogh) Reward for pecking when presented a particular artist

(e.g. Van Gogh)

17.05.11 AI NN 46

17.05.11 AI NN 47

Presenter

Presentation Notes

Canal with Women Washing, 1888

17.05.11 AI NN 48

17.05.11 AI NN 49

Pigeons were able to discriminate between Van Gogh and Chagall with 95% accuracy (when presented with pictures they had been trained on)

Discrimination still 85% successful for previously unseen paintings of the artists

Pigeons do not simply memorise the pictures They can extract and recognise patterns (the ‘style’) They generalise from the already seen to make

predictions

This is what neural networks (biological and artificial) are good at (unlike conventional computer)

17.05.11 AI NN 50

Feeding data through the net:

(1 × 0.25) + (0.5 × (-1.5)) = 0.25 + (-0.75) = - 0.5

11+ e 0 . 5

= 0 . 3775Squashing:

17.05.11 AI NN 51

Data is presented to the network in the form of activations in the input layer

Examples Pixel intensity (for pictures) Molecule concentrations (for artificial nose) Share prices (for stock market prediction)

Data usually requires preprocessing Analogous to senses in biology

How to represent more abstract data, e.g. a name? Choose a pattern, e.g.

0-0-1 for “Chris” 0-1-0 for “Becky”

17.05.11 AI NN 52

• Weight settings determine the behaviour of a network

How can we find the right weights?

17.05.11 AI NN 53

Training the Network - Learning

Backpropagation Requires training set (input / output pairs) Starts with small random weights Error is used to adjust weights (supervised learning) Gradient descent on error landscape

17.05.11 AI NN 54

17.05.11 AI NN 55

Advantages It works! Relatively fast

Downsides Requires a training set Can be slow Probably not biologically realistic

Alternatives to Backpropagation Hebbian learning

Not successful in feed-forward nets Reinforcement learning

Only limited success Artificial evolution

More general, but can be even slower than backprop

17.05.11 AI NN 56

Example: Voice Recognition

Task: Learn to discriminate between two different voices saying “Hello”

Data Sources

Steve Simpson David Raubenheimer

Format Frequency distribution (60 bins) Analogy: cochlea

17.05.11 AI NN 57

Network architecture Feed forward network

60 input (one for each frequency bin) 6 hidden 2 output (0-1 for “Steve”, 1-0 for “David”)

17.05.11 AI NN 58

Presenting the dataSteve

David

17.05.11 AI NN 59

Presenting the data (untrained network)Steve

David

0.43

0.26

0.73

0.55

17.05.11 AI NN 60

Calculate errorSteve

David

0.43 – 0 = 0.43

0.26 –1 = 0.74

0.73 – 1 = 0.27

0.55 – 0 = 0.55

17.05.11 AI NN 61

Backprop error and adjust weightsSteve

David

0.43 – 0 = 0.43

0.26 – 1 = 0.74

0.73 – 1 = 0.27

0.55 – 0 = 0.55

1.17

0.82

17.05.11 AI NN 62

Repeat process (sweep) for all training pairs Present data Calculate error Backpropagate error Adjust weights

Repeat process multiple times

17.05.11 AI NN 63

Presenting the data (trained network)Steve

David

0.01

0.99

0.99

0.01

17.05.11 AI NN 64

Results – Voice Recognition

Performance of trained network

Discrimination accuracy between known “Hello”s• 100%

Discrimination accuracy between new “Hello”’s• 100%

17.05.11 AI NN 65

Results – Voice Recognition (ctnd.)

Network has learnt to generalise from original data

Networks with different weight settings can have same functionality

Trained networks ‘concentrate’ on lower frequencies

Network is robust against non-functioning nodes

17.05.11 AI NN 66

Applications of Feed-forward nets Pattern recognition

Character recognition Face Recognition

Sonar mine/rock recognition (Gorman & Sejnowksi, 1988)

Navigation of a car (Pomerleau, 1989)

Stock-market prediction

Pronunciation (NETtalk)(Sejnowksi & Rosenberg, 1987)

17.05.11 AI NN 67

Cluster analysis of hidden layer

17.05.11 AI NN 68

FFNs as Biological Modelling Tools

Signalling / Sexual Selection Enquist & Arak (1994)

Preference for symmetry not selection for ‘good genes’, but instead arises through the need to recognise objects irrespective of their orientation

Johnstone (1994) Exaggerated, symmetric ornaments facilitate

mate recognition

(but see Dawkins & Guilford, 1995)

17.05.11 AI NN 69

Recurrent Networks

Feed forward networks: Information only flows one way One input pattern produces one output No sense of time (or memory of previous state)

Recurrency Nodes connect back to other nodes or themselves Information flow is multidirectional Sense of time and memory of previous state(s)

Biological nervous systems show high levels of recurrency (but feed-forward structures exists too)

17.05.11 AI NN 70

Elman Nets

Elman nets are feed forward networks with partial recurrency

Unlike feed forward nets, Elman nets have a memoryor sense of time

17.05.11 AI NN 71

Classic experiment on language acquisition and processing (Elman, 1990)

Task Elman net to predict successive words in sentences.

Data Suite of sentences, e.g.

“The boy catches the ball.” “The girl eats an apple.”

Words are input one at a time

Representation Binary representation for each word, e.g.

0-1-0-0-0 for “girl”

Training method Backpropagation

17.05.11 AI NN 72

Internal representation of words

17.05.11 AI NN 73

Hopfield Networks

Sub-type of recurrent neural nets Fully recurrent Weights are symmetric Nodes can only be on or off Random updating

Learning: Hebb rule (cells that fire together wire together)

Biological equivalent to LTP and LTD

Can recall a memory, if presented with acorrupt or incomplete version

auto-associative orcontent-addressable memory

17.05.11 AI NN 74

Task: store images with resolution of 20x20 pixels Hopfield net with 400 nodes

Memorise:1. Present image2. Apply Hebb rule (cells that fire together, wire together)

Increase weight between two nodes if both have same activity, otherwise decrease

3. Go to 1

Recall:1. Present incomplete pattern2. Pick random node, update3. Go to 2 until settled

17.05.11 AI NN 75

Memories are attractors in state space

17.05.11 AI NN 76

Catastrophic forgetting

Problem: memorising new patterns corrupts the memory of older ones Old memories cannot be recalled, or spurious

memories arise

• Solution: allow Hopfield net to sleep

17.05.11 AI NN 77

Two approaches (both using randomness): Unlearning (Hopfield, 1986)

Recall old memories by random stimulation, but use an inverse Hebb rule

‘Makes room’ for new memories (basins of attraction shrink)

Pseudorehearsal (Robins, 1995)

While learning new memories, recall old memories by random stimulation

Use standard Hebb rule on new and old memories Restructure memory• Needs short-term + long term memory Mammals: hippocampus plays back new memories to

neo-cortex, which is randomly stimulated at the same time

17.05.11 AI NN 78

RNNs as Central Pattern Generators

CPGs: group of neurones creating rhythmic muscle activity for locomotion, heart-beat etc.

Identified in several invertebrates and vertebrates

Hard to study

Computer modelling E.g. lamprey swimming (Ijspeert et al., 1998)

17.05.11 AI NN 79

Evolution of Bipedal Walking (Reil & Husbands, 2001)

17.05.11 AI NN 80

CPG cycles are cyclic attractors in state space

17.05.11 AI NN 81

Recap – Neural Networks Components – biological plausibility

Neurone / node Synapse / weight

Feed forward networks Unidirectional flow of information Good at extracting patterns, generalisation and

prediction Distributed representation of data Parallel processing of data Training: Backpropagation Not exact models, but good at demonstrating

principles

Recurrent networks Multidirectional flow of information Memory / sense of time Complex temporal dynamics (e.g. CPGs) Various training methods (Hebbian, evolution) Often better biological models than FFNs

17.05.11 AI NN 82

Supervised Vs. Unsupervised

Previously discussed networks are ‘supervised’Need to be trained ahead of time with lots of

data

Unsupervised networks adapt to the input Applications in Clustering and reducing

dimensionality Learning may be very slow

17.05.11 AI NN 83

Current Applications

Investment Analysis Predicting movement of stocks Replacing earlier linear models

Signature Analysis Bank Checks, VISA, etc.

Process Control Chemistry related

Monitoring Sensor networks may gather more data than can be processed

by operators Inputs: Cues from camera data, vibration levels, sound, radar,

lydar, etc. Output: Number of people at a terminal, engine warning light,

control for light switch

17.05.11 AI NN 84

Current Work

Work of S. Cho, Chow and C. Leung Training Neural Network to approximate crowd

size based on image cues Network approximates a non linear mapping

between input and output space Their input cues: Black Pixels, White Pixels,

Edge Pixels Our proposed inputs: Edge Histogram, Blob

Size histogram (based on background model)

17.05.11 AI NN 85

References

[1] L. Smith, ed. (1996, 2001), "An Introduction to Neural Networks", URL: http://www.cs.stir.ac.uk/~lss/NNIntro/InvSlides.html

[2] Sarle, W.S., ed. (1997), Neural Network FAQ, URL: ftp://ftp.sas.com/pub/neural/FAQ.html

[3] StatSoft, "Neural Networks", URL: http://www.statsoftinc.com/textbook/stneunet.html

[4] S. Cho, T. Chow, and C. Leung, "A Neural-Based Crowd Estimation by Hybrid Global Learning Algorithm", IEEE Transactions on Systems, Man and Cybernetics, Part B, No. 4. 1999.

http://www.cs.stir.ac.uk/~lss/NNIntro/InvSlides.html

ftp://ftp.sas.com/pub/neural/FAQ.html

http://www.statsoftinc.com/textbook/stneunet.html

What is an Evolutionary Algorithm?

Lars Mehnen

Contents

Recap of Evolutionary MetaphorBasic scheme of an EABasic Components:Representation / Evaluation / Population /

Parent Selection / Recombination / Mutation / Survivor Selection / Termination

Examples : eight queens / knapsackTypical behaviours of EAsEC in context of global optimisation

Recap of EC metaphor

A population of individuals exists in an environment with limited resources

Competition for those resources causes selection of those fitter individuals that are better adapted to the environment

These individuals act as seeds for the generation of new individuals through recombination and mutation

The new individuals have their fitness evaluated and compete (possibly also with parents) for survival.

Over time Natural selection causes a rise in the fitness of the population

Recap 2:

EAs fall into the category of “generate and test” algorithms

They are stochastic, population-based algorithms Variation operators (recombination and mutation) create

the necessary diversity and thereby facilitate novelty Selection reduces diversity and acts as a force pushing

quality

General Scheme of EAs

Pseudo-code for typical EA

What are the different types of EAs

Historically different flavours of EAs have been associated with different representations Binary strings : Genetic Algorithms Real-valued vectors : Evolution Strategies Finite state Machines: Evolutionary Programming LISP trees: Genetic Programming

These differences are largely irrelevant, best strategy choose representation to suit problem choose variation operators to suit representation

Selection operators only use fitness and so are independent of representation

Representations

Candidate solutions (individuals) exist in phenotype space

They are encoded in chromosomes, which exist in genotype space Encoding : phenotype=> genotype (not necessarily one to one) Decoding : genotype=> phenotype (must be one to one)

Chromosomes contain genes, which are in (usually fixed) positions called loci (sing. locus) and have a value (allele)In order to find the global optimum, every feasible

solution must be represented in genotype space

Evaluation (Fitness) Function

Represents the requirements that the population should adapt to

a.k.a. quality function or objective function Assigns a single real-valued fitness to each phenotype

which forms the basis for selectionSo the more discrimination (different values) the

better Typically we talk about fitness being maximisedSome problems may be best posed as

minimisation problems, but conversion is trivial

Population

Holds (representations of) possible solutions Usually has a fixed size and is a multiset of genotypes Some sophisticated EAs also assert a spatial structure

on the population e.g., a grid. Selection operators usually take whole population into

account i.e., reproductive probabilities are relative to current generation

Diversity of a population refers to the number of different fitnesses / phenotypes / genotypes present (note not the same thing)

Parent Selection Mechanism

Assigns variable probabilities of individuals acting as parents depending on their fitnesses

Usually probabilistichigh quality solutions more likely to become

parents than low qualitybut not guaranteedeven worst in current population usually has

non-zero probability of becoming a parent This stochastic nature can aid escape from local optima

Variation Operators

Role is to generate new candidate solutions Usually divided into two types according to their arity

(number of inputs): Arity 1 : mutation operators Arity >1 : Recombination operators Arity = 2 typically called crossover

There has been much debate about relative importance of recombination and mutation Nowadays most EAs use both Choice of particular variation operators is representation

dependant

Mutation

Acts on one genotype and delivers another Element of randomness is essential and differentiates it

from other unary heuristic operators Importance ascribed depends on representation and

dialect: Binary GAs – background operator responsible for preserving

and introducing diversity EP for FSM’s/ continuous variables – only search operator GP – hardly used

May guarantee connectedness of search space and hence convergence proofs

Recombination

Merges information from parents into offspring Choice of what information to merge is stochastic Most offspring may be worse, or the same as the parents Hope is that some are better by combining elements of

genotypes that lead to good traits Principle has been used for millennia by breeders of

plants and livestock

Survivor Selection

a.k.a. replacement Most EAs use fixed population size so need a way of

going from (parents + offspring) to next generation Often deterministicFitness based : e.g., rank parents+offspring and

take best Age based: make as many offspring as parents

and delete all parents Sometimes do combination (elitism)

Initialisation / Termination

Initialisation usually done at random, Need to ensure even spread and mixture of possible allele

values Can include existing solutions, or use problem-specific

heuristics, to “seed” the population

Termination condition checked every generation Reaching some (known/hoped for) fitness Reaching some maximum allowed number of generations Reaching some minimum level of diversity Reaching some specified number of generations without fitness

improvement

Place 8 queens on an 8x8 chessboard insuch a way that they cannot check each other

Example: the 8 queens problem

The 8 queens problem: representation

1 23 45 6 7 8

Genotype:a permutation of the numbers 1 - 8

Phenotype:a board configuration

Obvious mapping

• Penalty of one queen:the number of queens she can check.

• Penalty of a configuration: the sum of the penalties of all queens.

• Note: penalty is to be minimized

• Fitness of a configuration: inverse penalty to be maximized

8 Queens Problem: Fitness evaluation

The 8 queens problem: Mutation

Small variation in one permutation, e.g.:• swapping values of two randomly chosen positions,

1 23 45 6 7 8 1 23 4 567 8

The 8 queens problem: Recombination

Combining two permutations into two new permutations:• choose random crossover point• copy first parts into children• create second part by inserting values from other parent:

• in the order they appear there • beginning after crossover point• skipping values already in child

8 7 6 42 5311 3 5 24 678

8 7 6 45 1231 3 5 62 874

Parent selection:Pick 5 parents and take best two to undergo

crossover Survivor selection (replacement)When inserting a new child into the population,

choose an existing member to replace by:sorting the whole population by decreasing fitnessenumerating this list from high to lowreplacing the first with a fitness lower than the given

child

The 8 queens problem: Selection

8 Queens Problem: summary

Note that is is only one possibleset of choices of operators and parameters

Typical behaviour of an EA

Early phase:quasi-random population distribution

Mid-phase:population arranged around/on hills

Late phase:population concentrated on high hills

Phases in optimising on a 1-dimensional fitness landscape

Typical run: progression of fitness

Typical run of an EA shows so-called “anytime behavior”

Bes

t fitn

ess

in p

opul

atio

n

Time (number of generations)

Best

fitn

ess

in p

opul

atio

n

Time (number of generations)

Progress in 1st half

Progress in 2nd half

Are long runs beneficial?

• Answer: - it depends how much you want the last bit of progress- it may be better to do more shorter runs

T: time needed to reach level F after random initialisation

TTime (number of generations)

Best

fitn

ess

in p

opul

atio

n

F: fitness after smart initialisationF

Is it worth expending effort on smart initialisation?

• Answer : it depends: - possibly, if good solutions/methods exist.- care is needed, see chapter on hybridisation

Evolutionary Algorithms in Context

There are many views on the use of EAs as robust problem solving tools

For most problems a problem-specific tool may:perform better than a generic search algorithm

on most instances, have limited utility, not do well on all instances

Goal is to provide robust tools that provide:evenly good performance over a range of problems and instances

Scale of “all” problems

Per

form

ance

of m

etho

ds o

n pr

oble

ms

Random search

Special, problem tailored method

Evolutionary algorithm

EAs as problem solvers: Goldberg’s 1989 view

EAs and domain knowledge

Trend in the 90’s:adding problem specific knowledge to EAs(special variation operators, repair, etc)

Result: EA performance curve “deformation”: better on problems of the given typeworse on problems different from given typeamount of added knowledge is variable

Recent theory suggests the search for an “all-purpose” algorithm may be fruitless

Scale of “all” problems

P

Michalewicz’ 1996 viewP

erfo

rman

ce o

f met

hods

on

prob

lem

s

EA 1

EA 4

EA 3EA 2

EC and Global Optimisation

Global Optimisation: search for finding best solution x*

out of some fixed set S Deterministic approachese.g. box decomposition (branch and bound etc)Guarantee to find x* , but may run in super-

polynomial time Heuristic Approaches (generate and test)rules for deciding which x ∈ S to generate nextno guarantees that best solutions found are

globally optimal

EC and Neighbourhood Search

Many heuristics impose a neighbourhood structure on S Such heuristics may guarantee that best point found is

locally optimal e.g. Hill-Climbers: But problems often exhibit many local optima Often very quick to identify good solutions

EAs are distinguished by:Use of population,Use of multiple, stochastic search operators Especially variation operators with arity >1Stochastic selection

GA Quick Overview

Developed: USA in the 1970’s Early names: J. Holland, K. DeJong, D. Goldberg Typically applied to:

discrete optimization Attributed features:

not too fast good heuristic for combinatorial problems

Special Features: Traditionally emphasizes combining information from good

parents (crossover)many variants, e.g., reproduction models, operators

Genetic algorithms

Holland’s original GA is now known as the simple genetic algorithm (SGA)Other GAs use different:RepresentationsMutationsCrossoversSelection mechanisms

SGA technical summary tableau

Representation Binary strings

Recombination N-point or uniform

Mutation Bitwise bit-flipping with fixed probability

Parent selection Fitness-Proportionate

Survivor selection All children replace parents

Speciality Emphasis on crossover

Genotype space = {0,1}L

Phenotype space

Encoding (representation)

Decoding(inverse representation)

011101001

01000100110010010

10010001

Representation

SGA reproduction cycle

1. Select parents for the mating pool (size of mating pool = population size)

2. Shuffle the mating pool3. For each consecutive pair apply crossover with

probability pc , otherwise copy parents4. For each offspring apply mutation (bit-flip with

probability pm independently for each bit)5. Replace the whole population with the resulting

offspring

SGA operators: 1-point crossover

Choose a random point on the two parents Split parents at this crossover point Create children by exchanging tails Pc typically in range (0.6, 0.9)

SGA operators: mutation

Alter each gene independently with a probability pm pm is called the mutation rate

Typically between 1/pop_size and 1/ chromosome_length

Main idea: better individuals get higher chanceChances proportional to fitnessImplementation: roulette wheel technique

Assign to each individual a part of the roulette wheelSpin the wheel n times to select n individuals

SGA operators: Selection

fitness(A) = 3

fitness(B) = 1

fitness(C) = 2

A C

1/6 = 17%

3/6 = 50%

B2/6 = 33%

An example after Goldberg ‘89 (1)

Simple problem: max x2 over {0,1,…,31}GA approach:Representation: binary code, e.g. 01101 ↔ 13Population size: 41-point xover, bitwise mutation Roulette wheel selectionRandom initialisation

We show one generational cycle done by hand

x2 example: selection

X2 example: crossover

X2 example: mutation

The simple GA

Has been subject of many (early) studiesstill often used as benchmark for novel GAs

Shows many shortcomings, e.g.Representation is too restrictiveMutation & crossovers only applicable for bit-string &

integer representationsSelection mechanism sensitive for converging

populations with close fitness valuesGenerational population model (step 5 in SGA repr.

cycle) can be improved with explicit survivor selection

Alternative Crossover Operators

Performance with 1 Point Crossover depends on the order that variables occur in the representation

more likely to keep together genes that are near each otherCan never keep together genes from opposite

ends of stringThis is known as Positional BiasCan be exploited if we know about the

structure of our problem, but this is not usually the case

n-point crossover

Choose n random crossover points Split along those points Glue parts, alternating between parents Generalisation of 1 point (still some positional bias)

Uniform crossover

Assign 'heads' to one parent, 'tails' to the other Flip a coin for each gene of the first child Make an inverse copy of the gene for the second child Inheritance is independent of position

Crossover OR mutation?

Decade long debate: which one is better / necessary / main-background

Answer (at least, rather wide agreement): it depends on the problem, but in general, it is good to have both both have another rolemutation-only-EA is possible, xover-only-EA would not work

Exploration: Discovering promising areas in the search space, i.e. gaining information on the problem

Exploitation: Optimising within a promising area, i.e. using informationThere is co-operation AND competition between them

Crossover is explorative, it makes a big jump to an area somewhere “in between” two (parent) areas

Mutation is exploitative, it creates random smalldiversions, thereby staying near (in the area of ) the parent

Crossover OR mutation? (cont’d)

Only crossover can combine information from two parents

Only mutation can introduce new information (alleles)

Crossover does not change the allele frequencies of the population (thought experiment: 50% 0’s on first bit in the population, ?% after performing ncrossovers)

To hit the optimum you often need a ‘lucky’ mutation

Crossover OR mutation? (cont’d)

Other representations

Gray coding of integers (still binary chromosomes) Gray coding is a mapping that means that small changes in

the genotype cause small changes in the phenotype (unlike binary coding). “Smoother” genotype-phenotype mapping makes life easier for the GA

Nowadays it is generally accepted that it is better to encode numerical variables directly as

Integers

Floating point variables

Integer representations

Some problems naturally have integer variables, e.g. image processing parameters

Others take categorical values from a fixed set e.g. {blue, green, yellow, pink}

N-point / uniform crossover operators work Extend bit-flipping mutation to make

“creep” i.e. more likely to move to similar value Random choice (esp. categorical variables) For ordinal problems, it is hard to know correct range for

creep, so often use two mutation operators in tandem

Real valued problems

Many problems occur as real valued problems, e.g. continuous parameter optimisation f : ℜ n ℜ

Illustration: Ackley’s function (often used in EC)

Mapping real values on bit strings

z ∈ [x,y] ⊆ ℜ represented by {a1,…,aL} ∈ {0,1}L

• [x,y] → {0,1}L must be invertible (one phenotype per genotype)

• Γ: {0,1}L → [x,y] defines the representation

Only 2L values out of infinite are represented L determines possible maximum precision of solution High precision long chromosomes (slow evolution)

],[)2(12

),...,(1

01 yxaxyxaa j

L

jjLLL ∈⋅⋅

−−

+=Γ ∑−

=−

Floating point mutations 1

General scheme of floating point mutations

Uniform mutation:

Analogous to bit-flipping (binary) or random resetting (integers)

ll xxxx xx ′′=′→= ..., , ...,, 11

[ ]iiii UBLBxx ,, ∈′

[ ]iii UBLBx , from (uniform)randomly drawn ′

Floating point mutations 2

Non-uniform mutations:Many methods proposed,such as time-varying

range of change etc.Most schemes are probabilistic but usually only

make a small change to valueMost common method is to add random deviate

to each variable separately, taken from N(0, σ) Gaussian distribution and then curtail to rangeStandard deviation σ controls amount of

change (2/3 of deviations will lie in range (- σ to + σ)

Crossover operators for real valued GAs

Discrete:each allele value in offspring z comes from one of its

parents (x,y) with equal probability: zi = xi or yi Could use n-point or uniform

Intermediateexploits idea of creating children “between” parents

(hence a.k.a. arithmetic recombination)zi = α xi + (1 - α) yi where α : 0 ≤ α ≤ 1. The parameter α can be:

• constant: uniform arithmetical crossover• variable (e.g. depend on the age of the population) • picked at random every time

Single arithmetic crossover

• Parents: ⟨x1,…,xn ⟩ and ⟨y1,…,yn⟩• Pick a single gene (k) at random, • child1 is:

• reverse for other child. e.g. with α = 0.5nkkk xxyxx ..., ,)1( , ..., ,1 ⋅−+⋅ αα

Simple arithmetic crossover

• Parents: ⟨x1,…,xn ⟩ and ⟨y1,…,yn⟩• Pick random gene (k) after this point mix values• child1 is:

• reverse for other child. e.g. with α = 0.5nxkxkykxx ⋅−+⋅+⋅−++⋅ )1(ny ..., ,1)1(1 , ..., ,1 αααα

• Most commonly used• Parents: ⟨x1,…,xn ⟩ and ⟨y1,…,yn⟩• child1 is:

• reverse for other child. e.g. with α = 0.5

Whole arithmetic crossover

yaxa ⋅−+⋅ )1(

Permutation Representations

Ordering/sequencing problems form a special type Task is (or can be solved by) arranging some objects in

a certain order Example: sort algorithm: important thing is which elements

occur before others (order) Example: Travelling Salesman Problem (TSP) : important thing

is which elements occur next to each other (adjacency)

These problems are generally expressed as a permutation: if there are n variables then the representation is as a list of n

integers, each of which occurs exactly once

Permutation representation: TSP example

Problem:• Given n cities• Find a complete tour with

minimal length Encoding:

• Label the cities 1, 2, … , n• One complete tour is one

permutation (e.g. for n =4 [1,2,3,4], [3,4,2,1] are OK)

Search space is BIG: for 30 cities there are 30! ≈ 1032

possible tours

Mutation operators for permutations

Normal mutation operators lead to inadmissible solutionse.g. bit-wise mutation : let gene i have value jchanging to some other value k would mean that k

occurred twice and j no longer occurred Therefore must change at least two valuesMutation parameter now reflects the probability

that some operator is applied once to the whole string, rather than individually in each position

Insert Mutation for permutations

Pick two allele values at randomMove the second to follow the first,

shifting the rest along to accommodateNote that this preserves most of the order

and the adjacency information

Swap mutation for permutations

Pick two alleles at random and swap their positionsPreserves most of adjacency information

(4 links broken), disrupts order more

Inversion mutation for permutationsPick two alleles at random and then invert

the substring between them.Preserves most adjacency information

(only breaks two links) but disruptive of order information

Scramble mutation for permutationsPick a subset of genes at randomRandomly rearrange the alleles in those

positions

(note subset does not have to be contiguous)

“Normal” crossover operators will often lead to inadmissible solutions

Many specialised operators have been devised which focus on combining order or adjacency information from the two parents

Crossover operators for permutations

1 2 3 4 5

5 4 3 2 1

1 2 3 2 1

5 4 3 4 5

Order 1 crossover

Idea is to preserve relative order that elements occur Informal procedure:

1. Choose an arbitrary part from the first parent2. Copy this part to the first child3. Copy the numbers that are not in the first part, to

the first child: starting right from cut point of the copied part, using the order of the second parent and wrapping around at the end

4. Analogous for the second child, with parent roles reversed

Order 1 crossover example

Copy randomly selected set from first parent

Copy rest from second parent in order 1,9,3,8,2

Informal procedure for parents P1 and P2:1. Choose random segment and copy it from P1 2. Starting from the first crossover point look for elements in that

segment of P2 that have not been copied3. For each of these i look in the offspring to see what element j has

been copied in its place from P14. Place i into the position occupied j in P2, since we know that we will

not be putting j there (as is already in offspring)5. If the place occupied by j in P2 has already been filled in the

offspring k, put i in the position occupied by k in P26. Having dealt with the elements from the crossover segment, the rest

of the offspring can be filled from P2. Second child is created analogously

Partially Mapped Crossover (PMX)

PMX example

Step 1

Step 2

Step 3

Cycle crossover

Basic idea: Each allele comes from one parent together with its position.

Informal procedure:1. Make a cycle of alleles from P1 in the following way.

(a) Start with the first allele of P1. (b) Look at the allele at the same position in P2.(c) Go to the position with the same allele in P1. (d) Add this allele to the cycle.(e) Repeat step b through d until you arrive at the first allele of P1.

2. Put the alleles of the cycle in the first child on the positions they have in the first parent.

3. Take next cycle from second parent

Cycle crossover example

Step 1: identify cycles

Step 2: copy alternate cycles into offspring

Edge Recombination

Works by constructing a table listing which edges are present in the two parents, if an edge is common to both, mark with a +e.g. [1 2 3 4 5 6 7 8 9] and [9 3 7 8 2 6 5 1

4]

Edge Recombination 2

Informal procedure once edge table is constructed

1. Pick an initial element at random and put it in the offspring2. Set the variable current element = entry3. Remove all references to current element from the table4. Examine list for current element:

If there is a common edge, pick that to be next element Otherwise pick the entry in the list which itself has the shortest list Ties are split at random

5. In the case of reaching an empty list: Examine the other end of the offspring is for extension Otherwise a new element is chosen at random

Edge Recombination example

Multiparent recombination

Recall that we are not constricted by the practicalities of nature

Noting that mutation uses 1 parent, and “traditional” crossover 2, the extension to a>2 is natural to examine

Been around since 1960s, still rare but studies indicate useful

Three main types: Based on allele frequencies, e.g., p-sexual voting generalising

uniform crossover Based on segmentation and recombination of the parents, e.g.,

diagonal crossover generalising n-point crossover Based on numerical operations on real-valued alleles, e.g.,

center of mass crossover, generalising arithmetic recombination operators

Population Models

SGA uses a Generational model:each individual survives for exactly one generationthe entire set of parents is replaced by the offspring

At the other end of the scale are Steady-State models: one offspring is generated per generation, one member of population replaced,

Generation Gap the proportion of the population replaced1.0 for GGA, 1/pop_size for SSGA

Fitness Based Competition

Selection can occur in two places:Selection from current generation to take part in

mating (parent selection) Selection from parents + offspring to go into next

generation (survivor selection) Selection operators work on whole individuali.e. they are representation-independent

Distinction between selectionoperators: define selection probabilities algorithms: define how probabilities are implemented

Implementation example: SGA

Expected number of copies of an individual iE( ni ) = µ • f(i)/ ⟨f⟩

(µ = pop.size, f(i) = fitness of i, ⟨f⟩ avg. fitness in pop.)

Roulette wheel algorithm:Given a probability distribution, spin a 1-armed

wheel n times to make n selectionsNo guarantees on actual value of ni

Baker’s SUS algorithm:n evenly spaced arms on wheel and spin onceGuarantees floor(E( ni ) ) ≤ ni ≤ ceil(E( ni ) )

Problems includeOne highly fit member can rapidly take over if rest of

population is much less fit: Premature ConvergenceAt end of runs when fitnesses are similar, lose

selection pressure Highly susceptible to function transposition

Scaling can fix last two problemsWindowing: f’(i) = f(i) - β t

where β is worst fitness in this (last n) generations

Sigma Scaling: f’(i) = max( f(i) – (⟨ f ⟩ - c • σf ), 0.0)where c is a constant, usually 2.0

Fitness-Proportionate Selection

Function transposition for FPS

Rank – Based Selection

Attempt to remove problems of FPS by basing selection probabilities on relativerather than absolute fitnessRank population according to fitness and

then base selection probabilities on rank where fittest has rank µ and worst rank 1This imposes a sorting overhead on the

algorithm, but this is usually negligible compared to the fitness evaluation time

Linear Ranking

Parameterised by factor s: 1.0 < s ≤ 2.0measures advantage of best individualin GGA this is the number of children allotted to

it Simple 3 member example

Exponential Ranking

Linear Ranking is limited to selection pressureExponential Ranking can allocate more

than 2 copies to fittest individualNormalise constant factor c according to

population size

Tournament Selection

All methods above rely on global population statisticsCould be a bottleneck esp. on parallel

machinesRelies on presence of external fitness function

which might not exist: e.g. evolving game players

Informal Procedure:Pick k members at random then select the best

of theseRepeat to select more individuals

Tournament Selection 2

Probability of selecting i will depend on:Rank of iSize of sample k

higher k increases selection pressureWhether contestants are picked with replacement

Picking without replacement increases selection pressure

Whether fittest contestant always wins (deterministic) or this happens with probability p

For k = 2, time for fittest individual to take over population is the

same as linear ranking with s = 2 • p

Survivor Selection

Most of methods above used for parent selectionSurvivor selection can be divided into two

approaches:Age-Based Selectione.g. SGAIn SSGA can implement as “delete-random” (not

recommended) or as first-in-first-out (a.k.a. delete-oldest)

Fitness-Based SelectionUsing one of the methods above or

Two Special Cases

ElitismWidely used in both population models (GGA,

SSGA)Always keep at least one copy of the fittest solution

so farGENITOR: a.k.a. “delete-worst”From Whitley’s original Steady-State algorithm (he

also used linear ranking for parent selection)Rapid takeover : use with large populations or “no

duplicates” policy

Example application of order based GAs: JSSP

Precedence constrained job shop scheduling problem J is a set of jobs. O is a set of operations M is a set of machines Able ⊆ O × M defines which machines can perform which

operations Pre ⊆ O × O defines which operation should precede which Dur : ⊆ O × M → IR defines the duration of o ∈ O on m ∈ M

The goal is now to find a schedule that is: Complete: all jobs are scheduled Correct: all conditions defined by Able and Pre are satisfied Optimal: the total duration of the schedule is minimal

Precedence constrained job shop scheduling GA

Representation: individuals are permutations of operations Permutations are decoded to schedules by a decoding procedure

take the first (next) operation from the individual look up its machine (here we assume there is only one) assign the earliest possible starting time on this machine, subject to

machine occupation precedence relations holding for this operation in the schedule created so far

fitness of a permutation is the duration of the corresponding schedule (to be minimized)

use any suitable mutation and crossover use roulette wheel parent selection on inverse fitness Generational GA model for survivor selection use random initialisation

JSSP example: operator comparison

04.02.2012 AI HH 1

Hodgkin-Huxley Modeland

FitzHugh-Nagumo Model


04.02.2012 AI HH 2

Alan Lloyd Hodgkin

Andrew Fielding Huxley

The squid (Loligo)

A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of Physiology 117:500-544 (1952)

Nobelpreice for medicine, 1963

04.02.2012 AI HH 3

Loligo giant axones

04.02.2012 AI HH 4

Nervous System

Signals are propagated from nerve cell to nerve cell (neuron) via electro-chemical mechanisms

~100 billion neurons in a person Hodgkin and Huxley experimented on squid giant

axons and discovered how the signal is produced and propagate within the neuron

H.-H. model was published in Jour. of Physiology(1952)

H.-H. were awarded 1963 Nobel Prize FitzHugh-Nagumo model is a simplification

04.02.2012 AI HH 5

Neurons

Basic morphologyPresynaptic terminal (axon or dendrite)Postsynaptic process (dendrite or cell body)

Cerebral cortex108 - 109 synapses/mm3

0.5 - 1.0 m diameter of synaptic contactµ

04.02.2012 AI HH 6

Neuron

C. George Boeree: www.ship.edu/~cgboeree/

http://www.ship.edu/~cgboeree/

04.02.2012 AI HH 7

Electron Micrograph of a Real Neuron

04.02.2012 AI HH 8

Remember? Transmenbrane Ionic Transport Ion channels act as gates that allow or

block the flow of specific ions into and out of the cell.

04.02.2012 AI HH 9

04.02.2012 AI HH 10

Ein Aktionspotential besteht aus verschiedenen Phasen

Restingpotential

Depolarisation Repolarisation

Post-hyperpolarisation

Restingpotential

04.02.2012 AI HH 11

Das Modell von Hodgkin and Huxley

The Aktionspotential:

• During the resting-potential the Na+-channels are closed (not inactivated!, not activated)

• After depolarization the Na+-channels open fast, GNa is increased

• The membrane depolarizes further more

• Slowly the K+-channels, GK increases

• Na+-channels close (inactivate)

• K+-flow (current) dominates den allover „current“

• The membrane-potential repolarizes

• The K+-channels remain still in an open state with decreased GNa

• After hyper-polarization (Vm near EK)

• As long as Na+-channels are inactivated: absolute refractory period

04.02.2012 AI HH 12

Synaptic transmission

Depolarization due to action potential Influx of calciumExocystosis of synaptic vesiclesDiffusion of neurotransmitterBinding to postsynaptic receptors

04.02.2012 AI HH 13

The basic Hodgkin - Huxley model

The standard model of an exicitatory neuron consists of the equation for the total membrane current IM

M m K Na LdVI C I I Idt

= + + +

04.02.2012 AI HH 14

Currents

V denotes the membrane voltage IK is the potassium current INa is the sodium current IL is a leakage current

04.02.2012 AI HH 15

Circuit

04.02.2012 AI HH 16

The Cable Equation

See http://diwww.epfl.ch/~gerstner/SPNM/SPNM.html

for excellent additional material Just a piece of passive dendrite can yield

complicated differential equations which have been extensively studied by electro-technicians in the context of the study of coaxial cables (TV antenna cable):

http://diwww.epfl.ch/~gerstner/SPNM/SPNM.html

04.02.2012 AI HH 17

Action Potential

Axon membrane potential difference

V = Vi – VeWhen the axon is excited, V spikes because sodium Na+ and potassium K+ions flow through the membrane.10 msec

mV

_ 30

-70

V_ 0

04.02.2012 AI HH 18

Charge across a capacitor is q = CE, C is capacitance, E is the voltage across the capacitor, differentiating:

Ohm’s law:

Hodkin Huxley equations K and Na nonlinear functions of membrane voltage and time.

V-Vc is the voltage difference across the membrane and the equilibrium value for the ion is Vc, which may change in time.

dq dECdt dt

=

/V IR I V R gV= ⇒ = =

( , ) ( ( ) ( ))c cI g V t V t E t= ⋅ −

04.02.2012 AI HH 19

Circuit Model for Axon MembraneSince the membrane separates charge, it is modeled as a capacitor with capacitance C. Ion channels are resistors.

1/R = g = conductance

iC = C dV/dt

iNa = gNa (V – VNa)

iK= gK (V – VK)

iL = gL (V – V:)

04.02.2012 AI HH 20C. George Boeree: www.ship.edu/~cgboeree/

Nernst Potential

VNa , VK and Vr

Ion flow due to electrical signal

Traveling wave

http://www.ship.edu/~cgboeree/

04.02.2012 AI HH 21

Circuit EquationsSince the sum of the currents is 0, it follows that

( ) ( ) ( )Na KKr r ap

Na

dVC V V g V V IgV Vgdt= − − − − − − +

where Iap is applied current. If ion conductances are constants then group constants to obtain 1st order, linear eq

apIVVgdtdVC +−−= *)(

Solving gives gIVtV ap /*)( +→

04.02.2012 AI HH 22

Variable Conductance

Experiments showed that gNa and gK varied with time and V. After stimulus, Na responds much more rapidly than K .

g

04.02.2012 AI HH 23

Hodgkin-Huxley System

Four state variables are used:v(t)=V(t)-Veq is membrane potential,m(t) is Na activation,n(t) is K activation and h(t) is Na inactivation.

In terms of these variables gK=gKn4 and gNa=gNam3h. The resting potential Veq≈-70mV. Voltage clamp experiments determined gK and n as functions of t and hence the parameter dependences on v in the differential eq. for n(t). Likewise for m(t) and h(t).

04.02.2012 AI HH 24

Hodgkin-Huxley System

IVvgVvngVvhmgdtdvC aprrKNa

KNa+−−−−−−= )()()( 43

mvmvdtdm

mm )()1)(( βα −−=

nvnvdtdn

nn )()1)(( βα −−=

hvhvdtdh

hh )()1)(( βα −−=

04.02.2012 AI HH 25

Iap =8, v(t)

Iap=7, v(t)

m(t)

n(t)

h(t)

10msec

1.2

110 mV

40msec

04.02.2012 AI HH 26

Fast-Slow Dynamics

v, m are on a fast time scale and n, h are slow.

ρm(v) dm/dt = m∞(v) – m.

ρm(v) is much smaller than

ρn(v) and ρh(v). An increase in v results in an increase in m∞(v) and a large dm/dt. Hence Na activates more rapidly than K in response to a change in v.

m(t)

n(t)

h(t)

10msec

04.02.2012 AI HH 27

FitzHugh-Nagumo System

Iwvfdtdv

+−= )(ε wvdtdw 5.0−=and

I represents applied current, ε is small and f(v) is a cubic nonlinearity. Observe that in the (v,w) phase plane

which is small unless the solution is near f(v)-w+I=0. Thus the slowmanifold is the cubic w=f(v)+I which is the nullcline of the fast variable v. And w is the slow variable with nullcline w=2v.

Iwvfwv

dvdw

+−−

=)(

)5.0(ε

04.02.2012 AI HH 28

v

I=0 I=0.2Stable rest state Stable oscillation

Take f(v)=v(1-v)(v-a) .

w w

v

04.02.2012 AI HH 29

FitzHugh-Nagumo Orbits

04.02.2012 AI HH 30

How do the channels really work: Patch Clamp

04.02.2012 AI HH 31

The Patch Clamp Technique – single channel measurements

Bert Sakmann, Erwin NeherNobelpreice for medicin, 1991

04.02.2012 AI HH 32

04.02.2012 AI HH 33

Major Functional Areas

Primary motor: voluntary movement Primary somatosensory: tactile, pain, pressure, position, temp., mvt. Motor association: coordination of complex movements Sensory association: processing of multisensorial information Prefrontal: planning, emotion, judgement Speech center (Broca’s area): speech production and articulation Wernicke’s area: comprehen- sion of speech Auditory: hearing Auditory association: complex auditory processing Visual: low-level vision Visual association: higher-level vision

04.02.2012 AI HH 34Felleman & Van Essen, 1991

Interconnect

04.02.2012 AI HH 35

Detailed Neural ModelingA simulator, called “Neuron” has been developed at Yale to simulate

the Hodgkin-Huxley equations, as well as other membranes/channels/etc. See http://www.neuron.yale.edu/

http://www.neuron.yale.edu/

04.02.2012 AI HH 36

Basic neurobiology: Combine inhibitory and excitatory signals at different times to fire a neuron.

04.02.2012 AI HH 37

Computer science: Programing a simple early computer demonstrates basic computer science concepts.

04.02.2012 AI HH 38

Neural networks: Use a learning algorithm to understand how the brain stabilizes the eyes during head movement.

04.02.2012 AI HH 39

Active neural membranes: Explore the Hodgkin-Huxley equations, the classical explanation of neural membrane biophysics.

04.02.2012 AI HH 40

04.02.2012 AI HH 41

References1. C.G. Boeree, The Neuron, www.ship.edu/~cgboeree/.2. R. FitzHugh, Mathematical models of excitation and

propagation in nerve, In: Biological Engineering, Ed: H.P. Schwan, McGraw-Hill, New York, 1969.

3. L. Edelstein-Kesket, Mathematical Models in Biology, Random House, New York, 1988.

4. A.L. Hodgkin, A.F. Huxley and B. Katz, J. Physiology 116, 424-448,1952.

5. A.L. Hodgkin and A.F. Huxley, J. Physiol. 116, 449-566, 1952.6. F.C. Hoppensteadt and C.S. Peskin, Modeling and Simulation

in Medicine and the Life Sciences, 2nd ed, Springer-Verlag, New York, 2002.

7. J. Keener and J. Sneyd, Mathematical Physiology, Springer-Verlag, New York, 1998.

8. J. Rinzel, Bull. Math. Biology 52, 5-23, 1990.9. E.K. Yeargers, R.W. Shonkwiler and J.V. Herod, An

Introduction to the Mathematics of Biology: with Computer Algebra Models, Birkhauser, Boston, 1996.

ai_all

Documents

episode itselfsolitaire observable

internet shopping partial

genetic algorithms variant

knowledge base contd

mutex relation holds

local beam search

unlike natural language

spare tire probleminit